Multi-task parallel processing three-layer hierarchical scanning method and system based on distributed framework

Abstract

The invention specifically discloses a multi-task parallel processing three-layer hierarchical scanning method and system based on a distributed framework, and the method comprises the following steps: constructing MQTT message middleware to control the transmission of information, and building a distributed transmission calculation architecture; single-file-level rule scanning is carried out, a client obtains to-be-scanned project source file information and issues the information to the MQTT, scanning results are returned to the server, and integration of single-file-level scanning results is completed at the client; project-level and clustering-level scanning tasks are executed in parallel, project-level scanning is undertaken by the Leader Node, the Server performs clustering according to a file reference relationship in a single-file-level scanning result, and clustering-level scanning is completed by issuing a clustering task set to the Node by the Server; and the client receives the results of the single file level, the clustering level and the project level and then integrates the results to obtain a final complete scanning result. By adopting the technical scheme, loads are shared, multi-end request concurrent response and dynamic node group scheduling are carried out, and distributed scanning under a multi-task scene is realized.

Description

Technical Field

[0001] This invention belongs to the field of distributed transmission technology and relates to a three-layer hierarchical scanning method and system for multi-task parallel processing based on a distributed framework. Background Technology

[0002] As software scale grows exponentially, with leading companies exceeding ten million lines of code in a single project, static code rule scanning faces the dual challenges of efficiency and accuracy. Especially in safety-critical sectors such as automotive and aerospace, multi-file collaborative development models adhering to coding standards like MISRAC and AUTOSAR make cross-file dependency analysis a core requirement for ensuring software quality.

[0003] The efficiency bottleneck of single-machine static scanning tools has driven the rise of distributed solutions, but early distributed architecture designs generally overlooked the special characteristics of code dependencies. Although the traditional Hadoop architecture can achieve parallel processing of large-scale data, its "stateless task splitting" characteristic makes it difficult to support cross-file dependency analysis such as function calls and class inheritance. Task sharding in this type of architecture is usually based on the physical structure of files rather than logical dependencies, resulting in a large number of cross-file defects that cannot be detected (such as timing problems caused by global variable modifications).

[0004] Distributed scanning architectures roughly categorize rule-based scanning tasks into single-file and cross-file scanning tasks. However, actual rule-based scanning tasks include not only single-file and cross-file rules but also project-level rules, which are an indispensable part of the task. Project-level scanning refers to using the entire project as the scanning unit, performing rule validation based on the project's global metadata, and aiming to discover "global-level issues" affecting the project's availability, security, compliance, and maintainability. Its core characteristic is that it is "not limited to a single or multiple files" but focuses on whether the "project as a whole" conforms to preset rules. However, introducing project-level scanning also introduces new load balancing issues.

[0005] Existing distributed transport computing frameworks still have the following problems:

[0006] 1. The scanning task is mainly broken down into individual tasks, but the case of multiple tasks scanning in parallel is not considered.

[0007] 2. The scanning task hierarchy lacks project-level information, resulting in incomplete scanning results. Furthermore, the system consistently requests source files from the client, making transmission latency the biggest factor affecting distributed scanning.

[0008] 3. The clustering method is based on the reference relationship of files. Taking C language as an example, if ah is referenced by ac and bc, then ah is used as the cluster head and forms a cluster with {ac, bc}. However, from the perspective of compilation, the .c file is the center and other .h files are recursively included. This clustering method makes the dependency relationship of .c files broken when performing cross-file rule scanning (there is only one .h file in the cluster and no other referenced .h files), which affects the accuracy of the scanning results. Summary of the Invention

[0009] The purpose of this invention is to address the aforementioned problems in existing technologies by proposing a three-layer hierarchical scanning method and system based on a distributed framework for multi-task parallel processing.

[0010] To achieve the above objectives, the basic solution of this invention is: a three-layer hierarchical scanning method for multi-task parallel processing based on a distributed framework, comprising the following steps:

[0011] Based on the publish / subscribe model, an MQTT message middleware is built to control the transmission of information, and a distributed transmission computing architecture based on the Client-Server-Node group [LeaderNode-Node] architecture is constructed.

[0012] A single-file-level rule scan is performed. The client obtains the source file information of the project to be scanned and publishes it to MQTT. The server selects a node as the LeaderNode of the new task and determines the ordinary nodes of the assisting task to form a node group to complete the single-file-level scan. The scan results are returned to the server. The results are used for subsequent clustering. The client completes the integration of the single-file-level scan results.

[0013] Project-level and cluster-level scanning tasks are executed in parallel. Project-level scanning is undertaken by LeaderNode, while the server performs clustering based on the file reference relationships in the single-file-level scanning results. Cluster-level scanning is performed by the server distributing cluster task sets to ordinary nodes.

[0014] The client receives the results at the single-file level, cluster level, and project level, and integrates them to obtain the final complete scan result.

[0015] The working principle and beneficial effects of this basic solution are as follows: Based on the MQTT (Message Queuing Telemetry Transport) protocol with a publish / subscribe mechanism, this technical solution proposes a client-server-[Leadernode-node] architecture to achieve concurrent response to multi-end requests, dynamic node group scheduling, and a three-level hierarchical rule scanning mode of single file, cluster, and project level to scan project files step by step. A two-stage file transfer mode is used to reduce overall transmission latency and improve scanning accuracy. The implementation of the three-level hierarchical rule scanning mode and the two-stage file transfer mode first performs single file scanning, followed by parallel project-level and cluster scanning, progressing layer by layer to ensure complete scan results and reduce the impact of transmission latency on distributed scanning efficiency.

[0016] Furthermore, the method for building an MQTT message middleware based on the publish / subscribe pattern is as follows:

[0017] The MQTT protocol manages the transmission of information through topics. Publishers send topics carrying data to the MQTT broker, and the broker distributes the data to nodes that have subscribed to the topic.

[0018] To ensure that messages for a specified Topic are sent to the correct nodes, the Topic name is used as the scheduling keyword. A Topic name is named "Source / Function / Symbol / TaskID". Source is the name of the device that published this Topic, Function is the name of the function that is communicated through this Topic, Symbol is the content identifier name, which the receiving end can use to distinguish the message's origin, and TaskID is used to distinguish different users or different tasks of the same user. It is composed of the Client device number that issued the scanning task and the task number.

[0019] Based on the publish / subscribe pattern, an MQTT message middleware is built, which is beneficial for subsequent use.

[0020] Furthermore, at the start of the single-file level scanning phase, the process by which the client obtains the source file information of the project to be scanned is represented as follows:

[0021] ,

[0022] ,

[0023] ,

[0024] Here, Project represents the project to be scanned, TaskDict is a dictionary summarizing the scan results, and FileInfo represents the single-file scan results, including the file path and size information. The scan result for the i-th single file includes the file path. and size information This process means that the Scanf function scans the specified items. For each source file to be scanned, a FileInfo containing the file path and file size will be generated. The TaskDict output by the function is a collection of FileInfo from all source files. The TaskDict will be published to the Task Request Topic of the MQTT broker.

[0025] At the start of the single-file-level scanning phase, the Client obtains information about the source files of the project to be scanned, which is a simple operation.

[0026] Furthermore, after initialization, the server connects to the MQTT broker, subscribes to and obtains the TaskDict, and parses it to obtain a single-file-level task queue (taskqueue) sorted by file size.

[0027] ,

[0028] Where sort is the sorting function;

[0029] The server performs node group adjustments, specifically as follows:

[0030] Select a LeaderNode from the non-Leader mode Nodes to serve as the LeaderNode for the new task, and allocate some base Nodes to assist in the task. The allocation process is completed by modifying the NodeMap on the Server. The Leader is set by publishing messages to specify the Topic in the node mode of the MQTT broker.

[0031] Obtain a single-file-level task queue for single-file scanning.

[0032] Furthermore, the method for constructing the [LeaderNode-Node] node group is as follows:

[0033] Initially, no tasks are running in the architecture, all nodes are in normal mode, and a table called NodeInfo is input to the server node to rate the nodes based on their hardware and network conditions, providing a basis for the subsequent selection of the LeaderNode.

[0034] When the server receives a task request, it will select one node to appoint as the LeaderNode, and the other nodes will be ordinary nodes, together forming a node group to complete the scanning task.

[0035] When the server receives n task requests, it will select n nodes to appoint as LeaderNodes, and divide the remaining ordinary nodes into groups according to the size of the task to be scanned. Each node group is responsible for one scan task.

[0036] [LeaderNode-Node] is a node group. Nodes have two modes: Leader and Normal. The mode is controlled by the Server, and the allocation of node groups is dynamically adjusted by the Server.

[0037] Furthermore, it also includes methods for Server selection and control of LeaderNode state switching, as detailed below:

[0038] The server searches for the non-LeaderNode node with the highest rating in NodeInfo. If multiple nodes with the same rating exist, it selects one, marks it as the Leader, sends the appointment information to this node via the MQTT broker, and waits for a successful appointment reply. If no reply is received within the timeout period, the server enters the exception handling process.

[0039] After receiving the confirmation message that the appointment was successful, the server will distribute the ordinary nodes into each [LeaderNode-Node] node group according to the task size.

[0040] When the number of ordinary nodes in the [LeaderNode-Node] node group is less than MinSize, the reception of new tasks is paused. MinSize is defined according to the actual situation.

[0041] After receiving the task completion information, the server sends a command to the LeaderNode bound to this task to switch to normal mode and waits for a successful reply. If no reply is received within the timeout period, the exception handling process begins.

[0042] After receiving a successful switchover response, the server removes the LeaderNode from the Leader state and restores it to a normal state. The LeaderNode, along with the other Nodes in its [LeaderNode-Node] group, is then assigned to the corresponding [LeaderNode-Node] group based on the size of the tasks in progress.

[0043] The server selects and controls the LeaderNode state switching, which is simple to operate.

[0044] Furthermore, the single-file level, cluster level, and project level rule sets used in the three-layer hierarchical scanning were obtained by manually annotating and classifying industry standard rule sets. The classification and annotation criteria were as follows:

[0045] Scope-first: Prioritize the division based on the "file scope required for analysis", from single file → cluster → project. Scope is the most intuitive distinguishing criterion.

[0046] Dependency Auxiliary: If the scope is ambiguous, it is determined by "whether it depends on project-level global connection information". If it depends, it is project-level; otherwise, it is cluster-level.

[0047] Industry Adaptation: Based on the rule descriptions of specific industry standards, if the rules explicitly mention the keywords "within the module" or "in the project," they can be directly used as the classification basis.

[0048] The single-file level, cluster level, and project level rule sets used in the three-level hierarchical scanning were obtained by manually annotating and classifying industry standard rule sets, which facilitates the division.

[0049] Furthermore, the first-stage file transfer and single-file-level scanning: After completing the node group allocation, the Server will issue single-file-level tasks via the Singletaskassign function, as follows:

[0050] , ,

[0051] Among them, taskqueue is an ordered single-file-level scan task queue generated by the server after receiving a task request, taskID is a task identifier used to match node groups for task allocation, and NodeMap is a node group allocation table.

[0052] After processing by the function, the Single_task allocation set will be output. Issue the i-th single-file level task, which includes the task issuance target. and task content Similarly, it is published to the corresponding Topic on the MQTT broker, and the Node subscribes to the relevant Topic based on the NodeID to obtain it;

[0053] After receiving a task, the Node starts requesting source files from the Client for scanning. The MQTT broker publishes a Phase 1 request Topic message. The Client receives the corresponding request message by matching the task ID and publishes a message carrying the source files to the Phase 1 source file transmission Topic. Ordinary nodes in the node group receive the requested source files by matching the Node ID. The LeaderNode subscribes to all Phase 1 source file transmission Topics under the corresponding task ID, thereby caching all source files on the LeaderNode and enabling the LeaderNode to become the near-end source file provider node in the Phase 2 source file transmission.

[0054] After receiving the source file, Node performs a single-file level scan. This process is represented as follows:

[0055] ,

[0056] ,

[0057] ,

[0058] Where FileInfo is the source file content, Rule is the single-file level rule set, and Includ is the file reference relationship set in Single_res.

[0059] The SinglelesSet, a collection of vulnerability scan results for single-file rules, is returned to the client. Each result is recorded using a unified ResultStruct structure. The SinglelesSet is represented as follows:

[0060] ,

[0061] After receiving the IncludeSet, the Server parses out the Include information and integrates it into an IncludeInfo collection containing all file reference information.

[0062] .

[0063] After the node group allocation is completed, a first-stage file transfer and single-file level scan are performed. The client provides the source file transfer, and the Leadernode listens for and collects all source file information during this stage, which is then used as the file transfer source for subsequent cluster scans and project-level scans.

[0064] Furthermore, two-stage file transfer and cluster-level and project-level scanning:

[0065] After the single-file level scan is completed, the cluster level and project level scans will begin simultaneously in parallel. The project level scan is performed on the LeaderNode, and the specific steps are as follows:

[0066] When the LeaderNode receives the project-level scan start command from the Server, it checks whether all source files in the cache have been collected. If any are missing, it requests the missing files from the Client again to complete them. If the files are complete, it creates a new thread to perform a project-level scan.

[0067] ,

[0068] Among them, FileDict is a dictionary of source files, with filenames as keys and corresponding source file contents as values; Prule is a project-level rule set; ProgramresSet is a project-level scan result set, where each result is recorded using a unified ResultStruct structure.

[0069] The task performed in the ProgramScan function is as follows:

[0070] It iterates through all project files, obtains file references, class references, function calls and global parameter information for each file, and transforms the source code into an abstract syntax tree through a syntax parser, thus completing the preliminary information acquisition work required for scanning.

[0071] Based on the acquired prior information Establish global connection information across the entire project :

[0072] ,

[0073] ,

[0074] Among them, IncludeList contains file reference information, ClassList contains class reference information, FunctionList contains function call information, and GlobalVue contains global parameter information;

[0075] Traverse the Abstract Syntax Tree (AST) tree structure and determine the relationships between nodes in the AST throughout the project using global relationship information. Trigger project-level rule checks for each node, specifically:

[0076] The project-level rules defined in Prule are matched sequentially. If a rule is found to be in violation, the violation is added to the result information. If no rule is found to be in violation, the process continues to the next node.

[0077] Meanwhile, the original LeaderNode thread continues to handle message responses from the MQTT broker.

[0078] After completing the single-file level scan, the cluster level and project level scans will begin simultaneously in parallel, enabling concurrent responses to multi-terminal requests.

[0079] Furthermore, the server performs clustering based on the file reference relationships in the single-file level scan results. The specific process is as follows:

[0080] S101, Select an unclustered entry, Include1, from IncludeInfo. Construct a stack based on the file reference relationship information Include1. The corresponding construction method is as follows:

[0081] ,

[0082] The referenced file elements in Include correspond to the referenced file elements in Stack;

[0083] S102, pop the top element from the stack, add it to the cluster, index this element in IncludeInfo, obtain the Include information of this element, and push the Include information onto the stack, represented as:

[0084] ,

[0085] S103, Repeat step S102 until the stack is empty, then perform deduplication on the files in the cluster to complete the clustering. The final cluster representation is as follows:

[0086] ,

[0087] Among them, ClusterHead in Cluster corresponds to FileName in the initially selected Include1;

[0088] S104, After the Include completes the clustering construction, it is marked as clustered information, and the clustering of this reference relationship is completed;

[0089] S105, Traverse all unclustered information in IncludeInfo, repeat steps S101-S104 until all file clustering tasks are completed;

[0090] The clustering results are represented as follows:

[0091] ,

[0092] After clustering is completed, the Server will distribute cluster-level tasks via the Clustertaskassign function, as follows:

[0093] , ,

[0094] ClusterDict is the cluster task dictionary, taskID is the task identifier, and Nodemap is the node group allocation table. The node group is found by matching taskID with NodeMap and the task is sent to the Node in the node group.

[0095] TaskInfo is a list of clustered source files. After the Node receives the task and parses the TaskInfo information, it no longer requests source files from the remote Client at this stage, but instead requests source files from the local Node LeaderNode.

[0096] After receiving the task, the Node no longer requests the source file from the remote Client at this stage, but instead requests the source file from the local LeaderNode.

[0097] Once Node receives the source file, it begins a cluster-level scan, which is represented as follows:

[0098] .

[0099] Among them, ClusterInfo contains the contents of all files in the cluster, with filenames and contents paired one-to-one; Crule is the cluster-level rule set; and ClusterresSet is the cluster-level scan result set, where each result is recorded using a unified ResultStruct structure.

[0100] The server clusters files based on file reference relationships in the single-file level scan results, changing the source of the second-stage file transfer from the remote client (user) to the local Leadernode (server node).

[0101] Furthermore, the client will receive and integrate the results at the single-file level, cluster level, and project level to obtain the final complete scan result.

[0102] After project-level and cluster-level scans are completed, the results are returned to the client. These results are then integrated with the single-file-level scan results to obtain a complete three-level hierarchical scan result. The output information of the three-level hierarchical scans all use a unified structure, ResultStruct. After alignment, they can be merged.

[0103] ,

[0104] Among them, RuleId is the rule number of the scan anomaly, FilePath is the file path information of the file where the problem was found during the scan, Code is the specific code content of the problem found during the scan, Warning is the warning message, Suggestion is the optimization suggestion corresponding to the rule, and ResultType is the result level type, which is used by the client to distinguish and integrate results at different levels.

[0105] In the three-layer hierarchical scanning, a two-stage file transfer strategy is adopted. In the first stage, the source file is obtained from the remote client, and in the second stage, the source file is obtained from the local server Leadernode node, which reduces the impact of transmission latency on the efficiency of distributed scanning.

[0106] This invention also provides a three-layer hierarchical scanning system based on a distributed framework and multi-task parallel processing, including a distributed transmission and computing architecture module based on a client-server-node group [LeaderNode-Node] architecture. The distributed transmission and computing architecture module executes the method described in this invention to obtain complete scanning results.

[0107] This system is based on a two-stage file transfer mode, which realizes three-level hierarchical rule scanning of single files, clusters, and projects, distributes the load, and performs distributed scanning in multi-task scenarios. Attached Figure Description

[0108] Figure 1 This is a schematic diagram of the structure of the three-layer hierarchical scanning method for multi-task parallel processing based on a distributed framework according to the present invention;

[0109] Figure 2 This is a schematic diagram of the two-stage file transfer process of the three-layer hierarchical scanning method based on a distributed framework in this invention.

[0110] Figure 3 This is a flowchart illustrating the three-layer hierarchical scanning method for multi-task parallel processing based on a distributed framework, as described in this invention.

[0111] Figure 4 This is a schematic diagram of the one-stage file transfer process of the three-layer hierarchical scanning method based on a distributed framework in this invention.

[0112] Figure 5 This is a schematic diagram of the two-stage file transfer process of the three-layer hierarchical scanning method based on a distributed framework for multi-task parallel processing according to the present invention. Detailed Implementation

[0113] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0114] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0115] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0116] In actual production use, it is common for multiple tasks to request at the same time. Introducing multi-task parallel processing will bring new problems of task diversion and node overload. Therefore, it is necessary to design a new distributed architecture to achieve error-free multi-task parallel processing and balance the load of each module in the architecture so that it can run stably.

[0117] This invention discloses a three-tiered hierarchical scanning method based on a distributed framework and multi-task parallel processing. It proposes a three-tiered hierarchical rule scanning mode of single file, cluster, and project level to distribute the load and achieve distributed scanning in multi-task scenarios. The source file is always provided by the client node; remote transmission makes transmission latency the biggest factor affecting distributed scanning. Figure 1 and Figure 3 As shown, the three-layer hierarchical scanning method for multi-task parallel processing based on a distributed framework includes the following steps:

[0118] Based on the publish / subscribe model, an MQTT message middleware is built to control the transmission of information, and a distributed transmission computing architecture based on the Client-Server-Node group [LeaderNode-Node] architecture is constructed.

[0119] A single-file-level rule scan is performed. The client obtains the source file information of the project to be scanned and publishes it to MQTT. The server selects a node as the LeaderNode of the new task and determines the ordinary nodes of the assisting task to form a node group to complete the single-file-level scan. The scan results are returned to the server. The results are used for subsequent clustering. The client completes the integration of the single-file-level scan results.

[0120] Project-level and cluster-level scanning tasks are executed in parallel. Project-level scanning is undertaken by LeaderNode, while the server performs clustering based on the file reference relationships in the single-file-level scanning results. Cluster-level scanning is performed by the server distributing cluster task sets to ordinary nodes.

[0121] The client receives the results at the single-file level, cluster level, and project level, and integrates them to obtain the final complete scan result.

[0122] This invention references the node caching concept and proposes a two-stage file transfer mode. In the single-file scanning stage, the client provides the source file transfer. During this stage, the Leadernode listens for and collects all source file information and uses it as the file transfer source for subsequent cluster scanning and project-level scanning. This changes the file transfer source in the second stage from the remote client (user) to the near Leadernode (server node).

[0123] In actual production use, users may submit multiple scan tasks simultaneously, or multiple users may submit their own scan tasks at the same time, which will bring additional user message matching issues. The Gossip protocol solves the problem of eventual consistency of multi-user data by achieving node state synchronization through decentralized random message propagation.

[0124] This invention proposes a client-server-[Leadernode-node] architecture based on the MQTT (Message Queuing Telemetry Transport) protocol with a publish / subscribe mechanism. It enables concurrent response to multi-end requests, dynamic node group scheduling, and adopts a three-level hierarchical rule scanning mode of single file, cluster, and project level to scan project files step by step. It also reduces the overall transmission latency through a two-stage file transfer mode.

[0125] In a preferred embodiment of the present invention, the method for constructing an MQTT message middleware based on a publish / subscribe pattern is as follows:

[0126] The MQTT protocol manages the transmission of information through topics. Publishers send topics carrying data to the MQTT broker, and the broker distributes the data to nodes that have subscribed to the topic.

[0127] To ensure that messages for a specified Topic are sent to the correct nodes, the Topic name is used as the scheduling keyword. A Topic name is named "Source / Function / Symbol / TaskID". Source is the device name that published the Topic, Function is the name of the function communicated through this Topic, Symbol is the content identifier used by the receiving end to distinguish message ownership, and TaskID is used to distinguish different users or different tasks of the same user. It consists of the Client device number that issued the scan task and the task number. For example, if a Client with device number 0022 requests a second scan task before the first scan task is completed, and the TaskID of the first scan task is 002201, then the TaskID of the second scan task will be 002202, as shown in Table 1.

[0128] Table 1 Topic Structure Table

[0129]

[0130] The above example could form "Client / Newtask / Taskdict / 002201". The Topic identifier means: a new task request issued by the Client, carrying task list information, and is task number 01 for Client 0022. This Topic identifier-based message routing lays the foundation for responding to multi-terminal task requests.

[0131] In a preferred embodiment of the present invention, the process by which the client obtains the source file information of the project to be scanned at the start of the single-file level scanning phase is represented as follows:

[0132] ,

[0133] ,

[0134] ,

[0135] Here, Project represents the project to be scanned, TaskDict is a dictionary summarizing the scan results, and FileInfo represents the single-file scan results, including the file path and size information. The scan result for the i-th single file includes the file path. and size information This process means that the Scanf function scans the specified items. For each source file to be scanned, a FileInfo containing the file path and file size will be generated. The TaskDict output by the function is a collection of FileInfo from all source files. The TaskDict will be published to the Task Request Topic of the MQTT broker.

[0136] In a preferred embodiment of the present invention, after the Server initializes, it connects to the MQTT broker. The Server subscribes to and obtains the TaskDict, and parses it to obtain a single-file-level task queue (taskqueue) sorted by file size.

[0137] ,

[0138] Where sort is the sorting function;

[0139] The server performs node group adjustments, specifically as follows:

[0140] Select a LeaderNode from the non-Leader mode Nodes as the LeaderNode for the new task, and allocate some basic Nodes to assist in the task. The allocation process is completed by modifying the NodeMap table on the Server (as shown in Table 2). The Leader is set by publishing messages by specifying the Topic in the node mode of the MQTT broker.

[0141] Table 2 NodeMap Structure

[0142]

[0143] In a preferred embodiment of the present invention, nodes have two modes: Leader and Normal. The mode is controlled by the Server, and the node group allocation is dynamically adjusted by the Server. Communication between the Client, Server, and [LeaderNode-Node] nodes is implemented through an MQTT broker. The method for constructing the [LeaderNode-Node] node group is as follows:

[0144] Initially, no tasks are running in the architecture, all nodes are in normal mode, and a table called NodeInfo is input to the server node to rate the nodes based on their hardware and network conditions, providing a basis for the subsequent selection of the LeaderNode.

[0145] When the server receives a task request, it will select one node to appoint as the LeaderNode, and the other nodes will be ordinary nodes, together forming a node group to complete the scanning task.

[0146] When the server receives n task requests, it will select n nodes to appoint as LeaderNodes, and divide the remaining ordinary nodes into groups according to the size of the task to be scanned. Each node group is responsible for one scan task.

[0147] In a preferred embodiment of the present invention, a method for Server to select and control LeaderNode state switching is further included, as follows:

[0148] The server searches for the non-LeaderNode node with the highest rating in NodeInfo. If multiple nodes with the same rating exist, it selects one, marks it as the Leader, sends the appointment information to this node via the MQTT broker, and waits for a successful appointment reply. If no reply is received within the timeout period, the server enters the exception handling process.

[0149] After receiving the confirmation message that the appointment was successful, the server will distribute the ordinary nodes into each [LeaderNode-Node] node group according to the task size.

[0150] To ensure system efficiency, when the number of ordinary nodes in a [LeaderNode-Node] node group is less than MinSize, the reception of new tasks will be paused. MinSize is defined according to the actual situation.

[0151] After receiving the task completion information, the server sends a command to the LeaderNode bound to this task to switch to normal mode and waits for a successful reply. If no reply is received within the timeout period, the exception handling process begins.

[0152] After receiving a successful switchover response, the server removes the LeaderNode from the Leader state and restores it to a normal state. The LeaderNode, along with the other Nodes in its [LeaderNode-Node] group, is then assigned to the corresponding [LeaderNode-Node] group based on the size of the tasks in progress.

[0153] In a preferred embodiment of the present invention, the single-file level, cluster level, and project level rule sets used during three-layer hierarchical scanning are obtained by manually annotating and classifying industry standard rule sets, with the classification and annotation criteria being:

[0154] Scope-first: Prioritize the division based on the "file scope required for analysis", from single file → cluster → project. Scope is the most intuitive distinguishing criterion.

[0155] Dependency assistance: If the scope is ambiguous (e.g., spanning multiple files but without clearly defined cluster boundaries), determine whether it depends on project-level global relationship information. If it does, the dependency is project-level; otherwise, it is cluster-level.

[0156] Industry Adaptation: Based on the rule descriptions of specific industry standards, if the rules explicitly mention keywords such as "within the module" or "in the project," they can be directly used as the classification basis.

[0157] For example, CERT rule DCL01-C prohibits mixing declarations and code in the same block. This rule only focuses on the order of declarations and statements "within a single block" and does not require other file information, so it is marked as a single-file level rule.

[0158] CERT rule EXP34-C ensures that function parameter types match function definitions. If the function definition and call are in different files, the abstract syntax trees of both the call point and the definition point need to be analyzed to compare whether the parameter types are consistent. Therefore, it is marked as a cluster-level rule.

[0159] CERT - MSC04. Ensure that the encryption algorithm used in the project meets the security requirements. This rule checks the overall security of the encryption algorithm of the project and is an architectural-level project risk, therefore it is marked as a project-level rule.

[0160] In a preferred embodiment of the present invention, such as Figure 4 As shown, the first-stage file transfer and single-file-level scanning: After completing the node group allocation, the Server will issue single-file-level tasks through the Singletaskassign function, as follows:

[0161] , ,

[0162] Among them, taskqueue is an ordered single-file-level scan task queue generated by the server after receiving a task request, taskID is a task identifier used to match node groups for task allocation, and NodeMap is a node group allocation table.

[0163] After processing by the function, the Single_task allocation set will be output. Issue the i-th single-file level task, which includes the task issuance target. and task content Similarly, it is published to the corresponding Topic on the MQTT broker, and the Node subscribes to the relevant Topic based on the NodeID to obtain it;

[0164] After receiving a task, the Node starts requesting source files from the Client for scanning. The MQTT broker publishes a Phase 1 request Topic message. The Client receives the corresponding request message by matching the task ID and publishes a message carrying the source files to the Phase 1 source file transmission Topic. Ordinary nodes in the node group receive the requested source files by matching the Node ID. The LeaderNode subscribes to all Phase 1 source file transmission Topics under the corresponding task ID, thereby caching all source files on the LeaderNode and enabling the LeaderNode to become the near-end source file provider node in the Phase 2 source file transmission.

[0165] After receiving the source file, Node performs a single-file level scan. This process is represented as follows:

[0166] ,

[0167] ,

[0168] ,

[0169] In this context, FileInfo represents the source file content, Rule represents the single-file level rule set, IncludeSet (the set of file reference relationships in Single_res) is returned to the Server for constructing clusters, where Include represents the direct reference relationships of a single file, and SingleesSet (the set of single-file rule vulnerability scanning results) is returned to the Client. Each result is recorded using a unified ResultStruct structure, and SingleesSet is represented as follows:

[0170] ,

[0171] After receiving the IncludeSet, the Server parses out the Include information and integrates it into an IncludeInfo collection containing all file reference information.

[0172] .

[0173] In a preferred embodiment of the present invention, such as Figure 5 As shown, the two-stage file transfer and cluster-level and project-level scanning are performed:

[0174] After the single-file level scan is completed, the cluster level and project level scans will begin simultaneously in parallel. The project level scan is performed on the LeaderNode, and the specific steps are as follows:

[0175] When the LeaderNode receives the project-level scan start command from the Server, it checks whether all source files in the cache have been collected. If any are missing, it requests the missing files from the Client again to complete them. If the files are complete, it creates a new thread to perform a project-level scan.

[0176] ,

[0177] Among them, FileDict is a dictionary of source files, with filenames as keys and corresponding source file contents as values; Prule is a project-level rule set; ProgramresSet is a project-level scan result set, where each result is recorded using a unified ResultStruct structure.

[0178] The task performed in the ProgramScan function is as follows:

[0179] It iterates through all project files, obtains file references, class references, function calls and global parameter information for each file, and transforms the source code into an abstract syntax tree (AST) through a syntax parser, thus completing the preliminary information acquisition work required for scanning.

[0180] Based on the acquired prior information Establish global connection information across the entire project :

[0181] ,

[0182] ,

[0183] Among them, IncludeList contains file reference information, ClassList contains class reference information, FunctionList contains function call information, and GlobalVue contains global parameter information;

[0184] Traverse the abstract syntax tree (AST) structure and determine the relationships between nodes in the AST throughout the project using global relationship information. Trigger project-level rule checks for each node, specifically:

[0185] The project-level rules defined in Prule are matched sequentially. If a rule is found to be in violation, the violation is added to the result information. If no rule is found to be in violation, the process continues to the next node.

[0186] Meanwhile, the original LeaderNode thread continues to handle message responses from the MQTT broker.

[0187] In a preferred embodiment of the present invention, the server performs clustering based on the file reference relationships in the single-file level scan results, and the specific process is as follows:

[0188] S101, Select an unclustered entry, Include1, from IncludeInfo. Construct a stack based on the file reference relationship information Include1. The corresponding construction method is as follows:

[0189] ,

[0190] The referenced file elements in Include correspond to the referenced file elements in Stack;

[0191] S102, pop the top element from the stack, add it to the cluster, index this element in IncludeInfo, obtain the Include information of this element, and push the Include information onto the stack, represented as:

[0192] ,

[0193] S103, Repeat step S102 until the stack is empty, then perform deduplication on the files in the cluster to complete the clustering. The final cluster representation is as follows:

[0194] ,

[0195] Among them, ClusterHead in Cluster corresponds to FileName in the initially selected Include1;

[0196] S104, After the Include completes the clustering construction, it is marked as clustered information, and the clustering of this reference relationship is completed;

[0197] S105, Traverse all unclustered information in IncludeInfo, repeat steps S101-S104 until all file clustering tasks are completed;

[0198] The clustering results are represented as follows:

[0199] ,

[0200] After clustering is completed, the Server will distribute cluster-level tasks via the Clustertaskassign function, as follows:

[0201] , ,

[0202] ClusterDict is the cluster task dictionary, taskID is the task identifier, and Nodemap is the node group allocation table. The node group is found by matching taskID with NodeMap and the task is sent to the Node in the node group.

[0203] TaskInfo is a list of clustered source files. After the Node receives the task and parses the TaskInfo information, it no longer requests source files from the remote Client at this stage, but instead requests source files from the local Node LeaderNode.

[0204] After receiving the task, the Node no longer requests the source file from the remote Client at this stage, but instead requests the source file from the local LeaderNode.

[0205] Once Node receives the source file, it begins a cluster-level scan, which is represented as follows:

[0206] .

[0207] Among them, ClusterInfo contains the contents of all files in the cluster, with filenames and contents paired one-to-one; Crule is the cluster-level rule set; and ClusterresSet is the cluster-level scan result set, where each result is recorded using a unified ResultStruct structure.

[0208] Clustering based on file references (using .h files as cluster heads) fragments intra-cluster dependencies, hindering subsequent cluster-level rule scanning. Therefore, a compilation unit-based clustering approach (using .c files as cluster heads) is needed. Compilation unit-based clustering ensures complete dependencies within clusters, considering nested references. This clustering process can be divided into three steps: "resolving direct references → constructing nested references → building clusters," aligning with the characteristics of compilation units.

[0209] In a preferred embodiment of the present invention, the client integrates the received single-file level, cluster level, and project level results to obtain the final complete scan result (each result is a ResultStruct) as follows:

[0210] After project-level and cluster-level scans are completed, the results are returned to the client. These results are then integrated with the single-file-level scan results to obtain a complete three-level hierarchical scan result. The output information of the three-level hierarchical scans all use a unified structure, ResultStruct. After alignment, they can be merged.

[0211] ,

[0212] Among them, RuleId is the rule number of the scan anomaly, FilePath is the file path information of the file where the problem was found during the scan, Code is the specific code content of the problem found during the scan, Warning is the warning message, Suggestion is the optimization suggestion corresponding to the rule, and ResultType is the result level type (such as Single / Cluster / Project), which is used by the client to distinguish and integrate results from different levels.

[0213] Simple single-file and cluster-level two-layer hierarchical scanning results in missing project-level vulnerability warnings, and traditional client-to-server file transfer strategies are time-consuming. Therefore, it is necessary to implement a three-layer hierarchical rule scanning mode of single file, cluster, and project level, and a two-stage file transfer mode. First, perform single-file scanning, and then perform project-level and cluster scanning in parallel, progressing layer by layer to ensure complete scanning results.

[0214] The file transfer process involves two phases: the first phase retrieves the source file from the remote client, and the second phase retrieves it from the local server (LeaderNode). This reduces the impact of transmission latency on the efficiency of distributed scanning. Table 3 shows the comparison results of the three-layer hierarchical rule scanning, and the two-stage file transfer process is as follows: Figure 2 As shown.

[0215] Table 3 Comparison Results of Three-Level Hierarchical Rule Scanning

[0216]

[0217] This invention also provides a three-layer hierarchical scanning system based on a distributed framework and multi-task parallel processing, including a distributed transmission and computing architecture module based on a client-server-node group [LeaderNode-Node] architecture. The distributed transmission and computing architecture module executes the method described in this invention to obtain complete scanning results.

[0218] Static rule scanning tools play a crucial role in detecting coding rule vulnerabilities. However, most current rule scanning tools only support single-user, single-machine operation, and some distributed architecture tools do not consider simultaneous multi-task detection, resulting in low detection efficiency. This invention mainly studies distributed static coding rule scanning tools, constructs a distributed transmission and computing architecture that supports parallel execution of multiple tasks, and proposes a two-stage file transfer and a three-layer hierarchical scanning execution process to improve system performance.

[0219] This invention constructs a distributed transmission computing architecture that supports the parallel execution of multiple tasks, enabling the parallel operation of multiple tasks. This is suitable for situations where multiple tasks request simultaneously in actual use, effectively improving the scanning efficiency of static encoding rules. Furthermore, node group resources can be dynamically allocated, resulting in high server resource utilization.

[0220] This invention proposes a three-tiered hierarchical rule scanning mode: single-file, cluster, and project-level. Single-file scanning is performed first, followed by parallel project-level and cluster scanning. This multi-level detection reduces omissions during detection. A two-stage file transfer strategy is employed in the three-tiered scanning: the first stage retrieves the source file from the remote client, and the second stage retrieves the source file from the local server leadernode node, minimizing the impact of transmission latency on distributed scanning efficiency.

[0221] Furthermore, clustering based on the perspective of compilation units ensures complete file dependencies within the clusters, making the scan results more reliable and reducing false negatives and missed positives caused by fragmented dependencies.

[0222] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A multi-task parallel processing three-layer hierarchical scanning method based on a distributed framework, characterized in that, Comprising the following steps: Based on the publish / subscribe mode, the MQTT message middleware is constructed to control the transmission of information, and the distributed transmission computing architecture based on the client-server-node group [LeaderNode-Node] architecture is built; Single file level rule scanning is performed, the client obtains the project source file information to be scanned and publishes it to MQTT, the server selects a node as the LeaderNode of the new task and determines the ordinary node Node assisting the task, forms a node group to complete the single file level scanning, and returns the scanning result to the server Server, which is used for subsequent clustering, and the client completes the integration of the single file level scanning result; Parallel execution of project level and clustering level scanning tasks, project level scanning is undertaken by the LeaderNode, the server Server performs clustering according to the file reference relationship in the single file level scanning result, and the server Server sends the clustering task set to the ordinary node Node to complete the clustering level scanning; The client Client integrates the results of single file level, clustering level and project level after receiving them, that is, the final complete scanning result is obtained. 2.The multi-task parallel processing three-tier hierarchical scanning method based on a distributed framework according to claim 1, wherein, Based on the publish / subscribe mode, the method for building the MQTT message middleware is: The MQTT protocol manages the transmission of information through the topic Topic, the publisher sends the topic carrying data to the message broker MQTT broker, and the broker distributes the data to the nodes subscribing to the topic; In order to control that the specified Topic message can be sent to the correct node, the Topic name is used as the scheduling keyword, a Topic name is named "Source / Function / Symbol / TaskID", Source is the device name publishing this Topic, Function is the function name completed through this Topic communication, Symbol is the content mark name, the receiving end can distinguish the message ownership according to this identification, and TaskID is used to distinguish different user or different task information of the same user, which is composed of the client device number and the task number. 3.The multi-task parallel processing three-tier hierarchical scanning method based on distributed framework according to claim 1, wherein, The process of the client Client obtaining the project source file information to be scanned at the beginning of the single file level scanning stage is represented as: ， ， ， Where Project is the project to be scanned, TaskDict is the result dictionary of the scan summary, and FileInfo is the single file scan result, containing single file path and size information, ScanResult is the scan result of the i-th single file, containing single file path and size information ; This process represents the Scanf function scanning the specified project, and each source file to be scanned will generate a FileInfo containing the file path and file size. The TaskDict output by the function is the collection of all source file FileInfo, and TaskDict will be published to the task request Topic of the MQTT broker. 4.The method of claim 1, wherein, After the server Server is initialized, it is connected to the MQTT broker, the server Server subscribes to obtain TaskDict, and parses to obtain the single file level task queue taskqueue sorted by file size: ， Wherein, Sort is the sorting function; The server Server adjusts the node group, specifically: Select one from the non-Leader mode Node as the LeaderNode of the new task, and divide part of the basic Node to assist the task, the allocation process is completed by changing the node mapping table NodeMap on the server Server, and the LeaderNode is set by publishing messages on the node mode specified Topic of the MQTT broker.

5. The method of claim 1, wherein the method is a multi-task parallel processing three-tier hierarchical scanning method based on a distributed framework. The construction method of the [LeaderNode-Node] node group is specifically: Initially, no task is being carried out in the architecture, all Nodes are in normal mode, and a table NodeInfo is input to the Server node, which rates the Nodes according to the conditions of the hardware devices and network conditions where the Nodes are located, providing a basis for subsequent LeaderNode selection; When the Server receives a task request, it will select a Node to appoint as a LeaderNode, and other Nodes as normal nodes, which together form a node group to complete the scanning task; When the Server receives n task requests, it will select n Nodes to appoint as LeaderNodes, and divide the group affiliation of the remaining normal Nodes according to the size of the tasks to be scanned, with one node group responsible for one scanning task.

6. The multi-task parallel processing three-tier hierarchical scanning method based on a distributed framework according to claim 5, characterized in that, It also includes a method for Server to select and control LeaderNode state switching, as follows: The Server searches for the node with the highest rating among the non-LeaderNodes in NodeInfo, and if there are multiple Nodes with the same rating, it randomly selects one of them, marks it as a Leader, sends an appointment information to this Node through the MQTT broker, and waits for a successful reply to the appointment. If there is no reply within a certain time, it enters an abnormal state; After the Server receives the reply information of a successful appointment, it divides the normal nodes into each [LeaderNode-Node] node group according to the size of the task in proportion; When the number of normal Nodes in a [LeaderNode-Node] node group is less than MinSize, it suspends the reception of new tasks, and MinSize is defined according to actual conditions; After the Server receives the task completion information, it sends a command to switch to normal mode to the LeaderNode bound to this task, and waits for a reply to the successful switching. If there is no reply within a certain time, it enters an abnormal state; After the Server receives the reply information of a successful switching, it removes the LeaderNode from the Leader state and restores it to the normal state, and together with the remaining Nodes in the [LeaderNode-Node] node group, it divides them into the corresponding [LeaderNode-Node] node group according to the size of the ongoing task in proportion to work.

7. The method of claim 1, wherein the method is a multi-task parallel processing three-tier hierarchical scanning method based on a distributed framework. The single-file-level, cluster-level, and project-level rule sets used in three-level hierarchical scanning are obtained by manually annotating and classifying the rule sets of industry standards, with the following classification annotation criteria: Scope first: preferentially divide according to "the scope of the file required for analysis", single file -> cluster -> project, and the scope is the most intuitive standard of differentiation; Dependence assistance: if the scope is ambiguous, determine whether it depends on project-level global contact information, and if it does, it is project-level, otherwise it is cluster-level; Industry adaptation: combined with the rule description of specific industry standards, if the rule explicitly mentions the keywords "within the module" or "in the project", it can be directly used as a classification basis.

8. The multi-task parallel processing three-tier hierarchical scanning method based on a distributed framework according to claim 7, characterized in that, One-stage file transmission and single-file level scanning: After the node group allocation is completed, the Server will issue single-file level tasks through the Singletaskassign function, represented as: ， ， Where taskqueue is the ordered single-file level scanning task queue generated by the Server after receiving the task request, taskID is the task identification number used to match the node group for task allocation, and NodeMap is the node group allocation table; After function processing, the output Single_task distribution set is distributed, The i-th single file level task is issued, which contains the task issuing target And task content , also published to the corresponding Topic of MQTT broker, and the Node node subscribes to the related Topic according to NodeID acquisition; After receiving the task, the Node begins to request the source file from the Client for scanning, publishes a one-stage request Topic message on the MQTT broker, and the Client receives the corresponding request message by matching the taskID. The LeaderNode subscribes to all one-stage source file transmission Topics under the corresponding taskID, and caches all source files on the LeaderNode, so that the LeaderNode can become a near-end source file providing node in the two-stage source file transmission; After receiving the source file, the Node performs single-file level scanning, which is represented as: ， ， ， Where FileInfo is the source file content, Srule is the single-file level rule set, and the file reference relationship set IncludeSet in Single_res is returned to the Server for constructing the cluster, where Include is the direct reference relationship of the single file. The single-file rule vulnerability scanning result set SingleresSet is returned to the Client, and each result information is recorded in the unified structure ResultStruct structure. SingleresSet is represented as: ， After receiving IncludeSet, the Server parses the Include information and integrates it into the IncludeInfo set containing all file reference information: 。 9.The multi-task parallel processing three-tier hierarchical scanning method based on distributed framework according to claim 1, wherein, Two-stage file transmission and cluster level and project level scanning: After completing single-file level scanning, cluster level and project level scanning will start simultaneously and in parallel. Project level scanning is performed on the LeaderNode, and the specific steps are as follows: After receiving the project level scanning start instruction issued by the Server, the LeaderNode detects whether all source files in the cache are complete. If missing, it requests the missing files from the Client again for completion. If the files are complete, a new thread is created for project level scanning, represented as: ， Where FileDict is the source file dictionary, the key value is the file name, and the value is the corresponding source file content. Prule is the project level rule set, and ProgramresSet is the project level scanning result set, where each result information is recorded in the unified structure ResultStruct structure; The task performed in the ProgramScan function is: Traverse all project files to obtain file reference, class reference, function call and global parameter information of each file, and convert the source code into abstract syntax tree AST through the syntax parser to complete the pre-information acquisition required by scanning; According to the acquired pre-information Constructing global contact information between entire projects : ， ， Wherein, IncludeList is file reference information, ClassList is class reference information, FunctionList is function call information, and GlobalVue is global parameter information; Traverse the abstract syntax tree tree structure, and determine the connection of the nodes in the abstract syntax tree in the whole project through global connection information, trigger project-level rule checking for each node, specifically: Match the project-level rules defined in Prule one by one, and add the rule violation to the result information if the rule matching output violates the rule, and continue to traverse the next node if the rule does not violate the rule; At the same time, the original thread of the LeaderNode still processes the message response of the MQTT broker.

10. The multi-task parallel processing three-tier hierarchical scanning method based on a distributed framework according to claim 9, wherein, The Server clusters according to the file reference relationship in the single file level scanning result, and the specific process is as follows: S101, select an unclustered information Include1 from IncludeInfo, construct a stack according to the file reference relationship information Include1, and the corresponding construction method is: ， Wherein, the reference file element in Include corresponds to the reference file element in Stack; S102, pop the top element of the stack into the cluster, index this element in IncludeInfo, obtain the Include information of this element, and push the Include information into the stack, which is expressed as: ， S103, repeat step S102 until the stack is empty, and then perform a de-duplication operation on the files in the cluster to complete clustering, and the final obtained cluster representation is: ， Wherein, ClusterHead in Cluster corresponds to FileName in Include1 selected initially; S104, Include is marked as clustered information after completing the construction of clustering, and this reference relationship clustering is completed; S105, traverse all unclustered information in IncludeInfo, repeat steps S101-S104 until all file clustering tasks are completed; The clustering result is expressed as: ， After completing the clustering, the Server will assign the clustering level task through the Clustertaskassign function, which is expressed as: ， ， Wherein, ClusterDict is the cluster task dictionary, taskID is the task identifier, NodeMap is the node group allocation table, find out the node group through taskID and NodeMap matching, and assign the task to the Node in the node group; Wherein, TaskInfo is the list of clustered source files, Node receives the task and parses the information of TaskInfo, and in this stage, it no longer requests the source file from the remote Client, but requests the source file from the near-end node LeaderNode; After receiving the source file, the Node starts the clustering level scanning, and the process is expressed as: ， Wherein, ClusterInfo is the clustered file content, the file name and content are one-to-one, Crule is the clustered level rule set, ClusterresSet is the clustered level scan result set, wherein each result information is recorded by using the uniform structure body ResultStruct structure.

11. The multi-task parallel processing three-tier hierarchical scanning method based on a distributed framework according to claim 10, wherein, The method for the client Client to integrate the single file level, clustered level and project level results is as follows: After the project level scan and the clustered level scan are completed, the results are returned to the Client end, and the single file level scan result is integrated to obtain the complete three-level hierarchical scan result, the result information output by the three-level hierarchical scan is all recorded by using the uniform structure body ResultStruct, and after alignment, the results can be integrated: ， Wherein, RuleId is the number of the rule of the scan exception, FilePath is the file path information of the problem found in the scan, Code is the specific code content of the problem found in the scan, Warning is the warning information, Suggestion is the optimization suggestion corresponding to the rule, ResultType is the result belonging to the hierarchical type, and is used for the client Client to distinguish and integrate the results of different levels.

12. A multi-task parallel processing three-tier hierarchical scanning system based on a distributed framework, characterized in that, The distributed transmission computing architecture module based on the client Client-server Server-node group [LeaderNode-Node] architecture, the distributed transmission computing architecture module executes the method in any one of claims 1-11, and complete scan results are obtained.

Images