Distributed processing system for circuit board cloud detection data

Abstract

The invention relates to the technical field of distributed processing, in particular to a circuit board cloud detection data distributed processing system which comprises a task monitoring module, a computing power scheduling module, a task distribution module, a defect comparison module and an archiving dual-backup module. In the invention, through careful monitoring and intelligent scheduling of analysis requirements of circuit board detection data, the efficiency and accuracy of data processing are improved, the matching degree of the task data volume and the node uploading bandwidth is evaluated, the reasonable distribution of the data processing tasks among the nodes is ensured, and the data processing efficiency is improved. The task load can be dynamically adjusted according to the availability of real-time computing resources, so that the resource utilization rate is optimized, the defect part in the image can be quickly recognized by extracting and comparing key features of the circuit image, the recognition accuracy is improved, the integrity and safety of data are ensured through a dual-standby filing mechanism, and the method is suitable for large-scale popularization and application. And when the main archiving node goes wrong, the standby node can take over quickly, so that the risk of data loss is reduced.

Description

Technical Field

[0001] The present invention relates to the field of distributed processing technology, and in particular to a distributed processing system for cloud-based detection data of circuit boards. Background Art

[0002] The field of distributed processing technology involves technical solutions for parallel or collaborative data storage, computing, and transmission in a multi-node environment. Specifically, this involves distributing the data processing load across multiple computing devices through task partitioning and scheduling, ensuring efficient and consistent data exchange between nodes through network communication protocols, and improving overall system reliability and scalability through redundancy and fault-tolerance mechanisms. This field often utilizes specific technologies such as consistency algorithms, distributed storage systems, and distributed computing frameworks to address high-concurrency access, massive data processing, and high-availability system deployment in large-scale data environments.

[0003] The PCB cloud-based distributed inspection data processing system processes test data generated during the PCB inspection process through a distributed cloud server cluster. Through task partitioning and node scheduling, the system distributes raw data from PCB inspection equipment to multiple processing nodes, utilizing parallel computing to perform data analysis, defect identification, and storage of processing results. Applications include improving the efficiency and accuracy of PCB inspection data processing, reducing the risk of single points of failure, and enabling rapid archiving, remote access, and intelligent analysis of test data. It is suitable for large-scale PCB production inspection scenarios.

[0004] Traditional processing systems lack flexibility and specificity in allocating data tasks, leading to an uneven load between nodes and impacting the overall system's processing efficiency and response time. Furthermore, when processing complex or large-scale data, traditional systems lack effective real-time monitoring and adjustment mechanisms, making them unable to effectively respond to node failures or network fluctuations, which can easily lead to data processing delays or interruptions. Traditional systems also lack the accuracy and efficiency of identifying data defects, requiring additional time and resources to ensure the quality and accuracy of data analysis. Furthermore, data backup and recovery strategies are not effectively optimized in many existing technologies. Once a system failure occurs, the complexity and uncertainty of data recovery increase significantly, impacting the stability and security of the entire production process. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a distributed processing system for circuit board cloud detection data.

[0006] In order to achieve the above objectives, the present invention adopts the following technical solution: a distributed processing system for circuit board cloud detection data, the system comprising: The task monitoring module obtains the inspection task data uploaded by multiple circuit board inspection devices, calls the upload status information of each node in the node monitoring list, evaluates the adaptability of the node upload based on the task content type and node distribution success rate, and generates a list of adapted distribution nodes; The computing power scheduling module calls the processor occupancy rate, number of active threads, and number of available memory blocks of each node based on the adapted distribution node list, calculates the node computing power level, and obtains the node computing power quantification result; The task allocation module calls the node computing power quantification result, calculates and evaluates the task adaptability based on the number of image blocks and the node computing power level, combined with the task processing step size and the memory idle ratio, combines and binds the task number and the node number, and generates task binding scheduling information; The defect comparison module performs a difference operation between the image feature vector and the reference image vector of the same structure in the standard image library based on the task binding scheduling information, records the corresponding position index, circuit number and defect type for the image block exceeding the threshold, and generates image defect location information.

[0007] As a further solution of the present invention, the adaptation distribution node list specifically includes the node upload bandwidth value, the node distribution historical success rate, and the node data synchronization delay; the node computing power matching list includes the instantaneous available CPU value, the thread processing capability value, and the memory idle ratio; the task binding scheduling information specifically refers to the image block quantity binding information, the circuit structure partition data information, and the processing step matching index; the image defect location information specifically includes the boundary direction feature offset, the continuous change abnormal point, and the signal jump intensity abnormality record.

[0008] As a further solution of the present invention, the task monitoring module includes: The task data extraction submodule obtains the inspection task data uploaded by multiple circuit board inspection devices, extracts the data volume parameters, circuit image quantity parameters, and inspection content type parameters from the inspection task data, performs total statistics on the extracted data volume parameters, sums the number of image blocks for the circuit image quantity parameters, and performs category classification and counting on the inspection content type parameters to generate an inspection task data information table; Based on the detection task data information table, the node status monitoring submodule calls the upload status information of each node in the node monitoring list, obtains the node's current upload bandwidth rate, the number of successful upload tasks per unit time, the number of upload task interruptions, and the task processing request queue length, and calculates the node upload performance index after data standardization. The upload performance index corresponding to each node is screened, and the node number and corresponding bandwidth value whose performance index ranking is within the target range are obtained to generate the node bandwidth capacity screening result; The node adaptation evaluation submodule uses the node bandwidth capacity screening results, calls the historical distribution task success rate of the screened nodes, performs a weighted combined score on the success rate and the number of task data content types, and judges whether the score value is greater than the task content adaptation threshold to obtain a set of nodes that meet the adaptability standards and generate a list of adapted distribution nodes.

[0009] As a further solution of the present invention, the formula for calculating the node upload performance index is: ; in, Representative Node upload performance index, Representative The upload bandwidth rate of each node, Representative The number of upload task interruptions per node per unit time, Representative The length of the task processing request queue of each node, Representative The number of tasks that are successfully uploaded by a node per unit time.

[0010] As a further solution of the present invention, the computing power scheduling module includes: The processor resource extraction submodule obtains the processor occupancy rate of the node based on the adaptation distribution node list, extracts the processor occupancy rate parameter and the node identification number, calculates the processor idle rate, calls the processor idle rate and the node identification number, filters the nodes whose processor idle rate is greater than the set idle threshold, and generates a node processor idle rate list; The thread and memory extraction submodule calls the number of active threads and available memory blocks of the filtered node based on the node processor idle rate list, obtains the thread activity amount and the memory available ratio respectively, performs a proportional operation on the thread activity amount and the memory available ratio, sets the ratio threshold as the thread memory load balancing threshold, determines whether the ratio value falls within the load balancing interval, identifies the node information that meets the load balancing conditions, and generates a node load balancing data table; The node computing power synthesis submodule extracts the processor idle rate, thread activity, and memory available ratio corresponding to each node based on the node load balancing data table, assigns weight parameters to each of them, and performs weighted summation. The weighted result is compared with the node computing power level division benchmark value to obtain the corresponding level range of the node computing power and establish the node computing power quantification result.

[0011] As a further solution of the present invention, the task allocation module includes: The task information extraction submodule calls the node computing power quantization result, obtains the data content of each detection task, extracts the number of image blocks, the number of circuit structure partitions and the processing step length, records the total value of the number of image blocks, marks the number of circuit structure partitions by level, classifies the processing step length by interval, and generates a task processing structure data table; The task adaptation evaluation submodule calls the node computing power level data according to the task processing structure data table, extracts the node memory idle ratio, combines the number of image blocks, the number of circuit structure partitions and the processing step length to construct the task load, performs quantitative matching based on the adaptation between the task load and the node computing power level and the memory idle ratio, calculates the task adaptation value, combines the task nodes whose task adaptation values ​​are greater than the matching reference value, and generates a task node adaptation matching table; The number binding generation submodule calls the correspondence between task numbers and node numbers based on the task node adaptation matching table, sorts the task numbers in partition order, prioritizes the node numbers according to the adaptation value, combines and binds the sorted number sets, and establishes task binding scheduling information.

[0012] As a further solution of the present invention, the formula for calculating the task adaptation value is: ; in, Indicates the Task and The adaptation value of each node, Indicates the The computing power level of each node, Indicates the The memory idle ratio of each node, Indicates the The number of image patches for each task, Indicates the The number of circuit structure partitions for each task, Indicates the The processing step of each task, Indicates the average task processing step length.

[0013] As a further solution of the present invention, the defect comparison module includes: The feature value extraction submodule extracts the boundary direction feature value, boundary continuous change value, and signal jump strength value of each image block based on the task binding scheduling information and the circuit image block data assigned to the corresponding node, classifies and organizes the three extracted feature values ​​according to the image block sequence, classifies and stores them according to the image block number, and generates an image block feature classification table; The feature vector comparison submodule calls the reference image feature vector corresponding to each image block structure in the standard image library based on the image block feature classification table, calculates the difference between each feature value and the corresponding reference feature value, sets a feature offset threshold for the difference value, and filters and records the image block numbers that exceed the feature offset threshold to obtain a feature offset anomaly list; The defect record generation submodule extracts the position index corresponding to the abnormal image block based on the feature offset anomaly list, retrieves the circuit number and defect type classification code associated with the image block, forms a triplet format of the position index, circuit number and defect type, and establishes image defect location information.

[0014] As a further embodiment of the present invention, the system further comprises: The archiving dual-standby module calls the image defect location information, obtains the defect type, circuit number and task number of the defective image block, detects the processing load rate and cache occupancy ratio of the current archiving node, calculates the node hot standby value, determines whether the node hot standby value is lower than the hot standby standard value, and when it is lower, synchronously writes the image block and metadata to the primary archiving node and the standby node, records the write confirmation information and the data block check code, and generates distributed archiving write record information; The distributed archiving write record information includes a defective image block index, a data block check code and archiving node confirmation information.

[0015] As a further solution of the present invention, the archiving dual-backup module includes: The archiving node detection submodule calls the image defect location information, and calls the processing load rate and cache occupancy ratio data of the current archiving node according to the defect type, circuit number and task number of the defective image block, normalizes the archiving node processing load rate and cache occupancy ratio and calculates the average value, classifies and filters the nodes according to the changing trends of the processing load rate and cache occupancy ratio, and generates an archiving node load status table; The node hot standby judgment submodule extracts the average load value and cache occupancy rate of the archive node based on the archive node load status table, performs weighted accumulation operation on the average load value and the cache occupancy rate, compares the weighted accumulation value with the node hot standby standard value, and screens a list of nodes with values ​​lower than the node hot standby standard value to obtain an archive node hot standby screening table; The dual-standby synchronous writing submodule extracts the node information that meets the hot standby requirements according to the archive node hot standby screening table, writes the corresponding defective image blocks and associated metadata into the primary archive node and the backup archive node at the same time, generates a check code for the written data block, records the synchronous write confirmation information, and establishes distributed archive write record information.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, through careful monitoring and intelligent scheduling of the analysis needs of circuit board detection data, the efficiency and accuracy of data processing are improved, the matching degree of task data volume and node upload bandwidth is evaluated, and then the node upload adaptability is reasonably evaluated, thereby ensuring the reasonable distribution of data processing tasks among various nodes. The task load can be dynamically adjusted according to the availability of real-time computing resources, thereby optimizing resource utilization and reducing processing delays. By extracting and comparing key features of circuit images, defective parts in the image can be quickly identified, improving the recognition accuracy and shortening the diagnosis time. The dual-backup archiving mechanism ensures the integrity and security of the data. When a problem occurs in the main archiving node, the backup node can quickly take over, reducing the risk of data loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a system flow chart of the present invention; Figure 2 Schematic diagram of the system framework of the present invention; Figure 3 This is a flowchart of the task monitoring module of the present invention; Figure 4 This is a flow chart of the computing power scheduling module of the present invention; Figure 5 A flowchart of the task allocation module of the present invention; Figure 6 This is a flow chart of the defect comparison module of the present invention; Figure 7 The flowchart of the dual standby module of the present invention is archived. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0019] In the description of the present invention, it should be understood that the terms "length," "width," "up," "down," "front," "back," "left," "right," "vertical," "horizontal," "top," "bottom," "inside," "outside," and the like, indicating positions or relationships, are based on the positions or relationships shown in the accompanying drawings and are intended only to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limiting the present invention. Furthermore, in the description of the present invention, "plurality" means two or more, unless otherwise expressly and specifically defined.

[0020] See also Figure 1 , a distributed processing system for circuit board cloud detection data, the system includes: The task monitoring module obtains the inspection task data uploaded by multiple circuit board inspection devices, extracts the data volume parameters, circuit image number parameters, and inspection content type parameters in the inspection task, calls the upload status information of each node in the node monitoring list, compares the inspection task data volume with the node upload bandwidth value, evaluates the adaptability of the node upload based on the task content type and the node distribution historical success rate, and generates a list of adapted distribution nodes; The computing power scheduling module, based on the list of adapted distribution nodes, calls each node's processor occupancy rate, number of active threads, and number of available memory blocks, extracts the node's instantaneous available CPU value, thread processing capacity value, and memory idle ratio, performs weighted synthesis, calculates the node computing power level, and obtains the node computing power quantification result; The task allocation module uses the node computing power quantification results to extract the number of image blocks, the number of circuit structure partitions, and the processing step size in each detection task. Based on the number of image blocks and the node computing power level, combined with the task processing step size and the memory idle ratio, it calculates and evaluates the task adaptability, combines the task number and the node number, and generates task binding scheduling information. The defect comparison module is based on task binding scheduling information. Each node calls the assigned circuit image block data, extracts the image block boundary direction feature value, boundary continuous change value, and signal jump strength value, and establishes a set of feature value difference vectors. Each set of feature vectors is subtracted from the reference image vector of the same structure in the standard image library, and a feature offset threshold is set. For image blocks that exceed the threshold, the corresponding position index, circuit number, and defect type are recorded to generate image defect location information. The archive dual-backup module calls image defect location information, obtains the defect type, circuit number, and task number of the defective image block, detects the processing load rate and cache occupancy ratio of the current archive node, calculates the node hot standby value, and determines whether the node hot standby value is lower than the hot standby standard value. If it is lower, the image block and metadata are synchronously written to the primary archive node and the backup node, and the write confirmation information and data block check code are recorded to generate distributed archive write record information; The list of adapted distribution nodes specifically includes the node upload bandwidth value, the node distribution historical success rate, and the node data synchronization delay. The node computing power matching list includes the instantaneous available CPU value, thread processing capability value, and memory idle ratio. The task binding scheduling information specifically refers to the image block quantity binding information, circuit structure partition data information, and processing step matching index. The image defect positioning information specifically includes the boundary direction feature offset, continuous change anomaly points, and signal jump intensity anomaly records. The distributed archive write record information includes the defective image block index, data block check code, and archive node confirmation information.

[0021] See also Figure 2 and Figure 3,The task monitoring module includes the task data extraction submodule, the node ,status monitoring submodule and the node adaptation evaluation submodule; The task data extraction submodule obtains the inspection task data uploaded by multiple circuit board inspection devices, extracts the data volume parameters, circuit image quantity parameters, and inspection content type parameters from the inspection task data, performs total statistics on the extracted data volume parameters, sums the number of image blocks for the circuit image quantity parameters, and performs category classification and counting on the inspection content type parameters to generate an inspection task data information table; The task data extraction submodule obtains the inspection task data uploaded by multiple circuit board inspection devices. For the acquisition operation, the circuit board inspection devices are connected one by one, and the task data file stored in the device is obtained through the communication protocol. The data volume parameter, circuit image number parameter and inspection content type parameter recorded in the task data file are read. The data volume parameter corresponds to the number of inspection data items in each task, the circuit image number parameter corresponds to the number of circuit images generated for each task, and the inspection content type parameter corresponds to the number of categories of inspection items in each task. In the example, three devices are connected, numbered A1, A2, and A3 respectively. The task data uploaded by A1 contains 1200 data items, 350 circuit images, and 4 types of inspection content. The task data uploaded by A2 contains 800 data items, 200 circuit images, and 4 types of inspection content. There are 3 types of detection content. The A3 uploaded task data contains 1000 data items, 250 circuit images, and 5 types of detection content. The total amount of the extracted data volume parameters is counted. Specifically, the data volume uploaded by each device is directly added up, and the total amount of data is calculated as 1200+800+1000=3000. The number of image blocks is summed for the circuit image quantity parameter. Specifically, the number of images uploaded by each device is added up, and the total number of images is calculated as 350+200+250=800. The detection content type parameters are classified and counted. Specifically, the content type is used as the classification basis, and the type content in A1, A2, and A3 is marked respectively. The number of occurrences and distribution of different types are counted to form a corresponding table of types and corresponding quantities, and then the detection task data information table is generated.

[0022] The node status monitoring submodule, based on the detection task data information table, calls the upload status information of each node in the node monitoring list to obtain the node's current upload bandwidth rate, the number of successful upload tasks per unit time, the number of upload task interruptions, and the task processing request queue length. After data standardization, it calculates the node upload performance index, screens the upload performance index corresponding to each node, obtains the node number and corresponding bandwidth value whose performance index ranking is within the target range, and generates the node bandwidth capacity screening result; The formula for calculating the node upload performance index is: ; in, Representative Node upload performance index, Representative The upload bandwidth rate of each node, Representative The number of upload task interruptions per node per unit time, Representative The length of the task processing request queue of each node, Representative The number of tasks uploaded successfully per node per unit time; The node upload performance index is used to measure the network transmission stability and bandwidth utilization efficiency of distributed nodes during the task upload phase. It is a key indicator for screening node upload capabilities before task scheduling. Operational item logic: Item 1 : Indicates the bandwidth availability correction item. The more interruptions there are, the smaller the value is, reflecting the negative impact of network instability. Item 2 : Indicates the request response efficiency adjustment item. When the upload amount is successfully Below the queue When , the value becomes larger, indicating that the node response is insufficient.

[0023] therefore, The larger the value, the better the upload stability, processing response efficiency and transmission bandwidth capability of the node, and the more suitable it is as a task distribution entry.

[0024] The node status monitoring submodule calls each node in the node monitoring list to upload status information based on the detection task data information table. For the call operation, it sends an upload status information request instruction to each node. The node list is set to include nodes N1, N2, and N3. N1 returns an upload bandwidth rate of 800 Mbps, 90 successful upload tasks per unit time, 2 upload task interruptions, and a task processing request queue length of 15. N2 returns an upload bandwidth rate of 600 Mbps, 70 successful upload tasks, 4 interruptions, and a request queue length of 20. N3 returns an upload bandwidth rate of 750 Mbps, 80 successful upload tasks, 3 interruptions, and a request queue length of 18. The data of each node parameter obtained is normalized, and the maximum and minimum value normalization method is used to process each parameter. The formula is: ,in is the original data, For the standardized data, 、 They are the minimum and maximum values ​​of the same column of data respectively, and then the normalized bandwidth rate, number of successes, number of interruptions, and queue length of each node are compared under a unified scale. According to the formula Compute node upload performance index, where Representative The upload bandwidth rate of each node, is the number of upload task interruptions per unit time, The length of the task processing request queue, The number of tasks that have been successfully uploaded per unit time. The actual data brought in is as follows: , calculate N2 and N3 respectively in this way to obtain the upload performance index of all nodes, sort the performance index in descending order, and select the node numbers and corresponding bandwidth values ​​with performance index ranking in the top 30% as the node bandwidth capacity screening results.

[0025] The node adaptation assessment submodule filters the results based on the node bandwidth capacity, calls the historical distribution task success rate of the filtered nodes, and performs a weighted combined score on the success rate and the number of task data content types. It then determines whether the score is greater than the task content adaptation threshold, obtains the node set that meets the adaptability criteria, and generates a list of adapted distribution nodes. The node adaptation evaluation submodule filters the results based on the node bandwidth capacity, calls the historical distribution task success rate of the filtered node, queries its task distribution record for each node for the call operation, extracts the number of successful and failed distribution tasks in the past 30 days, and sets the number of successful tasks distributed by node N1 in 30 days to 270 and the number of failed tasks to 30. The success rate is calculated as , the success rate of N2 is After the success rate is obtained, the node success rate and the number of task data content types are weighted and scored. The weighted scoring formula is set as ,in 、 , ContentTypeCount is the number of task data content types, the number of N1 node task content types is set to 4, then ,Set the task content adaptation threshold as 0.8, and judge based on whether the score value is greater than 0.8. The N1 node score 0.84 is greater than 0.8, N1 meets the adaptability standard, and the N2 score is lower than 0.8, which does not meet the standard. Finally, the node set that meets the adaptability standard is obtained to generate the adaptation distribution node list.

[0026] See also Figure 2 and Figure 4 ,The computing power scheduling module includes a processor resource extraction submodule, a thread and memory extraction submodule, and a node computing power synthesis submodule; The processor resource extraction submodule obtains the processor occupancy rate of the node based on the list of adapted distribution nodes, extracts the processor occupancy rate parameters and the node identification number, calculates the processor idle rate, calls the processor idle rate and the node identification number, filters out the nodes whose processor idle rate is greater than the set idle threshold, and generates a node processor idle rate list; The processor resource extraction submodule is based on the list of adapted distribution nodes. First, it reads the node identification numbers in the list of adapted distribution nodes one by one, and sends a query instruction to each node to obtain the current processor occupancy of the node. The processor occupancy is recorded in percentage form. For example, the processor occupancy of node A is 65%, node B is 40%, and node C is 75%. After extracting the processor occupancy parameters and node identification number of each node, the processor idle rate is calculated. The idle rate = 100% - occupancy method is used for node-by-node calculation. For example, the processor idle rate of node A is 35%, node B is 60%, and node C is 75%. Point C is 25%. The processor idle rate and node identification number are called to form a temporary correspondence table. Nodes with processor idle rates greater than the set idle threshold are screened out. The idle threshold is set to 50%. During the screening process, the processor idle rate of each node is compared with the set idle threshold. When the idle rate is greater than 50%, the node number is recorded. Otherwise, it is eliminated. For example, if the idle rate of node B is 60% and greater than 50%, it is retained. If the idle rates of nodes A and C are 35% and 25% respectively, both less than 50%, they are eliminated. Finally, the node B numbers and their idle rates that meet the conditions are screened out to generate a node processor idle rate list.

[0027] The thread and memory extraction submodule, based on the node processor idle rate list, calls the number of active threads and available memory blocks of the filtered node, obtains the thread activity amount and the available memory ratio respectively, performs a proportional operation on the thread activity amount and the available memory ratio, sets the ratio threshold as the thread memory load balancing threshold, determines whether the ratio value falls into the load balancing range, identifies the node information that meets the load balancing conditions, and generates a node load balancing data table; The thread and memory extraction submodule is based on the node processor idle rate list. First, it calls each node in the node processor idle rate list one by one, sends a query instruction to request the node to return the number of active threads and the number of available memory blocks. The thread activity is measured by the number of active threads. For example, node B returns 80 active threads. The memory availability ratio is expressed as the ratio of the number of available memory blocks to the total number of memory blocks. For example, the number of available memory blocks of node B is 160 and the total number of memory blocks is 256. The available memory ratio is 160 / 256=62.5%. After obtaining the thread activity and memory availability ratio respectively, the proportional calculation is performed. The calculation method is to divide the thread activity by the available memory ratio. For example, the ratio value of node B is 80 / (62.5%)=80 / 0.625=128. The ratio threshold range is set as the load balancing threshold, and the load balancing interval is [100,150]. It is judged whether the ratio value falls into the load balancing interval. The node B ratio value 128 is judged. Since 128 is between 100 and 150, it is determined that node B meets the load balancing conditions. The node information that meets the load balancing conditions is identified, and the node B and its corresponding thread activity and memory available ratio are recorded to generate a node load balancing data table.

[0028] The node computing power synthesis submodule extracts the processor idle rate, thread activity, and memory available ratio corresponding to each node based on the node load balancing data table, assigns weight parameters to each, and then performs weighted summation. The weighted result is compared with the node computing power level division benchmark value to obtain the node computing power corresponding level range and establish the node computing power quantification result; The node computing power synthesis submodule first extracts the processor idle rate, thread activity, and memory available ratio corresponding to each node based on the node load balancing data table. The data extracted from node B is a processor idle rate of 60%, a thread activity of 80, and a memory available ratio of 62.5%. Weight parameters are assigned to each of them, setting the processor idle rate weight to 0.4, the thread activity weight to 0.3, and the memory available ratio weight to 0.3. A weighted sum is performed, and the weighted summation formula is: , substituting the parameter values ​​into the calculation, the weighted sum score of node B is , based on the weighted results and the node computing power level classification benchmark value, the classification benchmark value is set as: 0-40 is low level, 40-70 is medium level, and above 70 is high level. Node B's weighted score of 66.75 is between 40 and 70, and is classified as a medium-level node. Finally, the node computing power quantification result is established, and node B is recorded as a medium-level node and its corresponding weighted score.

[0029] See also Figure 2 and Figure 5 ,The task assignment module includes a task information extraction submodule, a task ,adaptation evaluation submodule, and a number binding generation submodule; The task information extraction submodule calls the node computing power quantification results to obtain the data content of each detection task, extracts the number of image blocks, the number of circuit structure partitions and the processing step length, records the total value of the number of image blocks, marks the number of circuit structure partitions by level, classifies the processing step length by interval, and generates a task processing structure data table; The task information extraction submodule calls the node computing power quantization result, reads the node number corresponding to each detection task one by one, obtains the data content of each detection task, and extracts the number of image blocks, the number of circuit structure partitions and the processing step from the data content. The number of image blocks is obtained by parsing the image block field in the task data packet. For example, the number of image blocks for task 1 is 120 blocks, for task 2 is 150 blocks, and for task 3 is 90 blocks. When extracting the number of circuit structure partitions, the circuit diagram topology field is read, and the number of structural branch nodes is counted as the number of partitions. For example, the number of partitions for task 1 is 5, the number of partitions for task 2 is 7, and the number of partitions for task 3 is 4. When extracting the processing step, the step value is read according to the processing parameter field in the data packet. The unit is pixel. For example, the step length of task 1 is 10 pixels, the step length of task 2 is 15 pixels, and the step length of task 3 is 8 pixels. The total number of blocks is recorded, and the number of image blocks of all tasks is added together to form the total number of blocks, for example, 120+150+90=360 blocks. The number of circuit structure partitions is marked with a hierarchy, and the number of partitions is divided into three levels: 0-3 is a low level, 4-6 is a medium level, and 7 and above is a high level. Task 1 has a partition number of 5 that belongs to the medium level, task 2 has a partition number of 7 that belongs to the high level, and task 3 has a partition number of 4 that belongs to the medium level. The processing step size is classified by intervals, and the step size is divided into short step size (0-8 pixels), medium step size (9-14 pixels), and long step size (above 15 pixels). Task 1 has a step size of 10 pixels that is classified as a medium step size, task 2 has a step size of 15 pixels that is classified as a long step size, and task 3 has a step size of 8 pixels that is classified as a short step size. Finally, a task processing structure data table is generated based on the number of image blocks, the hierarchy of the number of circuit structure partitions, and the classification of processing steps.

[0030] The task adaptation evaluation submodule calls the node computing power level data based on the task processing structure data table, extracts the node memory idle ratio, combines the number of image blocks, the number of circuit structure partitions and the processing step length to construct the task load, performs quantitative matching based on the adaptation between the task load and the node computing power level and the memory idle ratio, calculates the task adaptation value, combines the task nodes whose task adaptation values ​​are greater than the matching benchmark value, and generates a task node adaptation matching table; The formula for calculating the task adaptation value is: ; in, Indicates the Task and The adaptation value of each node, Indicates the The computing power level of each node, Indicates the The memory idle ratio of each node, Indicates the The number of image patches for each task, Indicates the The number of circuit structure partitions for each task, Indicates the The processing step of each task, Represents the average value of task processing step length; The task adaptation value is used to measure the comprehensive degree of adaptation between a task (especially a circuit image processing task) and a computing power node in terms of resource matching.

[0031] Operational logic analysis: molecular : represents the current schedulable capability of the node; Denominator : represents the load intensity index of the task; When the number of images is large, the number of partitions is large, or the processing step size deviates from the mean, the computational intensity increases.

[0032] Therefore, the adaptation value The larger the value, the more "matched" the task is to be processed on the node. That is, the node has sufficient capacity and can meet the task load, which is conducive to subsequent scheduling and binding.

[0033] The task adaptation evaluation submodule first reads each task data in the task processing structure data table according to the task processing structure data table, calls the node computing power level data and the node memory idle ratio data, extracts the node computing power level, for example, node 1 level is 3, node 2 level is 4, node 3 level is 2, extracts the node memory idle ratio, for example, node 1 is 65%, node 2 is 70%, node 3 is 55%, and combines the number of image blocks, the number of circuit structure partitions and the processing step length of each task to construct the task load. For example, the load of task 1 is 120 image blocks, 5 circuit partitions and a step length of 10 pixels. According to the formula Compute the task adaptation value, where is the node computing power level, is the node memory idle ratio, is the number of task image blocks, is the number of task circuit partitions, is the task processing step length, is the average value of task processing step length, set Pixel, the adaptation value of node 1 and task 1 is , perform the same calculation on all task node combinations, set the matching benchmark value to 0.02, and screen out task node combinations with adaptation values ​​greater than 0.02. The adaptation value of node 2 and task 2 is 0.025, which meets the condition. The adaptation value of node 3 and task 3 is 0.018, which does not meet the condition. Finally, a task node adaptation matching table is generated, recording the task number and the node number of the successful match.

[0034] The number binding generation submodule calls the correspondence between task numbers and node numbers based on the task node adaptation matching table, sorts the task numbers according to the partition order, prioritizes the node numbers according to the adaptation value, combines and binds the sorted number sets, and establishes task binding scheduling information; The number binding generation submodule is based on the task node adaptation matching table. It first calls the correspondence between the task number and the node number, reads each task and the corresponding node number, and sorts the task numbers in partition order. The partition order is sorted from small to large according to the number of circuit structure partitions. For example, task 3 has 4 partitions and ranks first, task 1 has 5 partitions and ranks second, and task 2 has 7 partitions and ranks third. The node numbers are prioritized according to the adaptation value, and the adaptation value is arranged from high to low. For example, the adaptation value of node 2 is 0.025, which is higher than that of node 1 of 0.0192, and the adaptation value of node 3 is 0.018, which is the lowest. Therefore, node 2 takes priority, node 1 is second, and node 3 is finally. The sorted task numbers are combined and bound with the node number set, and priority node numbers are assigned according to the task partition order. Task 3 is bound to node 2, task 1 is bound to node 1, and task 2 is bound to node 3. Finally, the task binding scheduling information is established.

[0035] See also Figure 2 and Figure 6 ,The defect comparison module includes a feature value extraction submodule, a feature vector comparison submodule, and a defect record generation submodule; The feature value extraction submodule extracts the boundary direction feature value, boundary continuous change value, and signal jump strength value of each image block based on the task binding scheduling information and the circuit image block data assigned to the corresponding node. The three extracted feature values ​​are classified and sorted according to the image block sequence, and the image blocks are classified and stored according to the image block number to generate an image block feature classification table. The eigenvalue extraction submodule is based on the task binding scheduling information. It first reads the correspondence between each task and the binding node one by one, calls the data content assigned to the node, parses each circuit image block, extracts the boundary direction eigenvalue of each image block, and extracts the main direction angle value in the image boundary area through the image gradient direction detection operator. The unit is degree. For example, the boundary direction eigenvalue of image block 001 is extracted as 45 degrees, and that of image block 002 is extracted as 30 degrees. When extracting the continuous change value of the boundary, the gradient change amplitude of the continuous boundary points in the image block is accumulated. For example, the continuous change value of the boundary of image block 001 is 120, and that of image block 002 is 95. The signal jump strength is extracted. When the image block has a value, the intensity difference between the high and low grayscale areas in the image block is detected and the maximum change value is recorded. For example, the signal jump intensity value of image block 001 is 200, and that of image block 002 is 180. According to the order of the image blocks, the three extracted eigenvalues ​​are sorted separately and classified according to the image block number. Each image block number corresponds to a set of eigenvalue records. The image block number is determined by the sequential number field in the task data structure. For example, the image block numbers of task A are from 001 to 100 in sequence, and those of task B are from 101 to 200. Finally, an image block feature classification table is generated, which records each image block number and the corresponding boundary direction eigenvalue, boundary continuous change value and signal jump intensity value.

[0036] The feature vector comparison submodule uses the image block feature classification table to call the reference image feature vector corresponding to each image block structure in the standard image library, calculates the difference between each feature value and the corresponding reference feature value, sets a feature offset threshold for the difference value, and filters and records the image block numbers that exceed the feature offset threshold to obtain a feature offset anomaly list; The feature vector comparison submodule is based on the image block feature classification table, calls the data group corresponding to each image block number one by one, and reads the reference image feature vector corresponding to the image block structure in the standard image library. The reference feature vector stores the standard boundary direction, boundary continuous change and signal jump strength parameter values. For example, the standard boundary direction of image block 001 is 47 degrees, the standard boundary continuous change value is 115, and the standard signal jump strength is 210. The difference between each feature value and the corresponding reference feature value is calculated respectively. The difference value is calculated in the absolute value difference method. For example, the boundary direction difference value of image block 001 is |45-47|=2 degrees, the boundary continuous change difference value is |120-115|=5, and the signal jump strength difference value is |200-21 0|=10, set the feature offset threshold for the above difference values, set the boundary direction offset threshold to 3 degrees, the boundary continuous change offset threshold to 10, and the signal jump intensity offset threshold to 15, compare each difference value with the set threshold one by one, and determine it as abnormal when the difference value is greater than the corresponding threshold. For example, the boundary direction difference of image block 001 is 2 degrees less than 3 degrees, which is not abnormal, the boundary continuous change is 5 less than 10, which is not abnormal, and the signal jump intensity is 10 less than 15, which is not abnormal. Therefore, image block 001 has no abnormality. If the boundary direction difference of image block 002 is 4 degrees greater than 3 degrees, the boundary direction is marked as abnormal. Filter and record the image block numbers that exceed the feature offset threshold, collect all the image block numbers with abnormalities, and obtain a feature offset abnormality list.

[0037] The defect record generation submodule extracts the position index corresponding to the abnormal image block based on the feature offset anomaly list, retrieves the circuit number and defect type classification code associated with the image block, and forms a triplet format of the position index, circuit number, and defect type to establish image defect location information; The defect record generation submodule extracts the position index corresponding to each abnormal image block one by one based on the feature offset anomaly list. The position index consists of the image block number and its relative position information in the overall circuit diagram. For example, the position index of image block 002 is (x=100, y=200). The circuit number associated with the image block is retrieved. The circuit number is determined by the circuit structure field in the task data. For example, the circuit number associated with image block 002 is CIR12345. The defect type classification code is retrieved. The defect type classification code is specified based on the feature dimension of the abnormal feature value. For example, the boundary direction anomaly corresponds to the classification code 01, the boundary continuous change anomaly corresponds to the classification code 02, and the signal jump intensity anomaly corresponds to the classification code 03. If the boundary direction of image block 002 is abnormal, the classification code is 01. The position index, circuit number, and defect type classification code are combined to form a triple format, for example, ((100, 200), CIR12345, 01). The same operation is performed for each abnormal image block to finally establish the image defect location information.

[0038] See also Figure 2 and Figure 7 ,The archiving dual standby module includes the archiving node detection submodule, the node hot standby judgment submodule, and the dual standby synchronous writing submodule; The archiving node detection submodule calls the image defect location information and calls the processing load rate and cache occupancy ratio data of the current archiving node according to the defect type, circuit number and task number of the defective image block. It normalizes the archiving node processing load rate and cache occupancy ratio and calculates the average value. It then classifies and filters the nodes according to their changing trends, and generates an archiving node load status table. The archiving node detection submodule calls the image defect positioning information. First, it reads the corresponding records one by one according to the defect type, circuit number and task number of the defective image block, and calls the processing load rate data and cache occupancy ratio data of the current archiving node. The processing load rate is recorded as the percentage of the number of tasks processed by the node per minute to the maximum processing capacity of the node. For example, the processing load rate of node A is 65%, node B is 50%, and node C is 75%. The cache occupancy ratio is recorded as the percentage of the node's used cache capacity to the maximum cache capacity. For example, the cache occupancy ratio of node A is 70%, node B is 55%, and node C is 80%. The processing load rate and cache occupancy ratio are normalized using the maximum and minimum normalization formula. ,in is the original value, and are the minimum and maximum values ​​in the data set respectively. The normalized result of node A’s processing load rate is: , the normalized result of node A’s cache occupancy ratio is After normalization, the normalized processing load rate and normalized cache occupancy ratio of each node are averaged. The average value of node A is , node B is , node C is ,Classification and screening are performed based on the changing trends of node processing load rate and cache occupancy ratio, and the trend change range is set to 0-0.4 as low load, 0.4-0.7 as medium load, and 0.7-1 as high load. Node A is classified as medium load, node B is classified as low load, and node C is classified as high load. Finally, an archived node load status table is generated to record the node number and the corresponding load classification results.

[0039] The node hot standby judgment submodule extracts the average load value and cache occupancy rate of the archive node based on the archive node load status table, performs weighted accumulation operation on the average load value and cache occupancy rate, compares the weighted accumulation value with the node hot standby standard value, and screens the node list with a value lower than the node hot standby standard value to obtain the archive node hot standby screening table; The node hot standby judgment submodule extracts the average load value and cache occupancy data of the archive nodes one by one based on the archive node load status table, sets the average load value weight to 0.6 and the cache occupancy weight to 0.4, and performs weighted accumulation operation on the average load value and cache occupancy. The calculation method is: ,in is the average node load rate, is the node cache occupancy ratio, and the weighted cumulative value of node A is , the weighted cumulative value of node B is , the weighted cumulative value of node C is , set the node hot standby standard value to 60, and compare the weighted cumulative value with the node hot standby standard value. The weighted value of node A is 67, which is greater than 60 and does not meet the hot standby condition. The weighted value of node B is 52, which is less than 60 and meets the hot standby condition. The weighted value of node C is 77, which is greater than 60 and does not meet the hot standby condition. Filter the nodes with weighted cumulative values ​​less than the node hot standby standard value, and filter out node B as a node that meets the hot standby condition. Finally, obtain the archived node hot standby screening table and record the node numbers that meet the hot standby condition.

[0040] The dual-standby synchronous writing submodule extracts node information that meets the hot standby requirements based on the archive node hot standby screening table, writes the corresponding defective image blocks and associated metadata to the primary archive node and the backup archive node simultaneously, generates a checksum for the written data blocks, records the synchronous writing confirmation information, and establishes distributed archive writing record information; The dual-standby synchronous writing submodule extracts node information that meets the hot standby requirements based on the archive node hot standby screening table, selects the archive node master node and the standby node corresponding to the task number and node number respectively. For example, the defective image block 001 selects the master node A and the standby node B. The corresponding defective image block data and associated metadata are organized in data block format. The data block contains the image block number, location index, circuit number and defect type classification code, and is sent to the storage interface of the master archive node and the standby archive node respectively. The data block integrity is checked during writing. An independent check code is generated for each data block by generating an MD5 or SHA256 check code. For example, the MD5 check code generated for image block 001 is d41d8cd98f00b204e9800998ecf8427e. The synchronous write confirmation information of the master node and the standby node is recorded, including the write completion time, data block number and corresponding check code. The distributed archive write record information is established to ensure that the writing process of each defective data is completed synchronously between the two nodes and a traceable record is formed.

[0041] The above are merely preferred embodiments of the present invention and do not limit the present invention in any other form. Any technician familiar with the profession may use the technical content disclosed above to change or modify it into an equivalent embodiment with equivalent changes and apply it to other fields. However, any simple modification, equivalent change and modification made to the above embodiment based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A distributed processing system for circuit board cloud detection data, characterized in that: The system comprises: The task monitoring module obtains the inspection task data uploaded by multiple circuit board inspection devices, calls the upload status information of each node in the node monitoring list, evaluates the adaptability of the node upload based on the task content type and node distribution success rate, and generates a list of adapted distribution nodes; The computing power scheduling module calls the processor occupancy rate, number of active threads, and number of available memory blocks of each node based on the adapted distribution node list, calculates the node computing power level, and obtains the node computing power quantification result; The task allocation module calls the node computing power quantification result, calculates and evaluates the task adaptability based on the number of image blocks and the node computing power level, combined with the task processing step size and the memory idle ratio, combines and binds the task number and the node number, and generates task binding scheduling information; The defect comparison module performs a difference operation between the image feature vector and the reference image vector of the same structure in the standard image library based on the task binding scheduling information, records the corresponding position index, circuit number and defect type for the image block exceeding the threshold, and generates image defect location information.

2. The circuit board cloud detection data distributed processing system according to claim 1 is characterized in that: The adaptation distribution node list specifically includes the node upload bandwidth value, the node distribution historical success rate, and the node data synchronization delay. The node computing power matching list includes the instantaneous available CPU value, the thread processing capability value, and the memory idle ratio. The task binding scheduling information specifically refers to the image block quantity binding information, the circuit structure partition data information, and the processing step matching index. The image defect positioning information specifically includes the boundary direction feature offset, the continuous change abnormal point, and the signal jump intensity abnormality record.

3. The circuit board cloud detection data distributed processing system according to claim 2, characterized in that: The task monitoring module includes: The task data extraction submodule obtains the inspection task data uploaded by multiple circuit board inspection devices, extracts the data volume parameters, circuit image quantity parameters, and inspection content type parameters from the inspection task data, performs total statistics on the extracted data volume parameters, sums the number of image blocks for the circuit image quantity parameters, and performs category classification and counting on the inspection content type parameters to generate an inspection task data information table; Based on the detection task data information table, the node status monitoring submodule calls the upload status information of each node in the node monitoring list, obtains the node's current upload bandwidth rate, the number of successful upload tasks per unit time, the number of upload task interruptions, and the task processing request queue length, and calculates the node upload performance index after data standardization. The upload performance index corresponding to each node is screened, and the node number and corresponding bandwidth value whose performance index ranking is within the target range are obtained to generate the node bandwidth capacity screening result; The node adaptation evaluation submodule uses the node bandwidth capacity screening results, calls the historical distribution task success rate of the screened nodes, performs a weighted combined score on the success rate and the number of task data content types, and judges whether the score value is greater than the task content adaptation threshold to obtain a set of nodes that meet the adaptability standards and generate a list of adapted distribution nodes.

4. The circuit board cloud detection data distributed processing system according to claim 3 is characterized in that: The formula for computing the node upload performance index is: ； in, Representative Node upload performance index, Representative The upload bandwidth rate of each node, Representative The number of upload task interruptions per node per unit time, Representative The length of the task processing request queue of each node, Representative The number of tasks that are successfully uploaded by a node per unit time.

5. The circuit board cloud detection data distributed processing system according to claim 4 is characterized in that: The computing power scheduling module includes: The processor resource extraction submodule obtains the processor occupancy rate of the node based on the adaptation distribution node list, extracts the processor occupancy rate parameter and the node identification number, calculates the processor idle rate, calls the processor idle rate and the node identification number, filters the nodes whose processor idle rate is greater than the set idle threshold, and generates a node processor idle rate list; The thread and memory extraction submodule calls the number of active threads and available memory blocks of the filtered node based on the node processor idle rate list, obtains the thread activity amount and the memory available ratio respectively, performs a proportional operation on the thread activity amount and the memory available ratio, sets the ratio threshold as the thread memory load balancing threshold, determines whether the ratio value falls within the load balancing interval, identifies the node information that meets the load balancing conditions, and generates a node load balancing data table; The node computing power synthesis submodule extracts the processor idle rate, thread activity, and memory available ratio corresponding to each node based on the node load balancing data table, assigns weight parameters to each of them, and performs weighted summation. The weighted result is compared with the node computing power level division benchmark value to obtain the corresponding level range of the node computing power and establish the node computing power quantification result.

6. The circuit board cloud detection data distributed processing system according to claim 5, characterized in that: The task allocation module includes: The task information extraction submodule calls the node computing power quantization result, obtains the data content of each detection task, extracts the number of image blocks, the number of circuit structure partitions and the processing step length, records the total value of the number of image blocks, marks the number of circuit structure partitions by level, classifies the processing step length by interval, and generates a task processing structure data table; The task adaptation evaluation submodule calls the node computing power level data according to the task processing structure data table, extracts the node memory idle ratio, combines the number of image blocks, the number of circuit structure partitions and the processing step length to construct the task load, performs quantitative matching based on the adaptation between the task load and the node computing power level and the memory idle ratio, calculates the task adaptation value, combines the task nodes whose task adaptation values ​​are greater than the matching reference value, and generates a task node adaptation matching table; The number binding generation submodule calls the correspondence between task numbers and node numbers based on the task node adaptation matching table, sorts the task numbers in partition order, prioritizes the node numbers according to the adaptation value, combines and binds the sorted number sets, and establishes task binding scheduling information.

7. The circuit board cloud detection data distributed processing system according to claim 6, characterized in that: The formula for calculating the task adaptation value is: ； in, Indicates the Task and The adaptation value of each node, Indicates the The computing power level of each node, Indicates the The memory idle ratio of each node, Indicates the The number of image patches for each task, Indicates the The number of circuit structure partitions for each task, Indicates the The processing step of each task, Indicates the average task processing step length.

8. The circuit board cloud detection data distributed processing system according to claim 7, characterized in that: The defect comparison module includes: The feature value extraction submodule extracts the boundary direction feature value, boundary continuous change value, and signal jump strength value of each image block based on the task binding scheduling information and the circuit image block data assigned to the corresponding node, classifies and organizes the three extracted feature values ​​according to the image block sequence, classifies and stores them according to the image block number, and generates an image block feature classification table; The feature vector comparison submodule calls the reference image feature vector corresponding to each image block structure in the standard image library based on the image block feature classification table, calculates the difference between each feature value and the corresponding reference feature value, sets a feature offset threshold for the difference value, and filters and records the image block numbers that exceed the feature offset threshold to obtain a feature offset anomaly list; The defect record generation submodule extracts the position index corresponding to the abnormal image block based on the feature offset anomaly list, retrieves the circuit number and defect type classification code associated with the image block, forms a triplet format of the position index, circuit number and defect type, and establishes image defect location information.

9. The circuit board cloud detection data distributed processing system according to claim 8, characterized in that: The system further comprises: The archiving dual-standby module calls the image defect location information, obtains the defect type, circuit number and task number of the defective image block, detects the processing load rate and cache occupancy ratio of the current archiving node, calculates the node hot standby value, determines whether the node hot standby value is lower than the hot standby standard value, and when it is lower, synchronously writes the image block and metadata to the primary archiving node and the standby node, records the write confirmation information and the data block check code, and generates distributed archiving write record information; The distributed archiving write record information includes a defective image block index, a data block check code and archiving node confirmation information.

10. The circuit board cloud detection data distributed processing system according to claim 9, characterized in that: The archiving dual-backup module includes: The archiving node detection submodule calls the image defect location information, and calls the processing load rate and cache occupancy ratio data of the current archiving node according to the defect type, circuit number and task number of the defective image block, normalizes the archiving node processing load rate and cache occupancy ratio and calculates the average value, classifies and filters the nodes according to the changing trends of the processing load rate and cache occupancy ratio, and generates an archiving node load status table; The node hot standby judgment submodule extracts the average load value and cache occupancy rate of the archive node based on the archive node load status table, performs weighted accumulation operation on the average load value and the cache occupancy rate, compares the weighted accumulation value with the node hot standby standard value, and screens a list of nodes with values ​​lower than the node hot standby standard value to obtain an archive node hot standby screening table; The dual-standby synchronous writing submodule extracts the node information that meets the hot standby requirements according to the archive node hot standby screening table, writes the corresponding defective image blocks and associated metadata into the primary archive node and the backup archive node at the same time, generates a check code for the written data block, records the synchronous write confirmation information, and establishes distributed archive write record information.

Images