A PCB component image target detection method based on graph network

Abstract

The present invention discloses a PCB component image target detection method based on a graph network, comprising the steps of: dividing four-fifths of the samples in the PCB image sample data set into a training set, and the remaining samples into a verification set, using a generative adversarial network to learn and generate new samples for all samples in the training set; constructing a K nearest neighbor matrix for each new sample using superpixels; establishing an RPN network, extracting the border positions of multiple PCB superpixel training set sample images and continuously correcting the probability of being close to the correct annotation data frame through a loss function; using a graph convolutional network to refine the feature map to generate feature map information; using a similarity prediction algorithm to classify each feature map of the previous stage, and then calculating the similarity probability of each map to the component template and classifying it into the category of the corresponding component template based on the probability. The present invention realizes fast and effective classification and annotation of various components on the PCB board, greatly saving labor costs.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a PCB component image target detection method based on a graph network. Background Art

[0002] The detection of PCB components is a necessary step in the automated detection of hardware. Traditionally, quality inspection engineers conduct visual inspections to manually frame components (such as resistors and capacitors) and then identify component types. A PCB generally contains hundreds of components. Quality inspection engineers need to understand various types of electronic components to accurately mark them. Quality inspection engineers need to spend about two days marking a PCB, which takes a lot of time and energy, has a high threshold, and is expensive to mark. Therefore, it is necessary to provide a method for quickly marking and identifying components. By using the conditional generative adversarial network in deep learning to eliminate image ambiguity, and using the ideas of superpixels and graph neural networks to build a target detection model framework, it can quickly and effectively classify and mark each component on the PCB, saving a lot of manpower costs. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a PCB component image target detection method based on a graph network.

[0004] The technical solution of the present invention is as follows:

[0005] A PCB component image target detection method based on a graph network comprises the following steps:

[0006] S1, obtain a PCB image sample data set, and divide four fifths of the samples in the PCB image sample data set into a training set, and the other fifth of the samples into a validation set; use a GANs generative adversarial network to learn all samples in the training set to generate new samples;

[0007] S2, for each new sample generated in step S1, a K nearest neighbor matrix is ​​constructed using superpixels, with weights as:

[0008]

[0009] Where xi and xj are the two-dimensional coordinates of superpixels i and j respectively, and σx is the scaling parameter;

[0010] S3, establish an RPN network, extract the border positions of multiple PCB superpixel training set sample images, and use functions to continuously translate and scale to correct the probability of being close to the correct labeled data frame, thereby providing a feature map for the next step;

[0011] S4, using a graph convolutional network and combining it with the component template graph to refine the feature graph provided in step S3 to generate feature graph information;

[0012] S5, using a similarity prediction algorithm to classify each feature graph generated in step S3 according to the feature graph information generated in step S4; then using the similarity prediction algorithm to calculate the similarity probability of each graph processed by steps S3 and S4 to the component template, and classify it into the category of the corresponding component template based on the probability.

[0013] Furthermore, the specific implementation process of the feature refinement in step S4 is as follows:

[0014] Let the entities of the graph in the graph convolutional network be represented as nodes, the connections between nodes be represented as edges, the system-level features be represented by global attributes, nodes are represented as vi, edges are represented as ek, global attributes are represented as u, and sk and rk are used to represent the indexes of the sending node and receiving node of edge k respectively;

[0015] The graph is represented by a 3-tuple G = (u; V; E), where u represents a global attribute, which is a feature of the entire graph; V represents the set of nodes in the graph, which is represented as:

[0016]

[0017] E represents the set of edges, which is expressed as:

[0018]

[0019] Where Ne and Nv represent the number of edges and nodes respectively, ek is the feature of the kth edge, sk and rk are the numbers of the receiving node and the sending node respectively;

[0020] a. First, use the current edge information, node information, and global attribute u to generate new edge information;

[0021] b. Generate new node information based on new edge information;

[0022] c. Based on the new node and edge information, a new global attribute u is generated, and the whole process is completed.

[0023] Furthermore, the GANs generative adversarial network includes a generative model and a discriminative model, wherein the generative model is used to learn the real image distribution of samples in the training set so that the new samples generated by the GANs generative adversarial network are more realistic, and the discriminative model is used to determine whether the image is a real image or a generated image;

[0024] The GANs generative adversarial network adopts a deep convolutional neural network.

[0025] Furthermore, the step S3 is to continuously translate, scale and correct the data frame to be close to the correct annotation data frame through the loss function.

[0026] By adopting the above scheme, the present invention has the following beneficial effects:

[0027] A method for detecting PCB component images based on graph networks is provided. The GANs generative adversarial network is used to eliminate blurred images and improve sample quality. Superpixel image segmentation is used to improve target detection performance and labeling efficiency. The correlation between graphs captured and analyzed based on the graph convolutional network is used to improve the target recognition rate and labeling accuracy, so that each component on the PCB can be quickly and effectively classified and labeled, greatly saving labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0029] Figure 1 A flowchart of the steps of the PCB component image target detection method based on the graph network provided by the present invention;

[0030] Figure 2(a) shows the definition of the graph of a graph convolutional network.

[0031] Figure 2(b) is a schematic diagram of the GN block (graph network block) calculation process.

[0032] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0033] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0034] Reference Figure 1 As shown, the present invention provides a PCB component image target detection method based on a graph network, comprising the following steps:

[0035] S1, obtain a PCB image sample data set (the PCB image sample data set is derived from an open source data set on the Internet, and is obtained after a large number of PCB board components are manually annotated for training), and four-fifths of the samples in the PCB image sample data set are divided into a training set, and the other one-fifth of the samples are divided into a verification set; use the GANs generative adversarial network to learn and generate new samples for all samples in the training set, so as to eliminate image ambiguity; specifically, the GANs generative adversarial network includes a generative model and a discriminant model, the generative model is used to learn the real image distribution of samples in the training set, so that the new samples generated by the GANs generative adversarial network are more real, and the discriminant model is used to determine whether the image is a real image or a generated image. As the GANs generative adversarial network is continuously iterated in training, the generative model and the discriminant model are continuously adversarially learned, and finally a Nash equilibrium is reached; more specifically, the GANs generative adversarial network uses a deep convolutional neural network to adapt to the processing of image environment data of different PCB samples;

[0036] S2, for each new sample generated in step S1, a K nearest neighbor matrix is ​​constructed using superpixels (assuming a predefined parameter K, where K is the number of pre-generated superpixel blocks, i.e., it is expected that an image of size M*N (the number of pixels is M*N) will be divided into K superpixel blocks, and each superpixel block size contains [(M*N) / K] pixels), and the weights are:

[0037]

[0038] Where x i and x j are the two-dimensional coordinates of superpixels i and j, σ x is the scaling parameter (by calculating the average distance x of the k nearest neighbors of each node (i.e., the block segmentation graph after the new sample is segmented) k get);

[0039] S3, establish the RPN network (Region Proposal Network), extract the bounding box (box, actually the predicted box, including four coordinates P (x, y, w, h)) position of multiple PCB superpixel training set sample images (confirm the target box position by confirming the coordinates of the center point and the length and width), and continuously translate and scale the loss function to correct the probability of approaching GroundTruth (correctly labeled data frame, coordinates are ground truth G (x, y, w, h)) (set value> 0.6), so as to determine the positive sample (the target box including components), thereby providing a feature map for the next step;

[0040] S4, using a graph convolutional network (GCN) and combining the component template graph to refine the feature map provided in step S3, and generate feature map information; further, the specific implementation process of feature refinement in step S4 is as follows:

[0041] Since the main computing unit of the graph convolutional network is the GN block, and the GN block is a graph-to-graph module whose input and output are both graphs, in the graph of the graph convolutional network, entities (entities, since the graph network is composed of interconnected graph network blocks (GN blocks), entities are also called "nodes" in the neural network implementation, and the connections between nodes are called "edges", which represent the dependencies between nodes) are represented as nodes, relations (connections between nodes) are represented as edges, and system-level features are represented by global attributes; refer to Figure 2(a):

[0042] The node is represented by vi, the edge is represented by ek, the global attribute is represented by u, and sk and rk are used to represent the index of the sender node and the receiver node of edge k (i.e., the kth edge) respectively;

[0043] A graph is represented by a 3-tuple G = (u; V; E), where u represents a global information (i.e., a global attribute), which is a feature of the entire graph; V represents the set of nodes in the graph, which is represented as:

[0044]

[0045] E represents the set of edges, which is expressed as:

[0046]

[0047] Where N e and N v Represent the number of edges and nodes respectively, ek is the attribute of the k-th edge, sk and rk are the numbers of the receiver node and sender node respectively;

[0048] The calculation process is shown in Figure 2(b). In Figure 2(b), 1 is to calculate and update three variables: node, edge, and u; 2 is to loop and calculate the update function to iteratively update each edge; 3 is to update the features of each node based on the previous loop; 4 is to generate V′ and E′ and aggregate them to generate corresponding information. and 5 is to update u, 6 is to calculate and finally update the (edge, node, u) feature. The whole process can be summarized as follows:

[0049] a. First, use the current edge information, node information, and global information (global attributes) u to generate new edge information;

[0050] b. Generate new node information based on the new edge information;

[0051] c. Generate new global information (global attribute) u based on the new node and edge information, and the entire update process is completed;

[0052] S5, using the similarity prediction algorithm (SVM) to classify each feature map generated in the previous stage (i.e., step S3) for the feature map information generated in step S4; then using the similarity prediction algorithm (SVM) to calculate the similarity probability of each Proposal (the block diagram after RPN framing + GCN feature extraction, i.e., the diagram processed by steps S3 and S4) to the component template, and classify it into the category of the corresponding component template based on the probability, so that the various components on the PCB board can be quickly and effectively classified and labeled, saving a lot of manpower costs.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] A method for detecting PCB component images based on graph networks is provided. The GANs generative adversarial network is used to eliminate blurred images and improve sample quality. Superpixel image segmentation is used to improve target detection performance and labeling efficiency. The correlation between graphs captured and analyzed based on the graph convolutional network is used to improve the target recognition rate and labeling accuracy, so that each component on the PCB can be quickly and effectively classified and labeled, greatly saving labor costs.

[0055] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A PCB component image target detection method based on graph network, characterized in that: The steps include: S1, obtain a PCB image sample data set, and divide four fifths of the samples in the PCB image sample data set into a training set, and the other fifth of the samples into a validation set; use a GANs generative adversarial network to learn all samples in the training set to generate new samples; S2, for each new sample generated in step S1, a K nearest neighbor matrix is ​​constructed using superpixels, with weights as: ； Where xi and xj are the two-dimensional coordinates of superpixels i and j respectively, and σx is the scaling parameter; S3, establish an RPN network, extract the border positions of multiple PCB superpixel training set sample images, and use functions to continuously translate and scale to correct the probability of being close to the correct labeled data frame, thereby providing a feature map for the next step; S4, using a graph convolutional network and combining it with the component template graph to refine the feature graph provided in step S3 to generate feature graph information; S5, using a similarity prediction algorithm to classify each feature graph generated in step S3 according to the feature graph information generated in step S4; then using the similarity prediction algorithm to calculate the similarity probability of each graph processed by step S3 and step S4 to the component template, and classify it into the category of the corresponding component template based on the probability; The specific implementation process of the feature refinement in step S4 is as follows: Let the entities of the graph in the graph convolutional network be represented as nodes, the connections between nodes be represented as edges, the system-level features be represented by global attributes, nodes are represented as vi, edges are represented as ek, global attributes are represented as u, and sk and rk are used to represent the indexes of the sending node and receiving node of edge k respectively; The graph is represented by a 3-tuple G = (u; V; E), where u represents a global attribute, which is a feature of the entire graph; V represents the set of nodes in the graph, which is represented as: ； E represents the set of edges, which is expressed as: ； Where Ne and Nv represent the number of edges and nodes respectively, ek is the feature of the kth edge, sk and rk are the numbers of the receiving node and the sending node respectively; a. First, use the current edge information, node information, and global attribute u to generate new edge information; b. Generate new node information based on new edge information; c. Based on the new node and edge information, a new global attribute u is generated, and the whole process is completed.

2. The PCB component image target detection method based on graph network according to claim 1 is characterized in that: The GANs generative adversarial network includes a generative model and a discriminative model. The generative model is used to learn the real image distribution of samples in the training set to make the new samples generated by the GANs generative adversarial network more realistic. The discriminative model is used to determine whether an image is a real image or a generated image.

3. The PCB component image target detection method based on graph network according to claim 1 is characterized in that: The GANs generative adversarial network adopts a deep convolutional neural network.

4. The PCB component image target detection method based on graph network according to claim 1 is characterized in that: The step S3 is to continuously translate, scale and correct the data frame close to the correct annotation data frame through the loss function.

Images