Component identification methods, media and equipment for PCBA boards

By building and training neural network models, PCBA board image segmentation and target box redundancy updates are solved, the problem of small electronic components recognition is improved, the recognition accuracy is reduced, and the calculation resource consumption is solved, and the identification error in the existing technology is solved. Low accuracy problem.

CN118865001BActive Publication Date: 2024-12-06HUNAN AEROSPACE TIANLU NEW MATERIAL TESTING CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411348361.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-26
Publication Date
2024-12-06
Estimated Expiration
2044-09-26

AI Technical Summary

Technical Problem

The prior art is difficult to effectively identify small electronic components on PCBA boards, resulting in low recognition errors and accuracy, high computing resources consumption, and only part of the components may be obtained when cutting images.

Method used

The neural network model is built and trained to take the PCBA board image data as input, segment the image and map the target box, and update the target box with redundancy in overlapping areas to ensure the accuracy of identification and reduce the consumption of computing resources.

Benefits of technology

It improves the accuracy and accuracy of PCBA board component recognition, reduces computing resource consumption, and avoids identification errors and redundant labeling problems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118865001B_ABST
    Figure CN118865001B_ABST
Patent Text Reader

Abstract

The present invention relates to a component identification method, medium and device for PCBA boards. By constructing and training a neural network model with image data of the PCBA board as input and target categories, target frame data and confidence of each component on the PCBA board as output, the original image of the current PCBA board to be identified is obtained, and it is segmented into a plurality of segmented images, and input into the neural network model, the target category, target frame data and confidence of each component on each segmented image are obtained, and then each target frame is mapped to the corresponding position of the original image, and the overlapping target frames are redundantly updated to obtain an updated target frame, and the target category, target frame data and confidence of the updated target frame are annotated on the corresponding target frame of the original image with a target label. The problem of high computing resource consumption, large data processing volume, large recognition error and low recognition accuracy in the prior art is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a component recognition method, medium and device for a PCBA board. Background Art

[0002] With the rapid development of the electronics manufacturing industry, as a basic component of the modern information industry, printed circuit boards are widely used in multiple manufacturing fields, such as computers, communications, electronic equipment, and aerospace systems. With the development of integrated circuits, electronic systems are constantly evolving towards miniaturization, micro-miniaturization, and high integration, and the size of electronic components on printed circuit boards is getting smaller and smaller. With the rapid development of camera technology, the resolution of images is getting higher and higher, and the processing tasks for raw images are also increasing. In actual engineering applications, in order to see the overall picture of a relatively large object, the camera's field of view is required to be relatively large, resulting in a larger resolution of the entire image. Therefore, the proportion of packaged components such as chip resistors, color ring resistors, and chip capacitors in the entire image is very small. The existing target detection algorithm cannot directly detect such small targets, and when using the original image to train the detection model, a large amount of computing resources is required. Compressing the image can reduce the amount of data processing to reduce the consumption of computing resources, but it will affect the detection effect.

[0003] In order to solve the above problems, the original image of the PCBA board can be cut, and the obtained cut image can be used to train the detection model. Then, the cut image of the original image to be detected is input into the trained neural network model to identify the electronic components contained in each cut image. However, since the electronic components are unevenly distributed on the PCBA board, the obtained cut image may not contain complete electronic components. In addition, because some structures of many electronic components are very similar, such as a diode and a light-emitting diode differing only by an arrow, and some only differ in chip pins, etc.; therefore, only part of the structure of the electronic component is obtained when the image is cut, which may lead to recognition errors and reduced accuracy.

[0004] Therefore, how to improve it is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0005] Based on this, the purpose of this application is to provide a component identification method, medium and device for PCBA board to solve at least one technical problem mentioned in the above background technology.

[0006] In a first aspect, the present application provides a component identification method for a PCBA board, comprising:

[0007] Build and train a neural network model that takes the image data of the PCBA board as input and outputs the target category, target box data, and confidence of each component on the PCBA board;

[0008] Obtain the original image of the PCBA board to be identified and segment it into several segmented images; there are overlapping areas between adjacent segmented images;

[0009] Input each segmented image into the neural network model to obtain the target category, target frame data and confidence of each component on each segmented image;

[0010] According to the target frame data of each component on each segmented image, each target frame is mapped to the corresponding position of the original image;

[0011] According to the target category, target frame data and confidence of each component, redundantly update the overlapping target frames to obtain updated target frames;

[0012] The target category, target frame data and confidence of the updated target frame are annotated on the corresponding target frame of the original image with the target label.

[0013] Furthermore, the specific structure of the neural network model is:

[0014] The input layer receives the input image data and then passes the image data to the convolutional layer;

[0015] The convolution layer is connected to the input layer and is used to extract local features from the input image data to obtain the initial feature map;

[0016] The batch normalization layer is connected to the convolutional layer and is used to normalize the output of each layer to improve training stability and accelerate convergence, reduce internal covariate shift, and obtain the normalized updated feature map;

[0017] The pooling layer is connected to the batch normalization layer to reduce the spatial dimension of the updated feature map and retain the main features. The updated feature map is then pooled to reduce the amount of calculation and prevent overfitting to obtain the optimized feature map.

[0018] The feature fusion layer is connected to the pooling layer and is used to fuse feature maps from different levels to capture information of objects of different sizes and obtain the final feature map;

[0019] The detection head is connected to the feature fusion layer and is used to further process the final feature map to generate detection results, including the target box position, component category and confidence level;

[0020] The output layer is connected to the detection head and is used to integrate the target box, category and confidence generated by the detection head into the final detection result, which is the target label corresponding to each target.

[0021] Furthermore, the convolutional layer includes:

[0022] A first convolution unit, used to extract a texture feature map of the image data;

[0023] A second convolution unit, used to extract a color feature map of the image data;

[0024] A third convolution unit, used to extract a shape feature map of the image data;

[0025] The fusion unit is used to fuse the texture feature map, color feature map and shape feature map obtained by each convolution unit to obtain an initial feature map with more distinguishability and relevance.

[0026] Further, the steps of building and training the neural network model include:

[0027] Build an initial neural network model;

[0028] Collect several original images, mark each component in the original image, and obtain a marked image;

[0029] Set the size, moving step, moving direction and starting position of the cut blocks in each original image;

[0030] The cutting block is moved in sequence according to the starting position, moving step length and moving direction, and the pixel points within the coverage range of the cutting block are extracted according to the size of the cutting block to obtain a segmented image; there are overlapping areas between adjacent segmented images;

[0031] Determine whether each segmented image contains components. If not, discard the segmented image. If so, obtain the target category and target frame data corresponding to each component based on the annotations of the components in each segmented image, and use the components and their target categories and target frame data on each segmented image as the original image subset;

[0032] Each segmented image in the original image subset is rotated several times by a predetermined angle to obtain new subsets with different angles;

[0033] Construct a training data set based on the original image subset and the new subset;

[0034] The training data set is input into the initial neural network model to optimize the parameters of the initial neural network model and obtain the final neural network model.

[0035] Furthermore, the step of obtaining the original image of the PCBA board to be identified and segmenting it into a plurality of segmented images also includes:

[0036] Get the coordinates of the segmented image feature points in the original image; the segmented image feature points may include the image center point and image corner points.

[0037] Further, the step of redundantly updating the overlapping target frames according to the target categories, target frame data and confidence of each component to obtain an updated target frame includes: target frame data, including target frame position and target frame size;

[0038] According to the target frame data, the overlap index between two overlapping target frames is calculated, and the target frames whose overlap index is greater than the overlap threshold are set as the same overlapping target frame group;

[0039] Determine whether the target categories corresponding to each target frame in each overlapping target frame group are the same. If so, retain the target frame with the highest confidence and remove the other target frames in the group; if not, retain the target frame with the largest target frame size and remove the other target frames in the group.

[0040] Furthermore, the overlap index between the two overlapping target frames is calculated, specifically, the ratio of the overlap area of ​​the two overlapping target frames to the sum of their total areas is calculated.

[0041] Further, the step of performing redundant updating on the overlapping target frames according to the target categories, target frame data and confidence of each component to obtain an updated target frame also includes:

[0042] According to the confidence and size of each target frame, its target feature index is calculated, the target frame with a larger target feature index is retained, and the other target frames in the group are removed.

[0043] In a second aspect, the present application further provides a computer storage medium storing an executable program code; the executable program code is used to execute the component identification method for a PCBA board as described in any one of the first aspects.

[0044] In a third aspect, the present application also provides a terminal device, including a memory and a processor; the memory stores a program code that can be executed by the processor; the program code is used to execute the component identification method for the PCBA board described in any one of the first aspects.

[0045] The present invention provides a component identification method, medium and device for PCBA boards. First, a special neural network model is constructed and trained, which takes the image data of the PCBA board as input and the target category, target frame data and confidence of each component on the PCBA board as output; then the original image of the current PCBA board to be identified is obtained and segmented into a plurality of segmented images. The key is to avoid the failure to obtain a complete image of the electronic component during cutting, and to ensure that there are overlapping areas between adjacent segmented images during segmentation, so as to improve the recognition accuracy of the subsequent neural network model. At the same time, the resolution of the segmented image is low, which can reduce the consumption of computing resources; on this basis, each segmented image is segmented into a plurality of segmented images. The image is input into the neural network model to obtain the target category, target frame data and confidence of each component on each segmented image; then, according to the target frame data of each component on each segmented image, each target frame is mapped to the corresponding position of the original image, and then according to the target category, target frame data and confidence of each component, the overlapping target frame is redundantly updated to obtain the updated target frame. The key is to avoid only obtaining part of the structure of the electronic component when cutting the image, resulting in recognition errors, reduced accuracy and redundant labeling; finally, the target category, target frame data and confidence of the updated target frame are annotated on the corresponding target frame of the original image with a target label. The key of the present invention is that the early segmentation includes the overlapping part, and the later recognition includes the redundant removal part, which solves the problems of high computing resource consumption, large data processing volume, large recognition error and low recognition accuracy in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 The present invention is a flowchart of an embodiment of a component identification method for a PCBA board. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] It should be noted that if the embodiments of the present invention involve directional indications, such as up, down, left, right, front, back, etc., then the directional indication is only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly. In addition, if the embodiments of the present invention involve descriptions of "first, second", "S1, S2", "step one, step two", etc., then such descriptions are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of indicated technical features or indicating the execution order of the method, etc. Those skilled in the art can understand that anything that does not violate the gist of the invention under the technical concept of the invention should be included in the protection scope of the present invention.

[0049] like Figure 1 As shown, the present invention provides a component identification method for a PCBA board:

[0050] S1: Build and train a neural network model that takes the image data of the PCBA board as input and outputs the target category, target box data and confidence of each component on the PCBA board;

[0051] Specifically, the neural network model can adopt any image recognition model in the prior art to construct an initial neural network model; then, by collecting several original images of the PCBA board, marking the targets in the original images, and segmenting them into several segmented images, a training data set is constructed; then, according to the training data set, the initial neural network model is input to train various parameters of the initial neural network model to obtain a trained neural network model.

[0052] Preferably, the target category may include various subdivided electronic components such as chip resistors, color ring resistors, chip capacitors, etc.; the target frame data may include the target frame shape, target frame size and target frame feature point coordinates, etc. The target frame shape may be selected as the minimum circumscribed rectangle, the minimum circumscribed circle, etc.; the target frame size may be selected as the length, width, area, number of pixels, etc.; the target frame feature point coordinates may be selected as the coordinates of a corner point, the coordinates of the center point, etc.; it is worth noting that this step is a preparatory work, which can be constructed and trained in advance, or it can be performed simultaneously with step S2, as long as it is constructed and trained before the neural network model is needed for prediction in step S3.

[0053] More preferably, the specific structure of the neural network model may include but is not limited to:

[0054] Input Layer: It is used to receive input image data and then pass the image data to the convolutional layer;

[0055] Convolutional Layers: connected to the input layer, used to extract local features from the input image data to obtain the initial feature map;

[0056] Batch Normalization: connected to the convolutional layer, it is used to normalize the output of each layer to improve training stability and accelerate convergence, reduce internal covariate shift, and obtain the normalized updated feature map;

[0057] Pooling Layers: Connected with the batch normalization layer, it is used to reduce the spatial dimensions (width and height) of the updated feature map, retain the main features, and then perform a pooling operation on the updated feature map to reduce the amount of calculation and prevent overfitting, and obtain an optimized feature map;

[0058] Feature Fusion Layers: Connected to the pooling layer, it is used to fuse feature maps from different levels to capture information of objects of different sizes and obtain the final feature map;

[0059] Detection Heads: connected to the feature fusion layer, used to further process the final feature map to generate detection results, including target box location, component category and confidence;

[0060] Output Layer: connected to the detection head, used to integrate the target box, category and confidence generated by the detection head into the final detection result, which is the target label corresponding to each target.

[0061] More preferably, the convolutional layer may optionally include but is not limited to:

[0062] The first convolution unit is used to extract the texture feature map of the image data; specifically, the texture feature map usually shows some irregular texture patterns or edge information, and these texture patterns may form complex patterns in the feature map, corresponding to the patterns, edge lines, etc. on the surface of the components in the image data;

[0063] a second convolution unit, for extracting a color feature map of the image data; specifically, the color feature map will highlight the distribution of different color areas, which helps to identify color combinations corresponding to different color coatings on the surface of each component in the image data;

[0064] The third convolution unit is used to extract the shape feature map of the image data; specifically, the shape feature map may show obvious geometric shapes or contours, which helps to distinguish the shapes and sizes of components to correspond to components of different shapes in PCBA;

[0065] The fusion unit is used to fuse the texture feature map, color feature map and shape feature map obtained by each convolution unit to obtain an initial feature map with more distinguishability and relevance.

[0066] In this embodiment, a preferred embodiment of the convolution layer is given. Since the input image data may contain several electronic components of different shapes, colors, and sizes, the textures, shapes, and colors of the surfaces of various components are different. Even for the same type of electronic components, such as resistors, the color combinations of the surfaces are different due to the different internal resistance sizes. Therefore, the feature extraction layer can selectively extract any one or more features of texture, color, shape, etc. to classify and distinguish the resistors; preferably, it includes three convolution units, i.e., three feature extraction units, to respectively extract texture feature maps, color feature maps, and shape feature maps, and then select the most partitioned and characteristic features through the fusion unit to obtain the initial feature map corresponding to the input image data, which can improve the accuracy and efficiency of subsequent component type judgment. It is worth noting that the three local features of texture, color, and shape are only preferred examples with higher predictive recognition, but are not limited to this. The feature extraction layer can extract any one or more of the three features, and can also be used to extract other features for distinction.

[0067] Further preferably, the steps of constructing and training the neural network model may optionally include but are not limited to:

[0068] S11: Build an initial neural network model; preferably use the YOLOv8n deep learning network;

[0069] S12: Collect several original images, mark each component in the original image, and obtain a marked image;

[0070] Specifically, it is optional but not limited to using a labeling tool labelImg to label the targets in the image, so as to label the target categories of all components at corresponding positions in the original image to obtain a labeled image.

[0071] S13: setting the size, moving step, moving direction and starting position of the cut blocks in each original image;

[0072] S14: moving the cutting block in sequence according to the starting position, the moving step length and the moving direction, and extracting the pixel points within the coverage range of the cutting block according to the size of the cutting block to obtain a segmented image; there are overlapping areas between adjacent segmented images;

[0073] For example, the cutting block size is set to 640x640, and the moving step is 512 pixels, that is, each extracted segmented image has a 20% overlap area with the previous extracted segmented image. Starting from the upper left corner of the original image, it moves from left to right and from top to bottom until the original image is segmented. It is worth noting that when the uncut part of the original image is smaller than the moving step, the moving step can be adjusted to increase the overlapping area of ​​the adjacent target image so that the original image is fully cut without reducing the size of the target image.

[0074] Preferably, since the moving step size is constant except for special circumstances when cutting the image, but the electronic components on the PCBA board are unevenly distributed, the segmented image obtained by cutting may not necessarily contain the target, so it is necessary to execute step S15 to eliminate it to reduce the amount of data processing.

[0075] S15: Determine whether each segmented image contains components. If not, discard the segmented image. If so, obtain the target category and target frame data corresponding to each component according to the annotations of the components in each segmented image, and use each component and its target category and target frame data on each segmented image as the original image subset;

[0076] Specifically, it is optional but not limited to judging whether each segmented image contains components based on whether there is a target annotation in each segmented image. If not, the segmented image is discarded. If yes, the target category corresponding to each component is obtained based on the annotation of the component in each segmented image, and then the minimum circumscribed figure of each component is obtained as the target frame of the component, and the size and corner point coordinates of the corresponding target frame are obtained to obtain the target frame data of the target frame corresponding to the component, and each component on each segmented image and its target category and target frame data are used as the original image subset; the minimum circumscribed figure may optionally include a minimum circumscribed rectangle, a minimum circumscribed circle, etc. It is worth noting that when the minimum circumscribed figure is a minimum circumscribed rectangle, its feature point is the center point and / or corner point of the minimum circumscribed rectangle, and when the minimum circumscribed figure is a minimum circumscribed circle, its feature point is the center point of the minimum circumscribed circle, and the feature point of the target frame is arbitrarily set according to the specific shape of the minimum circumscribed figure.

[0077] S16: rotating each segmented image in the original image subset by a predetermined angle several times to obtain new subsets with different angles;

[0078] S17: construct a training data set based on the original image subset and the new subset;

[0079] S18: Input the training data set into the initial neural network model to optimize various parameters of the initial neural network model to obtain the final neural network model.

[0080] Specifically, the above-mentioned preset size, number of rotations, angle, and direction can be arbitrarily set by those skilled in the art.

[0081] Preferably, 80.0% of the original image subset and the new subset are set as training sets, 20% of the original image subset and the new subset are set as verification sets, and the training data set is input into the initial recognition network model. At the same time, the training parameters epochs (number of training times) are set to 300 and imgsz (image size) is set to 640 to optimize the various parameters of the initial recognition network model according to the test data set, and the recognition accuracy of the recognition network model is verified by the verification data set to determine whether it meets the set conditions to obtain the final recognition network model.

[0082] Further preferably, the original image is segmented into several segmented images of 640x640 pixels, which are subsets of the original image. The original image subset is rotated 60 degrees each time for 5 times to obtain five new subsets at different angles; then the original image subset and the new subset together constitute a training data set, or a portion of each set is selected to constitute a training data set together.

[0083] S2: Obtain the original image of the PCBA board to be identified and segment it into several segmented images; there are overlapping areas between adjacent segmented images;

[0084] Specifically, it is optional but not limited to segmenting the original image of the PCBA board to be identified into a plurality of segmented images according to steps S13-S14.

[0085] Preferably, since it is also necessary to map each target frame to a corresponding position of the original image according to the coordinates of the segmented image in the original image and the position coordinates of the target frame in each segmented image, step S2 further includes:

[0086] S21: Obtain the coordinates of the segmented image feature points in the original image; the segmented image feature points may optionally include the image center point and the image corner point. It is worth noting that the coordinates of the image center point and the corner point are the coordinates in the original image; as their coordinate mapping in the segmented image.

[0087] Specifically, it is optional but not limited to moving the cutting block each time to extract pixel points within the coverage range of the cutting block in the original image. When a segmented image is obtained, the center point coordinates and corner point coordinates of the cutting block in the original image are obtained as the feature point coordinates of the segmented image.

[0088] S3: Input each segmented image into the neural network model to obtain the target category, target frame data and confidence of each component on each segmented image;

[0089] Specifically, it is optional but not limited to inputting several segmented images obtained in step S2 into the trained neural network model obtained in step S1 to identify the target category, target frame data and confidence of each component on each segmented image.

[0090] S4: mapping each target frame to a corresponding position of the original image according to the target frame data of each component on each segmented image;

[0091] Specifically, it is optional but not limited to obtaining the positional relationship between the target frame of each component and the original image based on the position of each segmented image in the original image and the coordinates of the corner points of the target frame of each component in the corresponding segmented image, thereby mapping the target frame of each component to the corresponding position of the original image.

[0092] For example, it is optional but not limited to establishing a plane rectangular coordinate system with the upper left corner of the original image as the origin, establishing a segmented image coordinate system with the upper left corner of the segmented image as the origin, assuming that the coordinates of the upper left corner and the lower right corner of the target box in the segmented image coordinate system are (1,1) and (3,3), and the coordinates of the upper left corner of the segmented image in the original image are (5,5). Then, according to the relative position between the original image and the segmented image and the relative position between the target box and the segmented image, the coordinates of the corner points of the target box in the original image can be obtained as (6,6) and (8,8).

[0093] Preferably, since each extracted segmented image has a 20% overlap area with the previous extracted segmented image, the target frames may overlap, so after mapping the target labels of each segmented image to the corresponding positions in the original image, it is also necessary to remove the overlapping target frames;

[0094] S5: performing redundant updates on the overlapping target frames according to the target categories, target frame data and confidence of each component to obtain updated target frames;

[0095] Specifically, it is optional but not limited to removing overlapping target frames according to set rules based on the target category, target frame data and confidence of each component, and retaining the target frame with the highest accuracy as the updated target frame.

[0096] Preferably, the step of redundantly updating the target frames of the overlapping areas according to the target categories, target frame data and confidence of each component may optionally include but is not limited to:

[0097] Target frame data, including target frame position and target frame size; target frame position can be represented by target frame feature point coordinates, corner point coordinates, center point coordinates, etc.; target frame size can be represented by length, width, area, number of pixels, etc.;

[0098] S51: calculating the overlap index between two overlapping target frames according to the target frame data, and setting the target frames whose overlap index is greater than the overlap threshold as the same overlapping target frame group;

[0099] Specifically, due to the existence of overlapping areas in the original image cutting, the identified target frames will overlap. At the same time, due to the influence of environmental factors, image distortion and offset may occur during scanning, resulting in recognition errors, etc., which may cause partial overlap of target frames corresponding to different components. Therefore, it is optional but not limited to determining the overlapping target frames according to the target frame positions, and then determining the area of ​​the overlapping parts according to the target frame sizes; specifically, the overlapping area between the two overlapping target frames is obtained, and the ratio of the overlapping area to the sum of the two total areas is calculated as the overlapping index, and it is determined whether the overlapping index is greater than the overlapping threshold. If it is, the two belong to the same overlapping frame group. If not, it means that the targets corresponding to the two are not the same component.

[0100] S52: Determine whether the target categories corresponding to the target frames in each overlapping target frame group are the same. If so, retain the target frame with the highest confidence and remove the other target frames in the group; if not, retain the target frame with the largest target frame size and remove the other target frames in the group;

[0101] Specifically, it is optional but not limited to judging whether the target categories obtained by the neural network model for the targets contained in each target frame in the same overlapping target frame group after grouping in step S1 are the same; if the categories are the same, obtaining the confidence corresponding to each target frame, removing the target frame with low confidence, retaining the target frame with the highest confidence as the target frame finally determined in the group of overlapping target frames; if the target categories obtained by the neural network model for the targets contained in each target frame are different, the complete image of the component may not be obtained during image cutting, causing the neural network model to make errors during recognition; since the more complete the component image is, the higher the recognition accuracy of the neural network model is, the target frame with a larger target frame size should be retained.

[0102] For example, the overlapping indexes of three target frames 1-3 are greater than the set threshold and are classified into the same overlapping target frame group. The target categories, target frame data, and partial confidence data of the corresponding components are as follows:

[0103] Target box 1:

[0104] Target category: resistor;

[0105] Target box size: 12×12;

[0106] Confidence: Resistance 80%

[0107] Capacitance 20%

[0108] Target box 2:

[0109] Target category: resistor;

[0110] Target box size: 15×15;

[0111] Confidence: Resistance 90%

[0112] Capacitor 10%

[0113] Target box 3:

[0114] Target category: Capacitor;

[0115] Target box size: 13×13;

[0116] Confidence: Capacitance 90%

[0117] Resistor 10%

[0118] In the above three target frames, since the target categories of target frame 1 and target frame 2 are the same, target frame 2 with high confidence is retained, and target frame 1 with low confidence is removed; target frame 2 and target frame 3 have different target categories, target frame 2 with a larger target frame size is retained, and target frame 3 with a smaller target frame size is removed; finally, in this group of overlapping target frames, target frame 2 is retained as the final target frame.

[0119] More preferably, in step S52, the target frame with the highest reliability is retained as the final target frame according to the target category, target frame size and confidence corresponding to each target frame in each overlapping target frame group. However, when there are multiple target frames with the highest confidence or the largest size, they need to be further screened. Therefore, step S52 further includes:

[0120] S53: Calculate the target feature index of each target frame according to the confidence level and target frame size corresponding to each target frame, retain the target frame with a larger target feature index, and remove other target frames in the group.

[0121] Specifically, it is optional but not limited to obtaining the product of the confidence and target frame size corresponding to each target frame in the overlapping target frame group as the target feature index corresponding to each target frame, and comparing the sizes of the target feature indexes corresponding to each target frame, retaining the target frame with a larger target feature index as the final target frame.

[0122] S6: Annotate the target category, target frame data and confidence of the updated target frame on the corresponding target frame of the original image with a target label.

[0123] Specifically, after deduplication of all overlapping target frames according to step S5, each electronic component has only one target frame corresponding to it, and the target category, confidence, and target frame data such as the target frame shape, target frame size, and target frame feature point coordinates corresponding to each component target frame are combined to form a target label. The target label is marked on the target frame corresponding to each electronic component in the original image to obtain the final recognition result.

[0124] In this embodiment, a component recognition method for a PCBA board of the present invention is provided. First, a special neural network model is constructed and trained, which takes the image data of the PCBA board as input and the target category, target frame data and confidence of each component on the PCBA board as output; then, the original image of the current PCBA board to be recognized is obtained and segmented into several segmented images. The key is to avoid the failure to obtain a complete image of the electronic component during cutting, and to ensure that there are overlapping areas between adjacent segmented images during segmentation, so as to improve the recognition accuracy of the subsequent neural network model. At the same time, the resolution of the segmented image is low, which can reduce the consumption of computing resources; on this basis, each segment is segmented into several segments. The image is cut and input into the neural network model to obtain the target category, target frame data and confidence of each component on each segmented image; then, according to the target frame data of each component on each segmented image, each target frame is mapped to the corresponding position of the original image, and then according to the target category, target frame data and confidence of each component, the overlapping target frames are redundantly updated to obtain the updated target frame. The key is to avoid obtaining only part of the structure of the electronic component when cutting the image, which leads to recognition errors, reduced accuracy and redundant annotation problems; finally, the target category, target frame data and confidence of the updated target frame are annotated on the corresponding target frame of the original image with a target label. The key to the present invention is that the early segmentation includes overlapping parts, and the later recognition includes redundant removal parts, especially according to the target category, target frame data and confidence of each component, the overlapping target frames are redundantly updated, which is more accurate than the traditional solution which only deletes by confidence. As described in the background technology, some structures of many electronic components are very similar, such as a diode and a light-emitting diode which only differ by an arrow, and some are only different in chip pins, etc., so the recognition types of some overlapping target frames may be inconsistent. Simply removing redundancy by confidence will greatly reduce the recognition accuracy; it solves the problems of high computing resource consumption, large data processing volume, large recognition error and low recognition accuracy in the prior art.

[0125] On the other hand, the present invention further provides a computer storage medium storing an executable program code; the executable program code is used to execute any of the above-mentioned component identification methods for PCBA boards.

[0126] On the other hand, the present invention also provides a terminal device, including a memory and a processor; the memory stores a program code that can be executed by the processor; the program code is used to execute any of the above-mentioned component identification methods for PCBA boards.

[0127] Exemplarily, the program code may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the program code in the terminal device.

[0128] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the terminal device may also include an input / output device, a network access device, a bus, etc.

[0129] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0130] The memory may be an internal storage unit of the terminal device, such as a hard disk or a memory. The memory may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device. Furthermore, the memory may include both an internal storage unit of the terminal device and an external storage device. The memory is used to store the program code and other programs and data required by the terminal device. The memory may also be used to temporarily store data that has been output or is to be output.

[0131] The above-mentioned computer storage medium and terminal device are created based on the above-mentioned component identification method for PCBA board. Its technical functions and beneficial effects are not repeated here. The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0132] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A component identification method for a PCBA board, characterized in that: include: Build and train a neural network model that takes the image data of the PCBA board as input and outputs the target category, target box data, and confidence of each component on the PCBA board; Obtain the original image of the PCBA board to be identified and segment it into several segmented images; there are overlapping areas between adjacent segmented images; Input each segmented image into the neural network model to obtain the target category, target frame data and confidence of each component on each segmented image; According to the target frame data of each component on each segmented image, each target frame is mapped to the corresponding position of the original image; According to the target category, target frame data and confidence of each component, redundantly update the overlapping target frames to obtain updated target frames; including: target frame data, including target frame position and target frame size; according to the target frame data, calculate the overlap index between the two overlapping target frames, and set the target frames with overlap index greater than the overlap threshold as the same overlapping target frame group; determine whether the target categories corresponding to the target frames in each overlapping target frame group are the same, if so, retain the target frame with the highest confidence and remove other target frames in the group; if not, retain the target frame with the largest target frame size and remove other target frames in the group; The target category, target frame data and confidence of the updated target frame are annotated on the corresponding target frame of the original image with the target label.

2. The method according to claim 1, characterized in that The specific structure of the neural network model is: The input layer receives the input image data and then passes the image data to the convolutional layer; The convolution layer is connected to the input layer and is used to extract local features from the input image data to obtain the initial feature map; The batch normalization layer is connected to the convolutional layer and is used to normalize the output of each layer to improve training stability and accelerate convergence, reduce internal covariate shift, and obtain the normalized updated feature map; The pooling layer is connected to the batch normalization layer to reduce the spatial dimension of the updated feature map and retain the main features. The updated feature map is then pooled to reduce the amount of calculation and prevent overfitting to obtain the optimized feature map. The feature fusion layer is connected to the pooling layer and is used to fuse feature maps from different levels to capture information of objects of different sizes and obtain the final feature map; The detection head is connected to the feature fusion layer and is used to further process the final feature map to generate detection results, including the target box position, component category and confidence level; The output layer is connected to the detection head and is used to integrate the target box, category and confidence generated by the detection head into the final detection result, which is the target label corresponding to each target.

3. The method according to claim 2, characterized in that The convolutional layer consists of: A first convolution unit, used to extract a texture feature map of the image data; A second convolution unit, used to extract a color feature map of the image data; A third convolution unit, used to extract a shape feature map of the image data; The fusion unit is used to fuse the texture feature map, color feature map and shape feature map obtained by each convolution unit to obtain an initial feature map with more distinguishability and relevance.

4. The method according to claim 1, characterized in that: The steps to build and train a neural network model include: Build an initial neural network model; Collect several original images, mark each component in the original image, and obtain a marked image; Set the size, moving step, moving direction and starting position of the cut blocks in each original image; The cutting block is moved in sequence according to the starting position, moving step length and moving direction, and the pixel points within the coverage range of the cutting block are extracted according to the size of the cutting block to obtain a segmented image; there are overlapping areas between adjacent segmented images; Determine whether each segmented image contains components. If not, discard the segmented image. If so, obtain the target category and target frame data corresponding to each component based on the annotations of the components in each segmented image, and use the components and their target categories and target frame data on each segmented image as the original image subset; Each segmented image in the original image subset is rotated several times by a predetermined angle to obtain new subsets with different angles; Construct a training data set based on the original image subset and the new subset; The training data set is input into the initial neural network model to optimize the parameters of the initial neural network model and obtain the final neural network model.

5. The method according to claim 1, characterized in that The step of obtaining the original image of the PCBA board to be identified and segmenting it into a plurality of segmented images also includes: Get the coordinates of the segmented image feature points in the original image; segmented image feature points, optionally including image center points and image corner points.

6. The method according to claim 1, characterized in that The overlapping index between the two overlapping target frames is calculated, specifically: the ratio of the overlapping area of ​​the two overlapping target frames to the sum of their total areas is calculated.

7. The method according to claim 6, characterized in that The step of redundantly updating the overlapping target frames according to the target categories, target frame data and confidence of each component to obtain an updated target frame also includes: According to the confidence and size of each target frame, its target feature index is calculated, the target frame with a larger target feature index is retained, and the other target frames in the group are removed.

8. A computer storage medium, characterized in that An executable program code is stored; when the executable program code is executed by a processor, it is used to implement the component identification method for a PCBA board as described in any one of claims 1 to 7.

9. A terminal device, characterized in that: It comprises a memory and a processor; the memory stores a program code executable by the processor; when the program code is executed by the processor, it is used to implement the component identification method for a PCBA board as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • PCB component defect detection method

    CN115775246A

  • Circuit board electronic component detection method and device and computer equipment

    CN116245882A