A PCB component accurate identification method based on difficult sample mining

Abstract

The application discloses a PCB component accurate identification method based on difficult sample mining, wherein a training picture sample set is used to train a rough classifier; then, samples with incorrect identification results in the rough classifier and samples with correct identification results and classification confidence lower than a set threshold theta in the rough classifier are used as difficult sample samples; the difficult sample samples are used to train a super-resolution reconstruction network; and images and labels output by the super-resolution reconstruction network are used to train a fine classifier. Pictures to be measured are sequentially input into the trained rough classifier, the super-resolution reconstruction network and the fine classifier for accurate identification, and identification results are output. The application can quickly and simply identify relatively clear and obvious PCB component pictures, accurately and effectively identify relatively small or relatively blurred PCB component pictures, and has good practicability.

Description

Technical Field

[0001] This invention belongs to the technical field of industrial inspection, specifically relating to a method for accurate identification of PCB components based on difficult sample mining. Background Technology

[0002] Industrial inspection is a crucial area in the industrial sector, directly impacting the yield rate and performance of industrial products. Image recognition technology is a core technology in automated industrial inspection. Automated industrial inspection systems based on intelligent image recognition technology can improve product performance, reduce scrap rates, and enhance supply chain management. PCB defect detection is a significant area within automated industrial inspection. Among defects, incorrect component types represent a serious product failure. Compared to less severe failures such as over-soldering or misaligned leads, incorrect component types directly render the PCB unusable. Therefore, accurate identification of PCB components is a vital aspect of PCB manufacturing.

[0003] Unlike natural scene images where targets are relatively large and sparsely distributed, PCB images show extremely densely packed components. A typical PCB contains hundreds of components, primarily capacitors, inductors, and resistors. Due to the limited board area and the sheer number of components, each individual component is relatively small. Therefore, PCB component identification is a problem of detecting and recognizing densely packed small targets.

[0004] Most current PCB component identification software systems employ deep learning methods. Deep learning is a data-driven approach that, compared to traditional image recognition, can learn and iterate on a massive number of samples, thereby improving system performance. Existing component type identification algorithms use a single deep network from the natural image domain for identification, employing a simple data iteration pattern to train the neural network. This simplistic approach cannot adaptively process components of different sizes and types, resulting in low identification performance. Therefore, this invention provides a method for accurate PCB component identification based on hard sample mining. Summary of the Invention

[0005] The purpose of this invention is to provide a method for accurate identification of PCB components based on difficult sample mining, aiming to solve the aforementioned problems. This invention first uses a shallow deep neural network for coarse identification and filters out difficult samples. Then, it employs super-resolution reconstruction technology to improve resolution and image size, restoring more detailed information. Finally, it uses a deep neural network with higher recognition performance for fine identification, thereby improving recognition performance and achieving accurate identification of PCB component types.

[0006] This invention is mainly achieved through the following technical solutions:

[0007] A method for accurate identification of PCB components based on hard sample mining includes the following steps:

[0008] Step S1: Use an industrial camera to capture images of the PCB board;

[0009] Step S2: Manually label and create a training image sample set;

[0010] Step S3: Train a coarse classifier using the training image sample set;

[0011] Step S4: Take the samples with incorrect identification results in the coarse classifier and the samples with correct identification results in the coarse classifier and classification confidence scores below the set threshold θ as hard sample samples, and add the sample images and corresponding label categories to the hard sample library.

[0012] Step S5: Train the super-resolution reconstruction network using hard sample images, enlarge the hard sample images using bilinear interpolation, input the hard sample images into the super-resolution reconstruction network, and use the enlarged hard sample images as the output images of the super-resolution reconstruction network.

[0013] Step S6: Train a fine classifier using the images and labels output by the super-resolution reconstruction network;

[0014] Step S7: Input the PCB image to be tested captured by the industrial camera into the trained coarse classifier for recognition. If the confidence level of the recognition is higher than the set threshold, the recognition result is directly output; otherwise, it is input into the trained super-resolution reconstruction network for super-resolution reconstruction, and then output into the trained fine classifier for accurate recognition, and the recognition result is output.

[0015] This invention is a method for accurate identification of PCB (Printed Circuit Board) components based on deep learning and image processing. It is used to identify the types of components (capacitors, inductors, resistors, etc.) in PCB images captured by industrial cameras, thereby realizing automated identification of PCB components, reducing the workload and difficulty of manual inspection, and improving the pass rate of PCB products.

[0016] To better implement the present invention, in step S3, the coarse classifier adopts the AlexNet neural network, and retains the pooling operation of the odd-numbered layers of the AlexNet neural network while removing the pooling operation of the even-numbered layers.

[0017] In this invention, AlexNet, with its relatively small number of layers, is used as the coarse classifier neural network. Compared to current neural networks with higher performance and more layers, AlexNet has lower computational cost, fewer parameters, and lower requirements for the number of samples. To address the small image size processed by the coarse classifier, this invention adjusts and optimizes the AlexNet network by eliminating some pooling operations. This avoids the problem of excessively low network output resolution due to frequent downsampling, which would lead to an insufficiently small input dimension in the last few layers of the network, hindering effective training. Eliminating pooling operations effectively preserves the position and size information of the component images, improving the recognition performance of the coarse classifier AlexNet without sacrificing overall classification performance.

[0018] Visualizing the outputs of each layer in the AlexNet neural network reveals that lower-level neural networks output detailed image information such as pixels and edges, while higher-level neural networks output semantic information such as object components and object types. For accurate identification of PCB components, these components have clearly defined edges and contours, and this edge and contour information is crucial for distinguishing between components. For example, a resistor's contour is elliptical and relatively complete, while solder's contour is square and may have gaps. Furthermore, component information is also essential for accurate identification, as different components are composed of different parts.

[0019] For example, a complete capacitor unit consists of several components such as capacitor, pins, and solder. These components constitute the characteristic information of the capacitor unit. Therefore, when performing desampling design, it is necessary to retain four types of network output information that are significantly important for PCB component identification: edges, contours, components, and types. In the AlexNet network output, the second convolutional neural network outputs edge and contour information, the fourth convolutional neural network outputs component information, and the sixth convolutional neural network outputs type information. Therefore, it is necessary to retain the complete outputs of the second, fourth, and sixth layers. Since pooling operations will lose network layer output information, it is necessary to cancel the pooling operations of the second, fourth, and sixth layers, and retain the pooling operations of the first, third, and fifth layers.

[0020] To better implement the present invention, in step S4, the normalized feature norm of the image is used as a measure of image recognizability, thereby distinguishing difficult sample samples into difficult identifiable samples and difficult unidentifiable samples, and removing difficult unidentifiable samples from the difficult sample library.

[0021] To better realize the present invention, step S4 further includes the following steps:

[0022] Step A: Let the features of the image after passing through the convolutional layer be:

[0023] (1)

[0024] in, N For the number of images, M The dimension of the convolutional features;

[0025] Step B: Calculate the mean and variance of each feature dimension:

[0026] ;

[0027] (2)

[0028] Step C: Calculate the normalized feature norm:

[0029] (3)

[0030] if norm (F i If the value is greater than the threshold σ, it is considered a difficult-to-identify sample and no action is taken; if... norm (F i If the value is less than or equal to the threshold σ, it is considered a difficult unrecognizable sample, and the corresponding image and label are removed from the difficult sample library.

[0031] To better implement the present invention, the threshold θ is further set to 0.7 and the threshold σ is set to 0.4.

[0032] In the recognition process of the coarse classifier based on AlexNet, some samples can be accurately classified with a high confidence level. These samples are easy to identify and do not require further fine-grained recognition. In the accurate identification of PCB components, images with a straight shooting angle and clear image are easier to identify, as shown in Figure 1(a). These images can be accurately identified by the AlexNet coarse classifier and do not require fine-grained recognition. Images that are clear but have a large shooting angle (as shown in Figure 1(b)) or a long shooting distance (as shown in Figure 1(c)) are difficult for the AlexNet coarse classifier to identify accurately, but can be accurately identified by a finer recognition network after super-resolution reconstruction. Images that are blurred or distorted due to factors such as optical jitter or insufficient lighting have too much loss of detail information. Even after super-resolution reconstruction, the detail information cannot be recovered, so fine-grained recognition is not possible. Moreover, adding such blurry or distorted samples to the training sample library will reduce the recognition performance. Therefore, these samples need to be removed, as shown in Figure 1(d) and Figure 1(e).

[0033] Existing methods can only distinguish between easily identifiable samples (Fig. 1(a)) and difficult-to-identify samples (Fig. 1(b)-Fig. 1(e)), but cannot distinguish between difficult-to-identify samples and difficult-to-unidentify samples. Adding difficult-to-unidentify samples to the coarse and fine classifiers will reduce classification performance. In PCB component identification, due to the large variety and dense distribution of components, there are many difficult-to-identify training images, requiring effective screening methods. Removing unidentifiable samples can not only eliminate the accuracy loss of the fine classifier caused by low-quality unidentifiable samples, but also reduce the number of difficult sample libraries and improve the training speed of the fine classifier.

[0034] When analyzing PCB components, it was found that after images passed through the AlexNet convolutional layer, clear images had higher feature norms, while blurry or severely distorted images had lower normalized feature norms. Therefore, this invention proposes using the normalized feature norm of an image as a measure of its recognizability, thereby distinguishing between images that are difficult to recognize (Fig. 1(b)-Fig. 1(c)) and images that are difficult to recognize (Fig. 1(d)-Fig. 1(e)).

[0035] To better implement this invention, the super-resolution reconstruction network in step S5 further employs the BasicVSR++ network. The reason for misclassification or low confidence in some difficult sample images is that the image size is too small, resulting in insufficient image information and target detail information, making accurate identification by the classifier impossible. To address the performance degradation caused by images that are too small or too blurry, this invention uses the latest and most advanced super-resolution algorithm, BasicVSR++, for image super-resolution reconstruction.

[0036] The BasicVSR++ network has a moderate computational cost and number of parameters, resulting in reconstructed images with minimal distortion and rich detail. The difficult sample images from step S4 are enlarged using the bilinear interpolation algorithm. The original image is used as the input image for BasicVSR++, and the enlarged image is used as the output image, thus enabling the training of the BasicVSR++ algorithm.

[0037] To better implement this invention, in step S6, the fine classifier uses the ShuffleNet V2 network. The images after super-resolution reconstruction are large in size and rich in detail, but are difficult to recognize. Therefore, a more advanced neural network with better performance and higher accuracy is needed for training. This invention uses the lightweight recognition neural network ShuffleNet V2, which is more mobile-friendly and has excellent classification performance, as the fine classifier. Compared to neural networks such as ResNet and EfficientNet, which have extremely deep layers and a large number of parameters, ShuffleNet V2 has less computation, fewer parameters, and excellent classification performance.

[0038] To better realize the present invention, in step S7, the area where the component is located is selected to cut out the PCB image and obtain a sub-image, and the sub-image is input into the trained coarse classifier for recognition.

[0039] The beneficial effects of this invention are:

[0040] (1) The present invention adopts a hierarchical recognition framework based on difficult sample mining, which can quickly and easily identify relatively clear and obvious PCB component images, and accurately and effectively identify PCB component images that are small in size or relatively blurry.

[0041] (2) The present invention uses an improved AlexNet neural network as a coarse classifier, reduces the pooling operation in the AlexNet network, and can save the position information and detail information of relatively small component images, thereby improving classification performance;

[0042] (3) The present invention uses the coarse classifier recognition results and recognition confidence to select difficult samples for recognition, and then uses the image normalization feature norm to distinguish difficult identifiable samples from difficult unidentifiable samples. In this way, not only can the recognition performance of clear and identifiable samples be improved, but also the loss of recognition performance caused by severely blurred and distorted image samples can be eliminated, and the training of fine classifier can be accelerated.

[0043] (4) The present invention uses the BasicVSR++ algorithm to perform super-resolution reconstruction on relatively blurry component images, thereby enhancing the detail and size information of the component images;

[0044] (5) The present invention combines the PCB schematic diagram to confirm and accurately cut out the component images, thus avoiding the component positioning deviation introduced by the image recognition method. Attached Figure Description

[0045] Figure 1(a) shows a PCB board image of an easily identifiable sample;

[0046] Figure 1(b) shows a PCB board image of the difficult sample I that can be finely identified;

[0047] Figure 1(c) shows a PCB board image of a difficult sample II that can be finely identified;

[0048] Figure 1(d) shows a PCB board image of difficult sample I that cannot be clearly identified;

[0049] Figure 1(e) shows a PCB board image of the difficult sample II that cannot be clearly identified;

[0050] Figure 2 This is a flowchart of the neural network training process of the present invention;

[0051] Figure 3 This is a flowchart illustrating the PCB component identification process of the present invention.

[0052] Figure 4 This is a flowchart of the screening process for difficult sample cases. Detailed Implementation

[0053] Example 1:

[0054] A method for accurate identification of PCB components based on hard sample mining, such as Figure 2 , Figure 3 As shown, it includes the following steps:

[0055] Step S1: Use an industrial camera to capture images of the PCB board;

[0056] Step S2: Manually label and create a training image sample set;

[0057] Step S3: Train a coarse classifier using the training image sample set;

[0058] Step S4: Take the samples with incorrect identification results in the coarse classifier and the samples with correct identification results in the coarse classifier and classification confidence scores below the set threshold θ as hard sample samples, and add the sample images and corresponding label categories to the hard sample library.

[0059] Step S5: Train the super-resolution reconstruction network using hard sample images, enlarge the hard sample images using bilinear interpolation, input the hard sample images into the super-resolution reconstruction network, and use the enlarged hard sample images as the output images of the super-resolution reconstruction network.

[0060] Step S6: Train a fine classifier using the images and labels output by the super-resolution reconstruction network;

[0061] Step S7: Input the PCB image to be tested captured by the industrial camera into the trained coarse classifier for recognition. If the recognition confidence is higher than the set threshold, the recognition result is directly output. Otherwise, it is input into the trained super-resolution reconstruction network for super-resolution reconstruction, and then output into the trained fine classifier for accurate recognition, and the recognition result is output.

[0062] This invention employs a hierarchical recognition framework based on difficult example mining, which can quickly and easily identify relatively clear and obvious PCB component images, as well as accurately and effectively identify smaller or blurry PCB component images.

[0063] Example 2:

[0064] A method for accurate identification of PCB components based on hard sample mining, such as Figure 2 , Figure 3 As shown, it includes the following steps:

[0065] Step S1: Use an industrial camera to capture images of the PCB board;

[0066] Step S2: Manually label and create a training image sample set;

[0067] Step S3: Train a coarse classifier using the training image sample set. The coarse classifier uses the AlexNet neural network, and cancels the pooling operations of the 2nd, 4th, and 6th layers of the AlexNet neural network, while retaining the pooling operations of the 1st, 3rd, and 5th layers.

[0068] Step S4: Take the samples with incorrect identification results in the coarse classifier and the samples with correct identification results in the coarse classifier and classification confidence scores below the set threshold θ as hard sample samples, and add the sample images and corresponding label categories to the hard sample library.

[0069] Step S5: Train the super-resolution reconstruction network using hard sample images. The super-resolution reconstruction network uses the BasicVSR++ network. Use bilinear interpolation to enlarge the hard sample images and input the hard sample images into the super-resolution reconstruction network. Use the enlarged hard sample images as the output images of the super-resolution reconstruction network.

[0070] Step S6: Train a fine classifier using the images and labels output by the super-resolution reconstruction network. The fine classifier uses the ShuffleNet V2 network.

[0071] Step S7: Take the PCB image to be tested captured by the industrial camera, select the area where the components are located, cut out the PCB image and obtain a sub-image, input the sub-image into the trained coarse classifier for recognition, if the recognition confidence is higher than the set threshold, the recognition result is directly output, otherwise it is input into the trained super-resolution reconstruction network for super-resolution reconstruction, and then output into the trained fine classifier for accurate recognition, and output the recognition result.

[0072] This invention employs an improved AlexNet neural network as a coarse classifier, reducing pooling operations in the AlexNet network and preserving the positional and detail information of relatively small component images, thereby improving classification performance. This invention uses the BasicVSR++ algorithm to perform super-resolution reconstruction on blurry component images, enhancing the detail and size information of the component images. This invention combines PCB schematics to confirm and accurately cut out component images, avoiding the component positioning errors introduced by image recognition methods.

[0073] Example 3:

[0074] This embodiment is an optimization based on Embodiment 1 or Embodiment 2, such as... Figure 4 As shown, in step S4, the normalized feature norm of the image is used as a measure of image identifiability, thereby distinguishing difficult sample samples into difficult identifiable samples and difficult unidentifiable samples, and removing difficult unidentifiable samples from the difficult sample library. This includes the following steps:

[0075] Step A: Let the features of the image after passing through the convolutional layer be:

[0076] (1)

[0077] in, N For the number of images, M The dimension of the convolutional features;

[0078] Step B: Calculate the mean and variance of each feature dimension:

[0079] ;

[0080] (2)

[0081] Step C: Calculate the normalized feature norm:

[0082] (3)

[0083] if norm (F i If the value is greater than the threshold σ, it is considered a difficult-to-identify sample and no action is taken; if... norm (F i If the value is less than or equal to the threshold σ, it is considered a difficult unrecognizable sample, and the corresponding image and label are removed from the difficult sample library.

[0084] This invention uses the coarse classifier recognition results and recognition confidence to select difficult samples for recognition, and then uses the image normalized feature norm to distinguish between difficult identifiable samples and difficult unidentifiable samples. In this way, not only can the recognition performance of clear and identifiable samples be improved, but the loss of recognition performance caused by severely blurred and distorted image samples can also be eliminated, and the training of fine classifier can be accelerated.

[0085] The other parts of this embodiment are the same as those in Embodiment 1 or Embodiment 2, so they will not be described again.

[0086] Example 4:

[0087] A method for accurate identification of PCB components based on hard sample mining, such as Figures 2-4 As shown, it includes the following steps:

[0088] (1) Step S1: Use an industrial camera to take images of the PCB board.

[0089] (2) Step S2: Based on the PCB schematic, manually annotate and create a training image sample set. The training samples include background images, capacitor images, inductor images, and resistor images.

[0090] (3) Step S3: Train a coarse classifier using the training image sample set.

[0091] We use AlexNet, a neural network with fewer layers, as the coarse classifier, and cancel the pooling operations in layers 2, 4, and 6, while retaining the pooling operations in layers 1, 3, and 5.

[0092] (4) Step S4: As Figure 4 As shown, the images of difficult samples corresponding to the coarse classifier are selected.

[0093] Step 1: Check the coarse classifier's recognition results. If the classification result is incorrect, the current sample is a difficult sample, and the image and its corresponding label category are added to the difficult sample library; if the classification result is correct, proceed to Step 2.

[0094] Step 2: Check the confidence level of the coarse classifier. If the confidence level is less than... θ If the current sample is a hard sample, then the image and its corresponding label category are added to the hard sample library. θ The threshold value is set.

[0095] After using Step 1-Step 2 to screen difficult samples, it is necessary to remove unidentifiable samples. This not only eliminates the accuracy loss of the fine classifier caused by low-quality unidentifiable samples, but also reduces the number of difficult sample sets and improves the training speed of the fine classifier.

[0096] Furthermore, the normalized feature norm of the image is used as a measure of image identifiability to distinguish between hard-to-identify and hard-to-unidentify samples, and then hard-to-unidentify samples are removed from the hard sample library. The specific steps are as follows:

[0097] Step 1: Let the features of the image after passing through the convolutional layer be:

[0098] (1)

[0099] in, N For the number of images, M denoted as the convolution feature dimension.

[0100] Step 2: Calculate the mean and variance of each feature dimension:

[0101] ;

[0102] (2)

[0103] Step 3: Calculate the normalized feature norm:

[0104] (3)

[0105] Step 4: If norm (F i If the value is greater than the threshold σ, it is considered a difficult-to-identify sample and no action is taken; if... norm (F i If the value is less than or equal to the threshold σ, it is considered a difficult unrecognizable sample, and the corresponding image and label are removed from the difficult sample library.

[0106] (5) Step S5: Train the super-resolution reconstruction network using difficult example images

[0107] This invention employs the latest and most advanced super-resolution algorithm, BasicVSR++, for image super-resolution reconstruction. The difficult sample images from step S4 are enlarged using a bilinear interpolation algorithm. The original image is used as the input image for BasicVSR++, and the enlarged image is used as the output image, thus enabling the training of the BasicVSR++ algorithm.

[0108] (6) Step S6: Train the fine classifier using the super-resolution reconstructed images and labels.

[0109] This invention employs ShuffleNet V2, a lightweight neural network with excellent classification performance and good mobile deployment capabilities, as the fine classifier. Compared to neural networks such as ResNet and EfficientNet, which have extremely deep layers and a large number of parameters, ShuffleNet V2 has lower computational cost, fewer parameters, and superior classification performance.

[0110] (7) Step S7: Combine the PCB schematic diagram with the PCB image captured by the camera to identify components;

[0111] After the industrial camera captures images of the PCB, the area where the components are located is selected for image matting based on the PCB image and schematic diagram. The sub-image is then fed into the coarse classifier AlexNet for recognition. If the recognition confidence is high, the recognition result is directly output; if the recognition confidence is low, it is fed into the super-resolution reconstruction module BasicVSR++ for super-resolution reconstruction. The reconstructed image is then fed into the fine classifier ShuffleNet V2 for accurate recognition, and the recognition result is output.

[0112] This invention employs a hierarchical recognition framework based on hard sample mining, enabling both rapid and simple recognition of relatively clear and obvious PCB component images, as well as accurate and effective recognition of smaller or blurry PCB component images. The invention uses the coarse classifier's recognition results and recognition confidence to select hard samples for identification, and then uses the image normalized feature norm to distinguish between hard-to-identify and hard-to-unidentify samples. This not only improves the recognition performance of clear and identifiable images but also eliminates the performance loss caused by severely blurred or distorted image samples, accelerating the training of the fine classifier.

[0113] Example 5:

[0114] A method for accurate identification of PCB components based on hard sample mining employs an industrial digital camera with an active light source and lens to capture PCB images. The software system uses Microsoft Windows 11 Home Chinese Edition as the operating system, Facebook's PyTorch deep learning framework, and Mathworks' Matlab software for image acquisition and preprocessing. The processor for deep learning training and inference is an Intel i7, and the computing platform for training and inference is an NVIDIA GetForce RTX 3050 dedicated graphics card. The specific steps include:

[0115] (1) PCB image acquisition

[0116] Using an industrial digital camera, images of PCBs of different types and sizes were repeatedly taken from multiple angles and distances. The brightness of the active light source was also adjusted to varying degrees, and the corresponding PCB model for each set of images was recorded. This resulted in image sets of various PCB models in different scenarios.

[0117] (2) Training sample preparation

[0118] For each PCB, images were used in conjunction with the PCB schematics. The open-source annotation software LabelImg was employed to select sub-images for various components such as capacitors, inductors, and resistors, and assign corresponding category labels. Considering the complexity of PCB components in terms of appearance and type, the number of samples for each component exceeded 6000. PyTorch's built-in sample preprocessing scripts, combined with Matlab, were used to convert the image and label sets into the corresponding PyTorch data format.

[0119] (3) Coarse classifier training

[0120] The coarse classifier was trained using all samples with default configuration parameters. Based on the image characteristics of the original PCB components, the training batch size was set to 16 and the number of training rounds was set to 20. The training algorithm used was stochastic gradient descent (SGD).

[0121] The coarse classifier removes the pooling operation of even-numbered layers, and the remaining network parameters use the pre-trained parameter values, which are then fine-tuned.

[0122] (4) Filter the hard sample images corresponding to the coarse classifier.

[0123] Filter difficult sample images, filter the completed difficult sample images and their labels, and set a threshold. θ The threshold σ was set to 0.7, and the threshold value σ was set to 0.4. Data augmentation was used to expand the sample size based on the number of samples in each class, increasing the number of samples for classes with fewer samples. After the sample augmentation, the number of hard cases for each class exceeded 1500.

[0124] (5) Training the super-resolution reconstruction network

[0125] For the augmented hard sample images, the bilinear interpolation method was used to enlarge them by 2x, 4x, and 8x respectively, and the parameters of BasicVSR++ were adjusted. The images were then trained using the original images and the three enlarged images.

[0126] Training was performed using default configuration parameters, with batch sizes of 8, 4, and 2 for the three training iterations, and 20 training epochs. The training algorithm used was stochastic gradient descent (SGD). Pre-trained parameter values ​​were used and then fine-tuned.

[0127] (6) Training of the fine classifier

[0128] The high-resolution image was used to train the classifier. The training used default configuration parameters, with a batch size of 4, epochs of 20, and stochastic gradient descent (SGD) as the training algorithm. Pre-trained parameters from ShuffleNet V2 were used, and fine-tuning was performed based on these pre-trained values.

[0129] (7) Real-time recognition

[0130] PCB images are captured using a high-frame-rate industrial digital camera to obtain classification results for each component. The classification results and their corresponding locations are then output to calibration personnel. These personnel compare the standard types on the PCB schematic with the category information output by the software system, focusing on identifying components with incorrect types and addressing the problematic PCBs.

[0131] The results of testing on 1000 different types of components collected in practice are shown in Table 1. When only the AlexNet coarse classifier is used for identification, the identification accuracy is 86.4%. When the hierarchical identification method is used without adding the feature norm-based distinguishability judgment, the identification accuracy is 93.7%. After adding the feature norm-based distinguishability judgment strategy, the identification accuracy is 95.9%. It can be seen that the present invention can significantly improve the identification performance of components.

[0132] Table 1

[0133]

[0134] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for accurate identification of PCB components based on hard sample mining, characterized in that, Includes the following steps: Step S1: Use an industrial camera to capture images of the PCB board; Step S2: Manually label and create a training image sample set; Step S3: Train a coarse classifier using the training image sample set; Step S4: Take the samples with incorrect identification results in the coarse classifier and the samples with correct identification results in the coarse classifier and classification confidence scores below the set threshold θ as hard sample samples, and add the sample images and corresponding label categories to the hard sample library. Step S5: Train the super-resolution reconstruction network using hard sample images, enlarge the hard sample images using bilinear interpolation, input the hard sample images into the super-resolution reconstruction network, and use the enlarged hard sample images as the output images of the super-resolution reconstruction network. Step S6: Train a fine classifier using the images and labels output by the super-resolution reconstruction network; Step S7: Input the PCB image to be tested captured by the industrial camera into the trained coarse classifier for recognition. If the confidence level of the recognition is higher than the set threshold, the recognition result is directly output; otherwise, it is input into the trained super-resolution reconstruction network for super-resolution reconstruction, and then output into the trained fine classifier for accurate recognition, and the recognition result is output. In step S4, the normalized feature norm of the image is used as a measure of image recognizability, thereby distinguishing difficult sample samples into difficult identifiable samples and difficult unidentifiable samples, and removing difficult unidentifiable samples from the difficult sample library. Step S4 includes the following steps: Step A: Let the features of the image after passing through the convolutional layer be: （1） in, N For the number of images, M The dimension of the convolutional features; Step B: Calculate the mean and variance of each feature dimension: ； （2） Step C: Calculate the normalized feature norm: （3） if norm (F i If the value is greater than the threshold σ, it is considered a difficult-to-identify sample and no action is taken; if... norm (F i If the value is less than or equal to the threshold σ, it is considered a difficult unrecognizable sample, and the corresponding image and label are removed from the difficult sample library.

2. The method for accurate identification of PCB components based on hard sample mining according to claim 1, characterized in that, In step S3, the coarse classifier uses the AlexNet neural network, retains the pooling operations of the odd-numbered layers of the AlexNet neural network, and removes the pooling operations of the even-numbered layers.

3. The method for accurate identification of PCB components based on hard sample mining according to claim 1, characterized in that, The threshold θ The value is 0.7, and the threshold σ is 0.

4.

4. The method for accurate identification of PCB components based on hard sample mining according to claim 1, characterized in that, In step S5, the super-resolution reconstruction network uses the BasicVSR++ network.

5. The method for accurate identification of PCB components based on hard sample mining according to claim 1, characterized in that, In step S6, the fine classifier uses the ShuffleNet V2 network.

6. The method for accurate identification of PCB components based on hard sample mining according to claim 1, characterized in that, In step S7, the area where the components are located is selected to cut out the PCB image and obtain a sub-image. The sub-image is then input into the trained coarse classifier for recognition.

Images