A PCB defect detection and identification method based on MAIRNet

Abstract

The present invention discloses a PCB defect detection and identification method based on MAIRNet, including: collecting PCB template images and images to be detected, constructing a data set, and dividing it into test set images and training set images; detecting whether there is a situation of missing components, and finally outputting the location information of the missing components and marking them in the image to be detected; outputting the location of polar components, the type and location information of color ring resistors, and the category and location information of solder joint defects; outputting the cropped image to be detected of the PCB polar components and the template image, constructing a PCB component polarity discrimination method, and outputting the polarity connection of components; summarizing and displaying the acquired information. The present invention can detect and identify common problems such as missing components, incorrect polarity connection of components, and solder joint defects on the surface of PCB, output the location information and category information of the defective area, and can detect and identify the category of color ring resistors, which has significant advantages over manual detection.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision and deep learning, and in particular to a PCB defect detection and identification method based on MAIRNet. Background Art

[0002] With the rapid development of machine vision and artificial intelligence technology, deep learning, neural network and other technologies have gradually been applied to the process of industrial inspection. PCB has the characteristics of high density, light weight and high integration, which also makes the PCB defect detection process face the problems of diversified target components, small size of detection objects and difficult identification.

[0003] The main methods of PCB inspection are currently divided into three categories: manual visual inspection, contact inspection and non-contact inspection. Manual visual inspection is inefficient, and contact inspection is very likely to damage components on the PCB surface. With the development of image processing technology and visual sensors, vision-based PCB inspection has gradually been applied to actual production. However, the commonly used visual inspection method cannot identify and inspect multiple components at the same time, and still requires a relatively complex manual operation process, which has limitations.

[0004] The detection and recognition methods based on traditional image processing technology do not require complex and expensive hardware, but they have certain limitations. The detection and recognition accuracy of complex backgrounds of PCBs with many types of targets is low, and it is impossible to accurately distinguish the types of components and defects. The current target detection and recognition technology based on deep learning does not have a specific neural network model built for PCB defects. The detection and recognition technology for various common PCB defects, such as solder joints and cold solder joints, missing components, and reverse polarity, is relatively immature, making it difficult to perform targeted detection and recognition of multiple types of defects in PCBs. At the same time, due to the complex network structure and the deepening of the number of layers, the deep neural network leads to a complex model and increased computing resources, making it difficult to transplant it to embedded development boards for industrial production detection.

[0005] Therefore, a new technical solution is needed to solve these problems. Summary of the invention

[0006] Purpose of the invention: In order to overcome the shortcomings of the prior art, a PCB defect detection and identification method based on MAIRNet is provided, which uses a detection method based on deep learning and machine vision to replace the traditional detection method. In actual application scenarios, the cost is relatively low and no expensive automatic optical inspection (AOI) system is required. At the same time, compared with manual inspection and contact inspection, the efficiency is greatly improved, and it is not easy to cause damage to components on the PCB surface. The method provided by the present invention is based on the construction of a neural network model, and the model has a low degree of complexity, which is convenient for transplantation to an embedded development board and application in actual industrial production inspection. At the same time, it can realize the detection function of multiple types of defect targets in a single inspection operation, improve production efficiency and reliability, meet the production needs of high-performance and high-complexity products, and can greatly improve the overall detection level of PCB on the production line of smart grid equipment panels, which has important practical application value.

[0007] Technical solution: To achieve the above purpose, the present invention provides a PCB defect detection and identification method based on MAIRNet, comprising the following steps:

[0008] S1: Collect template images and images to be tested on both sides of the PCB respectively, build a data set, and divide it into test set images and training set images. Label the training set images with component categories and defects, and generate corresponding label files.

[0009] S2: Input all the images to be detected and the template images in the training set into the component existence detection module to detect whether there are any missing components, and finally output the location information of the missing components and mark them in the images to be detected;

[0010] S3: Input the PCB images in the test set into the built and trained MAIRNet-based PCB defect detection and recognition model for detection and recognition operations, and output the positioning of polar components, color ring resistor types and positioning information, and solder joint defect types and positioning information;

[0011] S4: using the polarity component positioning situation output in step S3 to crop and screen the image to be detected and the template image, output the cropped PCB polarity component image to be detected and the template image, construct a PCB component polarity discrimination method, use the component existence detection module to operate, by segmenting the polarity component existence area, separately compare the polarity component area of ​​the image to be detected and the template image, input the comparison situation into the polarity detection module for discrimination, and finally output the component polarity connection situation;

[0012] S5: The acquired missing component location information and the marking conditions on the PCB image, component polarity connection conditions, color ring resistor types and location information, and solder joint defect types and location information are summarized and displayed.

[0013] Furthermore, the detection method of the component existence detection module in step S2 is:

[0014] D1: Preprocess the input image to be detected, and use ORB features to perform image registration between the image to be detected and the original image to ensure that the relevant features contained in the image to be detected and the template image are as close as possible;

[0015] D2: Perform image grayscale operation on the image to be detected and the template image after image registration to generate a grayscale image to be detected and a grayscale image of the template;

[0016] D3: Perform image difference between the grayscale image to be detected and the template grayscale image;

[0017] D4: performing image binarization processing on the differential grayscale image obtained in step D3 to obtain a differential binary image, and performing image morphological processing, using corrosion and expansion operations to further reduce the noise and influence of the PCB background environment on the positioning of the missing component area;

[0018] D5: Filter pixels of the differential binary image after the image morphological processing in step D4 to filter out the largest connected area in the image;

[0019] D6: Mark the largest connected area selected in step D5 with a rectangular frame, and map the rectangular frame of the marked component missing area to the input image to be detected, thereby outputting the missing component positioning information and achieving marking.

[0020] Furthermore, the method for building a PCB defect detection and identification model based on MAIRNet in step S3 includes the following steps:

[0021] A1: Build a MAIRNet-based network to extract features of PCB images in the training set and output feature maps of multiple scales. MAIRNet is a multidimensional attention inverse residual network.

[0022] A2: PCB component target detection and recognition are performed on feature maps of various scales. The bounding box location information of the target contains the category information corresponding to the target. The category information includes the specific category of the target component and the defect situation.

[0023] Furthermore, the step A1 specifically includes the following steps:

[0024] B1: MAIRNet consists of an inverse residual network and a multi-dimensional attention module, and uses a multi-scale feature fusion mechanism in the network construction process;

[0025] B2: A multidimensional attention module is added on the basis of the inverse residual network. The multidimensional attention module is used to optimize each inverse residual module in the inverse residual network to obtain an inverse residual module optimized by the multidimensional attention module.

[0026] B3: Build MAIRNet according to the multi-scale feature fusion mechanism.

[0027] Furthermore, the step A2 specifically includes the following steps:

[0028] C1: The 5-layer feature map output by the multi-dimensional attention enhanced neural network is input into the PCB component target detection and recognition module to perform three types of operations, namely polar component positioning, color ring resistor recognition, and solder joint detection. In each type of operation, the 5-layer feature map passes through 4 groups of convolutional neural networks; each group of convolutional neural networks includes a convolution with a convolution kernel of 3×3 and a stride of 1, a group normalization operation, and finally an activation using the ReLU activation function;

[0029] C2: In each type of operation, bounding box regression is performed on the feature maps output by the convolution of the four groups of convolutional neural networks to determine the area where the component target is located, and the generalized intersection over union function GIoU (Generalized Intersection over Union) is used as the bounding box regression loss function;

[0030] C3: In each type of operation, the feature maps output by the convolution of 4 groups of convolutional neural networks are classified to determine the type of component target and the defect category, and the focal loss function FocalLoss is used as the classification loss function.

[0031] Furthermore, the specific calculation process of the generalized intersection-over-union function GIoU in step C2 is:

[0032] L GIoU =1-GIoU (1)

[0033]

[0034]

[0035] In the formula, A represents the predicted bounding box, B represents the true bounding box, C represents the minimum enclosing rectangular area of ​​the predicted bounding box and the true bounding box, and L GIoUrepresents the bounding box regression loss function, IoU represents the intersection over union function between the predicted bounding box and the true bounding box, and GIoU represents the generalized intersection over union function between the predicted bounding box and the true bounding box.

[0036] Furthermore, the specific calculation process of the focal loss function FocalLoss in step C3 is:

[0037] FL(p,y)=-y(1-p) γ log(p)-(1-y)p γ log(1-p) (4)

[0038] In the formula, p represents the predicted value of a certain category label, which is between 0 and 1, y represents the actual category label, which is 0 or 1, and γ represents an artificially set constant.

[0039] Furthermore, the method for determining the polarity of PCB components in step S4 includes the following steps:

[0040] E1: Preprocess the input image of the PCB polarity component to be inspected, and use the ORB feature to perform image registration between the image to be inspected and the original image to ensure that the relevant features contained in the image of the PCB polarity component to be inspected and the template image of the PCB polarity component are as close as possible;

[0041] E2: performing image grayscale operation on the PCB polarity component to be detected image and the PCB polarity component template image obtained after image registration, and generating a grayscale image of the PCB polarity component to be detected and a grayscale image of the PCB polarity component template;

[0042] E3: Perform image difference between the grayscale image of the PCB polarity component to be detected and the grayscale image of the PCB polarity component template;

[0043] E4: performing image binarization processing on the PCB polarity component differential grayscale image obtained in step E3 to obtain a PCB polarity component differential binary image, and performing image morphology processing, using corrosion and dilation operations to further reduce the noise and influence of the PCB background environment on the positioning of the polarity component difference area;

[0044] E5: Pixel screening is performed on the differential binary image after image morphology processing in step E4, and the maximum connected area in the differential binary image of PCB polarity components is screened out;

[0045] E6: Input the largest connected area selected in step E5 into the polarity detection module for further identification, and set the threshold S t , if the maximum connected area is greater than the set threshold S tThis indicates that there may be a polarity error. If the maximum connected area does not exceed the set threshold S t This indicates that the polarity is correct.

[0046] Furthermore, the main body of MAIRNet in step B1 is an inverse residual network constructed by 17 inverse residual modules. The first end of each inverse residual module is a 1×1 convolution, which is used to expand the feature matrix channel and enrich the number of features. The middle part of the inverse residual module is composed of a 3×3 deep convolution kernel, which is used to generate a feature matrix consistent with the number of channels of the input feature matrix, thereby reducing the number of parameters and computing costs. Finally, the ReLU6 activation function is used to enhance the expression ability of the network.

[0047] The specific operation of the multidimensional attention module in step B2 is as follows: the feature matrix of each inverse residual module is divided into two channels for average pooling operation, the feature matrices of the two channels after average pooling are connected and 1×1 convolved respectively, Batch Norm normalization operation is used, and activated by Swish activation function, the activated feature matrix is ​​decomposed into feature matrices of two channels again, and the feature matrices of the two channels are activated by 1×1 convolution and Sigmoid activation function respectively, and the obtained feature matrices of the two channels are merged with the original feature matrix output by the inverse residual module, so as to obtain a feature map enhanced by multidimensional attention, and the enhanced feature map can more effectively express the information in the original image;

[0048] The step B3 is specifically as follows: the inverse residual modules optimized by the 17 multi-dimensional attention modules are divided into a 7-layer network, wherein the first layer to the seventh layer, each layer sequentially comprises: 1, 2, 3, 4, 3, 3, 1 multi-dimensional attention optimized inverse residual modules, and the feature maps processed by the second layer, the third layer, the fifth layer, and the seventh layer are respectively defined as C 2 , C 3 , C 5 , C 7 Perform 1×1 convolution to get F 2 、F 3 、F 5 、F 7 , and F 7 Perform convolution again with a step size of 2 to get F 7 ', fuse the feature maps of different scales, and finally fuse and output 5 layers of feature maps F 2 、F 3 、F 5 、F 7 、F 7 '.

[0049] Furthermore, in step S5, summary display is performed by building a user software system interface.

[0050] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0051] 1. The present invention collects PCB images in industrial production, annotates the images to be inspected and the template images, constructs a data set, collects images through an industrial camera, and performs inspection and recognition after configuring the software operating environment. No expensive inspection equipment and fixtures are required, and damage to PCB components caused by contact inspection is avoided. The cost is low, less manual assistance is required, and the implementation method is quick, convenient, and easy to operate.

[0052] 2. The present invention adopts a neural network based on multi-dimensional attention enhancement for detection, which has a good detection and recognition effect on defects in small targets such as solder joints.

[0053] 3. The present invention can detect and identify multiple types of common PCB defects at one time without complicated manual operation and multiple tests. At the same time, an interactive interface is constructed to output defect detection and identification results. It has the advantages of high recognition accuracy, fast detection speed, simple operation and intuitive display results. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of the method of the present invention;

[0055] Figure 2 It is a structural diagram of the PCB defect detection and identification model based on MAIRNet in the present invention;

[0056] Figure 3 : is a structural diagram of the inverse residual module optimized by the multi-dimensional attention module used in the present invention;

[0057] Figure 4 This is the principle diagram of the generalized intersection-over-union function GIoU;

[0058] Figure 5 This is the result of the detection and identification of common PCB defects based on MAIRNet;

[0059] Figure 6 It is a component missing condition detection diagram output by the component existence detection module in the present invention;

[0060] Figure 7 It is a result diagram of component polarity discrimination output by the polarity detection module in the present invention;

[0061] Figure 8 It is a schematic diagram of the user software system interface constructed by the present invention. DETAILED DESCRIPTION

[0062] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0063] The present invention provides a PCB defect detection and identification method based on a multidimensional attention enhanced neural network (MAIRNet), such as Figure 1 As shown, it includes the following steps:

[0064] S1: Collect the images to be tested and the template images on both sides of the PCB. The images to be tested may have missing components, incorrect polarity of components, wrong connection of color ring resistors, and solder joint defects. Divide the test set images and training set images in a ratio of 3:7, annotate the training set images with component categories and defects, and generate corresponding label files;

[0065] S2: Build a PCB defect detection and identification model based on MAIRNet;

[0066] S3: Input all the images to be detected and the template images in the training set into the component existence detection module to detect whether there are any missing components. Finally, the location information of the missing components is output and marked in the images to be detected.

[0067] S4: Input the PCB images in the test set into the trained MAIRNet-based PCB defect detection and recognition model for detection and recognition operations, and output the positioning of polar components, the types and positioning information of color ring resistors, and the types and positioning information of solder joint defects;

[0068] S5: Use the polarity component positioning output in step S4 to crop and screen the image to be detected and the template image, and output the cropped PCB polarity component image to be detected and the template image. Construct a PCB component polarity discrimination method, use the component existence detection module in S3 to operate, segment the polarity component existence area, compare the polarity component area of ​​the image to be detected and the template image separately, input the comparison situation into the polarity detection module for discrimination, and finally output the component polarity connection situation;

[0069] S6: Build a user software system interface, and display the missing component location information output in step S3 together with the marking on the PCB image, component polarity connection status, color ring resistor type and location information, solder joint defect type and location information in the user software system interface.

[0070] The specific process of step S2 in this embodiment is as follows:

[0071] A1: Build a MAIRNet-based network to extract features of PCB images in the training set and output feature maps of multiple scales. MAIRNet is a multidimensional attention inverse residual network.

[0072] A2: PCB component target detection and recognition are performed on feature maps of various scales. The bounding box location information of the target contains the category information corresponding to the target. The category information includes the specific category and defect status of the target component. The specific structure is as follows: Figure 2 As shown;

[0073] The specific process of step A1 in this embodiment is as follows:

[0074] B1: MAIRNet is composed of an inverse residual network and a multi-dimensional attention module, and a multi-scale feature fusion mechanism is used in the network construction process. The main body of MAIRNet is an inverse residual network constructed by 17 inverse residual modules. The first end of each inverse residual module is a 1×1 convolution, which is used to expand the feature matrix channel and enrich the number of features. The middle part of the inverse residual module is composed of a 3×3 deep convolution kernel, which is used to generate a feature matrix consistent with the number of channels of the input feature matrix, thereby reducing the number of parameters and computing costs. Finally, the ReLU6 activation function is used to enhance the network's expressiveness;

[0075] B2: A multidimensional attention module is added on the basis of the inverse residual network. The multidimensional attention module is used to optimize each inverse residual module in the inverse residual network to obtain an inverse residual module optimized by the multidimensional attention module. The specific operation of the multidimensional attention module is as follows: the feature matrix of each inverse residual module is divided into two channels for average pooling operation. The feature matrices of the two channels after average pooling are connected and 1×1 convolved respectively, and the Batch Norm normalization operation is used. It is activated by the Swish activation function, and the activated feature matrix is ​​decomposed again into feature matrices of two channels. The feature matrices of the two channels are activated by 1×1 convolution and Sigmoid activation function respectively. The feature matrices of the two channels are merged with the original feature matrices output by the inverse residual module to obtain a feature map enhanced by multidimensional attention. The enhanced feature map can more effectively express the information in the original image; the specific structure of the inverse residual module optimized by the multidimensional attention module is as follows Figure 3 As shown;

[0076] B3: MAIRNet is built according to the multi-scale feature fusion mechanism. The inverse residual modules optimized by 17 multi-dimensional attention modules are divided into a 7-layer network, where each layer from the 1st to the 7th layer contains: 1, 2, 3, 4, 3, 3, 1 multi-dimensional attention optimized inverse residual module, and the feature maps processed by the 2nd, 3rd, 5th, and 7th layers of the network are defined as C 2 , C 3 , C 5 , C 7 Perform 1×1 convolution to get F 2 、F 3 、F 5 、F 7 , and F 7 Perform convolution again with a step size of 2 to get F 7 ', fuse the feature maps of different scales, and finally fuse and output 5 layers of feature maps F 2 、F 3 、F 5 、F 7 、F 7 '.

[0077] The specific process of step A2 in this embodiment is as follows:

[0078] C1: The 5-layer feature map output by the multi-dimensional attention enhanced neural network is input into the PCB component target detection and recognition module for three types of operations: polar component positioning, color ring resistor recognition, and solder joint detection. In each type of operation, the 5-layer feature map passes through 4 groups of convolutional neural networks. Each group of convolutional neural networks includes a convolution with a convolution kernel of 3×3 and a stride of 1, a group normalization operation, and finally an activation using the ReLU activation function;

[0079] C2: In each type of operation, bounding box regression is performed on the feature maps output by the convolution of the four groups of convolutional neural networks to determine the area where the component target is located, and the generalized intersection over union function GIoU (Generalized Intersection over Union) is used as the bounding box regression loss function;

[0080] C3: In each type of operation, the feature maps output by the convolution of four groups of convolutional neural networks are classified to determine the type of component target and the defect category, and the focal loss function is used as the classification loss function.

[0081] It should be noted here that the schematic diagram of the GIoU function in step C2 is as follows Figure 4 As shown, specifically:

[0082] L GIoU =1-GIoU (1)

[0083]

[0084]

[0085] In the formula, A represents the predicted bounding box, B represents the true bounding box, C represents the minimum enclosing rectangular area of ​​the predicted bounding box and the true bounding box, and L GIoU represents the bounding box regression loss function, IoU represents the intersection over union function between the predicted bounding box and the true bounding box, and GIoU represents the generalized intersection over union function between the predicted bounding box and the true bounding box.

[0086] It should be noted here that the calculation process of the focal loss function FocalLoss in step C3 is as follows:

[0087] FL(p,y)=-y(1-p) γ log(p)-(1-y)p γ log(1-p) (4)

[0088] In the formula, p represents the predicted value of a certain category label, which is between 0 and 1, y represents the actual category label, which is 0 or 1, and γ represents an artificially set constant.

[0089] The specific process of step S3 in this embodiment is as follows:

[0090] D1: Preprocess the input image to be detected, and use ORB features to perform image registration between the image to be detected and the original image to ensure that the relevant features contained in the image to be detected and the template image are as close as possible;

[0091] D2: Perform image grayscale operation on the image to be detected and the template image after image registration to generate a grayscale image to be detected and a grayscale image of the template;

[0092] D3: Perform image difference between the grayscale image to be detected and the template grayscale image;

[0093] D4: performing image binarization processing on the differential grayscale image obtained in step D3 to obtain a differential binary image, and performing image morphological processing, using corrosion and expansion operations to further reduce the noise and influence of the PCB background environment on the positioning of the missing component area;

[0094] D5: Filter pixels of the differential binary image after the image morphological processing in step D4 to filter out the largest connected area in the image;

[0095] D6: Mark the largest connected area selected in step D5 with a rectangular frame, and map the rectangular frame of the marked component missing area to the input image to be detected, thereby outputting the missing component positioning information and achieving marking.

[0096] The specific process of step S5 in this embodiment is as follows:

[0097] E1: Preprocess the input image of the PCB polarity component to be inspected, and use the ORB feature to perform image registration between the image to be inspected and the original image to ensure that the relevant features contained in the image of the PCB polarity component to be inspected and the template image of the PCB polarity component are as close as possible;

[0098] E2: performing image grayscale operation on the PCB polarity component to be detected image and the PCB polarity component template image obtained after image registration, and generating a grayscale image of the PCB polarity component to be detected and a grayscale image of the PCB polarity component template;

[0099] E3: Perform image difference between the grayscale image of the PCB polarity component to be detected and the grayscale image of the PCB polarity component template;

[0100] E4: performing image binarization processing on the PCB polarity component differential grayscale image obtained in step F3 to obtain a PCB polarity component differential binary image, and performing image morphology processing, using corrosion and dilation operations to further reduce the noise and influence of the PCB background environment on the positioning of the polarity component difference area;

[0101] E5: Filter pixels of the differential binary image after image morphology processing in step F4 to filter out the largest connected area in the differential binary image of PCB polarity components;

[0102] E6: Input the largest connected area selected in step F5 into the polarity detection module for further identification, and set the threshold S t , if the maximum connected area is greater than the set threshold S t This indicates that there may be a polarity error. If the maximum connected area does not exceed the set threshold S t This indicates that the polarity is correct.

[0103] In order to verify the actual effect of the method of the present invention, the method of the present invention was actually applied to the detection, wherein the actual detection and identification of the resistor and the solder joint based on the MAIRNet PCB defect detection and identification model are as follows: Figure 5 As shown in (a) and (b), it can be seen that Figure 5 (a) and (b) can clearly and efficiently identify the type of resistor target and the area where the solder joint has defects; the results of the component existence detection module for common component detection are as follows: Figure 6 As shown in FIG. 1 , it can be seen that the component presence detection module can accurately locate the area where the components are missing; the judgment result of the polarity component connection is as follows: Figure 7As shown, the final judgment result is output on the user software system interface. It can be seen that the polarity detection module can complete the judgment of the connection status of common polarity components. Figure 5 , Figure 6 , Figure 7 The detection and recognition results verify the practical effect of the method of the present invention. The user software system interface is as follows: Figure 8 shown.

[0104] It can be seen that the present invention can detect and identify common problems such as missing components, incorrect component polarity connection, and solder joint defects on the PCB surface, output the location information and category information of the defective area, and can detect and identify the color ring resistor category. The detection accuracy is as high as 95%, and the average detection and identification speed of a single image does not exceed 2 seconds, which has significant advantages over manual detection methods.

Claims

1. A PCB defect detection and identification method based on MAIRNet, It is characterized in that The steps include: S1: Collect template images and images to be tested on both sides of the PCB respectively, build a data set, and divide it into test set images and training set images. Label the training set images with component categories and defects, and generate corresponding label files. S2: Input all the images to be detected and the template images in the training set into the component existence detection module to detect whether there are any missing components, and finally output the location information of the missing components and mark them in the images to be detected; S3: Input the PCB images in the test set into the built and trained MAIRNet-based PCB defect detection and recognition model for detection and recognition operations, and output the positioning of polar components, color ring resistor types and positioning information, and solder joint defect types and positioning information; S4: using the polarity component positioning situation output in step S3 to crop and screen the image to be detected and the template image, output the cropped PCB polarity component image to be detected and the template image, construct a PCB component polarity discrimination method, use the component existence detection module to operate, by segmenting the polarity component existence area, separately compare the polarity component area of ​​the image to be detected and the template image, input the comparison situation into the polarity detection module for discrimination, and finally output the component polarity connection situation; S5: The acquired missing component location information and the marking conditions on the PCB image, component polarity connection conditions, color ring resistor types and location information, solder joint defect types and location information are summarized and displayed; The method for building a PCB defect detection and identification model based on MAIRNet in step S3 includes the following steps: A1: Build a MAIRNet-based framework to extract features of PCB images in the training set and output feature maps of various scales; A2: PCB component target detection and recognition are performed on feature maps of various scales. The bounding box location information of the target contains the category information corresponding to the target. The category information includes the specific category of the target component and the defect situation. The step A1 specifically includes the following steps: B1: MAIRNet consists of an inverse residual network and a multi-dimensional attention module, and uses a multi-scale feature fusion mechanism in the network construction process; B2: A multidimensional attention module is added on the basis of the inverse residual network. The multidimensional attention module is used to optimize each inverse residual module in the inverse residual network to obtain an inverse residual module optimized by the multidimensional attention module. B3: Build MAIRNet according to the multi-scale feature fusion mechanism; The step A2 specifically includes the following steps: C1: The 5-layer feature map output by the multi-dimensional attention enhanced neural network is input into the PCB component target detection and recognition module to perform three types of operations, namely polar component positioning, color ring resistor recognition, and solder joint detection. In each type of operation, the 5-layer feature map passes through 4 groups of convolutional neural networks; each group of convolutional neural networks includes a convolution with a convolution kernel of 3×3 and a stride of 1, a group normalization operation, and finally an activation using the ReLU activation function; C2: In each type of operation, bounding box regression is performed on the feature map output by the convolution of the four groups of convolutional neural networks to determine the area where the component target is located, and the generalized intersection-over-union function GIoU is used as the bounding box regression loss function; C3: In each type of operation, the feature maps output by the convolution of four groups of convolutional neural networks are classified to determine the type of component target and the defect category, and the focal loss function is used as the classification loss function.

2. According to the PCB defect detection and identification method based on MAIRNet according to claim 1, It is characterized in that The detection method of the component existence detection module in step S2 is: D1: Preprocess the input image to be detected and use ORB features to perform image registration between the image to be detected and the original image; D2: Perform image grayscale operation on the image to be detected and the template image after image registration to generate a grayscale image to be detected and a grayscale image of the template; D3: Perform image difference between the grayscale image to be detected and the template grayscale image; D4: performing image binarization processing on the differential grayscale image obtained in step D3 to obtain a differential binary image, and performing image morphological processing, using corrosion and expansion operations to further reduce the noise and influence of the PCB background environment on the positioning of the missing component area; D5: Filter pixels of the differential binary image after the image morphological processing in step D4 to filter out the largest connected area in the image; D6: Mark the largest connected area selected in step D5 with a rectangular frame, and map the rectangular frame of the marked component missing area to the input image to be detected, thereby outputting the missing component positioning information and achieving marking.

3. According to the PCB defect detection and identification method based on MAIRNet according to claim 1, It is characterized in that The specific calculation process of the generalized intersection-over-union function GIoU in step C2 is: L GIoU =1-GIoU (1) In the formula, A represents the predicted bounding box, B represents the true bounding box, C represents the minimum enclosing rectangular area of ​​the predicted bounding box and the true bounding box, and L GIoU represents the bounding box regression loss function, IoU represents the intersection over union function between the predicted bounding box and the true bounding box, and GIoU represents the generalized intersection over union function between the predicted bounding box and the true bounding box.

4. According to the MAIRNet-based PCB defect detection and identification method of claim 1, It is characterized in that The specific calculation process of the focal loss function in step C3 is: FL(p,y)=-y(1-p) γ log(p)-(1-y)p γ log(1-p) (4) In the formula, p represents the predicted value of a certain category label, which is between 0 and 1, y represents the actual category label, which is 0 or 1, and γ represents an artificially set constant.

5. According to the MAIRNet-based PCB defect detection and identification method of claim 1, It is characterized in that The method for determining the polarity of PCB components in step S4 comprises the following steps: E1: Preprocess the input image of the PCB polarity components to be inspected, and use the ORB feature to perform image registration between the image to be inspected and the original image; E2: performing image grayscale operation on the PCB polarity component to be detected image and the PCB polarity component template image obtained after image registration, and generating a grayscale image of the PCB polarity component to be detected and a grayscale image of the PCB polarity component template; E3: Perform image difference between the grayscale image of the PCB polarity component to be detected and the grayscale image of the PCB polarity component template; E4: performing image binarization processing on the PCB polarity component differential grayscale image obtained in step E3 to obtain a PCB polarity component differential binary image, and performing image morphology processing, using corrosion and dilation operations to further reduce the noise and influence of the PCB background environment on the positioning of the polarity component difference area; E5: Pixel screening is performed on the differential binary image after image morphology processing in step E4, and the maximum connected area in the differential binary image of PCB polarity components is screened out; E6: Input the largest connected area selected in step E5 into the polarity detection module for further identification, and set the threshold S t , if the maximum connected area is greater than the set threshold S t This indicates that there may be a polarity error. If the maximum connected area does not exceed the set threshold S t This indicates that the polarity is correct.

6. According to the MAIRNet-based PCB defect detection and identification method of claim 1, It is characterized in that The main body of MAIRNet in step B1 is an inverse residual network constructed by 17 inverse residual modules. The first end of each inverse residual module is a 1×1 convolution, which is used to expand the feature matrix channel and enrich the number of features. The middle part of the inverse residual module is composed of a 3×3 deep convolution kernel, which is used to generate a feature matrix consistent with the number of channels of the input feature matrix, thereby reducing the number of parameters and computing costs. Finally, the ReLU6 activation function is used to enhance the expression ability of the network; The specific operation of the multidimensional attention module in step B2 is as follows: the feature matrix of each inverse residual module is divided into two channels for average pooling operation, the feature matrices of the two channels after average pooling are respectively connected and 1×1 convolved, Batch Norm normalization operation is used, and activated by Swish activation function, the activated feature matrix is ​​decomposed into feature matrices of two channels again, and the feature matrices of the two channels are respectively activated by 1×1 convolution and Sigmoid activation function, and the obtained feature matrices of the two channels are merged with the original feature matrix output by the inverse residual module, so as to obtain a feature map enhanced by multidimensional attention; The step B3 is specifically as follows: the inverse residual modules optimized by the 17 multi-dimensional attention modules are divided into a 7-layer network, wherein the first layer to the seventh layer, each layer sequentially comprises: 1, 2, 3, 4, 3, 3, 1 multi-dimensional attention optimized inverse residual modules, and the feature maps processed by the second layer, the third layer, the fifth layer, and the seventh layer are respectively defined as C 2 , C 3 , C 5 , C 7 Perform 1×1 convolution to get F 2 、F 3 、F 5 、F 7 , and F 7 Perform convolution again with a step size of 2 to get F 7 ', fuse the feature maps of different scales, and finally fuse and output 5 layers of feature maps F 2 、F 3 、F 5 、F 7 、F 7 '.

7. According to the MAIRNet-based PCB defect detection and identification method of claim 1, It is characterized in that In the step S5, a summary display is performed by building a user software system interface.

Images