PCB defect detection method and device and electronic equipment

Abstract

The invention provides a PCB defect detection method and device and electronic equipment, and the method comprises the steps: carrying out the multi-scale morphological transformation operation of a PCB gray image after collecting and obtaining the PCB gray image of a to-be-detected PCB, and obtaining a transformation image; and performing RGB pseudo color space mapping in combination with the transformed image and the PCB template drawing to generate a fused image. And processing the fused image by using a pre-trained detection model, and outputting defect information of the PCB. According to the scheme, after the multi-scale morphological transformation operation is carried out on the PCB gray level image, the RGB pseudo color space mapping processing is carried out, more comprehensive feature representation is obtained, deep analysis of a detection model is further facilitated, and high-precision recognition and positioning of PCB defects are achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a PCB defect detection method, device, and electronic equipment. Background Art

[0002] As electronic products advance toward miniaturization, high density, and multifunctionality, the manufacturing process for printed circuit boards (PCBs) is becoming increasingly complex, posing unprecedented challenges to the accuracy, real-time performance, and automation of PCB defect detection technologies. Current PCB defect detection methods are primarily divided into two categories: manual visual inspection and automated computer vision inspection. Manual visual inspection, however, is inefficient and highly dependent on human input. Its results are susceptible to subjective operator influence, resulting in high rates of missed and false positives, making it unable to meet the demands of high-speed, high-quality production lines in the modern electronics manufacturing industry.

[0003] Computer vision inspection methods primarily include reference image comparison, rule-driven methods, and machine learning. Reference image comparison methods rely on high-precision image registration and are susceptible to lighting variations and noise, resulting in insufficient robustness in practical applications. Rule-driven methods use predefined detection rules tailored to specific defect types, limiting their flexibility and versatility, making them difficult to adapt to the diverse range of PCB defect types.

[0004] In recent years, deep learning technology has achieved tremendous success in general image recognition, but it has encountered significant bottlenecks in PCB defect detection scenarios. This is primarily due to the limited ability of existing deep learning models to extract subtle features, as PCB defects are often small and hidden. This leads to inadequate feature extraction and insufficient defect recognition performance. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a PCB defect detection method, device and electronic equipment to improve the recognition and positioning accuracy of PCB defects.

[0006] In a first aspect, the present invention provides a PCB defect detection method, the method comprising:

[0007] Acquire a PCB grayscale image of the PCB to be inspected;

[0008] Performing a multi-scale morphological transformation operation on the PCB grayscale image to obtain a transformed image;

[0009] Performing RGB pseudo color space mapping on the transformed image and the PCB template image to generate a fused image;

[0010] The fused image is processed using a pre-trained detection model to output defect information of the PCB board.

[0011] In an optional embodiment, the step of performing a multi-scale morphological transformation operation on the PCB grayscale image to obtain a transformed image includes:

[0012] Performing a plurality of different types of morphological transformation operations on the PCB grayscale image using designed structural elements of different sizes to obtain a plurality of processed images;

[0013] The multiple images are fused according to the set weight coefficients to obtain a transformed image.

[0014] In an optional embodiment, the multiple different types of morphological transformation operations include a closing operation, an opening operation, a morphological gradient operation, and a black hat transformation operation;

[0015] The closing operation is used to fill the open circuit defects on the PCB board, the opening operation is used to remove the short circuit defects on the PCB board, the morphological gradient operation is used to enhance the edge area of the PCB board, and the black hat transformation operation is used to enhance the defects in the dark area of the PCB board.

[0016] In an optional embodiment, the step of performing RGB pseudo color space mapping in combination with the transformed image and the PCB template image to generate a fused image includes:

[0017] Obtaining a pixel value of each pixel point in the transformed image and a pixel value of each pixel point in the PCB template image;

[0018] Setting the brightness value of each color channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the transformed image and the pixel value of each pixel point in the PCB template image;

[0019] A fused image is generated based on the brightness value of each color channel in the RGB color space.

[0020] In an optional embodiment, the step of setting the brightness value of each color channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the transformed image and the pixel value of each pixel point in the PCB template image includes:

[0021] Determine the brightness value of the B channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the PCB template image;

[0022] Determining the brightness value of the G channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel in the transformed image;

[0023] Set the brightness value of the R channel of the RGB color space of the fused image to be generated to the default value.

[0024] In an optional embodiment, the step of processing the fused image using a pre-trained detection model and outputting defect information of the PCB board includes:

[0025] Importing the fused image into the input layer of a pre-trained detection model to obtain an image tensor of the fused image;

[0026] Processing the image tensor through the feature extraction layer of the detection model to output feature information;

[0027] The feature information is processed based on the prediction layer of the detection model to output defect information of the PCB board, where the defect information includes an identification box identifying each defect on the PCB board and a type of each defect.

[0028] In an optional embodiment, the method further includes a step of pre-training to obtain the detection model, which step includes:

[0029] Obtaining a sample image of a PCB board, wherein the sample image includes a real frame identifying defects on the PCB board and a real type of each defect;

[0030] Importing each of the sample images into the constructed deep learning network model, and outputting a predicted box identifying each defect in each of the sample images and a predicted type of each defect;

[0031] A loss function is constructed based on the true box and predicted box of each defect, as well as the true type and predicted type of each defect. The deep learning network model is trained using the loss function as a guide until the preset stopping condition is met, thereby obtaining a trained detection model.

[0032] In an optional embodiment, the step of acquiring a PCB grayscale image of the PCB board to be inspected includes:

[0033] Shoot the PCB to be inspected at multiple angles and under multiple lighting conditions to obtain multiple segmented images;

[0034] performing image preprocessing on each of the segmented images;

[0035] Image registration technology is used to stitch multiple segmented images to obtain a PCB grayscale image.

[0036] In a second aspect, the present invention provides a PCB defect detection device, comprising:

[0037] An acquisition module is used to acquire a PCB grayscale image of the PCB board to be inspected;

[0038] A transformation module, configured to perform a multi-scale morphological transformation operation on the PCB grayscale image to obtain a transformed image;

[0039] A mapping module, configured to perform RGB pseudo color space mapping on the basis of the transformed image and the PCB template image to generate a fused image;

[0040] The detection module is used to process the fused image using a pre-trained detection model and output defect information of the PCB board.

[0041] In a third aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement any of the methods described in the foregoing embodiments.

[0042] The present invention provides a PCB defect detection method, device, and electronic device. After acquiring a PCB grayscale image of a PCB board to be inspected, a multi-scale morphological transformation operation is performed on the PCB grayscale image to obtain a transformed image. RGB pseudo-color space mapping is performed on the transformed image and a PCB template image to generate a fused image. The fused image is processed using a pre-trained detection model, and defect information of the PCB board is output. In this solution, a more comprehensive feature representation is obtained by performing a multi-scale morphological transformation operation on the PCB grayscale image and then performing RGB pseudo-color space mapping processing, which facilitates in-depth analysis of the detection model and achieves high-precision identification and positioning of PCB defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments of the present invention. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 A structural block diagram of an electronic device provided by an embodiment of the present invention;

[0045] Figure 2 A flow chart of a PCB defect detection method provided by an embodiment of the present invention;

[0046] Figure 3 for Figure 2 Flowchart of the sub-steps included in S11;

[0047] Figure 4 for Figure 2 Flowchart of the sub-steps included in S12;

[0048] Figure 5 for Figure 2 Flowchart of the sub-steps included in S13;

[0049] Figure 6 for Figure 5 Flowchart of the sub-steps included in S132;

[0050] Figure 7 A flowchart of a training method in a PCB defect detection method provided by an embodiment of the present invention;

[0051] Figure 8 for Figure 2 Flowchart of the sub-steps included in S14;

[0052] Figure 9 One of the test result schematic diagrams provided in the embodiment of the invention;

[0053] Figure 10 The second schematic diagram of the detection results provided by the embodiment of the present invention;

[0054] Figure 11 The third schematic diagram of the detection results provided by the embodiment of the present invention;

[0055] Figure 12 This is a functional module block diagram of the PCB defect detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

[0057] See also Figure 1 , an electronic device provided in an embodiment of the present invention, and the PCB defect detection method provided in an embodiment of the present invention can be applied to this electronic device. The electronic device includes a memory, a processor, and a communication module. The memory, processor, and communication module components are electrically connected to each other directly or indirectly to enable data transmission or exchange. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0058] Memory is used to store programs or data. Memory can be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), and electrically erasable programmable read-only memory (EEPROM).

[0059] The processor is used to read / write data or programs stored in the memory and execute corresponding functions.

[0060] The communication module is used to establish a communication connection between the electronic device and other communication terminals through the network, and is used to send and receive data through the network.

[0061] It should be understood that Figure 1 The structure shown is only a schematic diagram of the structure of the electronic device. The electronic device may also include Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown. Figure 1 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0062] In some embodiments, the electronic device can be a computer device, a server, etc. in an automatic detection system, which can be communicated with a camera device to obtain image information captured by the camera device and analyze and process the image information to execute the PCB defect detection method provided in an embodiment of the present invention.

[0063] The following, combined Figure 2 The PCB defect detection method provided by the embodiment of the present invention is described. Figure 2 1 is a flow chart of a PCB defect detection method provided by an embodiment of the present invention, the PCB defect detection method comprising the following steps:

[0064] S11, acquiring a PCB grayscale image of the PCB board to be inspected.

[0065] S12, performing a multi-scale morphological transformation operation on the PCB grayscale image to obtain a transformed image.

[0066] S13, performing RGB pseudo color space mapping on the transformed image and the PCB template image to generate a fused image.

[0067] S14: Process the fused image using a pre-trained detection model to output defect information of the PCB board.

[0068] In this embodiment, PCB defect detection can be performed using an automated detection system and a motion control platform. The automated detection system can be developed using Python and run on an Ubuntu operating system. The motion control platform can be, for example, an ACS precision motion control platform. The automated detection system can be driven and operated by the ACS motion control platform.

[0069] The PCB to be inspected is placed at a designated location on the platform and captured using a camera. The captured image is then converted to grayscale for ease of subsequent processing.

[0070] On this basis, a multi-scale morphological transformation is performed on the PCB grayscale image. Multi-scale here refers to the use of structuring elements of various sizes to perform morphological transformation on the PCB grayscale image. This morphological transformation can enhance the characteristics of potential defects on the PCB board in the image, enabling subsequent inspection models to achieve more accurate identification and location.

[0071] Defects on PCB boards mainly include open circuits, short circuits, breaks, protrusions, copper slag, pinholes, depressions and other types of defects.

[0072] To enhance the image's color information and facilitate subsequent processing using a detection model built using a deep learning network, in this embodiment, RGB pseudo-color space mapping is performed on the transformed image and the PCB template image to produce a fused image. Before PCB production, engineers draw a PCB template image and then laser-etch the PCB according to that template image. Therefore, the PCB template image can be understood as PCB production data exported in image format. The resulting fused image contains rich color information in the RGB space.

[0073] On this basis, the fused image is detected and processed through the pre-trained detection model to identify and locate defects on the PCB board and obtain specific defect information.

[0074] The PCB defect detection method provided in this embodiment uses multi-channel image processing technology to perform RGB pseudo-color space mapping on PCB grayscale images, improving the detection capability of sensitive defects and overcoming the limitations of traditional single-channel grayscale image processing methods. It fully utilizes multi-channel image information and multi-scale morphological transformation technology to enhance the representation of defect features. Combined with the powerful feature extraction and classification capabilities of deep learning, it achieves high-precision and high-efficiency detection of PCB defects.

[0075] The following describes in detail how to implement each of the above steps.

[0076] In actual application scenarios, the surface area of the PCB board may be large, while the field of view of the camera device is limited. It may be impossible to fully present the surface information of the PCB board by taking a single image. Figure 3 In this embodiment, the step of acquiring the grayscale image of the PCB to be inspected can be implemented in the following manner:

[0077] S111 , photographing the PCB to be inspected under multiple angles and lighting conditions to obtain multiple segmented images.

[0078] S112: performing image preprocessing on each of the segmented images.

[0079] S113, using image registration technology to stitch the multiple segmented images together to obtain a PCB grayscale image.

[0080] In this embodiment, high-resolution cameras can be used for image capture. The automated inspection system, using precise displacement control, controls the cameras to capture images of a PCB placed in a specified position from multiple angles and under various lighting conditions. The total captured image must ensure complete coverage of the entire PCB surface, preserving rich image information and detailed features.

[0081] In order to improve the image quality, each segmented image collected can be preprocessed first, including geometric correction processing, illumination balancing processing and noise suppression processing.

[0082] The pre-processed segmented images are then stitched together, using image registration technology to form a complete, high-resolution grayscale image of the PCB. The system automatically corrects for displacement errors and uneven lighting during the stitching process, ensuring seamless and consistent stitching.

[0083] Based on the PCB grayscale image obtained by the above preprocessing, morphological transformation is performed on the PCB grayscale image to highlight the specific defect areas in the PCB grayscale image, thereby better identifying defects on the PCB board. Figure 4 In this embodiment, the step of performing a multi-scale morphological transformation operation on the PCB grayscale image can be implemented in the following manner:

[0084] S121 , performing a plurality of different types of morphological transformation operations on the PCB grayscale image using designed structural elements of different sizes to obtain a plurality of processed images.

[0085] S122: performing fusion processing on the multiple images according to the set weight coefficients to obtain a transformed image.

[0086] In this embodiment, for defect features of different sizes, morphological transformation processing can be performed using structural elements of different sizes, for example, including structural elements of different sizes such as 3×3, 5×5, and 7×7. Among them, the structural element is the basic geometric shape unit used to define image morphological transformations (such as corrosion, expansion, etc.). In essence, it is a local window or template with a specific shape (such as rectangle, circle, diamond, etc.) and size. It is equivalent to a "ruler" that interacts with the image (such as sliding pixel by pixel) to detect structural features in the image that match its own shape and scale. For example, smaller structural elements can capture subtle details in the image, while larger structural elements are used to process morphological features in macroscopic areas.

[0087] There are many different types of morphological transformation operations including closing operations, opening operations, morphological gradient operations, and black hat transformation operations.

[0088] Among them, the closing operation is used to fill the open circuit defects on the PCB board, the opening operation is used to remove the short circuit defects on the PCB board, the morphological gradient operation is used to enhance the edge area of the PCB board, and the black hat transformation operation is used to enhance the defects in the dark area of the PCB board.

[0089] Specifically, the closing operation is a process of first performing a dilation operation on the PCB grayscale image and then performing an erosion operation. The original PCB grayscale image is represented as I(x, y), and the used structuring element is represented as B(x, y). The dilation operation performed on the PCB grayscale image can be expressed as follows:

[0090] DILATE(I,B)=max (s,t)∈B {I(xs,yt)}

[0091] The expansion operation can be written as Its function is to expand the target area, fill small holes, and connect disconnected areas. Therefore, it can effectively deal with disconnection defects in PCB grayscale images.

[0092] In addition, the corrosion operation performed on the PCB grayscale image can be expressed as follows:

[0093] ERODE(I,B)=min (s,t)∈B {I(x+s,y+t)}

[0094] The corrosion operation can be written as Its function is to reduce the target area, thereby removing small interference areas. It is very effective in removing small noise and small pseudo defects in PCB grayscale images.

[0095] On this basis, the closing operation is a process of expansion followed by corrosion, which can be expressed as follows:

[0096]

[0097] Performing a closing operation on a PCB grayscale image can fill small holes in the target area of the PCB grayscale image, connect adjacent areas, and smooth the target outline while maintaining its original size. This is primarily used to fill open circuit defects. Therefore, performing a closing operation on a PCB grayscale image can effectively identify defects such as broken circuits and detached pads during PCB defect detection.

[0098] The opening operation is a process of corrosion followed by expansion, which can be expressed as follows:

[0099]

[0100] Performing an opening operation on a PCB grayscale image can separate connected areas, remove small interference objects, and smooth the target contour. This is primarily used to remove short circuit defects. Therefore, performing an opening operation on a PCB grayscale image can effectively identify defects such as wire bridging and solder residue during PCB defect detection.

[0101] In addition, edge detection and feature enhancement can be performed through morphological gradient operations and black hat transformation operations. The morphological gradient operation can be expressed as follows:

[0102]

[0103] The morphological gradient operation can be used to highlight edge regions in PCB grayscale images. This is achieved by subtracting the erosion result from the dilation result. This can effectively detect edge defects in PCB grayscale images, such as trace edge burrs and pad deformation.

[0104] The black hat transformation operation can be expressed as follows:

[0105]

[0106] The black hat transformation operation can be used to highlight defects in dark areas, such as missing copper and insufficient etching. The black hat transformation operation is performed by subtracting the original image from the closed result, which can effectively identify concave defects in PCB boards.

[0107] On the basis of the above, multiple images obtained by different types of morphological transformation operations are subjected to weighted fusion processing, which can be expressed as follows:

[0108] I processed =α1·MORPH_CLOSE(I)+α2·MORPH_OPEN(I)

[0109] +α3·MORPH_GRADIENT(I)+α4·BLACK_HAT(I)

[0110] Among them, α1, α2, α3, and α4 are weight coefficients, and they satisfy

[0111] In this embodiment, multi-scale morphological transformation technology is used to design differentiated processing strategies for PCB defects of different sizes and shapes, enabling the system to capture large-scale structural defects and tiny fine defects at the same time, thereby improving the adaptability and comprehensiveness of detection.

[0112] During the fusion processing, different weight coefficients can be adaptively adjusted according to the specific PCB manufacturing process, board characteristics and common defect types, enabling the system to flexibly respond to the diverse needs of different production lines and further improve the robustness and accuracy of defect detection.

[0113] Furthermore, in order to overcome the limitations of the traditional single-channel grayscale image processing method, in this embodiment, based on the above, the transformed image and the PCB template image are combined to perform RGB pseudo color space mapping to obtain a fused image. Specifically, please refer to Figure 5 , which can be achieved by:

[0114] S131, obtaining the pixel value of each pixel point in the transformed image and the pixel value of each pixel point in the PCB template image.

[0115] S132: Setting the brightness value of each color channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the transformed image and the pixel value of each pixel point in the PCB template image.

[0116] S133: Generate a fused image based on the brightness value of each color channel in the RGB color space.

[0117] In this embodiment, the transformed image is a grayscale image, with each pixel having a value between 0 and 255. To enhance color information, a mapping relationship is established between the pixel values in the grayscale image and the brightness values of each channel in the RGB color space. The brightness values of each color channel at each pixel in the RGB color space determine the final image color of that pixel.

[0118] See also Figure 6 In this embodiment, the brightness value of each color channel of the fused image to be generated is determined by:

[0119] S1321: Determine the brightness value of the B channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the PCB template image.

[0120] S1322: Determine the brightness value of the G channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the transformed image.

[0121] S1323 , setting the brightness value of the R channel of the RGB color space of the fused image to be generated to a default value.

[0122] In this embodiment, each pixel in the transformed image, the PCB template image, and the finally generated fused image is in one-to-one correspondence, and the difference is reflected in the difference in the pixel value or color channel brightness value of the pixel.

[0123] For each pixel in the fused image, the brightness value of the B channel of that pixel is determined by the pixel value of the corresponding pixel in the PCB template image, and the brightness value of the G channel of that pixel is determined by the pixel value of the corresponding pixel in the transformed image. The brightness value of the R channel of that pixel is set to the default value and can be set as needed, with no specific restrictions.

[0124] Thus, the brightness values of the R channel, G channel, and B channel of each pixel in the fused image can be determined, and the final color of the fused image can be determined.

[0125] In this embodiment, by using RGB pseudo color space mapping for PCB grayscale images, the complementarity of multi-channel information is fully exploited, the detection capability of color-sensitive defects is significantly improved, and the limitations of traditional single-channel grayscale image processing methods are overcome.

[0126] On this basis, the pre-trained detection model can be used to process the obtained fused image, and finally obtain the defect information on the PCB board.

[0127] Existing deep learning technologies have performed poorly in PCB defect detection scenarios. This is primarily due to the difficulty in obtaining large amounts of labeled defect sample data. PCB defects are often small and hidden, and existing deep learning models have limited ability to extract subtle features. These models also suffer from poor generalization, making them difficult to handle unknown defect types. They also require significant computing resources and are unable to effectively meet the demands of real-time detection.

[0128] Existing technologies fail to effectively utilize the multi-source information in PCB images, particularly the potential of multi-channel and multi-scale image information, resulting in inadequate feature extraction and insufficient defect recognition performance. The detection method provided in this embodiment leverages the synergistic advantages of color pseudo-space feature enhancement, multi-scale morphological transformation, and deep learning to significantly improve the accuracy and real-time performance of automatic PCB defect detection.

[0129] See also Figure 7 ,The following first introduces the implementation method of the pre-trained detection model.

[0130] S21 , obtaining a sample image of a PCB board, wherein the sample image includes a real frame identifying defects on the PCB and a real type of each defect.

[0131] S22: Import each of the sample images into the constructed deep learning network model, and output a prediction box identifying each defect in each of the sample images and a prediction type of each defect.

[0132] S23, constructing a loss function based on the real box and predicted box of each defect, as well as the real type and predicted type of each defect, and training the deep learning network model with the loss function as a guide until the preset stopping condition is met, thereby obtaining a trained detection model.

[0133] Similarly, a camera is used to capture images of the PCB board, and then a sample image is obtained after pre-processing and stitching. The sample image may include multiple images obtained from different angles and under different lighting conditions.

[0134] The sample image has defect information identified by manual identification or other identification methods, including a real frame that frames the defect and a real type of the defect.

[0135] In this embodiment, each sample image can be processed in the same manner as the morphological transformation processing and RGB pseudo color space mapping processing in the above-mentioned practical application stage. The processing method can be found in the above description and will not be described in detail in this embodiment.

[0136] In addition, in this embodiment, a deep learning network model is pre-built, which includes an input layer, a feature extraction layer, and a prediction layer, and of course may also include some other network layers.

[0137] Sample images containing the true defect bounding boxes and true defect types are fed into the deep learning network model for processing. The deep learning network model identifies and locates PCB defects in the sample images and outputs the predicted bounding boxes and predicted defect types for each defect in the sample images.

[0138] Among them, the input layer can process the input sample image to obtain its image tensor representation, as shown below:

[0139]

[0140] Where H, W, and C represent the image height, width, and number of channels of the sample image, respectively.

[0141] The feature extraction layer can extract features from the image tensor and obtain feature information, which is represented as follows:

[0142] F l =σ(Conv(Fl-1 )+b l )

[0143] Among them, F l is the feature map of the first layer, σ is the activation function (such as ReLU), b l is the bias parameter.

[0144] Finally, the prediction box position regression is performed through the prediction layer of the deep learning network model, which can be expressed as follows:

[0145] B pred ={(x c ,y c ,w,h,p)}

[0146] Among them, (x c ,y c ) is the center coordinate, (w,h) is the width and height, and p is the confidence level.

[0147] While obtaining the predicted boxes representing each defect in the sample image, the predicted type of each defect can also be output.

[0148] In this embodiment, the goal of training the deep learning network model is to ensure that the defect location information and type ultimately output by the model are as consistent as possible with the actual location information and type of the sample image. Therefore, a loss function is constructed based on the actual and predicted bounding boxes of each defect, as well as the actual and predicted types of each defect. This loss function serves as a guide for training the deep learning network model. In this embodiment, the constructed loss function can be as follows:

[0149]

[0150] Among them, λ coord ,λ noobj represents the weighting coefficient, are indicator functions, x i 、y i 、w i 、h i 、C i Respectively represent the real location information, width and height, and communication information, Represent the predicted position information, width and height, and communication information respectively.

[0151] In this embodiment, the loss function comprehensively considers the position error, confidence error and type error, and iteratively updates the model from multiple aspects, so that the final trained detection model has high-precision recognition and positioning functions.

[0152] The above is the implementation method of the pre-trained detection model. In the actual application stage, the obtained detection model can be used to perform defect detection on the PCB board to be inspected. For the obtained PCB grayscale image, after the above-mentioned morphological transformation processing and RGB pseudo color space mapping to obtain a fused image, the fused image is imported into the detection model to perform defect detection. For details, please refer to Figure 8 In this embodiment, this step can be implemented by:

[0153] S141, importing the fused image into the input layer of a pre-trained detection model to obtain an image tensor of the fused image.

[0154] S142: Process the image tensor through the feature extraction layer of the detection model to output feature information.

[0155] S143: Process the feature information based on the prediction layer of the detection model, and output defect information of the PCB board, where the defect information includes identification boxes for identifying each defect on the PCB board and the type of each defect.

[0156] In this embodiment, the processing process of the detection model on the fused image in the actual application stage is similar to the processing process of the sample image by the above-mentioned deep learning network model. For details, please refer to the above, and this embodiment will not be repeated here.

[0157] After the fused image is processed by the detection model, the detection model can output the identification boxes of each defect on the PCB board in the fused image and the type of each defect. In addition, it also carries the confidence level of each recognition result.

[0158] In this embodiment, in order to ensure the accuracy and precision of the final recognition result, the recognition result can also be verified and corrected. For example, the recognition result can be screened based on the confidence level of each recognition result to select the recognition result with a confidence level greater than a set confidence threshold, as shown below:

[0159] D final ={d i |P(d i )>θ confidence}

[0160] Among them, θ confidence is the confidence threshold, d i represents the i-th recognition result, D final Represents a collection of filtered recognition results.

[0161] In addition, there may be overlap between the recognition results on the PCB board, and the existence of overlapping parts affects the identification of defects. Therefore, the IoU calculation method of non-maximum suppression (NMS) can be used to determine the final defect area. For example, it can be characterized as follows:

[0162]

[0163] The final defect area is determined as follows:

[0164]

[0165] Among them, θ nms is the NMS threshold, IoU(r i ,r j ) indicates defect r i and r j The IOU of non-maximum suppression between them.

[0166] Furthermore, in this embodiment, a post-processing morphological verification method may be used to verify the recognition result.

[0167] In the verification process, the adaptive threshold is used as the judgment standard, and the adaptive threshold T adaptive (x,y) is calculated as follows:

[0168] T adaptive (x,y)=μ(x,y)+kσ(x,y)

[0169] Where μ(x,y) is the mean of the local area, σ(x,y) is the standard deviation of the local area, and k is the adjustment coefficient.

[0170] In this embodiment, a cross-validation mechanism is used to compare the detection results of morphological post-processing and deep learning network models. This mechanism eliminates possible false positives and improves the accuracy and reliability of the final detection results. Post-processing verification can achieve effects such as eliminating false positives, optimizing boundaries, and integrating results. Specifically, the reliability of detection results is verified through local statistical features, defect boundaries can be refined using adaptive thresholds, and the results of multiple detection steps are integrated to improve the accuracy of the final detection results.

[0171] In addition, the inspection results can be recorded and a detailed defect report can be generated, including information such as defect type, location, size and confidence level. Figure 9 、 Figure 10 and Figure 111 shows defect information of a PCB board obtained based on the PCB defect detection method provided in this embodiment, including identification boxes for identifying each defect and the type of each defect, such as burr, mousebite, short, copper, pinhole, open, etc.

[0172] Preliminary experimental results demonstrate that the PCB defect detection method employed in this embodiment achieves a detection accuracy of 98.7% and a missed detection rate of less than 1.2% across a variety of complex PCB board defect detection tests. This significantly improves sensitivity to detecting minor defects compared to traditional methods. The processing speed meets the real-time detection requirements of industrial production lines, fully demonstrating the superiority and practical value of combining multi-channel morphological multi-scale transformation preprocessing with deep feature learning for PCB defect detection.

[0173] The PCB defect detection method provided in this embodiment utilizes multi-scale morphological transformation technology to implement differentiated feature enhancement for defects of varying sizes and shapes. This enables the system to capture both large structural defects and tiny, fine defects, improving detection adaptability and comprehensiveness. Furthermore, the enhancement of defect features through morphological preprocessing reduces the subsequent deep learning network model's reliance on a large number of labeled samples, effectively addressing the practical challenge of obtaining PCB defect samples and enhancing the model's generalization and engineering practicality.

[0174] In addition, an RGB pseudo-color space mapping technology is proposed to map a single-channel grayscale image into a three-channel color image to enhance the significance of defect features, fully tap the complementarity of multi-channel information, significantly improve the detection capability of color-sensitive defects, overcome the limitations of traditional single-channel grayscale image processing methods, and break through the information bottleneck of traditional single-channel processing.

[0175] In this embodiment, an innovative multi-channel and multi-scale morphological feature fusion mechanism is designed to intelligently integrate complementary information in different channels and at different scales, providing a richer and more comprehensive feature representation for the deep learning model and significantly improving the detection performance.

[0176] Furthermore, a deep feature learning mechanism is introduced to automatically extract high-level semantic features from preprocessed multi-channel fused images, reducing the subjectivity and limitations of manually designed features and improving the robustness of defect detection in complex backgrounds. By integrating morphological processing with deep learning, the system maintains high detection accuracy while reducing reliance on large-scale annotated samples, addressing the practical challenge of obtaining PCB defect samples and improving the model's generalization and practicality. This enables the detection system to maintain stable detection performance even under non-ideal conditions such as complex backgrounds, varying lighting conditions, and noise interference, significantly enhancing the reliability of industrial field applications.

[0177] In this embodiment, an end-to-end PCB defect detection system is designed to achieve full-process automation from image input to defect location and classification, meeting the high requirements of the modern electronics manufacturing industry for detection efficiency and automation level.

[0178] The method of the present invention is not only suitable for detecting common PCB defect types such as open circuits, short circuits, breaks, protrusions, copper slag, pinholes, etc., but can also effectively identify complex defects and new defects that are difficult to detect with traditional methods, as well as PCB defects under complex backgrounds and undesirable lighting conditions. It has broad engineering application value and significant technological innovation significance.

[0179] Through the above-mentioned innovative design and technology integration, the present invention has achieved a technological breakthrough in the field of PCB defect detection, providing a more efficient, accurate and reliable solution for quality control in the electronics manufacturing industry, with significant engineering application value and broad industrialization prospects.

[0180] In order to perform the corresponding steps in the above-mentioned PCB defect detection method embodiment and various possible methods, a method for implementing a PCB defect detection device is given below. Optionally, the PCB defect detection device can adopt the above-mentioned Figure 1 The device structure of the electronic device shown.

[0181] Further, see Figure 12 , Figure 12 This is a functional module diagram of a PCB defect detection device provided by an embodiment of the present invention. It should be noted that the basic principles and technical effects of the PCB defect detection device provided by this embodiment are the same as those of the corresponding method embodiments described above. For the sake of simplicity, any parts not mentioned in this embodiment can be referred to the corresponding contents of the corresponding method embodiments described above. The PCB defect detection device includes:

[0182] An acquisition module is used to acquire a PCB grayscale image of the PCB board to be inspected;

[0183] A transformation module, configured to perform a multi-scale morphological transformation operation on the PCB grayscale image to obtain a transformed image;

[0184] A mapping module, configured to perform RGB pseudo color space mapping on the basis of the transformed image and the PCB template image to generate a fused image;

[0185] The detection module is used to process the fused image using a pre-trained detection model and output defect information of the PCB board.

[0186] As a possible implementation, the above transformation module may be used to:

[0187] Performing a plurality of different types of morphological transformation operations on the PCB grayscale image using designed structural elements of different sizes to obtain a plurality of processed images;

[0188] The multiple images are fused according to the set weight coefficients to obtain a transformed image.

[0189] As a possible implementation, the multiple different types of morphological transformation operations include a closing operation, an opening operation, a morphological gradient operation, and a black hat transformation operation;

[0190] The closing operation is used to fill the open circuit defects on the PCB board, the opening operation is used to remove the short circuit defects on the PCB board, the morphological gradient operation is used to enhance the edge area of the PCB board, and the black hat transformation operation is used to enhance the defects in the dark area of the PCB board.

[0191] As a possible implementation, the above mapping module can be used to:

[0192] Obtaining a pixel value of each pixel point in the transformed image and a pixel value of each pixel point in the PCB template image;

[0193] Setting the brightness value of each color channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the transformed image and the pixel value of each pixel point in the PCB template image;

[0194] A fused image is generated based on the brightness value of each color channel in the RGB color space.

[0195] As a possible implementation, the above mapping module can be used to:

[0196] Determine the brightness value of the B channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the PCB template image;

[0197] Determining the brightness value of the G channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel in the transformed image;

[0198] Set the brightness value of the R channel of the RGB color space of the fused image to be generated to the default value.

[0199] As a possible implementation, the above detection module can be used to:

[0200] Importing the fused image into the input layer of a pre-trained detection model to obtain an image tensor of the fused image;

[0201] Processing the image tensor through the feature extraction layer of the detection model to output feature information;

[0202] The feature information is processed based on the prediction layer of the detection model to output defect information of the PCB board, where the defect information includes an identification box identifying each defect on the PCB board and a type of each defect.

[0203] As a possible implementation, the PCB defect detection device further includes a training module, which can be used to:

[0204] Obtaining a sample image of a PCB board, wherein the sample image includes a real frame identifying defects on the PCB and a real type of each defect;

[0205] Importing each of the sample images into the constructed deep learning network model, and outputting a predicted box identifying each defect in each of the sample images and a predicted type of each defect;

[0206] A loss function is constructed based on the true box and predicted box of each defect, as well as the true type and predicted type of each defect. The deep learning network model is trained using the loss function as a guide until the preset stopping condition is met, thereby obtaining a trained detection model.

[0207] As a possible implementation, the acquisition module can be used to:

[0208] Shoot the PCB to be inspected at multiple angles and under multiple lighting conditions to obtain multiple segmented images;

[0209] performing image preprocessing on each of the segmented images;

[0210] Image registration technology is used to stitch multiple segmented images to obtain a PCB grayscale image.

[0211] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory shown in FIG. 1 or the operating system (OS) of the electronic device may be fixed and may be Figure 1Meanwhile, the data and program codes required to execute the above modules may be stored in the memory.

[0212] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of the devices, methods, and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the boxes can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified functions or actions, or can be implemented using a combination of dedicated hardware and computer instructions.

[0213] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0214] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0215] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A PCB defect detection method, characterized in that: The method comprises: Acquire a PCB grayscale image of the PCB to be inspected; Performing a multi-scale morphological transformation operation on the PCB grayscale image to obtain a transformed image; Performing RGB pseudo color space mapping on the transformed image and the PCB template image to generate a fused image; The fused image is processed using a pre-trained detection model to output defect information of the PCB board.

2. The PCB defect detection method according to claim 1, characterized in that: The step of performing a multi-scale morphological transformation operation on the PCB grayscale image to obtain a transformed image includes: Performing a plurality of different types of morphological transformation operations on the PCB grayscale image using designed structural elements of different sizes to obtain a plurality of processed images; The multiple images are fused according to the set weight coefficients to obtain a transformed image.

3. The PCB defect detection method according to claim 2, characterized in that: The multiple different types of morphological transformation operations include closing operation, opening operation, morphological gradient operation and black hat transformation operation; The closing operation is used to fill the open circuit defects on the PCB board, the opening operation is used to remove the short circuit defects on the PCB board, the morphological gradient operation is used to enhance the edge area of the PCB board, and the black hat transformation operation is used to enhance the defects in the dark area of the PCB board.

4. The PCB defect detection method according to claim 1, wherein: The step of performing RGB pseudo color space mapping in combination with the transformed image and the PCB template image to generate a fused image includes: Obtaining a pixel value of each pixel point in the transformed image and a pixel value of each pixel point in the PCB template image; Setting the brightness value of each color channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the transformed image and the pixel value of each pixel point in the PCB template image; A fused image is generated based on the brightness value of each color channel in the RGB color space.

5. The PCB defect detection method according to claim 4, characterized in that: The step of setting the brightness value of each color channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the transformed image and the pixel value of each pixel point in the PCB template image includes: Determine the brightness value of the B channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel point in the PCB template image; Determining the brightness value of the G channel in the RGB color space of the fused image to be generated based on the pixel value of each pixel in the transformed image; Set the brightness value of the R channel of the RGB color space of the fused image to be generated to the default value.

6. The PCB defect detection method according to claim 1, characterized in that: The step of processing the fused image using a pre-trained detection model and outputting defect information of the PCB board includes: Importing the fused image into the input layer of a pre-trained detection model to obtain an image tensor of the fused image; Processing the image tensor through the feature extraction layer of the detection model to output feature information; The feature information is processed based on the prediction layer of the detection model to output defect information of the PCB board, where the defect information includes an identification box identifying each defect on the PCB board and a type of each defect.

7. The PCB defect detection method according to claim 1, characterized in that: The method further includes a step of pre-training to obtain the detection model, which step includes: Obtaining a sample image of a PCB board, wherein the sample image includes a real frame identifying defects on the PCB board and a real type of each defect; Importing each of the sample images into the constructed deep learning network model, and outputting a predicted box identifying each defect in each of the sample images and a predicted type of each defect; A loss function is constructed based on the true box and predicted box of each defect, as well as the true type and predicted type of each defect. The deep learning network model is trained using the loss function as a guide until the preset stopping condition is met, thereby obtaining a trained detection model.

8. The PCB defect detection method according to claim 1, wherein: The step of acquiring a PCB grayscale image of the PCB board to be inspected comprises: Shoot the PCB to be inspected at multiple angles and under multiple lighting conditions to obtain multiple segmented images; performing image preprocessing on each of the segmented images; Image registration technology is used to stitch multiple segmented images to obtain a PCB grayscale image.

9. A PCB defect detection device, characterized in that: The device comprises: An acquisition module is used to acquire a PCB grayscale image of the PCB board to be inspected; A transformation module, configured to perform a multi-scale morphological transformation operation on the PCB grayscale image to obtain a transformed image; A mapping module, configured to perform RGB pseudo color space mapping on the basis of the transformed image and the PCB template image to generate a fused image; The detection module is used to process the fused image using a pre-trained detection model and output defect information of the PCB board.

10. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the method according to any one of claims 1 to 8.

Images