Big data-based circuit board defect intelligent detection method

Abstract

The invention provides a circuit board defect intelligent detection method based on big data, and belongs to the technical field of circuit board detection and classification, and the method comprises the following steps: carrying out defect detection on a PCB by using visible light, infrared light and ultraviolet light, and carrying out data set preprocessing to obtain a preprocessed data set; inputting the obtained preprocessed data set into a feature extraction network, and performing feature extraction on each image to obtain a first feature map; inputting the obtained first feature map into a multi-modal feature fusion module to obtain a second feature map after multi-modal fusion; inputting the second feature map into a multi-scale feature fusion module to obtain a third feature map after multi-scale fusion; inputting the third feature map into a prediction head to obtain a prediction map; the method can solve the problems that the dependence degree on visible light is high, and tiny defects are difficult to discover by purely depending on visible light detection at present.

Description

Technical Field

[0001] The present invention belongs to the technical field of circuit board detection and classification. Specifically, it relates to an intelligent detection method for circuit board defects based on big data. Background Art

[0002] With the rapid development of the electronics industry, the electronics manufacturing industry has become a key component of modern manufacturing. The printed circuit board (PCB) is an information carrier integrating various electronic components and has a wide range of applications in the electronics field. Its quality directly affects the performance of products. In the process of circuit board manufacturing, surface mount technology is generally used for component installation on the PCB. With the advent of the 5G era, the development of electronic technology and the electronics manufacturing industry, electronic products tend to be lighter, smaller, and thinner. To ensure the performance of electronic products, circuit board defect detection technology has become a very crucial technology in the electronics industry. How to accurately, quickly, and precisely detect minute defects such as missing holes, rat bites, open circuits, and short circuits during the PCB manufacturing process has become an industry problem.

[0003] Problem 1: During conventional probe detection, it is usually necessary to test each connection point on the circuit board one by one. Although it can detect finely, it usually takes a lot of time and effort. With the increasing complexity of circuit board design, conventional probes are difficult to flexibly adapt to different types of circuit boards. During the detection process, conventional probes will be mechanically worn, resulting in damage to the probe head. And the damaged probes cannot perform effective detection, and it is necessary to frequently replace the probes, increasing the maintenance cost.

[0004] Problem 2: Traditional circuit board fault detection generally takes photos under visible light. Due to the low contrast of visible light, only obvious defects can be detected, and comprehensive and detailed detection cannot be achieved. The gloss or reflection characteristics of the circuit board surface cause high-light reflection when taking photos with visible light, interfering with the identification of defects. It is very difficult to detect small physical defects (such as minor soldering quality problems) simply relying on visible light detection. When taking photos, electronic components, solder joints, or other structures will block part of the circuit board, so the requirements and dependence on visible light are relatively high.

[0005] Problem 3: Currently, the use of artificial intelligence for image recognition and processing has too low accuracy for circuit boards. When distinguishing minute defects such as missing holes, rat bites, open circuits, and short circuits, multiple consecutive defects are easily recognized as one defect; at the same time, it is impossible to accurately distinguish between short circuits and open circuits and between minor solder joint defects and normal solder joints.

[0006] Therefore, the commonly used detection methods still need to be improved. Summary of the Invention

[0007] To solve the above problems, the present invention proposes an intelligent detection method for circuit board defects based on big data, which can solve the problems that visible light cannot achieve detailed and comprehensive detection and the detection equipment has a high dependence on visible light in conventional detection methods; at the same time, it solves the problems that ordinary probes take a lot of time for fine detection and are difficult to adapt to complex circuit boards.

[0008] The present invention is specifically implemented as follows: The present invention provides an intelligent detection method for circuit board defects based on big data, which includes the following steps: Step S1: Use visible light, infrared light, and ultraviolet light to irradiate the front and back sides of the PCB in sequence, obtain defect detection data images and perform preprocessing on the data set to obtain a preprocessed data set; wherein the preprocessed data set includes image data sets of small defects such as missing holes, rat bites, open circuits, and short circuits; Step S2: Input the preprocessed data set obtained through Step S1 into a feature extraction network, extract features from each image, and obtain a first feature map; Step S3: Input the first feature map obtained through Step S2 into a multi-modal feature fusion module to obtain a second feature map after multi-modal fusion; Step S4: Input the second feature map obtained through Step S3 into a multi-scale feature fusion module to obtain a third feature map after multi-scale fusion; Step S5: Input the third feature map obtained through Step S4 into a prediction head to obtain a prediction map; Step S6: Post-process the prediction map obtained through Step S5 to obtain a prediction box; Step S7: Use the trained network to perform defect detection on the prediction box obtained through Step S6 and output the result; Step S8: Classify and label the detected circuit board according to the output result.

[0009] The beneficial effects of adopting the above solution are as follows: The circuit board is irradiated with visible light and photographed in this state to restore the original state of the circuit board to be detected, which is used as the subsequent comparison basis; the circuit board is irradiated with infrared light and photographed in this state to remove the interference on the surface of the circuit board and reduce the influence of other non-defective substances on the subsequent detection of the circuit board; the circuit board is irradiated with ultraviolet light and photographed in this state to achieve focused photography of fine points and effectively detect defects that cannot be observed under other light sources; the PCB in the same state is irradiated with visible light, infrared light, and ultraviolet light, and the obtained images are combined. By combining the advantages of various lights, it can effectively avoid the influence of the non-defective state of the PCB on the subsequent analysis, and at the same time can accurately classify and determine various defects.

[0010] On the basis of the above technical solution, an intelligent detection method for circuit board defects based on big data according to the present invention can be further improved as follows: Further, the multi-modal feature fusion module includes a global feature fusion module branch and a local feature fusion module branch; the first feature map is processed by the global feature fusion module branch and the local feature fusion module branch and then spliced, and the spliced data is subjected to convolution processing to obtain the second feature map.

[0011] Further, the processing process of the global feature fusion module branch includes: Step S3-1-1: The optical image data obtains a visible light imaging picture feature map, an ultraviolet imaging picture feature map, and an infrared imaging picture feature map through Step S1 and Step S2; Step S3-1-2: The three feature maps obtained in Step S3-1-1 are input into the double cross-attention fusion module in pairs and output; Step S3-1-3: The data output in Step S3-1-2 is subjected to splicing processing and output; Step S3-1-4: The data obtained in Step S3-1-3 is subjected to convolution operation processing and output.

[0012] Further, the processing process of the local feature fusion module branch includes: Step S3-2-1: The optical image data obtains a visible light imaging picture feature map, an ultraviolet imaging picture feature map, and an infrared imaging picture feature map through Step S1 and Step S2; Step S3-2-2: The three feature maps obtained in Step S3-2-1 are respectively subjected to convolution operation processing and output; Step S3-2-3: The data obtained in Step S3-2-2 is subjected to splicing processing and output; Step S3-2-4: Input the data obtained in Step S3-2-3 into the coordinate attention module and output it. Step S3-2-5: Perform convolution operation on the data obtained in Step S3-2-3 and output it.

[0013] The beneficial effects of adopting the above further solution are as follows: Through integrating the global features of different modalities, global feature fusion can capture the common information and mutual relationships among modalities, thereby improving the overall performance of the model. The features of local regions can help make up for the missing information in other regions, thus enhancing the overall performance; at the same time, local feature fusion enables the model to share features when processing multiple tasks, improving efficiency.

[0014] Further, the processing process of the multi-scale feature fusion module includes: Step S4-1: Respectively perform transposed convolution with a stride of 2 and upsampling on multiple second feature maps synchronously, and output the data. Step S4-2: Add the data obtained by transposed convolution and the data obtained by upsampling in Step S4-1 and output it. Step S4-3: Perform convolution operation on the data obtained in Step S4-2 and output it. Step S4-4: Concatenate the data obtained in Step S4-3 with the high-scale feature map data and output it. Step S4-5: Perform convolution operation on the data obtained in Step S4-4 again and output it to obtain the third feature map.

[0015] Further, the preprocessing operations in Step S1 include: data cleaning, image enhancement, data augmentation, and data normalization.

[0016] Further, the postprocessing operations in Step S6 include: box screening, box adjustment, class assignment, and non-maximum suppression.

[0017] Compared with the prior art, the beneficial effects of an intelligent detection method for circuit board defects based on big data provided by the present invention are as follows: Through the multi-modal feature fusion module, the features of three types of images can be effectively fused, enabling the features of the three images to enhance and complement each other, so that the network can effectively capture important defect information in the circuit board image. For example, the tip of the circuit board solder joint has different images under various light source conditions. Using this principle, fixed-point detection of the solder joint is performed, and whether there are minor defects such as solder missing is determined based on each image. In addition, the multi-module and multi-scale cooperation proposed by the present invention and their cross-use can effectively fuse the multi-scale feature representations of multiple modules, which helps to capture defects of different sizes and shapes in the circuit board image and has broad application prospects in practical applications, and can provide a more efficient and accurate quality control solution for the circuit board manufacturing industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is the overall method flow chart; Figure 2 is the overall network flow chart; Figure 3 is the multi-modal feature fusion module structure diagram; Figure 4 is the multi-scale feature fusion module structure diagram; Figure 5 is the PCB input diagram; Figure 6 is the system output diagram. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0020] The present invention provides an intelligent detection method for circuit board defects based on big data, as Figure 1-4 shown. The specific process of the method includes the following steps: I. Use the light source switching control system to cooperate with the CCD and CMOS sensors to collect defect detection data of the PCB, and organize the data set , and then preprocess the data set to obtain the processed data set .

[0021] The above light source switching system is connected to multiple light source devices, and the switching between different light sources is carried out through a light source controller and a computer program, and the PCB is irradiated by reciprocally using visible light, infrared light and ultraviolet light. Among them, the visible light is a conventional light source, the infrared light uses a light source with a wavelength between 700um and 1350um, and the ultraviolet light uses a light source with a wavelength between 340nm and 400nm.

[0022] The following embodiments are described for the above imaging process: 1: Single camera three-light source mode An Andor Zyla series camera is used to take pictures of the circuit board to be detected with a single camera. A light source switching device is rotatably installed beside the camera. The light source switching device irradiates the circuit board to be detected in the order of visible light, infrared light and ultraviolet light, and at the same time the camera takes pictures of the circuit board and uploads the data. In this photographing mode, the circuit board needs to stay directly below the camera to ensure that it moves after the three light source replacements are completed.

[0023] 2: Multi-camera multi-light source mode The circuit board to be detected passes through a normal camera equipped with a visible light source, an infrared imaging camera equipped with an infrared light source, and an ultraviolet imaging camera equipped with an ultraviolet light source in sequence, and takes pictures of the circuit board and uploads the data in sequence. In this photographing mode, the circuit board does not need to stay and can directly pass through the camera, and the camera will capture the circuit board at the corresponding position.

[0024] Among them, v is the PCB image data obtained in the visible light state, u is the PCB image data obtained in the infrared light state, and r is the PCB image data obtained in the ultraviolet light state.

[0025] Each data set group may have various micro defects such as missing holes, mouse bites, open circuits, and short circuits.

[0026] The above preprocessing operations include: Using filtering technology to remove random noise in the acquired images; eliminating duplicate and low-quality images to avoid negative impacts on model training; Adjusting the brightness of the images to ensure the comparability of data under different lighting conditions; adjusting the color of the images to ensure their true colors; Normalizing the pixel values of the images by mean and variance to reduce the sensitivity of the model to different lighting and contrast.

[0027] Second, any group in the preprocessed image data set I , that is, the image data containing defects such as PCB missing holes, mouse bites, open circuits, and short circuits obtained under visible light, infrared light, and ultraviolet light states are respectively input into the Swin-T feature extraction network to obtain the first feature maps of different scales for the three images , , , , , , , and .

[0028] Among them, represents the PCB feature map under visible light, represents the PCB feature map under infrared light, represents the PCB feature map under ultraviolet light; the first feature map numbers are the same at the same scale.

[0029] Third, input the first feature maps of the three images of each scale into the multi-modal feature fusion module to obtain the second feature maps after multi-modal fusion , and .

[0030] Perform feature integration processing on the first feature maps obtained when the PCB is in visible light, infrared light, and ultraviolet light states at the same scale.

[0031] Fourth, input the second feature maps , and into the multi-scale feature fusion module to obtain the third feature maps after multi-scale fusion and .

[0032] Perform scale integration on multiple PCB second feature maps for subsequent observation.

[0033] Fifth, input the third feature maps and containing PCB defect data into the prediction head to obtain the prediction maps and .

[0034] Sixth, post-process the prediction maps and obtained by processing through the prediction head to obtain the prediction boxes.

[0035] The post-processing steps of the prediction maps aim to improve the accuracy and usability of the object detection results.

[0036] Extract high-quality target bounding boxes from the original output of the model through threshold setting, non-maximum suppression, bounding box adjustment, and class assignment.

[0037] VII. Train the original network using reliable data, and repeatedly monitor the reliable data during the training process to determine the training degree of the original network by checking the coincidence degree between the unmonitored data and the reliable data. Use the trained network to perform defect detection on the above-mentioned target bounding boxes and output the detection data.

[0038] Among them, the reliable data includes PCB image data with clearly identifiable defects such as missing holes, mouse bites, open circuits, excessive copper, missing copper, short circuits, and open circuits. Train the original network using the reliable data, and optimize the original network by comparing the output data with the original data.

[0039] VIII. Perform defect identification and classification on the circuit board according to the output data.

[0040] Optionally, in the above technical solution, Figure 2 , the multi-modal feature fusion module includes a global feature fusion module branch and a local feature fusion module branch; the first feature maps of PCBs with defects such as missing holes, mouse bites, open circuits, and short circuits under different light source states are respectively processed by the global feature fusion module branch and the local feature fusion module branch and then spliced, and the spliced data is subjected to 3X3 convolution processing to obtain the aforementioned second feature map. Among them, the global feature part includes the overall image state of the circuit board, mainly for obvious defects, determining the overall detection positions, and marking the positions; the local feature part includes the image states of each small position of the circuit board, mainly for detailed processing of each solder joint and notch position.

[0041] Optionally, in the above technical solution, the processing process of the global feature fusion module branch includes: 3-1-1. The optical image data of PCBs with various defects are preprocessed and feature-extracted to obtain the corresponding visible light imaging picture feature map, ultraviolet imaging picture feature map, and infrared imaging picture feature map of the PCB; 3-1-2. Input the three feature maps obtained in step 3-1-1 in pairs into the double-cross attention fusion module and output; 3-1-3. Splice and output the data output in step 3-1-2; 3-1-4. Perform 3X3 convolution operation processing on the data obtained in step 3-1-3 and output.

[0042] Optionally, in the above technical solution, the processing process of the local feature fusion module branch includes: 3-2-1. The optical image data are processed through step S1 and step S2 to obtain the visible light imaging picture feature map, the ultraviolet imaging picture feature map, and the infrared imaging picture feature map; 3-2-2. Perform 3X3 convolution operation processing on the three feature maps obtained in step S3-2-1 respectively and output; 3-2-3. Perform splicing processing on the data obtained in step S3-2-2 and output; 3-2-4. Input the data obtained in step S3-2-3 into the coordinate attention module and output; 3-2-5. Perform 3X3 convolution operation processing on the data obtained in step S3-2-3 and output.

[0043] Optionally, in the above technical solution, as Figure 4 , the processing process of the multi-scale feature fusion module includes: 4-1. Perform 4X4 deconvolution with a stride of 2 and upsampling on multiple second feature maps respectively, and output the data; 4-2. Add the data obtained by deconvolution in step S4-1 to the data obtained by upsampling and output; 4-3. Perform 3X3 convolution operation processing on the data obtained in step S4-2 and output; 4-4. Splice the data obtained in step S4-3 with the high-scale feature map data and output; 4-5. Perform 1X1 convolution operation processing on the data obtained in step S4-4 again and output to obtain the third feature map.

[0044] As shown in Table 1 below, a total of six circuit board fault modes are selected for the experiment, including missing holes, rat bites, open circuits, excessive copper, missing copper, and short circuits. 50 different circuit boards are selected for each mode for detection and comparison.

[0045] As Figure 5 shown, it is the display of the picture after post-processing taken in the single-camera three-light source mode under the input state of the system. As Figure 6 shown, it is the display of the picture after analysis and processing by the system network trained by this method in the output state of the system.

[0046] It can be seen from the observed and compared data that the traditional circuit board detection method is lower than this method in terms of accuracy, precision, and recall rate.

[0047] This method adopts a circuit board defect detection method that integrates infrared thermal imaging, ultraviolet imaging, and visible light imaging technologies. By integrating the characteristics of these three images through a multi-modal network, it significantly reduces the dependence on external lighting conditions and enhances the adaptability to complex surfaces. This method has significant advantages in circuit board defect detection, including improving accuracy, reliability, and robustness. Through algorithm improvement, the accuracy, recall rate, and mean average precision of the detection algorithm of the method are 99.5%, 99.6%, and 99.3% respectively. The multi-modal + multi-scale cooperation network can fully explore the complementarity between the three imaging technologies of infrared thermal imaging, ultraviolet imaging, and visible light imaging technologies, improving the overall detection effect.

[0048] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A circuit board defect intelligent detection method based on big data, characterized in that: The following steps are involved: Step S1: Use visible light, infrared light, and ultraviolet light to irradiate the front and back sides of the PCB in sequence, obtain defect detection data images, and perform data set preprocessing to obtain a preprocessed data set; wherein the preprocessed data set includes an image data set of micro defects such as missing holes, mouse bites, open circuits, and short circuits; Step S2: inputting the preprocessed data set obtained in step S1 into a feature extraction network, performing feature extraction on each image, and obtaining a first feature map; Step S3: inputting the first feature map of the defective PCB obtained in step S2 into a multimodal feature fusion module to obtain a second feature map after multimodal fusion; Step S4: inputting the second feature map of the defective PCB obtained in step S3 into a multi-scale feature fusion module to obtain a third feature map after multi-scale fusion; Step S5: inputting the third feature map obtained in step S4 into the prediction head to obtain a prediction map; Step S6: post-processing the prediction image obtained in step S5 to obtain a prediction frame; Step S7: using the trained network to perform PCB corresponding defect detection on the prediction frame obtained in step S6 and output the result; Step S8: Identify and classify the detected PCB according to the output results.

2. According to the big data-based intelligent circuit board defect detection method of claim 1, it is characterized in that: The multimodal feature fusion module includes a global feature fusion module branch and a local feature fusion module branch; the first feature map is spliced ​​after being processed by the global feature fusion module branch and the local feature fusion module branch, and the spliced ​​data is convolved to obtain the second feature map.

3. The method for intelligent detection of circuit board defects based on big data according to claim 2 is characterized in that: The global feature fusion module branch processing process includes: Step S3-1-1: The optical image data is subjected to step S1 and step S2 to obtain a visible light imaging picture feature map, an ultraviolet imaging picture feature map, and an infrared imaging picture feature map; Step S3-1-2: input the three feature maps obtained in step S3-1-1 into the double cross attention fusion module in pairs and output them; Step S3-1-3: splicing and outputting the data outputted in step S3-1-2; Step S3-1-4: Perform convolution operation on the data obtained in step S3-1-3 and output it.

4. The method for intelligent detection of circuit board defects based on big data according to claim 2 is characterized in that: The local feature fusion module branch processing process includes: Step S3-2-1: The optical image data is subjected to step S1 and step S2 to obtain a visible light imaging picture feature map, an ultraviolet imaging picture feature map, and an infrared imaging picture feature map; Step S3-2-2: performing convolution operations on the three feature maps obtained in step S3-2-1 respectively and outputting them; Step S3-2-3: splicing and outputting the data obtained in step S3-2-2; Step S3-2-4: input the data obtained in step S3-2-3 into the coordinate attention module and output it; Step S3-2-5: Perform convolution operation on the data obtained in step S3-2-3 and output it.

5. The method for intelligent detection of circuit board defects based on big data according to claim 2 is characterized in that: The multi-scale feature fusion module processing process includes: Step S4-1: synchronously perform deconvolution and upsampling processing with a step size of 2 on the plurality of the second feature maps, and output the data; Step S4-2: adding the data obtained by deconvolution in step S4-1 and the data obtained by upsampling and outputting the result; Step S4-3: performing convolution operation on the data obtained in step S4-2 and outputting the convolution operation; Step S4-4: splicing the processed data obtained in step S4-3 with the high-scale feature map data and outputting them; Step S4-5: The data obtained in step S4-4 is subjected to convolution operation again and output to obtain the third feature map.

6. The method for intelligent detection of circuit board defects based on big data according to claim 1 is characterized in that: The preprocessing operations in step S1 include: data cleaning, image enhancement, data enhancement, and data normalization.

7. The method for intelligent detection of circuit board defects based on big data according to claim 1, characterized in that: The post-processing operations in step S6 include: frame screening, frame adjustment, category assignment and non-maximum suppression.

Images