PCB production line defect detection system and method based on image processing

Abstract

According to the PCB production line defect detection system and method based on image processing, a plurality of high-resolution cameras, an infrared thermal imaging camera and a 3D laser scanner are adopted, a dynamically-controlled annular LED light source, polarized light illumination and a structured light projection technology are combined, a foreground and a background are separated through a self-adaptive threshold Otsu method, and the defect detection accuracy is improved. A convolutional neural network denoising model and a frequency domain filtering technology are introduced, welding spot and line features are extracted from a preprocessed image, a coding scheme is designed, multi-modal features are fused, spatial-temporal feature analysis is introduced, fine classification and positioning are performed in combination with SVM preliminary screening and a YOLOv5 deep learning model, an online learning mechanism is introduced, model parameters are dynamically optimized, and a multi-modal image is obtained. An intelligent early warning mechanism is established, automatic grading and alarm sending are carried out, a statistical report is automatically generated, production process parameters are recorded, root cause analysis is carried out in combination with big data analysis, and potential defects are early warned in advance.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a PCB production line defect detection system and method based on image processing. Background Art

[0002] At present, in the quality control link of printed circuit board (PCB) production, traditional visual inspection technology generally has certain limitations and challenges. Due to the complex structure, diverse layers, dense components, small color differences of PCB, and minor defects in the production process such as short circuit, open circuit, poor welding and other problems are difficult to identify with the naked eye or simple visual systems. Therefore, the existing PCB defect detection technology based on image processing often has shortcomings.

[0003] Image acquisition is not comprehensive. Single-view, low-resolution cameras may not be able to cover all potential defect areas, especially when facing multi-layered, obstructed PCBs, which can easily cause blind spots and misjudgments; poor image quality, uneven lighting conditions, severe noise interference, and texture problems under complex backgrounds require more efficient denoising and background separation technologies in the preprocessing stage, and some existing general methods may not be able to meet this demand; insufficient feature extraction. Existing technologies may be relatively rough in extracting key PCB features. For example, they fail to fully utilize detailed information such as solder joint shape, area ratio, and line width consistency, which limits the accuracy of subsequent defect detection; the detection algorithm is limited in performance. Traditional machine learning algorithms may have insufficient generalization capabilities when dealing with complex and subtle PCB defects. Although deep learning has shown great potential in the field of image recognition, in practical applications, how to effectively train models to deal with various types and forms of PCB defects is still a major challenge. Summary of the invention

[0004] The purpose of the present invention is to provide a PCB production line defect detection system and method based on image processing.

[0005] The problems to be solved by the present invention are: for complex PCBs with occlusion and multi-level structures, it is difficult to fully cover the traditional image acquisition method, resulting in the omission of some defects; the denoising and background separation technology in the preprocessing stage is not intelligent enough, cannot adapt to complex backgrounds and changing colors, and affects the accuracy of subsequent feature extraction; the feature extraction method is single and fails to make full use of multimodal data, resulting in insufficient detection ability for subtle defects; the defect detection algorithm relies on a fixed model, is difficult to adapt to changes in the production process, and has low detection accuracy and efficiency; the lack of an intelligent early warning and feedback mechanism, the inability to promptly discover and deal with urgent or major defects, and affects production quality and efficiency.

[0006] A PCB production line defect detection system based on image processing, comprising: Image acquisition module: Use multiple high-resolution cameras to shoot the PCB from different angles, introduce infrared thermal imaging cameras and 3D laser scanners to obtain temperature distribution and geometric structure information, dynamically control the ring LED light source, combine polarized light illumination and structured light projection technology, and automatically adjust the light source brightness and angle through real-time image analysis. All cameras and sensors use a high-speed synchronous trigger mechanism, combined with multi-view images for stereo matching and 3D reconstruction, to generate a three-dimensional model of the PCB; Image preprocessing module: De-noise, grayscale, smooth and edge enhance the collected images. For the complex and changeable background and component colors on the PCB, the adaptive threshold technology Otsu method is used to separate the foreground and background. A denoising model based on a convolutional neural network is introduced to automatically adjust the filter scale according to the feature complexity of different regions to remove noise and retain details. The frequency domain filtering technology and the ring filter are combined to effectively remove the interference of periodic texture noise and low-frequency parts, and the image is resized and normalized. Feature extraction module: Extract feature information about PCB components, circuits and solder joints from preprocessed images, including shape, color, texture and edge. Design a special feature encoding scheme for solder joint quality, extract key features of circularity and area ratio, integrate temperature distribution and height difference multimodal features obtained by infrared thermal imaging and 3D laser scanning, introduce spatiotemporal feature analysis, capture the change trend of defects over time, and analyze the spatial relationship between different areas in combination with the physical layout of the PCB to perform multi-dimensional feature encoding; Defect detection algorithm module: It adopts a method combining machine learning and deep learning. It first uses the classic algorithm SVM for preliminary screening, and then uses the deep learning model YOLOv5 for fine classification and positioning. It introduces an online learning mechanism. During the training process, it performs classification, segmentation and positioning tasks at the same time, shares the feature extraction layer, and dynamically optimizes the detection parameters through incremental learning and adaptive threshold adjustment. Result analysis and display module: Analyze and display the detected defect results, and provide timely feedback to operators and automation systems. The detection results are displayed in the form of images on the monitoring interface. Operators can view the enlarged image of the suspected defect area and compare it with normal samples. It also provides one-click marking confirmation and correction functions. An intelligent early warning mechanism is established to automatically classify defects according to their severity and type, and immediately issue alarms for urgent or major defects. Predictive maintenance functions are introduced to analyze historical defect data and combine time series prediction models to provide early warning of possible defect types and locations. Defect statistical reports are automatically generated, including defect type distribution and time trend analysis, to provide data support for continuous improvement of production lines and develop user-friendly interactive interfaces.

[0007] A PCB production line defect detection method based on image processing, based on a PCB production line defect detection system based on image processing, the technical solution adopted is as follows: S1: Use multiple high-resolution cameras to shoot PCBs from different angles, introduce infrared thermal imaging cameras and 3D laser scanners, dynamically control the ring LED light source, add polarized light illumination and structured light projection, and collect images of PCBs with obstructions and multi-level structures; S2: Preprocess the acquired images, use the adaptive threshold technology Otsu method to separate the foreground and background of the acquired PCB images, combine the frequency domain filtering technology Fourier transform to introduce a denoising model based on a convolutional neural network, and automatically adjust the filter scale to remove noise according to the feature complexity of different regions; S3: Extract feature information about PCB components, circuits and solder joints from preprocessed images, including shape, color, texture, and edge. Design a special feature encoding scheme for the roundness and area ratio of solder joint quality, as well as the consistency of circuit width and spacing. Integrate the temperature distribution and height difference multimodal features obtained by infrared thermal imaging and 3D laser scanning, and introduce spatiotemporal features to analyze the dynamic characteristics of the PCB production line. S4: The feature information of the pre-processed normal PCB images and different types of PCB defect images are used as data sets, and machine learning computer vision algorithms are used to detect defects in PCB images. SVM algorithms are used for preliminary screening, and YOLOv5 deep learning models are used for subsequent classification and positioning. Online learning mechanisms are introduced to continuously update and optimize during the production process. S5: Input the data set into the model for training and adjusting the model parameters. In the process of model training, introduce data enhancement and synthetic defect generation technology. Apply the trained model to the real-time PCB image to detect whether there are defects. S6: Feedback the detected defect results to the operator and the automation system in a timely manner, display the detection results in the form of images, establish an intelligent early warning mechanism, automatically classify the defects according to their severity and type, issue alarms for defects of different levels, introduce predictive maintenance functions, analyze historical defect data, and combine time series prediction models to warn of possible defect types and locations in advance; S7: Develop a user-friendly interactive interface that allows operators to directly view the enlarged image of the suspected defect area and compare it with the normal sample on the interface, and provide one-click marking confirmation and correction functions to improve operating efficiency. The system automatically generates defect statistical reports, including defect type distribution and time trend analysis, to provide data support for continuous improvement of the production line; S8: Introduce a defect traceability mechanism to record the production process parameters of each PCB, associate them with the final defect detection results, combine big data analysis and machine learning technology to perform root cause analysis, and analyze historical defect data to find out the main causes of defects.

[0008] Furthermore, the S1 introduces an infrared thermal imaging camera and a 3D laser scanner, dynamically controls the annular LED light source, adds polarized light illumination and structured light projection, and collects images of PCBs with obstructions and multi-level structures, including: S11: Introducing a highly sensitive infrared thermal imaging camera to capture the temperature distribution on the PCB surface, using a high-precision 3D laser scanner to obtain the 3D structural information of the PCB, including the height of the solder joints, the thickness of the circuit, and the shape of the components, using a polarized light illumination system combined with structured light projection technology to generate specific patterns on the PCB surface, and using an array of multiple high-resolution cameras to shoot the PCB from different angles; S12: Automatically identify under-exposure, over-exposure areas and shadow problems through real-time image analysis, and determine whether the brightness and angle of each LED light source partition need to be adjusted based on the results of real-time image analysis; S13: A high-speed synchronous trigger mechanism is used for all cameras and sensors, and multi-view images are combined with a stereo matching algorithm to perform 3D reconstruction to generate a 3D model of the PCB.

[0009] Furthermore, the adaptive threshold technology Otsu method is used in S2 to separate the foreground and background of the collected PCB image, and the denoising model based on convolutional neural network is introduced in combination with the frequency domain filtering technology Fourier transform, including: S21: Convert the color image to a grayscale image, calculate the grayscale histogram of the image, traverse all grayscale thresholds, calculate the corresponding inter-class variance and find the threshold that maximizes the inter-class variance. ,in, and Represents the pixel ratio of background and foreground respectively, and Represent the average gray value of the background and foreground respectively. Set the pixels with gray values ​​less than or equal to 5 in the image as the background, and the pixels with gray values ​​greater than 250 as the foreground, and separate the foreground and background; S22: resizing the image after foreground separation, checking whether the width and height of the image are integer powers of 2, if not, selecting the nearest larger power of 2 size to expand it by zero filling, applying two-dimensional discrete Fourier transform to the image, converting the image from the spatial domain to the frequency domain, and using an annular filter to filter out the texture information of the low-frequency part, the central part of the annular filter is set to 0, and the periphery is set to an attenuation coefficient for gradual transition, using two-dimensional discrete inverse Fourier transform to convert the filtered frequency domain image back to the spatial domain, and normalizing the inverse transformed image; S23: The ring filter transfer function is ,in Frequency point The distance to the origin, is the inner radius of the annular filter, is the attenuation coefficient, which controls the transition bandwidth of the filter.

[0010] Furthermore, in S3, a special feature encoding scheme is designed for the roundness and area ratio characteristics of solder joint quality, as well as the consistency of line width and spacing, and the temperature distribution and height difference multi-modal characteristics obtained by infrared thermal imaging and 3D laser scanning are integrated, and the spatiotemporal characteristics are introduced to analyze the dynamic characteristics of the PCB production line, including: S31: Shape feature encoding: perform contour detection on each object in the image, calculate the rectangularity, circularity, and aspect ratio of the contour, calculate the length, angle, and curvature geometric properties of the line segment, and extract the circularity feature of the welding point and encode it into a feature vector; Color feature coding: Use color space conversion from RGB to HSV and histogram statistics to record the difference between the solder joint color and the background color, and count the consistency of the circuit color; Texture feature encoding: Use local binary mode to encode the surface texture of PCB components and the surface quality of solder joints, and count the number and distribution of texture features in a specific area; Edge feature encoding: Use the Canny edge detection algorithm to extract solder joint and component edge information, calculate edge integrity and sharpness, and perform statistical continuity, smoothness, and width variation on line edges; Solder joint quality feature coding: Calculate the circular similarity index of the solder joint as the circularity error, standardize and code the error value, measure the ratio of the actual area of ​​the solder joint to its ideal area, and judge whether the solder joint is too large or too small according to the set threshold; Line feature coding: detect the width of each line section, calculate the standard deviation of the width, analyze the distance between lines, and calculate the standard deviation of the distance between adjacent lines; S32: Extract temperature distribution features from infrared thermal imaging images to identify thermal anomaly areas on the PCB. Analyze the continuous PCB image sequence to capture the change trend of defects over time. Combined with the physical layout of the PCB, analyze the spatial relationship between different areas. S33: Adoption Calculate the temporal change of each pixel in the image sequence, where is the time series characteristic value at time t, is the number of pixels in the image, is the weight of the ith pixel, which is adjusted according to whether it is located in the solder joint or the circuit area. and is the grayscale value of the i-th pixel at time t and t-1.

[0011] Furthermore, the S4 uses the SVM algorithm for preliminary screening, uses the YOLOv5 deep learning model for subsequent classification and positioning, and introduces an online learning mechanism to continuously update and optimize during the production process, including: S41: Classification, segmentation and localization tasks are performed simultaneously during training, feature extraction layers are shared, and the total loss function ,in is the loss function for the classification task, is the loss function of the positioning task, is the loss function for the segmentation task, is the weight of the corresponding task; S42: Introduce an incremental learning mechanism to automatically adjust model parameters and dynamically adjust the threshold of defect detection by collecting newly generated defect samples. When a certain type of defect appears frequently within a period of time, the system automatically lowers the detection threshold of this type of defect.

[0012] Furthermore, an intelligent early warning mechanism is established in S6 to automatically classify defects according to their severity and type, and issue alarms for defects of different levels, including: S61: A comprehensive evaluation system is constructed based on the defect type, location, size, and shape. Each dimension is assigned a different weight, and a comprehensive score is finally calculated to determine the severity of the defect. S62: Introduce fuzzy logic algorithm to deal with the uncertainty and fuzziness of defect assessment problems. A fuzzy membership function is calculated based on multiple input variables including defect area, shape, and location to ultimately determine the level of the defect. S63: Comprehensive evaluation system for defects , where n is the number of evaluation dimensions, is the weight of the i-th dimension, is the scoring function of the i-th dimension, It is the specific measurement value of this dimension, and the corresponding warning is triggered according to the evaluation result.

[0013] The beneficial effects of the present invention are as follows: multiple high-resolution cameras are used to shoot the PCB from different angles, and a 3D laser scanner is used to obtain the three-dimensional structural information of the PCB, ensuring that the PCB with obstructions and multi-layer structures can be fully covered to avoid missing key areas. Infrared thermal imaging cameras are introduced to capture temperature distribution, and a 3D laser scanner is used to obtain geometric features such as solder joint height and line thickness, providing multi-modal data, and enhancing the detection capability of hidden defects such as electrical faults and overheating areas; Dynamically control the annular LED light source, combined with polarized light illumination and structured light projection technology, to reduce reflected light interference and highlight surface texture. Automatically adjust the light source brightness and angle through real-time image analysis to ensure the best image quality in each area, especially for highly reflective materials and complex backgrounds. Introduce a denoising model based on convolutional neural network (CNN), automatically adjust the filter scale according to the feature complexity of different areas, remove noise while retaining more details. Combine frequency domain filtering technology and annular filter to effectively remove periodic texture noise and low-frequency interference, and improve image clarity and contrast. Design a special feature encoding scheme. In particular, for solder joint quality, extract key features such as circularity and area ratio, conduct a comprehensive assessment of solder joint quality, integrate the temperature distribution and height difference multi-modal features obtained by infrared thermal imaging and 3D laser scanning, provide richer feature information, analyze the continuous PCB image sequence, capture the change trend of defects over time, and analyze the spatial relationship between different areas in combination with the physical layout of the PCB to help identify progressive defects and process problems; A comprehensive evaluation system is constructed based on multiple dimensions such as defect type, location, size, shape, etc. Different weights are assigned to each dimension, and ultimately a comprehensive score is calculated to determine the severity of the defect. Fuzzy logic algorithms are introduced to deal with uncertainty and ambiguity in defect evaluation to ensure the scientificity and rationality of the grading results. Corresponding warnings are triggered based on the evaluation results, and alarms are immediately issued for urgent or major defects to help operators take timely measures. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A module diagram of a PCB production line defect detection system based on image processing according to the present invention; Figure 2 The present invention is a flow chart of a PCB production line defect detection method based on image processing. DETAILED DESCRIPTION

[0015] The present invention is further clearly and completely described below, but the protection scope of the present invention is not limited thereto.

[0016] A PCB production line defect detection system based on image processing, comprising: Image acquisition module: Use multiple high-resolution cameras to shoot the PCB from different angles, introduce infrared thermal imaging cameras and 3D laser scanners to obtain temperature distribution and geometric structure information, dynamically control the ring LED light source, combine polarized light illumination and structured light projection technology, and automatically adjust the light source brightness and angle through real-time image analysis. All cameras and sensors use a high-speed synchronous trigger mechanism, combined with multi-view images for stereo matching and 3D reconstruction, to generate a three-dimensional model of the PCB; Image preprocessing module: De-noise, grayscale, smooth and edge enhance the collected images. For the complex and changeable background and component colors on the PCB, the adaptive threshold technology Otsu method is used to separate the foreground and background. A denoising model based on a convolutional neural network is introduced to automatically adjust the filter scale according to the feature complexity of different regions to remove noise and retain details. The frequency domain filtering technology and the ring filter are combined to effectively remove the interference of periodic texture noise and low-frequency parts, and the image is resized and normalized. Feature extraction module: Extract feature information about PCB components, circuits and solder joints from preprocessed images, including shape, color, texture and edge. Design a special feature encoding scheme for solder joint quality, extract key features of circularity and area ratio, integrate temperature distribution and height difference multimodal features obtained by infrared thermal imaging and 3D laser scanning, introduce spatiotemporal feature analysis, capture the change trend of defects over time, and analyze the spatial relationship between different areas in combination with the physical layout of the PCB to perform multi-dimensional feature encoding; Defect detection algorithm module: It adopts a method combining machine learning and deep learning. It first uses the classic algorithm SVM for preliminary screening, and then uses the deep learning model YOLOv5 for fine classification and positioning. It introduces an online learning mechanism. During the training process, it performs classification, segmentation and positioning tasks at the same time, shares the feature extraction layer, and dynamically optimizes the detection parameters through incremental learning and adaptive threshold adjustment. Result analysis and display module: Analyze and display the detected defect results, and provide timely feedback to operators and automation systems. The detection results are displayed in the form of images on the monitoring interface. Operators can view the enlarged image of the suspected defect area and compare it with normal samples. It also provides one-click marking confirmation and correction functions. An intelligent early warning mechanism is established to automatically classify defects according to their severity and type, and immediately issue alarms for urgent or major defects. Predictive maintenance functions are introduced to analyze historical defect data and combine time series prediction models to provide early warning of possible defect types and locations. Defect statistical reports are automatically generated, including defect type distribution and time trend analysis, to provide data support for continuous improvement of production lines and develop user-friendly interactive interfaces.

[0017] refer to Figure 1 As shown, this is a module diagram of a PCB production line defect detection system based on image processing.

[0018] A PCB production line defect detection method based on image processing, based on a PCB production line defect detection system based on image processing, the technical solution adopted is as follows: S1: Use multiple high-resolution cameras to shoot PCBs from different angles, introduce infrared thermal imaging cameras and 3D laser scanners, dynamically control the ring LED light source, add polarized light illumination and structured light projection, and collect images of PCBs with obstructions and multi-level structures; S2: Preprocess the acquired images, use the adaptive threshold technology Otsu method to separate the foreground and background of the acquired PCB images, combine the frequency domain filtering technology Fourier transform to introduce a denoising model based on a convolutional neural network, and automatically adjust the filter scale to remove noise according to the feature complexity of different regions; S3: Extract feature information about PCB components, circuits and solder joints from preprocessed images, including shape, color, texture, and edge. Design a special feature encoding scheme for the roundness and area ratio of solder joint quality, as well as the consistency of circuit width and spacing. Integrate the temperature distribution and height difference multimodal features obtained by infrared thermal imaging and 3D laser scanning, and introduce spatiotemporal features to analyze the dynamic characteristics of the PCB production line. S4: The feature information of the pre-processed normal PCB images and different types of PCB defect images are used as data sets, and machine learning computer vision algorithms are used to detect defects in PCB images. SVM algorithms are used for preliminary screening, and YOLOv5 deep learning models are used for subsequent classification and positioning. Online learning mechanisms are introduced to continuously update and optimize during the production process. S5: Input the data set into the model for training and adjusting the model parameters. In the process of model training, introduce data enhancement and synthetic defect generation technology. Apply the trained model to the real-time PCB image to detect whether there are defects. S6: Feedback the detected defect results to the operator and the automation system in a timely manner, display the detection results in the form of images, establish an intelligent early warning mechanism, automatically classify the defects according to their severity and type, issue alarms for defects of different levels, introduce predictive maintenance functions, analyze historical defect data, and combine time series prediction models to warn of possible defect types and locations in advance; S7: Develop a user-friendly interactive interface that allows operators to directly view the enlarged image of the suspected defect area and compare it with the normal sample on the interface, and provide one-click marking confirmation and correction functions to improve operating efficiency. The system automatically generates defect statistical reports, including defect type distribution and time trend analysis, to provide data support for continuous improvement of the production line; S8: Introduce a defect traceability mechanism to record the production process parameters of each PCB, associate them with the final defect detection results, combine big data analysis and machine learning technology to perform root cause analysis, and analyze historical defect data to find out the main causes of defects.

[0019] refer to Figure 2 The figure shows a flow chart of a PCB production line defect detection method based on image processing.

[0020] Furthermore, the S1 introduces an infrared thermal imaging camera and a 3D laser scanner, dynamically controls the annular LED light source, adds polarized light illumination and structured light projection, and collects images of PCBs with obstructions and multi-level structures, including: S11: Introducing a highly sensitive infrared thermal imaging camera to capture the temperature distribution on the PCB surface, using a high-precision 3D laser scanner to obtain the 3D structural information of the PCB, including the height of the solder joints, the thickness of the circuit, and the shape of the components, using a polarized light illumination system combined with structured light projection technology to generate specific patterns on the PCB surface, and using an array of multiple high-resolution cameras to shoot the PCB from different angles; S12: Automatically identify under-exposure, over-exposure areas and shadow problems through real-time image analysis, and determine whether the brightness and angle of each LED light source partition need to be adjusted based on the results of real-time image analysis; S13: A high-speed synchronous trigger mechanism is used for all cameras and sensors, and multi-view images are combined with a stereo matching algorithm to perform 3D reconstruction to generate a 3D model of the PCB.

[0021] Furthermore, the adaptive threshold technology Otsu method is used in S2 to separate the foreground and background of the collected PCB image, and the denoising model based on convolutional neural network is introduced in combination with the frequency domain filtering technology Fourier transform, including: S21: Convert the color image to a grayscale image, calculate the grayscale histogram of the image, traverse all grayscale thresholds, calculate the corresponding inter-class variance and find the threshold that maximizes the inter-class variance. ,in, and Represents the pixel ratio of background and foreground respectively, and Represent the average gray value of the background and foreground respectively. Set the pixels with gray values ​​less than or equal to 5 in the image as the background, and the pixels with gray values ​​greater than 250 as the foreground, and separate the foreground and background; S22: resizing the image after foreground separation, checking whether the width and height of the image are integer powers of 2, if not, selecting the nearest larger power of 2 size to expand it by zero filling, applying two-dimensional discrete Fourier transform to the image, converting the image from the spatial domain to the frequency domain, and using an annular filter to filter out the texture information of the low-frequency part, the central part of the annular filter is set to 0, and the periphery is set to an attenuation coefficient for gradual transition, using two-dimensional discrete inverse Fourier transform to convert the filtered frequency domain image back to the spatial domain, and normalizing the inverse transformed image; S23: The ring filter transfer function is ,in Frequency point The distance to the origin, is the inner radius of the annular filter, is the attenuation coefficient, which controls the transition bandwidth of the filter.

[0022] Furthermore, in S3, a special feature encoding scheme is designed for the roundness and area ratio characteristics of solder joint quality, as well as the consistency of line width and spacing, and the temperature distribution and height difference multi-modal characteristics obtained by infrared thermal imaging and 3D laser scanning are integrated, and the spatiotemporal characteristics are introduced to analyze the dynamic characteristics of the PCB production line, including: S31: Shape feature encoding: perform contour detection on each object in the image, calculate the rectangularity, circularity, and aspect ratio of the contour, calculate the length, angle, and curvature geometric properties of the line segment, and extract the circularity feature of the welding point and encode it into a feature vector; Color feature coding: Use color space conversion from RGB to HSV and histogram statistics to record the difference between the solder joint color and the background color, and count the consistency of the circuit color; Texture feature encoding: Use local binary mode to encode the surface texture of PCB components and the surface quality of solder joints, and count the number and distribution of texture features in a specific area; Edge feature encoding: Use the Canny edge detection algorithm to extract solder joint and component edge information, calculate edge integrity and sharpness, and perform statistical continuity, smoothness, and width variation on line edges; Solder joint quality feature coding: Calculate the circular similarity index of the solder joint as the circularity error, standardize and code the error value, measure the ratio of the actual area of ​​the solder joint to its ideal area, and judge whether the solder joint is too large or too small according to the set threshold; Line feature coding: detect the width of each line section, calculate the standard deviation of the width, analyze the distance between lines, and calculate the standard deviation of the distance between adjacent lines; S32: Extract temperature distribution features from infrared thermal imaging images to identify thermal anomaly areas on the PCB. Analyze the continuous PCB image sequence to capture the change trend of defects over time. Combined with the physical layout of the PCB, analyze the spatial relationship between different areas. S33: Adoption Calculate the temporal change of each pixel in the image sequence, where is the time series characteristic value at time t, is the number of pixels in the image, is the weight of the ith pixel, which is adjusted according to whether it is located in the solder joint or the circuit area. and is the grayscale value of the i-th pixel at time t and t-1.

[0023] Furthermore, the S4 uses the SVM algorithm for preliminary screening, uses the YOLOv5 deep learning model for subsequent classification and positioning, and introduces an online learning mechanism to continuously update and optimize during the production process, including: S41: Classification, segmentation and localization tasks are performed simultaneously during training, feature extraction layers are shared, and the total loss function ,in is the loss function for the classification task, is the loss function of the positioning task, is the loss function for the segmentation task, is the weight of the corresponding task; S42: Introduce an incremental learning mechanism to automatically adjust model parameters and dynamically adjust the threshold of defect detection by collecting newly generated defect samples. When a certain type of defect appears frequently within a period of time, the system automatically lowers the detection threshold of this type of defect.

[0024] Furthermore, an intelligent early warning mechanism is established in S6 to automatically classify defects according to their severity and type, and issue alarms for defects of different levels, including: S61: A comprehensive evaluation system is constructed based on the defect type, location, size, and shape. Each dimension is assigned a different weight, and a comprehensive score is finally calculated to determine the severity of the defect. S62: Introduce fuzzy logic algorithm to deal with the uncertainty and fuzziness of defect assessment problems. A fuzzy membership function is calculated based on multiple input variables including defect area, shape, and location to ultimately determine the level of the defect. S63: Comprehensive evaluation system for defects , where n is the number of evaluation dimensions, is the weight of the i-th dimension, is the scoring function of the i-th dimension, It is the specific measurement value of this dimension, and the corresponding warning is triggered according to the evaluation result.

[0025] The present invention provides a PCB production line defect detection system and method based on image processing, adopts multiple high-resolution cameras, infrared thermal imaging cameras and 3D laser scanners, combines dynamically controlled annular LED light sources, polarized light illumination and structured light projection technology, separates foreground and background through adaptive threshold Otsu method, introduces convolutional neural network denoising model and frequency domain filtering technology, extracts solder joint and line features from preprocessed images, designs coding schemes, integrates multimodal features, introduces spatiotemporal feature analysis, combines SVM preliminary screening and YOLOv5 deep learning model for fine classification and positioning, introduces online learning mechanism, dynamically optimizes model parameters, establishes intelligent early warning mechanism, automatically classifies and issues alarms, automatically generates statistical reports, records production process parameters, combines big data analysis for root cause analysis, and warns of potential defects in advance.

Claims

1. A PCB production line defect detection system based on image processing, comprising: Image acquisition module: Use multiple high-resolution cameras to shoot the PCB from different angles, introduce infrared thermal imaging cameras and 3D laser scanners to obtain temperature distribution and geometric structure information, dynamically control the ring LED light source, combine polarized light illumination and structured light projection technology, and automatically adjust the light source brightness and angle through real-time image analysis. All cameras and sensors use a high-speed synchronous trigger mechanism, combined with multi-view images for stereo matching and 3D reconstruction, to generate a three-dimensional model of the PCB; Image preprocessing module: De-noise, grayscale, smooth and edge enhance the collected images. For the complex and changeable background and component colors on the PCB, the adaptive threshold technology Otsu method is used to separate the foreground and background. A denoising model based on a convolutional neural network is introduced to automatically adjust the filter scale according to the feature complexity of different regions to remove noise and retain details. The frequency domain filtering technology and the ring filter are combined to effectively remove the interference of periodic texture noise and low-frequency parts, and the image is resized and normalized. Feature extraction module: Extract feature information about PCB components, circuits and solder joints from preprocessed images, including shape, color, texture and edge. Design a special feature encoding scheme for solder joint quality, extract key features of circularity and area ratio, integrate temperature distribution and height difference multimodal features obtained by infrared thermal imaging and 3D laser scanning, introduce spatiotemporal feature analysis, capture the change trend of defects over time, and analyze the spatial relationship between different areas in combination with the physical layout of the PCB to perform multi-dimensional feature encoding; Defect detection algorithm module: It adopts a method combining machine learning and deep learning. It first uses the classic algorithm SVM for preliminary screening, and then uses the deep learning model YOLOv5 for fine classification and positioning. It introduces an online learning mechanism. During the training process, it performs classification, segmentation and positioning tasks at the same time, shares the feature extraction layer, and dynamically optimizes the detection parameters through incremental learning and adaptive threshold adjustment. Result analysis and display module: Analyze and display the detected defect results, and provide timely feedback to operators and automation systems. The detection results are displayed in the form of images on the monitoring interface. Operators can view the enlarged image of the suspected defect area and compare it with normal samples. It also provides one-click marking confirmation and correction functions. An intelligent early warning mechanism is established to automatically classify defects according to their severity and type, and immediately issue alarms for urgent or major defects. Predictive maintenance functions are introduced to analyze historical defect data and combine time series prediction models to provide early warning of possible defect types and locations. Defect statistical reports are automatically generated, including defect type distribution and time trend analysis, to provide data support for continuous improvement of production lines and develop user-friendly interactive interfaces.

2. A PCB production line defect detection method based on image processing, characterized in that: A PCB production line defect detection system based on image processing as claimed in claim 1, comprising: S1: Use multiple high-resolution cameras to shoot PCBs from different angles, introduce infrared thermal imaging cameras and 3D laser scanners, dynamically control the ring LED light source, add polarized light illumination and structured light projection, and collect images of PCBs with obstructions and multi-level structures; S2: Preprocess the acquired images, use the adaptive threshold technology Otsu method to separate the foreground and background of the acquired PCB images, combine the frequency domain filtering technology Fourier transform to introduce a denoising model based on a convolutional neural network, and automatically adjust the filter scale to remove noise according to the feature complexity of different regions; S3: Extract feature information about PCB components, circuits and solder joints from preprocessed images, including shape, color, texture, and edge. Design a special feature encoding scheme for the roundness and area ratio of solder joint quality, as well as the consistency of circuit width and spacing. Integrate the temperature distribution and height difference multimodal features obtained by infrared thermal imaging and 3D laser scanning, and introduce spatiotemporal features to analyze the dynamic characteristics of the PCB production line. S4: The feature information of the pre-processed normal PCB images and different types of PCB defect images are used as data sets, and machine learning computer vision algorithms are used to detect defects in PCB images. SVM algorithms are used for preliminary screening, and YOLOv5 deep learning models are used for subsequent classification and positioning. Online learning mechanisms are introduced to continuously update and optimize during the production process. S5: Input the data set into the model for training and adjusting the model parameters. In the process of model training, introduce data enhancement and synthetic defect generation technology. Apply the trained model to the real-time PCB image to detect whether there are defects. S6: Feedback the detected defect results to the operator and the automation system in a timely manner, display the detection results in the form of images, establish an intelligent early warning mechanism, automatically classify the defects according to their severity and type, issue alarms for defects of different levels, introduce predictive maintenance functions, analyze historical defect data, and combine time series prediction models to warn of possible defect types and locations in advance; S7: Develop a user-friendly interactive interface that allows operators to directly view the enlarged image of the suspected defect area and compare it with the normal sample on the interface, and provide one-click marking confirmation and correction functions to improve operating efficiency. The system automatically generates defect statistical reports, including defect type distribution and time trend analysis, to provide data support for continuous improvement of the production line; S8: Introduce a defect traceability mechanism to record the production process parameters of each PCB, associate them with the final defect detection results, combine big data analysis and machine learning technology to perform root cause analysis, and analyze historical defect data to find out the main causes of defects.

3. A PCB production line defect detection method based on image processing as claimed in claim 2, characterized in that: The S1 introduces an infrared thermal imaging camera and a 3D laser scanner, dynamically controls the annular LED light source, adds polarized light illumination and structured light projection, and collects images of PCBs with obstructions and multi-level structures, including: S11: Introducing a highly sensitive infrared thermal imaging camera to capture the temperature distribution on the PCB surface, using a high-precision 3D laser scanner to obtain the 3D structural information of the PCB, including the height of the solder joints, the thickness of the circuit, and the shape of the components, using a polarized light illumination system combined with structured light projection technology to generate specific patterns on the PCB surface, and using an array of multiple high-resolution cameras to shoot the PCB from different angles; S12: Automatically identify under-exposure, over-exposure areas and shadow problems through real-time image analysis, and determine whether the brightness and angle of each LED light source partition need to be adjusted based on the results of real-time image analysis; S13: A high-speed synchronous trigger mechanism is used for all cameras and sensors, and multi-view images are combined with a stereo matching algorithm to perform 3D reconstruction to generate a 3D model of the PCB.

4. The method for detecting defects in a PCB production line based on image processing according to claim 2, characterized in that: In S2, the adaptive threshold technology Otsu method is used to separate the foreground and background of the collected PCB image, and the denoising model based on convolutional neural network is introduced by combining the frequency domain filtering technology Fourier transform, including: S21: Convert the color image to a grayscale image, calculate the grayscale histogram of the image, traverse all grayscale thresholds, calculate the corresponding inter-class variance and find the threshold that maximizes the inter-class variance. ,in, and Represents the pixel ratio of background and foreground respectively, and Represent the average gray value of the background and foreground respectively. Set the pixels with gray values ​​less than or equal to 5 in the image as the background, and the pixels with gray values ​​greater than 250 as the foreground, and separate the foreground and background; S22: resizing the image after foreground separation, checking whether the width and height of the image are integer powers of 2, if not, selecting the nearest larger power of 2 size to expand it by zero filling, applying two-dimensional discrete Fourier transform to the image, converting the image from the spatial domain to the frequency domain, and using an annular filter to filter out the texture information of the low-frequency part, the central part of the annular filter is set to 0, and the periphery is set to an attenuation coefficient for gradual transition, using two-dimensional discrete inverse Fourier transform to convert the filtered frequency domain image back to the spatial domain, and normalizing the inverse transformed image; S23: The ring filter transfer function is ,in Frequency point The distance to the origin, is the inner radius of the annular filter, is the attenuation coefficient, which controls the transition bandwidth of the filter.

5. The method for detecting defects in a PCB production line based on image processing according to claim 2, characterized in that: In the S3, a special feature encoding scheme is designed for the roundness and area ratio characteristics of solder joint quality, as well as the consistency of line width and spacing. The temperature distribution and height difference multi-modal characteristics obtained by infrared thermal imaging and 3D laser scanning are integrated, and the spatiotemporal characteristics are introduced to analyze the dynamic characteristics of the PCB production line, including: S31: Shape feature encoding: perform contour detection on each object in the image, calculate the rectangularity, circularity, and aspect ratio of the contour, calculate the length, angle, and curvature geometric properties of the line segment, and extract the circularity feature of the welding point and encode it into a feature vector; Color feature coding: Use color space conversion from RGB to HSV and histogram statistics to record the difference between the solder joint color and the background color, and count the consistency of the circuit color; Texture feature encoding: Use local binary mode to encode the surface texture of PCB components and the surface quality of solder joints, and count the number and distribution of texture features in a specific area; Edge feature encoding: Use the Canny edge detection algorithm to extract solder joint and component edge information, calculate edge integrity and sharpness, and perform statistical continuity, smoothness, and width variation on line edges; Solder joint quality feature coding: Calculate the circular similarity index of the solder joint as the circularity error, standardize and code the error value, measure the ratio of the actual area of ​​the solder joint to its ideal area, and judge whether the solder joint is too large or too small according to the set threshold; Line feature coding: detect the width of each line section, calculate the standard deviation of the width, analyze the distance between lines, and calculate the standard deviation of the distance between adjacent lines; S32: Extract temperature distribution features from infrared thermal imaging images to identify thermal anomaly areas on the PCB. Analyze the continuous PCB image sequence to capture the change trend of defects over time. Combined with the physical layout of the PCB, analyze the spatial relationship between different areas. S33: Adoption Calculate the temporal change of each pixel in the image sequence, where is the time series characteristic value at time t, is the number of pixels in the image, is the weight of the ith pixel, which is adjusted according to whether it is located in the solder joint or the circuit area. and is the grayscale value of the i-th pixel at time t and t-1.

6. A PCB production line defect detection method based on image processing as claimed in claim 2, characterized in that: In the S4, the SVM algorithm is used for preliminary screening, the YOLOv5 deep learning model is used for subsequent classification and positioning, and the online learning mechanism is introduced to continuously update and optimize during the production process, including: S41: Classification, segmentation and localization tasks are performed simultaneously during training, feature extraction layers are shared, and the total loss function ,in is the loss function for the classification task, is the loss function of the positioning task, is the loss function for the segmentation task, is the weight of the corresponding task; S42: Introduce an incremental learning mechanism to automatically adjust model parameters and dynamically adjust the threshold of defect detection by collecting newly generated defect samples. When a certain type of defect appears frequently within a period of time, the system automatically lowers the detection threshold of this type of defect.

7. A PCB production line defect detection method based on image processing as claimed in claim 2, characterized in that: The S6 establishes an intelligent early warning mechanism, which automatically classifies defects according to their severity and type, and issues alarms for defects of different levels, including: S61: A comprehensive evaluation system is constructed based on the defect type, location, size, and shape. Each dimension is assigned a different weight, and a comprehensive score is finally calculated to determine the severity of the defect. S62: Introduce fuzzy logic algorithm to deal with the uncertainty and fuzziness of defect assessment problems. A fuzzy membership function is calculated based on multiple input variables including defect area, shape, and location to ultimately determine the level of the defect. S63: Comprehensive evaluation system for defects , where n is the number of evaluation dimensions, is the weight of the i-th dimension, is the scoring function of the i-th dimension, It is the specific measurement value of this dimension, and the corresponding warning is triggered according to the evaluation result.

Images