Circuit board defect detection system based on machine learning

Abstract

The invention belongs to the technical field of circuit boards, and particularly relates to a circuit board defect detection system based on machine learning. The method comprises the following steps of: 1, acquiring multi-modal data of a circuit board through various sensors, transmitting the data to a data fusion module, fusing and outputting a unified multi-modal data set; 2, the fused data are transmitted into an adaptive deep convolutional neural network, features in the image are extracted through a deep learning model, and preliminary defect detection is carried out; 3, comparing a result after each detection with an actual defect label, calculating an error and feeding back the error, adjusting a feedback mechanism through a recurrent neural network, and gradually optimizing model parameters after each detection; step 4, after the detection is completed, classifying the defects by using an ensemble learning algorithm, and outputting a final detection result; and 5, continuously collecting new data by the system in the production process and optimizing the model according to real-time feedback.

Description

Technical Field

[0001] The present invention relates to the technical field of circuit boards, and in particular to a circuit board defect detection system based on machine learning. Background Art

[0002] With the continuous development of electronic products, the quality of circuit boards, as core electronic components, directly affects the performance and reliability of products. Circuit board defects usually manifest as welding defects, open circuits, short circuits, and solder leaks. Any small defect may lead to failure of the entire product. Therefore, timely and accurate detection and repair of circuit board defects is crucial to ensuring product quality.

[0003] Traditional circuit board defect detection systems have the following disadvantages:

[0004] 1. At present, many automated inspection methods mainly rely on image processing technology to identify defects in circuit boards, especially traditional visual inspection systems. These systems capture images of circuit boards through cameras and use image processing algorithms to detect defects. However, these methods often have some limitations, such as high sensitivity to lighting changes and background noise, difficulty in identifying tiny defects, and poor recognition of defects with complex shapes;

[0005] 2. With the rapid development of deep learning technology, many studies have begun to try to apply deep learning to circuit board defect detection. Deep neural networks (DNN) and convolutional neural networks (CNN) have been proven to effectively extract features from images and perform efficient defect classification. However, existing deep learning methods still have some problems, such as overfitting, imbalanced training data, and low detection accuracy for some defects. In addition, these methods usually rely on a single image modality and lack the comprehensive use of multimodal data, which limits their robustness and detection capabilities;

[0006] 3. In recent years, some studies have attempted to combine multi-modal data (such as infrared images, X-ray images, etc.) with traditional RGB images. By fusion of multi-modal data, the advantages of different sensors can be utilized to improve the detection accuracy of the system. However, the existing multi-modal data fusion methods have not yet achieved sufficient adaptability and still have deficiencies when dealing with detection tasks under complex working conditions.

[0007] In response to the above-mentioned technical defects, a circuit board defect detection system solution based on machine learning is proposed. Summary of the invention

[0008] To solve the above problems, the present invention provides the following technical solutions:

[0009] A circuit board defect detection system based on machine learning, including an image acquisition and multimodal data fusion module, an adaptive deep convolutional neural network module, a combination module of a recurrent neural network and a generative adversarial network, a cyclic repetitive detection and optimization mechanism module, a defect recognition and classification module, an online real-time detection and adaptive optimization module, and an implementation plan and effect evaluation module;

[0010] The image acquisition and multimodal data fusion module includes multiple sensors, and collects circuit board image data through the multiple sensors to form a multimodal data set;

[0011] The adaptive deep convolutional neural network module includes an adaptive pooling layer and a multi-scale convolutional layer. Based on the fusion result of the multimodal image data, an adaptive deep convolutional neural network is used to extract features from the image of the image acquisition and multimodal data fusion module, that is, the convolution kernel and the pooling strategy are automatically adjusted according to the illumination change and resolution data of the input image, and the size of the convolution kernel and the pooling strategy are dynamically adjusted according to the resolution and illumination change conditions of the image, thereby improving the defect detection capability of the circuit board;

[0012] The combination module of the recurrent neural network and the generative adversarial network includes a recurrent neural network unit and a generative adversarial network unit; a recurrent neural network is introduced based on an adaptive deep convolutional neural network module, and the system's defect prediction capability in a dynamic change process is enhanced by modeling the temporal relationship of the image sequence. The system also introduces a generative adversarial network to generate various circuit board defect images, thereby enriching the training data set and improving the generalization ability of the model.

[0013] The loop detection and optimization mechanism module inputs the data information detected by the combination module of the recurrent neural network and the generative adversarial network into the loop detection and optimization mechanism module, and continuously feeds back and optimizes the model parameters through the loop detection and optimization mechanism module, thereby improving the detection accuracy through multiple rounds of iterations;

[0014] The defect recognition and classification module is based on an adaptive deep convolutional neural network module, a combination module of a recurrent neural network and a generative adversarial network, and a cyclic repeated detection and optimization mechanism module. In combination with the results of multiple rounds of repeated detection, the system uses an integrated learning algorithm to classify defects, and uses multiple classifiers in the defect recognition and classification module for fusion to identify and classify defects detected on the circuit board.

[0015] The online real-time detection and adaptive optimization module detects circuit board defects in real time based on the data of the defect recognition and classification module and performs real-time feedback optimization on the model;

[0016] The implementation plan and effect evaluation module: use multimodal data for real-time detection and classification, and the feedback mechanism after each iteration can gradually optimize the model.

[0017] Furthermore, the image acquisition and multimodal data fusion module includes multiple sensors (RGB camera, infrared camera, X-ray imaging device, 3D depth camera) to collect circuit board image data, and the collected data are recorded separately to form a multimodal data set, and the multimodal data set is fused as:

[0018] where represents the fused multimodal dataset, ω i is the weighting coefficient of each mode, D i is the i-th modal data.

[0019] Furthermore, the adaptive deep convolutional neural network module inputs the image after multimodal data fusion into the adaptive deep convolutional neural network, which has an adaptive pooling and multi-scale convolution layer structure, and dynamically adjusts the convolution kernel and pooling strategy according to the resolution and illumination change conditions of the input image. The adaptive pooling operation is performed by the following formula:

[0020] F adaptive =AdaptivePooling(F)

[0021] The one is the feature map after the convolution operation, and the one is the result after adaptive pooling.

[0022] Furthermore, based on improving the time series prediction ability of defect detection, the combination module of the recurrent neural network and the generative adversarial network includes a recurrent neural network (RNN) and a generative adversarial network (GAN). The system combines the recurrent neural network and the generative adversarial network. The recurrent neural network (RNN) enhances the model's adaptability to dynamic changes by processing the temporal information of the image sequence. The generative adversarial network is used to generate defect samples, expand the training data set, and improve the generalization ability of the model. The optimization objectives of the generative network G and the discriminative network D of the generative adversarial network are:

[0023]

[0024] Where G represents the generative network, D represents the discriminative network, x represents the real data, z represents random noise (used to generate samples), represents the data distribution, represents the noise distribution, and represents the expected value.

[0025] Furthermore, the analysis process of the loop repetition detection and optimization mechanism module is as follows:

[0026] After each round of detection, the system updates the model weights through an adaptive feedback adjustment mechanism to reduce detection errors. The output of each detection is compared with the true annotation, the error is calculated, and the model parameters are adjusted inversely according to the error:

[0027]

[0028] Where represents the error of the k-th round of detection, represents the number of samples, represents the true labeled value of the ith sample, and represents the predicted value of the ith sample.

[0029] Furthermore, the error feedback update analysis process of the cyclic repetition detection and optimization mechanism module is as follows:

[0030] After the error feedback is updated, the system will enter the next round of detection until the error reaches the predetermined stop condition:

[0031] if E k -E k-1 <∈or E k -E min then stop iteration

[0032] Where represents the error of the k-th round of detection, represents the error of the previous round of detection, is the error tolerance threshold, and is the minimum error threshold.

[0033] Furthermore, the defect recognition and classification module uses an integrated learning algorithm to classify defects based on multiple rounds of repeated detection results. At the same time, the defect recognition and classification module includes multiple classifiers (such as support vector machine SVM, random forest RF, deep neural network DNN) to jointly determine the final defect category to improve the accuracy of classification. The final detection result is determined by a weighted integration method:

[0034]

[0035] Where represents the final detection result, M represents the number of classifiers, represents the weighting coefficient of the k-th classifier, and represents the final prediction result of the k-th classifier.

[0036] Furthermore, the online real-time detection and adaptive optimization module analysis process includes the following:

[0037] The online real-time detection and adaptive optimization module performs real-time detection through an embedded processing platform. After each detection, the system continuously adjusts the model weights according to the feedback information. The deep feedback mechanism and incremental learning strategy enable the system to dynamically adapt to new defect types and improve detection accuracy through adaptive optimization.

[0038] Furthermore, the implementation plan and effect evaluation module use multimodal data for real-time detection and classification. The feedback mechanism after each iteration can gradually optimize the model, improve the accuracy of detection, and reduce the occurrence of false detections and missed detections. Through continuous optimization and incremental learning, the system can adapt to different production environments and changing defect types.

[0039] Furthermore, according to the machine learning-based circuit board defect detection system of claim 9, it is characterized in that the steps of using the machine learning-based circuit board defect detection method of any one of claims 1 to 9 are as follows:

[0040] Step 1: First, the multimodal data of the circuit board is collected through various sensors. These data are transmitted to the data fusion module, fused and output as a unified multimodal data set;

[0041] Step 2: The fused data is passed to the adaptive deep convolutional neural network, and the features in the image are extracted through the deep learning model, and preliminary defect detection is performed. The system performs adaptive optimization based on the detection results, and adjusts the convolution kernel and pooling strategy to cope with different production environments;

[0042] Step 3: The results after each inspection will be compared with the actual defect annotations, the error will be calculated and feedback will be given. The feedback mechanism will be adjusted through the recurrent neural network to ensure that the next round of inspection is more accurate. After each round of inspection, the model parameters will be gradually optimized to reduce the error and improve the accuracy of the inspection;

[0043] Step 4: After the inspection is completed, the defects are classified using an integrated learning algorithm and the final inspection results are output;

[0044] Step 5: The system continuously collects new data during the production process and optimizes the model based on real-time feedback to ensure that the system can adapt to new production conditions and defect types and maintain high-precision and high-robustness detection capabilities.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. In a circuit board defect detection system based on machine learning of the present invention, the system, with the support of multimodal data, can process data from different sensors in real time, quickly identify various types of defects, and maintain high accuracy under the conditions of noise and environmental changes;

[0047] 2. In a circuit board defect detection system based on machine learning of the present invention, the error feedback after each detection can gradually optimize the system, adapt to changes in the production process, and continuously improve the accuracy of defect detection;

[0048] 3. In a circuit board defect detection system based on machine learning, the present invention reduces the need for manual intervention and reduces the error and missed detection rate in manual detection through automated detection and feedback optimization;

[0049] 4. In a circuit board defect detection system based on machine learning of the present invention, the system can monitor the production line in real time, respond quickly, and immediately provide detection results, supporting real-time quality control in the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with the accompanying drawings;

[0051] Figure 1 This is an overall flow chart of a circuit board defect detection system based on machine learning in the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] like Figure 1 As shown, a circuit board defect detection system based on machine learning includes an image acquisition and multimodal data fusion module, an adaptive deep convolutional neural network module, a combination module of a recurrent neural network and a generative adversarial network, a cyclic repetitive detection and optimization mechanism module, a defect recognition and classification module, an online real-time detection and adaptive optimization module, and an implementation plan and effect evaluation module;

[0054] The image acquisition and multimodal data fusion module includes multiple sensors, which collect circuit board image data through multiple sensors to form a multimodal data set;

[0055] The adaptive deep convolutional neural network module includes an adaptive pooling layer and a multi-scale convolutional layer. Based on the fusion results of multimodal image data, an adaptive deep convolutional neural network is used to extract features from the images of the image acquisition and multimodal data fusion modules. That is, the convolution kernel and pooling strategy are automatically adjusted according to the illumination changes and resolution data of the input image. At the same time, the size of the convolution kernel and the pooling strategy are dynamically adjusted based on the resolution and illumination change conditions of the image, thereby improving the defect detection capability of the circuit board;

[0056] The combination module of recurrent neural network and generative adversarial network includes recurrent neural network unit and generative adversarial network unit. The recurrent neural network is introduced based on the adaptive deep convolutional neural network module. By modeling the temporal relationship of the image sequence, the system's defect prediction ability in the dynamic change process is enhanced. The system also introduces a generative adversarial network to generate various circuit board defect images, thereby enriching the training data set and improving the generalization ability of the model.

[0057] The loop detection and optimization mechanism module inputs the data information detected by the combination module of the recurrent neural network and the generative adversarial network into the loop detection and optimization mechanism module, and continuously feeds back and optimizes the model parameters through the loop detection and optimization mechanism module, thereby improving the detection accuracy through multiple rounds of iterations;

[0058] The defect recognition and classification module is based on the adaptive deep convolutional neural network module, the combination module of the recurrent neural network and the generative adversarial network, and the cyclic repeated detection and optimization mechanism module. In combination with the results of multiple rounds of repeated detection, the system uses an integrated learning algorithm to classify defects and uses multiple classifiers in the defect recognition and classification module to fuse and identify and classify defects detected in circuit boards.

[0059] The online real-time detection and adaptive optimization module detects circuit board defects in real time based on the data from the defect recognition and classification module and performs real-time feedback optimization on the model;

[0060] Implementation plan and effect evaluation module: Use multimodal data for real-time detection and classification, and the feedback mechanism after each iteration can gradually optimize the model.

[0061] Specifically, the image acquisition and multimodal data fusion module includes a variety of sensors (RGB camera, infrared camera, X-ray imaging equipment, 3D depth camera) to collect circuit board image data, and record the collected data separately to form a multimodal data set, which is fused into:

[0062] where represents the fused multimodal dataset, ω i is the weighting coefficient of each mode, D i is the i-th modal data.

[0063] Common sensors include RGB cameras, infrared cameras, X-ray imaging devices, and 3D depth cameras. The data collected by each sensor represents information at different levels of the circuit board, forming a multimodal data set;

[0064] RGB camera: captures the appearance image of the circuit board, mainly used to identify visible defects on the surface, such as welding defects, open circuits, short circuits, etc.

[0065] Infrared camera: used to detect thermal imaging information in the circuit board, which can effectively identify abnormal temperature areas caused by defects;

[0066] X-ray imaging equipment: used to obtain information about the internal structure of the circuit board, especially suitable for detecting deep defects that are not visible on the surface, such as solder leaks and cold solder joints;

[0067] 3D depth camera: provides three-dimensional topographic information of the circuit board surface, helping the system identify complex geometric defects;

[0068] The data from these different sources are combined to form a unified multimodal data set system through weighted fusion. The weighting coefficient ω of each modal data is adaptively adjusted according to the working environment and defect type of different production lines. i , and dynamically optimize through the following steps:

[0069] Weighting coefficient adjustment: Use online learning algorithms to dynamically adjust the weight coefficients of each modal data;

[0070] Data fusion: By weighted fusion of features from different modal data, a comprehensive and robust feature map is generated, enabling the system to effectively detect complex defects in different environments.

[0071] Specifically, the adaptive deep convolutional neural network module (ADCNN) inputs the image after multimodal data fusion into the adaptive deep convolutional neural network, which has an adaptive pooling and multi-scale convolution layer structure. The convolution kernel and pooling strategy are dynamically adjusted according to the resolution and illumination change conditions of the input image. The adaptive pooling operation is performed by the following formula:

[0072] F adaptive =AdaptivePooling(F)

[0073] The one is the feature map after the convolution operation, and the one is the result after adaptive pooling.

[0074] The fused multimodal data is input into the adaptive deep convolutional neural network (ADCNN) for processing. In this network, the convolution layer and pooling layer can be dynamically adjusted according to the resolution, illumination changes and other conditions of different input images. By introducing adaptive pooling and multi-scale convolution strategies, the system can effectively extract features with high robustness.

[0075] Adaptive pooling layer: In view of the characteristics of images with different resolutions and multimodal data, the system automatically adjusts the pooling size and stride through adaptive pooling operations to retain more detailed information and improve defect detection accuracy;

[0076] Multi-scale convolutional layer: The multi-scale convolutional layer structure can effectively process defects of different scales and sizes and improve the detection ability of various defects. Each convolutional layer extracts deep information of the image by learning local and global features of the data;

[0077] Data enhancement: Use data enhancement techniques (such as rotation, scaling, shearing, etc.) to generate diverse training samples to improve the generalization ability of the model:

[0078] Adaptive optimization: During the training process, the model can automatically adjust the learning rate and weights through an adaptive optimization algorithm, making the training process more efficient.

[0079] Combination of recurrent neural network and generative adversarial network (GAN): In the system of the present invention, recurrent neural network (RNN) and generative adversarial network (GAN) are used in combination to enhance the model's adaptability to timing changes and diversity of defect samples;

[0080] Recurrent Neural Network (RNN): RNN can process the temporal information in the image sequence, which is particularly important for dealing with defects that gradually appear in the production line (such as welding defects). The system enhances the recognition ability of dynamic defects by analyzing the changes in the continuous image sequence;

[0081] Generative Adversarial Network (GAN): GAN is used to generate new defect samples and expand the training data set. Especially when there are insufficient samples, it can enhance the generalization ability of the model by generating high-quality virtual defect samples. The optimization goal of the generative network G and the discriminative network D is to achieve the best effect of defect sample generation and recognition by minimizing the loss function.

[0082] GAN's generative network helps the system expand the training set when data is insufficient by generating various hypothetical defect samples, thereby effectively improving the robustness and accuracy of the model.

[0083] Specifically, based on improving the time series prediction ability of defect detection, the combination module of recurrent neural network and generative adversarial network includes recurrent neural network (RNN) and generative adversarial network (GAN). The system combines recurrent neural network and generative adversarial network. The recurrent neural network (RNN) enhances the model's adaptability to dynamic changes by processing the temporal information of the image sequence. The generative adversarial network is used to generate defect samples, expand the training data set, and improve the generalization ability of the model. The optimization objectives of the generative network G and the discriminative network D of the generative adversarial network are:

[0084] L GAN (G,D)=Ε x～Pdata(x) [logD(x)]+Ε z～Pz(z) [log(1-D(G(z)))]

[0085] Where G represents the generative network, D represents the discriminative network, x represents the real data, z represents random noise (used to generate samples), represents the data distribution, represents the noise distribution, and represents the expected value.

[0086] Specifically, the analysis process of the loop detection and optimization mechanism module is as follows:

[0087] After each round of detection, the system updates the model weights through an adaptive feedback adjustment mechanism to reduce detection errors. The output of each detection is compared with the true annotation, the error is calculated, and the model parameters are adjusted inversely according to the error:

[0088]

[0089] Where represents the error of the k-th round of detection, represents the number of samples, represents the true labeled value of the ith sample, and represents the predicted value of the ith sample.

[0090] Specifically, the error feedback update analysis process of the cyclic repeated detection and optimization mechanism module is as follows: After the error feedback is updated, the system will enter the next round of detection until the error reaches the predetermined stop condition:

[0091] if E k -E k-1 <∈or E k -E min then stop iteration

[0092] Where represents the error of the k-th round of detection, represents the error of the previous round of detection, is the error tolerance threshold, and is the minimum error threshold.

[0093] After each round of detection, the system will compare with the real annotation, calculate the error and adjust the model parameters according to the error. This cyclic detection process will continue until the error converges to a predetermined threshold. This mechanism ensures that the system can continuously optimize and improve the detection accuracy;

[0094] Error feedback mechanism: After each round of detection, the system calculates the error and compares it with the error of the previous round, and makes feedback adjustments. If the error change is less than the threshold or the error is less than the minimum error, the iteration stops;

[0095] Adaptive adjustment mechanism: During the cyclic detection process, the system will adaptively adjust the model weights based on real-time feedback to improve the accuracy of the detection results.

[0096] Specifically, in the defect recognition and classification module, based on multiple rounds of repeated detection results, the system uses an integrated learning algorithm to classify defects. At the same time, the defect recognition and classification module includes multiple classifiers (such as support vector machine SVM, random forest RF, deep neural network DNN) to jointly determine the final defect category to improve the accuracy of classification. The final detection result is determined by a weighted integration method:

[0097]

[0098] Where represents the final detection result, M represents the number of classifiers, represents the weighting coefficient of the k-th classifier, and represents the final prediction result of the k-th classifier.

[0099] Specifically, the online real-time detection and adaptive optimization module analysis process includes the following:

[0100] The online real-time detection and adaptive optimization module performs real-time detection through an embedded processing platform. After each detection, the system continuously adjusts the model weights based on feedback information. The deep feedback mechanism and incremental learning strategy enable the system to dynamically adapt to new defect types and improve detection accuracy through adaptive optimization.

[0101] The system is deployed in real time on the embedded processing platform of the production line to perform online real-time defect detection. After each inspection of the production line is completed, the system will adjust the model weights according to the feedback information of the current inspection results, thereby ensuring high accuracy and high robustness in different production environments.

[0102] Specifically, the implementation plan and effect evaluation module uses multimodal data for real-time detection and classification. The feedback mechanism after each iteration can gradually optimize the model, improve the accuracy of detection, and reduce the occurrence of false detections and missed detections. Through continuous optimization and incremental learning, the system can adapt to different production environments and changing defect types.

[0103] The defect detection system of the present invention can provide high-precision and high-robust defect detection under different working conditions, and is particularly excellent in detecting tiny defects and defects with complex shapes. Through continuous optimization and incremental learning, the system gradually adapts to new defect types, which can effectively reduce false detections and missed detections, and ensure production quality.

[0104] Specifically, according to a machine learning-based circuit board defect detection system according to claim 9, it is characterized in that the steps of a machine learning-based circuit board defect detection method according to any one of claims 1 to 9 are as follows:

[0105] Step 1: First, the multimodal data of the circuit board is collected through various sensors. These data are transmitted to the data fusion module, fused and output as a unified multimodal data set;

[0106] Step 2: The fused data is passed to the adaptive deep convolutional neural network, and the features in the image are extracted through the deep learning model, and preliminary defect detection is performed. The system performs adaptive optimization based on the detection results, and adjusts the convolution kernel and pooling strategy to cope with different production environments;

[0107] Step 3: The results after each inspection will be compared with the actual defect annotations, the error will be calculated and feedback will be given. The feedback mechanism will be adjusted through the recurrent neural network to ensure that the next round of inspection is more accurate. After each round of inspection, the model parameters will be gradually optimized to reduce the error and improve the accuracy of the inspection;

[0108] Step 4: After the inspection is completed, the defects are classified using an integrated learning algorithm and the final inspection results are output;

[0109] Step 5: The system continuously collects new data during the production process and optimizes the model based on real-time feedback to ensure that the system can adapt to new production conditions and defect types and maintain high-precision and high-robustness detection capabilities.

[0110] Working principle: The working principle of the present invention is based on the combination of deep learning, recurrent neural network (RNN), generative adversarial network (GAN) and multimodal data fusion. The system collects circuit board images through a variety of sensors (RGB, infrared, X-ray, 3D depth camera, etc.) and performs multimodal data fusion. The fused data is input into the adaptive deep convolutional neural network (ADCNN). Through multi-layer convolution and pooling processing, the feature information in the circuit board is extracted. The network can adaptively adjust the convolution kernel and pooling strategy to adapt to different working conditions and defect types. The detection results are optimized through the generative adversarial network (GAN) to generate new defect samples to expand the data set and improve the generalization ability of the model. At the same time, the system uses a recurrent neural network (RNN) to analyze timing information and enhance the model's adaptability to dynamic defects. After each round of detection, the model parameters are adjusted through an error feedback mechanism, and the defects are classified based on an integrated learning algorithm. Through multiple rounds of repeated detection, the system can be optimized in real time in a production environment to ensure high-precision and robust defect detection effects.

[0111] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A circuit board defect detection system based on machine learning, characterized in that: It includes image acquisition and multimodal data fusion module, adaptive deep convolutional neural network module, combination module of recurrent neural network and generative adversarial network, cyclic repetitive detection and optimization mechanism module, defect recognition and classification module, online real-time detection and adaptive optimization module, and implementation plan and effect evaluation module. The image acquisition and multimodal data fusion module includes multiple sensors, and collects circuit board image data through the multiple sensors to form a multimodal data set; The adaptive deep convolutional neural network module includes an adaptive pooling layer and a multi-scale convolutional layer. Based on the fusion result of the multimodal image data, an adaptive deep convolutional neural network is used to extract features from the image of the image acquisition and multimodal data fusion module, that is, the convolution kernel and the pooling strategy are automatically adjusted according to the illumination change and resolution data of the input image, and the size of the convolution kernel and the pooling strategy are dynamically adjusted according to the resolution and illumination change conditions of the image, thereby improving the defect detection capability of the circuit board; The combination module of the recurrent neural network and the generative adversarial network includes a recurrent neural network unit and a generative adversarial network unit; a recurrent neural network is introduced based on an adaptive deep convolutional neural network module, and the system's defect prediction capability in a dynamic change process is enhanced by modeling the temporal relationship of the image sequence. The system also introduces a generative adversarial network to generate various circuit board defect images, thereby enriching the training data set and improving the generalization ability of the model. The loop detection and optimization mechanism module inputs the data information detected by the combination module of the recurrent neural network and the generative adversarial network into the loop detection and optimization mechanism module, and continuously feeds back and optimizes the model parameters through the loop detection and optimization mechanism module, thereby improving the detection accuracy through multiple rounds of iterations; The defect recognition and classification module is based on an adaptive deep convolutional neural network module, a combination module of a recurrent neural network and a generative adversarial network, and a cyclic repeated detection and optimization mechanism module. In combination with the results of multiple rounds of repeated detection, the system uses an integrated learning algorithm to classify defects, and uses multiple classifiers in the defect recognition and classification module for fusion to identify and classify defects detected on the circuit board. The online real-time detection and adaptive optimization module detects circuit board defects in real time based on the data of the defect recognition and classification module and performs real-time feedback optimization on the model; The implementation plan and effect evaluation module: use multimodal data for real-time detection and classification, and the feedback mechanism after each iteration can gradually optimize the model.

2. A circuit board defect detection system based on machine learning according to claim 1, characterized in that: The image acquisition and multimodal data fusion module includes a variety of sensors (RGB camera, infrared camera, X-ray imaging equipment, 3D depth camera) to collect circuit board image data, and the collected data is recorded as D RGB , D IR , D X-ray , D 3D , and form a multimodal dataset, the multimodal dataset D = {D RGB ,D IR ,D X-ray ,D 3D } and merged into: where represents the fused multimodal dataset, ω i is the weighting coefficient of each mode, D i is the i-th modal data.

3. A circuit board defect detection system based on machine learning according to claim 2, characterized in that: The adaptive deep convolutional neural network module inputs the image after multimodal data fusion into the adaptive deep convolutional neural network. The network has an adaptive pooling and multi-scale convolution layer structure. The convolution kernel and pooling strategy are dynamically adjusted according to the resolution of the input image and the illumination change conditions to improve the defect detection capability. The adaptive pooling operation is performed by the following formula: F adaptive =AdaptivePooling(F) Where F is the feature map after the convolution operation, F adaptive This is the result after adaptive pooling.

4. A circuit board defect detection system based on machine learning according to claim 3, characterized in that: Based on improving the time series prediction ability of defect detection, the combination module of the recurrent neural network and the generative adversarial network includes a recurrent neural network (RNN) and a generative adversarial network (GAN). The system combines the recurrent neural network and the generative adversarial network. The recurrent neural network (RNN) enhances the model's adaptability to dynamic changes by processing the temporal information of the image sequence. The generative adversarial network is used to generate defect samples, expand the training data set, and improve the generalization ability of the model. The optimization objectives of the generative network G and the discriminative network D of the generative adversarial network are: Where G represents the generative network, D represents the discriminative network, x represents the real data, z represents random noise (used to generate samples), represents the data distribution, represents the noise distribution, and represents the expected value.

5. A circuit board defect detection system based on machine learning according to claim 4, characterized in that: The analysis process of the loop repetition detection and optimization mechanism module is as follows: After each round of detection, the system updates the model weights through an adaptive feedback adjustment mechanism to reduce the detection error. The output of each detection is compared with the true annotation, and the calculated error E k , and adjust the model parameters inversely according to the error: Where represents the error of the k-th round of detection, represents the number of samples, represents the true labeled value of the ith sample, and represents the predicted value of the ith sample.

6. A circuit board defect detection system based on machine learning according to claim 5, characterized in that: The error feedback update analysis process of the cyclic repetition detection and optimization mechanism module is as follows: After the error feedback is updated, the system will enter the next round of detection until the error reaches the predetermined stop condition: if E k -E k-1 <∈or E k -E min then stop iteration where represents the error of the kth round of detection, represents the error of the previous round of detection, ∈ is the error tolerance threshold, and E min is the minimum error threshold.

7. A circuit board defect detection system based on machine learning according to claim 6, characterized in that: The defect recognition and classification module uses an integrated learning algorithm to classify defects based on multiple rounds of repeated detection results. At the same time, the defect recognition and classification module includes multiple classifiers (such as support vector machine SVM, random forest RF, deep neural network DNN) to jointly determine the final defect category to improve the accuracy of classification. The final detection result is determined by a weighted integration method: Where represents the final detection result, M represents the number of classifiers, represents the weighting coefficient of the k-th classifier, and represents the final prediction result of the k-th classifier.

8. The circuit board defect detection system based on machine learning according to claim 7, characterized in that: The online real-time detection and adaptive optimization module analysis process includes the following: The online real-time detection and adaptive optimization module performs real-time detection through an embedded processing platform. After each detection, the system continuously adjusts the model weights according to the feedback information. The deep feedback mechanism and incremental learning strategy enable the system to dynamically adapt to new defect types and improve detection accuracy through adaptive optimization.

9. A circuit board defect detection system based on machine learning according to claim 8, characterized in that: The implementation plan and effect evaluation module use multimodal data for real-time detection and classification. The feedback mechanism after each iteration can gradually optimize the model, improve the accuracy of detection, and reduce the occurrence of false detections and missed detections. Through continuous optimization and incremental learning, the system can adapt to different production environments and changing defect types.

10. A circuit board defect detection system based on machine learning according to claim 9, characterized in that: The steps of the circuit board defect detection method based on machine learning described in any one of claims 1 to 9 are as follows: Step 1: First, the multimodal data of the circuit board is collected through various sensors. These data are transmitted to the data fusion module, fused and output as a unified multimodal data set; Step 2: The fused data is passed to the adaptive deep convolutional neural network, and the features in the image are extracted through the deep learning model, and preliminary defect detection is performed. The system performs adaptive optimization based on the detection results, and adjusts the convolution kernel and pooling strategy to cope with different production environments; Step 3: The results after each inspection will be compared with the actual defect annotations, the error will be calculated and feedback will be given. The feedback mechanism will be adjusted through the recurrent neural network to ensure that the next round of inspection is more accurate. After each round of inspection, the model parameters will be gradually optimized to reduce the error and improve the accuracy of the inspection; Step 4: After the inspection is completed, the defects are classified using an integrated learning algorithm and the final inspection results are output; Step 5: The system continuously collects new data during the production process and optimizes the model based on real-time feedback to ensure that the system can adapt to new production conditions and defect types and maintain high-precision and high-robustness detection capabilities.

Images