Printed circuit board defect detection system

Abstract

The invention discloses a printed circuit board defect detection system. The image acquisition module is used for acquiring a multispectral image of a printed circuit board to be detected, and the image preprocessing module is used for denoising, enhancing and normalizing the acquired image. The feature extraction module extracts geometric and spectral information from the preprocessed image, including the shape and layout structure of the circuit. The deep learning module processes the extracted preliminary geometric and spectral information, and the basic network sub-module performs feature extraction through a pre-trained convolutional neural network to generate high-level features. And the self-adaptive adjustment sub-module carries out fine adjustment on the high-level features through defect sample data with labels, and generates specific features for identifying and classifying various defects of the printed circuit board. The defect identification and classification sub-module combines high-level features and specific features to realize accurate detection of short circuit, open circuit, insufficient etching, welding defects and the like. And the visualization module is used for marking the position and the type of the defect and generating a detection report.

Description

Technical Field

[0001] The present invention relates to the technical field of printed circuit boards, and particularly to a printed circuit board defect detection system. Background Art

[0002] In the prior art, the defect detection of printed circuit boards (PCBs) usually relies on manual inspection, optical imaging or traditional rule-based algorithms. The optical detection system uses images to obtain the appearance information of the circuit board and analyzes possible defects through specific algorithms. Deep learning technology has also gradually been applied to the defect detection of printed circuit boards, using convolutional neural networks (CNNs) for feature extraction to identify different types of defects. These systems can improve the detection efficiency and accuracy to a certain extent.

[0003] However, there are still some problems with the existing defect detection technologies. For example, traditional rule-based methods are easily limited by complex circuit structures and cannot flexibly handle different types of defects. At the same time, existing deep learning systems rely on large-scale labeled datasets and it is difficult to achieve efficient defect detection with limited samples. In addition, the detection results of defects and the visual presentation are usually separated, which limits the operator's intuitive understanding of the defect location and type.

[0004] Therefore, there is an urgent need to develop a new type of printed circuit board defect detection system. Summary of the Invention

[0005] The present application provides a printed circuit board defect detection system to improve the accuracy and efficiency of printed circuit board defect detection.

[0006] The present application provides a printed circuit board defect detection system, including:

[0007] An image acquisition module for acquiring multi-spectral images of the printed circuit board to be detected;

[0008] An image preprocessing module connected to the image acquisition module for denoising, enhancing and normalizing the multi-spectral images to improve the image quality;

[0009] A feature extraction module connected to the image preprocessing module for extracting preliminary geometric and spectral information from the preprocessed images, wherein the geometric and spectral information includes the shape, layout structure of the circuit and specific spectral characteristics;

[0010] A deep learning module connected to the feature extraction module for processing the extracted preliminary geometric and spectral information; wherein, the deep learning module includes:

[0011] The basic network sub-module is used to perform feature extraction on the preliminary geometric and spectral information extracted by the feature extraction module using a pre-trained convolutional neural network to obtain high-level features, where the high-level features include specific circuit patterns and connection relationships in the printed circuit board;

[0012] The adaptive adjustment sub-module is connected to the basic network sub-module and uses the labeled printed circuit board defect sample data to fine-tune the high-level features to generate specific features that can reflect the circuit board defects, which are used to further accurately identify and classify various defects on the printed circuit board;

[0013] The defect recognition and classification sub-module is used to combine the high-level features provided by the basic network sub-module and the specific features provided by the adaptive adjustment sub-module to perform defect recognition and classification of the printed circuit board, where the defect types of the printed circuit board include short circuit, open circuit, insufficient etching, and soldering defects;

[0014] The visualization module is connected to the deep learning module and is used to label the detected defect positions and types on the original image and generate a report including the detection results.

[0015] Furthermore, the image acquisition module includes multiple spectral cameras for respectively acquiring multi-spectral images within different wavelength ranges; where the different wavelengths include visible light, near-infrared, and ultraviolet light bands.

[0016] Furthermore, the image preprocessing module includes a denoising processing unit, an image enhancement unit, and a normalization processing unit, where:

[0017] The denoising processing unit uses a wavelet transform-based denoising algorithm to filter out random noise in the multi-spectral image;

[0018] The image enhancement unit uses an adaptive histogram equalization algorithm to improve the contrast of the image;

[0019] The normalization processing unit normalizes the image to eliminate the influence of brightness differences on subsequent analysis.

[0020] Furthermore, the feature extraction module uses a Hough transform algorithm based on edge detection and Fourier transform to extract the geometric shape features, connection patterns, and spectral features of the printed circuit board.

[0021] Furthermore, the basic network sub-module uses a pre-trained convolutional neural network model and extracts the high-level features of the printed circuit board through the ResNet50 network architecture pre-trained on the ImageNet dataset.

[0022] Furthermore, the adaptive adjustment sub-module fine-tunes the pre-trained convolutional neural network by using the labeled printed circuit board defect sample data, and uses the following formula to fine-tune the high-level feature parameters of the basic network sub-module:

[0023]

[0024] where, θ represents the adjustable weight parameter of the basic network; θ * represents the optimal parameter after fine-tuning; N is the number of training samples; x i represents the input of the i-th sample; y i represents the label of the i-th sample; L(f θ (x i ),y i ) is the loss function, which is used to evaluate the difference between the prediction result L(f θ (x i ),y i ) and the actual label y i . The loss function is implemented by using cross-entropy loss; α is the weight parameter; t i represents the timestamp of the sample x i ; t ref represents the reference time; T max represents the maximum time span; T is the number of Monte Carlo samplings; represents the model output of the t-th forward propagation after activating Dropout; represents the mean value of the output after forward propagation; β is the weight parameter of the regularization term; θ k is the k-th high-level feature parameter; K is the total number of high-level feature parameters; |θ k | p is the p-norm of the parameter θ k ; λ t is a parameter that changes dynamically with time; γ is the weight parameter; L represents the number of samples in the dataset or a specific batch; represents the gradient of the neural network model f θ with respect to the input data x i .

[0025] Furthermore, the defect recognition and classification sub-module uses the features output by the convolutional neural network to train a support vector machine classifier for classifying printed circuit board defects; the support vector machine classifier is used to accurately classify the defects into short circuits, open circuits, insufficient etching, and welding defects according to the input high-level features and specific features.

[0026] Furthermore, the visualization module includes a 3D image processing unit for highlighting the detected defect positions on the original image with different colors, displaying different color codes according to the defect types, and generating a detection report in PDF format, which includes the exact positions, sizes, and types of the defects.

[0027] Furthermore, the visualization module includes a heat map generation unit for generating a visualized heat map based on the types and densities of the defects. The heat map marks the defect-dense areas with highlighted colors and shows the distribution of the defects through color gradients, thus intuitively providing the defect concentration information of different areas on the circuit board.

[0028] Furthermore, the visualization module includes an interactive user interface that allows users to dynamically view the detection results of the printed circuit board through zooming, rotating, and panning functions, and supports users to click on the marked points in the image to display the detailed information of the corresponding defects. The detailed information includes the defect type, position coordinates, and recommended repair solutions.

[0029] The beneficial effects of this application mainly include: (1) By introducing the acquisition of multi-spectral images and the denoising, enhancement, and normalization operations of the image preprocessing module, the system can provide high-quality input images, which greatly improves the accuracy of defect recognition by the subsequent deep learning module, especially in complex circuit board structures. (2) Through the adaptive adjustment sub-module, a small amount of labeled printed circuit board defect sample data is used to fine-tune the pre-trained high-level features, enabling the system to adapt to different circuit board defect detection tasks and effectively identify various types of defects, including short circuits, open circuits, insufficient etching, and soldering defects. (3) By combining the pre-trained convolutional neural network (basic network sub-module) with the adaptive adjustment module, the system can achieve effective defect detection under limited sample conditions, reducing the need for large-scale labeled data sets and improving the application flexibility of the system. (4) The visualization module can intuitively mark the positions and types of defects on the original image and generate a report to help the operator quickly understand the detection results, facilitate subsequent processing and decision-making, and improve the efficiency of the detection process. Description of the Drawings

[0030] Figure 1 is a schematic diagram of a printed circuit board defect detection system provided by the first embodiment of this application. Detailed Embodiments

[0031] Many specific details are set forth in the following description in order to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of this application. Therefore, this application is not limited by the specific embodiments disclosed below.

[0032] The first embodiment of the present application provides a printed circuit board defect detection system. Please refer to Figure 1 , which is a schematic diagram of the first embodiment of the present application. The following will be combined with Figure 1 to provide a detailed description of a printed circuit board defect detection system according to the first embodiment of the present application.

[0033] The printed circuit board defect detection system includes an image acquisition module 101, an image preprocessing module 102, a feature extraction module 103, a deep learning module 104, and a visualization module 105.

[0034] The image acquisition module 101 is used to obtain the multi-spectral image of the printed circuit board to be detected.

[0035] The image acquisition module 101 is used to obtain the multi-spectral image of the printed circuit board to be detected, ensuring that image information within different spectral ranges is obtained to improve the accuracy and coverage of subsequent detection. This module may include a multi-spectral camera that can simultaneously capture image information of multiple wavelengths such as visible light, infrared light, and ultraviolet light. The image information of these different bands can reveal different defects in the surface and internal structure of the circuit board, such as tiny soldering problems on the surface or defects inside the material.

[0036] In a specific implementation, the image acquisition module 101 may include a plurality of camera arrays that can obtain images of the printed circuit board from different angles, thus ensuring the comprehensiveness of image information. In addition, the module may be equipped with a light source system to optimize the imaging quality. Especially during the detection process, when there is reflection, shadow, or other visual interference on the circuit board surface, the light source within the module will automatically adjust the lighting conditions to ensure that the acquired images are clear and stable. The acquisition rate of the image acquisition module 101 can be dynamically adjusted according to the detection requirements. In the environment of a high-speed production line, the module can continuously capture images at a speed of multiple frames per second.

[0037] After being acquired, this multi-spectral image information will be transmitted to the image preprocessing module 102 for further denoising and enhancement processing, thereby providing high-quality basic data for subsequent defect detection.

[0038] Furthermore, the image acquisition module includes a plurality of spectral cameras for respectively obtaining multi-spectral images within different wavelength ranges; wherein, the different wavelengths include visible light, near-infrared, and ultraviolet light bands.

[0039] The image acquisition module contains multiple spectral cameras, aiming to obtain multi-spectral images of the printed circuit board within different wavelength ranges. These cameras can capture image information of different wavelengths, including visible light, near-infrared light, and ultraviolet light. Each wavelength range can reveal different types of features or defects on the printed circuit board, providing rich data information for subsequent defect detection.

[0040] The camera in the visible light band is mainly used to capture the visual features on the surface of the circuit board, such as the shape of components, the state of solder joints, and the layout of connecting lines. The camera in the near-infrared light band can penetrate the surface layer of certain materials to reveal abnormalities in the internal structure, such as poor soldering or component connection problems. The ultraviolet light band is particularly suitable for detecting tiny flaws or contaminants on the surface, which may not be obvious in images of other wavelengths.

[0041] Through the image information of these different wavelength ranges, the system can comprehensively detect the printed circuit board. The images captured by each spectral camera provide unique information in a specific wavelength band, and when combined, they can enhance the detection ability of the system. For example, in some cases, the near-infrared image can identify potential problems inside the material, while the ultraviolet light image can detect abnormalities in surface details. Using the data of these different wavelengths, the system can ensure that even tiny or deeply hidden defects can be accurately identified and located.

[0042] The functions of this module can be achieved through multi-spectral camera technology. These cameras can be used in a variety of industrial and scientific research applications, and different spectral ranges can be selected according to requirements. By reasonably adjusting the wavelength settings of the cameras and the light source conditions, it can be ensured that the captured multi-spectral images have sufficient clarity and details, laying a foundation for subsequent image processing and defect detection.

[0043] The image preprocessing module 102, connected to the image acquisition module, is used to perform denoising, enhancement, and normalization processing on the multi-spectral images to improve the image quality.

[0044] The image preprocessing module 102 is used to process the multi-spectral images obtained from the image acquisition module 101 to improve the image quality for subsequent analysis. The main functions of this module include denoising, enhancement, and normalization processing. To achieve these functions, the image preprocessing module can combine multiple image processing algorithms.

[0045] First, in the denoising process, the image preprocessing module can adopt common denoising algorithms such as wavelet transform-based, mean filtering, or median filtering. The specific choice depends on the type and intensity of the noise in the image. Through these algorithms, the module can effectively remove random noise in the image while keeping the edges and details of the image unaffected. For example, when noise points are introduced into the multi-spectral image during the acquisition process due to the thermal noise of the sensor or external light interference, the denoising algorithm can automatically identify and remove these noise points.

[0046] In terms of image enhancement, the module can use contrast enhancement algorithms such as histogram equalization or adaptive histogram equalization (CLAHE) to improve the brightness and contrast of the image, making the surface of the circuit board and subtle defects more clearly visible. Image enhancement is particularly important in printed circuit board inspection because some small defects such as open circuits or soldering problems may be difficult to distinguish in the original image, while the enhanced image can highlight these key areas.

[0047] Finally, the normalization process is to standardize the pixel values of the image so that they are within a unified dynamic range, thereby eliminating the brightness differences between different spectral images. The normalization process can be carried out according to the following formula:

[0048]

[0049] where I(x,y,λ) represents the brightness value of the pixel at the corresponding wavelength λ in the multi-spectral image; min(I) and max(I) represent the minimum and maximum brightness of the image respectively; I norm (x,y,λ) represents the data after normalization processing.

[0050] Through the above processing steps, the image preprocessing module can output high-quality multi-spectral images, ensuring that the geometric and spectral information in the images can be accurately extracted and used for subsequent defect detection.

[0051] Furthermore, the image preprocessing module includes a denoising processing unit, an image enhancement unit, and a normalization processing unit, where:

[0052] The denoising processing unit adopts a wavelet transform-based denoising algorithm to filter out random noise in the multi-spectral image;

[0053] The image enhancement unit adopts an adaptive histogram equalization algorithm to improve the contrast of the image;

[0054] The normalization processing unit normalizes the image to eliminate the influence of brightness differences on subsequent analysis.

[0055] The image preprocessing module consists of a denoising processing unit, an image enhancement unit, and a normalization processing unit, aiming to perform necessary processing on the collected multispectral images to ensure that the image quality is suitable for subsequent defect detection and analysis.

[0056] The denoising processing unit uses a wavelet transform-based denoising algorithm, which can effectively filter out the random noise in the multispectral images. During the multispectral imaging process, random noise may be introduced due to ambient light, sensor thermal noise, or other factors. These noises will interfere with the important details in the image and affect the accuracy of subsequent feature extraction. Wavelet transform is an advanced denoising method. It can analyze the image data at different scales, remove high-frequency noise by decomposing and reconstructing the image signal, while retaining the important details and edge information in the image. In this way, the denoised image is clearer, providing a stable basis for subsequent processing.

[0057] The image enhancement unit further enhances the contrast of the image by using the adaptive histogram equalization algorithm. There may be complex structures and subtle defects on the surface of the circuit board, which are sometimes not easily directly detected in the original image. Through adaptive histogram equalization, the contrast in the image is enhanced, especially the details in the bright and dark regions are more clearly visible. Different from traditional histogram equalization, adaptive histogram equalization can intelligently adjust the contrast of different regions according to the local brightness information of the image, thus effectively avoiding over-enhancement or distortion phenomena, enabling details to be clearly presented under different lighting conditions.

[0058] The role of the normalization processing unit is to perform normalization processing on the multispectral images, aiming to eliminate the brightness differences between spectral channels. This step ensures that the data of each spectral channel is within the same brightness range, avoiding misjudgment or incorrect analysis caused by inconsistent brightness. In multispectral imaging, the reflection and absorption of light with different wavelengths are different, which may lead to inconsistent image brightness. Normalization processing can balance these differences, enabling subsequent deep learning models to more accurately process and analyze image information without being affected by brightness changes.

[0059] Through this multi-step image preprocessing, the system can generate high-quality, unified, and enhanced multispectral images, laying a solid foundation for the subsequent steps of printed circuit board defect detection.

[0060] The feature extraction module 103 is connected to the image preprocessing module and is used to extract preliminary geometric and spectral information from the preprocessed images. The geometric and spectral information includes the shape, layout structure of the circuit, and specific spectral characteristics.

[0061] The feature extraction module 103 is used to extract preliminary geometric and spectral information from the multi-spectral images output by the image preprocessing module 102. The geometric information includes the circuit shapes, component layouts, and connection structures on the circuit board, while the spectral information includes the material properties obtained through spectral imaging at different wavelengths. This information serves as the basis for the subsequent processing of the deep learning module and determines whether the system can correctly identify the structural features of the printed circuit board and detect potential defects.

[0062] The implementation of the feature extraction module can be based on a variety of image processing techniques. First, the extraction of geometric features can be achieved through edge detection algorithms (such as Canny edge detection), which can identify the boundaries of circuit traces and the outlines of components on the printed circuit board. Further, the line detection algorithm based on the Hough transform can be used to identify straight paths and connection patterns in the circuit, while the Fourier transform can analyze periodic or repetitive circuit structures, especially in complex circuits, and can effectively identify minor open or short circuit problems.

[0063] The extraction of spectral information mainly involves analyzing the wavelength differences in the multi-spectral images to identify the spectral characteristics of different materials. For example, different materials have different abilities to reflect or absorb light at different wavelengths, and these differences can be extracted and classified through the spectral analysis algorithm in the feature extraction module. For different materials on the printed circuit board, the module can determine whether their constituent materials meet the standards by analyzing their reflection spectra, and can also detect material aging, oxidation, or other abnormalities.

[0064] In implementation, the feature extraction module can use open-source computer vision libraries (such as OpenCV) or pre-trained models based on deep learning for feature extraction. The comprehensive processing of geometric and spectral information can provide accurate input for the deep learning module, enabling the system to accurately identify defects in subsequent steps.

[0065] Furthermore, the feature extraction module uses the Hough transform algorithm based on edge detection and the Fourier transform to extract the geometric shape features, connection patterns, and spectral features of the printed circuit board.

[0066] The function of the feature extraction module is to extract the geometric shape features, connection patterns, and spectral features of the printed circuit board from the preprocessed multi-spectral images. To achieve this goal, the feature extraction module employs the Hough transform algorithm based on edge detection and the Fourier transform.

[0067] In terms of geometric shape feature extraction, the module first uses edge detection technology to identify the key edges in the image. Edge detection is a common image processing technique that can effectively identify the edges of circuit traces, component outlines, and other important geometric structures in a printed circuit board. Through this step, the system can generate a clear contour map showing various shape information on the circuit board. Edge detection can be implemented through various algorithms, such as Canny edge detection or Sobel operator, and a suitable algorithm is selected according to the noise level in the image and the requirement for details.

[0068] The Hough transform algorithm further processes this edge information to identify and extract straight or circular structures in the circuit board. The Hough transform is a mathematical method that can extract geometric shapes (such as straight lines or circles) from a set of edge points, and is especially suitable for the analysis of connection patterns in printed circuit boards. For example, the traces connecting different components on a printed circuit board are usually straight. Through the Hough transform, these straight lines can be quickly and accurately extracted, helping the system identify the circuit connection patterns on the circuit board and analyze whether there are problems such as open circuits or short circuits. For some arc structures or circular components on the circuit board, the Hough transform can also be used to extract these circular features to ensure the integrity of the connection pattern.

[0069] The Fourier transform is used for spectral feature extraction. It is a tool that transforms an image from the spatial domain to the frequency domain. In the frequency domain, the Fourier transform can effectively analyze the periodic structures or frequency distributions in the image, especially when dealing with the periodically arranged circuit structures in a printed circuit board, it has a significant effect. The Fourier transform can help detect potential pattern repeatability anomalies in the circuit board and identify defects that may be caused by deviations in the manufacturing process, such as uneven or incomplete etching on the printed circuit board. In addition, when analyzing the frequency characteristics of multi-spectral images, the Fourier transform can also extract the reflection characteristics of different materials at different wavelengths, which is of great significance for distinguishing circuit board materials and detecting material defects.

[0070] By combining the Hough transform algorithm of edge detection and the Fourier transform, the feature extraction module can comprehensively and accurately extract the geometric, connection, and spectral features of the printed circuit board, ensuring that the subsequent defect detection module can effectively use these features for analysis.

[0071] The deep learning module 104, connected to the feature extraction module, is used to process the extracted preliminary geometric and spectral information; wherein, the deep learning module includes:

[0072] The basic network sub-module is used to use a pre-trained convolutional neural network to perform feature extraction on the preliminary geometric and spectral information extracted by the feature extraction module to obtain high-level features, where the high-level features include specific circuit patterns and connection relationships in the printed circuit board.

[0073] An adaptive adjustment sub-module, connected to the basic network sub-module, uses the labeled printed circuit board defect sample data to fine-tune the high-level features, generating specific features that can reflect the circuit board defects for further accurate identification and classification of various defects on the printed circuit board.

[0074] A defect identification and classification sub-module is used to combine the high-level features provided by the basic network sub-module and the specific features provided by the adaptive adjustment sub-module to perform defect identification and classification of the printed circuit board. Among them, the defect types of the printed circuit board include short circuit, open circuit, insufficient etching, and soldering defects.

[0075] The deep learning module 104 is the core of the printed circuit board defect detection system, used to process the geometric and spectral information obtained from the feature extraction module 103 to identify and classify the defects on the printed circuit board. This module uses a convolutional neural network (CNN) to further analyze and process these initially extracted features, thereby extracting higher-level and more abstract features, especially the complex connection patterns and specific circuit structures in the circuit.

[0076] The deep learning module includes a basic network sub-module, which is based on a pre-trained convolutional neural network and has been pre-trained on a general large-scale dataset (such as the ImageNet dataset), so it has good feature extraction capabilities. The basic network sub-module will initially process the input geometric and spectral information to identify the global structure information and specific circuit patterns on the circuit board. These high-level features include the global layout of the printed circuit board, the connection relationships between components, and specific circuit patterns, which can help identify defects such as circuit breaks, short circuits, and soldering problems.

[0077] After the basic network sub-module, the system also includes an adaptive adjustment sub-module, which uses transfer learning technology to fine-tune the high-level features in the basic network by using the labeled printed circuit board defect sample data. Transfer learning allows this module to quickly adjust the network parameters in the case of using a small amount of specially labeled circuit board defect data, enabling it to adapt to specific circuit board detection tasks. In a specific implementation, the adaptive adjustment sub-module will fine-tune the high-level parameters of the convolutional neural network to generate specific features related to the circuit board defects. These specific features can more accurately capture and reflect various defects on the printed circuit board, including short circuits, open circuits, insufficient etching, and soldering defects.

[0078] Through this multi-level feature extraction and adjustment, the deep learning module can effectively convert the original geometric and spectral information into high-level information that can identify defects, ensuring that the system has a high-accuracy defect detection ability.

[0079] Furthermore, the basic network sub-module uses a pre-trained convolutional neural network model and extracts high-level features of the printed circuit board through the ResNet50 network architecture pre-trained on the ImageNet dataset.

[0080] The basic network sub-module adopts a pre-trained convolutional neural network model, and its core architecture is based on the ResNet50 network pre-trained on the ImageNet dataset. ResNet50 is a deep convolutional neural network with a 50-layer network structure. Its main feature is to introduce residual connections to solve the problem of gradient disappearance in the training of deep networks. This enables the network to still effectively extract complex image features while maintaining high depth.

[0081] In this printed circuit board defect detection system, the main role of the basic network sub-module is to extract high-level features of the printed circuit board image through the ResNet50 network architecture. Through the pre-trained convolutional neural network, the system can identify complex circuit board patterns and wiring structures from multi-spectral images. After large-scale pre-training on the ImageNet dataset, ResNet50 has mastered a wide range of visual features, including edges, textures, shapes, etc., which are common in many visual tasks. Utilizing this pre-trained ability, the basic network sub-module can quickly identify the overall structural features of the circuit and important connection patterns in printed circuit board detection.

[0082] The pre-trained ResNet50 processes the input multi-spectral image layer by layer through its convolutional layers. Each layer of convolutional operation extracts more abstract and high-level features. For example, the initial convolutional layer may extract basic edge information, while the subsequent deep convolutional layers can capture more complex shape, texture, and structure patterns. These high-level features are particularly suitable for analyzing the wiring patterns, component layouts, and overall geometric structures of printed circuit boards.

[0083] By using the pre-trained model, the basic network sub-module can reduce the training time and improve the accuracy of the model because ResNet50 has learned rich visual features through large-scale image data. According to actual needs, the network can be further fine-tuned to better adapt to the printed circuit board defect detection task. For example, through transfer learning techniques, a small number of labeled printed circuit board defect images can be used to fine-tune the high-level parameters of the ResNet50 model, so that the model can be more focused on identifying specific defects in printed circuit boards, such as short circuits, open circuits, or insufficient etching, etc.

[0084] In summary, by using the pre-trained ResNet50 model, the basic network sub-module can efficiently and accurately extract high-level features of the printed circuit board, providing key inputs for subsequent defect detection and classification.

[0085] Furthermore, the adaptive adjustment sub-module uses the labeled printed circuit board defect sample data, fine-tunes the pre-trained convolutional neural network using the transfer learning method, and fine-tunes the high-level feature parameters of the basic network sub-module using the following formula:

[0086]

[0087] where, θ represents the adjustable weight parameter of the basic network; θ * represents the optimal parameter after fine-tuning; N is the number of training samples; x i represents the input of the i-th sample; y i represents the label of the i-th sample; L(f θ (x i ),y i ) is the loss function, used to evaluate the difference between the prediction result L(f θ (x i ),y i ) and the actual label y i , and the loss function is implemented using cross-entropy loss; α is the weight parameter; t i represents the timestamp of the sample x i ; t ref represents the reference time; T max represents the maximum time span; T is the number of Monte Carlo sampling times; represents the model output of the t-th forward propagation after activating Dropout; represents the mean value of the output after forward propagation; β is the weight parameter of the regularization term; θ k is the k-th high-level feature parameter; K is the total number of high-level feature parameters; |θ k | p is the p-norm of the parameter θ k ; λ t is a dynamically adjusted parameter that changes over time; γ is the weight parameter; L represents the number of samples in the dataset or a specific batch; represents the gradient of the neural network model f θ with respect to the input data x i .

[0088] The adaptive adjustment sub-module uses the labeled printed circuit board defect sample data and fine-tunes the pre-trained convolutional neural network through transfer learning to adapt to the specific task of printed circuit board defect detection. The transfer learning method allows fine-tuning a model that has been trained on a large-scale general dataset with a smaller sample data, which enables the model to quickly adapt to new tasks, reduce training time, and improve performance. The key to fine-tuning lies in adjusting the high-level feature parameters of the base network sub-module to ensure that these parameters can better reflect the actual defects in the printed circuit board.

[0089] To achieve fine-tuning, an optimization formula is used, which optimizes the weight parameters of the model by minimizing the loss function to make the model output more accurate. Each term in the formula has a specific contribution to the fine-tuning process of the high-level feature parameters, ensuring that the model can adaptively adjust the feature extraction and classification capabilities from different perspectives.

[0090] The following is a detailed description of each term in the formula and how these terms act on the fine-tuning process of the model:

[0091] θ represents the adjustable weight parameters of the base network. In transfer learning, these parameters are the core of the neural network model adjusted through training, aiming to enable the model to better adapt to new tasks.

[0092] θ * is the optimal weight parameter after optimization. During the fine-tuning process, the system continuously iteratively adjusts the weights and finally finds a set of parameters that can minimize the loss. These parameters are most suitable for handling the printed circuit board defect detection task.

[0093] N is the number of training samples, representing the total number of samples used to train the model. The model is trained and fine-tuned through these labeled sample data.

[0094] x i is the input of the i-th sample. For printed circuit board defect detection, x i may be a preprocessed multi-spectral image.

[0095] y i is the label of the i-th sample, representing the actual defect category corresponding to this sample. The model calculates the error by comparing the prediction result with the actual label.

[0096] L(f θ (x i ),y i ) is the loss function, which is used to evaluate the prediction output f θ (x i ) and the actual label y iThe difference between them. In this system, the cross-entropy loss function is used to measure the error between the predicted result and the true category. This is a common loss function, especially suitable for classification tasks, and can effectively guide the training direction of the model.

[0097] α is a weight parameter used to control the influence of the time-related weighting term. It regulates the role of time correlation in the fine-tuning process.

[0098] t i is the timestamp of sample x i . The timestamp represents the time information of each sample during the training process and may be related to the time order of data collection.

[0099] t ref is the reference time point, usually set as the time when the model is initialized or training starts. It is used to calculate the difference of sample x i relative to the reference time.

[0100] T max is the maximum time span, representing the normalization range of the time difference. It ensures that the time correlation calculation is within an appropriate range and does not generate extreme weights due to excessive time differences.

[0101] T is the number of Monte Carlo samplings, indicating the number of times of forward propagation to calculate the uncertainty of the model output. Monte Carlo Dropout estimates the uncertainty of the model by activating the Dropout layer during forward propagation for sampling.

[0102] is the output of the model when activating Dropout after the t-th forward propagation. Since Dropout randomly discards some neurons, the output of the model for each forward propagation will be different.

[0103] is the output mean obtained through T times of forward propagation. It represents the predicted mean of the model for sample x i as a reference for uncertainty calculation.

[0104] β is the weight parameter of the regularization term used to regulate the influence of regularization on the model. Regularization helps the model avoid overfitting by constraining the complexity of the model, ensuring that the model performs well on the training data and also has good generalization ability on new data.

[0105] θ k is the k-th high-level feature parameter, representing the weight related to the k-th feature extraction in the network.

[0106] K is the total number of high-level feature parameters, representing the number of weight parameters used to extract high-level features in the network.

[0107] |θ k | p is the p-norm of the high-level feature parameters. The p-norm is used to measure the magnitude of the weights and is commonly used in L1 regularization (p = 1) or L2 regularization (p = 2) to constrain the values of the weights and prevent the model parameters from being too large, which may lead to overfitting.

[0108] λ t is a dynamically adjusted parameter that changes over time and is used to control the strength of regularization. As time progresses, the impact of regularization can gradually decrease or increase, which helps the model to remain robust in the early stages of training and adapt to more features in the later stages. λ t can be implemented using the following formula:

[0109] λ t =λ0·e -r·t

[0110] where λ0 is the initial regularization strength; r is the decay rate; and t is the current training epoch or time step.

[0111] γ is another weight parameter that is used to control the impact of gradient regularization. The gradient regularization term constrains the gradient changes of the model, preventing the model from producing drastic output changes in the face of small changes in the input, and ensuring the robustness of the model.

[0112] L represents the number of samples in the dataset or a specific batch, and is commonly used as the batch size in batch gradient descent training.

[0113] is the gradient of the neural network model f θ with respect to the input data x i and represents the impact of small changes in the input data on the output result. By constraining the gradient, the model can maintain the smoothness and stability of the output on complex datasets and reduce its sensitivity to noise.

[0114] This optimization formula combines the loss function, time-dependent weighting, uncertainty evaluation, and regularization terms, enabling the adaptive adjustment sub-module to fine-tune the basic network sub-module on limited defective data and generate optimal parameters for a specific printed circuit board defect detection task.

[0115] Furthermore, the defect identification and classification sub-module uses the features output by the convolutional neural network to train a support vector machine classifier for classifying printed circuit board defects; the support vector machine classifier is used to accurately classify the defects into short circuits, open circuits, insufficient etching, and soldering defects based on the input high-level features and specific features.

[0116] The described defect identification and classification sub-module is trained using a Support Vector Machine (SVM) classifier with high-level features and specific features extracted by a Convolutional Neural Network (CNN) to achieve precise classification of printed circuit board defects. The core of this module lies in that the Convolutional Neural Network is used to extract complex features from images, and the Support Vector Machine then completes the classification task of different types of defects based on these high-level features.

[0117] Specifically, when processing multi-spectral images of printed circuit boards, the Convolutional Neural Network first extracts important features such as geometric structures, wiring patterns, and spectral information in the circuit board through multiple convolutional operations. After being processed at the high level of the convolutional network, these features can form a deep understanding of the circuit board, including differentiating potential differences in different types of problems such as short circuits, open circuits, insufficient etching, and soldering defects. These extracted high-level features, along with the specific features generated by the adaptive adjustment module, are used as inputs to the Support Vector Machine classifier.

[0118] The Support Vector Machine classifier trains a classification model by learning these input features to accurately classify each defect sample as a short circuit, open circuit, insufficient etching, or soldering defect. The advantage of the SVM classifier is that it can handle high-dimensional data and improve classification accuracy by maximizing the classification boundary, making it particularly suitable for processing multi-dimensional feature vectors generated by the Convolutional Neural Network.

[0119] In specific implementation, the Convolutional Neural Network can be first used to extract features, and these features are input into the Support Vector Machine for training the classification model. During the training process, the SVM will continuously adjust its decision boundary to find the feature combination that can best distinguish different defect categories. After training, the system can input new printed circuit board images into the Convolutional Neural Network to generate features, and use the SVM classifier to classify the defects of each sample, finally obtaining an accurate prediction result of the printed circuit board defect type.

[0120] The following is the reference implementation code for the deep learning module:

[0121]

[0122]

[0123]

[0124]

[0125]

[0126]

[0127]

[0128] The training steps of the deep learning module are as follows:

[0129] First, obtain the multi-spectral images of the printed circuit board from the image acquisition module, and generate training data after preprocessing. Then, use the pre-trained ResNet50 model to extract high-level features, freeze the early layers, and fine-tune the last fully connected layer to adapt to the printed circuit board defect detection task. Through transfer learning, use the labeled defect data to fine-tune the model, and combine time-weighting and Monte Carlo Dropout sampling for training. Then, use a support vector machine (SVM) classifier to train the extracted high-level features to classify different defect types. Finally, generate defect detection results and a visualization report.

[0130] The visualization module 105, connected to the deep learning module, is used to label the detected defect positions and types on the original image and generate a report including the detection results.

[0131] The visualization module 105 is used to intuitively display the printed circuit board defect information identified and classified by the deep learning module 104. This module overlays the detected defect positions, types and other results on the original multi-spectral image for visual presentation, helping users accurately understand the problem areas on the circuit board. In this way, operators can directly see the specific positions and natures of each defect on the image, so as to quickly make decisions on repair or further inspection.

[0132] This visualization module can not only graphically label the detection results of defects on the image, but also distinguish different defect types using different colors or symbols. For example, short circuit defects can be labeled in red, open circuits can be labeled in yellow, and soldering problems can be represented in blue. In this way, different types of defects can be visually displayed, facilitating technicians to quickly identify the problem types.

[0133] In addition, the visualization module can generate a detailed detection report, which not only includes the defect position and type labels in the image, but also provides statistical information related to each defect, such as the size and severity of the defect. These data can be attached to the report in the form of text, charts or tables. The report can also be exported to standard file formats (such as PDF or CSV) for easy archiving, sharing or further analysis.

[0134] To ensure flexibility in actual operation, the visualization module may also include interactive functions, such as allowing users to zoom in and out of the image, rotate the view of the circuit board to inspect defects from different angles. Users can also click on the marked points in the image to obtain detailed information about the defect, such as the specific detection time, detection accuracy, and recommended repair operations. In this way, the visualization module not only provides a static result display but also improves the operational convenience for users.

[0135] Furthermore, the visualization module includes a three-dimensional image processing unit for highlighting the detected defect locations on the original image with different colors, displaying different color codes according to the defect types, and generating a detection report in PDF format, which includes the exact locations, sizes, and types of the defects.

[0136] The described visualization module has a three-dimensional image processing unit, whose core function is to highlight the detected printed circuit board defects on the original image and use different color codes for different types of defects. This three-dimensional image processing unit helps users intuitively understand the detection results through clear visual feedback, enabling them to quickly locate and identify problems on the circuit board.

[0137] First of all, the three-dimensional image processing unit can generate a three-dimensional view based on the multi-spectral image, which is particularly important for the detection of printed circuit boards. Printed circuit boards usually have complex laminated structures and fine component arrangements. The three-dimensional view can help technicians better understand the spatial structure of the circuit board and the specific locations of the defects. Through three-dimensional processing, users can not only see the defects on the plane but also observe the depth and spatial distribution of the defects from different angles, which is crucial for evaluating the severity and potential impact of the defects. For example, some defects may only affect the surface, while others may penetrate into the internal layers of the circuit board. Three-dimensional visualization helps distinguish these situations.

[0138] To improve the intuitiveness of recognition, the three-dimensional image processing unit uses different color codes to mark various defects. Each defect type has a corresponding color. For example, short-circuit defects can be marked in red, open-circuit defects in yellow, insufficient etching defects in green, and soldering defects may use blue. This color differentiation method allows users to quickly identify the problem types by color without in-depth analysis of complex texts or data. The highlighted display of the color codes on the image enables users to clearly determine the types and locations of the problems at a glance. Especially when dealing with large-area circuit boards, the color markings can help users efficiently scan and locate.

[0139] In addition to color marking, the system also generates a detection report in PDF format. This report not only includes the exact location of each detected defect (usually shown in the form of coordinates), but also details the size and type of the defect. For example, the report may include the exact location of the defect on the circuit board, such as the positioning information on the X, Y, and Z coordinate axes, enabling technicians to precisely locate the defect on the physical circuit board. The size of the defect is also listed, indicating the physical dimensions of the defect and helping to assess the severity of the problem. For a specific defect type, the report details the nature of the defect. For example, whether it will cause a short circuit, an open circuit in the circuit, or affect the function of the component.

[0140] The 3D image processing unit can also have additional interactive functions, allowing users to freely rotate, zoom in or out of the view in 3D space, so as to better view the details of the defect. This function is achieved through mouse dragging or touch screen operations. Users can zoom in on a certain area to carefully check the defects near complex components. In this way, not only can the defect location on the plane be observed, but also its spatial distribution can be explored for more in-depth analysis.

[0141] The extended functions of this visualization module can also include the backtracking and comparison of the detection history. The system can save the 3D images and report files of each detection, allowing users to compare the detection results at different time points during subsequent detections to determine whether the defects have an expanding or deteriorating trend. For example, if a slight under-etching is shown at a specific location in the first detection, and the defect at the same location has worsened in subsequent detections, the system can intuitively display this change through the comparison of different detection reports, providing a basis for preventive maintenance.

[0142] In terms of technical implementation, the 3D image processing unit can be implemented by using existing 3D image processing tools and libraries, such as 3D rendering technologies based on OpenGL or WebGL. Through these tools, the system can map the defect information obtained from the deep learning module and the defect classification sub-module into the 3D model and use custom color coding for marking. Those skilled in the art can adjust the rendering accuracy and speed according to the specific hardware conditions and image resolution to ensure a smooth user experience when processing high-resolution images.

[0143] The generation of the PDF report can use existing document generation tools (such as the ReportLab library in Python or the iText library in Java). By inserting data such as the exact location information, size, and type of the defect into a predefined PDF template, the system can generate a structured and easy-to-read report file. These reports can not only be used for current repair decisions, but also be archived for future reference or as a record for quality control.

[0144] Furthermore, the visualization module includes a heat map generation unit for generating a visualized heat map according to the type and density of defects. The heat map marks defect-dense areas with highlighted colors and displays the distribution of defects through color gradients, thereby intuitively providing defect concentration information in different areas on the circuit board.

[0145] The heat map generation unit of the visualization module generates a visual heat map by analyzing the defect type and density, helping users to intuitively understand the distribution of defects on the printed circuit board. The module uses color gradients and highlights to mark defect-intensive areas, allowing users to quickly identify areas on the circuit board where problems are concentrated and take appropriate actions. This heat map generation method is particularly useful in complex circuit board environments because it can simultaneously display the distribution and severity of multiple defects.

[0146] The core working principle of the heat map generation unit is color coding based on defect density. Areas with higher defect density are marked with brighter highlight colors, while areas with fewer or no defects are marked with softer colors. Through this color gradient, users can intuitively see the concentration of defects on the circuit board. For example, if an area appears red or dark red, this usually means that there are a large number of defects in that area with a high density, which may require intensive inspection or immediate repair. On the contrary, if an area appears light green or blue, this indicates that there are few defects in that area and there may be no serious problems. Through such color contrast, users can quickly identify the parts of the circuit board that need the most attention, saving time to check each area one by one.

[0147] In addition, the heat map generation unit can also display different types of defects by color classification according to their severity. For example, for short circuit defects, a color gradient scheme can be used, such as a gradual transition from yellow to red, to indicate the density of short circuit problems. For poor soldering problems, another set of color coding can be used, such as a transition from light blue to dark blue, to distinguish different types of defects. This design ensures that the distribution of different types of defects is presented on the same heat map at the same time, and users can directly identify the type and severity of the problem by color. This is very important for dealing with complex circuit boards with multiple defect types.

[0148] In terms of implementation details, the heatmap generation unit generates a heatmap with color gradients through data acquisition and processing steps. First, the system obtains the location and type information of each defect on the circuit board from the deep learning module and the defect recognition module. This information includes not only the exact coordinates of each defect but also the severity and category of the defect. The system aggregates this data into a unified coordinate system and calculates the defect density of each area. For example, the circuit board can be divided into several small areas, the number of defects in each area is counted, and then the defect density of each area is determined. Subsequently, according to the level of defect density, the heatmap generation unit assigns corresponding colors to each area and generates the final heatmap.

[0149] When implementing this function, existing image processing and data visualization tools can be utilized. For example, common visualization toolkits (such as Matplotlib or D3.js) can be used to create heatmaps with color gradients. These libraries provide rich color mapping and data visualization functions, enabling easy implementation of color gradient display based on data density. In addition, the system can adjust the resolution and color gradient range of the heatmap according to user needs to adapt to the detection requirements of different types of circuit boards. For large-area circuit boards, a heatmap with a lower resolution can be selected for display to highlight important areas; while for fine circuit board detection, the heatmap resolution can be increased to ensure that every detail can be accurately identified.

[0150] The heatmap generation unit is not limited to static heatmap display but can also implement dynamic interaction functions. Users can hover or click on a certain area of the heatmap to obtain specific defect information in that area, such as the specific type, size, and location of each defect. This interactive function helps users further understand the detailed information of the defects and quickly make corresponding repair decisions based on the display of the heatmap. In addition, the system can generate heatmaps for multiple time periods according to user needs, and by comparing the defect distribution at different times, judge the development trend of circuit board defects. For example, if the defect density in a certain area gradually increases during continuous detection, the system can highlight this area in the heatmap to prompt users that there may be potential serious problems in this area.

[0151] Finally, the results of the heatmap generation unit can not only be displayed on the screen but also be exported to different file formats, such as PNG, PDF, or SVG, etc. In this way, users can save, print, or share the heatmap together with the detection report, which is convenient for subsequent quality control and review. For large factories and manufacturing assembly lines, the heatmap can be used as a key visualization tool during the detection process to help inspectors quickly locate problems, optimize the repair and production processes, and ensure the normal operation of the production line.

[0152] In summary, the heat map generation unit can effectively display the distribution of defects on the printed circuit board through intuitive color coding and gradient methods, and provide detailed density information. This function greatly improves the detection efficiency, helps users quickly identify key problem areas, and at the same time, through rich extension functions and interactive designs, ensures that users can deeply understand and utilize the detection results.

[0153] Furthermore, the visualization module includes an interactive user interface that allows users to dynamically view the detection results of the printed circuit board through zooming, rotating, and panning functions, and supports users to click on the marked points in the image to display the detailed information of the corresponding defects. The detailed information includes the defect type, location coordinates, and recommended repair solutions.

[0154] The visualization module further includes an interactive user interface that allows users to view and operate the detection results of the printed circuit board in a dynamic manner. This interactive interface provides multiple functions, including zooming, rotating, and panning, enabling users to inspect the circuit board from multiple angles and different perspectives and to drill down into specific details. The purpose of this interactive design is to allow users to flexibly view the entire circuit board, thereby improving the efficiency and accuracy of defect detection. Especially when dealing with large-area or complex circuit boards, the interactive functions are particularly important.

[0155] Through the zoom function, users can easily zoom in on a certain area to more carefully inspect specific components or connection points. For example, some defects may be difficult to identify in the overall view of the circuit board, but by zooming in, users can clearly see whether there are problems such as poor soldering of components, disconnected wires, or insufficient etching. The zoom-out function allows users to return to the global view to observe the overall situation of the circuit board and ensure that no area is missed. The rotation function provides users with a more flexible perspective, especially in a three-dimensional display environment. This function enables users to observe the structure of the circuit board from different angles and discover problems that may be difficult to detect from a single perspective.

[0156] The panning function enables users to move the view at a fixed zoom ratio to browse different areas of the circuit board without frequently adjusting the zoom ratio. This is extremely convenient when inspecting large-area circuit boards. Users can view each area along the edge or the central part of the circuit board one by one without repeatedly adjusting the view settings.

[0157] In addition, the interactive user interface not only provides basic operation functions but also supports clicking on the marked points in the circuit board image to display detailed information related to the defect. When the user clicks on a marked point, a message box will pop up, showing the specific information of the defect, including the type of the defect, the precise position coordinates (such as the coordinates on the X, Y, and Z axes), and the repair suggestions for the defect. This design greatly enhances the user experience, enabling the user to immediately obtain key diagnostic information without having to search or analyze data in the system. Through the click function, the user can easily check each defect on the circuit board from different perspectives, and the detailed information provided by the system effectively supports the user in making further decisions.

[0158] The defect type information helps the user quickly understand the nature of the problem. For example, the marked point can directly indicate whether the defect is a short circuit, an open circuit, a soldering problem, or an insufficient etching. The display of the position coordinates ensures that the user can accurately locate the problem on the physical circuit board. Especially on large and complex circuit boards, precise positioning becomes particularly important. The automatically generated position coordinate information, combined with electronic measurement tools, can help technicians efficiently find and repair the problem during actual operation.

[0159] The system also provides recommended repair solutions. This function gives optimized repair methods by combining the historical data of previous detections, the defect type, and the location. For example, the system can recommend re-soldering a certain point to solve the soldering problem or suggest replacing damaged components to repair the circuit fault. For some recurring defects, the system can also recommend preventive measures, such as adjusting production process parameters to reduce the occurrence of certain specific defects. These suggestions not only improve the intelligence level of the system but also help users save time and labor costs.

[0160] In practical applications, technicians can implement this interactive user interface by using existing front-end development tools and frameworks. Commonly used tools include HTML5, JavaScript libraries (such as D3.js or Three.js), and WebGL and other technologies. These tools can support efficient 2D and 3D image rendering and can easily implement basic interaction functions such as zooming, rotating, and panning. In addition, combined with the data processing capabilities of the back-end system, users can obtain defect detection results in real time through the interface and perform interactive operations.

[0161] The system can be designed to be flexible enough to support users at different levels. For ordinary users, the system provides simple and easy-to-use operation functions to help them quickly locate and understand defect problems. For advanced users or engineers, the system can provide more detailed technical information and analysis tools to help them conduct in-depth fault diagnosis and make decisions. This hierarchical design can ensure that users with different backgrounds can benefit from the system.

[0162] The system can integrate more analysis tools and report generation functions. For example, users can choose to export the detection results into multiple file formats (such as PDF or CSV) for subsequent quality control or sharing with other team members. At the same time, the system can also record the user's operation process and viewing history, generate operation logs or reports to help users review the detection process or make improvement suggestions. For large-scale factories or production lines, the system can also be connected to a central database to store all detection results, facilitating future data analysis or further training of machine learning models.

[0163] In summary, the described interactive user interface not only provides rich operation functions to help users visually view the defects on the circuit board, but also provides detailed defect information and repair suggestions by clicking on the marked points, making the system a comprehensive and efficient defect detection tool.

[0164] Although this application is disclosed above with preferred embodiments, it is not intended to limit this application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of this application. Therefore, the protection scope of this application should be subject to the scope defined by the claims of this application.

Claims

1. A printed circuit board defect detection system, characterized in that: include: An image acquisition module, used for acquiring a multispectral image of a printed circuit board to be inspected; An image preprocessing module, connected to the image acquisition module, for performing denoising, enhancement and normalization processing on the multispectral image to improve image quality; A feature extraction module, connected to the image preprocessing module, for extracting preliminary geometric and spectral information from the preprocessed image, wherein the geometric and spectral information includes the shape, layout structure and specific spectral characteristics of the circuit; A deep learning module, connected to the feature extraction module, is used to process the extracted preliminary geometric and spectral information; wherein the deep learning module includes: A basic network submodule, for performing feature extraction on the preliminary geometric and spectral information extracted by the feature extraction module using a pre-trained convolutional neural network, to obtain high-level features, wherein the high-level features include specific circuit patterns and connection relationships in the printed circuit board; The adaptive adjustment submodule is connected to the basic network submodule, and uses the labeled PCB defect sample data to fine-tune the high-level features to generate specific features that can reflect the PCB defects, which are used to further accurately identify and classify various defects on the PCB; A defect identification and classification submodule, which is used to identify and classify defects of printed circuit boards by combining high-level features provided by the basic network submodule and specific features provided by the adaptive adjustment submodule, wherein the defect types of the printed circuit boards include short circuits, open circuits, insufficient etching, and welding defects; A visualization module is connected to the deep learning module and is used to mark the detected defect locations and types on the original image and generate a report including the detection results.

2. The printed circuit board defect detection system according to claim 1, characterized in that: The image acquisition module includes a plurality of spectral cameras for respectively acquiring multispectral images within different wavelength ranges; wherein the different wavelengths include visible light, near infrared and ultraviolet light bands.

3. The printed circuit board defect detection system according to claim 1, characterized in that: The image preprocessing module includes a denoising processing unit, an image enhancement unit and a normalization processing unit, wherein: The denoising processing unit uses a denoising algorithm based on wavelet transform to filter out random noise in multispectral images; The image enhancement unit uses an adaptive histogram equalization algorithm to improve the contrast of the image; The normalization processing unit performs normalization processing on the image to eliminate the influence of brightness difference on subsequent analysis.

4. The printed circuit board defect detection system according to claim 1, characterized in that: The feature extraction module uses a Hough transform algorithm based on edge detection and Fourier transform to extract geometric shape features, connection patterns and spectral features of the printed circuit board.

5. The printed circuit board defect detection system according to claim 1, characterized in that: The basic network submodule uses a pre-trained convolutional neural network model and extracts high-level features of the printed circuit board through a ResNet50 network architecture pre-trained on the ImageNet dataset.

6. The printed circuit board defect detection system according to claim 1, characterized in that: The adaptive adjustment submodule uses the labeled printed circuit board defect sample data and the transfer learning method to fine-tune the pre-trained convolutional neural network, and uses the following formula to fine-tune the high-level feature parameters of the basic network submodule: Among them, θ represents the adjustable weight parameter of the basic network; θ * represents the optimal parameters after fine-tuning; N is the number of training samples; x i represents the input of the i-th sample; y i represents the label of the i-th sample; L(f θ (x i ),y i ) is the loss function used to evaluate the prediction result L(f θ (x i ),y i ) and the actual label y i The difference between the two, the loss function is implemented using cross entropy loss; α is the weight parameter; t i Represents sample x i Timestamp; t ref Indicates the base time; T max represents the maximum time span; T is the number of Monte Carlo sampling; represents the model output of the tth forward propagation after activating Dropout; represents the mean of the output after forward propagation; β is the weight parameter of the regularization term; θ k is the kth high-level feature parameter; K is the total number of high-level feature parameters; |θ k | p is the parameter θ k The p-norm of t is a dynamically adjusted parameter that changes over time; γ is a weight parameter; L represents the number of samples in a dataset or a specific batch; Represents the neural network model f θ For input data x i gradient.

7. The printed circuit board defect detection system according to claim 1, characterized in that: The defect recognition and classification submodule uses features based on the output of the convolutional neural network to train a support vector machine classifier for classifying printed circuit board defects; the support vector machine classifier is used to accurately classify defects into short circuits, open circuits, insufficient etching and welding defects based on the input high-level features and specific features.

8. The printed circuit board defect detection system according to claim 1, characterized in that: The visualization module includes a three-dimensional image processing unit, which is used to highlight the detected defect locations on the original image with different colors, display different color codes according to the defect type, and generate a detection report in PDF format, which includes the precise location, size and type of the defect.

9. The printed circuit board defect detection system according to claim 1, characterized in that: The visualization module includes a heat map generation unit, which is used to generate a visualized heat map according to the type and density of defects. The heat map marks the defect-intensive areas with highlighted colors and displays the distribution of defects through color gradients, thereby intuitively providing defect concentration information in different areas on the circuit board.

10. The printed circuit board defect detection system according to claim 1, characterized in that: The visualization module includes an interactive user interface, which allows the user to dynamically view the inspection results of the printed circuit board through zooming, rotating and translating functions, and supports the user to click on the marked points in the image to display the detailed information of the corresponding defects, and the detailed information includes the defect type, location coordinates and recommended repair solutions.

Images