A circuit board component fault detection method based on infrared thermal images and a deep learning model

Abstract

The application discloses a circuit board component fault detection method based on an infrared thermal image and a deep learning model, first, an infrared thermal image sequence under the running state of a circuit board is collected, then a temperature maximum value sequence of a component region is extracted and data enhancement is carried out, a new temperature sequence is generated, continuous wavelet transformation is carried out on the new temperature sequence to be converted into a time-frequency image, finally, the time-frequency images of all circuit board components are sent into a deep learning model for feature extraction, and a fault type classification is carried out to obtain a fault classification result. Through the temperature maximum value, data enhancement and continuous wavelet transformation conversion into the time-frequency image, the problems of small temperature difference, few samples, large model and poor generalization in the prior art are overcome, and the accuracy of the circuit board component fault detection is improved.

Description

Technical Field

[0001] This invention belongs to the field of circuit board fault detection technology, and more specifically, relates to a method for detecting circuit board component faults based on infrared thermal images and deep learning models. Background Technology

[0002] With the continuous advancement of electronic technology, circuit board manufacturing technology is increasingly moving towards miniaturization and integration. As a core component of electronic devices, the circuit board supports the operation and use of the entire electronic system. Accurate fault detection of circuit boards is a crucial support method for the operation and maintenance of electronic systems. During the operation of a circuit board, components may experience various faults due to environmental changes and aging. These faults can range from affecting the normal operation of the system to causing the collapse of the entire electronic system.

[0003] In traditional circuit board fault detection techniques, researchers have used different physical methods to study circuit board fault data. For example, some researchers use instruments to detect the voltage, current, and frequency parameters of various components on the circuit board under test, and then analyze the data to deduce the fault. This method requires direct contact with the circuit board under test, which may cause secondary damage to the circuit board, and some tiny components may not be able to be contacted with probes at all.

[0004] In recent years, a major detection method involves sending a specific signal to the circuit board under test (PCB) and then analyzing the signal to detect faults. This data requires the PCB to generate corresponding stimuli. If the stimuli test signal is inappropriately selected, or if there is excessive interference with the PCB signal, the PCB will not be able to generate the correct stimuli characteristics. Therefore, PCB fault detection urgently needs a new physical method for non-contact, automated fault detection without external signal input. Infrared thermal imaging detection has thus been proposed for PCB fault detection.

[0005] However, infrared thermal imaging detection methods suffer from problems such as small temperature differences, limited sample sizes, large model sizes, and poor generalization, making them inaccurate for detecting faults in circuit board components. Summary of the Invention

[0006] The purpose of this invention is to address the problems of small temperature difference, limited sample size, large model size, and poor generalization in existing infrared thermal images. It provides a method for fault detection of circuit board components based on infrared thermal images and a deep learning model, thereby improving the accuracy of fault detection for circuit board components.

[0007] This invention relates to a circuit board component fault detection method based on infrared thermal images and a deep learning model, characterized by comprising the following steps:

[0008] (1) Acquire infrared thermal image sequence of the circuit board in operation

[0009] A sequence of infrared thermal images of the circuit board under test was captured using a thermal imager. :

[0010]

[0011] in, Indicates at time step Spatiotemporal location Temperature value, To represent the size of the timestamp, i.e. the length of time, Indicates the height of the infrared thermal image, Indicates the width of the infrared thermal image;

[0012] (2) Extract the temperature maxima sequence of the component region from the infrared thermal image sequence.

[0013] The acquired thermal image sequence is used to select a Region of Interest (ROI), which is the area where a circuit board component is located. Local temperature maximum sampling is used to extract the temperature of each ROI at each time step. The temperature maxima form a temperature maxima sequence. :

[0014]

[0015] in, For the first A set of spatial coordinates of a region of interest. Indicates the first The nth ROI region, i.e. A sequence of maximum temperature values ​​for components on a circuit board;

[0016] (3) Data augmentation of temperature maximum value series

[0017] 3.1) Sample splicing

[0018] The temperature maximum sequence of all circuit board components The temperature sequence sample dataset is spliced ​​together into a single whole. Represented as:

[0019]

[0020] in, The number of components on the circuit board;

[0021] 3.2) Generation of Autoregressive Temperature Series

[0022] An autoregressive parameterized model is established using a neural network. Through neural network training, it continuously learns and optimizes its parameters. And predict the temperature maximum sequence Every step of the future The conditional probability distribution, and thus the prediction of the future. The joint probability distribution of the sequences at each time step predicts the generation length to be... A new temperature sequence at each time step ;

[0023] 3.3) Uncertainty Interval Sampling

[0024] Using common-type prediction for temperature series Uncertainty quantification is performed by calculating a non-consistency score for the temperature at each time step. Based on the score distribution, a threshold is determined given a confidence level, and then the threshold and temperature sequence are analyzed. An uncertainty interval is constructed, and then the interval is sampled based on a normal distribution to generate a temperature sequence. ;

[0025] (4) Convert the temperature sequence into a time-frequency image through continuous wavelet transform.

[0026] First, the temperature series After performing bipolar normalization, a continuous wavelet transform is performed using the Complex Morlet mother wavelet function to obtain the wavelet coefficient matrix. :

[0027]

[0028] in, It is a temperature sequence Length, It is the normalized temperature sequence The Temperature values ​​at each time step For the mother wavelet function of Complex Morlet, For scale parameters, These are translation parameters;

[0029] The wavelet coefficient matrix obtained by the transformation Modulus extraction yields time-frequency image. :

[0030]

[0031] (5) Use deep learning models to extract features and classify fault types from time-frequency images.

[0032] Time-frequency images of all circuit board components The data is fed into a deep learning model for feature extraction and fault type classification to obtain the fault classification results.

[0033] The objective of this invention is achieved as follows:

[0034] This invention presents a circuit board component fault detection method based on infrared thermal images and a deep learning model. First, it acquires a sequence of infrared thermal images of the circuit board in operation. Then, it extracts the temperature maxima sequence of the component areas and performs data augmentation to generate a new temperature sequence. Next, it performs continuous wavelet transform on the new temperature sequence to convert it into a time-frequency image. Finally, it feeds the time-frequency images of all circuit board components into a deep learning model for feature extraction and fault type classification to obtain the fault classification result. By using temperature maxima, data augmentation, and continuous wavelet transform to convert to time-frequency images, this method overcomes the problems of small temperature difference, few samples, large model size, and poor generalization in existing infrared thermal images, thus improving the accuracy of circuit board component fault detection. Attached Figure Description

[0035] Figure 1 This is a flowchart of a specific implementation of the circuit board component fault detection method based on infrared thermal images and deep learning models of the present invention.

[0036] Figure 2 This is a schematic diagram of a specific example of a manually selected ROI region on a circuit board.

[0037] Figure 3 Yes Figure 2 A schematic diagram of a specific example of a temperature maximum sequence obtained by sampling the temperature maximum in the ROI region of the circuit board shown.

[0038] Figure 4 This embodiment describes the application process of enhancing the circuit board temperature sequence based on TimeGPT.

[0039] Figure 5 It is a time-frequency plot obtained by performing a continuous wavelet transform on the temperature sequence of a region of interest (ROI).

[0040] Figure 6 This is a schematic diagram of the network structure of the MSCB-CAS model in this embodiment;

[0041] Figure 7 This is a schematic diagram of the MSCB module in this embodiment;

[0042] Figure 8 This is a schematic diagram of the CSA module in this embodiment. Detailed Implementation

[0043] The specific embodiments of the present invention will now be described with reference to the accompanying drawings to enable those skilled in the art to better understand the invention. It should be particularly noted that in the following description, detailed descriptions of known functions and designs that might obscure the main content of the invention will be omitted here.

[0044] Figure 1 This is a flowchart of a specific implementation of the circuit board component fault detection method based on infrared thermal images and deep learning models of the present invention.

[0045] In this embodiment, as Figure 1 As shown, the circuit board component fault detection method based on infrared thermal images and deep learning models of the present invention includes the following steps:

[0046] Step S1: Acquire an infrared thermal image sequence of the circuit board during operation.

[0047] A sequence of infrared thermal images of the circuit board under test was captured using a thermal imager. :

[0048]

[0049] in, Indicates at time step Spatiotemporal location Temperature value, To represent the size of the timestamp, i.e. the length of time, Indicates the height of the infrared thermal image, Indicates the width of the infrared thermal image.

[0050] Thus, in time step Real-time thermal images It can be represented as:

[0051]

[0052] Specifically, in this embodiment, an infrared thermal image sequence of the circuit board in operation is acquired using a thermal imager to provide training data for the circuit board fault detection model. The acquired infrared thermal image sequence includes sequences of the circuit board in normal operation and sequences of the circuit board in fault operation.

[0053] Step S2: Extract the temperature maxima sequence of the component region from the infrared thermal image sequence.

[0054] The acquired thermal image sequence is used to select a Region of Interest (ROI), which is the area where a circuit board component is located. Local temperature maximum sampling is used to extract the temperature of each ROI at each time step. The temperature maxima form a temperature maxima sequence. :

[0055]

[0056] in, For the first A set of spatial coordinates of a region of interest. Indicates the first The nth ROI region, i.e. A sequence of maximum temperature values ​​for components on a circuit board.

[0057] In this embodiment, using the IResearch software accompanying the thermal imager, key component areas on the circuit board are manually selected as Regions of Interest (ROIs) in each frame of the acquired infrared thermal image, and the spatial coordinates of each ROI are recorded. Figure 2 A specific example of a manually selected ROI region on a circuit board is given.

[0058] The local temperature maximum sampling method is used to sample the temperature of each ROI region at each time step. Next, extract all spatial locations within the region. Corresponding temperature value The maximum value in the range forms a sequence of temperature maxima for each ROI region. , Figure 3 A specific example is given of the temperature maximum sequence obtained by sampling the temperature maximum in the ROI region of a circuit board.

[0059] Step S3: Perform data augmentation on the temperature maximum sequence

[0060] In this embodiment, data augmentation is performed on the temperature maximum value sequence based on the TimeGPT model, the basic model for time series prediction, such as... Figure 4 As shown, the specific steps include:

[0061] Step S3.1: Sample splicing

[0062] The temperature maximum sequence of all circuit board components The temperature sequence sample dataset is spliced ​​together into a single whole. Represented as:

[0063]

[0064] in, This refers to the number of components on the circuit board.

[0065] Step S3.2: Generation of autoregressive temperature series

[0066] For temperature sequence sample dataset Based on the probabilistic assumptions of the autoregressive model, the temperature maxima sequence for each ROI region is obtained. The joint probability distribution is:

[0067]

[0068] in, It is a series of temperature maxima. Mid-time step Previous temperature sequence.

[0069] In this way, an autoregressive parameterized model is established through a neural network. Through neural network training, it continuously learns and optimizes its parameters. And predict the temperature maximum sequence Every step of the future The conditional probability distribution, and thus the prediction of the future. The joint probability distribution of the sequences at each time step predicts the generation length to be... A new temperature sequence at each time step .

[0070] Step S3.3: Uncertainty Interval Sampling

[0071] Using common-type prediction for temperature series Uncertainty quantification is performed by calculating a non-consistency score for the temperature at each time step. Based on the score distribution, a threshold is determined given a confidence level, and then the threshold and temperature sequence are analyzed. An uncertainty interval is constructed, and then the interval is sampled based on a normal distribution to generate a temperature sequence. .

[0072] Specifically, in this embodiment, the TimeGPT model is fine-tuned by selecting SMAPE (symmetric mean absolute percentage error) as the loss function, setting the number of fine-tuning layers to 3, the number of iterations to 150, and the confidence level to 90%, so that the TimeGPT model learns the autoregressive features of the temperature sequence to generate new temperature sequence samples.

[0073] Step S4: Convert the temperature sequence into a time-frequency image using continuous wavelet transform.

[0074] First, the temperature series Bipolar normalization is performed to eliminate time-frequency plot errors caused by zero-fill boundary processing. The formula for bipolar normalization is:

[0075]

[0076] in, It is a temperature sequence The minimum value, It is a temperature sequence The maximum value, It is a temperature sequence No. Temperature values ​​at each time step.

[0077] Then, a continuous wavelet transform is performed using the Complex Morlet mother wavelet function to obtain the wavelet coefficient matrix. :

[0078]

[0079] in, It is a temperature sequence Length, It is a temperature sequence The Temperature values ​​at each time step For the mother wavelet function of Complex Morlet, For scale parameters, These are the translation parameters.

[0080] The expression for the complex Morlet mother wavelet function is:

[0081]

[0082] in, The normalization constant is For the center frequency, Used to control the width of the Gaussian function.

[0083] The wavelet coefficient matrix obtained by the transformation Modulus extraction yields time-frequency image. :

[0084]

[0085] Specifically, in this embodiment, the center frequency is set as the parameter of the mother wavelet function. =1.5Hz, Gaussian function width =2. Figure 5 This is a time-frequency diagram obtained by performing a continuous wavelet transform on a certain dimension of the circuit board temperature sequence sample in this embodiment.

[0086] Step S5: Use a deep learning model to extract features and classify fault types from time-frequency images.

[0087] Time-frequency images of all circuit board components The data is fed into a deep learning model for feature extraction and fault type classification to obtain the fault classification results.

[0088] In this embodiment, as Figure 6 As shown, the deep learning model is an MSCB-CAS model network structure, including an input layer, four multi-scale depthwise separable convolutional modules (MSCB convolutional blocks), a max pooling layer, a channel filtering attention mechanism module (CSA module), an average pooling layer, and a fully connected classification layer connected in sequence; the input layer contains time-frequency images of all circuit board components. Become The input tensor, where, For the number of channels, For width, For height, the first multi-scale depth-separable convolutional module convolves the input tensor to output a feature map with 64 channels. The second, third, and fourth multi-scale depth-separable convolutional modules convolve the feature map output by the first multi-scale depth-separable convolutional module to output feature maps with 128, 256, and 512 channels, respectively. The max pooling layer pools the feature map with 512 channels and then sends it to the channel filtering attention mechanism module for processing. The processed feature map is then sent to the average pooling layer for pooling, and the pooled feature map is sent to the fully connected classification layer for fault classification.

[0089] like Figure 7 As shown, the multi-scale depth separable convolutional module includes two scales of depth separable convolutional branches: 3×3 and 5×5. The output feature maps of each branch are concatenated in the channel dimension and then fused into a feature map with a preset number of channels through 1×1 convolution. Residual connections are introduced between adjacent MSCB convolutional blocks to avoid gradient vanishing.

[0090] The specific calculation process of the multi-scale depthwise separable convolution module is as follows:

[0091] 5.1.1 The first input feature map is denoted as... ,in This is the number of input channels. and These are the height and width of the input feature map, respectively;

[0092] 5.1.2 For the 3x3 scale convolution module, 3x3 depthwise convolution and pointwise convolution are used to convolve the input feature map, where the depthwise convolution is applied to each input channel. Perform convolutions separately, and use pointwise convolutions and 1x1 convolutions to adjust the feature map to the set number of output channels. Finally, a 3x3 scale convolutional output feature map is obtained. ;

[0093] 5.1.3 For the 5x5 scale convolution module, 5x5 depthwise convolution and pointwise convolution are used to convolve the input feature map, finally obtaining a 5x5 scale convolution output feature map. ;

[0094] 5.1.4 After obtaining feature representations at multiple scales through convolutional kernels of different scales, these feature maps need to be fused together. With feature map The feature maps are concatenated along the channel dimension to obtain the concatenated feature maps. Then, a 1x1 convolutional layer is used to process the feature map. Perform convolutional fusion operation, Adjust the number of channels back The final output feature map after fusion is obtained. ;

[0095] 5.1.5. Obtain the feature map Batch Normalization (BN) and ReLU non-linear activation are performed to facilitate model training.

[0096] like Figure 8 As shown, the channel filtering attention mechanism module includes parallel global average pooling and global max pooling branches. The feature vectors output by the two branches are processed by a fully connected multilayer perceptron (MLP layer, including dimensionality reduction layer, ReLU activation layer, and dimensionality increase layer) and then added together. Channel attention weights are generated by the Sigmoid function, and the channels are filtered in descending order of weight. One channel ( (Hyperparameters are selected for the channels), which are then input into a fully connected classification layer to obtain the fault classification results. Specifically, the fully connected classification layer receives the feature vector output by the CSA module, sets four output nodes, and uses the Softmax activation function to output the probability of each category, thereby achieving fault classification.

[0097] The specific process of the channel filtering algorithm in the channel filtering attention mechanism module is as follows:

[0098] 5.2.1 Input feature map, sort the channel attention weight vector A in descending order, and record the sort index;

[0099] 5.2.2, Before selection Use the selected_index as an index to construct a channel filtering vector;

[0100] 5.2.3 Extract the channel feature maps corresponding to the filtering index and stack them along the channel dimension to obtain the filtered feature maps.

[0101] Although the illustrative specific embodiments of the present invention have been described above to enable those skilled in the art to understand the invention, it should be understood that the invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the invention as defined and determined by the appended claims, and all inventions utilizing the concept of the present invention are protected.

Claims

1. A method for fault detection of circuit board components based on infrared thermal images and deep learning models, characterized in that, Includes the following steps: (1) Acquire infrared thermal image sequence of the circuit board in operation: A sequence of infrared thermal images of the circuit board under test was captured using a thermal imager. : ； in, Indicates at time step Spatiotemporal location Temperature value, To represent the size of the timestamp, i.e. the length of time, Indicates the height of the infrared thermal image, Indicates the width of the infrared thermal image; (2) Extract the temperature maximum value sequence of the component area from the infrared thermal image sequence; The acquired thermal image sequence is used to select a Region of Interest (ROI), which is the area where a circuit board component is located. Local temperature maximum sampling is employed to extract the temperature of each ROI at each time step. The temperature maxima form a temperature maxima sequence. : ； in, For the first A set of spatial coordinates of a region of interest. Indicates the first The nth ROI region, i.e. A sequence of maximum temperature values ​​for components on a circuit board; (3) Perform data augmentation on the temperature maximum value sequence; 3.1) Sample splicing; The temperature maximum sequence of all circuit board components The temperature sequence sample dataset is spliced ​​together into a single whole. Represented as: ； in, This refers to the number of components on the circuit board. 3.2) Generation of autoregressive temperature sequences; An autoregressive parameterized model is established using a neural network. Through neural network training, it continuously learns and optimizes its parameters. And predict the temperature maximum sequence Every step of the future The conditional probability distribution, and thus the prediction of the future. The joint probability distribution of the sequences at time steps predicts the generation length to be... A new temperature sequence at each time step ; 3.3) Uncertainty interval sampling; Using common-type prediction for temperature series Uncertainty quantification is performed by calculating a non-consistency score for the temperature at each time step. Based on the score distribution, a threshold is determined given a confidence level, and then the threshold and temperature sequence are analyzed. An uncertainty interval is constructed, and then the interval is sampled based on a normal distribution to generate a temperature sequence. ; (4) Convert the temperature sequence into a time-frequency image using continuous wavelet transform; First, the temperature series After performing bipolar normalization, a continuous wavelet transform is performed using the Complex Morlet mother wavelet function to obtain the wavelet coefficient matrix. : ； in, It is a temperature sequence Length, It is the normalized temperature sequence The Temperature values ​​at each time step For the mother wavelet function of Complex Morlet, For scale parameters, These are translation parameters; The wavelet coefficient matrix obtained by the transformation Modulus extraction yields time-frequency image. : ； (5) Use deep learning models to extract features and classify fault types from time-frequency images; Time-frequency images of all circuit board components The data is fed into a deep learning model for feature extraction and fault type classification to obtain the fault classification results.

2. The circuit board component fault detection method based on infrared thermal images and deep learning models according to claim 1, characterized in that, The deep learning model comprises, in sequence, an input layer, four multi-scale depthwise separable convolutional modules, a max pooling layer, a channel filtering attention mechanism module, an average pooling layer, and a fully connected classification layer; the input layer contains time-frequency images of all circuit board components. Become The input tensor, where, For the number of channels, For width, For height, the first multi-scale depth-separable convolutional module convolves the input tensor to output a feature map with 64 channels. The second, third, and fourth multi-scale depth-separable convolutional modules convolve the feature map output by the first multi-scale depth-separable convolutional module to output feature maps with 128, 256, and 512 channels, respectively. The max pooling layer pools the feature map with 512 channels and then sends it to the channel filtering attention mechanism module for processing. The processed feature map is then sent to the average pooling layer for pooling, and the pooled feature map is sent to the fully connected classification layer for fault classification.

3. The circuit board component fault detection method based on infrared thermal images and deep learning models according to claim 2, characterized in that, The multi-scale depth separable convolutional module includes two scales of depth separable convolutional branches: 3×3 and 5×5. The output feature maps of each branch are concatenated in the channel dimension and then fused into a feature map with a preset number of channels through 1×1 convolution. Residual connections are introduced between adjacent MSCB convolutional blocks. The computation process of the multi-scale depthwise separable convolution module is as follows: 5.1.1 The first input feature map is denoted as... ,in This is the number of input channels. and These are the height and width of the input feature map, respectively. 5.1.2 For the 3x3 scale convolution module, 3x3 depthwise convolution and pointwise convolution are used to convolve the input feature map, where, Depth convolution for each input channel Perform convolutions separately, and use pointwise convolutions and 1x1 convolutions to adjust the feature map to the set number of output channels. Finally, a 3x3 scale convolutional output feature map is obtained. ; 5.1.3 For the 5x5 scale convolution module, 5x5 depthwise convolution and pointwise convolution are used to convolve the input feature map, finally obtaining a 5x5 scale convolution output feature map. ; 5.1.4 After obtaining feature representations at multiple scales through convolutional kernels of different scales, these feature maps need to be fused together. With feature map The feature maps are concatenated along the channel dimension to obtain the concatenated feature maps. Then, a 1x1 convolutional layer is used to process the feature map. Perform convolutional fusion operation, Adjust the number of channels back The final output feature map after fusion is obtained. ; 5.1.

5. Obtain the feature map Batch Normalization (BN) and ReLU nonlinear activation are performed to facilitate model training.

4. The circuit board component fault detection method based on infrared thermal images and deep learning models according to claim 2, characterized in that, The channel selection attention mechanism module includes parallel global average pooling and global max pooling branches. The feature vectors output by the two branches are processed by fully connected multilayer perceptrons and then summed. Channel attention weights are generated using the Sigmoid function, and the channels are selected in descending order of weight. Each channel is input into a fully connected classification layer to obtain the fault classification result; The specific process of the channel filtering algorithm in the channel filtering attention mechanism module is as follows: 5.2.1 Input feature map, sort the channel attention weight vectors in descending order, and record the sort index; 5.2.2, Before selection Use one index to construct a channel filtering vector; 5.2.3 Extract the channel feature maps corresponding to the filtering index and stack them along the channel dimension to obtain the filtered feature maps.

Images