A Circuit Board Fault Detection Method Based on Infrared Thermography

Abstract

This invention discloses a circuit board fault detection method based on infrared thermometry, belonging to the field of fault identification. The invention first collects a dataset of temperature change patterns from normal circuit boards and various types of faulty circuit boards: Each circuit board is powered on for a fixed time, and an infrared thermal imager is used to collect temperature change data of each chip on the circuit board. The circuit board category is then labeled, such as normal or faulty, completing the dataset construction. Next, the dataset is used to train an improved Informer model to obtain model weight parameters. Finally, newly collected circuit board temperature data is input into the model, and the model outputs the corresponding category to achieve circuit board fault detection.

Description

Technical Field

[0001] This invention belongs to the field of fault identification, specifically a circuit board fault detection method based on infrared temperature measurement. Background Technology

[0002] Existing technologies have limited detection scope: Circuit board defect detection methods based on visible light images are mainly limited to detecting surface defects. They cannot effectively detect non-surface circuit board faults, such as internal chip damage or poor soldering of chip pins. These non-surface faults account for a large proportion of actual circuit board faults; therefore, detection methods based solely on visible light images have a significant detection blind spot.

[0003] Infrared image target identification is difficult: When using infrared thermal imagers to detect non-visual faults on circuit boards, the blurred edges of the infrared images make it impossible to accurately identify the target object whose temperature needs to be measured. Manual calibration of the target area is usually required, which increases the difficulty and workload of the inspection.

[0004] Existing circuit board fault detection methods based on infrared temperature measurement mostly use image difference or temperature profile methods. Image difference is an instantaneous detection method that only judges faults based on temperature data at a specific moment, resulting in low accuracy. Temperature profile methods compare temperature curves over a period of time to judge faults, but their drawback is that they can only determine whether the device temperature is abnormal, and cannot accurately locate the fault. Summary of the Invention

[0005] By combining visible light imaging technology and infrared thermometry technology, and utilizing an Informer-based deep learning classification model, comprehensive detection and classification of circuit board faults can be achieved, thereby improving the accuracy and efficiency of circuit board fault detection.

[0006] First, a dataset of temperature change patterns for normal circuit boards and various types of faulty circuit boards was collected: Each circuit board was powered on for a fixed time, and temperature change data of each chip on the board was collected using an infrared thermal imager. The circuit board category was then labeled, such as normal or faulty, completing the dataset construction. Next, an improved Informer-based model was trained using this dataset to obtain the model weight parameters. Finally, newly collected circuit board temperature data was input into the model, and the model outputs the corresponding category to achieve circuit board fault detection.

[0007] Combining infrared thermometry and time-series classification techniques, a process flow for infrared thermometry-based detection is established, such as... Figure 1As shown. Specifically, an infrared thermal imager is used to collect temperature data of normal circuit boards and circuit boards with different types of faults after being powered on for a period of time, and the data is converted into a multivariate time series. The fault identification result is obtained based on the classification result of the time series. Therefore, the technical solution of this invention is: a circuit board fault detection method based on infrared temperature measurement, the method comprising:

[0008] Step 1: Create a dataset;

[0009] The circuit board includes multiple chips. When all components of the circuit board are working normally, an infrared thermal imager is used to collect the time-series data of the temperature distribution on the front of the circuit board during operation. One chip is then disabled, and the circuit board is powered on again, collecting the time-series data of the temperature distribution on the front of the circuit board during operation. This process is repeated for each chip failure, collecting the time-series data of the temperature distribution on the front of the circuit board during operation. All collected data are compiled into a dataset and labeled with the corresponding faulty chip.

[0010] Step 2: Establish a fault detection network;

[0011] The image detection network includes: a statistical SENet module, a feature embedding module, and an encoder module. The input data is first pre-adjusted by the statistical SENet module to obtain the chip temperature data, and then the feature is encoded using the Embed module. Subsequently, it is fed into the encoder module. The output feature of the encoder module is input into the recognition module to obtain the recognition result.

[0012] Step 3: Use the dataset established in Step 1 to train the fault detection network established in Step 2, and use the trained fault detection network to perform fault detection on the temperature data of the same circuit board.

[0013] Furthermore, the statistical-SENet module in the fault detection network of step 2 includes: a statistical module, a channel weight estimator, and a normalization module. The input chip temperature data is first processed by the statistical module, which outputs eight statistical features of the chip temperature sequence: maximum value, minimum value, average value, median, variance, root mean square, sequence sum, and maximum absolute difference. These eight statistical features are then input into the channel weight estimator, which consists of three layers: ReLU, Tanh, and IN. ReLU and Tanh are two activation functions in machine learning, and IN is InstanceNorm, which is instance normalization computation in machine learning. The channel weight estimator outputs the channel weights of the input data. After normalization, each channel weight is multiplied by the original input data channels before being output.

[0014] Furthermore, the feature embedding module includes: a fully connected layer and a position encoding layer;

[0015] The input data is fed into a fully connected layer and a positional encoding layer, respectively. After the fully connected layer transforms the input data into a higher feature dimension, it is added to the output of the positional encoding layer and used as the output of the feature embedding module.

[0016] The position coding layer uses sine and cosine position coding, which is calculated using the following formula:

[0017]

[0018] Where pos represents the sequence position number in the sequence, d model This represents the feature dimension number, where 2i+1 and 2i represent the odd and even position indices in the feature dimension, respectively, ranging from 0 to d. model -1, PE(pos,2i) represents the position encoding result of the corresponding sequence position pos with feature dimension 2i, and similarly, PE(pos,2i+1) represents the position encoding result of the corresponding sequence position pos with feature dimension 2i+1.

[0019] Furthermore, the encoder module in step 2 comprises three encoders, each with the same structure, consisting of an attention module, a fully convolutional neural network, and a distillation module. The feature map output from the first encoder is then passed through a feature mapping layer, a pooling layer, and two fully connected layers before being input into the first prediction module a1. The feature map output from the second encoder is then passed through a feature mapping layer, a pooling layer, and two fully connected layers before being input into the second prediction module a2. The feature map output from the third encoder is then passed through a feature mapping layer and two fully connected layers before being input into the third prediction module a3. Finally, the prediction results output from the first prediction module a1, the second prediction module a2, and the third prediction module a3 are multiplied by the weight parameters of each module and then summed to obtain the final output. The weight parameters of the three layers are learnable parameters that are updated during network training, thereby implementing a hierarchical attention mechanism. The distillation module compresses the input data, reducing the size of the feature tensor to 1 / 4 of the input size.

[0020] Furthermore, the fully convolutional neural network sequentially includes: a first convolutional layer, a first batch normalization layer, a second convolutional layer, a second batch normalization layer, a third convolutional layer, and a third batch normalization layer; the distillation module sequentially includes: a convolutional layer, a batch normalization layer, a first feature mapping layer, a pooling layer, and a second feature mapping layer.

[0021] Furthermore, the system employing an infrared temperature measurement-based circuit board fault detection method comprises: a basic tool layer, a data layer, a device layer, an application layer, and an interaction layer; the basic tool layer abstracts the most basic functions; the data layer is used to record and maintain data resources; the device layer provides an interface for operating hardware devices; the application layer implements complex functional processes related to the project; and the interaction layer provides a user-friendly graphical user interface for users.

[0022] This invention combines visible light cameras and infrared thermal imagers to collect temperature data from faulty circuit boards, transforming it into a multivariate time series model and optimizing the data processing method. Simultaneously, improvements and optimizations based on the deep learning Informer model enhance the accuracy of circuit board fault detection. The designed data acquisition platform ensures the accuracy and reliability of the data, and experiments verify the advantages of the improved model. These results collectively demonstrate the practical value and contribution of this research to the field of circuit board fault detection. Attached Figure Description

[0023] Figure 1 This describes the process of an infrared temperature measurement detection method.

[0024] Figure 2 This is a system hardware block diagram.

[0025] Figure 3 This is a schematic diagram of an infrared thermal imager.

[0026] Figure 4 The diagram shows the data acquisition platform, where (a) is a model of the data acquisition platform and (b) is a physical image of the data acquisition platform.

[0027] Figure 5 This is a diagram of the software system architecture for the data acquisition platform.

[0028] Figure 6 Temperature acquisition graphs for different types of chips.

[0029] Figure 7 A schematic diagram of the improved distillation module.

[0030] Figure 8 A schematic diagram of the improved SENet network structure.

[0031] Figure 9 This is a schematic diagram of the improved overall network architecture. Detailed Implementation

[0032] 1. Data Acquisition and Processing

[0033] (1) System Hardware: This invention designs a high-efficiency and accurate circuit board image and temperature acquisition platform to provide reliable data support for circuit board defect detection and fault diagnosis. The system hardware structure is as follows: Figure 2As shown, the system is equipped with a three-axis motion platform, capable of adjusting the camera module to accommodate circuit boards of different sizes for clear imaging. The system includes a high-resolution visible light camera and an infrared thermal imager for visual inspection and temperature monitoring of the circuit boards, while a power module simulates the actual operating conditions of the circuit boards. The electrical control box supplies power to the camera module and motion platform, and uses closed-loop control of three motors to ensure precise position adjustment of the motion platform. The platform can simultaneously acquire image and temperature data from the circuit boards and transmit this data to a computer for recording and analysis. Specifically, the selected IRay AT61P infrared thermal imager (such as...) is used. Figure 3 It features high resolution (640×512), clearly displaying the temperature distribution of the circuit board during operation, and has an autofocus function to adapt to circuit boards of different sizes. Figure 4 As shown, the entire platform is ingeniously designed. Through the precise control of the three-axis motion platform and the packaged integration of the camera module, it achieves efficient and accurate imaging of the circuit board, providing strong technical support for circuit board defect detection and fault diagnosis.

[0034] (2) Platform Software System: This invention introduces a mature and stable data acquisition platform software system (such as...). Figure 5 This system employs a modular approach and a front-end / back-end isolation architecture, achieving superior decoupling and scalability. The system software architecture is divided into five layers: a basic tools layer, a data layer, a device layer, an application layer, and an interaction layer. Each layer is built upon the support of the underlying modules and provides functionality to the layers above it. The basic tools layer abstracts the most fundamental functions of the project, such as image processing and log processing; the data layer is used to record and maintain data resources; the device layer provides interfaces for operating hardware devices such as visible light cameras, infrared cameras, and three-axis motion platforms; the application layer implements complex functional processes related to the project; and the interaction layer provides a user-friendly graphical user interface. This layered design achieves functional isolation while ensuring high system performance, representing a balance between decoupling and efficiency.

[0035] (3) Dataset Construction: This invention describes a method for acquiring circuit board images and temperature data using a specially designed acquisition platform. Through registration and spatial transformation techniques, the system utilizes chip identification results from visible light images to guide an infrared thermal imager in accurately acquiring chip temperature data. Given that integrated circuit chips are the primary heat source on the circuit board, and that the operating states of chips influence each other, the experiment collected temperature data from most chips on the circuit board for fault analysis. To eliminate the influence of different ambient temperatures, the temperature data was preprocessed by subtracting the temperature value from the first frame to standardize the data, ensuring that the temperature curves could be aligned under different environments. The experiment used multiple circuit boards, including normal boards and various faulty boards, and acquired temperature data from 38 chips on each board, resulting in 210 data samples. The time series length was 1200, including one type of normal circuit board and eight types of faulty circuit boards. The data is as follows: Figure 6 It clearly reflects the temperature time series changes of chips in various circuit boards, providing strong support for circuit board fault analysis.

[0036] (4) Computing platform: The training and testing of the model of this invention are carried out on a high-performance Windows 10 operating system computing platform, using the Python programming language and deployed based on the PyTorch deep learning framework.

[0037] 2. Informer network structure optimization

[0038] (1) Optimization of the feedforward network based on FCN: Although the core self-attention mechanism of the Informer model can effectively analyze the correlation of all time points in a time series and extract contextual semantic features, its computational complexity is high when the sequence is long, and it is insufficient for extracting local and deep features. In contrast, convolutional neural networks (CNNs) are better at extracting local and deep features. Fully convolutional neural networks (FCNs) are a type of CNN. By setting the number of convolutional kernels to first increase and then decrease, features can be increased to a high-dimensional space, activation selection is performed on high-dimensional features, and then reduced to a low-dimensional space. This process has the same effect as the feedforward network layer in Informer. However, FCN has convolution operations, so it has a greater role in further extracting features. Based on the above reasons, FCN is used to replace the feedforward network layer FFN in Informer.

[0039] (2) Optimization based on multi-scale feature fusion: In deep learning tasks, it is crucial to improve model performance by fusing features at different scales. Shallow features have high resolution and rich details but low semantic meaning, while deep features have strong semantic meaning but low resolution. Therefore, improved distilling networks (such as...) Figure 7The sequence compression ratio was adjusted from 2:1 to 4:1. Simultaneously, a hierarchical attention mechanism was applied to weightedly fuse multi-level prediction results, thereby achieving multi-scale feature fusion and effectively improving the model's ability to perceive detailed information in shallow features.

[0040] (3) Optimization based on channel attention mechanism: To further accurately measure the correlation between data from each chip and fault type, this invention proposes an improved SENet network structure (e.g., Figure 8 To address the issue of limited feature extraction in the original SENet's Squeeze operation, a novel Squeeze method is designed, incorporating prior knowledge to enrich the feature representation by calculating statistical features (such as maximum, minimum, and average values) for each channel. Next, channel weights are initially calculated using two fully connected layers and ReLU and Tanh activation functions. Then, an instance normalization layer is added to further activate or suppress the channel weights through normalization distribution. This method not only considers the chip characteristics of the circuit board temperature time series and its correlation with faults but also enhances the accurate estimation of channel weights by introducing prior knowledge, thereby improving the accuracy of fault classification.

[0041] Based on the above three optimizations, the improved overall structure of the network model is obtained (e.g. Figure 9 The model first pre-conditions the input chip temperature data using the Statistic-SENet module, and then performs feature encoding using the Embed module. Subsequently, the data is fed into the Informer model encoder module, which consists of three Encoder Blocks: a multi-head probabilistic sparse attention layer, a fully convolutional feedforward network layer, and a distillation layer, used for global context understanding, deep local feature extraction, and sequence feature compression, respectively. Through multi-scale feature fusion and hierarchical attention mechanisms, combined with classifiers of different scales, the model performs importance determination and feature extraction on the chip data, ultimately improving the accuracy and robustness of prediction results while reducing computational load.

[0042] 3. Experimental Results

[0043] Experiments on circuit board fault detection were conducted based on the structurally optimized model, with 105 cases each in the training and test sets. Cross-entropy loss was used as the loss function, Adam was used as the optimizer, and the initial learning rate was set to 1e-6. The training run consisted of 355 epochs. Experimental results are shown in Table 1.

[0044] Table 1 Experimental results of fault detection on the improved model circuit board

[0045] Precision Recall F1 Score Forecast / Total Normal category 1.00 1.00 1.00 25 / 25 Fault Category 1 1.00 1.00 1.00 10 / 10 Fault Category 2 1.00 1.00 1.00 10 / 10 Fault Category 3 1.00 1.00 1.00 10 / 10 Fault Category 4 1.00 1.00 1.00 10 / 10 Fault Category 5 1.00 0.90 0.95 9 / 10 Fault Category 6 1.00 1.00 1.00 10 / 10 Fault Category 7 0.91 1.00 0.95 10 / 10 Fault Category 8 1.00 1.00 1.00 10 / 10 Weighted Avg 0.99 0.99 0.99 - Accuracy 0.990 104 / 105

[0046] As shown in Table 1, in the test set, there were 105 samples, with only one incorrectly identified, resulting in an accuracy of 0.990. The weighted precision of 0.99 indicates that 99% of the predicted categories were correct. The weighted recall of 0.99 indicates that the model can identify the vast majority of faults. The F1 score of 0.99 indicates that the model performs well considering both precision and recall. The precision for category 7 was 0.91, indicating that some samples from other categories were incorrectly predicted as category 7. The recall for category 5 was 0.90, indicating that a small number of samples in category 5 were incorrectly identified as other categories; the table shows that one sample from category 5 was incorrectly predicted as category 7. Based on the F1 scores for each category, categories 0, 1, 2, 3, 4, 6, and 8 were correctly predicted, while categories 5 and 7 had a small probability of error. Category 0 represents the standard board, indicating that the model can correctly identify all normal circuit boards and will not identify faulty circuit boards as normal. Circuit boards with fault category 5 have a small probability of being identified as fault category 7. In summary, the improved model can be concluded that it can complete the circuit board fault detection task. The original Informer network structure has three problems in circuit board fault detection, which are optimized accordingly. A fully convolutional network (FCN) replaces the feedforward network (FFN) to improve the model's feature extraction capability. Multi-scale feature fusion is used to optimize the classifier's prediction performance. Simultaneously, a channel attention mechanism module is added to the network front-end to measure the correlation between different chip data and faults, thereby improving the accuracy of circuit board fault detection.

Claims

1. A circuit board fault detection method based on infrared temperature measurement, the method comprising: Step 1: Create a dataset; The circuit board includes multiple chips. When all components of the circuit board are working normally, an infrared thermal imager is used to collect the time-series data of the temperature distribution on the front of the circuit board during operation. One chip is then disabled, and the circuit board is powered on again, collecting the time-series data of the temperature distribution on the front of the circuit board during operation. This process is repeated for each chip failure, collecting the time-series data of the temperature distribution on the front of the circuit board during operation. All collected data are compiled into a dataset and labeled with the corresponding faulty chip. Step 2: Establish a fault detection network; The image detection network includes: a statistical SENet module, a feature embedding module, and an encoder module. The input data is first pre-adjusted by the statistical SENet module to obtain the chip temperature data, and then the feature is encoded using the Embed module. Subsequently, it is fed into the encoder module. The output feature of the encoder module is input into the recognition module to obtain the recognition result. The encoder module comprises three encoders, each with the same structure, consisting of an attention module, a fully convolutional neural network, and a distillation module. The feature map output from the first encoder is then passed through a feature mapping layer, a pooling layer, and two fully connected layers before being input into the first prediction module a1. The feature map output from the second encoder is then passed through a feature mapping layer, a pooling layer, and two fully connected layers before being input into the second prediction module a2. The feature map output from the third encoder is then passed through a feature mapping layer and two fully connected layers before being input into the third prediction module a3. Finally, the prediction results output from the first prediction module a1, the second prediction module a2, and the third prediction module a3 are multiplied by the weight parameters of each module and then summed to obtain the final output. Step 3: Use the dataset established in Step 1 to train the fault detection network established in Step 2, and use the trained fault detection network to perform fault detection on the temperature data of the same circuit board.

2. The circuit board fault detection method based on infrared thermometry as described in claim 1, characterized in that, The statistical-SENet module in the fault detection network of step 2 includes: a statistical module, a channel weight estimator, and a normalization module. The input chip temperature data is first processed by the statistical module, which outputs eight statistical features of the chip temperature sequence: maximum value, minimum value, average value, median, variance, root mean square, sequence sum, and maximum absolute difference. These eight statistical features are then input into the channel weight estimator, which consists of three layers: ReLU, Tanh, and IN. ReLU and Tanh are two activation functions in machine learning, and IN is InstanceNorm, which is instance normalization computation in machine learning. The channel weight estimator outputs the channel weights of the input data. Each channel weight is normalized and then multiplied by the original input data channels before being output.

3. The circuit board fault detection method based on infrared thermometry as described in claim 1, characterized in that, The feature embedding module includes: a fully connected layer and a position encoding layer; The input data is fed into a fully connected layer and a positional encoding layer, respectively. After the fully connected layer transforms the input data into a higher feature dimension, it is added to the output of the positional encoding layer and used as the output of the feature embedding module. The position coding layer uses sine and cosine position coding, which is calculated using the following formula: ； Where pos represents the sequence position number in the sequence, d model This represents the feature dimension number, where 2i+1 and 2i represent the odd and even position indices in the feature dimension, respectively, ranging from 0 to d. model -1, This represents the position encoding result at sequence position pos with feature dimension 2i. Similarly, This represents the position encoding result of the corresponding sequence position pos with a feature dimension of 2i+1.

4. The circuit board fault detection method based on infrared thermometry as described in claim 1, characterized in that, The fully convolutional neural network comprises, in sequence: a first convolutional layer, a first batch normalization layer, a second convolutional layer, a second batch normalization layer, a third convolutional layer, and a third batch normalization layer; the distillation module comprises, in sequence: a convolutional layer, a batch normalization layer, a first feature mapping layer, a pooling layer, and a second feature mapping layer.

5. The circuit board fault detection method based on infrared thermometry as described in claim 1, characterized in that, A system employing an infrared temperature measurement-based circuit board fault detection method is disclosed. This system comprises: a basic tool layer, a data layer, a device layer, an application layer, and an interaction layer. The basic tool layer abstracts the most fundamental functions; the data layer records and maintains data resources; the device layer provides interfaces for operating hardware devices; the application layer implements complex functional processes related to the project; and the interaction layer provides a user-friendly graphical user interface.

Images