Defect detection method and system for prompting distillation based on thermodynamic diagram

Abstract

The invention discloses a defect detection method and system for prompting distillation based on a thermodynamic diagram, and the method comprises the steps: collecting and preprocessing a plurality of product surface pictures, and constructing a defect detection data set; constructing a teacher model and a student model, training the teacher model, and storing the weight of the trained teacher model; constructing a thermodynamic diagram prompt module and embedding the thermodynamic diagram prompt module into the teacher model to obtain a new teacher model, training the thermodynamic diagram prompt module in the new teacher model, and storing the weight of the trained new teacher model; respectively inputting the product surface picture into a new teacher model and a student model, calculating distillation loss according to multi-scale features output by the new teacher model and the student model, and updating the weight of the student model in combination with the distillation loss and the detection loss of the student model to obtain a trained student model; and deploying the student model into the equipment terminal and carrying out defect detection to obtain a detection result. Defect detection is carried out in a deep learning mode, and manpower and material resources are greatly saved.

Description

Technical Field

[0001] This invention relates to the field of product defect detection technology, and in particular to a defect detection method and system based on heat map-guided distillation. Background Technology

[0002] In the field of intelligent manufacturing, product defect detection is a crucial step in ensuring production quality and improving production efficiency. With the increasing automation and scale of production in the manufacturing industry, production line speeds are constantly accelerating, and product types and materials are becoming increasingly diverse, leading to a corresponding increase in the complexity of defect detection. During processing, assembly, and transportation, products may develop defects such as surface scratches, cracks, dents, foreign object residue, or assembly deviations. If these defects are not detected and addressed in a timely manner, product yield will decrease, rework and scrap costs will increase, and ultimately, overall production efficiency will be affected.

[0003] Existing defect detection methods primarily rely on manual quality inspection or machine vision methods based on hand-crafted features. Manual inspection is inefficient, unstable, and ill-suited to the demands of high-speed production lines; methods based on hand-crafted features often lack robustness when dealing with complex textures, different surface materials, and varying lighting conditions. With the development of deep learning and machine vision technologies, using industrial cameras to capture product images and combining them with deep learning models for defect identification has become a common approach to achieving automated inspection. In practical applications, defect detection systems typically need to be deployed on resource-constrained edge devices or embedded platforms to meet the real-time and low-latency requirements of production lines. However, existing deep learning models are complex in structure, have a large number of parameters, and slow inference speeds, making it difficult to meet the high-efficiency inspection needs of manufacturing scenarios. Therefore, improving the detection performance of small models under limited hardware conditions has become a critical issue that urgently needs to be addressed in product defect detection in industrial settings. Summary of the Invention

[0004] This invention provides a defect detection method and system based on heat map-guided distillation to solve the technical problems mentioned in the background art.

[0005] To achieve the above objectives, the technical solution of the present invention is implemented as follows: This invention provides a defect detection method based on thermogram-guided distillation, comprising the following steps: S1. Collect multiple product surface images, preprocess the selected product surface images, and use the preprocessed product surface images to construct a defect detection dataset. S2. Construct a teacher model and a student model for defect detection. Train the teacher model using the defect detection dataset and save the weights of the trained teacher model. The teacher model includes a first backbone network, a multi-scale feature fusion module, and a detection head for defect detection, all connected in sequence. The student model includes a second backbone network, a multi-scale feature fusion module, and a detection head for defect detection, all connected in sequence. S3. Construct a heatmap prompting module and embed it into the multi-scale feature fusion module and detection head of the teacher model to obtain a new teacher model. Load the weights of the original teacher model and train the new teacher model. During the training process, freeze the weights of the original teacher model and train only the heatmap prompting module. Save the weights of the trained new teacher model. S4. Load the weights saved in S3 into the new teacher model. Input the product surface images in the defect detection dataset into the new teacher model and the student model respectively. Calculate the distillation loss based on the multi-scale features output by the new teacher model and the student model and using the constructed multi-scale context enhancement network. Update the weights of the student model by combining the distillation loss and the original detection loss of the student model to obtain the trained student model. S5. Deploy the trained student model to the device and use the trained student model on the device to perform defect detection and obtain the detection results.

[0006] Furthermore, the preprocessing in S1 includes defect annotation and data augmentation; wherein, data augmentation includes one or more of scaling, cropping, horizontal flipping, and contrast enhancement.

[0007] Furthermore, the first backbone network adopts a first residual neural network with a depth of 50 layers, which is composed of multiple residual units stacked together; the first backbone network is used for multi-scale feature extraction. The multi-scale feature fusion module is used to perform multi-scale feature fusion; The detection head includes a parallel classification branch and a regression branch, where the classification branch is used to output the class probability of the corresponding position for each scale feature map; the regression branch is used to predict the coordinate offset of each preset anchor box. The second backbone network uses a first residual neural network with a depth of 18 layers.

[0008] Furthermore, step S3 specifically includes the following steps: S31. Construct a heatmap prompting module. The heatmap prompting module consists of three sequentially connected convolutional neural networks. Each convolutional neural network is composed of convolutional layers and activation functions, which are used to further process the input feature map and output a spatial response map. S32. Embed the heatmap prompting module into the multi-scale feature fusion module and the detection head of the teacher model to obtain a new teacher model; S33. Load the weights of the original teacher model into the new teacher model and freeze the weights of the original teacher model in the new teacher model to avoid the training of the heatmap prompting module interfering with the original detection capability of the original teacher model. S34. Train the new teacher model using the defect detection dataset. The loss function is the heatmap alignment loss. Continuously optimize the parameters of the heatmap prompting module through the standard backpropagation algorithm. After training, save the weights of the new teacher model, including all weight parameters of the original teacher model and the heatmap prompting module.

[0009] Furthermore, step S34 specifically includes the following steps: S341. Randomly select a preprocessed product surface image from the defect detection dataset and input it into the new teacher model. After the product surface image passes through the first backbone network and the multi-scale feature fusion module of the new teacher model, feature maps of different scales are obtained. , ; S342. Convert feature maps of different scales. The input is fed into the heatmap hint module to generate the corresponding heatmap hint features. Heatmaps highlight features Used to indicate the spatial area currently of interest to the heatmap prompt module; S343. Convert feature maps of different scales. Averaging is performed along the channel dimension to obtain the average heatmap of channel response intensity. Subsequently, the average heatmap of the channel response intensity was analyzed. Normalization was performed to normalize the numerical range to [0,1]. The normalization method used was min-max normalization, resulting in the normalized average heatmap of the channel response intensity. ; S344, Based on the normalized channel response intensity average thermogram The heatmap hint features output by the truth value and heatmap hint module Calculate the heatmap alignment loss; S345, loop through S341 to S344, continuously optimize the parameters of the heatmap prompting module using the standard backpropagation algorithm. After training is complete, save the weights of the new teacher model, including all weight parameters of the original teacher model and the heatmap prompting module.

[0010] Furthermore, the heatmap hint feature in S342 The calculation formula is as follows: ; in, , , To represent different convolution operations, This represents the activation function. Represents the Sigmoid function; The average thermogram of channel response intensity in S343 The calculation formula is as follows: ; in, The spatial coordinates of the representative feature map The number of channels in the feature map; This represents the k-th channel of the feature map; Represents spatial location coordinates Feature map at the location; The normalized channel response intensity average thermogram in S343 The calculation formula is as follows: ; in, The average heatmap representing the channel response intensity Minimum value in spatial dimension; The average heatmap representing the channel response intensity The maximum value in the spatial dimension; Represents a constant; The formula for calculating the heatmap alignment loss in S344 is as follows: ; in, This indicates the heatmap alignment loss; , These represent the height and width of the feature map, respectively. Represents spatial location coordinates The heatmap at that location suggests certain features; Represents spatial location coordinates The normalized channel response intensity average thermogram at the location.

[0011] Furthermore, step S4 specifically includes the following steps: S41. Load the weights saved in S3 into the new teacher model; S42. Input the defect detection dataset into the new teacher model and student model respectively; S43. After the product surface image is processed by the first backbone network and multi-scale feature fusion module of the new teacher model, multi-scale features of the new teacher model are generated. The product surface image, after passing through the first backbone network and multi-scale feature fusion module of the student model, generates the multi-scale features of the student model. ,in, ; S44. Multi-scale features output from the new teacher model and student model and The constructed multi-scale contextual enhancement network is used to calculate the distillation loss, which includes heatmap distillation loss. and context distillation loss ; S45. The weight parameters of the student model are updated using the backpropagation algorithm, combining the distillation loss and the original detection loss of the student model; whereby the original detection loss of the student model includes the classification loss. and regression loss ; S46. Iterate through S41 to S45 until the preset iteration stopping condition is met to obtain the trained student model.

[0012] Furthermore, S44 specifically includes the following steps: S441. Multi-scale features output by the new teacher model The input is fed into the trained heatmap cue module to generate corresponding heatmap cue features. ; S442. Using heatmaps to highlight features Respectively with multi-scale features Multi-scale features Perform element-wise dot product to obtain the weighted feature map of the new teacher model. and the weighted student model feature map ; S443. Calculate the heatmap distillation loss using the weighted feature map of the new teacher model and the weighted feature map of the student model. ; S444. Construct a multi-scale context augmentation network for extracting contextual relationships. The multi-scale context augmentation network includes three cascaded multi-scale context augmentation modules. Each multi-scale context enhancement module includes a first normalization layer, a deep convolutional layer, a first activation function SiLU, a second activation function SiLU, a two-dimensional spatial state selection module SS2D, and a second normalization layer. The output of the first normalization layer is split into two branches: a main branch and a gated branch. The gated branch is fully connected to the first activation function SiLU to obtain a gated feature G. The main branch is fully connected to the deep convolutional layer and then connected to the second activation function SiLU, the two-dimensional spatial state selection module SS2D, and the second normalization layer to obtain a context feature B. The gated feature G and the context feature B are multiplied element-wise and then fully connected to the input of the first normalization layer. S445, Multi-scale features Multi-scale features The inputs are fed into a multi-scale context enhancement network to obtain the context feature maps of the new teacher model. and the context feature map of the student model ; Utilizing context feature maps and context feature maps Constructing context distillation loss .

[0013] Furthermore, the weighted new teacher model feature map in S442 and the weighted student model feature map The calculation formulas are as follows: ; ; in, This represents the element-wise dot product operation; Thermographic distillation loss in S443 The calculation formula is as follows: ; Where c represents the current number of channels; Context distillation loss in S445 The calculation formula is as follows: ; The classification loss and regression loss The calculation formulas are as follows: ; ; in, This represents the predicted target probability. It is a factor used to control the weights of positive and negative samples. It is a weighting factor used to control the difficulty of samples. It is the difference between the predicted value and the actual value.

[0014] In another aspect, the present invention provides a defect detection system based on heatmap-guided distillation, which is configured to or execute a defect detection method based on heatmap-guided distillation.

[0015] The beneficial effects of this invention are: This invention discloses a defect detection method based on heatmap-guided distillation, which aims to solve the problems of low efficiency and poor robustness of traditional defect detection methods, as well as the limitations of existing deep learning models on edge devices in terms of computing resources and slow inference speed, making it difficult to achieve high-speed, real-time defect detection.

[0016] Meanwhile, this invention employs deep learning for defect detection, significantly saving human and material resources. Furthermore, it constructs a teacher-student knowledge distillation framework (a combination of a teacher model, a student model, and distillation loss), efficiently transferring knowledge from the teacher model to a lightweight student model through a heatmap cue module and a multi-scale contextual enhancement network. Specifically, this invention designs a heatmap cue module to guide the student model to focus on the target region and suppress background interference; simultaneously, combined with a multi-scale contextual enhancement network, it captures global contextual information on features at different scales, thereby providing richer contextual information during distillation and enhancing the student model's understanding and transfer of knowledge from the teacher model. Applying the lightweight student model trained by distillation to industrial inspection scenarios can effectively reduce model storage requirements while maintaining good detection accuracy and inference efficiency. Attached Figure Description

[0017] Figure 1 This is a flowchart of the defect detection method in this invention; Figure 2 This is a block diagram illustrating the operational principle of the defect detection method in this invention; Figure 3 This is a structural block diagram of the heat map prompting module in this invention; Figure 4 This is a structural block diagram of the multi-scale context enhancement network in this invention. Detailed Implementation

[0018] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many other different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0019] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0020] Reference Figure 1 and Figure 2 This application provides a defect detection method based on heat map-guided distillation, comprising the following steps: S1. Collect multiple product surface images, preprocess the selected product surface images, and use the preprocessed product surface images to construct a defect detection dataset. S2. Construct a teacher model and a (lightweight) student model for defect detection. Train the teacher model using the defect detection dataset and save the weights of the trained teacher model. The teacher model consists of a first backbone network, a multi-scale feature fusion module, and a detection head for defect detection, all connected in sequence. The student model consists of a second backbone network, a multi-scale feature fusion module, and a detection head for defect detection, all connected in sequence. S3. Construct a heatmap prompting module and embed it into the multi-scale feature fusion module and detection head of the teacher model to obtain a new teacher model. Load the weights of the original teacher model and train the new teacher model. During the training process, freeze the weights of the original teacher model and train only the heatmap prompting module. Save the weights of the trained new teacher model. S4. Load the weights saved in S3 into the new teacher model. Input the product surface images in the defect detection dataset into the new teacher model and the student model respectively. Calculate the distillation loss based on the multi-scale features output by the new teacher model and the student model and using the constructed multi-scale context enhancement network. Update the weights of the student model by combining the distillation loss and the original detection loss of the student model to obtain the trained student model. S5. Deploy the trained student model to the device and use the trained student model on the device to perform defect detection and obtain the detection results.

[0021] In some embodiments, the preprocessing in S1 includes defect labeling and data augmentation; wherein, data augmentation includes one or more of scaling, cropping, horizontal flipping, and contrast enhancement, which can increase the diversity and robustness of data and improve the generalization ability of the model.

[0022] In some embodiments, S1 specifically includes the following steps: S11. Collect multiple product surface images and select product surface images with different shooting times, angles, and scenes that meet the quality requirements. S12. Use annotation tools (such as the image annotation tool Label Me) to accurately annotate the defect targets in the image, ensuring that the bounding box of each defect is correctly covered, and generate an annotation file in the corresponding format (such as JSON). S13. Perform image enhancement processing on the labeled product surface image to expand the diversity of the sample. This completes the preprocessing of the product surface image. S14. Construct a defect detection dataset using the preprocessed product surface images, and divide the defect detection dataset into a training set and a validation set according to a set ratio for subsequent model training and performance evaluation.

[0023] In some embodiments, the first backbone network employs a first residual neural network with a depth of 50 layers, namely a ResNet50 network; the first residual neural network is composed of multiple stacked residual units; it has strong multi-scale feature extraction capabilities and can effectively capture the spatial and semantic information of the target. The first backbone network is used for multi-scale feature extraction; The Feature Pyramid Network (FPN) module is used to perform multi-scale feature fusion to improve the information fusion effect between features of different scales. The FPN module fuses large-scale features (i.e., low-level high-resolution feature maps) with small-scale features (i.e., high-level semantic features) through lateral connections, thereby enhancing the model's ability to perceive targets of different sizes and achieving more detailed and comprehensive feature representation. The detection head includes parallel classification and regression branches. Both the classification and regression branches use lightweight fully convolutional sub-networks and are respectively connected to feature maps of different scales in the multi-scale feature fusion module. The classification branch is used to output the class probability of the corresponding position for each scale feature map; the regression branch is used to predict the coordinate offset of each preset anchor box and regress it to the precise position of the target.

[0024] The second backbone network uses a first residual neural network with a depth of 18 layers, namely the ResNet18 network.

[0025] In some embodiments, S2 specifically includes the following steps: S21. Supervised training of the teacher model is performed using the training set. The loss function during the training process includes two parts: classification loss and regression loss. S22, iterate through S21 until the loss function converges. During the iteration, the weight parameters of the teacher model are continuously optimized using the standard backpropagation algorithm until the weights of the teacher model are validated using a validation set and the performance meets the preset requirements. After training is completed, all weight parameters of the teacher model are saved.

[0026] In some embodiments, S3 specifically includes the following steps: S31. Construct a heatmap prompting module. The heatmap prompting module consists of three sequentially connected convolutional neural networks. Each convolutional neural network consists of convolutional layers and activation functions. For the specific structure, please refer to [reference needed]. Figure 3It is used to further process the input feature map and output a spatial response map, wherein the convolutional layer has a built-in normalization layer; S32. The heatmap prompt module is embedded between the multi-scale feature fusion module and the detection head in the teacher model. This embedding position can make full use of the feature maps of different scales extracted by the multi-scale feature fusion module. After embedding, a new teacher model is obtained. S33. Load the weights of the original teacher model into the new teacher model and freeze the weights of the original teacher model in the new teacher model to avoid the training of the heatmap prompting module from interfering with the original detection capability of the original teacher model; freezing means setting all teacher model parameters not to participate in gradient backpropagation and update during the training process, thereby maintaining the stability of its feature extraction and detection performance. S34. Train the new teacher model using the training set in the defect detection dataset. The loss function is the heatmap alignment loss. Continuously optimize the parameters of the heatmap prompting module through the standard backpropagation algorithm. After training, save the weights of the new teacher model, including all weight parameters of the original teacher model and the heatmap prompting module.

[0027] In some embodiments, S34 specifically includes the following steps: S341. Randomly select a preprocessed product surface image from the defect detection dataset and input it into the new teacher model. After the product surface image passes through the first backbone network and the multi-scale feature fusion module of the new teacher model, feature maps of different scales are obtained. , ; S342. Convert feature maps of different scales. The input is fed into the heatmap hint module to generate the corresponding heatmap hint features. Heatmaps highlight features Used to indicate the spatial area currently of interest to the heatmap prompt module; S343. Convert feature maps of different scales. Averaging is performed along the channel dimension to obtain the average heatmap of channel response intensity. Subsequently, the average heatmap of the channel response intensity was analyzed. Normalization was performed to normalize the numerical range to [0,1]. The normalization method used was min-max normalization, resulting in the normalized average heatmap of the channel response intensity. ; S344, Based on the normalized channel response intensity average thermogram The heatmap hint features output by the truth value and heatmap hint module Calculate the heatmap alignment loss; S345, loop through S341 to S344, continuously optimize the parameters of the heatmap prompting module using the standard backpropagation algorithm. After training is complete, save the weights of the new teacher model, including all weight parameters of the original teacher model and the heatmap prompting module.

[0028] In some embodiments, the heatmap hint feature in S342 The calculation formula is as follows: ; in, , , To represent different convolution operations, This represents the activation function. Represents the Sigmoid function; The average thermogram of channel response intensity in S343 The calculation formula is as follows: ; in, The spatial coordinates of the representative feature map The number of channels in the feature map; This represents the k-th channel of the feature map; Represents spatial location coordinates Feature map at the location; The normalized channel response intensity average thermogram in S343 The calculation formula is as follows: ; in, The average heatmap representing the channel response intensity Minimum value in spatial dimension; The average heatmap representing the channel response intensity The maximum value in the spatial dimension; Represents a constant; The formula for calculating the heatmap alignment loss in S344 is as follows: ; in, This indicates the heatmap alignment loss; , These represent the height and width of the feature map, respectively. Represents spatial location coordinates The heatmap at that location suggests certain features; Represents spatial location coordinates The normalized channel response intensity average thermogram at the location.

[0029] In some embodiments, S4 specifically includes the following steps: S41. Load the weights saved in S3 into the new teacher model; S42. Input the training set in the defect detection dataset into the new teacher model and student model respectively; S43. After the product surface image is processed by the first backbone network and multi-scale feature fusion module of the new teacher model, multi-scale features of the new teacher model are generated. The product surface image, after passing through the first backbone network and multi-scale feature fusion module of the student model, generates the multi-scale features of the student model. ,in, ; S44. Multi-scale features output from the new teacher model and student model and The constructed multi-scale contextual enhancement network is used to calculate the distillation loss, which includes heatmap distillation loss. and context distillation loss ; S45. Construct a total loss function by combining the distillation loss and the original detection loss of the student model. Update the weight parameters of the student model based on the total loss function and using the backpropagation algorithm. The original detection loss of the student model includes the classification loss. and regression loss .

[0030] The total loss function is as follows: ; S46. Iterate through S41 to S45 until the preset iteration stopping condition is met to obtain the trained student model.

[0031] In some embodiments, S44 specifically includes the following steps: S441. Multi-scale features output by the new teacher model The input is fed into the trained heatmap cueing module to generate heatmap cueing features for guiding attention to key areas during the distillation process. ; S442. Using heatmaps to highlight features Respectively with multi-scale features Multi-scale features Perform element-wise dot product to obtain the weighted feature map of the new teacher model. and the weighted student model feature map ; S443. Calculate the heatmap distillation loss using the weighted feature map of the new teacher model and the weighted feature map of the student model. Thermographic distillation loss This is used to guide student models to focus on key areas indicated by heatmaps, thereby improving the detection performance and robustness of student models; S444. Construct a multi-scale context augmentation network for extracting contextual relationships. The multi-scale context augmentation network includes three cascaded multi-scale context augmentation modules. Reference Figure 4 Each multi-scale context enhancement module includes a first normalization layer, a deep convolutional layer, a first activation function SiLU, a second activation function SiLU, a two-dimensional spatial state selection module SS2D, and a second normalization layer. The output of the first normalization layer is split into two branches: a main branch and a gated branch. The gated branch is fully connected to the first activation function SiLU to obtain a gated feature G. The main branch is fully connected to the deep convolutional layer and then connected to the second activation function SiLU, the two-dimensional spatial state selection module SS2D, and the second normalization layer to obtain a context feature B. The gated feature G and the context feature B are multiplied element-wise and then fully connected to the input of the first normalization layer. S445, Multi-scale features Multi-scale features The inputs are fed into a multi-scale context enhancement network to obtain the context feature maps of the new teacher model. and the context feature map of the student model ; Utilizing context feature maps and context feature maps Constructing context distillation loss .

[0032] Specifically, firstly, multi-scale features Multi-scale features The inputs are fed into a multi-scale context enhancement network, and the first normalization layer processes the input features. (including multi-scale features) and multi-scale features Normalize the features to obtain the normalized features. Then, the output of the first normalization layer is split into two branches: a main branch and a gated branch. The gated branch is fully connected to the first activation function SiLU to obtain the gated features. After the main branch is fully connected to the deep convolutional layer, it is connected to the second activation function SiLU, the two-dimensional spatial modeling SS2D, and the second normalization layer to obtain contextual features. Gating features With context features After element-wise dot product, and then through a second normalization layer and a fully connected layer... The fusion result was obtained. Then merge the results Input features The residuals are summed to obtain the context feature output. ; Context feature output The output of the first multi-scale context enhancement module is expressed by the following formula: ; in, , SiLU represents the first activation function. This represents the second activation function, SiLU. Represents a deep convolutional layer. Represents two-dimensional space modeling. , , This represents a fully connected operation. , These represent the first normalization layer and the second normalization layer, respectively.

[0033] The second and third multi-scale context enhancement modules then operate in the same way as the first. After these three modules, the context feature map of the new teacher model is obtained. and the context feature map of the student model Utilizing context feature maps and context feature maps Constructing context distillation loss .

[0034] In some embodiments, the weighted new teacher model feature map in S442 and the weighted student model feature map The calculation formulas are as follows: ; ; in, This represents the element-wise dot product operation; Thermographic distillation loss in S443 The calculation formula is as follows: ; Where c represents the current number of channels; Context distillation loss in S445 The calculation formula is as follows: ; The classification loss and regression loss The calculation formulas are as follows: ; ; in, This represents the predicted target probability. It is a factor used to control the weights of positive and negative samples. It is a weighting factor used to control the difficulty of samples. It is the difference between the predicted value and the actual value.

[0035] This invention discloses a defect detection method based on heatmap-guided distillation, which aims to solve the problems of low efficiency and poor robustness of traditional defect detection methods, as well as the limitations of existing deep learning models on edge devices in terms of computing resources and slow inference speed, making it difficult to achieve high-speed, real-time defect detection.

[0036] Meanwhile, this invention employs deep learning for defect detection, significantly saving human and material resources. Furthermore, it constructs a teacher-student knowledge distillation framework (a combination of a teacher model, a student model, and distillation loss), efficiently transferring knowledge from the teacher model to a lightweight student model through a heatmap cue module and a multi-scale contextual enhancement network. Specifically, this invention designs a heatmap cue module to guide the student model to focus on the target region and suppress background interference; simultaneously, combined with a multi-scale contextual enhancement network, it captures global contextual information on features at different scales, thereby providing richer contextual information during distillation and enhancing the student model's understanding and transfer of knowledge from the teacher model. Applying the lightweight student model trained by distillation to industrial inspection scenarios can effectively reduce model storage requirements while maintaining good detection accuracy and inference efficiency.

[0037] In another aspect, the present invention provides a defect detection system based on heatmap-guided distillation, which is configured to or execute a defect detection method based on heatmap-guided distillation.

[0038] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A defect detection method based on thermographic indication of distillation, characterized in that, Includes the following steps: S1. Collect multiple product surface images, preprocess the selected product surface images, and use the preprocessed product surface images to construct a defect detection dataset. S2. Construct a teacher model and a student model for defect detection. Train the teacher model using the defect detection dataset and save the weights of the trained teacher model. The teacher model includes a first backbone network, a multi-scale feature fusion module, and a detection head for defect detection, all connected in sequence. The student model includes a second backbone network, a multi-scale feature fusion module, and a detection head for defect detection, all connected in sequence. S3. Construct a heatmap prompting module and embed it into the multi-scale feature fusion module and detection head of the teacher model to obtain a new teacher model. Load the weights of the original teacher model and train the new teacher model. During the training process, freeze the weights of the original teacher model and train only the heatmap prompting module. Save the weights of the trained new teacher model. S4. Load the weights saved in S3 into the new teacher model. Input the product surface images in the defect detection dataset into the new teacher model and the student model respectively. Calculate the distillation loss based on the multi-scale features output by the new teacher model and the student model and using the constructed multi-scale context enhancement network. Update the weights of the student model by combining the distillation loss and the original detection loss of the student model to obtain the trained student model. S5. Deploy the trained student model to the device and use the trained student model on the device to perform defect detection and obtain the detection results.

2. The defect detection method based on thermogram-guided distillation according to claim 1, characterized in that, The preprocessing in S1 includes defect annotation and data augmentation; wherein, data augmentation includes one or more of scaling, cropping, horizontal flipping, and contrast enhancement.

3. The defect detection method based on thermogram-guided distillation according to claim 1, characterized in that, The first backbone network adopts a first residual neural network with a depth of 50 layers, which is composed of multiple residual units stacked together; the first backbone network is used for multi-scale feature extraction. The multi-scale feature fusion module is used to perform multi-scale feature fusion; The detection head includes a parallel classification branch and a regression branch, where the classification branch is used to output the class probability of the corresponding position for each scale feature map; the regression branch is used to predict the coordinate offset of each preset anchor box. The second backbone network uses a first residual neural network with a depth of 18 layers.

4. The defect detection method based on thermogram-guided distillation according to claim 1, characterized in that, S3 specifically includes the following steps: S31. Construct a heatmap prompting module. The heatmap prompting module consists of three sequentially connected convolutional neural networks. Each convolutional neural network is composed of convolutional layers and activation functions, which are used to further process the input feature map and output a spatial response map. S32. Embed the heatmap prompting module into the multi-scale feature fusion module and the detection head of the teacher model to obtain a new teacher model; S33. Load the weights of the original teacher model into the new teacher model and freeze the weights of the original teacher model in the new teacher model to avoid the training of the heatmap prompting module interfering with the original detection capability of the original teacher model. S34. Train the new teacher model using the defect detection dataset. The loss function is the heatmap alignment loss. Continuously optimize the parameters of the heatmap prompting module through the standard backpropagation algorithm. After training, save the weights of the new teacher model, including all weight parameters of the original teacher model and the heatmap prompting module.

5. The defect detection method based on thermogram-guided distillation according to claim 4, characterized in that, S34 specifically includes the following steps: S341. Randomly select a preprocessed product surface image from the defect detection dataset and input it into the new teacher model. After the product surface image passes through the first backbone network and the multi-scale feature fusion module of the new teacher model, feature maps of different scales are obtained. , ; S342. Convert feature maps of different scales. The input is fed into the heatmap hint module to generate the corresponding heatmap hint features. Heatmaps highlight features Used to indicate the spatial area currently of interest to the heatmap prompt module; S343. Convert feature maps of different scales. Averaging is performed along the channel dimension to obtain the average heatmap of channel response intensity. ; Subsequently, the average heatmap of the channel response intensity was analyzed. Normalization was performed to normalize the numerical range to [0,1]. The normalization method used was min-max normalization, resulting in the normalized average heatmap of the channel response intensity. ; S344, Based on the normalized channel response intensity average thermogram The heatmap hint features output by the truth value and heatmap hint module Calculate the heatmap alignment loss; S345, loop through S341 to S344, continuously optimize the parameters of the heatmap prompting module using the standard backpropagation algorithm. After training is complete, save the weights of the new teacher model, including all weight parameters of the original teacher model and the heatmap prompting module.

6. The defect detection method based on thermogram-guided distillation according to claim 5, characterized in that, The heat map indication features in S342 The calculation formula is as follows: ； in, , , To represent different convolution operations, This represents the activation function. Represents the Sigmoid function; The average thermogram of channel response intensity in S343 The calculation formula is as follows: ； in, The spatial coordinates of the representative feature map The number of channels in the feature map; This represents the k-th channel of the feature map; Represents spatial location coordinates Feature map at the location; The normalized channel response intensity average thermogram in S343 The calculation formula is as follows: ； in, The average heatmap representing the channel response intensity Minimum value in spatial dimension; The average heatmap representing the channel response intensity The maximum value in the spatial dimension; Represents a constant; The formula for calculating the heatmap alignment loss in S344 is as follows: ； in, This indicates the heatmap alignment loss; , These represent the height and width of the feature map, respectively. Represents spatial location coordinates The heatmap at that location suggests certain features; Represents spatial location coordinates The normalized channel response intensity average thermogram at the location.

7. The defect detection method based on thermogram-guided distillation according to claim 6, characterized in that, S4 specifically includes the following steps: S41. Load the weights saved in S3 into the new teacher model; S42. Input the defect detection dataset into the new teacher model and student model respectively; S43. After the product surface image is processed by the first backbone network and multi-scale feature fusion module of the new teacher model, multi-scale features of the new teacher model are generated. The product surface image, after passing through the first backbone network and multi-scale feature fusion module of the student model, generates the multi-scale features of the student model. ,in, ; S44. Multi-scale features output from the new teacher model and student model and The constructed multi-scale contextual enhancement network is used to calculate the distillation loss, which includes heatmap distillation loss. and context distillation loss ; S45. The weight parameters of the student model are updated using the backpropagation algorithm, combining the distillation loss and the original detection loss of the student model; whereby the original detection loss of the student model includes the classification loss. and regression loss ; S46. Iterate through S41 to S45 until the preset iteration stopping condition is met to obtain the trained student model.

8. The defect detection method based on thermogram-guided distillation according to claim 7, characterized in that, S44 specifically includes the following steps: S441. Multi-scale features output by the new teacher model The input is fed into the trained heatmap cue module to generate corresponding heatmap cue features. ; S442. Using heatmaps to highlight features Respectively with multi-scale features Multi-scale features Perform element-wise dot product to obtain the weighted feature map of the new teacher model. and the weighted student model feature map ; S443. Calculate the heatmap distillation loss using the weighted feature map of the new teacher model and the weighted feature map of the student model. ; S444. Construct a multi-scale context augmentation network for extracting contextual relationships. The multi-scale context augmentation network includes three cascaded multi-scale context augmentation modules. Each multi-scale context enhancement module includes a first normalization layer, a deep convolutional layer, a first activation function SiLU, a second activation function SiLU, a two-dimensional spatial state selection module SS2D, and a second normalization layer. The output of the first normalization layer is split into two branches: a main branch and a gated branch. The gated branch is fully connected to the first activation function SiLU to obtain a gated feature G. The main branch is fully connected to the deep convolutional layer and then connected to the second activation function SiLU, the two-dimensional spatial state selection module SS2D, and the second normalization layer to obtain a context feature B. The gated feature G and the context feature B are multiplied element-wise and then fully connected to the input of the first normalization layer. S445, Multi-scale features Multi-scale features The inputs are fed into a multi-scale context enhancement network to obtain the context feature maps of the new teacher model. and the context feature map of the student model ; Utilizing context feature maps and context feature maps Constructing context distillation loss .

9. The defect detection method based on thermogram-guided distillation according to claim 8, characterized in that, The weighted new teacher model feature map in S442 and the weighted student model feature map The calculation formulas are as follows: ； ； in, This represents the element-wise dot product operation; Thermographic distillation loss in S443 The calculation formula is as follows: ； Where c represents the current number of channels; Context distillation loss in S445 The calculation formula is as follows: ； The classification loss and regression loss The calculation formulas are as follows: ； ； in, This represents the predicted target probability. It is a factor used to control the weights of positive and negative samples. It is a weighting factor used to control the difficulty of samples. It is the difference between the predicted value and the actual value.

10. A defect detection system based on thermographic indication of distillation, characterized in that, The defect detection method based on heat map-guided distillation as described in any one of claims 1 to 9 is configured or executed.

Images