A defect detection method and device based on a heat map prompt

Abstract

The application discloses a defect detection method and device based on a heat map prompt, and comprises the following steps: acquiring image data sets of various industrial defect types; obtaining a predicted heat map according to the image data sets; performing supervised training according to the average error loss between the predicted heat map and a heat map true label to obtain a trained industrial defect prompt model; and performing prediction on a defect image to be detected based on the trained industrial defect prompt model to obtain a defect region by segmentation. The application proposes a new skeletonized heat map prompt network based on a basic model, automatically provides prompt information for a visual basic model, and improves the precision of an automatic industrial defect detection task.

Description

Technical Field

[0001] This invention relates to the field of visual image technology, and in particular to a defect detection method and apparatus based on heatmap cues. Background Technology

[0002] In industrial production, limitations and deficiencies in existing technology and working conditions can easily affect the quality of finished products. Surface defects are the most direct manifestation of this quality issue. Therefore, to ensure a high pass rate and reliable quality, surface defect detection is essential. Quality control has long relied on manual inspection, which limits production quality and efficiency. Consequently, increasing research is focusing on automated industrial defect detection.

[0003] In recent years, foundational visual models have achieved significant success. In the field of vision, these models are pre-trained on large-scale datasets, with their transformer (a training model based on an attention mechanism) backbone demonstrating powerful feature extraction capabilities. These pre-trained foundational models are often fine-tuned and applied to downstream tasks, exhibiting excellent generalization and transfer capabilities. However, these fine-tuned foundational models are not suitable for existing automated industrial defect detection tasks, thus requiring manual prompting of the visual foundational models and preventing automated prompting.

[0004] Therefore, existing technologies still need improvement. Summary of the Invention

[0005] The technical problem to be solved by the present invention is that, in view of the defects of the prior art, the present invention provides a defect detection method and device based on heat map prompts, so as to solve the problem that the existing basic model cannot automatically provide visual basic model prompt information.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows:

[0007] In a first aspect, the present invention provides a defect detection method based on heatmap prompts, comprising:

[0008] Acquire image datasets of various types of industrial defects;

[0009] A predicted heatmap is extracted from the image dataset;

[0010] Supervised training is performed based on the average error loss between the predicted heatmap and the actual heatmap label to obtain the trained industrial defect alert model.

[0011] Based on the trained industrial defect indication model, the defect image to be tested is predicted and the defect region is segmented.

[0012] In one implementation, acquiring an image dataset of multiple types of industrial defects includes:

[0013] Obtain image datasets of various industrial defect types; wherein, the image datasets are anomaly datasets with pixel-level segmentation annotations;

[0014] The image dataset is divided into a training set and a test set according to a preset ratio;

[0015] The training set and the test set are preprocessed to resize all images to the same size.

[0016] In one implementation, the step of extracting the predicted heatmap from the image dataset includes:

[0017] The images in the training set are input into the transformer encoder in the frozen SAM base model to obtain intermediate layer image features and final image features;

[0018] The intermediate layer image features are input into a feature fusion extraction network to obtain fused features;

[0019] The fused features are input into the heatmap prediction head to obtain the predicted heatmap.

[0020] In one implementation, the heatmap prediction head includes: a first convolutional layer and a second convolutional layer; the first convolutional layer is a lower convolutional layer, and the second convolutional layer is a 1*1 convolutional layer.

[0021] In one implementation, the step of inputting the fused features into the heatmap prediction head to obtain the predicted heatmap further includes:

[0022] The predicted heatmap and the final image features are spatially aligned using a segmented coding network, and the aligned predicted heatmap and the final image features are then fused to obtain the prompted features.

[0023] The features following the prompt are input into the mask decoder in the frozen SAM base model, and the defect segmentation result of the defect image to be tested is output.

[0024] In one implementation, the step of supervised training based on the average error loss between the predicted heatmap and the actual heatmap label to obtain the trained industrial defect alerting model includes:

[0025] Calculate the average error loss between the predicted heatmap and the actual labels of the heatmap, and perform supervised training on the heatmap prediction head based on the average error loss;

[0026] Calculate the cross-entropy loss between the defect segmentation result and the segmentation label, and perform supervised training on the entire segmentation process based on the cross-entropy loss to obtain the trained industrial defect indication model.

[0027] In one implementation, the step of predicting and segmenting the defect region based on the trained industrial defect indication model for the test defect image includes:

[0028] The images in the test set are used as the defect images to be tested;

[0029] The defect image to be tested is input into the trained industrial defect alerting model for prediction, and the defect segmentation result of the defect image to be tested is output.

[0030] Secondly, the present invention provides a defect detection device based on heatmap prompts, comprising:

[0031] The acquisition module is used to acquire image datasets of various types of industrial defects;

[0032] The predicted heatmap module extracts a predicted heatmap based on the image dataset.

[0033] The supervised training module performs supervised training based on the average error loss between the predicted heatmap and the actual labels of the heatmap, and obtains the trained industrial defect indication model.

[0034] The prediction module predicts and segments the defect region from the defect image to be tested based on the trained industrial defect indication model.

[0035] Thirdly, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a defect detection program based on heatmap hints, and the defect detection program based on heatmap hints, when executed by the processor, is used to implement the operation of the defect detection method based on heatmap hints as described in the first aspect.

[0036] Fourthly, the present invention also provides a medium, which is a computer-readable storage medium, storing a heatmap-based defect detection program, which, when executed by a processor, is used to implement the operation of the heatmap-based defect detection method as described in the first aspect.

[0037] The present invention, by employing the above technical solution, has the following effects:

[0038] This invention acquires image datasets of various industrial defect types, extracts predicted heatmaps from these datasets, and conducts supervised training based on the average error loss between the predicted heatmaps and their true labels to obtain a trained industrial defect alerting model. This model is then used to predict and segment defect regions from images to be tested. Furthermore, this invention proposes a novel skeletonized heatmap alerting network based on a fundamental model, which automatically provides visual alerting information to the fundamental model, improving the accuracy of automated industrial defect detection tasks. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0040] Figure 1 This is a flowchart of a defect detection method based on heatmap prompts in one implementation of the present invention.

[0041] Figure 2 This is a framework diagram of a heatmap-based prompting network in one implementation of the present invention.

[0042] Figure 3 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0043] The objectives, features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0045] Exemplary methods

[0046] In recent years, foundational models have achieved significant success. In the field of computer vision, these foundational models are pre-trained on large-scale datasets, with their transformer (a training model based on an attention mechanism) backbone networks exhibiting powerful feature extraction capabilities. These pre-trained foundational models are often fine-tuned and applied to downstream tasks, demonstrating excellent generalization and transfer capabilities. However, these fine-tuned foundational models are not suitable for existing automated industrial defect detection tasks. Clearly, foundational models trained through generalization and transfer learning cannot yet meet the accuracy requirements of current automated industrial defect detection tasks.

[0047] To address the aforementioned technical problems, this invention provides a defect detection method based on heatmap cues. This method acquires image datasets of various industrial defect types, extracts predicted heatmaps from these datasets, and performs supervised training based on the average error loss between the predicted heatmaps and their true labels to obtain a trained industrial defect cues model. This trained model is then used to predict and segment defect regions from the target defect image. This invention proposes a novel skeletonized heatmap cues network based on a base model, which automatically provides visual base model cues, improving the accuracy of automated industrial defect detection tasks.

[0048] like Figure 1 As shown, this embodiment of the invention provides a defect detection method based on heatmap prompts, including the following steps:

[0049] Step S100: Obtain image datasets for various types of industrial defects.

[0050] In this embodiment, the defect detection method based on heatmap cues is a cuing method for industrial defect detection tasks implemented based on a visual basic model. It does not require manual cues to the visual basic model, but only uses information in the image to learn an innovative cues from the basic segmentation model. This method mainly uses skeletonized heatmap information and combines it with the proposed cuing fusion network to effectively and efficiently fuse cues and image feature information, thereby predicting the defect image to be tested in the form of heatmap cues and segmenting the defect region of the industrial product.

[0051] Specifically, in one implementation of this embodiment, acquiring multi-source heterogeneous data includes the following steps:

[0052] Step S101: Obtain image datasets of various industrial defect types; wherein, the image datasets are anomaly datasets with pixel-level segmentation annotations;

[0053] Step S102: Divide the image dataset into a training set and a test set according to a preset ratio;

[0054] Step S103: Preprocess the training set and the test set by adjusting all images to the same size.

[0055] In this embodiment, a frozen Segment Anything Model (SAM, visual large model) is used as the base model, and a set of prompting engineering networks (i.e., feature fusion network, heatmap prediction head, and prompting fusion network) for industrial defects is designed. In this embodiment, the encoder and decoder of the original SAM model are frozen, and a new prompting network is designed to automatically generate prompting information, replacing the prompts that the original SAM model required manual provision, resulting in a parameter-efficient automatic defect segmentation framework.

[0056] In this embodiment, by freezing all layers of the transformer encoder and all layers of the original SAM model, the parameter weights of the original SAM model's encoder and decoder are not updated during training, thereby saving resources during training. Therefore, in this embodiment, only the parameters in the designed cueing engineering network are updated during training, enabling the SAM model to adapt to downstream tasks of industrial defect detection.

[0057] First, select the commonly used public dataset MVtec-AD, which contains 15 types of industrial defects. This dataset contains outlier data with pixel-level segmentation annotations. Then, divide this dataset into training and test sets in an 8:2 ratio.

[0058] Before being fed into the SAM model, the image data in the training and test sets are first preprocessed. The images are processed through a convolutional network with extensions to resize them to a uniform size of [1024, 1024]. Then, the preprocessed images are fed into the encoder of the base model SAM as RGB images, outputting a final image feature X. img The final image feature X img The image features are the original features directly output by the encoder of the base model SAM, and these original features have not undergone input feature fusion.

[0059] like Figure 1 As shown, in one implementation of the present invention, the defect detection method based on heatmap prompts further includes the following steps:

[0060] Step S200: Extract the predicted heatmap based on the image dataset.

[0061] In this embodiment, an industrial defect indication model was designed based on the SAM basic model; such as Figure 2As shown, the industrial defect indication model includes: an encoder, a feature fusion extraction network, a heatmap prediction head, an indication fusion network, and a mask decoder; wherein, the heatmap prediction head and the indication fusion network are uniquely designed networks in this embodiment. The heatmap prediction head can predict the skeletonized heatmap information as described above, while the indication fusion network can effectively and efficiently fuse the indication information and image feature information to obtain the indicated feature X. prompted .

[0062] For the preprocessed dataset, in the forward pass of the industrial defect indication model, the transformer encoder in the frozen SAM base model is used for encoding, with an RGB image as input, to obtain the final image features and intermediate layer image features of the image, denoted as X. img X inter Then, the intermediate layer image features X inter The input is fed into a lightweight convolutional neural network (i.e., a feature fusion and extraction network) to obtain the fused features, denoted as X. fusion Finally, the fused feature X fusion Input the heatmap prediction header to obtain the predicted heatmap X. hm .

[0063] Specifically, in one implementation of this embodiment, step S200 includes the following steps:

[0064] Step S201: Input the images in the training set into the transformer encoder in the frozen SAM base model to obtain intermediate layer image features and final image features;

[0065] Step S202: Input the intermediate layer image features into the feature fusion extraction network to obtain fused features;

[0066] Step S203: Input the fused features into the heatmap prediction head to obtain the predicted heatmap.

[0067] In this embodiment, the encoder consists of a multi-layered transformer block network. Pre-trained with a large amount of data, the encoder possesses strong feature extraction capabilities. Shallow, mid-level, and deep features each carry different semantic information. Shallow features are basic features, capturing some fundamental information and local dependencies of the input image, which is beneficial for low-level feature extraction. Mid-level features capture richer semantic information, handling long-distance dependencies in the input image and providing better performance and robustness. Deep features have even richer semantic information, typically task-related information, providing richer and finer-grained feature representations. A lightweight feature extraction network fuses the features from the intermediate layers of the encoder to obtain the fused feature X.fusion .

[0068] In this embodiment, a simple convolutional neural network is designed to form a heatmap prediction head. This heatmap prediction head module contains two convolutional layers: a first convolutional layer and a second convolutional layer. The first convolutional layer is a lower convolutional layer, and the second convolutional layer is a 1*1 convolutional layer. By fusing features X... fusion By inputting these two convolutional layers, the predicted heatmap X can be obtained. hm The predicted heatmap X hm During training, the heatmap approaches the true skeletonized heatmap, which can be understood as the predicted skeletonized heatmap. The skeletonized heatmap is a heatmap related to the segmented shape and morphology, rather than a conventional center heatmap.

[0069] In response to the characteristics of industrial defect images, this embodiment designs a special prompting information, namely a skeletonized heatmap. The skeletonized heatmap can be obtained through regression using a simple convolutional neural network. Then, by fusing the skeletonized heatmap with image feature information, the automatic prompting process of the industrial defect prompting model can be realized.

[0070] Step S204: The predicted heatmap and the final image features are spatially aligned using a segmented coding network, and the aligned predicted heatmap and the final image features are fused to obtain the features after prompting.

[0071] Step S205: Input the features after the prompt into the mask decoder in the frozen SAM base model, and output the defect segmentation result of the defect image to be tested.

[0072] In this embodiment, for the predicted heatmap X obtained above... hm The predicted heatmap X can be input into the prompt fusion network designed in this embodiment, and through the segmented coding network therein, the predicted heatmap X will be... hm and final image features X img Align spatially. Then, through concave center operations and addition operations, the aligned predicted heatmap X is... hm and final image features X img Fusion, obtaining the prompted feature X prompted .

[0073] During the concave operation, the final image features X img A feature extraction operation aimed at selectively focusing on the final image features X. img The local region is defined. Specifically, the input feature x is multiplied by a soft learnable parameter (smooth), and then passed through a softmax function layer to obtain a mask feature. The mask feature is then multiplied by the input feature x to obtain the output feature.

[0074] The addition operation combines the image features that have undergone the concave operation with the predicted heatmap features. Since spatial alignment has been ensured previously, the addition operation can be performed directly.

[0075] Furthermore, for the feature X after the prompt... prompted In this embodiment, the mask decoder in the frozen SAM basic model is used for decoding; X prompted As input, the final output is the defect segmentation result of the input image.

[0076] like Figure 1 As shown, in one implementation of the present invention, the defect detection method based on heatmap prompts further includes the following steps:

[0077] Step S300: Supervised training is performed based on the average error loss between the predicted heatmap and the actual heatmap label to obtain the trained industrial defect indication model.

[0078] In this embodiment, the defect segmentation results output by the forward process of the initial industrial defect indication model can be used for supervised training by using the loss between these predicted defect segmentation results and the segmentation labels (i.e., the actual defect segmentation results) to obtain the trained industrial defect indication model.

[0079] Specifically, in one implementation of this embodiment, step S300 includes the following steps:

[0080] Step S301: Calculate the average error loss between the predicted heatmap and the actual label of the heatmap, and perform supervised training on the heatmap prediction head based on the average error loss;

[0081] Step S302: Calculate the cross-entropy loss between the defect segmentation result and the segmentation label, and perform supervised training on the entire segmentation process based on the cross-entropy loss to obtain the trained industrial defect indication model.

[0082] In this embodiment, the true label of the skeletonized heatmap is the label obtained after processing the segmentation label of the original image. Specifically, the segmentation label of the original image is transformed by distance to obtain the true label of the skeletonized heatmap. The transformation process is as follows: using L2 distance as the standard, points that are far from the background L2 distance have a high response value in the heatmap, and correspondingly, points that are close to the background have a low response value in the heatmap.

[0083] In this embodiment, a skeletonized heatmap is obtained through regression using a simple convolutional neural network. The mean absolute error between the predicted heatmap and the true heatmap label is minimized during training. This process can be represented as: {X} fusion}→X hmThat is, skeletalized heatmap cues are regressed by fusing features. Then, a parameter-efficient cue fusion network is used to fuse the cue information and the original image information to obtain a cueed image feature. This process can be represented as: {X img ,X hn}→X prompted By inputting the image features after the prompt into the original mask decoder of the frozen SAM, the defect data can be automatically segmented.

[0084] In the above supervised training process, the heatmap prediction head can be supervised by the average error loss between the predicted heatmap and the real heatmap label. The entire segmentation process can be supervised by the cross-entropy loss between the segmentation result and the segmentation label. Specifically, first, the average error loss between the predicted heatmap and the real heatmap label is calculated, and the heatmap prediction head and the feature fusion extraction network are trained under supervision based on this average error loss. Then, the cross-entropy loss between the defect segmentation result and the segmentation label is calculated, and the entire segmentation process is trained under supervision based on this cross-entropy loss, thus obtaining the trained industrial defect indication model.

[0085] like Figure 1 As shown, in one implementation of the present invention, the defect detection method based on heatmap prompts further includes the following steps:

[0086] Step S400: Based on the trained industrial defect indication model, predict the defect image to be tested and segment the defect region.

[0087] In this embodiment, for the trained industrial defect alerting model, by taking the defect image to be tested as input, the alerting information and the original image information can be automatically fused to output the defect segmentation result of the defect image to be tested.

[0088] Specifically, in one implementation of this embodiment, step S400 includes the following steps:

[0089] Step S401: Use the images in the test set as the defect images to be tested;

[0090] Step S402: Input the defect image to be tested into the trained industrial defect prompting model for prediction, and output the defect segmentation result of the defect image to be tested.

[0091] In this embodiment, given a defect image to be tested, the model can automatically segment the defect region. Specifically, the defect image is input into a transformer encoder to obtain the intermediate layer image features. Next, these intermediate layer features are input into a feature fusion extraction network to obtain fused features. Then, the fused features are input into a heatmap prediction head designed in this embodiment to obtain a skeletonized heatmap. Afterward, the skeletonized heatmap is input into a cue fusion network designed in this embodiment, fusing the skeletonized heatmap (i.e., cue information) with the image feature information to obtain cue-enabled features. Finally, the cue-enabled features are input into a mask decoder, and after decoding, the defect segmentation result with cue information is obtained.

[0092] As an example, in the actual application scenario of this embodiment, the task of the defect detection method based on heatmap prompts in this embodiment is to obtain accurate defect segmentation results.

[0093] (1) Compared with the original SAM model, the defect detection method based on heat map prompts in this embodiment no longer requires manual prompts, but achieves automatic defect segmentation.

[0094] (2) Compared with the original SAM model, the defect detection method based on heat map prompts in this embodiment designs a new prompt that combines industrial domain knowledge, enriches the prompt information, and makes the model more suitable for defect segmentation tasks.

[0095] (3) Compared with other segmentation models, the defect detection method based on heatmap prompts in this embodiment ensures both segmentation performance and parameter efficiency.

[0096] This embodiment achieves the following technical effects through the above technical solution:

[0097] This embodiment acquires image datasets of various industrial defect types, extracts predicted heatmaps from these datasets, and performs supervised training based on the average error loss between the predicted heatmaps and their true labels. This results in a trained industrial defect alerting model, which is then used to predict and segment defect regions from the target defect image. This embodiment proposes a novel skeletonized heatmap alerting network based on a base model, which automatically provides visual base model alerts, improving the accuracy of automated industrial defect detection tasks.

[0098] Exemplary device

[0099] Based on the above embodiments, the present invention also provides a defect detection device based on heatmap prompts, comprising:

[0100] The acquisition module is used to acquire image datasets of various types of industrial defects;

[0101] The predicted heatmap module extracts a predicted heatmap based on the image dataset.

[0102] The supervised training module performs supervised training based on the average error loss between the predicted heatmap and the actual labels of the heatmap, and obtains the trained industrial defect indication model.

[0103] The prediction module predicts and segments the defect region from the defect image to be tested based on the trained industrial defect indication model.

[0104] This embodiment achieves the following technical effects through the above technical solution:

[0105] This embodiment acquires image datasets of various industrial defect types, extracts predicted heatmaps from these datasets, and performs supervised training based on the average error loss between the predicted heatmaps and their true labels. This results in a trained industrial defect alerting model, which is then used to predict and segment defect regions from the target defect image. This embodiment proposes a novel skeletonized heatmap alerting network based on a base model, which automatically provides visual base model alerts, improving the accuracy of automated industrial defect detection tasks.

[0106] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 3 As shown.

[0107] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a storage medium and internal memory; the storage medium stores the operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external devices; the display screen is used to display relevant information; and the communication module is used to communicate with a cloud server or other devices.

[0108] When executed by the processor, this computer program is used to implement a defect detection method based on heatmap hints.

[0109] It will be understood by those skilled in the art that Figure 3 The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0110] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a heatmap-based defect detection program, which, when executed by the processor, is used to implement the operation of the heatmap-based defect detection method described above.

[0111] In one embodiment, a storage medium is provided, wherein the storage medium stores a heatmap-based defect detection program, which, when executed by a processor, is used to implement the operation of the heatmap-based defect detection method described above.

[0112] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, database, or other media used in the embodiments provided by this invention can include both non-volatile and volatile memory.

[0113] In summary, this invention provides a defect detection method and apparatus based on heatmap cues. The method includes: acquiring image datasets of various industrial defect types; extracting predicted heatmaps from the image datasets; performing supervised training based on the average error loss between the predicted heatmaps and the true labels of the heatmaps to obtain a trained industrial defect cues model; and predicting and segmenting defect regions based on the trained industrial defect cues model for a test defect image. This invention proposes a novel skeletonized heatmap cues network based on a base model, which automatically provides visual base model cues, improving the accuracy of automated industrial defect detection tasks.

[0114] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A defect detection method based on heatmap prompts, characterized in that, include: Acquire image datasets of various types of industrial defects; The process of extracting a predicted heatmap from the image dataset includes: inputting images from the training set partitioned from the image dataset into the transformer encoder of the frozen SAM base model to obtain intermediate layer image features and final image features; inputting the intermediate layer image features into a feature fusion extraction network to obtain fused features; inputting the fused features into a heatmap prediction head to obtain the predicted heatmap; aligning the predicted heatmap and the final image features spatially through a segmented encoding network, and fusing the aligned predicted heatmap and the final image features to obtain prompted features; inputting the prompted features into a mask decoder in the frozen SAM base model to output the defect segmentation result of the defect image to be tested; wherein, the predicted heatmap is a predicted skeletonized heatmap, and the skeletonized heatmap is a heatmap related to the segmentation shape and morphology; Supervised training is performed based on the average error loss between the predicted heatmap and the actual heatmap label to obtain the trained industrial defect alert model. Based on the trained industrial defect indication model, the defect image to be tested is predicted and the defect region is segmented.

2. The defect detection method based on heatmap prompts according to claim 1, characterized in that, The acquisition of image datasets for various types of industrial defects includes: Obtain image datasets of various industrial defect types; wherein, the image datasets are anomaly datasets with pixel-level segmentation annotations; The image dataset is divided into a training set and a test set according to a preset ratio; The training set and the test set are preprocessed to resize all images to the same size.

3. The defect detection method based on heatmap prompts according to claim 1, characterized in that, The heatmap prediction head includes: a first convolutional layer and a second convolutional layer; the first convolutional layer is a lower convolutional layer, and the second convolutional layer is a lower convolutional layer. 1 convolutional layer.

4. The defect detection method based on heatmap prompts according to claim 1, characterized in that, The process of supervised training based on the average error loss between the predicted heatmap and the actual heatmap labels to obtain the trained industrial defect alerting model includes: Calculate the average error loss between the predicted heatmap and the actual labels of the heatmap, and perform supervised training on the heatmap prediction head based on the average error loss; Calculate the cross-entropy loss between the defect segmentation result and the segmentation label, and perform supervised training on the entire segmentation process based on the cross-entropy loss to obtain the trained industrial defect indication model.

5. The defect detection method based on heatmap prompts according to claim 2, characterized in that, The process of predicting and segmenting the defect region based on the trained industrial defect indication model includes: The images in the test set are used as the defect images to be tested; The defect image to be tested is input into the trained industrial defect alerting model for prediction, and the defect segmentation result of the defect image to be tested is output.

6. A defect detection device based on heatmap prompts, used to implement the defect detection method based on heatmap prompts as described in any one of claims 1-5, characterized in that, include: The acquisition module is used to acquire image datasets of various types of industrial defects; The predicted heatmap module extracts a predicted heatmap based on the image dataset. The supervised training module performs supervised training based on the average error loss between the predicted heatmap and the actual labels of the heatmap, and obtains the trained industrial defect indication model. The prediction module predicts and segments the defect region from the defect image to be tested based on the trained industrial defect indication model.

7. A terminal, characterized in that, include: The processor and memory, the memory storing a heatmap-based defect detection program, which, when executed by the processor, is used to implement the operation of the heatmap-based defect detection method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a heatmap-based defect detection program, which, when executed by a processor, is used to implement the operation of the heatmap-based defect detection method as described in any one of claims 1-5.

Images