Defect detection model construction method, defect detection method, and related device
By constructing a teacher-student network model and introducing an autoencoder branch, combined with a defect segmentation model, the problems of fuzzy positioning and low efficiency in defect detection are solved, and high-precision and efficient defect detection is achieved.
Patent Information
- Application Number
- CN202411318189.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2025-10-17
- Estimated Expiration
- 2044-09-20
AI Technical Summary
Existing deep learning-based defect detection methods have problems of ambiguity and incompleteness in defect localization, which affects detection accuracy and efficiency. In addition, supervised learning methods rely on a large number of labeled samples, which are difficult to obtain.
A teacher-student network model is constructed, and an autoencoder branch is introduced. The defect location information and mask prompt information are generated by simulating abnormal training images. The defect segmentation model is combined for refinement and segmentation, and the trained defect recognition and segmentation models are integrated.
It effectively reduces the missed detection rate and passed detection rate in defect detection, improves detection accuracy, and takes into account detection efficiency.
Smart Images
Figure CN119295854B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of defect detection and relates to a defect detection model construction method, a defect detection method and related devices. Background Art
[0002] As a core link in industrial quality control, defect detection is of self-evident importance. With the rapid development of intelligent manufacturing and the Industrial Internet, various industries have placed higher demands on the accuracy, speed, and intelligence level of defect detection. In the 3C (computer, communications, and consumer electronics) field, the detection of tiny and complex component defects is directly related to product reliability and user experience; in mechanical manufacturing, high-precision parts inspection is key to ensuring machine performance and lifespan; the automotive industry faces multiple inspection challenges in body, coating, and parts; the appearance and functional defect inspection of home appliances is related to consumer satisfaction; and in the aerospace field, any minor defect may cause a serious safety accident, so its inspection standards are more stringent.
[0003] Traditional manual inspection methods are limited by subjective judgment, fatigue, and low efficiency, and are no longer able to meet the needs of modern industrial production. Automated inspection methods based on deep learning have emerged. By simulating the learning process of the human brain's neural network, they enable intelligent analysis of multi-dimensional information such as images and sounds, significantly improving the accuracy and efficiency of defect detection. However, supervised learning methods are highly dependent on a large number of well-labeled defect samples. Acquiring these samples often requires a significant amount of manpower, material resources, and time. Furthermore, for some rare or new defect types, it may be impossible to obtain sufficient samples for training.
[0004] To overcome the limitations of supervised learning, unsupervised defect detection technology has gradually become a research hotspot. Among them, the teacher-student network model based on knowledge distillation has shown great potential. This model constructs two networks with similar structures but different capacity or performance: the teacher branch and the student branch. The model leverages these differences in learning to capture anomalies and thus detect defects. The teacher branch is typically a pre-trained, high-performance model responsible for extracting deep features from the input data; the student branch is a simplified or incompletely trained model with relatively weaker learning capabilities. When both branches process the same data simultaneously, the student branch cannot fully replicate the output of the teacher branch. This discrepancy may be caused by defects in the input data. However, this approach often suffers from ambiguity and incompleteness in defect localization, affecting the ultimate detection accuracy and efficiency. Summary of the Invention
[0005] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a defect detection model construction method, a defect detection method and related devices.
[0006] To achieve the above object, the present application adopts the following technical solutions to achieve the above object:
[0007] In a first aspect, the present application provides a defect detection model construction method, comprising:
[0008] Obtain an initial training image, and add simulated anomalies to the initial training image to obtain a simulated anomaly training image;
[0009] Train a preset defect recognition model according to the initial training image and the simulated anomaly training image to obtain defect position information and mask prompt information;
[0010] Train a preset defect segmentation model according to the defect position information and the mask prompt information;
[0011] Fuse the trained defect recognition model and the defect segmentation model to obtain a defect detection model;
[0012] The defect recognition model comprises a teacher network branch, a student network branch, and a self-encoder branch; the teacher network branch adopts fixed network weights, and the student network branch and the self-encoder branch adopt trainable network weights; the output difference between the teacher network branch and the student network branch is the defect position information, and the output difference between the student network branch and the self-encoder branch is the mask prompt information.
[0013] Optionally, the teacher network branch and the student network branch adopt the same network structure and are both encoder structures; the network structure of the self-encoder branch is an encoder-decoder structure.
[0014] Optionally, adding simulated anomalies to the initial training image to obtain the simulated anomaly training image comprises: adding simulated anomalies to the initial training image by using an anomaly generator according to the following formula:
[0015]
[0016] wherein, I a is the simulated anomaly training image, I is the initial training image, M is a simulated anomaly mask, is the inverse of M, is a pixel-by-pixel multiplication, is an opacity parameter in mixing, and A is an additional texture image.
[0017] Optionally, when training the preset defect recognition model according to the initial training image and the simulated anomaly training image, the training loss function of the teacher network branch-student network branch is:
[0018]
[0019]
[0020] wherein, L ST denotes the loss function between the student network branch and the teacher network branch; L hard denotes the hard sample loss function; H, W and C represent the height, width and channel number of the output feature map respectively; S(P) c denotes the output of the student network branch; c represents the channel number; L STAE denotes the loss function between the autoencoder network branch and the student network branch; S'(I) c denotes the additional output of the student network branch; A(I) c denotes the output of the autoencoder network branch;
[0021] The training loss function of the teacher network branch-autoencoder branch is:
[0022]
[0023] wherein, L AE denotes the loss function between the autoencoder network branch and the teacher network branch; T(I) c denotes the output of the teacher network branch.
[0024] Optionally, the defect segmentation model is obtained by introducing a trainable branch on the basis of a pre-trained SAM model and modifying the self-attention mechanism in the Transformer module of the pre-trained SAM model; and when training the pre-set defect segmentation model according to the defect position information and the mask prompt information, the parameters of the pre-trained SAM model are fixed;
[0025] wherein, the generation process of the query Q, the key K and the value V in the modified self-attention mechanism is:
[0026] Q = X·W Q1 ·W Q2 + X·W Q
[0027] K = X·W K
[0028] V = X·W V1 ·W V2 + X·W V
[0029] wherein, X represents an input feature; W Q1 denotes the first part of the newly added query matrix based on the trainable branch; W Q2 denotes the second part of the newly added query matrix based on the trainable branch; W Q denotes the original query matrix of the SAM model; W K denotes the original key matrix of the SAM model; W V1denotes a first part of the newly added value matrix based on the trainable branch; W V2 denotes a second part of the newly added value matrix based on the trainable branch; W V denotes the original value matrix of the SAM model.
[0030] In a second aspect, the present application provides a defect detection model construction system, comprising:
[0031] An image processing module is configured to obtain initial training images and add simulated abnormality to the initial training images to obtain simulated abnormality training images.
[0032] A recognition model training module is configured to train a preset defect recognition model based on the initial training images and the simulated abnormality training images to obtain defect position information and mask prompt information.
[0033] A segmentation model training module is configured to train a preset defect segmentation model based on the defect position information and the mask prompt information.
[0034] A model fusion module is configured to fuse the trained defect recognition model and the defect segmentation model to obtain a defect detection model.
[0035] The defect recognition model comprises a teacher network branch, a student network branch and an auto-encoder branch. The teacher network branch adopts fixed network weights, and the student network branch and the auto-encoder branch adopt trainable network weights. The output difference between the teacher network branch and the student network branch is the defect position information, and the output difference between the student network branch and the auto-encoder branch is the mask prompt information.
[0036] In a third aspect, the present application provides a defect detection method, comprising:
[0037] Obtaining a sample image to be detected;
[0038] Inputting the sample image to be detected into a preset defect detection model to obtain a defect detection result of the sample to be detected; wherein the preset defect detection model is constructed by using the defect detection model construction method described above.
[0039] In a fourth aspect, the present application provides a defect detection system, comprising:
[0040] A sample image acquisition module is configured to obtain a sample image to be detected;
[0041] A defect detection module is configured to input the sample image to be detected into a preset defect detection model to obtain a defect detection result of the sample to be detected; wherein the preset defect detection model is constructed by using the defect detection model construction method described above.
[0042] In a fifth aspect, the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the defect detection model construction method or the steps of the defect detection method when executing the computer program.
[0043] In a sixth aspect, the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program implements the steps of the defect detection model construction method or the steps of the defect detection method when executed by a processor.
[0044] Compared with the prior art, the present application has the following beneficial effects:
[0045] The defect detection model construction method of the present application adds simulated abnormality to the initial training image to obtain a simulated abnormality training image, then trains a preset defect recognition model according to the initial training image and the simulated abnormality training image to obtain defect position information and mask prompt information, and further trains a preset defect segmentation model according to the defect position information and the mask prompt information, and finally fuses the trained defect recognition model and the defect segmentation model to obtain a defect detection model. The defect recognition model introduces an auto-encoder branch to provide mask prompt information, and finally uses a defect segmentation model to refine and segment the defect position information, so that when the defect detection model constructed by the construction method is used for product defect detection, the missed detection rate and the over-detection rate in the product defect detection process can be effectively reduced, and the defect detection accuracy is improved while the detection efficiency is taken into account. BRIEF DESCRIPTION OF DRAWINGS
[0046] Figure 1 The defect detection model construction method of the present application is a flowchart.
[0047] Figure 2 The defect detection model principle diagram of the present application is a flowchart.
[0048] Figure 3 The defect detection method of the present application is a flowchart.
[0049] Figure 4 The defect detection model construction system structure block diagram of the present application is a flowchart.
[0050] Figure 5 The defect detection system structure block diagram of the present application is a flowchart. DETAILED DESCRIPTION
[0051] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative work should fall within the protection scope of the present application.
[0052] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device including a series of steps or units does not necessarily have to be limited to the clearly listed steps or units, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.
[0053] The present application will be described in further detail below with reference to the drawings:
[0054] With the development of modern machine learning and artificial intelligence technology, more and more related technologies are applied to industrial scenarios. Due to the system or human factors in the industrial production process, scratches and cracks and other defects are inevitably generated on the surface of the product. The speed and accuracy of traditional manual defect detection have seriously limited the production efficiency of the product, and cannot meet the detection requirements of the industrial scenario.
[0055] Although the defect detection method based on deep neural network can be used to perform end-to-end feature extraction on the defect image at present, most methods cannot balance the accuracy and efficiency of detection.
[0056] Referring to Figure 1 and 2 In an embodiment of the present application, a defect detection model construction method is provided, which improves the detection accuracy while balancing the detection efficiency by constructing an innovative defect detection model. Specifically, the defect detection model construction method of the present application comprises the following steps:
[0057] S1: obtaining an initial training image, and adding the initial training image to a simulated abnormality to obtain a simulated abnormality training image.
[0058] S2: training a preset defect recognition model according to the initial training image and the simulated abnormality training image to obtain defect position information and mask prompt information.
[0059] S3: training the preset defect segmentation model according to the defect position information and the mask prompt information.
[0060] S4: fusing the trained defect recognition model and the defect segmentation model to obtain the defect detection model.
[0061] The defect recognition model comprises a teacher network branch, a student network branch and a self-encoder branch; the teacher network branch adopts fixed network weights, and the student network branch and the self-encoder branch adopt trainable network weights; the output difference value of the teacher network branch and the student network branch is the defect position information, and the output difference value of the student network branch and the self-encoder branch is the mask prompt information.
[0062] The defect detection model construction method comprises the following steps: adding simulation abnormity to an initial training image to obtain a simulation abnormity training image; training a preset defect recognition model according to the initial training image and the simulation abnormity training image to obtain defect position information and mask prompt information; training a preset defect segmentation model according to the defect position information and the mask prompt information; and fusing the trained defect recognition model and the defect segmentation model to obtain a defect detection model. The defect recognition model introduces a self-encoder branch to provide mask prompt information, and finally adopts a defect segmentation model to refine and segment the defect position information, so that the defect detection model constructed by the construction method can effectively reduce the omission rate and over-detection rate in the product defect detection process, improve the defect detection accuracy while taking into account the detection efficiency.
[0063] In a possible implementation, the teacher network branch and the student network branch adopt the same network structure, and are both encoder structures; and the network structure of the self-encoder branch is an encoder-decoder structure.
[0064] Specifically, a lightweight pre-training network is used as the teacher network branch, the network structure of the student network branch is consistent with that of the teacher network branch, but the parameters are not pre-trained, the self-encoder branch adopts a convolutional self-encoder structure, and the output structures of the three network branches are consistent.
[0065] The introduction of the self-encoder branch provides mask prompt information to overcome the defects of traditional teacher-student network based on knowledge distillation, such as fuzzy and incomplete defect positioning, so as to improve the detection accuracy.
[0066] In a possible implementation, the adding simulation abnormity to the initial training image to obtain the simulation abnormity training image comprises: adding simulation abnormity to the initial training image by using an anomaly generator according to the following formula:
[0067]
[0068] wherein, I a is the initial training image, M is the simulated abnormality mask, is the inverse of M, is the pixel-wise multiplication, is the opacity parameter in the blending, and A is the additional texture image.
[0069] The initial training image is added with artificial simulated abnormality by using the simulated abnormality generator to obtain the initial training image with added simulated abnormality, i.e., the simulated abnormality training image, so as to overcome the problem of difficulty in obtaining a large number of training samples.
[0070] In a possible implementation, when the preset defect recognition model is trained according to the initial training image and the simulated abnormality training image, the training loss function of the teacher network branch-student network branch is:
[0071]
[0072]
[0073] wherein, L ST represents the loss function between the student network branch and the teacher network branch; L hard represents the difficult sample loss function; H, W and C represent the height, width and channel number of the output feature map respectively; S(P) c represents the output of the student network branch; c represents the channel number; L STAE represents the loss function between the auto-encoder network branch and the student network branch; S'(I) c represents the additional output of the student network branch; A(I) c represents the output of the auto-encoder network branch.
[0074] The training loss function of the teacher network branch-auto-encoder branch is:
[0075]
[0076] wherein, L AE represents the loss function between the auto-encoder network branch and the teacher network branch; T(I) c represents the output of the teacher network branch.
[0077] Specifically, in training the preset defect recognition model according to the initial training image and the simulated abnormal training image, the simulated abnormal training image is input into the student network branch, the initial training image is input into the teacher network branch and the autoencoder branch, in the training process, the parameter weight of the teacher network branch remains unchanged, the student network branch is trained to predict the output of the teacher network branch and the autoencoder branch, the autoencoder branch is trained to predict the output of the teacher network branch, the output difference of the teacher network branch and the student network branch is taken as the defect position information, and the output difference of the student network branch and the autoencoder branch is taken as the mask prompt information. The defect position information and the mask prompt information are input into the initial defect segmentation model for training, and the ideal output of each sample in the training process is the known simulated abnormal binary mask.
[0078] In a possible implementation, the defect segmentation model is obtained by introducing a trainable branch on the basis of a pre-trained SAM model and modifying the self-attention mechanism in the Transformer module of the pre-trained SAM model; and when training the preset defect segmentation model according to the defect position information and the mask prompt information, the parameters of the pre-trained SAM model are fixed.
[0079] In the modified self-attention mechanism, the generation processes of the query Q, the key K and the value V are as follows:
[0080] Q = X · W Q1 · W Q2 + X · W Q
[0081] K = X · W K
[0082] V = X · W V1 · W V2 + X · W V
[0083] wherein X represents an input feature; W Q1 represents a first part of a newly added query matrix based on the trainable branch; W Q2 represents a second part of the newly added query matrix based on the trainable branch; W Q represents an original query matrix of the SAM model; W K represents an original key matrix of the SAM model; W V1 represents a first part of a newly added value matrix based on the trainable branch; W V2 represents a second part of the newly added value matrix based on the trainable branch; W V represents an original value matrix of the SAM model.
[0084] Specifically, the defect segmentation model selects a visual large model SAM as the basis, introduces a branch for training while retaining the original SAM model architecture, and provides relevant additional parameters for defect detection field knowledge. By modifying the self-attention mechanism in the Transformer module of the SAM model, the integration of general knowledge and defect detection specific knowledge is realized. In the original SAM model, the generation process of the query Q, key K and value V in the self-attention mechanism can be represented by the following formula:
[0085] Q = X·W Q
[0086] K = X·W K
[0087] V = X·W V
[0088] Where W Q , W K and W V are weight matrices that are multiplied by the input feature X to generate the corresponding query Q, key K and value V, which uses the properties of matrix multiplication to map the input feature to the space of query Q, key K and value V.
[0089] In this embodiment, low-rank adaptive technology is used to modify the generation of query Q, key K and value V in the SAM model. The generation of query Q, key K and value V can be represented as the combination of the original weight matrix and the additional linear layer W Q1 , W Q2 , W V1 and W V2 obtained by the trainable branch:
[0090] Q = X·W Q1 ·W Q2 + X·W Q
[0091] K = X·W K
[0092] V = X·W V1 ·W V2 + X·W V
[0093] In the above formula, X represents the input feature. The outputs of W Q1 and W Q2 actually form a new weight matrix, which represents the newly learned knowledge for the defect detection task. In this case, W Q1 and W Q2 can be regarded as components of the weight matrix, which are trained to capture specific features related to defect detection. In this setting, the original weight matrix W QThe general knowledge learned from the pre-trained SAM model is retained, while W Q1 ·W Q2 provides task-specific knowledge for defect detection. Finally, the trained defect recognition model and the defect segmentation model are fused to obtain a complete defect detection model.
[0094] Referring to Figure 3 In another embodiment of the present application, a defect detection method is provided, comprising the following steps:
[0095] S11: obtaining a sample image to be detected.
[0096] S12: inputting the sample image to be detected into a preset defect detection model to obtain a defect detection result of the sample to be detected; wherein the preset defect detection model is constructed by the defect detection model construction method described above.
[0097] Specifically, when performing defect detection, the sample image to be detected is input into the defect recognition model, and is input into the teacher network branch, the student network branch and the autoencoder branch respectively, and then the outputs of the teacher network branch, the student network branch and the autoencoder branch are obtained respectively. The difference between the outputs of the teacher network branch and the student network branch is used as defect position information, and the difference between the outputs of the student network branch and the autoencoder branch is used as mask prompt information. The defect position information and the mask prompt information are input into the defect segmentation model, and then it is judged whether there is a defect in the sample image to be detected, and the corresponding defect segmentation result, such as a defect mask image, is output.
[0098] The design idea of the network structure of the defect detection model of the present application is very novel, and provides a corresponding solution for the actual problem of industrial defect detection. In actual scenarios, it can be applied to object surface defect detection in most industrial fields, especially in cases where accuracy and real-time performance are required. The main points of the main steps of the defect detection method are: 1. Construct a data set for training, such as normal mobile phone screens and cloth surface images; 2. Integrate the defect detection model of the present application into a specific deep learning framework, such as TensorFlow, Caffe and PyTorch; 3. Train the defect detection model, and train the defect recognition model and the defect segmentation model according to a certain object category; 4. Test the defect detection model, construct a data set for testing, convert it into an input image of a specified size, input it into the defect detection model, and obtain the predicted defect position information. Compared with other methods, this method has higher accuracy and real-time performance for different object categories, and basically meets the requirements of real industrial scenarios.
[0099] The following is an apparatus embodiment of the present application, which can be used to execute the method embodiment of the present application. For details not disclosed in the apparatus embodiment, please refer to the method embodiment of the present application.
[0100] Referring to Figure 4 In another embodiment of the present application, a defect detection model construction system is provided, which can be used to implement the defect detection model construction method described above. Specifically, the defect detection model construction system comprises an image processing module, an identification model training module, a segmentation model training module, and a model fusion module.
[0101] The image processing module is configured to obtain initial training images and add simulated anomalies to the initial training images to obtain simulated anomaly training images. The identification model training module is configured to train a preset defect identification model based on the initial training images and the simulated anomaly training images to obtain defect position information and mask prompt information. The segmentation model training module is configured to train a preset defect segmentation model based on the defect position information and the mask prompt information. The model fusion module is configured to fuse the trained defect identification model and the defect segmentation model to obtain a defect detection model. The defect identification model comprises a teacher network branch, a student network branch, and an auto-encoder branch. The teacher network branch adopts fixed network weights, and the student network branch and the auto-encoder branch adopt trainable network weights. The output difference between the teacher network branch and the student network branch is the defect position information, and the output difference between the student network branch and the auto-encoder branch is the mask prompt information.
[0102] All related contents of the steps involved in the foregoing embodiments of the defect detection model construction method can be cited to the function description of the function modules corresponding to the defect detection model construction system in the embodiments of the present application, and will not be repeated here.
[0103] Referring to Figure 5 In another embodiment of the present application, a defect detection system is provided, which can be used to implement the defect detection method described above. Specifically, the defect detection system comprises a sample image acquisition module and a defect detection module. The sample image acquisition module is configured to obtain a sample image to be detected. The defect detection module is configured to input the sample image to be detected into a preset defect detection model to obtain a defect detection result of the sample to be detected. The preset defect detection model is constructed by using the defect detection model construction method described above.
[0104] All related contents of the steps involved in the foregoing embodiments of the defect detection method can be cited to the function description of the function modules corresponding to the defect detection system in the embodiments of the present application, and will not be repeated here.
[0105] The division of the modules in the embodiments of the present application is illustrative, and is merely logical function division. In actual implementation, another division manner can be used. In addition, each function module in each embodiment of the present application can be integrated in one processor, or can be physically separated, or two or more modules can be integrated in one module. The integrated module can be implemented in the form of hardware or in the form of a software function module.
[0106] In still another embodiment of the present application, a computer device is provided, which includes a processor and a memory. The memory is configured to store a computer program, and the computer program includes program instructions. The processor is configured to execute the program instructions stored in the computer storage medium. The processor can be a central processing unit (CPU), and can also be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or another programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and the like. The processor is a computing core and a control core of the terminal, and is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions in the computer storage medium to implement a corresponding method flow or a corresponding function. The processor in the embodiments of the present application can be used for the operation of the defect detection model construction method and the defect detection method.
[0107] In still another embodiment of the present application, the present application also provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in a computer device, used for storing programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the computer device, and of course can also include the expansion storage medium supported by the computer device. The computer readable storage medium provides a storage space, which stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory. The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to implement the corresponding steps of the defect detection model construction method and the defect detection method in the above embodiments.
[0108] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.
[0109] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the flow Figure 1 The function specified in one or more flows and / or blocks Figure 1 The means for performing the function specified in one or more flows and / or blocks.
[0110] These computer program instructions can also be stored in a computer readable storage medium capable of directing the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable storage medium produce a product including instruction means, which implements the flow Figure 1 The function specified in one or more flows and / or blocksFigure 1 the function specified in the one or more blocks.
[0111] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide processes for implementing the flows Figure 1 the flows or the flows and / or blocks Figure 1 the steps of the function specified in the one or more blocks.
[0112] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit it, although the above embodiments of the present application have been described in detail, those skilled in the art should understand: the specific embodiments of the present application can be modified or replaced by the same, without departing from the spirit and scope of the present application, any modification or equivalent replacement, which should be covered in the protection scope of the claims of the present application.
Claims
1. A defect detection model construction method, characterized in that: include: Acquire an initial training image, and add a simulated anomaly to the initial training image to obtain a simulated anomaly training image; The preset defect recognition model is trained based on the initial training image and the simulated abnormal training image to obtain the defect location information and mask prompt information; Train the preset defect segmentation model based on the defect location information and mask prompt information; The defect detection model is obtained by integrating the trained defect recognition model and defect segmentation model; The defect recognition model includes a teacher network branch, a student network branch, and an autoencoder branch. The teacher network branch uses fixed network weights, while the student network branch and the autoencoder branch use trainable network weights. The output difference between the teacher and student network branches is the defect location information, and the output difference between the student network branch and the autoencoder branch is the mask prompt information. The defect segmentation model is obtained by introducing a trainable branch on the basis of a pre-trained SAM model and modifying the self-attention mechanism in the Transformer module of the pre-trained SAM model; and when the preset defect segmentation model is trained according to the defect location information and the mask prompt information, the parameters of the pre-trained SAM model are fixed; Among them, the query in the modified self-attention mechanism Q ,key K Sum V The generation process is: in, Represents input features; represents the first part of the newly added query matrix based on the trainable branch; represents the second part of the newly added query matrix based on the trainable branch; Represents the original query matrix of the SAM model; represents the original bond matrix of the SAM model; Represents the first part of the newly added value matrix based on the trainable branch; The second part of the matrix representing the newly added value based on the trainable branch; Represents the raw value matrix of the SAM model.
2. The defect detection model construction method according to claim 1, characterized in that: The teacher network branch and the student network branch adopt the same network structure, and both are encoder structures; the network structure of the autoencoder branch is an encoder-decoder structure.
3. The defect detection model construction method according to claim 1, characterized in that: The step of adding a simulated anomaly to the initial training image to obtain a simulated anomaly training image includes: adding a simulated anomaly to the initial training image using an anomaly generator according to the following formula to obtain a simulated anomaly training image: in, To simulate abnormal training images, is the initial training image, To simulate anomaly masks, for The inverse, is pixel-by-pixel multiplication, is the opacity parameter in the blend, To attach a texture image.
4. The defect detection model construction method according to claim 1, characterized in that: When the preset defect recognition model is trained based on the initial training image and the simulated abnormal training image, the training loss function of the teacher network branch-student network branch is: in, represents the loss function between the student network branch and the teacher network branch; represents the difficult sample loss function; 、 and Represent the height, width and number of channels of the output feature map respectively; represents the output of the student network branch; c Indicates the number of channels; Represents the loss function between the autoencoder network branch and the student network branch; Represents the additional output of the student network branch; Represents the output of the autoencoder network branch; The training loss function of the teacher network branch-autoencoder branch is: in, represents the loss function between the autoencoder network branch and the teacher network branch; Represents the output of the teacher network branch.
5. A defect detection model construction system, characterized in that: include: An image processing module is used to obtain an initial training image and add a simulated anomaly to the initial training image to obtain a simulated anomaly training image; The recognition model training module is used to train the preset defect recognition model based on the initial training image and the simulated abnormal training image to obtain the defect location information and mask prompt information; Segmentation model training module, used to train a preset defect segmentation model based on defect location information and mask prompt information; Model fusion module, used to fuse the trained defect recognition model and defect segmentation model to obtain a defect detection model; The defect recognition model includes a teacher network branch, a student network branch, and an autoencoder branch. The teacher network branch uses fixed network weights, while the student network branch and the autoencoder branch use trainable network weights. The output difference between the teacher and student network branches is the defect location information, and the output difference between the student network branch and the autoencoder branch is the mask prompt information. The defect segmentation model is obtained by introducing a trainable branch on the basis of a pre-trained SAM model and modifying the self-attention mechanism in the Transformer module of the pre-trained SAM model; and when the preset defect segmentation model is trained according to the defect location information and the mask prompt information, the parameters of the pre-trained SAM model are fixed; Among them, the query in the modified self-attention mechanism Q ,key K Sum V The generation process is: in, Represents input features; represents the first part of the newly added query matrix based on the trainable branch; represents the second part of the newly added query matrix based on the trainable branch; Represents the original query matrix of the SAM model; represents the original bond matrix of the SAM model; Represents the first part of the newly added value matrix based on the trainable branch; The second part of the matrix representing the newly added value based on the trainable branch; Represents the raw value matrix of the SAM model.
6. A defect detection method, characterized in that: include: Obtaining a sample image to be detected; The image of the sample to be detected is input into a preset defect detection model to obtain the defect detection result of the sample to be detected; wherein the preset defect detection model is constructed using the defect detection model construction method described in any one of claims 1 to 4.
7. A defect detection system, characterized in that: include: A sample image acquisition module is used to acquire a sample image to be detected; A defect detection module is used to input the image of the sample to be detected into a preset defect detection model to obtain the defect detection result of the sample to be detected; wherein the preset defect detection model is constructed using the defect detection model construction method described in any one of claims 1 to 4.
8. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the defect detection model construction method according to any one of claims 1 to 4 are implemented, or the steps of the defect detection method according to claim 6 are implemented.
9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the defect detection model construction method according to any one of claims 1 to 4 are implemented, or the steps of the defect detection method according to claim 6 are implemented.