Defect detection model construction method, defect detection method and related device

Abstract

The invention belongs to the field of defect detection, and discloses a defect detection model construction method, a defect detection method and a related device, and the method comprises the steps: obtaining an initial training image, and adding a simulation anomaly to the initial training image to obtain a simulation anomaly training image; training a preset defect identification model according to the initial training image and the simulated abnormal training image to obtain defect position information and mask prompt information; training a preset defect segmentation model according to the defect position information and the mask prompt information; fusing the trained defect identification model and the defect segmentation model to obtain a defect detection model; the defect identification model comprises a teacher network branch, a student network branch and an auto-encoder branch; the output difference between the teacher network branch and the student network branch is defect position information, and the output difference between the student network branch and the auto-encoder branch is mask prompt information. The detection omission rate and the over-detection rate in the detection process can be effectively reduced, and the detection accuracy is improved while the detection efficiency is considered.

Description

Technical Field

[0001] The present invention belongs to the field of defect detection, and relates to a defect detection model construction method, a defect detection method and related devices. Background Art

[0002] As the core link of industrial quality control, the importance of defect detection is self-evident. With the rapid development of intelligent manufacturing and industrial Internet, various industries have put forward higher requirements for the accuracy, speed and intelligence level of defect detection. In the field of 3C (computers, communications and consumer electronics), the detection of tiny and complex component defects is directly related to the reliability of the product and user experience; in mechanical manufacturing, high-precision parts detection is the key to ensuring the performance and life of the machine; the automotive industry faces many challenges in the detection of body, painting and parts; the appearance and functional defect detection of home appliances is related to consumer satisfaction; and in the field of aerospace, any slight defect may cause serious safety accidents, so its detection standards are more stringent.

[0003] Traditional manual inspection methods are limited by subjective judgment, fatigue effects, and low efficiency, and can no longer meet the needs of modern industrial production. Automated inspection methods based on deep learning have emerged. By simulating the learning process of the human brain neural network, intelligent analysis of multi-dimensional information such as images and sounds has been achieved, significantly improving the accuracy and efficiency of defect detection. However, supervised learning methods are highly dependent on a large number of labeled defect samples. The acquisition of these samples often requires a lot of manpower, material resources, and time. In addition, for some rare or new defect types, it may be impossible to obtain enough samples for training.

[0004] In order to overcome the limitations of supervised learning, unsupervised defect detection technology has gradually become a research hotspot. Among them, the teacher-student network model based on knowledge distillation has shown great potential. This model constructs two networks with similar structures but different capacities or performances, namely the teacher branch and the student branch, and uses their differences in the learning process to capture abnormal information, thereby realizing defect detection. The teacher branch is usually a pre-trained, high-performance model responsible for extracting deep-level features of the input data; while the student branch is a simplified or incompletely trained model with relatively weak learning ability. When both process the same data at the same time, since the student branch cannot fully reproduce the output results of the teacher branch, this difference may be caused by defects in the input data. However, this method often has fuzzy and incomplete problems in defect location, which affects the final detection accuracy and efficiency. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned shortcomings of the prior art and to provide a defect detection model construction method, a defect detection method and related devices.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a defect detection model construction method, comprising:

[0008] Acquire an initial training image, and add a simulated anomaly to the initial training image to obtain a simulated anomaly training image;

[0009] The preset defect recognition model is trained according to the initial training image and the simulated abnormal training image to obtain the defect location information and mask prompt information;

[0010] Train the preset defect segmentation model based on the defect location information and mask prompt information;

[0011] The defect detection model is obtained by integrating the trained defect recognition model and defect segmentation model;

[0012] Among them, the defect recognition model includes a teacher network branch, a student network branch and an autoencoder branch; the teacher network branch adopts fixed network weights, and the student network branch and the autoencoder branch adopt trainable network weights; the output difference between the teacher network branch and the student network branch is the defect location information, and the output difference between the student network branch and the autoencoder branch is the mask prompt information.

[0013] Optionally, the teacher network branch and the student network branch adopt the same network structure, and both are encoder structures; the network structure of the autoencoder branch is an encoder-decoder structure.

[0014] Optionally, adding a simulated abnormality to the initial training image to obtain a simulated abnormality training image includes: adding a simulated abnormality to the initial training image using an abnormality generator to obtain a simulated abnormality training image by the following formula:

[0015]

[0016] Among them, I a is the simulated abnormal training image, I is the initial training image, M is the simulated abnormal mask, is the inverse of M, ⊙ is the pixel-by-pixel multiplication, α is the opacity parameter in the blend, and A is the additional texture image.

[0017] Optionally, when the preset defect recognition model is trained according to the initial training image and the simulated abnormal training image, the training loss function of the teacher network branch-student network branch is:

[0018]

[0019]

[0020] Among them, L ST represents the loss function between the student network branch and the teacher network branch; L hard represents the difficult sample loss function; H, W and C represent the height, width and number of channels of the output feature map respectively; S(P) c represents the output of the student network branch; c represents the number of channels; L STAE represents the loss function between the autoencoder network branch and the student network branch; S′(I) c A(I) represents the additional output of the student network branch; c represents the output of the autoencoder network branch;

[0021] The training loss function of the teacher network branch-autoencoder branch is:

[0022]

[0023] Among them, L AE represents the loss function between the autoencoder network branch and the teacher network branch; T(I) c Represents the output of the teacher network branch.

[0024] Optionally, the defect segmentation model is obtained by introducing a trainable branch on the basis of a pre-trained SAM model and modifying the self-attention mechanism in the Transformer module of the pre-trained SAM model; and when the preset defect segmentation model is trained according to the defect location information and the mask prompt information, the parameters of the pre-trained SAM model are fixed;

[0025] Among them, the generation process of query Q, key K and value V in the modified self-attention mechanism is:

[0026] Q=X·W Q1 ·W Q2 +X·W Q

[0027] K=X·W K

[0028] V=X·W V1 ·W V2 +X·W V

[0029] Among them, X represents the input feature; W Q1 represents the first part of the newly added query matrix based on the trainable branch; W Q2 represents the second part of the newly added query matrix based on the trainable branch; W Q represents the original query matrix of the SAM model; W K represents the original bond matrix of the SAM model; W V1represents the first part of the newly added value matrix based on the trainable branch; W V2 represents the second part of the newly added value matrix based on the trainable branch; W V Represents the raw value matrix of the SAM model.

[0030] A second aspect of the present invention provides a defect detection model building system, comprising:

[0031] An image processing module is used to obtain an initial training image and add a simulated anomaly to the initial training image to obtain a simulated anomaly training image;

[0032] The recognition model training module is used to train the preset defect recognition model according to the initial training image and the simulated abnormal training image to obtain the defect location information and mask prompt information;

[0033] A segmentation model training module is used to train a preset defect segmentation model according to defect location information and mask prompt information;

[0034] Model fusion module, used to fuse the trained defect recognition model and defect segmentation model to obtain a defect detection model;

[0035] Among them, the defect recognition model includes a teacher network branch, a student network branch and an autoencoder branch; the teacher network branch adopts fixed network weights, and the student network branch and the autoencoder branch adopt trainable network weights; the output difference between the teacher network branch and the student network branch is the defect location information, and the output difference between the student network branch and the autoencoder branch is the mask prompt information.

[0036] A third aspect of the present invention provides a defect detection method, comprising:

[0037] Obtaining a sample image to be detected;

[0038] The image of the sample to be detected is input into a preset defect detection model to obtain the defect detection result of the sample to be detected; wherein the preset defect detection model is constructed using the above-mentioned defect detection model construction method.

[0039] A fourth aspect of the present invention provides a defect detection system, comprising:

[0040] A sample image acquisition module is used to acquire a sample image to be detected;

[0041] The defect detection module is used to input the image of the sample to be detected into a preset defect detection model to obtain the defect detection result of the sample to be detected; wherein the preset defect detection model is constructed using the above-mentioned defect detection model construction method.

[0042] According to a fifth aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned defect detection model construction method or the steps of the above-mentioned defect detection method when executing the computer program.

[0043] According to a sixth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned defect detection model construction method are implemented, or the steps of the above-mentioned defect detection method are implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The defect detection model construction method of the present invention adds simulated abnormalities to the initial training image to obtain a simulated abnormality training image, then trains a preset defect recognition model based on the initial training image and the simulated abnormality training image to obtain defect location information and mask prompt information, and then trains a preset defect segmentation model based on the defect location information and the mask prompt information, and finally fuses the trained defect recognition model and defect segmentation model to obtain a defect detection model. Among them, the defect recognition model introduces an autoencoder branch to provide mask prompt information, and finally uses a defect segmentation model to refine and segment the defect location information, so that when the defect detection model constructed by the construction method is used for product defect detection, it can effectively reduce the missed detection rate and over-detection rate in the product defect detection process, and improve the defect detection accuracy while taking into account the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of a defect detection model construction method according to an embodiment of the present invention.

[0047] Figure 2 Schematic diagram of the defect detection model principle of an embodiment of the present invention.

[0048] Figure 3 The figure is a flow chart of a defect detection method according to an embodiment of the present invention.

[0049] Figure 4 A system structure block diagram of a defect detection model construction system according to an embodiment of the present invention.

[0050] Figure 5 4 is a structural block diagram of a defect detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0052] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0053] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0054] With the development of modern machine learning and artificial intelligence technologies, more and more related technologies are being applied in industrial scenarios. Since the system or human factors in the industrial production process will inevitably cause defects such as scratches and cracks on the product surface, the speed and accuracy of traditional manual defect detection have severely limited the production efficiency of the product and cannot meet the detection requirements of industrial scenarios.

[0055] Although deep neural network-based defect detection methods can currently be used to perform end-to-end feature extraction on defect images, most methods cannot achieve both accuracy and efficiency in detection.

[0056] See also Figure 1 and 2 In one embodiment of the present invention, a defect detection model construction method is provided. By constructing an innovative defect detection model, the detection accuracy is improved while taking into account the detection efficiency. Specifically, the defect detection model construction method of the present invention includes the following steps:

[0057] S1: Acquire an initial training image, and add simulated anomalies to the initial training image to obtain a simulated anomaly training image.

[0058] S2: Train a preset defect recognition model according to the initial training image and the simulated abnormal training image to obtain defect location information and mask prompt information.

[0059] S3: Train a preset defect segmentation model based on defect location information and mask prompt information.

[0060] S4: The defect recognition model and defect segmentation model after training are integrated to obtain a defect detection model.

[0061] Among them, the defect recognition model includes a teacher network branch, a student network branch and an autoencoder branch; the teacher network branch adopts fixed network weights, and the student network branch and the autoencoder branch adopt trainable network weights; the output difference between the teacher network branch and the student network branch is the defect location information, and the output difference between the student network branch and the autoencoder branch is the mask prompt information.

[0062] The defect detection model construction method of the present invention adds simulated abnormalities to the initial training image to obtain a simulated abnormality training image, then trains a preset defect recognition model based on the initial training image and the simulated abnormality training image to obtain defect location information and mask prompt information, and then trains a preset defect segmentation model based on the defect location information and the mask prompt information, and finally fuses the trained defect recognition model and defect segmentation model to obtain a defect detection model. Among them, the defect recognition model introduces an autoencoder branch to provide mask prompt information, and finally uses a defect segmentation model to refine and segment the defect location information, so that when the defect detection model constructed by the construction method is used for product defect detection, it can effectively reduce the missed detection rate and over-detection rate in the product defect detection process, and improve the defect detection accuracy while taking into account the detection efficiency.

[0063] In a possible implementation, the teacher network branch and the student network branch adopt the same network structure, and both are encoder structures; the network structure of the autoencoder branch is an encoder-decoder structure.

[0064] Specifically, a lightweight pre-trained network is used as the teacher network branch. The network structure of the student network branch is consistent with the teacher network branch, but the parameters are not pre-trained. The autoencoder branch adopts a convolutional autoencoder structure to ensure that the output structures of the three network branches are consistent.

[0065] By introducing an autoencoder branch to provide mask hint information, we overcome the problem of fuzzy and incomplete defect positioning in the traditional teacher-student network based on knowledge distillation, so as to improve the detection accuracy.

[0066] In a possible implementation, adding a simulated abnormality to the initial training image to obtain a simulated abnormality training image includes: adding a simulated abnormality to the initial training image using an abnormality generator to obtain a simulated abnormality training image according to the following formula:

[0067]

[0068] Among them, I a is the simulated abnormal training image, I is the initial training image, M is the simulated abnormal mask, is the inverse of M, ⊙ is the pixel-by-pixel multiplication, α is the opacity parameter in the blend, and A is the additional texture image.

[0069] A simulated anomaly generator is used to add artificial simulated anomalies to the initial training image to obtain an initial training image with added simulated anomalies, namely a simulated anomaly training image, so as to overcome the problem of difficulty in obtaining a large number of training samples.

[0070] In a possible implementation, when the preset defect recognition model is trained according to the initial training image and the simulated abnormal training image, the training loss function of the teacher network branch-student network branch is:

[0071]

[0072]

[0073] Among them, L ST represents the loss function between the student network branch and the teacher network branch; L hard represents the difficult sample loss function; H, W and C represent the height, width and number of channels of the output feature map respectively; S(P) c represents the output of the student network branch; c represents the number of channels; L STAE represents the loss function between the autoencoder network branch and the student network branch; S′(I) c A(I) represents the additional output of the student network branch; c represents the output of the autoencoder network branch;

[0074] The training loss function of the teacher network branch-autoencoder branch is:

[0075]

[0076] Among them, L AE represents the loss function between the autoencoder network branch and the teacher network branch; T(I) c Represents the output of the teacher network branch.

[0077] Specifically, when training a preset defect recognition model based on the initial training image and the simulated abnormal training image, the simulated abnormal training image is input into the student network branch, and the initial training image is input into the teacher network branch and the autoencoder branch. During the training process, the parameter weight of the teacher network branch remains unchanged, the student network branch is trained to predict the output of the teacher network branch and the autoencoder branch, and the autoencoder branch is trained to predict the output of the teacher network branch. The output difference between the teacher network branch and the student network branch is used as the defect location information, and the output difference between the student network branch and the autoencoder branch is used as the mask prompt information. The above defect location information and mask prompt information are input into the initial defect segmentation model for training, and the ideal output of each sample in the training process is the known simulated abnormal binary mask.

[0078] In one possible implementation, the defect segmentation model is obtained by introducing a trainable branch based on a pre-trained SAM model and modifying the self-attention mechanism in the Transformer module of the pre-trained SAM model; and when the preset defect segmentation model is trained according to the defect location information and mask prompt information, the parameters of the pre-trained SAM model are fixed.

[0079] Among them, the generation process of query Q, key K and value V in the modified self-attention mechanism is:

[0080] Q=X·W Q1 ·W Q2 +X·W Q

[0081] K=X·W K

[0082] V=X·W V1 ·W V2 +X·W V

[0083] Among them, X represents the input feature; W Q1 represents the first part of the newly added query matrix based on the trainable branch; W Q2 represents the second part of the newly added query matrix based on the trainable branch; W Q represents the original query matrix of the SAM model; W K represents the original bond matrix of the SAM model; W V1 represents the first part of the newly added value matrix based on the trainable branch; W V2 represents the second part of the newly added value matrix based on the trainable branch; W V Represents the raw value matrix of the SAM model.

[0084] Specifically, the defect segmentation model selects the visual large model SAM as the basis. While retaining the original SAM model architecture, a trainable branch is introduced to provide relevant additional parameters for defect detection domain knowledge. By modifying the self-attention mechanism in the Transformer module of the SAM model, the integration of general knowledge and defect detection specific knowledge is achieved. In the original SAM model, the generation process of query Q, key K and value V in the self-attention mechanism can be expressed by the following formula:

[0085] Q=X·W Q

[0086] K=X·W K

[0087] V=X·W V

[0088] Among them, W Q , W K and W V It is a weight matrix, which is multiplied with the input feature X to generate the corresponding query Q, key K and value V. The input feature is mapped to the space of query Q, key K and value V by using the properties of matrix multiplication.

[0089] In this embodiment, low-rank adaptive technology is used to modify the generation of query Q, key K and value V in the SAM model. The generation of query Q, key K and value V can be represented as the additional linear layer W obtained by the original weight matrix and the trainable branch Q1 , W Q2 , W V1 and W V2 Combination of:

[0090] Q=X·W Q1 ·W Q2 +X·W Q

[0091] K=X·W K

[0092] V=X·W V1 ·W V2 +X·W V

[0093] In the above formula, X represents the input feature. Q1 and W Q2 The output of actually forms a new weight matrix, which represents the newly learned knowledge for the defect detection task. In this case, W Q1 and W Q2 can be viewed as components of a weight matrix that are trained to capture specific features relevant to defect detection. In this setting, the original weight matrix W QThe general knowledge learned from the pre-trained SAM model is retained, while W Q1 ·W Q2 It provides task-specific knowledge for defect detection. Finally, the trained defect recognition model and defect segmentation model are fused to obtain a complete defect detection model.

[0094] See also Figure 3 In another embodiment of the present invention, a defect detection method is provided, comprising the following steps:

[0095] S11: Obtain a sample image to be detected.

[0096] S12: Inputting the image of the sample to be detected into a preset defect detection model to obtain the defect detection result of the sample to be detected; wherein the preset defect detection model is constructed using the above-mentioned defect detection model construction method.

[0097] Specifically, when performing defect detection, the sample image to be detected is input into the defect recognition model, and then input into the teacher network branch, the student network branch and the autoencoder branch respectively, and then the outputs of the teacher network branch, the student network branch and the autoencoder branch are obtained respectively. The difference between the outputs of the teacher network branch and the student network branch is used as the defect location information, and the difference between the student network branch and the autoencoder branch is used as the mask prompt information. The defect location information and the mask prompt information are input into the defect segmentation model, and then it is determined whether there are defects in the sample image to be detected, and the corresponding defect segmentation result, such as a defect mask map, is output.

[0098] The design concept of the network structure of the defect detection model of the present invention is very novel, and provides a corresponding solution to the actual problem of industrial defect detection. In actual scenarios, it can be applied to the surface defect detection of objects in most industrial fields, especially in situations where both accuracy and real-time performance are required. It has good application value. The main steps of the defect detection method are: 1. Constructing a data set for training, such as a normal mobile phone screen, cloth surface image, etc.; 2. Integrating the defect detection model of the present invention into a specific deep learning framework, such as TensorFlow, Caffe and PyTorch; 3. Training the defect detection model, according to a certain object category, respectively training the defect recognition model and the defect segmentation model; 4. Testing the defect detection model, constructing a data set for testing, converting it into an input image of a set size and inputting it into the defect detection model, and obtaining the predicted defect location information. By comparing other methods, this method has a higher accuracy and real-time performance for different categories of inspected objects, and can basically meet the requirements of real industrial scenarios.

[0099] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0100] See also Figure 4 In another embodiment of the present invention, a defect detection model construction system is provided, which can be used to implement the above-mentioned defect detection model construction method. Specifically, the defect detection model construction system includes an image processing module, a recognition model training module, a segmentation model training module and a model fusion module.

[0101] Among them, the image processing module is used to obtain the initial training image, and add simulated anomalies to the initial training image to obtain a simulated anomaly training image; the recognition model training module is used to train a preset defect recognition model according to the initial training image and the simulated anomaly training image to obtain defect location information and mask prompt information; the segmentation model training module is used to train a preset defect segmentation model according to the defect location information and mask prompt information; the model fusion module is used to fuse the trained defect recognition model and defect segmentation model to obtain a defect detection model; wherein the defect recognition model includes a teacher network branch, a student network branch and an autoencoder branch; the teacher network branch adopts fixed network weights, and the student network branch and the autoencoder branch adopt trainable network weights; the output difference between the teacher network branch and the student network branch is the defect location information, and the output difference between the student network branch and the autoencoder branch is the mask prompt information.

[0102] All relevant contents of each step involved in the above-mentioned embodiment of the defect detection model construction method can be referred to the functional description of the functional modules corresponding to the defect detection model construction system in the embodiment of the present invention, and will not be repeated here.

[0103] See also Figure 5 In another embodiment of the present invention, a defect detection system is provided, which can be used to implement the above-mentioned defect detection method. Specifically, the defect detection system includes a sample image acquisition module and a defect detection module. The sample image acquisition module is used to acquire the sample image to be detected; the defect detection module is used to input the sample image to be detected into a preset defect detection model to obtain the defect detection result of the sample to be detected; the preset defect detection model is constructed by the above-mentioned defect detection model construction method.

[0104] All relevant contents of each step involved in the above-mentioned embodiment of the defect detection method can be referred to the functional description of the functional module corresponding to the defect detection system in the embodiment of the present invention, and will not be repeated here.

[0105] The division of modules in the embodiments of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0106] In another embodiment of the present invention, a computer device is provided, the computer device includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used for the operation of the defect detection model construction method and the defect detection method.

[0107] In another embodiment of the present invention, the present invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It can be understood that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the defect detection model construction method and the defect detection method in the above embodiment.

[0108] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0109] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A defect detection model construction method, characterized in that: include: Acquire an initial training image, and add a simulated anomaly to the initial training image to obtain a simulated anomaly training image; The preset defect recognition model is trained according to the initial training image and the simulated abnormal training image to obtain the defect location information and mask prompt information; Train the preset defect segmentation model based on the defect location information and mask prompt information; The defect detection model is obtained by integrating the trained defect recognition model and defect segmentation model; Among them, the defect recognition model includes a teacher network branch, a student network branch and an autoencoder branch; the teacher network branch adopts fixed network weights, and the student network branch and the autoencoder branch adopt trainable network weights; the output difference between the teacher network branch and the student network branch is the defect location information, and the output difference between the student network branch and the autoencoder branch is the mask prompt information.

2. The defect detection model construction method according to claim 1, characterized in that: The teacher network branch and the student network branch adopt the same network structure, and both are encoder structures; the network structure of the autoencoder branch is an encoder-decoder structure.

3. The defect detection model construction method according to claim 1, characterized in that: The adding of simulated abnormality to the initial training image to obtain the simulated abnormality training image comprises: adding simulated abnormality to the initial training image using an abnormality generator to obtain the simulated abnormality training image by the following formula: Among them, I a is the simulated abnormal training image, I is the initial training image, M is the simulated abnormal mask, is the inverse of M, ⊙ is the pixel-by-pixel multiplication, α is the opacity parameter in the blend, and A is the additional texture image.

4. The defect detection model construction method according to claim 1, characterized in that: When the preset defect recognition model is trained according to the initial training image and the simulated abnormal training image, the training loss function of the teacher network branch-student network branch is: Among them, L ST represents the loss function between the student network branch and the teacher network branch; L hard represents the difficult sample loss function; H, W and C represent the height, width and number of channels of the output feature map respectively; S(P) c represents the output of the student network branch; c represents the number of channels; L STAE represents the loss function between the autoencoder network branch and the student network branch; S′(I) c A(I) represents the additional output of the student network branch; c represents the output of the autoencoder network branch; The training loss function of the teacher network branch-autoencoder branch is: Among them, L AE represents the loss function between the autoencoder network branch and the teacher network branch; T(I) c Represents the output of the teacher network branch.

5. The defect detection model construction method according to claim 1, characterized in that: The defect segmentation model is obtained by introducing a trainable branch on the basis of a pre-trained SAM model and modifying the self-attention mechanism in the Transformer module of the pre-trained SAM model; and when the preset defect segmentation model is trained according to the defect position information and the mask prompt information, the parameters of the pre-trained SAM model are fixed; Among them, the generation process of query Q, key K and value V in the modified self-attention mechanism is: Q=X·W Q1 ·W Q2 +X·W Q K=X·W K V=X·W V1 ·W V2 +X·W V Among them, X represents the input feature; W Q1 represents the first part of the newly added query matrix based on the trainable branch; W Q2 represents the second part of the newly added query matrix based on the trainable branch; W Q represents the original query matrix of the SAM model; W K represents the original bond matrix of the SAM model; W V1 represents the first part of the newly added value matrix based on the trainable branch; W V2 represents the second part of the newly added value matrix based on the trainable branch; W V Represents the raw value matrix of the SAM model.

6. A defect detection model building system, characterized in that: include: An image processing module is used to obtain an initial training image and add a simulated anomaly to the initial training image to obtain a simulated anomaly training image; The recognition model training module is used to train the preset defect recognition model according to the initial training image and the simulated abnormal training image to obtain the defect location information and mask prompt information; A segmentation model training module is used to train a preset defect segmentation model according to defect location information and mask prompt information; Model fusion module, used to fuse the trained defect recognition model and defect segmentation model to obtain a defect detection model; Among them, the defect recognition model includes a teacher network branch, a student network branch and an autoencoder branch; the teacher network branch adopts fixed network weights, and the student network branch and the autoencoder branch adopt trainable network weights; the output difference between the teacher network branch and the student network branch is the defect location information, and the output difference between the student network branch and the autoencoder branch is the mask prompt information.

7. A defect detection method, characterized in that: include: Obtaining a sample image to be detected; The image of the sample to be detected is input into a preset defect detection model to obtain the defect detection result of the sample to be detected; wherein the preset defect detection model is constructed using the defect detection model construction method described in any one of claims 1 to 5.

8. A defect detection system, characterized in that: include: A sample image acquisition module is used to acquire a sample image to be detected; A defect detection module is used to input the image of the sample to be detected into a preset defect detection model to obtain the defect detection result of the sample to be detected; wherein the preset defect detection model is constructed using the defect detection model construction method described in any one of claims 1 to 5.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the processor implements the steps of the defect detection model construction method according to any one of claims 1 to 5, or implements the steps of the defect detection method according to claim 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the defect detection model construction method according to any one of claims 1 to 5 are implemented, or the steps of the defect detection method according to claim 7 are implemented.

Images