Unsupervised defect detection model training method and device based on continuous learning, defect detection method and device, electronic equipment, storage medium and computer program product

Abstract

The invention relates to an unsupervised defect detection model training method and device based on continuous learning, a defect detection method and device, electronic equipment, a storage medium and a computer program product. The method comprises the steps of obtaining product sample image features corresponding to a current task; feature disturbance is added for product sample image features corresponding to the current task; inputting the product sample image features corresponding to the current task after the feature disturbance is added and the learnable query embedded vector corresponding to the current task into a defect detection model; calculating loss based on the reconstructed image features and the product sample image features; and adjusting the parameters based on the loss. Therefore, by introducing the prompt signal of the historical task, the query embedding vectors of the new task and the old task can be close to each other as much as possible, so that the defect detection model can be helped to train the new task and keep the memory of the old task at the same time, and the forgetting resistance and the generalization ability of the defect detection model on the new task can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to a training method of an unsupervised defect detection model based on continuous learning, a defect detection method, an apparatus, an electronic device, a storage medium, and a computer program product. Background Art

[0002] In modern manufacturing, defect detection is generally performed at the end of production, and its main purpose is to try to identify defects in products to ensure product quality. In the past, defect detection in industrial production was mainly performed by experienced workers. However, this method consumes high labor costs, and if the workers are in a relatively tired state, it will also lead to the occurrence of false positives, where "false positives" refer to samples that are actually abnormal but are identified as normal samples. In order to solve the problem of high labor costs brought about by the above-mentioned manual defect detection method, the relevant technology mainly adopts an unsupervised method to perform product defect detection.

[0003] However, current unsupervised defect detection methods are usually only effective for existing static product data. When faced with dynamic product data that is continuously updated on industrial production lines, the defect detection capability of the model will be greatly reduced. In other words, the generalization ability of current defect detection models is poor, resulting in the inability to effectively detect defects in products that are continuously updated on industrial production lines. Summary of the invention

[0004] The present disclosure provides a training method, defect detection method, device, electronic device, storage medium and computer program product of an unsupervised defect detection model based on continuous learning, so as to at least solve the problem in the above-mentioned related technologies that the generalization ability of the defect detection model is poor, resulting in the inability to effectively detect defects in products that are constantly updated on industrial production lines.

[0005] According to a first aspect of an embodiment of the present disclosure, a training method for an unsupervised defect detection model based on continuous learning is provided, comprising: obtaining product sample image features corresponding to a current task, wherein the current task is one of a plurality of tasks performed sequentially, each of the plurality of tasks corresponds to a plurality of types of product sample images, and the product sample image features are image features extracted from the plurality of types of product sample images; adding feature perturbations to the product sample image features corresponding to the current task; inputting the product sample image features corresponding to the current task after the feature perturbations are added and a learnable query embedding vector corresponding to the current task into the defect detection model to obtain reconstructed image features, wherein the learnable query embedding vector corresponding to the current task is a query embedding vector obtained by performing knowledge distillation on a trained query embedding vector corresponding to a previous task before the current task and an initialized query embedding vector of the current task; calculating a loss based on the reconstructed image features and the product sample image features; and training the defect detection model by adjusting the parameters of the defect detection model based on the loss.

[0006] Optionally, obtaining product sample image features corresponding to the current task includes: performing feature extraction on multiple types of product sample images corresponding to the current task to obtain current product sample image features corresponding to the current task; randomly selecting some product sample image features from multiple product sample image features corresponding to each task in at least one task before the current task; and determining the current product sample image features and the randomly selected some product sample image features as the product sample image features corresponding to the current task.

[0007] Optionally, the defect detection model includes a feature reconstruction network, which includes an encoder and a decoder; the product sample image features corresponding to the current task after the feature perturbation is added and the learnable query embedding vector corresponding to the current task are input into the defect detection model to obtain reconstructed image features, including: inputting the product sample image features corresponding to the current task after the feature perturbation is added into the encoder to obtain an encoding result; inputting the encoding result and the learnable query embedding vector corresponding to the current task into the decoder to obtain the reconstructed image features.

[0008] Optionally, the encoder includes a plurality of cascaded encoders, and the decoder includes a plurality of cascaded decoders.

[0009] According to a second aspect of an embodiment of the present disclosure, a defect detection method is provided, comprising: acquiring target image features of an image containing a product to be inspected; inputting the target image features and a query embedding vector into a defect detection model trained according to the training method of the present disclosure to obtain target reconstructed image features, wherein the query embedding vector is a query embedding vector obtained by updating an initialized query embedding vector for a preset task during the training process of the defect detection model; and determining whether the product to be inspected has defects based on the target reconstructed image features and the target image features.

[0010] Optionally, determining whether the product to be inspected has defects based on the target reconstructed image features and the target image features includes: calculating a target mean square loss error between the target reconstructed image features and the target image features; when the target mean square loss error is less than or equal to a preset threshold, determining that the product to be inspected does not have abnormalities; otherwise, determining that the product to be inspected has abnormalities.

[0011] According to a third aspect of an embodiment of the present disclosure, a training device for a defect detection model is provided, comprising: an image feature acquisition module, configured to acquire product sample image features corresponding to a current task, wherein the current task is one of a plurality of tasks executed in sequence, each of the plurality of tasks corresponds to a plurality of types of product sample images, and the product sample image features are image features extracted from the plurality of types of product sample images; a feature perturbation adding module, configured to add feature perturbations to the product sample image features corresponding to the current task; a reconstruction module, configured to input the product sample image features corresponding to the current task after the feature perturbations are added and the learnable query embedding vector corresponding to the current task into an unsupervised defect detection model based on continuous learning to obtain reconstructed image features, wherein the learnable query embedding vector corresponding to the current task is a query embedding vector obtained by performing knowledge distillation on a trained query embedding vector corresponding to a previous task before the current task and an initialized query embedding vector of the current task; a loss calculation module, configured to calculate losses based on the reconstructed image features and the product sample image features; and a parameter adjustment module, configured to train the defect detection model by adjusting the parameters of the defect detection model based on the losses.

[0012] Optionally, the image feature acquisition module is configured to: perform feature extraction on multiple types of product sample images corresponding to the current task to obtain current product sample image features corresponding to the current task; randomly select some product sample image features from multiple product sample image features corresponding to each task in at least one task before the current task; and determine the current product sample image features and the randomly selected some product sample image features as the product sample image features corresponding to the current task.

[0013] Optionally, the defect detection model includes a feature reconstruction network, which includes an encoder and a decoder; the reconstruction module is configured to: input the product sample image features corresponding to the current task after the feature perturbation is added into the encoder to obtain the encoding result; input the encoding result and the learnable query embedding vector corresponding to the current task into the decoder to obtain the reconstructed image features.

[0014] Optionally, the encoder includes a plurality of cascaded encoders, and the decoder includes a plurality of cascaded decoders.

[0015] According to a fourth aspect of an embodiment of the present disclosure, a defect detection device is provided, comprising: an image feature acquisition module to be detected, configured to acquire target image features of an image containing a product to be detected; a reconstruction feature acquisition module, configured to input the target image features and a query embedding vector into a defect detection model trained according to the training method of the present disclosure, to obtain target reconstructed image features, wherein the query embedding vector is a query embedding vector obtained by updating an initialized query embedding vector for a preset task during the training process of the defect detection model; a defect determination module, configured to determine whether the product to be detected has a defect based on the target reconstructed image features and the target image features.

[0016] Optionally, the defect determination module is configured to: calculate the target mean square loss error between the target reconstructed image features and the target image features; when the target mean square loss error is less than or equal to a preset threshold, determine that there is no abnormality in the product to be inspected; otherwise, determine that there is an abnormality in the product to be inspected.

[0017] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a training method for an unsupervised defect detection model based on continuous learning according to the present disclosure, or to implement a defect detection method according to the present disclosure.

[0018] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of an unsupervised defect detection model based on continuous learning according to the present disclosure, or to execute the defect detection method according to the present disclosure.

[0019] According to the seventh aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the training method of an unsupervised defect detection model based on continuous learning according to the present disclosure, or implements the defect detection method according to the present disclosure.

[0020] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0021] In the present disclosure, a learnable query embedding vector can be set for each of the multiple tasks. When a new task comes, the previous task that is executed before the current task, that is, the knowledge of the previous task, can be transferred to the current task through knowledge distillation. In this way, by introducing the prompt signal of the historical task, the query embedding vectors of the new and old tasks can be made as close as possible, that is, it can be ensured that the query of the old task will not deviate during the decoding process, which can help the defect detection model to maintain the memory of the old task while training the new task, thereby improving the defect detection model's anti-forgetfulness and generalization ability on new tasks.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0024] Figure 1 is a flow chart illustrating a training method of an unsupervised defect detection model based on continuous learning according to an exemplary embodiment of the present disclosure;

[0025] Figure 2 is a schematic diagram showing a process of acquiring training sample images and dividing continuous learning tasks according to an exemplary embodiment of the present disclosure;

[0026] Figure 3 is a schematic diagram illustrating extraction of product sample image features from a product sample image according to an exemplary embodiment of the present disclosure;

[0027] Figure 4is a schematic diagram showing adding feature disturbance to product sample image features according to an exemplary embodiment of the present disclosure;

[0028] Figure 5 is a schematic diagram showing the working process of a continuous distillation prompt module according to an exemplary embodiment of the present disclosure;

[0029] Figure 6 is a schematic diagram showing the structure of a feature reconstruction network according to an exemplary embodiment of the present disclosure;

[0030] Figure 7 is a flow chart illustrating a defect detection method according to an exemplary embodiment of the present disclosure;

[0031] Figure 8 is a block diagram illustrating a training apparatus for a defect detection model according to an exemplary embodiment of the present disclosure;

[0032] Fig. 9 is a block diagram illustrating a defect detection apparatus according to an exemplary embodiment of the present disclosure;

[0033] Fig.10 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0036] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0037] In modern manufacturing, defect detection is generally performed at the last stage of production and manufacturing, and its main purpose is to try to identify defects in products to ensure product quality. In the past, defect detection in industrial production was mainly carried out by experienced workers. However, this method will consume high labor costs, and if the workers are in a relatively tired state, it will also lead to the occurrence of false positives, where "false positives" refer to samples that are actually abnormal but are identified as normal samples. In order to solve the problem of high labor costs brought about by the above-mentioned manual defect detection method, the relevant technology mainly adopts an unsupervised method to detect product defects. It should be noted that the object of industrial defect detection and positioning in the field of computer vision is an image containing the product to be detected, that is, by identifying and locating abnormal images, it is determined whether the product shown in the image has defects.

[0038] However, current unsupervised defect detection methods are usually only effective for existing static product data. When faced with dynamic product data that is continuously updated on industrial production lines, the defect detection capability of the model will be greatly reduced. In other words, the generalization ability of current defect detection models is poor, resulting in the inability to effectively detect defects in products that are continuously updated on industrial production lines.

[0039] In order to solve the above-mentioned problems existing in the related art, the present disclosure provides a training method, defect detection method, device, electronic device, storage medium and computer program product for an unsupervised defect detection model based on continuous learning. For each of the multiple tasks, a learnable query embedding vector can be set. When a new task comes, the previous task whose execution order is before the current task, that is, the knowledge of the previous task, can be transferred to the current task through knowledge distillation. In this way, by introducing the prompt signal of the historical task, the query embedding vectors of the new and old tasks can be made as close as possible, that is, it can be ensured that the query of the old task will not deviate during the decoding process, which can help the defect detection model to maintain the memory of the old task while training the new task, thereby improving the defect detection model's anti-forgetting and generalization ability on new tasks.

[0040] Figure 1 is a flowchart illustrating a training method of an unsupervised defect detection model based on continuous learning according to an exemplary embodiment of the present disclosure.

[0041] Reference Figure 1In step 101, the product sample image features corresponding to the current task may be obtained, wherein the current task may be one of a plurality of tasks to be performed in sequence, each of the plurality of tasks may correspond to a plurality of types of product sample images, and the product sample image features may be image features extracted from the plurality of types of product sample images. Specifically, normal images containing various types of industrial objects may be obtained as training sample images, and these training sample images may be randomly shuffled. Then, these training sample images may be respectively classified into different continuous learning tasks according to the number of tasks to meet the requirements of continuous learning.

[0042] Figure 2 is a schematic diagram showing a process of acquiring training sample images and dividing continuous learning tasks according to an exemplary embodiment of the present disclosure. Figure 2 First, normal sample images containing various types of products can be obtained through industrial cameras or other imaging devices, where "normal sample images" refer to sample images containing normal products, that is, samples without abnormal products. Then, the large number of normal sample images obtained can be broken up according to product types and divided into different continuous learning tasks for continuous learning. Exemplarily, assuming that there are N types of products in total and the number of continuous learning tasks is T, the sample images of these N types of products can be broken up and divided into different continuous learning tasks among the T continuous learning tasks for continuous learning.

[0043] It should be noted that in the present disclosure, the public industrial defect detection dataset MVTecAD can be used to train the defect detection model. The MVTecAD dataset may contain normal training sample images of 15 different types of industrial products, that is, training sample images without abnormalities. Exemplarily, the 15 different types of industrial products may include 10 object-type industrial products and 5 texture-type industrial products. "Object-type industrial products" may include but are not limited to: camera lenses, bottles, pills, metal nuts, screws, capsules, etc.; "texture-type industrial products" may include but are not limited to: wooden industrial products, blankets and other industrial products with patterned images.

[0044] In addition, continuous learning tasks can be set according to actual needs, and the normal training sample images of the above 15 different types of industrial products can be randomly shuffled and classified into different continuous learning tasks for continuous learning. Exemplarily, one task setting method can be: a total of 5 continuous learning tasks are set, and each continuous learning task can correspond to normal training sample images of 3 types of industrial products, that is, each continuous learning task can only learn 3 industrial product categories, and at the same time, continuous learning can also be carried out in a quasi-incremental learning manner; another task setting method can be: a total of 6 continuous learning tasks are set, and the first continuous learning task can learn 10 industrial product categories, and each of the subsequent 5 continuous learning tasks can correspond to incremental learning of one industrial product category, and at the same time, continuous learning can also be carried out in a quasi-incremental learning manner.

[0045] In this way, by obtaining normal training sample images of various different types of industrial products, it can be ensured that a large number of image samples of different types of industrial products can be effectively used in the training of defect detection models, which can ensure the adaptability of defect detection models to diversified image data, and thus make the defect detection model have better robustness.

[0046] It should be noted that after obtaining various types of product sample images, the pre-trained convolutional neural network can be used to extract image features from these product sample images, and then the product sample image features can be obtained, that is, the feature map after multi-scale feature fusion can be obtained. Figure 3 is a schematic diagram illustrating extracting product sample image features from a product sample image according to an exemplary embodiment of the present disclosure. Figure 3 , the EfficientNet_b4 model pre-trained on the large-scale public dataset ImageNet can be used to extract features from product sample images, and then the final product sample image features can be obtained through multi-scale feature fusion processing.

[0047] Thus, in the present disclosure, since the EfficientNet model has a powerful feature extraction capability, the multi-scale feature extraction function of the EfficientNet model can be used to capture the detail information and global information in the product sample image, and the richness and accuracy of the feature expression can be improved through multi-scale feature fusion processing.

[0048] According to an exemplary embodiment of the present disclosure, feature extraction can be performed on multiple types of product sample images corresponding to the current task to obtain the current product sample image features corresponding to the current task. Then, some product sample image features can be randomly selected from the multiple product sample image features corresponding to each task in at least one task before the current task. Next, the current product sample image features and some product sample image features randomly selected from the multiple product sample image features corresponding to the previous task can be determined as the product sample image features corresponding to the current task, that is, the product sample image features corresponding to the current task itself and some product sample image features randomly selected from the multiple product sample image features corresponding to the previous task itself can be used together to perform training for the current task.

[0049] Specifically, a representative feature subset can be selected as the coreset from the product sample image feature pool corresponding to each previous task of the current task. For example, a greedy selection method can be used to select a representative feature subset as the coreset from the product sample image feature pool corresponding to each previous task of the current task. Then, during the training process for the current task, the above coreset can be replayed to retain the important feature information representative of the previous task of the current task, that is, the old task, so as to improve the recognition and positioning capabilities of the defect detection model. That is, through this feature playback method, the defect detection model can retain the memory of the old task when training a new task, thereby improving its overall performance.

[0050] Exemplarily, assume that the product sample image feature pool corresponding to the first continuous learning task contains a total of 100 product sample image features, and assume that the product sample image feature pool corresponding to the second continuous learning task contains a total of 200 product sample image features. When training the second continuous learning task, several product sample image features can be randomly selected from the 100 product sample image features contained in the product sample image feature pool corresponding to the first continuous learning task. Exemplarily, 5 product sample image features can be randomly selected. Then, the 200 product sample image features corresponding to the second continuous learning task itself and several product sample image features randomly selected from the 100 product sample image features under the first continuous learning task can be used together for training the second continuous learning task.

[0051] Similarly, assuming that the product sample image feature pool corresponding to the third continuous learning task contains a total of 150 product sample image features, before training the third continuous learning task, several product sample image features can be randomly selected from the 200 product sample image features contained in the product sample image feature pool corresponding to the second continuous learning task. Then, several product sample image features randomly selected from the 100 product sample image features contained in the product sample image feature pool corresponding to the first continuous learning task, several product sample image features randomly selected from the 200 product sample image features corresponding to the second continuous learning task itself, and the 150 product sample image features corresponding to the third continuous learning task itself can be used together for training the third continuous learning task.

[0052] In the present disclosure, a quasi-incremental learning method can be adopted, and the memory ability of the defect detection model for old tasks can be improved through feature playback. That is, the coreset downsampling method can be used for feature selection, and then a representative feature subset can be selected from the feature pool corresponding to the previous task of the current task, that is, a more representative feature map can be obtained from the feature distribution corresponding to the previous task of the current task. In this way, in the process of training a new task, the more representative feature map corresponding to the previous task obtained by the coreset downsampling method can be used for playback, and then the representative important feature information in the old task can be retained, thereby improving the anti-forgetting ability and recognition and positioning capabilities of the defect detection model.

[0053] In step 102, feature disturbance may be added to the product sample image features corresponding to the current task. Figure 4 is a schematic diagram showing the addition of feature disturbance to product sample image features according to an exemplary embodiment of the present disclosure. Figure 4 , perturbations can be sampled from the Gaussian distribution, and then the perturbations sampled from the Gaussian distribution can be used to add feature perturbations to the product sample image features corresponding to the current task, so as to guide the defect detection model to learn the knowledge in the normal product sample images through denoising.

[0054] Specifically, noise disturbance can be sampled and the sampled noise disturbance can be added to the product sample image feature f, that is, the feature map f, with a fixed jitter probability p. For the product sample image feature f with a dimension C, the sampled disturbance can be:

[0055]

[0056] Among them, α is the characteristic disturbance coefficient, which can be used to control the jitter scale of the noise level. The noise disturbance D obeys the Gaussian distribution N, where μ is the mean and σ2 is the variance.

[0057] Thus, in the present disclosure, by adding feature perturbations, i.e., noise perturbations, to the product sample image features, i.e., feature maps, the model can better learn and adapt to the noise in the image, i.e., the addition of feature perturbations can enhance the defect detection model's ability to reconstruct positive sample images, i.e., it can enhance the defect detection model's ability to reconstruct sample images of products without abnormalities, thereby enhancing the robustness and generalization ability of the defect detection model, so that it can still maintain a good normal image, i.e., the ability to reconstruct defect-free images, when facing various image noises.

[0058] It should be noted that, in the present disclosure, by adding noise disturbance to the product sample image features, it is equivalent to artificially adding defects to the product sample images, that is, it is equivalent to artificially creating defects. In addition, the purpose of the feature reconstruction network is to reconstruct a normal sample image, that is, to reconstruct a product sample image that is defect-free, non-abnormal, and in an ideal state before adding noise disturbance. In this way, by calculating the loss between the reconstructed image features and the original product sample image features and updating the parameters of the defect detection model based on the calculated loss, that is, by calculating the loss between the reconstructed image features without abnormalities in an ideal state and the normal product sample image features before adding noise disturbance and updating the parameters of the defect detection model based on the calculated loss, the reconstruction ability of the defect detection model can be gradually improved, that is, the restoration ability of the defect detection model can be gradually improved, that is, it can ensure that the defect detection model can more accurately restore the original state of the product sample image before adding noise disturbance, that is, the real state, that is, the state without abnormalities.

[0059] In this way, after the defect detection model is trained, the image to be detected can be directly input into the defect detection model, and what the normal image of the image to be detected should look like under ideal conditions without defects, that is, without abnormal defects, can be reconstructed. Then, the original image to be detected can be compared with the normal image under ideal conditions reconstructed by the defect detection model. When the comparison result indicates that the gap between the two is relatively small, it means that the image to be detected is relatively close to the normal image under ideal conditions, and the reason why the two are relatively close is probably that the image to be detected itself is a normal image with almost no abnormalities. Therefore, at this time, it can be determined that the image to be detected is an image without abnormalities, that is, it can be determined that the product contained in the image to be detected is a normal product without abnormalities;

[0060] Alternatively, when the comparison result indicates that the difference between the two is relatively large, it means that the image to be detected is quite different from the normal image under ideal conditions, and the reason for the large difference between the two is probably that the image to be detected itself is an abnormal image with many defects. At this time, it can be determined that the image to be detected is an abnormal image, that is, it can be determined that the product contained in the image to be detected is an abnormal product with defects. In this way, effective identification of whether an industrial product is abnormal is achieved.

[0061] In step 103, the product sample image features corresponding to the current task after adding feature perturbations and the learnable query embedding vector corresponding to the current task can be input into the defect detection model to obtain reconstructed image features. That is, the feature map after adding feature perturbations can be input into the Transformer-based encoder-decoder reconstruction network to reconstruct the feature map.

[0062] It should be noted that the "learnable query embedding vector corresponding to the current task" can be the query embedding vector obtained by performing knowledge distillation on the trained query embedding vector corresponding to the previous task before the current task and the initialized query embedding vector of the current task. That is, in the continuous learning process, each continuous learning task can learn a separate prompt embedding as the query of the decoder. In addition, a continuous distillation prompt module can be introduced in the decoder, so that when a new task comes, the knowledge of the previous task can be transferred to the current task by weight distillation, that is, the query of the previous task can be distilled in the subsequent task.

[0063] Figure 5 Schematic diagram showing the working process of the continuous distillation prompt module according to an exemplary embodiment of the present disclosure. Figure 5 , the learnable query embedding vector of the current task can be optimized to distill the query embedding vector of the old task to obtain the knowledge of the old task. Then, the query embedding vector obtained after distillation can be input into the decoder contained in the defect detection model. In addition, the multi-head self-attention module in the Transformer can be modified to a multi-head neighbor mask self-attention module to avoid the feature reconstruction network from falling into the shortcut of directly copying the original image through the multi-head neighbor mask self-attention module, thereby greatly improving the defect detection model's ability to capture and reconstruct image features.

[0064] In this way, in the present disclosure, when a new task arrives, knowledge distillation can be performed on the trained query embedding vector corresponding to the old task before the new task, which can ensure that the query embedding vector of the new task is as close as possible to the query embedding vector of the old task, that is, it can prevent the query of the old task from deviating during the decoding process, thereby improving the anti-forgetting ability of the defect detection model.

[0065] According to an exemplary embodiment of the present disclosure, the above-mentioned defect detection model may include a feature reconstruction network, which may include an encoder and a decoder. The product sample image features corresponding to the current task after adding feature perturbation (perturbation) can be input into the encoder to obtain the encoding result, and then the encoding result and the learnable query embedding vector (query prompt embedding) corresponding to the current task can be input into the decoder to obtain the reconstructed image features (reconstruction token).

[0066] Figure 6 is a schematic diagram showing the structure of a feature reconstruction network according to an exemplary embodiment of the present disclosure. Figure 6 , the encoder can encode the feature map with added feature perturbations into the latent space, and can modify the multi-head self-attention module of each layer in the encoder into a multi-head neighbor masked self-attention module. Specifically, the encoder can include a multi-head neighbor masked self-attention module and a feed forward net. After adding feature perturbations (perturbation) to the product sample image features (feature tokens) corresponding to the current task, the product sample image features corresponding to the current task with added feature perturbations can be input into the multi-head neighbor masked self-attention module contained in the encoder. At this time, any one of K, V, and Q input into the multi-head neighbor masked self-attention module can be the product sample image features with added feature perturbations. Then, the multi-head neighbor masked self-attention module can process K, V, and Q by the following formula:

[0067]

[0068] Among them, D(K) is the channel dimension of K.

[0069] Next, residual processing can be performed on the output result of the multi-head neighbor mask self-attention module and the product sample image features with the feature perturbation added above to obtain the residual processing result. Then, the residual processing result can be input into the feedforward network to obtain the output result of the feedforward network. Next, residual processing can be performed again on the output result of the feedforward network and the residual processing result above to obtain the final output result of the encoder, that is, the encoding result.

[0070] According to an exemplary embodiment of the present disclosure, referring to Figure 6, the decoder can modify the multi-head attention module into a multi-head neighbor masked attention module. Exemplarily, the decoder can include two multi-head neighbor masked self-attention modules and a feedforward network. In addition, the key (K) and value (V) input to the first multi-head neighbor masked self-attention module can come from the output of the encoder; the query of the first multi-head neighbor masked self-attention module can use a learnable prompt embedding vector (query prompt embedding), and the learnable prompt embedding vector can be a query embedding vector obtained by performing knowledge distillation on the trained query embedding vector corresponding to the previous task before the current task and the initialized query embedding vector of the current task.

[0071] According to an exemplary embodiment of the present disclosure, the key (K) and value (V) of the second multi-headed neighbor mask self-attention module in the input decoder can come from the output of the previous decoder (output of lastlayer) cascaded with the current decoder, and the query of the second multi-headed neighbor mask attention module of the input decoder can come from the output of the first multi-headed neighbor mask attention module.

[0072] Specifically, the first multi-head neighbor masked attention module can process K, V, and Q to obtain the output result of the first multi-head neighbor masked attention module. Then, residual processing can be performed on the output result of the first multi-head neighbor masked attention module and the learnable query embedding vector input into the first multi-head neighbor masked attention module to obtain a first residual processing result.

[0073] Next, K, V contained in the output result of the previous decoder cascaded with the current decoder and the above-mentioned first residual processing result can also be used as the input of the second multi-head neighbor mask attention module, and then the output result of the second multi-head neighbor mask attention module can be obtained.

[0074] Then, the output result of the second multi-head neighbor mask attention module and the first residual processing result can be subjected to residual processing again to obtain a second residual processing result. Next, the second residual processing result can be input into the feedforward network to obtain the output result of the feedforward network. Then, the output result of the feedforward network and the second residual processing result can be subjected to residual processing again to obtain the final reconstructed image features.

[0075] According to an exemplary embodiment of the present disclosure, the encoder may include a plurality of cascaded encoders, and the decoder may include a plurality of cascaded decoders. Exemplarily, the number of cascades may be, but is not limited to, 4.

[0076] It should be noted that, in the case of multiple decoders being cascaded, each of the multiple decoders may include two multi-head neighbor mask self-attention modules and a feedforward network. Moreover, the inputs of the first multi-head neighbor mask self-attention module included in each decoder: K and V, may come from the encoding result of the last encoder in the cascaded multiple encoders; in addition, the inputs of the second multi-head neighbor mask self-attention module included in the first decoder: K and V, may also come from the encoding result of the last encoder in the cascaded multiple encoders; and for the second multi-head neighbor mask self-attention module included in each decoder from the second decoder to the Nth decoder in the cascaded multiple decoders, the inputs of the second multi-head neighbor mask self-attention module: K and V may come from the output result of the previous decoder cascaded with the current decoder.

[0077] In step 104, the loss may be calculated based on the reconstructed image features and the product sample image features. Exemplarily, the mean squared loss error between the input feature map and the reconstructed feature map may be calculated.

[0078] In step 105, the defect detection model may be trained by adjusting the parameters of the defect detection model based on the loss. Specifically, the parameters of the defect detection model may be updated using a back propagation algorithm to minimize the mean square loss error.

[0079] In this way, by continuously adjusting the parameters of the defect detection model based on the loss, the defect detection model can be helped to better reconstruct image features, that is, the feature reconstruction capability and overall performance of the defect detection model can be improved.

[0080] It should be noted that after the defect detection model is trained, the performance of the defect detection model can also be tested. Specifically, the sample image to be tested can be input into the trained defect detection model, and then it can be predicted whether there are defects in the sample image to be tested and the specific location of the defects. Then, the test results of the defect detection model can be compared with the true value labels corresponding to the sample to be tested. Moreover, the more similar the test results are to the true value labels, the better the performance of the defect detection model. In addition, the defect detection model can be further optimized based on the test results.

[0081] In the present disclosure, an efficient unsupervised industrial visual defect detection method based on continuous learning is provided. The performance and anti-forgetting ability of the defect detection model can be greatly improved by adding continuous prompt distillation in the feature reconstruction network; the recognition and positioning capabilities of the defect detection model can be further improved by replaying the features through coreset sampling; the addition of feature perturbations enhances the reconstruction capability of the defect detection model for positive sample images; and the use of the neighbor mask attention mechanism in the feature reconstruction network based on the Transformer encoder-decoder effectively prevents the feature reconstruction network from falling into the "copy shortcut", greatly improving the feature capture and reconstruction capabilities of the defect detection model. The defect detection method provided by the present disclosure can effectively improve the accuracy and efficiency of industrial visual defect detection, reduce the errors and costs of manual defect detection, and improve the automation and intelligence levels of industrial product production lines.

[0082] Figure 7 is a flowchart illustrating a defect detection method according to an exemplary embodiment of the present disclosure.

[0083] Reference Figure 7 In step 701, the target image features of the image containing the product to be inspected can be obtained. Specifically, a pre-trained convolutional neural network can be used to extract image features of the image of the product to be inspected, and then the target image features can be obtained, that is, the feature map after multi-scale feature fusion can be obtained. Exemplarily, the EfficientNet_b4 model pre-trained on the large-scale public dataset ImageNet can be used to extract features of the image of the product to be inspected, and then the final target image features can be obtained through multi-scale feature fusion processing.

[0084] Thus, in the present disclosure, since the EfficientNet model has a powerful feature extraction capability, the multi-scale feature extraction function of the EfficientNet model can be used to capture the detail information and global information in the image of the product to be inspected, and the richness and accuracy of the feature expression can be improved through multi-scale feature fusion processing.

[0085] In step 702, the target image features and the query embedding vector may be input into a defect detection model trained according to the training method of the present disclosure to obtain target reconstructed image features, wherein the query embedding vector may be a query embedding vector obtained by updating an initialized query embedding vector for a preset task during the training process of the defect detection model.

[0086] It should be noted that, as described in the previous embodiment, multiple continuous learning tasks can be pre-set, and each continuous learning task can correspond to a learnable query embedding vector. In addition, when a new task comes, the knowledge of the previous task can be transferred to the current task by weight distillation, that is, the query of the previous task can be distilled in the subsequent task. Exemplarily, assuming that a total of 5 continuous learning tasks are set, then for the first continuous learning task, it can correspond to an initialized query embedding vector, which can contain multiple randomly generated digital elements. In the process of training the first continuous learning task, the initialized query embedding vector corresponding to the first continuous learning task can be continuously updated, and finally the trained query embedding vector corresponding to the first continuous learning task can be obtained.

[0087] Next, for the second continuous learning task, it can also correspond to an initialized query embedding vector, and the trained query embedding vector corresponding to the first continuous learning task before the second continuous learning task can be knowledge distilled with the initialized query embedding vector of the second continuous learning task. Then, the query embedding vector obtained by knowledge distillation can be used as the input of the first multi-head neighbor mask self-attention module of the decoder included in the feature reconstruction network, so as to train the second continuous learning task. Moreover, in the process of training the second continuous learning task, the learnable query embedding vector corresponding to the second continuous learning task can be continuously updated, and finally the trained query embedding vector corresponding to the second continuous learning task can be obtained.

[0088] Similarly, for the third continuous learning task, it can also correspond to an initialized query embedding vector, and the trained query embedding vector corresponding to the second continuous learning task before the third continuous learning task can be knowledge distilled with the initialized query embedding vector of the third continuous learning task. Then, the query embedding vector obtained by knowledge distillation can be used as the input of the first multi-head neighbor mask self-attention module in the decoder included in the feature reconstruction network, so as to train the third continuous learning task. Moreover, in the process of training the third continuous learning task, the learnable query embedding vector corresponding to the third continuous learning task can be continuously updated, and finally the trained query embedding vector corresponding to the third continuous learning task can be obtained.

[0089] By analogy, after completing the training for the above five continuous learning tasks, the final fixed query embedding vector can be obtained, which can be used as the query input of the first multi-head neighbor mask self-attention module in the decoder contained in the feature reconstruction network of the trained defect detection model when reconstructing the target image features of the image of the product to be inspected.

[0090] In this way, during the training process of the defect detection model, when a new task arrives, knowledge distillation can be performed on the trained query embedding vector corresponding to the old task before the new task, so as to ensure that the query embedding vector of the new task is as close as possible to the query embedding vector of the old task, that is, the query of the old task will not deviate during the decoding process, thereby improving the anti-forgetting ability of the defect detection model.

[0091] In step 703, it can be determined whether the product to be inspected has defects based on the target reconstructed image features and the target image features. That is, it can be determined whether the product to be inspected has defects based on the normal image features of the image of the product to be inspected in an ideal state reconstructed by the trained feature reconstruction network and the original image features of the image of the product to be inspected.

[0092] According to an exemplary embodiment of the present disclosure, a target mean square loss error between target reconstructed image features and target image features may be calculated.

[0093] When the target mean square loss error is less than or equal to the preset threshold, it means that the image of the product to be detected is relatively close to the normal image under ideal conditions, and the reason why the two are relatively close is probably that the image of the product to be detected itself is a normal image with almost no abnormalities. Therefore, at this time, it can be determined that the image of the product to be detected is a normal image, that is, it can be determined that the product to be detected contained in the image of the product to be detected has no abnormalities, and that the product to be detected is a normal product.

[0094] Otherwise, that is, when the target mean square loss error is greater than the preset threshold, it means that the image of the product to be detected is quite different from the normal image under ideal conditions, and the reason for the large difference between the two is probably that the image of the product to be detected itself is an abnormal image with many defects. At this time, it can be determined that the image of the product to be detected is an abnormal image, that is, it can be determined that the product to be detected contained in the image of the product to be detected is abnormal, and that the product to be detected is an unqualified and inferior product. In this way, effective identification of whether there is an abnormality in industrial products is achieved.

[0095] In the present disclosure, an efficient unsupervised industrial visual defect detection method based on continuous learning is provided. The performance and anti-forgetting ability of the defect detection model can be greatly improved by adding continuous prompt distillation in the feature reconstruction network; the recognition and positioning capabilities of the defect detection model can be further improved by replaying the features through coreset sampling; the addition of feature perturbations enhances the reconstruction capability of the defect detection model for positive sample images; and the use of the neighbor mask attention mechanism in the feature reconstruction network based on the Transformer encoder-decoder effectively prevents the feature reconstruction network from falling into the "copy shortcut", greatly improving the feature capture and reconstruction capabilities of the defect detection model. The defect detection method provided by the present disclosure can effectively improve the accuracy and efficiency of industrial visual defect detection, reduce the errors and costs of manual defect detection, and improve the automation and intelligence levels of industrial product production lines.

[0096] Figure 8 is a block diagram illustrating a training apparatus for a defect detection model according to an exemplary embodiment of the present disclosure.

[0097] Reference Figure 8 The training device 800 of the defect detection model may include an image feature acquisition module 801, a feature disturbance adding module 802, a reconstruction module 803, a loss calculation module 804 and a parameter adjustment module 805.

[0098] The image feature acquisition module 801 can acquire the product sample image features corresponding to the current task, wherein the current task can be one of multiple tasks executed in sequence, each of the multiple tasks can correspond to multiple types of product sample images, and the above-mentioned product sample image features can be image features extracted from multiple types of product sample images. Specifically, normal images containing various types of industrial objects can be acquired as training sample images, and these training sample images can be randomly shuffled. Then, these training sample images can be respectively classified into different continuous learning tasks according to the number of tasks to meet the requirements of continuous learning.

[0099] According to an exemplary embodiment of the present disclosure, the image feature acquisition module 801 can perform feature extraction on multiple types of product sample images corresponding to the current task to obtain the current product sample image features corresponding to the current task. Then, some product sample image features can be randomly selected from the multiple product sample image features corresponding to each task in at least one task before the current task. Next, the current product sample image features and some product sample image features randomly selected from the multiple product sample image features corresponding to the previous task can be determined as the product sample image features corresponding to the current task, that is, the product sample image features corresponding to the current task itself and some product sample image features randomly selected from the multiple product sample image features corresponding to the previous task itself can be used together to train for the current task.

[0100] Specifically, a representative feature subset can be selected as the coreset from the product sample image feature pool corresponding to each previous task of the current task. For example, a greedy selection method can be used to select a representative feature subset as the coreset from the product sample image feature pool corresponding to each previous task of the current task. Then, during the training process for the current task, the above coreset can be replayed to retain the important feature information representative of the previous task of the current task, that is, the old task, so as to improve the recognition and positioning capabilities of the defect detection model. That is, through this feature playback method, the defect detection model can retain the memory of the old task when training a new task, thereby improving its overall performance.

[0101] In the present disclosure, a quasi-incremental learning method can be adopted, and the memory ability of the defect detection model for old tasks can be improved through feature playback. That is, the coreset downsampling method can be used for feature selection, and then a representative feature subset can be selected from the feature pool corresponding to the previous task of the current task, that is, a more representative feature map can be obtained from the feature distribution corresponding to the previous task of the current task. In this way, in the process of training a new task, the more representative feature map corresponding to the previous task obtained by the coreset downsampling method can be used for playback, and then the representative important feature information in the old task can be retained, thereby improving the anti-forgetting ability and recognition and positioning capabilities of the defect detection model.

[0102] The feature disturbance adding module 802 can add feature disturbance to the product sample image features corresponding to the current task.

[0103] Thus, in the present disclosure, by adding feature perturbations, i.e., noise perturbations, to the product sample image features, i.e., feature maps, the model can better learn and adapt to the noise in the image, i.e., the addition of feature perturbations can enhance the defect detection model's ability to reconstruct positive sample images, i.e., it can enhance the defect detection model's ability to reconstruct sample images of products without abnormalities, thereby enhancing the robustness and generalization ability of the defect detection model, so that it can still maintain a good normal image, i.e., the ability to reconstruct defect-free images, when facing various image noises.

[0104] The reconstruction module 803 can input the product sample image features corresponding to the current task after adding feature perturbations and the learnable query embedding vector corresponding to the current task into the defect detection model to obtain reconstructed image features. That is, the feature map after adding feature perturbations can be input into the Transformer-based encoder-decoder reconstruction network to reconstruct the feature map.

[0105] It should be noted that the "learnable query embedding vector corresponding to the current task" can be the query embedding vector obtained by performing knowledge distillation on the trained query embedding vector corresponding to the previous task before the current task and the initialized query embedding vector of the current task. That is, in the continuous learning process, each continuous learning task can learn a separate prompt embedding as the query of the decoder. In addition, a continuous distillation prompt module can be introduced in the decoder, so that when a new task comes, the knowledge of the previous task can be transferred to the current task by weight distillation, that is, the query of the previous task can be distilled in the subsequent task.

[0106] In this way, in the present disclosure, when a new task arrives, knowledge distillation can be performed on the trained query embedding vector corresponding to the old task before the new task, which can ensure that the query embedding vector of the new task is as close as possible to the query embedding vector of the old task, that is, it can prevent the query of the old task from deviating during the decoding process, thereby improving the anti-forgetting ability of the defect detection model.

[0107] According to an exemplary embodiment of the present disclosure, the above-mentioned defect detection model may include a feature reconstruction network, which may include an encoder and a decoder. The reconstruction module 803 may input the product sample image features corresponding to the current task after adding feature perturbations to the encoder to obtain the encoding result, and then the reconstruction module 803 may input the encoding result and the learnable query embedding vector corresponding to the current task into the decoder to obtain the reconstructed image features.

[0108] According to an exemplary embodiment of the present disclosure, the encoder may include a plurality of cascaded encoders, and the decoder may include a plurality of cascaded decoders. Exemplarily, the number of cascades may be, but is not limited to, 4.

[0109] The loss calculation module 804 may calculate the loss based on the reconstructed image features and the product sample image features. For example, the mean squared loss error between the input feature map and the reconstructed feature map may be calculated.

[0110] The parameter adjustment module 805 can train the defect detection model by adjusting the parameters of the defect detection model based on the loss. Specifically, the back propagation algorithm can be used to update the parameters of the defect detection model to minimize the mean square loss error.

[0111] In this way, by continuously adjusting the parameters of the defect detection model based on the loss, the defect detection model can be helped to better reconstruct image features, that is, the feature reconstruction capability and overall performance of the defect detection model can be improved.

[0112] In the present disclosure, an efficient unsupervised industrial visual defect detection method based on continuous learning is provided. The performance and anti-forgetting ability of the defect detection model can be greatly improved by adding continuous prompt distillation in the feature reconstruction network; the recognition and positioning capabilities of the defect detection model can be further improved by replaying the features through coreset sampling; the addition of feature perturbations enhances the reconstruction capability of the defect detection model for positive sample images; and the use of the neighbor mask attention mechanism in the feature reconstruction network based on the Transformer encoder-decoder effectively prevents the feature reconstruction network from falling into the "copy shortcut", greatly improving the feature capture and reconstruction capabilities of the defect detection model. The defect detection method provided by the present disclosure can effectively improve the accuracy and efficiency of industrial visual defect detection, reduce the errors and costs of manual defect detection, and improve the automation and intelligence levels of industrial product production lines.

[0113] Fig. 9 is a block diagram illustrating a defect detection apparatus according to an exemplary embodiment of the present disclosure.

[0114] Reference Fig. 9 The defect detection device 900 may include a to-be-detected image feature acquisition module 901 , a reconstructed feature acquisition module 902 , and a defect determination module 903 .

[0115] The image feature acquisition module 901 to be detected can obtain the target image features of the image containing the product to be detected. Specifically, a pre-trained convolutional neural network can be used to extract image features of the image of the product to be detected, and then the target image features can be obtained, that is, the feature map after multi-scale feature fusion can be obtained. Exemplarily, the EfficientNet_b4 model pre-trained on the large-scale public dataset ImageNet can be used to extract features of the image of the product to be detected, and then the final target image features can be obtained through multi-scale feature fusion processing.

[0116] Thus, in the present disclosure, since the EfficientNet model has a powerful feature extraction capability, the multi-scale feature extraction function of the EfficientNet model can be used to capture the detail information and global information in the image of the product to be inspected, and the richness and accuracy of the feature expression can be improved through multi-scale feature fusion processing.

[0117] The reconstruction feature acquisition module 902 can input the target image features and the query embedding vector into the unsupervised defect detection model based on continuous learning trained according to the training method of the present invention to obtain the target reconstructed image features, wherein the query embedding vector can be a query embedding vector obtained by updating the initialized query embedding vector for a preset task during the training process of the defect detection model.

[0118] In this way, during the training process of the defect detection model, when a new task arrives, knowledge distillation can be performed on the trained query embedding vector corresponding to the old task before the new task, so as to ensure that the query embedding vector of the new task is as close as possible to the query embedding vector of the old task, that is, the query of the old task will not deviate during the decoding process, thereby improving the anti-forgetting ability of the defect detection model.

[0119] The defect determination module 903 can determine whether the product to be inspected has defects based on the target reconstructed image features and the target image features. That is, it can determine whether the product to be inspected has defects based on the normal image features of the image of the product to be inspected in an ideal state reconstructed by the trained feature reconstruction network and the original image features of the image of the product to be inspected.

[0120] According to an exemplary embodiment of the present disclosure, the defect determination module 903 may calculate the target reconstructed image features and the target mean square loss error between the target image features.

[0121] When the target mean square loss error is less than or equal to the preset threshold, it means that the image of the product to be detected is relatively close to the normal image under ideal conditions, and the reason why the two are relatively close is probably that the image of the product to be detected itself is a normal image with almost no abnormalities. Therefore, at this time, it can be determined that the image of the product to be detected is a normal image, that is, it can be determined that the product to be detected contained in the image of the product to be detected has no abnormalities, and that the product to be detected is a normal product.

[0122] Otherwise, that is, when the target mean square loss error is greater than the preset threshold, it means that the image of the product to be detected is quite different from the normal image under ideal conditions, and the reason for the large difference between the two is probably that the image of the product to be detected itself is an abnormal image with many defects. At this time, it can be determined that the image of the product to be detected is an abnormal image, that is, it can be determined that the product to be detected contained in the image of the product to be detected is abnormal, and that the product to be detected is an unqualified and inferior product. In this way, effective identification of whether there is an abnormality in industrial products is achieved.

[0123] In the present disclosure, an efficient unsupervised industrial visual defect detection method based on continuous learning is provided. The performance and anti-forgetting ability of the defect detection model can be greatly improved by adding continuous prompt distillation in the feature reconstruction network; the recognition and positioning capabilities of the defect detection model can be further improved by replaying the features through coreset sampling; the addition of feature perturbations enhances the reconstruction capability of the defect detection model for positive sample images; and the use of the neighbor mask attention mechanism in the feature reconstruction network based on the Transformer encoder-decoder effectively prevents the feature reconstruction network from falling into the "copy shortcut", greatly improving the feature capture and reconstruction capabilities of the defect detection model. The defect detection method provided by the present disclosure can effectively improve the accuracy and efficiency of industrial visual defect detection, reduce the errors and costs of manual defect detection, and improve the automation and intelligence levels of industrial product production lines.

[0124] Fig.10 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure.

[0125] Reference Fig.10 The electronic device 1000 includes at least one memory 1001 and at least one processor 1002, wherein the at least one memory 1001 stores instructions, and when the instructions are executed by the at least one processor 1002, a training method for an unsupervised defect detection model based on continuous learning or a defect detection method according to an exemplary embodiment of the present disclosure is executed.

[0126] As an example, the electronic device 1000 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instructions. Here, the electronic device 1000 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instructions (or instruction sets) individually or in combination. The electronic device 1000 may also be part of an integrated control system or a system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0127] In the electronic device 1000, the processor 1002 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0128] The processor 1002 may execute instructions or codes stored in the memory 1001, wherein the memory 1001 may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0129] The memory 1001 may be integrated with the processor 1002, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 1001 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 1001 and the processor 1002 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 1002 can read files stored in the memory.

[0130] In addition, the electronic device 1000 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 1000 may be connected to each other via a bus and / or a network.

[0131] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, and when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the above-mentioned training method of an unsupervised defect detection model based on continuous learning or a defect detection method. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0132] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, including a computer program, which, when executed by a processor, implements the training method of the unsupervised defect detection model based on continuous learning or the defect detection method according to the present disclosure.

[0133] According to the training method, defect detection method, device, electronic device, storage medium and computer program product of the unsupervised defect detection model based on continuous learning disclosed in the present invention, a learnable query embedding vector can be set for each of the multiple tasks, and when a new task comes, the previous task whose execution order is before the current task, that is, the knowledge of the previous task, can be transferred to the current task through knowledge distillation. In this way, by introducing the prompt signal of the historical task, the query embedding vectors of the new and old tasks can be made as close as possible, that is, it can be ensured that the query of the old task will not deviate during the decoding process, which can help the defect detection model to maintain the memory of the old task while training the new task, thereby improving the defect detection model's anti-forgetting and generalization ability on new tasks.

[0134] According to an exemplary embodiment of the present disclosure, a quasi-incremental learning method can be adopted, and the memory ability of the defect detection model for old tasks can be improved through feature playback. That is, the coreset downsampling method can be used for feature selection, and then a representative feature subset can be selected from the feature pool corresponding to the previous task of the current task, that is, a more representative feature map can be obtained from the feature distribution corresponding to the previous task of the current task. In this way, in the process of training a new task, the more representative feature map corresponding to the previous task obtained by the coreset downsampling method can be used for playback, and then the representative important feature information in the old task can be retained, thereby improving the anti-forgetting ability and the recognition and positioning ability of the defect detection model.

[0135] According to an exemplary embodiment of the present disclosure, by adding feature perturbations, i.e., noise perturbations, to product sample image features, i.e., feature maps, the model can better learn and adapt to the noise in the image, i.e., the addition of feature perturbations can enhance the defect detection model's ability to reconstruct positive sample images, i.e., it can enhance the defect detection model's ability to reconstruct sample images of products without abnormalities, thereby enhancing the robustness and generalization ability of the defect detection model, so that it can still maintain a good normal image, i.e., the ability to reconstruct defect-free images, when facing various image noises.

[0136] According to an exemplary embodiment of the present disclosure, when a new task arrives, knowledge distillation can be performed on the trained query embedding vector corresponding to the old task before the new task, which can ensure that the query embedding vector of the new task is as close as possible to the query embedding vector of the old task, that is, the query of the old task will not deviate during the decoding process, thereby improving the anti-forgetting ability of the defect detection model.

[0137] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0138] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A training method for an unsupervised defect detection model based on continuous learning, characterized in that: include: Acquire a product sample image feature corresponding to a current task, wherein the current task is one of a plurality of tasks to be performed in sequence, each of the plurality of tasks corresponds to a plurality of types of product sample images, and the product sample image feature is an image feature extracted from the plurality of types of product sample images; Adding feature disturbance to the product sample image features corresponding to the current task; Inputting the product sample image features corresponding to the current task after adding feature perturbations and the learnable query embedding vector corresponding to the current task into the defect detection model to obtain reconstructed image features, wherein the learnable query embedding vector corresponding to the current task is a query embedding vector obtained by performing knowledge distillation on the trained query embedding vector corresponding to the previous task before the current task and the initialization query embedding vector of the current task; Calculating a loss based on the reconstructed image features and the product sample image features; The defect detection model is trained by adjusting parameters of the defect detection model based on the loss.

2. The training method according to claim 1, characterized in that: The obtaining of product sample image features corresponding to the current task includes: Extracting features of multiple types of product sample images corresponding to the current task to obtain features of the current product sample images corresponding to the current task; Randomly select part of the product sample image features from a plurality of product sample image features corresponding to each task in at least one task before the current task; The current product sample image features and the randomly selected part of the product sample image features are determined as the product sample image features corresponding to the current task.

3. The training method according to claim 1, characterized in that: The defect detection model includes a feature reconstruction network, and the feature reconstruction network includes an encoder and a decoder; The step of inputting the product sample image features corresponding to the current task after adding feature disturbance and the learnable query embedding vector corresponding to the current task into the defect detection model to obtain the reconstructed image features includes: Inputting the product sample image features corresponding to the current task after adding feature disturbance into the encoder to obtain an encoding result; The encoding result and the learnable query embedding vector corresponding to the current task are input into the decoder to obtain the reconstructed image features.

4. The training method according to claim 3, characterized in that: The encoder includes a plurality of cascaded encoders, and the decoder includes a plurality of cascaded decoders.

5. A defect detection method, characterized in that: include: Acquire target image features of an image containing a product to be inspected; Inputting the target image feature and the query embedding vector into a defect detection model trained according to any one of the training methods of claims 1 to 4 to obtain a target reconstructed image feature, wherein the query embedding vector is a query embedding vector obtained by updating an initialization query embedding vector for a preset task during the training process of the defect detection model; Based on the target reconstructed image features and the target image features, it is determined whether the product to be inspected has defects.

6. The defect detection method according to claim 5, characterized in that: The determining whether the product to be inspected has a defect based on the target reconstructed image feature and the target image feature includes: Calculating a target mean square loss error between the target reconstructed image features and the target image features; When the target mean square loss error is less than or equal to a preset threshold, determining that there is no abnormality in the product to be detected; Otherwise, it is determined that the product to be detected has an abnormality.

7. A training device for a defect detection model, characterized in that: include: An image feature acquisition module is configured to acquire a product sample image feature corresponding to a current task, wherein the current task is one of a plurality of tasks to be performed in sequence, each of the plurality of tasks corresponds to a plurality of types of product sample images, and the product sample image feature is an image feature extracted from the plurality of types of product sample images; A feature disturbance adding module, configured to add feature disturbance to the product sample image features corresponding to the current task; A reconstruction module is configured to input the product sample image features corresponding to the current task after adding feature perturbations and the learnable query embedding vector corresponding to the current task into an unsupervised defect detection model based on continuous learning to obtain reconstructed image features, wherein the learnable query embedding vector corresponding to the current task is a query embedding vector obtained by performing knowledge distillation on a trained query embedding vector corresponding to a previous task before the current task and an initialized query embedding vector of the current task; a loss calculation module, configured to calculate the loss based on the reconstructed image features and the product sample image features; A parameter adjustment module is configured to train the defect detection model by adjusting the parameters of the defect detection model based on the loss.

8. A defect detection device, characterized in that: include: A module for acquiring features of an image to be detected, configured to acquire features of a target image of an image containing a product to be detected; A reconstruction feature acquisition module, configured to input the target image feature and the query embedding vector into a defect detection model trained according to any one of the training methods of claims 1 to 4, to obtain a target reconstructed image feature, wherein the query embedding vector is a query embedding vector obtained by updating an initialization query embedding vector for a preset task during the training process of the defect detection model; The defect determination module is configured to determine whether the product to be inspected has defects based on the target reconstructed image features and the target image features.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the training method of an unsupervised defect detection model based on continuous learning as described in any one of claims 1 to 4, or to implement the defect detection method as described in any one of claims 5 to 6.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of an unsupervised defect detection model based on continuous learning as described in any one of claims 1 to 4, or to execute the defect detection method as described in any one of claims 5 to 6.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the training method of an unsupervised defect detection model based on continuous learning as described in any one of claims 1 to 4 is implemented, or the defect detection method as described in any one of claims 5 to 6 is implemented.

Images