Denoising-based image unsupervised anomaly detection method and related device

Abstract

The invention discloses an image unsupervised anomaly detection method based on denoising and a related device, and the method comprises the steps: constructing an image reconstruction network model which comprises a feature embedding module, an input mask module and a plurality of Mask Transform Block modules connected in series; constructing a training data set, wherein training images in the training data set are normal images to which composite noise is applied; sequentially inputting the training images in the training data set into the image reconstruction network model for unsupervised training processing until a convergent image reconstruction network model is obtained; and obtaining a to-be-detected image, and inputting the to-be-detected image into the convergent image reconstruction network model for unsupervised anomaly detection processing to obtain an anomaly detection result of the to-be-detected image. In the embodiment of the invention, unsupervised training is realized in the training stage, and no defect sample is needed; and high-precision pixel-level positioning can be carried out in a detection stage.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an unsupervised image anomaly detection method and related apparatus based on denoising. Background Technology

[0002] As the core of next-generation display technology, MiniLED is being widely integrated into high-end TVs, professional monitors, and smart mobile devices due to its significant advantages in high brightness, high contrast, and low power consumption. A typical MiniLED panel consists of tens of thousands of tiny RGB LED units arranged in a high-density array. This precise structure places extremely high demands on the manufacturing process. Any tiny manufacturing defect, such as foreign matter adhesion, abnormal chip color, missing or misaligned units, can seriously affect the display quality and reliability of the final product.

[0003] Currently, defect detection in MiniLEDs faces two major challenges. First, traditional manual visual inspection methods are extremely inefficient and costly due to the tiny size (typically between 100-300 micrometers) and large number of MiniLED units. Furthermore, they are prone to missed or false detections due to human eye fatigue and subjective judgment, making them unsuitable for large-scale industrial production. While researchers have begun applying deep learning methods to improve automated inspection, mainstream supervised learning methods suffer from a fundamental bottleneck: they rely on large-scale, diverse labeled defect samples for training. However, in actual MiniLED production, thanks to mature process control, product yields are extremely high, making defect sample collection itself extremely difficult. Even more challenging is the fact that the shape, size, color, and location of defects exhibit significant randomness and unpredictability, making it nearly impossible to completely cover all potential abnormal patterns with a limited sample set. This results in supervised learning models exhibiting poor generalization ability when faced with novel or rare defects not present in the training set.

[0004] To address this challenge, unsupervised anomaly detection methods that only require training on normal samples have become a more suitable technical approach for this scenario. Existing unsupervised methods can be broadly categorized into several types: 1. Memory-based methods: These methods construct a "memory" that stores local features of normal samples. During testing, anomalies are determined by comparing the distance between the query feature and features in the memory. While these methods achieve high accuracy in certain scenarios, they typically require nearest-neighbor searches across the entire memory, resulting in significant computational overhead during inference and consuming substantial memory to store the feature library. 2. Teacher-Student based methods: These methods utilize a pre-trained, fixed "teacher" network on a large dataset to guide a "student" network in learning feature representations of normal samples. Since the model has only seen normal samples, its response to anomalous inputs may be biased. However, this method may lack sensitivity to complex texture anomalies, and its effectiveness is largely limited by the knowledge scope of the pre-trained model. 3. Reconstruction-based methods: These methods (such as autoencoders) train a network to reconstruct the input normal image, assuming that the network cannot reconstruct anomalous regions it has never seen before. While this approach is intuitive and effective, traditional reconstruction models are prone to the "identity mapping" problem during training. This means that the model may learn to "perfectly" reconstruct abnormal regions as well, resulting in the abnormalities not being obvious in the residual map, thus reducing detection performance. At the same time, its training process often faces the challenge of instability.

[0005] In summary, existing unsupervised anomaly detection techniques still struggle to achieve an ideal balance between detection generalization, training stability, and sensitivity to small and complex anomalies when applied to manufacturing scenarios such as MiniLED. Therefore, there is an urgent need for a novel technical solution that can fundamentally improve the model's ability to generalize and detect unknown anomalies. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides an image unsupervised anomaly detection method and related device based on denoising, which has excellent generalization ability and unknown anomaly detection ability. It achieves unsupervised training in the training phase and does not require any defect samples; in the detection phase, it can achieve high-precision pixel-level positioning.

[0007] To address the aforementioned technical problems, embodiments of the present invention provide an unsupervised image anomaly detection method based on denoising, the method comprising: An image reconstruction network model is constructed, which consists of a feature embedding module, an input mask module, and several cascaded Mask Transformer Block modules; Construct a training dataset, wherein the training images in the training dataset are normal images with compound noise applied; The training images in the training dataset are sequentially input into the image reconstruction network model for unsupervised training until a converged image reconstruction network model is obtained. The image to be detected is acquired and input into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain the anomaly detection result of the image to be detected.

[0008] Optionally, the input mask module is used to perform high-proportion random occlusion processing on noisy features in the input image at the input end of the image reconstruction network model; The Mask Transformer Block module integrates an attention masking mechanism, which is used to simulate incomplete information during the computation process within the image reconstruction network model, ensuring that the image reconstruction network model can adapt to inputs with different information densities.

[0009] Optionally, the training images in the training dataset are normal images with compound noise applied, including: Several normal images are obtained, and several noise types are randomly selected for each normal image from a predefined noise pool. The predefined noise pool includes additive Gaussian noise, salt and pepper noise, multiplicative speckle noise, and Poisson noise. For each selected noise type, a randomly determined intensity parameter is configured, and each selected noise type is sequentially superimposed onto the corresponding normal image according to the corresponding intensity parameter in a random order to form a training image. The training dataset consists of several training images.

[0010] Optionally, the step of sequentially inputting the training images from the training dataset into the image reconstruction network model for unsupervised training until a converged image reconstruction network model is obtained includes: The training images in the training dataset are sequentially input into the image reconstruction network model for training processing. During the training process, the feature embedding layer in the image reconstruction network model converts the input training images into a series of feature blocks. A series of feature blocks are input into the input mask module to perform masking processing, and the masked feature blocks are input into several cascaded Mask Transformer Block modules for image reconstruction processing, and the reconstruction result of each training image is output. The pixel-level difference calculation results are obtained by using the reconstruction results of each training image and the corresponding normal image to perform pixel-level difference calculation. Based on the pixel-level difference calculation results corresponding to the reconstruction results of each training image, the absolute value error loss function is used to optimize the parameters of each layer of the trained image reconstruction network model. After optimization, the training images in the training dataset are sequentially input into the image reconstruction network model for training processing until a converged image reconstruction network model is obtained.

[0011] Optionally, the step of inputting a series of feature blocks into the input mask module to perform masking processing, and then inputting the masked feature blocks into several cascaded Mask Transformer Block modules for image reconstruction processing, outputting the reconstruction result of each training image, including: After the input mask module receives a series of feature blocks, it traverses all feature blocks and performs mask judgment on each feature block according to the preset mask probability to form a mask judgment result for each feature block. When the masking judgment result indicates that the corresponding feature block needs to be masked, the original feature vector corresponding to the feature block that needs to be masked is replaced with the masking marker vector in the training process to form the masked feature block, and the dimension of the masking marker vector is the same as that of the original feature vector. The masked feature blocks are input into several cascaded Mask Transformer Block modules for image reconstruction, and the reconstruction result of each training image is output.

[0012] Optionally, the masked feature blocks are input into several concatenated Mask Transformer Block modules for image reconstruction processing, and the reconstruction result of each training image is output, including: The masked feature block is input into several cascaded Mask Transformer Block modules, and a self-attention computation layer is used in each Mask Transformer Block module to perform independent random masking processing on the masked feature block to obtain random masking results. Based on the random masking result, determine whether the attention connection between the corresponding feature block and several cascaded Mask TransformerBlock modules needs to be masked. When a feature block needs to be masked, the weight of the feature block to be masked in the attention matrix is ​​set to negative infinity or a minimum value, and after softmax normalization, the attention weight of the feature block to be masked is made close to zero. After several cascaded Mask Transformer Block modules process the masked feature blocks according to attention weights, the reconstruction result of each training image is output.

[0013] Optionally, the step of inputting the image to be detected into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain the anomaly detection result of the image to be detected includes: The image to be detected is input into a converged image reconstruction network model for image reconstruction processing to obtain the reconstructed image to be detected. Calculate the pixel-level absolute difference between the reconstructed image to be detected and the image to be detected, and generate a residual image based on the calculated pixel-level absolute difference; If the residual image is a nearly completely black image, then the image to be detected is determined to be a normal image, and an anomaly detection result for the image to be detected is generated; If the residual image contains bright patches, it is determined that there are abnormal regions in the image to be detected, and the abnormal regions are located based on the bright patches to generate the abnormal detection result of the image to be detected.

[0014] In addition, embodiments of the present invention also provide an image unsupervised anomaly detection device based on denoising, the device comprising: The first building module is used to build an image reconstruction network model, which consists of a feature embedding module, an input mask module, and several cascaded Mask Transformer Block modules. The second building module is used to build a training dataset, wherein the training images in the training dataset are normal images with compound noise applied. Training module: used to sequentially input the training images in the training dataset into the image reconstruction network model for unsupervised training until a converged image reconstruction network model is obtained; Anomaly detection module: used to acquire the image to be detected, and input the image to be detected into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain the anomaly detection result of the image to be detected.

[0015] In addition, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein the processor runs a computer program or code stored in the memory to implement the image unsupervised anomaly detection method as described in any of the above.

[0016] In addition, embodiments of the present invention also provide a computer-readable storage medium for storing a computer program or code, which, when executed by a processor, implements the image unsupervised anomaly detection method as described in any of the above embodiments.

[0017] In this embodiment of the invention, it has excellent generalization ability and detection capability for unknown anomalies, and achieves completely unsupervised training without any defect samples. The training process does not require any form of defect samples, which perfectly solves the fundamental problem of scarce defect samples, high collection costs and inability to cover all types in industrial scenarios. High-precision pixel-level positioning, by calculating the pixel-level residual map between the input image and the model reconstructed image, can accurately delineate the contour and location of anomalies at high resolution, providing accurate visual basis for subsequent defect classification or equipment maintenance. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the image unsupervised anomaly detection method based on denoising in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structural composition of the image unsupervised anomaly detection device based on denoising in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structural composition of the image reconstruction network model in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of the Mask Transformer Block module in an embodiment of the present invention; Figure 6 This is a comparison chart of the detection results of two MimiED abnormal samples in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1, please refer to Figure 1 , Figure 1 This is a flowchart illustrating the image unsupervised anomaly detection method based on denoising in an embodiment of the present invention.

[0022] like Figure 1 As shown, an unsupervised image anomaly detection method based on denoising is described, the method comprising: S101: Construct an image reconstruction network model, which consists of a feature embedding module, an input mask module, and several cascaded Mask Transformer Block modules; In the specific implementation of this invention, the input masking module is used to perform high-proportion random occlusion processing on the noisy features in the input image at the input end of the image reconstruction network model; the Mask Transformer Block module integrates an attention masking mechanism, which is used to simulate the situation of incomplete information in the internal calculation of the image reconstruction network model, so as to ensure that the image reconstruction network model can adapt to inputs with different information densities.

[0023] Specifically, the first step is to build an image reconstruction network model, such as... Figure 4 As shown, the system comprises a feature embedding module, an input masking module, and several cascaded Mask Transformer Block modules. The feature embedding module converts the input image into image embedding features. The input end of the image reconstruction network model in the feature embedding module performs high-proportion random occlusion processing on the noisy features in the input image. The Mask Transformer Block module integrates an attention masking mechanism, which is used to simulate incomplete information in the computation process within the image reconstruction network model, ensuring that the image reconstruction network model can adapt to inputs with different information densities.

[0024] S102: Construct a training dataset, wherein the training images in the training dataset are normal images with compound noise applied; In the specific implementation of this invention, the training images in the training dataset are normal images with compound noise applied, including: obtaining several normal images, and randomly selecting several noise types for each normal image from a predefined noise pool, wherein the predefined noise pool includes additive Gaussian noise, salt-and-pepper noise, multiplicative speckle noise, and Poisson noise; configuring randomly determined intensity parameters for each selected noise type, and sequentially superimposing each selected noise type onto the corresponding normal image according to the corresponding intensity parameters in a random order to form training images, and the training dataset is composed of several training images.

[0025] Specifically, before training the image reconstruction network model, a training dataset needs to be constructed, which is a training dataset containing normal image samples. The image reconstruction network model will then be trained using the training dataset to make it more robust. The trained image reconstruction network model will be able to restore the original and clean state to the greatest extent possible, no matter how severely the input normal image is contaminated.

[0026] First, prepare a set of normal images (in this embodiment, a normal MiniLED image is used as an example) as the initial training set. Then, apply a dynamically generated noise composed of multiple basic noises to each normal image in the initial training set to generate a noisy image, thus forming the training dataset.

[0027] First, several basic noise types are randomly selected from a predefined noise pool and superimposed. This noise pool includes, but is not limited to, additive Gaussian noise, salt-and-pepper noise, multiplicative speckle noise, and Poisson noise. During the noise application process, each selected noise type is configured with a randomly determined intensity parameter. All types of noise are superimposed onto the original image in a random order to form a composite noise effect. This ensures that the noisy images generated in each training iteration have different combinations of noise features, thereby enhancing the model's robustness to diverse noise.

[0028] S103: Input the training images in the training dataset into the image reconstruction network model in sequence for unsupervised training until a converged image reconstruction network model is obtained; In the specific implementation of this invention, the step of sequentially inputting the training images from the training dataset into the image reconstruction network model for unsupervised training until a converged image reconstruction network model is obtained includes: sequentially inputting the training images from the training dataset into the image reconstruction network model for training, and during the training process, the feature embedding layer in the image reconstruction network model converts the input training images into a series of feature blocks; inputting the series of feature blocks into the input masking module to perform masking processing, and inputting the masked feature blocks into several cascaded Mask Transformer Block modules for image reconstruction processing, outputting the reconstruction result of each training image; performing pixel-level difference calculation processing using the reconstruction result of each training image and the corresponding normal image to obtain the pixel-level difference calculation result corresponding to the reconstruction result of each training image; optimizing the parameters of each layer of the trained image reconstruction network model using the absolute value error loss function based on the pixel-level difference calculation result corresponding to the reconstruction result of each training image, and returning to the step of sequentially inputting the training images from the training dataset into the image reconstruction network model for training until a converged image reconstruction network model is obtained.

[0029] Furthermore, the step of inputting a series of feature blocks into the input masking module for masking processing, and then inputting the masked feature blocks into several cascaded Mask Transformer Block modules for image reconstruction processing, outputting the reconstruction result of each training image, includes: after the input masking module receives a series of feature blocks, it iterates through all feature blocks and performs masking judgment on each feature block according to a preset masking probability to form a masking judgment result corresponding to each feature block; when the masking judgment result indicates that the corresponding feature block needs to be masked, the original feature vector corresponding to the feature block that needs to be masked is replaced with the masking marker vector from the training process to form the masked feature block, and the dimension of the masking marker vector is the same as that of the original feature vector; the masked feature block is input into several cascaded Mask Transformer Block modules for image reconstruction processing, and the reconstruction result of each training image is output.

[0030] Specifically, training an image reconstruction network model requires enabling the model to randomly occlude a high proportion of noisy features in the input image data. This allows the trained network model to move beyond simple noise pattern recognition and instead learn the inherent structure, contextual relationships, and deep semantics of normal images in order to reconstruct the image from extremely limited information.

[0031] During training, each noisy image in the training dataset is input as follows: Figure 4 In the constructed image reconstruction network model, the input noisy image is first converted into a series of feature blocks (tokens) through the feature embedding module (layer) within the image reconstruction network model. Then, the series of feature blocks are input into the input mask module (InputMask). In the input mask module, a mask probability parameter is set, and then all feature blocks in the series are traversed. Each feature block is independently masked according to the mask probability parameter. When a feature block is determined to need to be masked, its original feature vector is replaced with a mask token vector [mask-token] that can be learned during training. The dimension of the token vector is the same as that of the original feature block, and the initial value can be randomly initialized or set to zero vector. It is optimized and updated through backpropagation in subsequent training. Through this random masking mechanism, the model is forced to learn and infer information about the masked region from the unmasked feature blocks, thereby enhancing the robustness of feature extraction.

[0032] The masked feature blocks are then input into the network body, which consists of several cascaded Mask Transformer Blocks (MTBs), for processing. Within each Mask Transformer Block (MTB), an attention mask is applied, such as... Figure 5 As shown, in the attention mask of each MTB, the implementation method is to redetermine which attention connections between feature block pairs need to be masked based on the new random mask probability when calculating the attention weight matrix. The weights corresponding to the masked positions in the attention matrix are set to negative infinity or minimum values, so that the attention weights at that position approach zero after softmax normalization, thereby blocking the flow of information. This attention-level mask is fundamentally different from the input mask of the feature embedding layer in terms of execution location and mechanism: the input mask directly replaces the content of the feature block in the initial stage of feature extraction, affecting the feature vector itself; while the attention mask dynamically adjusts the interaction relationship between feature blocks in the self-attention mechanism inside the Transformer block, affecting the topological structure of information aggregation. The randomness of the two masks is independent of each other, so that the model faces the challenge of missing information at different levels, further strengthening the model's generalization ability and the ability to reconstruct incomplete information.

[0033] After processing by multiple MTBs, the network finally outputs a reconstructed image I_denoise. The training objective of the image reconstruction network model is to minimize the pixel-level difference between the reconstructed image I_denoise and the original, clean, normal image I_clean. The L1 loss function (absolute value error loss function) can be used for optimization to drive the model to learn how to "remove" all contamination (including injected noise and applied masks) and recover the original normal image content.

[0034] S104: Acquire the image to be detected, and input the image to be detected into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain the anomaly detection result of the image to be detected.

[0035] In a specific implementation of this invention, the step of inputting the image to be detected into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain anomaly detection results for the image to be detected includes: inputting the image to be detected into a converged image reconstruction network model for image reconstruction processing to obtain a reconstructed image to be detected; calculating the pixel-level absolute difference between the reconstructed image to be detected and the image to be detected, and generating a residual image based on the calculated pixel-level absolute difference; if the residual image is a nearly completely black image, then the image to be detected is determined to be a normal image, and anomaly detection results for the image to be detected are generated; if the residual image contains bright patches, then the image to be detected contains an abnormal region, and the abnormal region is located based on the bright patches, and anomaly detection results for the image to be detected are generated.

[0036] Specifically, once a converged image reconstruction network model is obtained, the image to be detected can be input into the converged image reconstruction network model to perform detection of the image to be detected and detect whether there are any anomalies.

[0037] The image to be detected is input into a converged image reconstruction network model (no noise injection or masking is performed at this stage) to obtain a reconstructed image I_reconstructed. By calculating the pixel-level absolute difference between the input image to be detected I and the reconstructed image I_reconstructed, a residual map (mask image) M = |I - I_reconstructed| is generated. If the input image is normal, the model can reconstruct it well, and the residual map M will be close to black. If the input image contains anomalies, the model will try to "repair" the abnormal areas using the learned knowledge, causing the areas to appear as bright patches in the residual map, thereby achieving accurate detection and localization of anomalies.

[0038] Specific effects, such as Figure 6 As shown, Figure 6Two abnormal images with foreign objects are shown. After the images are processed by the model, the foreign object area is "filled" in the output "denoised image" based on the normal background knowledge learned by the model. Therefore, when the original abnormal image is subtracted from the reconstructed image, only the foreign object part shows a clear bright outline on the resulting "mask image", while the normal area is black, thus realizing the accurate detection and localization of unknown abnormal shapes.

[0039] This result clearly demonstrates the method's ability to accurately detect and locate anomalies of unknown morphology.

[0040] In this embodiment of the invention, it has excellent generalization ability and detection capability for unknown anomalies, and achieves completely unsupervised training without any defect samples. The training process does not require any form of defect samples, which perfectly solves the fundamental problem of scarce defect samples, high collection costs and inability to cover all types in industrial scenarios. High-precision pixel-level positioning, by calculating the pixel-level residual map between the input image and the model reconstructed image, can accurately delineate the contour and location of anomalies at high resolution, providing accurate visual basis for subsequent defect classification or equipment maintenance.

[0041] Example 2, please refer to Figure 2 , Figure 2 This is a schematic diagram of the structural composition of the image unsupervised anomaly detection device based on denoising in an embodiment of the present invention.

[0042] like Figure 2 As shown, an unsupervised image anomaly detection device based on denoising is disclosed, the device comprising: First construction module 201: used to construct an image reconstruction network model, the image reconstruction network model comprising a feature embedding module, an input mask module and several cascaded Mask Transformer Block modules; In the specific implementation of this invention, the input masking module is used to perform high-proportion random occlusion processing on the noisy features in the input image at the input end of the image reconstruction network model; the Mask Transformer Block module integrates an attention masking mechanism, which is used to simulate the situation of incomplete information in the internal calculation of the image reconstruction network model, so as to ensure that the image reconstruction network model can adapt to inputs with different information densities.

[0043] Specifically, the first step is to build an image reconstruction network model, such as... Figure 4As shown, the system comprises a feature embedding module, an input masking module, and several cascaded Mask Transformer Block modules. The feature embedding module converts the input image into image embedding features. The input end of the image reconstruction network model in the feature embedding module performs high-proportion random occlusion processing on the noisy features in the input image. The Mask Transformer Block module integrates an attention masking mechanism, which is used to simulate incomplete information in the computation process within the image reconstruction network model, ensuring that the image reconstruction network model can adapt to inputs with different information densities.

[0044] Second construction module 202: used to construct a training dataset, wherein the training images in the training dataset are normal images with compound noise applied; In the specific implementation of this invention, the training images in the training dataset are normal images with compound noise applied, including: obtaining several normal images, and randomly selecting several noise types for each normal image from a predefined noise pool, wherein the predefined noise pool includes additive Gaussian noise, salt-and-pepper noise, multiplicative speckle noise, and Poisson noise; configuring randomly determined intensity parameters for each selected noise type, and sequentially superimposing each selected noise type onto the corresponding normal image according to the corresponding intensity parameters in a random order to form training images, and the training dataset is composed of several training images.

[0045] Specifically, before training the image reconstruction network model, a training dataset needs to be constructed, which is a training dataset containing normal image samples. The image reconstruction network model will then be trained using the training dataset to make it more robust. The trained image reconstruction network model will be able to restore the original and clean state to the greatest extent possible, no matter how severely the input normal image is contaminated.

[0046] First, prepare a set of normal images (in this embodiment, a normal MiniLED image is used as an example) as the initial training set. Then, apply a dynamically generated noise composed of multiple basic noises to each normal image in the initial training set to generate a noisy image, thus forming the training dataset.

[0047] First, several basic noise types are randomly selected from a predefined noise pool and superimposed. This noise pool includes, but is not limited to, additive Gaussian noise, salt-and-pepper noise, multiplicative speckle noise, and Poisson noise. During the noise application process, each selected noise type is configured with a randomly determined intensity parameter. All types of noise are superimposed onto the original image in a random order to form a composite noise effect. This ensures that the noisy images generated in each training iteration have different combinations of noise features, thereby enhancing the model's robustness to diverse noise.

[0048] Training module 203: used to sequentially input the training images in the training dataset into the image reconstruction network model for unsupervised training until a converged image reconstruction network model is obtained; In the specific implementation of this invention, the step of sequentially inputting the training images from the training dataset into the image reconstruction network model for unsupervised training until a converged image reconstruction network model is obtained includes: sequentially inputting the training images from the training dataset into the image reconstruction network model for training, and during the training process, the feature embedding layer in the image reconstruction network model converts the input training images into a series of feature blocks; inputting the series of feature blocks into the input masking module to perform masking processing, and inputting the masked feature blocks into several cascaded Mask Transformer Block modules for image reconstruction processing, outputting the reconstruction result of each training image; performing pixel-level difference calculation processing using the reconstruction result of each training image and the corresponding normal image to obtain the pixel-level difference calculation result corresponding to the reconstruction result of each training image; optimizing the parameters of each layer of the trained image reconstruction network model using the absolute value error loss function based on the pixel-level difference calculation result corresponding to the reconstruction result of each training image, and returning to the step of sequentially inputting the training images from the training dataset into the image reconstruction network model for training until a converged image reconstruction network model is obtained.

[0049] Furthermore, the step of inputting a series of feature blocks into the input masking module for masking processing, and then inputting the masked feature blocks into several cascaded Mask Transformer Block modules for image reconstruction processing, outputting the reconstruction result of each training image, includes: after the input masking module receives a series of feature blocks, it iterates through all feature blocks and performs masking judgment on each feature block according to a preset masking probability to form a masking judgment result corresponding to each feature block; when the masking judgment result indicates that the corresponding feature block needs to be masked, the original feature vector corresponding to the feature block that needs to be masked is replaced with the masking marker vector from the training process to form the masked feature block, and the dimension of the masking marker vector is the same as that of the original feature vector; the masked feature block is input into several cascaded Mask Transformer Block modules for image reconstruction processing, and the reconstruction result of each training image is output.

[0050] Specifically, training an image reconstruction network model requires enabling the model to randomly occlude a high proportion of noisy features in the input image data. This allows the trained network model to move beyond simple noise pattern recognition and instead learn the inherent structure, contextual relationships, and deep semantics of normal images in order to reconstruct the image from extremely limited information.

[0051] During training, each noisy image in the training dataset is input as follows: Figure 4 In the constructed image reconstruction network model, the input noisy image is first converted into a series of feature blocks (tokens) through the feature embedding module (layer) within the image reconstruction network model. Then, the series of feature blocks are input into the input mask module (InputMask). In the input mask module, a mask probability parameter is set, and then all feature blocks in the series are traversed. Each feature block is independently masked according to the mask probability parameter. When a feature block is determined to need to be masked, its original feature vector is replaced with a mask token vector [mask-token] that can be learned during training. The dimension of the token vector is the same as that of the original feature block, and the initial value can be randomly initialized or set to zero vector. It is optimized and updated through backpropagation in subsequent training. Through this random masking mechanism, the model is forced to learn and infer information about the masked region from the unmasked feature blocks, thereby enhancing the robustness of feature extraction.

[0052] The masked feature blocks are then input into the network body, which consists of several cascaded Mask Transformer Blocks (MTBs), for processing. Within each Mask Transformer Block (MTB), an attention mask is applied, such as... Figure 5 As shown, in the attention mask of each MTB, the implementation method is to redetermine which attention connections between feature block pairs need to be masked based on the new random mask probability when calculating the attention weight matrix. The weights corresponding to the masked positions in the attention matrix are set to negative infinity or minimum values, so that the attention weights at that position approach zero after softmax normalization, thereby blocking the flow of information. This attention-level mask is fundamentally different from the input mask of the feature embedding layer in terms of execution location and mechanism: the input mask directly replaces the content of the feature block in the initial stage of feature extraction, affecting the feature vector itself; while the attention mask dynamically adjusts the interaction relationship between feature blocks in the self-attention mechanism inside the Transformer block, affecting the topological structure of information aggregation. The randomness of the two masks is independent of each other, so that the model faces the challenge of missing information at different levels, further strengthening the model's generalization ability and the ability to reconstruct incomplete information.

[0053] After processing by multiple MTBs, the network finally outputs a reconstructed image I_denoise. The training objective of the image reconstruction network model is to minimize the pixel-level difference between the reconstructed image I_denoise and the original, clean, normal image I_clean. The L1 loss function (absolute value error loss function) can be used for optimization to drive the model to learn how to "remove" all contamination (including injected noise and applied masks) and recover the original normal image content.

[0054] Anomaly detection module 204: used to acquire the image to be detected, and input the image to be detected into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain the anomaly detection result of the image to be detected.

[0055] In a specific implementation of this invention, the step of inputting the image to be detected into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain anomaly detection results for the image to be detected includes: inputting the image to be detected into a converged image reconstruction network model for image reconstruction processing to obtain a reconstructed image to be detected; calculating the pixel-level absolute difference between the reconstructed image to be detected and the image to be detected, and generating a residual image based on the calculated pixel-level absolute difference; if the residual image is a nearly completely black image, then the image to be detected is determined to be a normal image, and anomaly detection results for the image to be detected are generated; if the residual image contains bright patches, then the image to be detected contains an abnormal region, and the abnormal region is located based on the bright patches, and anomaly detection results for the image to be detected are generated.

[0056] Specifically, once a converged image reconstruction network model is obtained, the image to be detected can be input into the converged image reconstruction network model to perform detection of the image to be detected and detect whether there are any anomalies.

[0057] The image to be detected is input into a converged image reconstruction network model (no noise injection or masking is performed at this stage) to obtain a reconstructed image I_reconstructed. By calculating the pixel-level absolute difference between the input image to be detected I and the reconstructed image I_reconstructed, a residual map (mask image) M = |I - I_reconstructed| is generated. If the input image is normal, the model can reconstruct it well, and the residual map M will be close to black. If the input image contains anomalies, the model will try to "repair" the abnormal areas using the learned knowledge, causing the areas to appear as bright patches in the residual map, thereby achieving accurate detection and localization of anomalies.

[0058] Specific effects, such as Figure 6 As shown, Figure 6 Two abnormal images with foreign objects are shown. After the images are processed by the model, the foreign object area is "filled" in the output "denoised image" based on the normal background knowledge learned by the model. Therefore, when the original abnormal image is subtracted from the reconstructed image, only the foreign object part shows a clear bright outline on the resulting "mask image", while the normal area is black, thus realizing the accurate detection and localization of unknown abnormal shapes.

[0059] This result clearly demonstrates the method's ability to accurately detect and locate anomalies of unknown morphology.

[0060] In this embodiment of the invention, it has excellent generalization ability and detection capability for unknown anomalies, and achieves completely unsupervised training without any defect samples. The training process does not require any form of defect samples, which perfectly solves the fundamental problem of scarce defect samples, high collection costs and inability to cover all types in industrial scenarios. High-precision pixel-level positioning, by calculating the pixel-level residual map between the input image and the model reconstructed image, can accurately delineate the contour and location of anomalies at high resolution, providing accurate visual basis for subsequent defect classification or equipment maintenance.

[0061] This invention provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the image unsupervised anomaly detection method of any of the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disk, hard disk, optical disk, CD-ROM, and magneto-optical disk), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (e.g., a computer, a mobile phone), and can be a read-only memory, a disk, or an optical disk, etc.

[0062] This invention also provides a computer application running on a computer, which is used to execute the image unsupervised anomaly detection method of any of the above embodiments.

[0063] also, Figure 3 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention.

[0064] This invention also provides an electronic device, such as... Figure 3 As shown. The electronic device includes a processor 302, a memory 303, an input unit 304, and a display unit 305, among other devices. Those skilled in the art will understand that... Figure 3The structural components of the illustrated electronic device do not constitute a limitation on all devices and may include more or fewer components than illustrated, or combine certain components. Memory 303 can be used to store application program 301 and various functional modules. Processor 302 runs application program 301 stored in memory 303, thereby performing various functional applications and data processing of the device. Memory can be internal memory or external memory, or both. Internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. External memory may include hard disks, floppy disks, ZIP disks, USB flash drives, magnetic tapes, etc. The memory disclosed in this invention includes, but is not limited to, these types of memory. The memory disclosed in this invention is only an example and not a limitation.

[0065] Input unit 304 is used to receive signal input and user-input keywords. Input unit 304 may include a touch panel and other input devices. The touch panel can collect user touch operations on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel) and drive the corresponding connection device according to a pre-set program; other input devices may include, but are not limited to, one or more of physical keyboards, function keys (such as play control buttons, power buttons, etc.), trackballs, mice, joysticks, etc. Display unit 305 can be used to display user-input information or information provided to the user, as well as various menus of the terminal device. Display unit 305 may be in the form of a liquid crystal display, organic light-emitting diode, etc. Processor 302 is the control center of the terminal device, connecting various parts of the entire device through various interfaces and lines, and performing various functions and processing data by running or executing software programs and / or modules stored in memory 303, and calling data stored in memory.

[0066] As one embodiment, the electronic device includes: one or more processors 302, a memory 303, and one or more application programs 301, wherein the one or more application programs 301 are stored in the memory 303 and configured to be executed by the one or more processors 302, and the one or more application programs 301 are configured to perform the image unsupervised anomaly detection method corresponding to any of the embodiments described above.

[0067] In this embodiment of the invention, it has excellent generalization ability and detection capability for unknown anomalies, and achieves completely unsupervised training without any defect samples. The training process does not require any form of defect samples, which perfectly solves the fundamental problem of scarce defect samples, high collection costs and inability to cover all types in industrial scenarios. High-precision pixel-level positioning, by calculating the pixel-level residual map between the input image and the model reconstructed image, can accurately delineate the contour and location of anomalies at high resolution, providing accurate visual basis for subsequent defect classification or equipment maintenance.

[0068] Furthermore, the above provides a detailed description of the image unsupervised anomaly detection method and related apparatus based on denoising provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for unsupervised image anomaly detection based on denoising, characterized in that, The method includes: An image reconstruction network model is constructed, which consists of a feature embedding module, an input mask module, and several cascaded Mask Transformer Block modules; Construct a training dataset, wherein the training images in the training dataset are normal images with compound noise applied; The training images in the training dataset are sequentially input into the image reconstruction network model for unsupervised training until a converged image reconstruction network model is obtained. The image to be detected is acquired and input into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain the anomaly detection result of the image to be detected.

2. The image unsupervised anomaly detection method according to claim 1, characterized in that, The input masking module is used to perform high-proportion random occlusion processing on noisy features in the input image at the input end of the image reconstruction network model. The Mask Transformer Block module integrates an attention masking mechanism, which is used to simulate incomplete information during the computation process within the image reconstruction network model, ensuring that the image reconstruction network model can adapt to inputs with different information densities.

3. The image unsupervised anomaly detection method according to claim 1, characterized in that, The training images in the training dataset are normal images with compound noise applied, including: Several normal images are obtained, and several noise types are randomly selected for each normal image from a predefined noise pool. The predefined noise pool includes additive Gaussian noise, salt and pepper noise, multiplicative speckle noise, and Poisson noise. For each selected noise type, a randomly determined intensity parameter is configured, and each selected noise type is sequentially superimposed onto the corresponding normal image according to the corresponding intensity parameter in a random order to form a training image. The training dataset consists of several training images.

4. The image unsupervised anomaly detection method according to claim 1, characterized in that, The step of sequentially inputting the training images from the training dataset into the image reconstruction network model for unsupervised training until a converged image reconstruction network model is obtained includes: The training images in the training dataset are sequentially input into the image reconstruction network model for training processing. During the training process, the feature embedding layer in the image reconstruction network model converts the input training images into a series of feature blocks. A series of feature blocks are input into the input mask module to perform masking processing, and the masked feature blocks are input into several cascaded Mask Transformer Block modules for image reconstruction processing, and the reconstruction result of each training image is output. The pixel-level difference calculation results are obtained by using the reconstruction results of each training image and the corresponding normal image to perform pixel-level difference calculation. Based on the pixel-level difference calculation results corresponding to the reconstruction results of each training image, the absolute value error loss function is used to optimize the parameters of each layer of the trained image reconstruction network model. After optimization, the training images in the training dataset are sequentially input into the image reconstruction network model for training processing until a converged image reconstruction network model is obtained.

5. The image unsupervised anomaly detection method according to claim 4, characterized in that, The process involves inputting a series of feature blocks into the input mask module for masking, and then inputting the masked feature blocks into several cascaded Mask Transformer Block modules for image reconstruction. The reconstruction result for each training image is then output, including: After the input mask module receives a series of feature blocks, it traverses all feature blocks and performs mask judgment on each feature block according to the preset mask probability to form a mask judgment result for each feature block. When the masking judgment result indicates that the corresponding feature block needs to be masked, the original feature vector corresponding to the feature block that needs to be masked is replaced with the masking marker vector in the training process to form the masked feature block, and the dimension of the masking marker vector is the same as that of the original feature vector. The masked feature blocks are input into several cascaded Mask Transformer Block modules for image reconstruction, and the reconstruction result of each training image is output.

6. The image unsupervised anomaly detection method according to claim 5, characterized in that, The masked feature blocks are input into several concatenated Mask Transformer Block modules for image reconstruction processing, and the reconstruction result of each training image is output, including: The masked feature block is input into several cascaded Mask Transformer Block modules, and a self-attention computation layer is used in each Mask Transformer Block module to perform independent random masking processing on the masked feature block to obtain random masking results. Based on the random masking result, determine whether the attention connection between the corresponding feature block and several cascaded Mask Transformer Block modules needs to be masked; When a feature block needs to be masked, the weight of the feature block to be masked in the attention matrix is ​​set to negative infinity or a minimum value, and after softmax normalization, the attention weight of the feature block to be masked is made close to zero. After several cascaded Mask Transformer Block modules process the masked feature blocks according to attention weights, the reconstruction result of each training image is output.

7. The image unsupervised anomaly detection method according to claim 1, characterized in that, The step of inputting the image to be detected into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain the anomaly detection result of the image to be detected includes: The image to be detected is input into a converged image reconstruction network model for image reconstruction processing to obtain the reconstructed image to be detected. Calculate the pixel-level absolute difference between the reconstructed image to be detected and the image to be detected, and generate a residual image based on the calculated pixel-level absolute difference; If the residual image is a nearly completely black image, then the image to be detected is determined to be a normal image, and an anomaly detection result for the image to be detected is generated; If the residual image contains bright patches, it is determined that there are abnormal regions in the image to be detected, and the abnormal regions are located based on the bright patches to generate the abnormal detection result of the image to be detected.

8. An image unsupervised anomaly detection device based on denoising, characterized in that, The device includes: The first building module is used to build an image reconstruction network model, which consists of a feature embedding module, an input mask module, and several cascaded Mask Transformer Block modules. The second building module is used to build a training dataset, wherein the training images in the training dataset are normal images with compound noise applied. Training module: used to sequentially input the training images in the training dataset into the image reconstruction network model for unsupervised training until a converged image reconstruction network model is obtained; Anomaly detection module: used to acquire the image to be detected, and input the image to be detected into a converged image reconstruction network model for unsupervised anomaly detection processing to obtain the anomaly detection result of the image to be detected.

9. An electronic device comprising a processor and a memory, characterized in that, The processor runs a computer program or code stored in the memory to implement the image unsupervised anomaly detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium for storing computer programs or code, characterized in that, When the computer program or code is executed by a processor, the image unsupervised anomaly detection method as described in any one of claims 1 to 7 is implemented.

Images