Image anomaly detection method, device and electronic equipment

Abstract

The application discloses an image anomaly detection method, device and electronic equipment. The method comprises the following steps: obtaining a target image; performing noise adding processing on the target image at N noise levels respectively, to obtain N noise images, wherein N is an integer greater than 1; based on the N noise images, predicting the target image before noise adding, to obtain N predicted images and image reconstruction untrustworthiness corresponding to the N predicted images respectively; based on the N predicted images and the image reconstruction untrustworthiness corresponding to the N predicted images, generating a reconstructed image; and based on the target image and the reconstructed image, performing image anomaly detection on the target image, to obtain an image anomaly detection result. The application solves the technical problem of defect missing detection and defect mis-detection when a defect detection method based on a reconstructed image is used in the related art.

Description

Technical Field

[0001] The present invention relates to the field of electric power, and in particular to an image anomaly detection method, device and electronic equipment. Background Art

[0002] Image-based equipment defect recognition technology has enormous application demand in the power industry. In the field of power transmission and transformation engineering construction alone, nearly 20 types of defects require identification, including those in substations, cables, aerial drones, and mobile phones. While supervised learning-based defect detection methods theoretically achieve high detection accuracy, they suffer from the high cost of collecting defect training samples and their inability to identify undefined defects. Therefore, applying self-supervised learning-based image anomaly detection algorithms, which require only normal image samples for training, to power equipment defect detection is a valuable complement to supervised training approaches. Among various self-supervised learning-based image anomaly detection algorithms, those based on image reconstruction have the advantages of not requiring defective samples in the training set, not relying on pre-trained models, and providing highly interpretable recognition results. These advantages make them particularly suitable for power equipment defect detection. Image reconstruction models typically train an autoencoder in a self-supervised manner. When an abnormal image is fed into the trained model, the model outputs a reconstructed image that approximates a normal image without defects. Therefore, the reconstruction error per pixel can be used to pinpoint the defect location. However, since the autoencoder has a certain robustness in reconstructing images, even defective textures can often be reconstructed, which easily leads to the problem of missed detection and false detection of defective samples.

[0003] Therefore, in the related art, when defect detection is performed based on the image reconstruction method, there are problems of missed defect detection and false defect detection.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] The embodiments of the present invention provide an image anomaly detection method, device, and electronic device to at least solve the technical problem in the related art of missed or false defect detection when performing defect detection based on image reconstruction methods.

[0006] According to one aspect of an embodiment of the present invention, there is provided an image anomaly detection method, comprising: acquiring a target image; performing noise addition processing on the target image at N noise levels, respectively, to obtain N noise images, wherein N is an integer greater than 1; predicting the target image before noise addition based on the N noise images, respectively, to obtain N predicted images, and image reconstruction unreliability corresponding to the N predicted images; generating a reconstructed image based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images; performing image anomaly detection on the target image based on the target image and the reconstructed image, to obtain an image anomaly detection result.

[0007] Optionally, performing noise addition processing of N noise levels on the target image to obtain N noise images includes: selecting a noise level from a predetermined number T of noise levels to obtain N noise levels; and superimposing noise of the N noise levels on the target image to obtain N noise images.

[0008] Optionally, the noise level selected from the predetermined number T of noise levels can mask abnormal textures of a predetermined scale in the target image without completely masking information of the target object in the target image.

[0009] Optionally, generating a reconstructed image based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images respectively includes: determining the image reconstruction reliability corresponding to the N predicted images respectively based on the image reconstruction unreliability corresponding to the N predicted images respectively; obtaining target weighted results corresponding to the N predicted images based on the N predicted images and the image reconstruction reliability corresponding to the N predicted images respectively, and the image reconstruction unreliability corresponding to the target image and the N predicted images respectively; and summing the target weighted results corresponding to the N predicted images to obtain the reconstructed image.

[0010] Optionally, the target weighted results corresponding to the N predicted images are obtained based on the N predicted images and the image reconstruction credibility corresponding to the N predicted images, and the image reconstruction unreliability corresponding to the target image and the N predicted images, respectively, including: for any target predicted image among the N predicted images, based on the target predicted image and the image reconstruction credibility corresponding to the target predicted image, obtaining a first weighted result of the target predicted image, and based on the target image and the image reconstruction unreliability corresponding to the target predicted image, obtaining a second weighted result of the target predicted image; summing the first weighted result and the second weighted result to obtain the target weighted result of the target predicted image; and adopting the method of obtaining the target weighted result by the target predicted image to obtain the target weighted results corresponding to the N predicted images.

[0011] Optionally, the method of predicting the target image before noise addition based on the N noise images, and obtaining N predicted images and image reconstruction unreliability corresponding to the N predicted images respectively, includes: inputting the N noise images into the image denoising model respectively, predicting the target image before noise addition, and obtaining N predicted images and image reconstruction unreliability corresponding to the N predicted images respectively, wherein the image denoising model is trained based on image sample data, and the image sample data includes: a sample image and a sample predicted image corresponding to the sample image and an image reconstruction unreliability corresponding to the sample predicted image.

[0012] Optionally, before inputting the N noisy images into the image denoising model respectively, predicting the target image before adding noise, and obtaining N predicted images respectively, and the image reconstruction unreliability corresponding to the N predicted images respectively, the method also includes: constructing an initial denoising model, and a loss function corresponding to the initial denoising model, wherein the loss function includes a first loss term and a second loss term, the first loss term is used to mark the loss value of image noise, and the second loss term is used to mark the loss value of image reconstruction unreliability; based on the loss function, the image sample data is used to train the initial denoising model to obtain the image denoising model.

[0013] According to another aspect of the present invention, an image anomaly detection device is provided, comprising: an acquisition module for acquiring a target image; a processing module for performing noise addition processing on the target image at N noise levels, respectively, to obtain N noisy images, wherein N is an integer greater than 1; a prediction module for predicting the target image before noise addition based on the N noisy images, respectively, to obtain N predicted images, and image reconstruction unreliability corresponding to the N predicted images; a generation module for generating a reconstructed image based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images; and a detection module for performing image anomaly detection on the target image based on the target image and the reconstructed image, to obtain an image anomaly detection result.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned image anomaly detection methods.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes any one of the above-mentioned image anomaly detection methods when running.

[0016] In an embodiment of the present invention, the target image before noise is added is predicted based on the N noise images to obtain N predicted images and the image reconstruction unreliability corresponding to the N predicted images respectively; a reconstructed image is generated based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images respectively; and image anomaly detection is performed on the target image based on the target image and the reconstructed image to obtain an image anomaly detection result. By using the image reconstruction unreliability corresponding to the N predicted images respectively, the purpose of distinguishing whether the N predicted images are reliable is achieved, thereby realizing the technical effect of avoiding defect missed detection and false detection, and further solving the technical problem of defect missed detection and false detection in the related art when defect detection is performed based on the method of reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a flowchart of an image anomaly detection method according to an embodiment of the present invention;

[0019] Figure 2is a structural diagram of a model training device provided according to an optional embodiment of the present invention;

[0020] Figure 3 is a schematic structural diagram of an abnormal image prediction device provided according to an optional embodiment of the present invention;

[0021] Figure 4 is a structural block diagram of an image anomaly detection device provided according to an embodiment of the present invention;

[0022] Figure 5 is a structural block diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] According to an embodiment of the present invention, a method embodiment of an image anomaly detection method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] Figure 1 FIG. 1 is a flow chart of an image anomaly detection method according to an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0027] Step S102, acquiring a target image;

[0028] As an optional embodiment, the method of this embodiment can be executed by a terminal or server used for image anomaly detection. For example, when applied to a terminal used for image anomaly detection, it can easily implement image processing in simple image scenarios. For another example, when applied to a server, it can utilize the server's rich computing resources or a relatively larger and more accurate document model, thereby more accurately detecting whether an image is abnormal.

[0029] It should be noted that the aforementioned terminals can be of various types, such as mobile terminals with certain computing capabilities or fixed computers with identification capabilities. The aforementioned servers can also be of various types, such as local servers or virtual cloud servers. Depending on the computing power, the servers can be single computers or computer clusters integrating multiple computers.

[0030] As an optional embodiment, the target image may be any of various types of images requiring anomaly detection, such as images of power equipment, based on which defects in the power equipment are detected. Power equipment may include various types, such as power generation equipment, transmission equipment, and power-consuming equipment, as well as monitoring equipment, servers, or image processors involved in power processing.

[0031] Step S104, performing noise processing at N noise levels on the target image to obtain N noisy images, where N is an integer greater than 1;

[0032] As an optional embodiment, before subjecting the target image to N noise levels to obtain N noise images, the target image can be preprocessed to make the processed target image easier to detect defects. For example, the target image includes areas with a high probability of defects and areas with a low probability of defects. In order to efficiently detect defects, defect detection can be performed only on the areas with a high probability of defects. Therefore, the areas with a high probability of defects can be cropped from the target image and defect detection can be performed based on these areas, making the detection results more direct and more efficient.

[0033] As an optional embodiment, the N noise levels mentioned above may be N noise levels selected from predetermined standard noise levels, and the predetermined standard noise levels may be the standard of noise levels in the field of image processing.

[0034] As an optional embodiment, the target image can be subjected to N noise levels of noise processing to obtain N noisy images in various ways. For example, a noise level can be selected from a predetermined number T of noise levels to obtain N noise levels; and noise of each of the N noise levels can be superimposed on the target image to obtain N noisy images. The noise level can also be selected from the predetermined number T of noise levels in various ways. For example, a noise level can be randomly selected from the predetermined number T of noise levels, and the randomization method can be an average randomization method. Alternatively, based on the image detection requirements, the noise level selected from the predetermined number T of noise levels can mask abnormal texture of a predetermined scale in the target image without completely masking the target object information in the target image. The predetermined scale can be determined based on the image's requirements for abnormal texture.

[0035] Step S106 , predicting the target image before adding noise based on the N noise images, and obtaining N predicted images and image reconstruction unreliability corresponding to the N predicted images;

[0036] As an optional embodiment, based on N noise images, the target image before adding noise is predicted to obtain N predicted images and the image reconstruction unreliability corresponding to the N predicted images can be obtained in a variety of ways. For example, some simple image comparison methods can be used to perform prediction to obtain the N predicted images; in order to make the N predicted images more accurate, an artificial intelligence model can be used for prediction.

[0037] For example, when predicting a target image before noise addition based on N noisy images, and obtaining N predicted images and image reconstruction unreliability corresponding to each of the N predicted images, the N noisy images can be input into an image denoising model to predict the target image before noise addition, and obtaining N predicted images and image reconstruction unreliability corresponding to each of the N predicted images, wherein the image denoising model is trained based on image sample data, and the image sample data includes: sample images, sample predicted images corresponding to the sample images, and image reconstruction unreliability corresponding to the sample predicted images. When the image denoising model is used to predict the target image before noise addition based on N noisy images, and obtaining N predicted images and image reconstruction unreliability corresponding to each of the N predicted images, since the model is trained based on a large amount of image sample data, the image denoising model can learn the reliability of various types of defects. Therefore, when performing image prediction based on the image denoising model, not only is the prediction result accurate, but the efficiency is also high.

[0038] As an optional embodiment, before inputting N noisy images into an image denoising model, predicting the target image before adding noise, and obtaining N predicted images and the image reconstruction unreliability corresponding to each of the N predicted images, an initial denoising model and a loss function corresponding to the initial denoising model may be constructed. The loss function includes a first loss term and a second loss term, the first loss term being used to mark the loss value of image noise, and the second loss term being used to mark the loss value of image reconstruction unreliability. Based on the loss function, the initial denoising model is trained using image sample data to obtain an image denoising model. Compared to conventional denoising models, this image denoising model introduces a prediction of the image reconstruction unreliability corresponding to the predicted image. Therefore, when training the image denoising model, training can be based on either the loss of the predicted image or the loss of the image reconstruction unreliability corresponding to the predicted image. The combination of these two losses can result in more accurate predictions of the predicted image and more accurate image reconstruction unreliability corresponding to the predicted image. Therefore, when using the loss function including the above-mentioned first loss term and the second loss term for model training, the above-mentioned two training objectives can be effectively treated separately, and when the above-mentioned two items are combined, the model training results can take into account the above-mentioned two training objectives and achieve the overall optimal training results.

[0039] Step S108 , generating a reconstructed image based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images;

[0040] As an optional embodiment, when generating a reconstructed image based on N predicted images and the image reconstruction unreliability corresponding to each of the N predicted images, image reconstruction can be performed based on the overall prediction results of the N predicted images. For example, this can be achieved by using the following processing method: based on the image reconstruction unreliability corresponding to each of the N predicted images, the image reconstruction reliability corresponding to each of the N predicted images is determined. For example, when the image reconstruction unreliability of any of the N predicted images is W, the image reconstruction reliability of that predicted image is 1-W; based on the image reconstruction reliability corresponding to the N predicted images and the N predicted images, as well as the image reconstruction unreliability corresponding to the target image and the N predicted images, target weighted results corresponding to the N predicted images are obtained; and the target weighted results corresponding to the N predicted images are summed to obtain a reconstructed image. Therefore, using the above processing, the target weighted results of the N predicted images can be summed to obtain a reconstructed image of the target image from the N predicted images as a whole.

[0041] As an optional embodiment, when target weighted results corresponding to the N predicted images are obtained based on N predicted images and the image reconstruction credibility corresponding to the N predicted images, and the image reconstruction unreliability corresponding to the target image and the N predicted images, the target weighted results corresponding to the N predicted images can be obtained in the following manner: first, for any target predicted image among the N predicted images, a first weighted result of the target predicted image is obtained based on the target predicted image and the image reconstruction credibility corresponding to the target predicted image, and a second weighted result of the target predicted image is obtained based on the target image and the image reconstruction unreliability corresponding to the target predicted image; the first weighted result and the second weighted result are summed to obtain the target weighted result of the target predicted image; and the target weighted results corresponding to the N predicted images are obtained in the manner of obtaining the target weighted result from the target predicted image.

[0042] Step S110 : performing image anomaly detection on the target image based on the target image and the reconstructed image to obtain an image anomaly detection result.

[0043] As an optional embodiment, when image anomaly detection is performed on the target image based on the target image and the reconstructed image to obtain the image anomaly detection result, when image anomaly detection is performed on the target image, the target image can be directly compared with the reconstructed image. There can be many comparison methods, for example, direct pixel-level comparison or direct three-color image comparison, which will not be elaborated here one by one.

[0044] Through the above steps, the target image before adding noise is predicted based on N noise images, and N predicted images and the image reconstruction unreliability corresponding to the N predicted images are obtained respectively; a reconstructed image is generated based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images; image anomaly detection is performed on the target image based on the target image and the reconstructed image to obtain the image anomaly detection result. Through the image reconstruction unreliability corresponding to the N predicted images, the purpose of distinguishing whether the N predicted images are reliable is achieved, thereby realizing the technical effect of avoiding defect missed detection and false detection, and further solving the technical problem of defect missed detection and false detection in the related technology when performing defect detection based on the method of reconstructed image.

[0045] It should be noted that, when predicting the target image, not only the noise in the noise image is predicted, but also the image reconstruction unreliability corresponding to the predicted image is predicted, and the image reconstruction unreliability reflects to a certain extent the degree of credibility of the predicted image as the target image. Therefore, when predicting the target image based on the predicted image, it will not be unilaterally and directly assumed that the image has defects or the image does not have defects.

[0046] For example, if the target image is an image of a piece of electrical equipment, there is a strong reflective area on the equipment. When noise is superimposed on this image, a noise image is generated. When predicting this noise image, the added noise may cause the strong reflective area to be significantly different from other areas. As a result, the predicted image will differ significantly from the original equipment image. This means that the strong reflective area will be directly identified as a defect in the equipment. However, this strong reflective area is merely a reflection of the equipment and does not indicate a defect. This means that the anomaly detection result is incorrect, resulting in a false detection. After adopting the solution of the embodiment of the present invention, since there will be a prediction of image reconstruction unreliability for the predicted image with added noise, therefore, when the possibility of abnormality obtained by abnormality detection based on the predicted image is relatively large, since this area is identified as a strong reflective area during the prediction, the possibility of it being a defect is relatively low, that is, the unreliability of the reconstructed image is relatively high. Therefore, after combining the abnormality detection result obtained based on the predicted image with the higher unreliability of the reconstructed image, the result of the device having defects based on the predicted image is relatively low. Therefore, the situation of false detection of device defects is effectively avoided, and the accuracy of device detection is effectively improved.

[0047] Based on the above embodiment and optional embodiment, an optional implementation manner is provided.

[0048] In response to the aforementioned issues in the related art, anomaly detection methods based on de-watermarked image reconstruction have been proposed. These methods first apply watermarks of various scales to the image to mask defective textures, then use a model to remove the watermarks and reconstruct the original image. Because the features of the anomaly region are completely erased after watermarking, the defective features are theoretically guaranteed not to be reconstructed, thus reducing the missed detection rate. However, a new problem arises: because the model's reconstruction of the original image from the degraded image has a large posterior variance, the reconstructed image often differs significantly from the original image in scenes with a wide range of angles and textures of normal objects (for example, highly reflective areas on equipment), which in turn leads to an increase in false positives. Other related techniques have also been proposed for image anomaly detection based on denoising diffusion models, but these methods still have two major drawbacks: first, they fail to suppress the interference of the image reconstruction posterior variance on anomaly detection, resulting in a high false positive rate; second, each sample of image anomaly detection requires hundreds of calls to the denoising model, resulting in low algorithm efficiency.

[0049] Therefore, in view of the above-mentioned problems in the related art, in an optional embodiment of the present invention, a new neural network output is designed based on the denoising diffusion model to predict the reconstruction unreliability of each pixel in the image, and a training method for the model-predicted image reconstruction unreliability is proposed. Furthermore, based on the image reconstruction unreliability proposed in this optional embodiment of the present invention, a new image reconstruction and image anomaly detection method is proposed. Furthermore, based on the new model, new training method, new image reconstruction, and anomaly detection method proposed in this optional embodiment of the present invention, a new image anomaly detection device is proposed. These are described below.

[0050] 1) Artificial neural network structure of optional embodiment of the present invention

[0051] The denoising model provided in the optional embodiment of the present invention is based on the Denoising Diffusion Probabilistic Models in the field of image generation. The denoising diffusion model adopts the U-Net artificial neural network. The dimensions of the input and output tensors of the traditional U-Net network are the same, both [N, C, H, W], where N is the number of batch samples, C is the number of color channels (usually RGB three channels), H is the image height, and W is the image width. The improved U-Net network proposed in the optional embodiment of the present invention adds an output dimension to the output layer so that the dimension of the output tensor is [N, C+1, H, W]. The added channel is used to predict the image reconstruction unreliability of each pixel in the image. The activation function of this channel is Sigmoid, so that the value range of the predicted image reconstruction unreliability is between [0,1].

[0052] II) Model training method of optional embodiment of the present invention

[0053] During the model training process, noise can be superimposed on the original image x0 in the training set according to the randomly sampled noise level variable t.

[0054] The noise level variable t is an integer whose value is randomly sampled from a uniform distribution between [1, T]. The constant T is usually 1000. When the noise level is 1000, the target texture in the image is completely submerged by the noise, approximating to a white noise image.

[0055] Add noise level t to the original image x0 to get the noise image x t , the formula is:

[0056]

[0057] In formula (1), It is a constant obtained by looking up the table according to the t value. The constant range is between (0, 1) and represents the noise image x t The energy proportion of the original image x0 in . The value of decreases monotonically with the increase of t. When t is T, The value of is close to 0. The white noise image ∈ is distributed according to the standard normal distribution. Obtained by random sampling.

[0058] Model training is to adjust the input and output weights of each neuron in the artificial neural network to minimize the loss function. Usually, all network weights of the denoising model are represented as θ. The prediction of the denoising model for the image noise ∈ is expressed as a function, denoted as ∈ θ (x t ,t). The noise prediction loss function of the denoising model is:

[0059]

[0060] In formula (2), E represents the mean of the losses of multiple samples in a Batch.

[0061] The channel added to the U-Net output layer is used to predict the image reconstruction unreliability. The output of this channel uses the Sigmoid activation function to make the value range between [0,1]. The image reconstruction unreliability predicted by the model can be expressed as M θ (x t ,t). M θ (x t ,t) has a tensor dimension of [N, 1, H, W].

[0062] Training M θ (x t ,t) is dynamically generated based on the mean square error of the noise prediction of the training sample, and is defined as:

[0063]

[0064] In formula (3), tanh is the tanh activation function (when the input value range of this function is between [0, +∞), the output is between [0, 1)).

[0065] S is a preset constant (a positive real number), is the anomaly detection noise set (will be introduced in the next chapter)

[0066] Training M θ (x t ,t) is the binary cross entropy:

[0067]

[0068] The total training loss function is:

[0069] L all =L simple +L M (5)

[0070] In the above formulas, formulas (1) and (2) are conventional formulas of the noise reduction diffusion model. Formulas (3), (4), and (5) are different from conventional formulas based on optional embodiments of the present invention.

[0071] The model training process involves extracting N samples from the training set to form a batch. This batch is used for model inference and to calculate the total loss. The gradient of the network weights is then calculated by taking the derivative of the total loss function. The network weights are then updated based on the gradient according to the pre-set learning rate and optimization method. This cycle continues until the total loss converges to a very small value.

[0072] III) Image reconstruction method and image anomaly detection method in optional embodiments of the present invention

[0073] The conventional denoising model is based on the noise image x with noise level t t To directly predict the original image x0 before adding noise, the image generated by this prediction is recorded as The prediction method is:

[0074]

[0075] Using formula (6) to predict the original image will produce a large error. Therefore, a new reconstruction method is proposed in an optional embodiment of the present invention.

[0076] In an optional embodiment of the present invention, a noise level set for image anomaly detection is set It selects N levels among all T noise levels.

[0077]

[0078] The principle of selecting the noise level is to make the noise cover up abnormal textures of a specific scale in the image without completely covering up the information of normal targets. For example, 25 noise levels with t being [350, 360, …, 589, 599] can be selected from 1000 noise levels.

[0079] In an optional embodiment of the present invention, the proposed image reconstruction method is:

[0080]

[0081] The meaning of formula (8) is:

[0082] 1) The image reconstruction unreliability mask is M θ (x τ ,τ);

[0083] 2) The image reconstruction credibility mask is (1-M θ (x τ ,τ))

[0084] 3) The original image predicted based on the noise level τ is predicted by the conventional method (6) The weighted sum between and the original image x0. The weight of is the image reconstruction credibility mask, and the weight of x0 is the image reconstruction untrustworthy mask.

[0085] 4) The final reconstructed image is the noise level set The weighted sum of the predicted images for each noise level τ in . Among them, the weight W τ Can be set to (but not limited to)

[0086] The image anomaly score graph is recorded as E, and the image anomaly detection method is:

[0087]

[0088] In formula (9), the general image comparison function Compare includes but is not limited to:

[0089] 1) Calculate the RGB mean square error of corresponding pixels in the two images;

[0090] 2) Calculate the uniform color space color difference between the corresponding pixels of the two images;

[0091] 3) Structural Similarity (SSIM) and other image texture structure comparison functions;

[0092] 4) Image comparison model based on artificial neural network.

[0093] IV) Image Anomaly Detection Device Proposed in an Alternative Embodiment of the Present Invention

[0094] Figure 2 : is a schematic diagram of the structure of a model training device provided according to an optional embodiment of the present invention, such as Figure 2 As shown, the model training device includes: a training image loading module, a noise level sampling module, a first noise addition module, a first noise reduction module, a loss calculation module and a network weight update module.

[0095] Training image loading module: The training image loading module randomly obtains N images from the training set. N is the number of images in the batch. The training set can only contain normal image samples.

[0096] Noise level sampling module: The noise level sampling module randomly samples N noise levels from the range [1, T] with an average distribution.

[0097] The first noise adding module: The first noise adding module calls formula (1) to generate N noise images.

[0098] First denoising module: The first denoising module calls the U-Net artificial neural network to predict the noise and image reconstruction unreliability of each of N images.

[0099] Loss calculation module: The loss calculation module uses the above formulas (2)(3)(4)(5) to calculate the loss and output the model update gradient.

[0100] Network weight update module: The network weight update module uses a preset artificial neural network optimization method (such as stochastic gradient descent (SGD)) to update the weights of U-Net.

[0101] Figure 3 is a schematic structural diagram of an abnormal image prediction device according to an optional embodiment of the present invention. Figure 3 As shown, the abnormal image prediction device includes: an image acquisition module, a target detection and cutout module, a second noise addition module, a second noise reduction module, an image reconstruction module, an image comparison module and an abnormality judgment module. The device is described below.

[0102] Image acquisition module: The image acquisition module can obtain video streams or image files from cameras or storage devices (such as DVRs), decode the video streams or image files, and output full-frame image data.

[0103] Object Detection Cutout Module: This module uses an object detection algorithm (such as YOLOv5 or YOLOv7) to extract the target region of the image (e.g., detecting an insulator region from a full-frame image captured by a drone) where anomalies (defects) are to be detected. A small image is then cut out of the full-frame image based on the detected target region and scaled to a suitable size for input to the U-Net. The resulting image is the original image x0.

[0104] Second Noise Adding Module: The second noise adding module calls formula (1) to generate N noise images. (Note that N here is different from N in the model training device. N here refers to the noise level set Number of levels in

[0105] Second denoising module: The second denoising module uses the U-Net artificial neural network to predict the noise and image reconstruction unreliability of each of N images.

[0106] Image reconstruction module: The image reconstruction module uses formula (6) (8) to calculate the reconstructed image

[0107] Image comparison module: The image comparison module uses formula (9) to calculate the abnormality score map.

[0108] Anomaly Judgment Module: This module identifies abnormal image regions based on whether the abnormality score of each pixel in the anomaly score map exceeds a preset threshold. Optionally, an image morphology algorithm can be used to process abnormal image regions to eliminate outliers that may be false positives. Alternatively, an image connected domain analysis algorithm can be used to determine the connected domain of the abnormal region. Various rules can be used to determine the presence of anomalies (defects) in the original image, including but not limited to the presence of an abnormal region, whether the number of pixels in the abnormal region exceeds a specific threshold, and whether the area or size of the abnormal connected domain exceeds a specific threshold.

[0109] In an embodiment of the present invention, an image anomaly detection device is also provided. Figure 4 FIG. 1 is a structural block diagram of an image anomaly detection device according to an embodiment of the present invention. Figure 4 As shown, the device includes: an acquisition module 42, a processing module 44, a prediction module 46, a generation module 48 and a detection module 40. The device is described below.

[0110] An acquisition module 42 is used to acquire a target image; a processing module 44 is connected to the acquisition module 42 and is used to perform noise addition processing on the target image at N noise levels to obtain N noise images, where N is an integer greater than 1; a prediction module 46 is connected to the processing module 44 and is used to predict the target image before noise addition based on the N noise images to obtain N predicted images and image reconstruction unreliability corresponding to the N predicted images; a generation module 48 is connected to the prediction module 46 and is used to generate a reconstructed image based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images; a detection module 40 is connected to the generation module 48 and is used to perform image anomaly detection on the target image based on the target image and the reconstructed image to obtain an image anomaly detection result.

[0111] It should be noted that the above-mentioned acquisition module 42 may partially correspond to the above-mentioned image acquisition module and target detection cutout module, the above-mentioned processing module 44 may partially correspond to the above-mentioned second noise addition module, the above-mentioned prediction module 46 may partially correspond to the above-mentioned second noise reduction module, the generation module 48 may partially correspond to the above-mentioned image reconstruction module, and the detection module 40 may partially correspond to the above-mentioned image comparison module and abnormality judgment module.

[0112] In an embodiment of the present invention, a computer-readable storage medium is further provided, which includes a stored executable program, wherein when the executable program runs, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned image anomaly detection methods.

[0113] In an embodiment of the present invention, an electronic device is provided. Figure 5 is a structural block diagram of an electronic device provided according to an embodiment of the present invention, such as Figure 5 As shown, the electronic device includes: a memory 52 storing an executable program; a processor 54 for running the program, wherein the program executes any one of the above-mentioned image anomaly detection methods when running.

[0114] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0115] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0117] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0118] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0120] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting anomalies in an image, characterized in that: include: Acquire the target image; Performing noise processing on the target image at N noise levels to obtain N noisy images, where N is an integer greater than 1; Based on the N noise images, predict the target image before adding noise, and obtain N predicted images and image reconstruction unreliability corresponding to the N predicted images respectively; generating a reconstructed image based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images; performing image anomaly detection on the target image based on the target image and the reconstructed image to obtain an image anomaly detection result; Wherein, the generating of the reconstructed image based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images respectively includes: determining the image reconstruction reliability corresponding to the N predicted images respectively based on the image reconstruction unreliability corresponding to the N predicted images respectively; for any target predicted image among the N predicted images, obtaining a first weighted result of the target predicted image based on the target predicted image and the image reconstruction reliability corresponding to the target predicted image, and obtaining a second weighted result of the target predicted image based on the target image and the image reconstruction unreliability corresponding to the target predicted image; summing the first weighted result and the second weighted result to obtain a target weighted result of the target predicted image; obtaining the target weighted results corresponding to the N predicted images respectively by adopting the method of obtaining the target weighted result from the target predicted image; summing the target weighted results corresponding to the N predicted images to obtain the reconstructed image.

2. The method according to claim 1, characterized in that The step of performing noise addition processing at N noise levels on the target image to obtain N noisy images includes: selecting a noise level from a predetermined number T of noise levels to obtain N noise levels; The noises of the N noise levels are respectively superimposed on the target image to obtain N noise images.

3. The method according to claim 2, characterized in that The noise level selected from the predetermined number T of noise levels can mask abnormal textures of a predetermined scale in the target image without completely masking information of the target object in the target image.

4. The method according to any one of claims 1 to 3, characterized in that The step of predicting the target image before adding noise based on the N noise images to obtain N predicted images and image reconstruction unreliability corresponding to the N predicted images includes: The N noisy images are respectively input into the image denoising model to predict the target image before adding noise, and N predicted images and the image reconstruction unreliability corresponding to the N predicted images are obtained respectively, wherein the image denoising model is trained based on image sample data, and the image sample data includes: a sample image and a sample predicted image corresponding to the sample image and the image reconstruction unreliability corresponding to the sample predicted image.

5. The method according to claim 4, characterized in that Before inputting the N noisy images into an image denoising model, predicting the target image before adding noise, and obtaining N predicted images and image reconstruction unreliability corresponding to the N predicted images, the method further includes: Constructing an initial denoising model and a loss function corresponding to the initial denoising model, wherein the loss function includes a first loss term and a second loss term, the first loss term is used to mark a loss value of image noise, and the second loss term is used to mark a loss value of image reconstruction unreliability; Based on the loss function, the initial denoising model is trained using the image sample data to obtain the image denoising model.

6. An image anomaly detection device, characterized in that: include: An acquisition module, used to acquire a target image; a processing module, configured to perform noise addition processing at N noise levels on the target image to obtain N noisy images, where N is an integer greater than 1; A prediction module, configured to predict the target image before noise addition based on the N noise images, to obtain N predicted images and image reconstruction unreliability corresponding to the N predicted images; a generating module, configured to generate a reconstructed image based on the N predicted images and the image reconstruction unreliability corresponding to the N predicted images; a detection module, configured to perform image anomaly detection on the target image based on the target image and the reconstructed image, and obtain an image anomaly detection result; Among them, the generation module is also used to determine the image reconstruction credibility corresponding to the N predicted images respectively based on the image reconstruction unreliability corresponding to the N predicted images respectively; for any target predicted image among the N predicted images, based on the target predicted image and the image reconstruction credibility corresponding to the target predicted image, obtain a first weighted result of the target predicted image, and based on the target image and the image reconstruction unreliability corresponding to the target predicted image, obtain a second weighted result of the target predicted image; sum the first weighted result and the second weighted result to obtain the target weighted result of the target predicted image; adopt the method of obtaining the target weighted result of the target predicted image to obtain the target weighted results corresponding to the N predicted images respectively; sum the target weighted results corresponding to the N predicted images to obtain the reconstructed image.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the computer-readable storage medium is located is controlled to execute the image anomaly detection method according to any one of claims 1 to 6.

8. An electronic device, characterized in that: include: a memory storing an executable program; A processor is configured to run the program, wherein the image anomaly detection method according to any one of claims 1 to 6 is executed when the program is run.

Images