A method, system, device and medium for generating diversified abnormal image data
By combining Markov random field approximation models and generative adversarial networks, diverse anomalous image data is generated, solving the problems of single generated samples and computational complexity in existing technologies, and realizing the efficient generation of anomalous images with multiple anomalous patterns.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI UNIV
- Filing Date
- 2024-06-18
- Publication Date
- 2026-08-04
AI Technical Summary
Existing methods for generating anomalous images lack diversity in the samples they produce, fail to fully reflect the original data, and have high computational complexity, relying on external auxiliary datasets or cumbersome pre-trained models.
By combining a Markov random field approximation model with a generative adversarial network, diverse anomalous image data is generated through adversarial training of the generator and discriminator. Fake pixel patches are generated by utilizing the edge distribution of real pixel patches, and random noise is introduced to generate various anomalous patterns.
It generates diverse anomalous image data, covering different anomalous patterns, using only normal data, without relying on external auxiliary datasets or heavy pre-trained models, and has high computational efficiency.
Smart Images
Figure CN118644745B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of anomalous image generation technology, and in particular to a method, system, device and medium for generating diverse anomalous image data. Background Technology
[0002] In recent years, various deep generative models have emerged, among which the mainstream deep generative models include Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and diffusion models. In the field of industrial image detection, detecting the presence and nature of anomalies in images is crucial; therefore, there is a significant demand for diverse anomaly image data. Existing anomaly image generation methods mainly include:
[0003] 1) A Generative Adversarial Network (GAN) consists of a generator and a discriminator. The generator uses random values sampled from a normal distribution to produce synthetic samples, while the discriminator attempts to distinguish between real and generated samples. The generator is trained to produce realistic outputs that can fool the discriminator, while the discriminator is trained to correctly distinguish between real and generated data, thus creating an adversarial process. Because the adversarial loss cannot cover the entire data distribution, it is prone to pattern collapse, meaning the generator tends to generate only a subset of samples from the data distribution, resulting in a lack of diversity in the generated samples and an inability to fully reflect the original data.
[0004] 2) In a variational autoencoder (VAE), the encoder maps high-dimensional input data to a low-dimensional representation, and the decoder reconstructs the original high-dimensional input data by mapping this representation back to its original form. Because VAEs use random encoding and decoding processes, the generated images often have some degree of blurring or distortion.
[0005] 3) The diffusion model consists of forward diffusion and backward diffusion processes. Forward diffusion is a Markov chain that gradually adds noise to the input data until white noise is obtained. The backward diffusion process uses a trainable neural network to reverse the forward process and gradually remove noise to recover the original data. Since the diffusion model usually requires multiple iterations to generate samples, the computational complexity of the generation process is high, and sample generation is slow.
[0006] In summary, existing techniques rely on external auxiliary datasets to generate anomalous data, or they require complex pre-trained models. Furthermore, the generated anomalous data is often quite homogeneous and fails to encompass all anomalous patterns. Summary of the Invention
[0007] The purpose of this invention is to provide a method, system, device, and medium for generating diverse abnormal image data that generates various abnormal images.
[0008] The objective of this invention can be achieved through the following technical solutions:
[0009] A method for generating diverse anomalous image data includes the following steps:
[0010] Normal image data is acquired, preprocessed, and segmented into real pixel patches. The patches are then processed using a combination of a Markov random field approximation model and a pre-trained generative adversarial network to generate diverse abnormal image data. The generative adversarial network includes a generator and a discriminator.
[0011] The specific steps for generating diverse anomalous image data include:
[0012] Based on the real pixel patch, the edge distribution of fake pixel patches at different levels is obtained by using a Markov random field approximation model, wherein the edge distribution of the fake pixel patch is consistent with the edge distribution of the real pixel patch.
[0013] By introducing a random noise vector, inputting it into generators of different levels, and combining it with the edge distribution of the corresponding level of fake pixel patches, different levels of fake pixel patches are generated, and further, diverse anomalous image data are generated.
[0014] Furthermore, the preprocessing steps include:
[0015] The normal image data is uniformly scaled to a fixed size and then standardized.
[0016] Furthermore, the expression for the Markov random field approximation model is:
[0017]
[0018] In the formula, c∈C w It is a pixel patch of size w*w. Given a distribution The w-th order Markov random field approximation, Given an image x c The distribution of φ c (x c ) represents the edge density function of a pixel patch.
[0019] Furthermore, the training process of the generative adversarial network includes:
[0020] Initialization: Randomly initialize the parameters of the generator and discriminator;
[0021] Generating anomalous images: A generator and a Markov random field approximation model are used to generate anomalous image data;
[0022] Constructing training samples: Obtain normal image data and combine it with the generated abnormal image data to form training samples;
[0023] Training the discriminator: The discriminator's loss function is used to enable the discriminator to distinguish between normal image data and generated abnormal image data, thereby maximizing the correct recognition rate of normal image data;
[0024] Generator and discriminator adversarial: The generator's loss function is used to guide the generator to generate abnormal image data to deceive the discriminator's judgment, so that the discriminator continuously improves the correct recognition rate of normal image data;
[0025] Alternating training: The discriminator and generator are trained alternately. In each training phase, the discriminator is trained first, and then the generator is trained using the updated discriminator.
[0026] Backpropagation and parameter update: The parameters of the generator and discriminator are updated using the backpropagation algorithm until the preset conditions are met, at which point the training process ends.
[0027] Furthermore, the loss function of the discriminator is:
[0028]
[0029] In the formula, For the adversarial loss of discriminator D, |C w | indicates the number of pixel patches, that is, the number of blocks into which a normal image is divided. This represents the expected probability that the discriminator D outputs the true value on pixel patch c in a normal image x. Let z represent the expected value of the probability that the discriminator D outputs a false value on pixel patch c under the generated abnormal image G(z), where z is the noise vector input to the generator G.
[0030] Furthermore, the loss function of the generator is:
[0031]
[0032] In the formula, For the total loss of the life-separating device G, I represents the adversarial loss of the differential G, β is the balance coefficient used to weigh the effects of the adversarial loss and the mutual information term. Θ (G(z),Z) represents the mutual information between the generated image G(z) and the input noise Z, where G(z) is the generated anomalous image, and |C w | indicates the number of pixel patches, that is, the number of blocks into which the image is divided. Let D be the expected value of the probability that the discriminator D outputs a true value on pixel patch c in the generated anomalous image G(z). It is a transformation function used to calculate mutual information. This represents the expected value of the transformation function T under the joint distribution p(X,Z). Let T be the expected value of the exponent of the transformation function T under distributions p(X) and p(Z).
[0033] Furthermore, the diverse abnormal image data includes multiple abnormal patterns, namely structural abnormal patterns and logical abnormal patterns.
[0034] The present invention also provides a system for generating diverse abnormal image data, comprising:
[0035] Image acquisition module: used to acquire normal image data, preprocess it, and segment it into real pixel patches;
[0036] Anomaly image output module: Based on the real pixel patch, it processes the data using a combination of Markov random field approximation model and a pre-trained generative adversarial network to generate diverse anomaly image data, wherein the generative adversarial network includes a generator and a discriminator;
[0037] The specific steps for generating diverse anomalous image data include:
[0038] Based on the real pixel patch, the edge distribution of fake pixel patches at different levels is obtained by using a Markov random field approximation model, wherein the edge distribution of the fake pixel patch is consistent with the edge distribution of the real pixel patch.
[0039] By introducing a random noise vector, inputting it into generators of different levels, and combining it with the edge distribution of the corresponding level of fake pixel patches, different levels of fake pixel patches are generated, and further, diverse anomalous image data are generated.
[0040] The present invention also provides an electronic device comprising: one or more processors; a memory; and one or more programs stored in the memory, said one or more programs including instructions for performing the method for generating diverse anomalous image data as described above.
[0041] The present invention also provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, the one or more programs including instructions for performing a method for generating diverse anomalous image data as described above.
[0042] Compared with the prior art, the present invention has the following beneficial effects:
[0043] (1) Based on the idea of Markov random field approximation, this invention obtains the edge distribution of false pixel patches of different levels that are consistent with the edge distribution of real pixel patches, and the classifier of the corresponding level generates false pixel patches, so that the false pixel patches have different anomalies, and obtains abnormal images that expose different anomaly patterns.
[0044] (2) The present invention can generate diverse abnormal image data, revealing multiple abnormal patterns, thereby covering different abnormal situations.
[0045] (3) This invention can generate abnormal data using only normal data, without relying on prior knowledge of a specific field and additional auxiliary data, or using a heavy pre-trained model to produce abnormal data. Attached Figure Description
[0046] Figure 1 This is a schematic diagram of the method flow of the present invention;
[0047] Figure 2 This is a schematic diagram of the overall structure of the present invention. Detailed Implementation
[0048] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.
[0049] Example 1
[0050] This embodiment provides a method for generating diverse abnormal image data, such as Figure 1 As shown, the method includes the following steps:
[0051] 1. Data Preparation
[0052] Only normal images are needed. The images are uniformly scaled to a fixed size of 2w*2w, and all data is standardized.
[0053] 2. Data Preprocessing and Segmentation
[0054] The prepared image data is divided into real pixel patches of size w*w for subsequent training of the generator and discriminator.
[0055] 3. Markov random field approximation model
[0056] 1) Inspired by text data, letter-level recombination can expose simple spelling anomalies, while word-level recombination can expose higher-level grammatical anomalies. For example, letter-level: rdoonyaa.dd otca e,d elregggadfo / / word-level: green add lead good,go for day cat. The main idea behind exposing multi-level anomalies is to disrupt the global structure while maintaining the local structure at different levels.
[0057] 2) This embodiment uses the Markov random field approximation method to extend this idea to image data. Specifically, for each w*w real pixel patch, a Markov random field approximation model is constructed. This model ensures that the edge distribution of each patch is consistent with the corresponding part of the original image. This ensures that the generated anomalous image is very similar to the original image in local details, but due to the lack of long-distance dependencies between patches, the overall structure is disrupted, leading to global anomalies.
[0058] The Markov random field approximation model is:
[0059]
[0060] Then q X Given a distribution p X The w-th order Markov random field approximation, where p X Given the distribution of an image X, c∈C w It is a pixel patch of size w*w, φ c (x c ) represents the edge density function of the patch. The first formula indicates that for each patch block c, q X marginal distribution and p X The edge distribution is consistent. The second formula represents the high-dimensional data q. x The density function can be decomposed into the product of multiple patch block variables.
[0061] Understanding of the model:
[0062] Equation (1) comes from The abnormal image cannot be distinguished from the normal image through a window of size w*w pixels, which means that the local structure of the normal image is well preserved;
[0063] Equation (2) shows that there is no long-range interaction between the patches, which means that the global structure is broken, consistent with the idea mentioned above in this embodiment.
[0064] 4. Generate diverse anomaly data
[0065] To generate anomalous images at different levels, this embodiment proposes a generative adversarial network (GAN)-based method, which uses two neural networks (a generator and a discriminator) to compete in generating anomalous image data. Specifically... Figure 2 As shown:
[0066] Similar to common generative adversarial networks, the generator G accepts a random noise vector and transforms it into an image by learning the edge distribution features of the target Markov field.
[0067] Discriminator D evaluates the realism of each image based on patches and outputs an anomaly score D for each patch c. c The discriminator at level w has a receptive field of size w*w, allowing it to focus on determining local structures. The adversarial loss used is shown below:
[0068] A. For discriminator D, its goal is to distinguish between real images and generated anomalous images, maximizing the correct recognition rate of real images. Where |C w | indicates the number of pixel patches, that is, the number of blocks into which the image is divided. This represents the expected probability that the discriminator D outputs the true value on patch c in the context of the true image x. Let represent the expected probability that the discriminator D outputs a false value on patch c under the generated anomalous image G(z). Here, z is the noise vector input to the generator G.
[0069]
[0070] B. For generator G, the goal is to optimize the generator so that the generated images can "deceive" the discriminator as much as possible. The generator tries to maximize the probability that the discriminator will mistakenly identify the generated images as real.
[0071]
[0072] Here we maximize the entropy of the output of G to make the generated samples more closely resemble the distribution of the original data.
[0073]
[0074] The loss of the final generator G is shown below:
[0075]
[0076] The loss function is iteratively optimized until a dynamic adversarial balance is reached between the generator and the discriminator. At this point, the generator learns to introduce global anomalies while maintaining local realism, thereby effectively generating diverse anomalous data and exposing various anomaly patterns, including structural and logical anomaly patterns.
[0077] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0078] Example 2
[0079] This embodiment provides a system for generating diverse abnormal image data, including:
[0080] Image acquisition module: used to acquire normal image data, preprocess it, and segment it into real pixel patches;
[0081] Anomaly image output module: Based on the real pixel patch, it processes the data using a combination of Markov random field approximation model and a pre-trained generative adversarial network to generate diverse anomaly image data, wherein the generative adversarial network includes a generator and a discriminator;
[0082] The specific steps for generating diverse anomalous image data include:
[0083] Based on the real pixel patch, the edge distribution of fake pixel patches at different levels is obtained by using a Markov random field approximation model, wherein the edge distribution of the fake pixel patch is consistent with the edge distribution of the real pixel patch.
[0084] By introducing a random noise vector, inputting it into generators of different levels, and combining it with the edge distribution of the corresponding level of fake pixel patches, different levels of fake pixel patches are generated, and further, diverse anomalous image data are generated.
[0085] The rest are as in Example 1.
[0086] Example 3
[0087] This embodiment provides an electronic device, including: one or more processors; a memory; and one or more programs stored in the memory, said one or more programs including instructions for performing the method for generating diverse anomalous image data as described in Embodiment 1.
[0088] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.
[0089] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0091] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0092] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.
[0093] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for generating diverse abnormal image data, characterized in that, Includes the following steps: Normal image data is acquired, preprocessed, and segmented into real pixel patches. These patches are then processed using a Markov random field approximation model combined with a pre-trained generative adversarial network to generate diverse anomalous image data. The expression for the Markov random field approximation model is: ， and In the formula, It is w A pixel patch of size w Given distribution The w-th order Markov random field approximation, Given an image The distribution The edge density function representing a pixel patch; The generative adversarial network includes a generator and a discriminator, and the training process of the generative adversarial network includes: Initialization: Randomly initialize the parameters of the generator and discriminator; Generating anomalous images: A generator and a Markov random field approximation model are used to generate anomalous image data; Constructing training samples: Obtain normal image data and combine it with the generated abnormal image data to form training samples; Training the discriminator: The discriminator's loss function is used to enable the discriminator to distinguish between normal image data and generated abnormal image data, thereby maximizing the correct recognition rate of normal image data; Generator and discriminator adversarial: The generator's loss function is used to guide the generator to generate abnormal image data to deceive the discriminator's judgment, so that the discriminator continuously improves the correct recognition rate of normal image data; Alternating training: The discriminator and generator are trained alternately. In each training phase, the discriminator is trained first, and then the generator is trained using the updated discriminator. Backpropagation and parameter update: The parameters of the generator and discriminator are updated using the backpropagation algorithm until the preset conditions are met, at which point the training process ends. The specific steps for generating diverse anomalous image data include: Based on the real pixel patch, the edge distribution of fake pixel patches at different levels is obtained by using a Markov random field approximation model, wherein the edge distribution of the fake pixel patch is consistent with the edge distribution of the real pixel patch. By introducing a random noise vector, inputting it into generators of different levels, and combining it with the edge distribution of the corresponding level of fake pixel patches, different levels of fake pixel patches are generated, and further, diverse anomalous image data are generated.
2. The method for generating diverse abnormal image data according to claim 1, characterized in that, The preprocessing steps include: The normal image data is uniformly scaled to a fixed size and then standardized.
3. The method for generating diverse abnormal image data according to claim 1, characterized in that, The loss function of the discriminator is: In the formula, For the adversarial loss of discriminator D, |C w | indicates the number of pixel patches, that is, the number of blocks into which a normal image is divided. This represents the expected probability that the discriminator D outputs the true value on pixel patch c in a normal image x. Let z represent the expected value of the probability that the discriminator D outputs a false value on pixel patch c under the generated abnormal image G(z), where z is the noise vector input to the generator G.
4. The method for generating diverse abnormal image data according to claim 1, characterized in that, The loss function of the generator is: In the formula, For the total loss of the life-separating device G, For the countermeasures loss of the life-differentiation device G, This is a balancing coefficient used to weigh the effects of adversarial loss and mutual information term. To generate an image and input noise Mutual information between them For the generated anomalous image, |C w | indicates the number of pixel patches, that is, the number of blocks into which the image is divided. Let D be the expected value of the probability that the discriminator D outputs a true value on pixel patch c in the generated anomalous image G(z). It is a transformation function used to calculate mutual information. This represents the transformation function under the joint distribution p(X,Z). Expected value Let T be the expected value of the exponent of the transformation function T under distributions p(X) and p(Z).
5. The method for generating diverse abnormal image data according to claim 1, characterized in that, The diverse abnormal image data includes multiple abnormal patterns, namely structural abnormal patterns and logical abnormal patterns.
6. A system for generating diverse abnormal image data, characterized in that, include: Image acquisition module: used to acquire normal image data, preprocess it, and segment it into real pixel patches; Anomaly Image Output Module: This module generates diverse anomaly image data based on the real pixel patches, using a combination of a Markov random field approximation model and a pre-trained generative adversarial network. The expression for the Markov random field approximation model is: ， and In the formula, It is w A pixel patch of size w Given distribution The w-th order Markov random field approximation, Given an image The distribution The edge density function representing a pixel patch; The generative adversarial network includes a generator and a discriminator, and the training process of the generative adversarial network includes: Initialization: Randomly initialize the parameters of the generator and discriminator; Generating anomalous images: A generator and a Markov random field approximation model are used to generate anomalous image data; Constructing training samples: Obtain normal image data and combine it with the generated abnormal image data to form training samples; Training the discriminator: The discriminator's loss function is used to enable the discriminator to distinguish between normal image data and generated abnormal image data, thereby maximizing the correct recognition rate of normal image data; Generator and discriminator adversarial: The generator's loss function is used to guide the generator to generate abnormal image data to deceive the discriminator's judgment, so that the discriminator continuously improves the correct recognition rate of normal image data; Alternating training: The discriminator and generator are trained alternately. In each training phase, the discriminator is trained first, and then the generator is trained using the updated discriminator. Backpropagation and parameter update: The parameters of the generator and discriminator are updated using the backpropagation algorithm until the preset conditions are met, at which point the training process ends. The specific steps for generating diverse anomalous image data include: Based on the real pixel patch, the edge distribution of fake pixel patches at different levels is obtained by using a Markov random field approximation model, wherein the edge distribution of the fake pixel patch is consistent with the edge distribution of the real pixel patch. By introducing a random noise vector, inputting it into generators of different levels, and combining it with the edge distribution of the corresponding level of fake pixel patches, different levels of fake pixel patches are generated, and further, diverse anomalous image data are generated.
7. An electronic device, characterized in that, include: One or more processors; Memory; and One or more programs stored in memory, the one or more programs including instructions for performing the method for generating diverse anomalous image data as described in any one of claims 1-5.
8. A computer-readable storage medium, characterized in that, It includes one or more programs that are executed by one or more processors of an electronic device, the one or more programs including instructions for performing the method for generating diverse anomalous image data as described in any one of claims 1-5.