An abnormal image generation method, system, computer and storage medium

Abstract

This invention relates to the field of sample generation technology, and provides an anomaly image generation method, system, computer, and storage medium. The anomaly image generation method includes: acquiring a normal image and a ReFlow model; encoding the normal image into a source latent representation; inversely converting the source latent representation into an inverse latent representation; acquiring anomaly cue words and a coarse anomaly mask; performing low-frequency structure-preserving processing on the inverse latent representation to obtain an updated latent representation; performing high-frequency anomaly detail synthesis processing on the updated latent representation to form a final latent representation; and decoding the final latent representation into an anomaly image. By employing the above method, high-quality anomaly samples can be generated even with a small sample size.

Description

Technical Field

[0001] This invention relates to the field of sample generation technology, and in particular to an abnormal image generation method, system, computer, and storage medium. Background Technology

[0002] In industrial inspection, medical imaging, and other scientific analysis fields, anomaly detection is a core task for ensuring system reliability and security. Especially for the detection of rare defects, accurate anomaly sample generation is a crucial step. However, anomaly samples are scarce and difficult to label, making the acquisition of sufficient labeled samples often extremely challenging. Therefore, generating realistic and diverse anomaly samples under limited labeled sample conditions has become key to improving the robustness and accuracy of downstream detection tasks.

[0003] Traditional anomaly generation methods often rely on manually designed heuristic rules, such as cropping, pasting, and texture synthesis. While these methods can generate certain anomalous images, their lack of contextual consistency and structure preservation often results in visually unnatural and semantically drifting anomalous samples, failing to effectively improve the generalization ability of downstream models.

[0004] In existing technologies, generative models such as diffusion models and ReFlow models have gradually become effective means of generating anomalies. However, existing methods still face problems such as insufficient structure preservation and poor distribution consistency. Especially in small sample settings, due to the lack of sufficient labeled data for retraining, there is a significant distribution shift between the generated anomaly samples and the target domain. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide an anomaly image generation method, system, computer, and storage medium. This invention seeks to solve the technical problems in existing technologies where, in the case of small sample sizes, the generated anomaly samples exhibit significant distributional offset and insufficient structure preservation compared to the target region.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution: An abnormal image generation method includes the following steps: Obtain a normal image and a ReFlow model, encode the normal image into a source latent representation, and based on the ReFlow model, inversely convert the source latent representation into an inverse latent representation; Obtain anomaly warning words and a coarse anomaly mask. Based on the source latent representation, the anomaly warning words, and the coarse anomaly mask, perform low-frequency structure preservation processing on the inverse latent representation to obtain an updated latent representation. The updated latent representation is subjected to high-frequency anomalous detail synthesis processing to form the final latent representation; The final latent representation is decoded into an anomalous image.

[0007] Furthermore, the normal image is encoded into the source latent representation using a variational autoencoder, and the final latent representation is decoded into the abnormal image using a variational autodecoder.

[0008] Furthermore, the step of inversely converting the source latent representation into an inverse latent representation based on the ReFlow model includes: Obtain the velocity prediction network of the ReFlow model; Based on the velocity prediction network and its first derivative approximation term, the source latent representation is iteratively updated several times to form the inverse latent representation.

[0009] Furthermore, the formula used for iterative updates is:

[0010] in, Indicates after iterative update The potential representation of the source of time, Indicates before iterative update The potential representation of the source of time, This indicates a speed prediction network. This represents the first derivative approximation term of the velocity prediction network.

[0011] Furthermore, the step of performing low-frequency structure-preserving processing on the inverse latent representation based on the source latent representation, the anomaly cue words, and the coarse anomaly mask to obtain an updated latent representation includes: The inverse latent representation is denoised based on the abnormal prompt words to form a denoised latent representation; Based on the source latent representation and the inverse latent representation, calculate the linear interpolation latent representation; Based on the coarse anomaly mask, the normal background of the linear interpolation latent representation is obtained, the anomalous foreground of the denoised latent representation is obtained, and the normal background and the anomalous foreground are fused to obtain an updated latent representation.

[0012] Furthermore, the formula for the potential representation of the linear interpolation is:

[0013] in, express The latent linear interpolation representation at time t, Represents the potential representation of the source. This represents the reverse latent representation; The formula for updating the latent representation is:

[0014] in, This indicates an update to the potential representation. Indicates an abnormal prospect. Indicates a normal background;

[0015] in, Indicates a rough anomaly mask. This indicates element-wise multiplication. This represents the latent representation for denoising;

[0016] Furthermore, the step of performing high-frequency anomalous detail synthesis processing on the updated latent representation to form the final latent representation includes: Extract normal background keys and normal background values ​​from the linear interpolation latent representation; Extract the foreground query, foreground key, and foreground value from the updated potential representation; Based on the normal background key, the normal background value, the foreground query, the foreground key, and the foreground value, self-attention is calculated to synthesize high-frequency anomalous details to form the final latent representation.

[0017] An abnormal image generation system, employing the abnormal image generation method described in the above technical solution, the system comprising: The reverse module is used to acquire a normal image and a ReFlow model, encode the normal image into a source latent representation, and reversely convert the source latent representation into a reverse latent representation based on the ReFlow model. The low-frequency module is used to acquire anomaly prompt words and a coarse anomaly mask, and to perform low-frequency structure preservation processing on the inverse latent representation based on the source latent representation, the anomaly prompt words and the coarse anomaly mask to obtain an updated latent representation; A high-frequency module is used to perform high-frequency anomalous detail synthesis processing on the updated latent representation to form the final latent representation; A decoding module is used to decode the final latent representation into an anomalous image.

[0018] A computer includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the abnormal image generation method as described above.

[0019] A storage medium storing a computer program that, when executed by a processor, implements the abnormal image generation method as described in any of the above technical solutions.

[0020] Compared with existing technologies, the advantages of this invention are as follows: By introducing a training-free inverse process based on the ReFlow model, the statistical structural information of the normal image is incorporated into the generation process, which helps to alleviate the distribution offset problem and maintain the structural consistency of the generated image; the generation process of abnormal images is conceptually decoupled into preserving the normal background and generating abnormal foreground. By dividing the denoising process into two stages, low-frequency structure preservation and high-frequency detail synthesis, spatial masking technology is used to display and fuse the normal background area during low-frequency processing, thereby ensuring that the normal background structure is not destroyed. During high-frequency processing, a correction attention mechanism is used to decouple the foreground information and background information in the latent representation of the image to achieve high-fidelity abnormal detail synthesis, ensuring that the generated abnormal details maintain contextual consistency with the background, thereby synthesizing more realistic abnormal images. Compared with traditional training-based generation methods, the method of this invention does not require additional training. Through the inverse flow model and frequency-aware decoupling strategy, high-quality abnormal images can be generated even with small sample settings, avoiding generation bias caused by insufficient training data. Attached Figure Description

[0021] Figure 1 This is a flowchart of the abnormal image generation method in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the abnormal image generation method in the first embodiment of the present invention; Figure 3 This is a structural block diagram of the abnormal image generation system in the second embodiment of the present invention; The following detailed description, in conjunction with the accompanying drawings, will further illustrate the present invention. Detailed Implementation

[0022] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Several embodiments of the invention are illustrated in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

[0023] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there may be an intervening component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there may be an intervening component. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this document are for illustrative purposes only.

[0024] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0025] Please see Figure 1 The abnormal image generation method in the first embodiment of the present invention includes the following steps: Step S10: Obtain a normal image and a ReFlow model, encode the normal image into a source latent representation, and based on the ReFlow model, inversely convert the source latent representation into an inverse latent representation; In step S10, the normal image is encoded into the source latent representation using a variational autoencoder.

[0026] Preferably, this embodiment uses a pre-trained ReFlow model to perform a training-free inverse process. The ReFlow model employs a technique for accelerating diffusion model sampling, which is an optimization method under the flow matching framework. The normal image is encoded by the variational autoencoder (VAE), which can incorporate the structural and distribution information of the normal sample into the generation process. When mapping the source latent representation to the inverse latent representation that follows a standard normal distribution, the RF-Solver inversion process is specifically used.

[0027] Step S10 includes: S110: Obtain the velocity prediction network of the ReFlow model; S120: Based on the velocity prediction network and the first derivative approximation term of the velocity prediction network, the source latent representation is iteratively updated several times to form the inverse latent representation; The formula used for iterative updates is:

[0028] in, Indicates after iterative update The potential representation of the source of time, Indicates before iterative update The potential representation of the source of time, This indicates a speed prediction network. This represents the first derivative approximation term of the velocity prediction network.

[0029] Preferably, the first derivative approximation term of the velocity prediction network is used to correct the discretization error, which can effectively alleviate the distribution shift caused by the Reflow model not being trained in the target domain.

[0030] Step S20: Obtain the anomaly prompt word and the coarse anomaly mask. Based on the source latent representation, the anomaly prompt word and the coarse anomaly mask, perform low-frequency structure preservation processing on the inverse latent representation to obtain the updated latent representation. Preferably, the coarse anomaly mask is used to indicate and mark the approximate location of the anomaly, and the anomaly message is used to describe the anomaly in the image. Please refer to [link / reference]. Figure 2 For example, in this embodiment, the abnormal prompt word is "A photo of a bottle with broken on it". The coarse abnormality mask is a binarized mask. The original latent feature in the image is the source latent representation. The coarse abnormality mask is obtained by interpolation. The image information processing specifically adopts the MM-DiT architecture.

[0031] Step S20 includes: S210: Denoise the inverse latent representation based on the abnormal prompt words to form a denoised latent representation; Preferably, the denoising process is divided into early denoising and late denoising, which is determined by setting a time threshold. Please refer to [link / reference]. Figure 2 T represents the time threshold. In the early denoising process, the noise level is relatively high, which mainly determines the low-frequency structure and global semantics of the image. The abnormal prompt words and the inverse latent representation are both inputs to the MM-DiT module. The MM-DiT module can process both text and image modalities to achieve better information transmission and integration.

[0032] S220: Calculate the linear interpolation latent representation based on the source latent representation and the inverse latent representation; Preferably, spatial masking is used to reveal the decoupled and fused potential representations.

[0033] S230: Based on the coarse anomaly mask, obtain the normal background of the linear interpolation latent representation, obtain the anomaly foreground of the denoised latent representation, and fuse the normal background and the anomaly foreground to obtain an updated latent representation.

[0034] Preferably, the normal background portion of the linear interpolation latent representation is preserved by using the coarse anomaly mask, and the anomaly foreground portion of the denoised latent representation is used for synthesis. Understandably, the preservation of low-frequency structures is achieved through latent fusion in physical space, which can effectively prevent the corruption of the normal background region structure.

[0035] The formula for the potential representation of the linear interpolation is:

[0036] in, express The latent linear interpolation representation at time t, Represents the potential representation of the source. This represents the reverse latent representation; The formula for updating the latent representation is:

[0037] in, This indicates an update to the potential representation. Indicates an abnormal prospect. Indicates a normal background;

[0038] in, Indicates a rough anomaly mask. This indicates element-wise multiplication. This represents the latent representation for denoising;

[0039] Step S30: Perform high-frequency anomalous detail synthesis processing on the updated latent representation to form the final latent representation; Preferably, high-frequency anomalous detail synthesis is performed in the later denoising process. The noise level is low in the later denoising process, which mainly determines the high-frequency details and local realism of the image. In this embodiment, the foreground and background information in the latent representation of the image are decoupled by a correction attention mechanism to achieve high-fidelity anomalous detail synthesis while ensuring contextual consistency.

[0040] Step S30 includes: S310: Extract the normal background key and normal background value from the linear interpolation latent representation; S320: Extract the foreground query, foreground key, and foreground value from the updated latent representation; Preferably, the foreground query is extracted based on the abnormal foreground, and the foreground key and the foreground value integrate the complete contextual information of the foreground and background.

[0041] S330: Based on the normal background key, the normal background value, the foreground query, the foreground key, and the foreground value, calculate self-attention to synthesize high-frequency anomalous details based on the self-attention, so as to form the final latent representation.

[0042] Preferably, the self-attention is calculated based on the softmax function, and the corrected attention is only applied to the self-attention calculation of the abnormal foreground markers. This makes the synthesis of local anomalies more focused, while ensuring the semantic consistency and structural coherence of the generated details with the surrounding environment, thereby enabling the synthesis of high-fidelity abnormal details.

[0043] Step S40: Decode the final latent representation into an anomalous image. In step S40, the final latent representation is decoded into the anomalous image using a variational autodecoder.

[0044] Understandably, compared with traditional training-based generation methods, the method of the present invention does not require additional training. Through the inverse flow model and frequency-aware decoupling strategy, it can generate high-quality anomalous images even with a small sample size, avoiding generation bias caused by insufficient training data.

[0045] Preferably, based on the anomalous image generation method in this embodiment, comparative experiments are conducted with existing NAS, AnomalyDiffusion, Dual AnoDiff, and Anomaly Any methods. The experiments are based on the widely used MVTec AD benchmark dataset, which contains 15 categories (including 10 object types and 5 texture types). Each category contains a large number of normal images and approximately 20 anomalous samples with pixel-level annotations. The dataset is used to evaluate the quality of the generated images and their performance in anomaly detection tasks. The Inception Score (IS) is used to evaluate the fidelity of the anomalous sample images, and intra-cluster pairwise LPIPS (IC-LPIPS) is used to evaluate the diversity of the anomalous sample images. The results of the comparative experiments are shown in the table below:

[0046] In terms of the IS metric, the method in this embodiment demonstrates excellent image quality, reaching 9.82, showing a significant advantage over other methods. Compared to AnomalyAny (8.97) and DualAnoDiff (8.56), the method in this embodiment exhibits stronger capabilities in terms of image realism and sharpness. This indicates that the generated anomaly images are visually closer to real defect images, providing more reliable training samples for downstream anomaly detection tasks. In terms of the IC-LPIPS metric, the method in this embodiment achieves 0.56, compared to other benchmark methods such as NSA (0.43), DualAnoDiff (0.49), and A The value of nomalyAny (0.45) demonstrates a significant advantage, indicating that the method in this embodiment can generate more diverse anomalous samples while preserving image quality, thereby improving the generation capability and sample diversity of anomalous images. This advantage is particularly prominent when generating images with complex structures and textures, enabling the generated anomalous images to exhibit higher adaptability and generalization capabilities in different industrial application scenarios. The excellent performance of the anomalous image generation method in this embodiment on both IS and IC-LPIPS metrics proves its significant advantages in image quality and diversity, and provides a more efficient and stable solution for small-sample anomalous generation tasks.

[0047] Please see Figure 3 The abnormal image generation system provided in the second embodiment of the present invention applies the abnormal image generation method described in the first embodiment above, and the system includes: The reverse module 10 is used to acquire a normal image and a ReFlow model, encode the normal image into a source latent representation, and reversely convert the source latent representation into a reverse latent representation based on the ReFlow model. In the reverse module 10, the normal image is encoded into the source latent representation using a variational autoencoder.

[0048] The reverse module 10 includes: The first unit is used to obtain the velocity prediction network of the ReFlow model; The second unit is used to perform several iterative updates on the source latent representation based on the velocity prediction network and the first derivative approximation term of the velocity prediction network to form the inverse latent representation. The formula used for iterative updates is:

[0049] in, Indicates after iterative update The potential representation of the source of time, Indicates before iterative update The potential representation of the source of time, This indicates a speed prediction network. This represents the first derivative approximation term of the velocity prediction network.

[0050] Low-frequency module 20 is used to acquire anomaly prompt words and coarse anomaly masks, and based on the source latent representation, the anomaly prompt words and coarse anomaly masks, to perform low-frequency structure preservation processing on the inverse latent representation to obtain an updated latent representation; The low-frequency module 20 includes: The third unit is used to denoise the inverse latent representation based on the abnormal prompt words to form a denoised latent representation; The fourth unit is used to calculate the linear interpolation latent representation based on the source latent representation and the inverse latent representation; The fifth unit is used to obtain the normal background of the linear interpolation latent representation and the abnormal foreground of the denoised latent representation based on the coarse anomaly mask, and to fuse the normal background and the abnormal foreground to obtain an updated latent representation.

[0051] The formula for the potential representation of the linear interpolation is:

[0052] in, express The latent linear interpolation representation at time t, Represents the potential representation of the source. This represents the reverse latent representation; The formula for updating the latent representation is:

[0053] in, This indicates an update to the potential representation. Indicates an abnormal prospect. Indicates a normal background;

[0054] in, Indicates a rough anomaly mask. This indicates element-wise multiplication. This represents the latent representation for denoising;

[0055] The high-frequency module 30 is used to perform high-frequency anomalous detail synthesis processing on the updated latent representation to form the final latent representation; The high-frequency module 30 includes: The sixth unit is used to extract the normal background key and normal background value from the linear interpolation latent representation; The seventh unit is used to extract the foreground query, foreground key, and foreground value from the updated latent representation; The eighth unit is used to calculate self-attention based on the normal background key, the normal background value, the foreground query, the foreground key, and the foreground value, so as to perform high-frequency anomalous detail synthesis based on the self-attention to form the final latent representation.

[0056] Decoding module 40 is used to decode the final latent representation into an anomalous image.

[0057] In the decoding module 40, the final latent representation is decoded into the anomalous image using a variational autodecoder.

[0058] A third embodiment of the present invention provides a computer, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the abnormal image generation method as described in any one of the first embodiments.

[0059] The fourth embodiment of the present invention provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the abnormal image generation method as described in any one of the first embodiments.

[0060] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0061] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for generating abnormal images, characterized in that, Includes the following steps: Obtain a normal image and a ReFlow model, encode the normal image into a source latent representation, and based on the ReFlow model, inversely convert the source latent representation into an inverse latent representation; Obtain anomaly warning words and a coarse anomaly mask. Based on the source latent representation, the anomaly warning words, and the coarse anomaly mask, perform low-frequency structure preservation processing on the inverse latent representation to obtain an updated latent representation. The updated latent representation is subjected to high-frequency anomalous detail synthesis processing to form the final latent representation; The final latent representation is decoded into an anomalous image.

2. The abnormal image generation method according to claim 1, characterized in that, The normal image is encoded into the source latent representation using a variational autoencoder, and the final latent representation is decoded into the abnormal image using a variational autodecoder.

3. The abnormal image generation method according to claim 1, characterized in that, The step of inversely converting the source latent representation into an inverse latent representation based on the ReFlow model includes: Obtain the velocity prediction network of the ReFlow model; Based on the velocity prediction network and its first derivative approximation term, the source latent representation is iteratively updated several times to form the inverse latent representation.

4. The abnormal image generation method according to claim 3, characterized in that, The formula used for iterative updates is: in, Indicates after iterative update The potential representation of the source of time, Indicates before iterative update The potential representation of the source of time, This indicates a speed prediction network. This represents the first derivative approximation term of the velocity prediction network.

5. The abnormal image generation method according to claim 1, characterized in that, The step of performing low-frequency structure preservation processing on the inverse latent representation based on the source latent representation, the anomaly cue words, and the coarse anomaly mask to obtain an updated latent representation includes: The inverse latent representation is denoised based on the abnormal prompt words to form a denoised latent representation; Based on the source latent representation and the inverse latent representation, calculate the linear interpolation latent representation; Based on the coarse anomaly mask, the normal background of the linear interpolation latent representation is obtained, the anomalous foreground of the denoised latent representation is obtained, and the normal background and the anomalous foreground are fused to obtain an updated latent representation.

6. The abnormal image generation method according to claim 5, characterized in that, The formula for the potential representation of the linear interpolation is: in, express The latent linear interpolation representation at time t, Represents the potential representation of the source. This represents the reverse latent representation; The formula for updating the latent representation is: in, This indicates an update to the potential representation. Indicates an abnormal prospect. Indicates a normal background; in, Indicates a rough anomaly mask. This indicates element-wise multiplication. This represents the latent representation for denoising; 。 7. The abnormal image generation method according to claim 5, characterized in that, The step of performing high-frequency anomalous detail synthesis on the updated latent representation to form the final latent representation includes: Extract normal background keys and normal background values ​​from the linear interpolation latent representation; Extract the foreground query, foreground key, and foreground value from the updated potential representation; Based on the normal background key, the normal background value, the foreground query, the foreground key, and the foreground value, self-attention is calculated to synthesize high-frequency anomalous details to form the final latent representation.

8. An abnormal image generation system, employing the abnormal image generation method as described in any one of claims 1 to 7, characterized in that, The system includes: The reverse module is used to acquire a normal image and a ReFlow model, encode the normal image into a source latent representation, and reversely convert the source latent representation into a reverse latent representation based on the ReFlow model. The low-frequency module is used to acquire anomaly prompt words and a coarse anomaly mask, and to perform low-frequency structure preservation processing on the inverse latent representation based on the source latent representation, the anomaly prompt words and the coarse anomaly mask to obtain an updated latent representation; A high-frequency module is used to perform high-frequency anomalous detail synthesis processing on the updated latent representation to form the final latent representation; A decoding module is used to decode the final latent representation into an anomalous image.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the abnormal image generation method as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the abnormal image generation method as described in any one of claims 1 to 7.

Images