Generation method and device of image with defect, electronic equipment and storage medium

Abstract

The invention relates to a generation method and device of a defective image, electronic equipment and a storage medium, and the method comprises the steps: obtaining a defective sample image, a defect-free sample image, a sample control condition and a sample instruction, the sample control condition is used for indicating a defect background, a defect position and a defect form in the defective sample image, and the sample control condition is used for indicating the defect background, the defect position and the defect form in the defect-free sample image; the sample instruction is used for describing a main category and a sub-category of the defect; sequentially performing coarse training and fine training on the initial image generation model based on the defective sample graph, the defect-free sample graph, the sample control condition and the sample instruction to obtain a trained image generation model; and inputting the defect-free image, the target control condition and the target instruction into the trained image generation model to obtain an output image with defects. According to the invention, the generation efficiency of the defect graph is improved.

Description

Technical Field

[0001] The present application relates to the field of neural network technology, and in particular to a method, device, electronic device and storage medium for generating defective images. Background Art

[0002] Currently, generative adversarial networks are generally used to generate images with defects. In the generated images, there is no intuitive and clear labeling information about the defect location and range. If the images are applied to downstream tasks, such as training a defect detection model, the detection model cannot directly detect the specific location and range of the defects from these images. This requires manual annotation of the defect location, that is, marking the coordinates, shape and other information of the defects in the image and converting them into a data format that the model can understand. Only then can the detection model learn from this labeled data, grasp the characteristics and location information of the defects, and then identify defects in actual inspections. However, manual defect annotation of images will reduce the efficiency of defect map generation. Summary of the Invention

[0003] The present application provides a method, device, electronic device and storage medium for generating defective images to solve the problem of low efficiency in defect image generation.

[0004] In a first aspect, the present application provides a method for generating a defective image, the method comprising: Obtaining a defective sample image, a non-defective sample image, a sample control condition, and a sample instruction, wherein the sample control condition is used to indicate the defect background, defect location, and defect morphology in the defective sample image, and the sample instruction is used to describe the main category and subcategory of the defect; Based on the defective sample image, the non-defective sample image, the sample control conditions, and the sample instructions, coarse training and fine training are sequentially performed on the initial image generation model to obtain a trained image generation model, wherein the coarse training is used to learn the complete morphology of defects under different main categories, and the fine training is used to learn the detailed morphology of defects under different subcategories of the same main category; The defect-free image, target control conditions and target instructions are input into the trained image generation model to obtain an output defective image.

[0005] Optionally, before obtaining the sample control condition, the process of generating the sample control condition includes: Extracting a background control condition from the defective sample image, wherein the background control condition is used to indicate a portion of the background of the defective sample image that contains defects but is not marked with defects; extracting a discriminative control condition from the defective sample image, wherein the discriminative control condition is used to indicate a defect mask in the background control condition; The background control condition and the discriminative control condition are combined with the defect as a reference point to obtain the sample control condition.

[0006] Optionally, the discriminative control condition includes a primary discriminative condition and a secondary discriminative condition, and the background control condition and the discriminative control condition are combined with the defect as a reference point to obtain the sample control condition, which includes: Taking the defect as a reference point, the background control condition and the primary distinguishing condition are combined to obtain a primary control condition, wherein the primary control condition is used to indicate the background of the defect and the overall defect mask; The background control condition and the secondary distinguishing condition are merged with the defect as a reference point to obtain a secondary control condition, wherein the secondary control condition is used to indicate the background of the defect and a partial defect mask at the position of the defect detail morphology, wherein the defect detail morphology of defects of different subcategories is different.

[0007] Optionally, the sample instructions include a first-level instruction and a second-level instruction, and based on the defective sample image, the non-defective sample image, the sample control conditions, and the sample instructions, the initial image generation model is subjected to coarse training and fine training in sequence, to obtain a trained image generation model, which includes: performing coarse training on an initial image generation model using the defective sample images, the non-defective sample images, the primary control conditions, and the primary instructions, so that the image generation model learns complete forms of defects belonging to different main categories, wherein the primary instructions are used to describe the main categories of the defects; After the rough training is completed, the image generation model is finely trained using the defective sample image, the defect-free sample image, the secondary control conditions, and the secondary instructions, so that the image generation model learns the detailed morphology of defects belonging to the same main category but different subcategories, thereby obtaining a trained image generation model, wherein the secondary instructions are used to describe the subcategories of the defects.

[0008] Optionally, performing rough training on the initial image generation model using the defective sample image, the non-defective sample image, the first-level control condition, and the first-level instruction includes: Inputting the defect-free sample image, the first-level control conditions, and the first-level instructions into an initial image generation model to obtain an output defect prediction image; Determine pixel differences between the defect prediction image and the defect sample image at defective parts and non-defective parts respectively; Performing weighted summation of pixel differences in corresponding position areas according to a first weight of the defective part and a second weight of the non-defective part to obtain a loss value, wherein the first weight is greater than the second weight; The model parameters are adjusted by minimizing the loss value to obtain a roughly trained image generation model.

[0009] Optionally, fine-training the image generation model using the defective sample image, the non-defective sample image, the secondary control condition, and the secondary instruction includes: Inputting the defect-free sample image, the secondary control conditions, and the secondary instructions into the roughly trained image generation model to obtain an output defect result image; Determine pixel differences between the defect result image and the defect sample image in defect detail morphology and other remaining areas, wherein the defect detail morphology refers to detail features between defects of different subcategories under the same main category; Performing weighted summation of pixel differences in corresponding position areas according to a third weight of the defect detail morphology and a fourth weight of the other remaining areas to obtain a loss value, wherein the third weight is greater than the fourth weight; The model parameters are adjusted by minimizing the loss value to obtain a finely trained image generation model.

[0010] Optionally, inputting the defect-free image, target control conditions, and target instructions into the trained image generation model to obtain an output defective image includes: Inputting the defect-free image, the target control conditions, and the target instructions into the trained image generation model, wherein the target control conditions are used to specify the defect background, defect location, defect complete form, and defect detailed form, and the target instructions are used to specify the main type and subtype of the defect; The trained image generation model generates a specified defect according to the target control condition and the target instruction, and obtains an output defective image.

[0011] In a second aspect, the present application provides a device for generating a defective image, the device comprising: an acquisition module, configured to acquire a defective sample image, a non-defective sample image, a sample control condition, and a sample instruction, wherein the sample control condition is used to indicate the defect background, defect location, and defect morphology in the defective sample image, and the sample instruction is used to describe the main category and subcategory of the defect; a training module, configured to sequentially perform coarse training and fine training on an initial image generation model based on the defective sample image, the defect-free sample image, the sample control conditions, and the sample instructions, to obtain a trained image generation model, wherein the coarse training is used to learn the complete morphology of defects under different main categories, and the fine training is used to learn the detailed morphology of defects under different subcategories of the same main category; The input and output module is used to input the defect-free image, target control conditions and target instructions into the trained image generation model to obtain the output defective image.

[0012] In a third aspect, the present application provides an electronic device comprising: at least one communication interface; at least one bus connected to the at least one communication interface; at least one processor connected to the at least one bus; and at least one memory connected to the at least one bus.

[0013] In a fourth aspect, the present application further provides a computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the method for generating a defective image as described in any one of the above items of the present application.

[0014] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: through the rough and fine training of the image generation model in the early stage, the model learns the macroscopic morphological differences of defects in the main category and the detailed features in the subcategory, thereby mastering the law of defect characteristics and having the ability to automatically generate defect images. In this way, when the model is subsequently applied in practice, it is only necessary to input a defect-free image, target control conditions and target instructions. The image generation model can determine the position, morphology, background, type and other elements of the defect according to the requirements of the target control conditions and target instructions, and quickly output a defect map with obvious defect markings. The present application can process a large number of images in a short period of time and generate defect maps with obvious defects. There is no need to manually mark the defects in the generated images, thereby improving the efficiency of defect map generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0018] Figure 1 Schematic diagram of a system for generating defective images provided in an embodiment of the present application; Figure 2A flow chart of a method for generating a defective image provided in an embodiment of the present application; Figure 3 A schematic diagram of a defective sample with cracks at the joint provided in an embodiment of the present application; Figure 4 A schematic diagram of a defect-free sample provided in an embodiment of the present application; Figure 5 A schematic diagram of the defect background provided in the embodiment of the present application; Figure 6 A schematic diagram of the overall defect mask provided in an embodiment of the present application; Figure 7 A schematic diagram of the superposition of background control conditions and distinguishing control conditions provided in an embodiment of the present application; Figure 8 A schematic diagram of the ControlNet process provided in an embodiment of the present application; Figure 9 A schematic structural diagram of a device for generating defective images provided in an embodiment of the present application; Figure 10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0020] The disclosure below provides many different embodiments or examples for implementing different structures of the present application. In order to simplify the disclosure of the present application, the components and settings of specific examples are described below. Of course, these are merely examples and are not intended to limit the present application. In addition, the present application may repeat reference numbers and / or letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or settings discussed.

[0021] In order to solve the problem of low defect map generation efficiency mentioned in the background technology, the embodiment of the present application processes the defect-free image through a neural network model to generate a defective image with obvious defects, without the need for manual defect labeling, thereby improving the efficiency of defect map generation.

[0022] Optionally, in the embodiment of the present application, the above-mentioned method for generating a defective image can be applied to Figure 1In the hardware environment composed of the terminal 101 and the server 103 shown in FIG. Figure 1 As shown, server 103 is connected to terminal 101 via a network. A user inputs a defect-free image and a defect generation request through terminal 101. Terminal 101 uploads this data to server 103. Server 103 uses a trained model to output a defective image with obvious defects marked and feeds this back to terminal 101. A database 105 can be set up on the server or independently of the server to provide data storage services for server 103. The aforementioned network includes, but is not limited to, a wide area network, a metropolitan area network, or a local area network. Terminal 101 includes, but is not limited to, a PC, a mobile phone, a tablet computer, etc.

[0023] The following will describe in detail a method for generating a defective image provided by an embodiment of the present application in conjunction with a specific implementation method, taking application to a server as an example. Figure 2 The specific steps are as follows: Step 201: Obtain a defective sample image, a non-defective sample image, a sample control condition, and a sample instruction, wherein the sample control condition is used to indicate the defect background, defect location, and defect morphology in the defective sample image, and the sample instruction is used to describe the main category and subcategory of the defect; Step 202: Based on the defective sample images, the defect-free sample images, the sample control conditions, and the sample instructions, the initial image generation model is subjected to coarse training and fine training in sequence to obtain a trained image generation model. The coarse training is used to learn the complete morphology of defects under different main categories, and the fine training is used to learn the detailed morphology of defects under different subcategories of the same main category. Step 203: Input the defect-free image, target control conditions, and target instructions into the trained image generation model to obtain an output defective image.

[0024] Some of the terms used in the embodiments of this application are explained below, as shown in the following content.

[0025] Defective sample images: Image data containing various defect features is an important material for the model to learn defect information. Defect-free sample image: completely normal image data without any defects, used for comparison with defective sample images to assist the model in identifying defect features. Sample control conditions: A set of structured parameters that precisely define the context, location, and specific form of defects in defective sample images, ensuring that defect presentation meets specific requirements.

[0026] Sample instructions: Specify the defect categories in text form, covering the main categories and subcategories under the main categories, to provide direction for model generation. Image generation model: A model built based on a deep learning algorithm that learns the defect characteristics of sample data and generates defect images that meet specific conditions. Coarse training: Allow the image generation model to learn the overall morphology of defects of different main categories from a macro perspective, and establish basic defect recognition and generation capabilities. Fine training: Based on coarse training, this method focuses on the subtle differences between defects in different subcategories under the same main category, and refines the model's learning of defect detail features.

[0027] In step 201, the data samples obtained by the server include defective sample images, non-defective sample images and their description information. The defective sample images record various defect instances, such as cracks, dents, spots, etc. Figure 3 The diagram is a schematic diagram of a defective sample with cracks at the joint; the diagram of a non-defective sample is a standard image without any flaws. Figure 4 This is a schematic diagram of a defect-free sample. These sample images are the basis for model training. Rich and representative samples allow the model to learn more about the characteristic differences between defects and normal states. Among them, all models mentioned in the embodiments of this application are image generation models.

[0028] The sample description includes sample control conditions and sample instructions. Sample control conditions are used to describe the defect background, defect location (the specific coordinate area of ​​the defect in the image), and defect morphology in the defect sample image. Sample instructions are used to clarify the defect classification hierarchy, including main categories (such as cracks and depressions) and subcategories (such as linear cracks and bifurcated cracks), providing semantic annotation guidance for the model.

[0029] like Figure 5 As shown, the defect background can be the background of the defect itself, edge information, or line drawing, etc.

[0030] The context itself refers to the basic environment in which the defect exists, such as Figure 5 In image processing, defects are located within a surface environment with a specific material texture and color. This background environment affects the visual appearance of the defect and is also interrelated with the defect's characteristics. Image generation models learn from this background information to more accurately grasp the scene in which the defect appears, taking this environmental factor into account when generating or identifying defects. For example, the visual appearance of the same type of crack on surfaces of different materials may vary. With this background information, the model can better distinguish and process them.

[0031] Edge information is a crucial component in describing the relationship between a defect and its surroundings. It defines the defect's contours and helps image generation models determine its shape. For example, edge information can be used to determine the direction and width of a crack. It also reveals the transition between the defect and the background.

[0032] Line drawings are a simplified image representation that highlights key line structures within an image. In defect recognition and generation, line drawings can extract key line features of defects, such as the direction and branching of cracks. By presenting important morphological information of defects in a concise manner, they enable image generation models to more efficiently learn and memorize typical defect features, helping them better handle defect-related tasks.

[0033] In step 202, the server uses the collected defective and non-defective sample images, sample control conditions, and sample instructions to perform a two-stage progressive training process on the initial image generation model, namely, coarse training and fine training. This training process enables the model to generate specific defects on demand based on defect-free images. During the coarse training phase, the model aims to distinguish between primary defect categories (e.g., cracks and dents). By comparing defective and non-defective sample images, the model learns the macroscopic morphological differences between defects of different primary categories, such as the linear appearance of cracks and the concave structure of dents. Combined with the position coordinates in the sample control conditions, the model generates a complete defect in the specified area of ​​the defect-free image, such as a linear crack in the center of a metal plate. This initially achieves matching of defect location with primary category morphology, but the generated defect details are relatively coarse.

[0034] During the refined training phase, the image generation model, building on its understanding of the differences between main categories, focuses on learning the microscopic details of defects within different subcategories within the same main category. For example, within the main crack category, the straightness of a linear crack and the branching structure of a bifurcated crack are analyzed. By focusing on the local defects through an attention mechanism and combining subcategory labels with background features (such as material texture and edge information), the model deeply learns the defect's geometric parameters (such as branch angle and crack width) and its semantic consistency with the background. For example, it can generate bifurcated cracks with jagged edges that conform to the distribution of aggregate at a specific location in the concrete background, resulting in a more refined defect shape that blends naturally with the background.

[0035] The core generation logic of the image generation model is based on a conditional generation mechanism: first, the defect semantics are parsed based on sample instructions and sample control conditions, converted into feature vectors, and fused with the visual features of the defect-free image. A decoder is then used to predict the pixel values ​​of the defect area. Meanwhile, background features such as edge information and line drawings are used to constrain the edge transitions of the defect, ensuring that the generated defects meet the input requirements in terms of position, shape, and background logic. After two stages of training, the model is able to refine defect features layer by layer from the macro to the micro level, ultimately accurately generating images with designated main and subcategory defects on defect-free images.

[0036] Among them, the image generation module can use ControlNet, which is a derivative network of Stable Diffusion (a text-to-image generation model based on deep learning). The content of the image generated by Stable Diffusion is random. Using ControlNet, customized images can be generated by setting specific control conditions.

[0037] In step 203, the fully trained image generation model enters the application phase. Its core function is to automatically generate defective images based on the input defect-free image, target control conditions, and target instructions. The target control conditions provide physical constraints for defect generation by defining the defect background (e.g., metallic glossy surface, wood texture), spatial location (e.g., the coordinate area of ​​the lower right corner of the image), and morphological parameters (e.g., crack length 10mm, width 0.5mm). The target instructions semantically define the defect category (e.g., linear crack), ensuring that the generated results meet preset requirements.

[0038] When the image generation model performs a generation task, the semantic parsing module first converts the target instruction into a defect feature vector. Combined with the background and position information in the target control conditions, the defect generation module is triggered in a specified area of ​​the defect-free image. Based on the defect morphology patterns learned during the training phase (e.g., linear continuity and edge sharpness of straight cracks) and background fusion logic (e.g., reflective consistency of cracks on metal surfaces), the defect generation module generates physically accurate defect regions pixel by pixel. Simultaneously, the model's built-in automatic annotation module simultaneously extracts the defect's geometric features (location coordinates, length, and direction) and semantic labels (main category, subcategory), overlaying them on the image as visual outlines and text. For example, given an image of a defect-free ceramic plate and the specified conditions of "smooth glaze background + central area + bifurcated cracks," the model generates a defect image with natural-looking glaze cracks and branching angles of approximately 45°. This automated process requires no human intervention and rapidly generates defect images.

[0039] For example, in a metal part quality inspection scenario, the server collects images of metal parts with crack defects as defective sample images, and images of normal metal parts as defect-free sample images. The sample control conditions set the defect background to the smooth surface of the metal part, the defect location to the defect coordinates in the defect sample image, and the defect morphology to an irregular crack. The sample instructions define the main category as "crack" and the subcategory as "bifurcation crack." Through coarse training, the image generation model learns to distinguish between defects in metal parts and grasps the general morphology of defects in different main categories. Subsequently, through fine training, the image generation model deeply understands the differences in details such as texture and angle between the two subcategories of "straight crack" and "bifurcation crack," resulting in a trained image generation model. During model operation, the input is an image of a defect-free metal part, the target instruction is to generate a "bifurcation crack" defect, and the target control conditions are that the crack is located at the edge of the part, is approximately 10 mm long, and has an irregular shape. The model outputs a defective image that clearly displays information such as the crack location, type, and morphology, which can be used in subsequent quality inspection scenarios.

[0040] In this application, through the rough and fine training of the image generation model in the early stage, the model learns the macroscopic morphological differences of defects in the main category and the detailed features in the subcategory, thereby mastering the law of defect characteristics and having the ability to automatically generate defect images. In this way, when the model is actually applied in the future, it is only necessary to input a defect-free image, target control conditions and target instructions. The image generation model can determine the location, morphology, background, type and other factors of the defect according to the requirements of the target control conditions and target instructions, and quickly output a defect map with obvious defect markings. This application can process a large number of images in a short period of time and generate defect maps with obvious defects. There is no need to manually mark the defects in the generated images, which improves the generation efficiency of defect maps.

[0041] As an optional implementation, before obtaining the sample control conditions, the process of generating the sample control conditions includes the following content.

[0042] Step S11: extracting a background control condition from the defective sample image, wherein the background control condition is used to indicate a portion of the background in the defective sample image that contains defects but is not marked with defects; Step S12: extracting a discriminative control condition from the defective sample image, wherein the discriminative control condition is used to indicate a defect mask in the background control condition; Step S13: The background control condition and the discriminative control condition are combined with the defect as a reference point to obtain the sample control condition.

[0043] In step S11, different background control conditions in the defective sample image result in different background extraction processes. The background control condition indicates the portion of the background in the defective sample image that contains defects but is not marked. By retaining the unmarked background portion, the image generation model can learn the natural association between defects and the environment, improving the physical credibility of defect generation.

[0044] If the background itself is to be extracted, since the defective sample image itself contains the background, the image of the defective sample image can be directly extracted.

[0045] To extract edge information, the server uses edge detection algorithms (such as the Canny operator) to extract structural edges in the background. These edges may be spatially correlated with the defect. These edges include both the outline of objects in the background (such as plate seams) and the trajectory of the defect (such as the direction of crack propagation). By overlaying edge information with the defect location, the model learns the spatial correlation between the defect and the background structure (for example, the distribution of cracks along the edge of an object). This allows the model to follow the physical logic of "edge alignment" when generating defects, improving the rationality of defect location.

[0046] If extracting line drawings, the server converts the defective sample image into a simplified line representation using an image sketching algorithm. This approach utilizes non-photorealistic rendering technology to calculate the image's gradient changes and color contrast to extract key lines that characterize the object's structure (e.g., contour lines and texture orientation lines). The line drawing discards the background's color and texture details, retaining only the topological relationships of the geometric structure, allowing the model to focus on the defect's morphological skeleton (e.g., the branching veins of a crack or the boundary lines of a depression). This abstract feature representation helps the model learn the essential morphological patterns of defects and reduces the interference of redundant visual information. It is particularly suitable for generation tasks that require highlighting the structural characteristics of defects (e.g., topological analysis of cracks in mechanical parts).

[0047] In step S12, the core of the discriminative control condition is to generate defect mask information, which represents the defect characteristics by pixel-level binarization or multi-channel color coding. The defect mask includes two forms, one is the overall defect mask and the other is the detail feature mask. These two masks are described in detail below.

[0048] Overall defect mask: Generate a binary image covering the defect area through manual annotation or automatic segmentation algorithm (for example, white marks the defect and black is the background). It is used to define the spatial range of the defect, extract geometric parameters such as area and position, and provide macroscopic morphological constraints for the defect. Figure 6 Schematic diagram of the overall defect mask.

[0049] Detail feature mask: For the internal subdivision structure of the defect (such as crack bifurcation points and wear areas), sub-features are marked with different colors or grayscale values ​​(for example, white marks the overall defect, red marks the straight part of the crack, and focuses on the sub-features marked in red) to characterize the microscopic differences of the defect.

[0050] The embodiment of the present application decouples defects from the background through mask information, forming a differentiated feature representation of "defect-background", which not only clarifies the scope of defect existence, but also injects sub-feature details to support the model to learn the hierarchical features of defects.

[0051] In step S13, the server matches the positional features in the background control conditions (such as the coordinates of the defect center) with the geometric parameters in the discriminative control conditions (such as the crack length) to obtain the sample control conditions. The sample control conditions can completely preserve the physical relationship between the defect and the background, so that the model can not only follow the constraints of the background environment during the generation process, but also accurately reproduce the morphological characteristics of the defect, thereby achieving precise control of defect generation. Figure 7 Schematic diagram of the superposition of background control conditions and discriminative control conditions.

[0052] In this application, background control conditions ensure that the semantics of defects are consistent with the environment, and discriminative masks support the generation of detail features, thereby improving the semantic richness of the generated image. By superimposing sample control conditions and discriminative control conditions, the background, location, and morphology of defects can be accurately specified.

[0053] As an optional implementation, the discriminative control conditions include primary discriminative conditions and secondary discriminative conditions, and the sample control conditions generated based on the discriminative control conditions also include two types. In step S13, the process of obtaining each sample control condition includes the following content.

[0054] In step S131 , the background control condition and the first-level distinguishing condition are combined with the defect as a reference point to obtain the first-level control condition, wherein the first-level control condition is used to indicate the background of the defect and the overall defect mask; In step S132: the background control condition and the secondary distinguishing condition are merged with the defect as the reference point to obtain the secondary control condition, wherein the secondary control condition is used to indicate the background where the defect is located and a partial defect mask at the position of the defect detail morphology, wherein the defect detail morphology of different subcategories of defects is different.

[0055] In step S131, the server uses the defect as a spatial reference point and merges the background control conditions with the primary discriminative conditions (used to determine the overall defect mask). The background control conditions provide environmental characteristics of the defect, such as the gloss of the metal surface and the texture of the defect location. The primary discriminative conditions define the defect's macroscopic outline and location through the overall defect mask. The merging process aligns the environmental characteristics with the defect location through coordinate mapping. For example, the texture characteristics of the "concrete background" are superimposed on the "overall crack mask" to ensure that the crack is generated in a specific area of ​​the concrete surface. The generated primary control conditions exist as a composite feature vector, which combines the visual environment information of the background and the overall morphological parameters of the defect, forming a coarse-grained description of the defect's "context + macroscopic morphology."

[0056] Through the first-level control conditions, the image generation model can follow the "environment-position" consistency principle when generating defects, avoiding logical errors such as "metal cracks appearing against a wood grain background". At the same time, based on the geometric constraints of the overall mask, it can quickly construct the macroscopic morphological framework of the defects (such as linear cracks and circular depressions), laying the foundation for subsequent detail generation.

[0057] In step S132, the server integrates the background control condition with the secondary distinguishing condition (used to determine the defect detail morphological mask), focusing on the precise control of the defect's microscopic features. The secondary distinguishing condition annotates the detailed structure within the defect, such as the location of the crack's bifurcation point and the roughness distribution of the wear area, and is represented by multi-channel color coding or grayscale value differences (for example, red marks a straight crack). When merged, the edge information and line features in the background control condition are deeply coupled with the detail mask. For example, the "jagged edge information of the metal crack edge" is combined with the "detail mask of the bifurcated crack" to generate crack branches with realistic physical characteristics. The resulting secondary control condition not only includes the background environment and the macroscopic location of the defect, but also injects the morphological detail parameters of the local defect area (such as bifurcation angle and edge roughness), achieving a fine-grained description of the defect's "context + microstructure".

[0058] By introducing detailed defect features, the secondary control condition forces the image generation model to focus on morphological differences in local areas during the generation process, addressing the problem of single defect morphology and lack of detailed variation in traditional methods. For example, for the "bifurcation crack" subcategory, the model can generate cracks of different morphologies based on the branching angle parameters of the secondary mask, improving the diversity and physical realism of the generated results.

[0059] In this application, the first-level control condition realizes the coarse-grained matching of "background-macro morphology" to ensure that defect generation conforms to the scene logic; the second-level control condition realizes the fine-grained alignment of "background-micro details" to make the defect morphology closer to the real physical laws. The combination of these two forms a full-chain control from macro to micro.

[0060] As an optional implementation, the sample instructions include first-level instructions and second-level instructions. In step 202, the training process of the image generation model includes the following content.

[0061] Step S21: performing rough training on the initial image generation model using defective sample images, non-defective sample images, first-level control conditions, and first-level instructions, so that the image generation model learns the complete morphology of defects belonging to different main categories, wherein the first-level instructions are used to describe the main categories of defects; Step S22: After the rough training is completed, the image generation model is finely trained using defective sample images, non-defective sample images, secondary control conditions, and secondary instructions, so that the image generation model learns the detailed morphology of defects belonging to the same main category but different subcategories, and obtains a trained image generation model, wherein the secondary instructions are used to describe the subcategories of defects.

[0062] The training process of the image generation model uses a two-stage progressive mechanism of coarse training and fine training to achieve hierarchical learning of defect features, as shown below.

[0063] Coarse training: macroscopic feature modeling of main category defects.

[0064] The server trains the image generation model using defective and non-defective sample images, first-level control conditions (background + overall defect mask), and first-level instructions (e.g., cracks, dents). The model learns the core morphological features of defects in different main categories by comparing the visual differences between defective and non-defective sample images and combining the background environment (e.g., metal surface texture) and the overall defect outline (e.g., linear crack mask) in the first-level control conditions. For example, when inputting "concrete background + overall crack mask," the model uses adversarial and reconstruction losses to learn to generate linear cracks that match the material properties of concrete and to understand the spatial correlation between cracks and background (e.g., distribution along aggregate gaps). This stage establishes the model's foundational understanding of main category defects, locking in the correlation between the main category and background through the first-level control conditions to avoid cross-category logical errors.

[0065] Fine training: learning detailed features of subcategory defects under the same main category.

[0066] Based on the rough training, the server introduces secondary control conditions (background + defect detail morphology mask) and secondary instructions (e.g., linear cracks, bifurcated cracks). The image generation model uses an attention mechanism to focus on the local defect area and, combined with the detail mask, learns the microscopic differences between different subtypes within the same main category. For example, when inputting "metal background + bifurcated crack detail mask," the model generates cracks with a specific bifurcation angle (e.g., 60°) and edge roughness, achieving refined differentiation from the main "crack" category to the "bifurcated crack" subtype.

[0067] In this application, two-stage training reduces the reliance on fully labeled data through feature hierarchical learning. Coarse training only requires main category-level annotation, while fine training relies on a small amount of subcategory detail annotation, reducing data collection and annotation costs. In addition, the coarsely trained model can generate defects that meet the main category characteristics. The defect prediction map it outputs has little difference in overall morphology from the real sample, providing high-quality feature initialization for learning subcategory details in the fine training stage, reducing the convergence time of subsequent training, and improving training efficiency and accuracy.

[0068] As an optional implementation, in step S21, the image generation model is roughly trained including the following contents.

[0069] Step S211: inputting the defect-free sample image, the first-level control conditions, and the first-level instructions into the initial image generation model to obtain an output defect prediction image; Step S212: determining pixel differences between the defect prediction image and the defect sample image in the defective part and the non-defective part respectively; Step S213: performing weighted summation of pixel differences in corresponding position areas according to a first weight of the defective part and a second weight of the non-defective part to obtain a loss value, wherein the first weight is greater than the second weight; Step S214: Adjust the model parameters by minimizing the loss value to obtain a roughly trained image generation model.

[0070] In step S211, the server uses the defect-free sample image as the basic input, overlays the first-level control conditions and the first-level instructions, and feeds this into the initial image generation model. The image generation model uses an encoder to extract visual features (such as material texture and color distribution) from the defect-free image. Then, combining the defect background, location, and morphology from the first-level control conditions with the primary defect type from the first-level instructions, it generates a predicted image containing the target primary defect category. For example, given a defect-free concrete image, a "concrete background + crack mask," and the "crack" instruction, the image generation model outputs a predicted image with cracks, essentially implanting defects within the specified background.

[0071] In step S212, the server compares the defect prediction image with the defective sample image pixel by pixel, calculating the pixel differences between the defective portion (the masked area) and the non-defective portion (the background area). The defective portion is used to highlight the morphological differences between the predicted defect area and the actual defect, while the non-defective portion is used to detect abnormal changes in the background area caused by the defect generation, ensuring that the model does not destroy the original background structure.

[0072] In step S213, a differentiated weighting mechanism is introduced, where the first weight (for defects) is greater than the second weight (for non-defects). This forces the model to prioritize defect generation accuracy, ensuring that the contribution of defect morphological errors to the loss value dominates. The loss value is then calculated based on the weighted sum of the pixel differences in the defect and the first weight, and the pixel differences in the non-defect and the second weight. This weighting adjustment ensures that the image generation model prioritizes the morphological authenticity of the defect area when generating defects, while also constraining the perturbation amplitude of the background area.

[0073] In step S214, the server minimizes the loss using a backpropagation algorithm and adjusts the model's network parameters, such as the convolution kernel parameters and attention weights. After multiple rounds of iteration, the image generation model gradually learns to generate the overall defect shape that matches the main category characteristics based on the primary control conditions, completing the coarse training phase.

[0074] In this application, by designing the first weight to be higher than the second weight, it is ensured that the image generation model concentrates training resources on morphological fitting of the defect area, avoiding blurring of defect features due to interference from background details.

[0075] As an optional implementation, in step S22, the image generation model is finely trained including the following contents.

[0076] Step S221: inputting the defect-free sample image, the secondary control conditions, and the secondary instructions into the roughly trained image generation model to obtain an output defect result image; Step S222: Determine the pixel differences between the defect result image and the defect sample image in terms of defect detail morphology and other remaining areas, wherein the defect detail morphology refers to the detail features between defects of different subcategories under the same main category; Step S223: performing weighted summation of pixel differences in the corresponding position area according to the third weight of the defect detail morphology and the fourth weight of other remaining areas to obtain a loss value, wherein the third weight is greater than the fourth weight; Step S224: Adjust the model parameters by minimizing the loss value to obtain a finely trained image generation model.

[0077] In step S221, the server inputs the defect-free sample image, the secondary control conditions, and the secondary instructions into the roughly trained image generation model. The model extracts features from the defect-free image through an encoder and, combined with the detail mask from the secondary control conditions, generates a defect result image with details for a specific subcategory. For example, given an image of a defect-free metal plate, a "metal background + bifurcated crack detail mask," and the "bifurcated crack" instruction, the model outputs a crack image with a precise branching structure, where the branching angles are consistent with the mask annotations.

[0078] In step S222, the server performs regional segmentation and comparison on the defect result image and the defect sample image, mainly dividing them into defect detail morphology areas and other remaining areas.

[0079] The defect detail morphology area focuses on the microscopic differences between subcategories (such as the number of branches of bifurcated cracks and the edge curvature of depressions), and calculates the pixel-level difference between the predicted results and the real samples; the other remaining areas include the background and defect non-detail areas, and detect abnormal disturbances caused by detail generation.

[0080] In step S223, a detail enhancement weighting mechanism is introduced, where the third weight (defect detail morphology) is higher than the fourth weight (other remaining areas), ensuring that detail differences dominate the contribution to the loss value. The image generation model calculates the loss value through weighted summation. During the calculation process, it prioritizes correcting subcategory-specific detail features (such as crack branching angle deviation) while simultaneously constraining perturbations in the background area.

[0081] In step S224, the server uses a backpropagation algorithm to minimize the loss value and fine-tune the model's attention module and decoder parameters. After multiple rounds of iteration, the model solidifies the subcategory details into the parameters, completing the fine-tuning training.

[0082] In this application, the enhanced effect of the third weight enables the model to capture subtle morphological differences within the same main category. The presence of the fourth weight limits the modification of non-detail areas, preventing the model from destroying the background structure or the overall morphology of the defect in pursuit of detail. For example, when generating complex branching cracks, the background metal texture retains its original clarity, and the overall continuity of the crack is unaffected.

[0083] As an optional implementation, in step 203, the defect-free image, target control conditions, and target instructions are input into the trained image generation model, and the output defective image includes the following content.

[0084] Step S31: Inputting the defect-free image, target control conditions, and target instructions into the trained image generation model, wherein the target control conditions are used to specify the defect background, defect location, complete defect form, and detailed defect form, and the target instructions are used to specify the main type and subtype of the defect; Step S32: Generate a specified defect according to the target control conditions and target instructions through the trained image generation model to obtain an output defective image.

[0085] After receiving input data, the image generation model converts the target instructions and target control conditions into multi-level feature vectors (such as background feature vectors, position mask vectors, and morphological parameter vectors). These are then integrated with the visual features of the defect-free image (such as pixel values ​​and texture distribution) to form a complete generation instruction. Based on the knowledge learned during the training phase, the image generation model executes the following generation logic. The resulting defective image is visually realistic and can be directly used in downstream inspection tasks. The executed logic includes the following.

[0086] Spatial positioning: Generate defects in designated areas of defect-free images based on defect location parameters to prevent defects from appearing in irrelevant areas.

[0087] Morphological rendering: Based on the complete morphological parameters of the defect, a basic outline that meets the main category characteristics is generated, and then the detailed morphological parameters of the defect are superimposed to refine the sub-category characteristics.

[0088] Background fusion: Adjust the visual attributes of the defect by using the material, lighting and other characteristics of the background control conditions to ensure the physical consistency between the defect and the background.

[0089] Automatic labeling: The built-in labeling module automatically generates the defect location box, category label and detail parameters based on the mask information in the target control conditions, and superimposes them on the generated image.

[0090] Figure 8 The figure is a flow chart of ControlNet. It can be seen that the input data includes the defect-free image in the upper left corner and the target control condition in the lower left corner. The target control condition marks the defect background, defect location, complete defect form and detailed defect form. By synthesizing the defect-free image and the target control condition and adding prompt words (target instructions), ControlNet outputs an image with defects.

[0091] Based on the same technical concept, the present application provides a device for generating defective images, such as Figure 9 As shown, the device includes: An acquisition module 901 is configured to acquire a defective sample image, a non-defective sample image, a sample control condition, and a sample instruction. The sample control condition is used to indicate the defect background, defect location, and defect morphology in the defective sample image, and the sample instruction is used to describe the main category and subcategory of the defect. Training module 902 is used to perform coarse training and fine training on the initial image generation model based on defect sample images, defect-free sample images, sample control conditions, and sample instructions to obtain a trained image generation model. The coarse training is used to learn the complete morphology of defects under different main categories, and the fine training is used to learn the detailed morphology of defects under different subcategories of the same main category. The input and output module 903 is used to input the defect-free image, target control conditions and target instructions into the trained image generation model to obtain the output defective image.

[0092] Optionally, the device is further used to: Extracting a background control condition from the defective sample image, wherein the background control condition is used to indicate a portion of the background in the defective sample image that contains defects but is not marked with the defects; Extracting a discriminative control condition from a defective sample image, wherein the discriminative control condition is used to indicate a defect mask in a background control condition; The background control condition and the discriminative control condition are combined with the defect as the reference point to obtain the sample control condition.

[0093] Optionally, the discriminative control condition includes a primary discriminative condition and a secondary discriminative condition, and the device is further configured to: Taking the defect as the reference point, the background control condition and the first-level discriminative condition are combined to obtain the first-level control condition, where the first-level control condition is used to indicate the background of the defect and the overall defect mask; The background control condition and the secondary distinguishing condition are combined with the defect as the reference point to obtain the secondary control condition, wherein the secondary control condition is used to indicate the background where the defect is located and a partial defect mask at the position of the defect detail morphology, wherein the defect detail morphology of different subcategories of defects is different.

[0094] Optionally, the sample instructions include primary instructions and secondary instructions, and the training module 902 is configured to: The initial image generation model is roughly trained using defective sample images, non-defective sample images, first-level control conditions, and first-level instructions, so that the image generation model learns the complete morphology of defects belonging to different main categories. The first-level instructions are used to describe the main categories of defects. After the rough training is completed, the image generation model is fine-trained using defective sample images, non-defective sample images, secondary control conditions, and secondary instructions, so that the image generation model can learn the detailed morphology of defects belonging to the same main category but different subcategories, and obtain a trained image generation model, where the secondary instructions are used to describe the subcategories of defects.

[0095] Optionally, the training module 902 is specifically configured to: Input the defect-free sample image, the first-level control conditions, and the first-level instructions into the initial image generation model to obtain the output defect prediction image; Determine the pixel differences between the defect prediction image and the defect sample image in the defect part and the non-defect part respectively; Performing weighted summation of pixel differences in corresponding position areas according to a first weight of the defective part and a second weight of the non-defective part to obtain a loss value, wherein the first weight is greater than the second weight; The model parameters are adjusted by minimizing the loss value to obtain a roughly trained image generation model.

[0096] Optionally, the training module 902 is specifically configured to: Input the defect-free sample image, the secondary control conditions, and the secondary instructions into the roughly trained image generation model to obtain the output defect result image; Determine the pixel differences between the defect result image and the defect sample image in terms of defect detail morphology and other remaining areas. The defect detail morphology refers to the detailed features between defects of different subcategories under the same main category. A loss value is obtained by weighted summing the pixel differences of the corresponding position area according to the third weight of the defect detail morphology and the fourth weight of other remaining areas, wherein the third weight is greater than the fourth weight; The model parameters are adjusted by minimizing the loss value to obtain a finely trained image generation model.

[0097] Optionally, the input / output module 903 is specifically configured to: Input the defect-free image, target control conditions, and target instructions into the trained image generation model, where the target control conditions are used to specify the defect background, defect location, complete defect form, and detailed defect form, and the target instructions are used to specify the main type and subtype of the defect; The trained image generation model generates the specified defects according to the target control conditions and target instructions, and the output defective image is obtained.

[0098] like Figure 10 As shown, an embodiment of the present application provides an electronic device, including a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other through the communication bus 1004.

[0099] The memory 1003 is used to store computer programs.

[0100] In one embodiment of the present application, the processor 1001 is configured to implement the method for generating a defective image provided by any one of the aforementioned method embodiments when executing the program stored in the memory 1003 .

[0101] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for generating a defective image provided by any of the aforementioned method embodiments are implemented.

[0102] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0103] Through the description of the above embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a general hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the relevant technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, a server, or a network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0104] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, for example, the singular forms "one", "an" and "said" used herein may also be meant to include plural forms. The terms "comprise", "include", "contain" and "have" are inclusive and therefore specify the presence of stated features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the specific order described or illustrated, unless the order of execution is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0105] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.

Claims

1. A method for generating a defective image, characterized in that: The method comprises: Obtaining a defective sample image, a non-defective sample image, a sample control condition, and a sample instruction, wherein the sample control condition is used to indicate the defect background, defect location, and defect morphology in the defective sample image, and the sample instruction is used to describe the main category and subcategory of the defect; Based on the defective sample image, the non-defective sample image, the sample control conditions, and the sample instructions, coarse training and fine training are sequentially performed on the initial image generation model to obtain a trained image generation model, wherein the coarse training is used to learn the complete morphology of defects under different main categories, and the fine training is used to learn the detailed morphology of defects under different subcategories of the same main category; The defect-free image, target control conditions and target instructions are input into the trained image generation model to obtain an output defective image.

2. The method according to claim 1, characterized in that Before obtaining the sample control conditions, the generation process of the sample control conditions includes: Extracting a background control condition from the defective sample image, wherein the background control condition is used to indicate a portion of the background of the defective sample image that contains defects but is not marked with defects; extracting a discriminative control condition from the defective sample image, wherein the discriminative control condition is used to indicate a defect mask in the background control condition; The background control condition and the discriminative control condition are combined with the defect as a reference point to obtain the sample control condition.

3. The method according to claim 2, characterized in that The discriminative control conditions include a primary discriminative condition and a secondary discriminative condition. The background control conditions and the discriminative control conditions are combined with the defect as a reference point to obtain the sample control conditions, including: Taking the defect as a reference point, the background control condition and the primary distinguishing condition are combined to obtain a primary control condition, wherein the primary control condition is used to indicate the background of the defect and the overall defect mask; The background control condition and the secondary distinguishing condition are merged with the defect as a reference point to obtain a secondary control condition, wherein the secondary control condition is used to indicate the background of the defect and a partial defect mask at the position of the defect detail morphology, wherein the defect detail morphology of defects of different subcategories is different.

4. The method according to claim 3, characterized in that The sample instructions include first-level instructions and second-level instructions. Based on the defective sample image, the non-defective sample image, the sample control conditions, and the sample instructions, the initial image generation model is subjected to coarse training and fine training in sequence. The trained image generation model includes: performing coarse training on an initial image generation model using the defective sample images, the non-defective sample images, the primary control conditions, and the primary instructions, so that the image generation model learns complete forms of defects belonging to different main categories, wherein the primary instructions are used to describe the main categories of the defects; After the rough training is completed, the image generation model is finely trained using the defective sample image, the defect-free sample image, the secondary control conditions, and the secondary instructions, so that the image generation model learns the detailed morphology of defects belonging to the same main category but different subcategories, thereby obtaining a trained image generation model, wherein the secondary instructions are used to describe the subcategories of the defects.

5. The method according to claim 4, characterized in that The rough training of the initial image generation model using the defective sample image, the non-defective sample image, the first-level control condition, and the first-level instruction includes: Inputting the defect-free sample image, the first-level control conditions, and the first-level instructions into an initial image generation model to obtain an output defect prediction image; Determine pixel differences between the defect prediction image and the defect sample image at defective parts and non-defective parts respectively; Performing weighted summation of pixel differences in corresponding position areas according to a first weight of the defective part and a second weight of the non-defective part to obtain a loss value, wherein the first weight is greater than the second weight; The model parameters are adjusted by minimizing the loss value to obtain a roughly trained image generation model.

6. The method according to claim 4, characterized in that The fine training of the image generation model using the defective sample image, the non-defective sample image, the secondary control conditions, and the secondary instructions includes: Inputting the defect-free sample image, the secondary control conditions, and the secondary instructions into the roughly trained image generation model to obtain an output defect result image; Determine pixel differences between the defect result image and the defect sample image in defect detail morphology and other remaining areas, wherein the defect detail morphology refers to detail features between defects of different subcategories under the same main category; Performing weighted summation of pixel differences in corresponding position areas according to a third weight of the defect detail morphology and a fourth weight of the other remaining areas to obtain a loss value, wherein the third weight is greater than the fourth weight; The model parameters are adjusted by minimizing the loss value to obtain a finely trained image generation model.

7. The method according to claim 1, characterized in that Inputting the defect-free image, target control conditions, and target instructions into the trained image generation model, the output defective image includes: Inputting the defect-free image, the target control conditions, and the target instructions into the trained image generation model, wherein the target control conditions are used to specify the defect background, defect location, defect complete form, and defect detailed form, and the target instructions are used to specify the main type and subtype of the defect; The trained image generation model generates a specified defect according to the target control condition and the target instruction, and obtains an output defective image.

8. A device for generating defective images, characterized in that: The device comprises: an acquisition module, configured to acquire a defective sample image, a non-defective sample image, a sample control condition, and a sample instruction, wherein the sample control condition is used to indicate the defect background, defect location, and defect morphology in the defective sample image, and the sample instruction is used to describe the main category and subcategory of the defect; a training module, configured to sequentially perform coarse training and fine training on an initial image generation model based on the defective sample image, the defect-free sample image, the sample control conditions, and the sample instructions, to obtain a trained image generation model, wherein the coarse training is used to learn the complete morphology of defects under different main categories, and the fine training is used to learn the detailed morphology of defects under different subcategories of the same main category; The input and output module is used to input the defect-free image, target control conditions and target instructions into the trained image generation model to obtain the output defective image.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method according to any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images