Industrial defect image generation method and device, electronic equipment and medium
By generating the exception mask and reconstructing the defect image, the problems of unstable generation effect of the deep generation model and poor correspondence between the defect and the mask are solved, and the accuracy of the generation and detection tasks of subtle defects is improved.
Patent Information
- Application Number
- CN202510427294.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-07
- Publication Date
- 2025-07-22
AI Technical Summary
The existing depth generation model has unstable effects when generating industrial defect images, making it difficult to ensure the correspondence between defects and masks, and making it difficult to generate subtle defects, affecting the accuracy and robustness of the detection task.
By obtaining the target input text and generating an exception mask, input it into the pre-trained defect imitation network model, masking the normal industrial image based on the exception mask, using the abnormal features of the diffusion model and the defect reference network model, reconstructing the target defect image, and computing similar results for the image and text to determine the defect image.
The stability of the generation effect and the accuracy of subtle defects are achieved, ensuring the correspondence between the generated defect image and the mask, and improving the accuracy and robustness of downstream detection tasks.
Smart Images
Figure CN120356033A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of image generation, and in particular, to an industrial defect image generation method, device, electronic device, and medium. Background Art
[0002] In recent years, deep generative models have made remarkable progress in the field of image generation technology. In particular, generative adversarial networks (GANs) and diffusion models included in deep generative models have demonstrated powerful generative capabilities in many applications. These technologies are used to generate defect images in the field of industrial inspection, aiming to alleviate the problem of scarce real defect image samples, thereby assisting neural network-based detection methods in industrial scenarios. However, due to the particularity of industrial defect images themselves and the imbalance of data, the existing deep generative models still have the following main problems:
[0003] 1. Unstable generation effect: Existing methods usually adopt a single-branch architecture of GAN or a single diffusion model, combining conditional processing with the generation task. This not only increases the learning burden of the network but also leads to unstable generation effects, facing greater challenges in generating high-quality defect images, thus affecting the overall generation effect and the accuracy of downstream detection tasks.
[0004] 2. Poor correspondence between defects and masks: Some methods are difficult to ensure that the abnormal regions in the generated defect images correspond to the preset defect masks, which may cause mismatches between data annotation and model training in actual industrial inspection, affecting the accuracy and robustness of downstream detection tasks.
[0005] 3. Difficulty in generating subtle defects: Existing methods are difficult to accurately reproduce small and complex defect features (such as cracks, holes, etc.) when generating images, resulting in deficiencies in detail restoration and authenticity of the generated images, and unable to meet the requirements of high-precision detection for tiny defects. Summary of the Invention
[0006] In view of this, the purpose of the present application is to provide an industrial defect image generation method, device, electronic device, and medium, which effectively solve the problems of unstable generation effect, poor correspondence between defects and masks, and difficulty in generating subtle defects existing in the existing deep generative models.
[0007] In a first aspect, an industrial defect image generation method provided by an embodiment of the present application includes:
[0008] Obtain a target input text and a normal industrial image corresponding to the target input text, and perform text inversion on the target input text to generate an abnormal mask; the abnormal mask includes a masked area;
[0009] Input the target input text, the normal industrial image, and the abnormal mask into a defect imitation network model pre-established based on a diffusion model; the defect imitation network model is established based on the abnormal features transmitted by a defect reference network model; the defect reference network model is established based on real industrial defect images;
[0010] Control the defect imitation network model to mask a target masked area corresponding to the masked area in the normal industrial image based on the abnormal mask, obtain a normal industrial image with the target masked area, and reconstruct the target masked area of the normal industrial image to obtain a target defect image;
[0011] Calculate a similarity result between the target defect image and the target input text, and determine the target defect image as the industrial defect image corresponding to the target input text based on the similarity result.
[0012] Combined with the first aspect, the embodiments of the present application provide a first possible implementation manner of the first aspect, wherein the specific steps for establishing the defect imitation network model based on the abnormal features transmitted by the defect reference network model include:
[0013] Construct an initial defect imitation network model based on a pre-trained diffusion model, and receive an abnormal reference feature combination transmitted by the defect reference network model;
[0014] Fuse the initial defect imitation network model and the abnormal reference feature reference combination to obtain a defect imitation network model.
[0015] Combined with the first aspect, the embodiments of the present application provide a second possible implementation manner of the first aspect, wherein the receiving the abnormal reference feature combination transmitted by the defect reference network model includes:
[0016] Extract various abnormal reference features and fusion weight parameters corresponding to multiple cross-attention layers respectively based on a pre-trained defect reference network model;
[0017] Summarize various abnormal reference features and fusion weight parameters corresponding to multiple cross-attention layers respectively to obtain an abnormal reference feature combination, and transmit the abnormal reference feature combination to the initial defect imitation network model based on a preset transmission method.
[0018] Combined with the first aspect, the embodiments of the present application provide a third possible implementation manner of the first aspect, wherein the fusing the initial defect imitation network model and the abnormal reference feature combination to obtain a defect imitation network model includes:
[0019] Determine the target level for integrating the initial defect imitation network model to invoke the sub - abnormal reference feature combination of the defect reference network model at the target level;
[0020] Process the sub - abnormal reference feature combination to obtain a sub - abnormal simulation feature combination, and based on the sub - abnormal simulation feature combination, obtain a defect imitation network model.
[0021] Combined with the first aspect, the embodiments of the present application provide a fourth possible implementation manner of the first aspect, where the abnormal mask masks the target mask area corresponding to the masked area in the normal industrial image to obtain a normal industrial image with the target mask area, including:
[0022] Traverse the normal industrial image to identify the target mask area corresponding to the mask area in the abnormal mask;
[0023] Generate the defect morphology included in the abnormal mask on the target mask area to mask the target mask area of the normal industrial image.
[0024] Combined with the first aspect, the embodiments of the present application provide a fifth possible implementation manner of the first aspect, where reconstructing the target mask area of the normal industrial image to obtain a target defect image includes:
[0025] Invoke the sub - abnormal simulation feature combination corresponding to the multi - layer cross - attention layer of the defect imitation network model;
[0026] Based on the sub - abnormal simulation feature combination, control the fusion of the abnormal mask and the normal industrial image in the target mask area to obtain a target defect image.
[0027] Combined with the first aspect, the embodiments of the present application provide a sixth possible implementation manner of the first aspect, where calculating the similarity result between the target defect image and the target input text includes:
[0028] Encode the target defect image and the target input text respectively to obtain image encoding information and text encoding information, and extract the corresponding key features;
[0029] Calculate the similarity between the key features corresponding to the image encoding information and the text encoding information respectively, and generate a similarity result.
[0030] In a second aspect, the embodiments of the present application provide an industrial defect image generation device, and the device includes:
[0031] An acquisition module, configured to acquire a target input text and the normal industrial image corresponding to the target input text, and perform text inversion on the target input text to generate an abnormal mask; the abnormal mask includes a masked area;
[0032] An input module, configured to input the target input text, the normal industrial image, and the abnormal mask into a pre-established defect imitation network model based on a diffusion model; the defect imitation network model is established based on abnormal features transmitted by a defect reference network model; the defect reference network model is established based on real industrial defect images;
[0033] A reconstruction module, configured to control the defect imitation network model to mask a target masked area corresponding to the masked area in the normal industrial image based on the abnormal mask, obtain a normal industrial image with the target masked area, and reconstruct the target masked area of the normal industrial image to obtain a target defect image;
[0034] A calculation module, configured to calculate a similarity result between the target defect image and the target input text, and determine the target defect image as the industrial defect image corresponding to the target input text based on the similarity result.
[0035] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of any one of the industrial defect image generation methods are executed.
[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of any one of the industrial defect image generation methods are executed.
[0037] An embodiment of the present application provides an industrial defect image generation method. The method first obtains a target input text and a normal industrial image corresponding to the target input text, and text-inverts the target input text to generate an abnormal mask; the abnormal mask includes a masked area. Secondly, the target input text, the normal industrial image, and the abnormal mask are input into a defect imitation network model established in advance based on a diffusion model; the defect imitation network model is established based on abnormal features transmitted by a defect reference network model; the defect reference network model is established based on real industrial defect images. Then, the defect imitation network model is controlled to mask a target masked area corresponding to the masked area in the normal industrial image based on the abnormal mask, obtaining a normal industrial image with the target masked area, and reconstructing the target masked area of the normal industrial image to obtain a target defect image. Finally, a similarity result between the target defect image and the target input text is calculated, and based on the similarity result, the target defect image is determined to be the industrial defect image corresponding to the target input text. The method proposed in the present application achieves the effect of corresponding the target defect image to the abnormal mask, and the abnormal mask contains subtle defects, and the generation effect is ensured, thereby solving the problems of unstable generation effect, poor correspondence between defects and masks, and difficulty in generating subtle defects existing in the prior art, and ensuring the superiority of the deep generation model in the image generation technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] To more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.
[0039] Figure 1 FIG. 1 shows the first flowchart of the first industrial defect image generation method provided by the embodiment of the present application;
[0040] Figure 2 FIG. 2 shows the second flowchart of the first industrial defect image generation method provided by the embodiment of the present application;
[0041] Figure 3 FIG. 3 shows a schematic diagram of the abnormal mask and the target defect image provided by the embodiment of the present application;
[0042] Figure 4 FIG. 4 shows the flowchart of generating the similarity result provided by the embodiment of the present application;
[0043] Figure 5 FIG. 5 shows the structural block diagram of the first industrial defect image generation device provided by the embodiment of the present application;
[0044] Figure 6 The structural block diagram of the first electronic device provided by the embodiments of the present application is shown. Detailed implementation manners
[0045] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the accompanying drawings in the present application are only for the purposes of illustration and description, and are not used to limit the protection scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in the present application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations in the flowchart may not be implemented in sequence, and the steps without logical context relationships may be reversed or implemented simultaneously. In addition, those skilled in the art may add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of the present application.
[0046] In addition, the described embodiments are only some embodiments of the present application, rather than all embodiments. The components of the embodiments of the present application usually described and illustrated in the accompanying drawings here may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application claimed, but only represents the selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the protection scope of the present application.
[0047] It should be noted that the term "including" will be used in the embodiments of the present application to indicate the existence of the features stated thereafter, but does not exclude the addition of other features.
[0048] Currently, due to factors such as the particularity of industrial defect images themselves and data imbalance, the existing deep generation models have the following main problems: the generation effect is unstable, affecting the overall generation effect and the accuracy of downstream detection tasks; it is difficult to ensure that the abnormal regions in the generated defect images correspond to the preset defect masks, affecting the accuracy and robustness of downstream detection tasks; it is difficult to accurately reproduce small and complex defect features (such as cracks, holes, etc.), resulting in deficiencies in the detail restoration and authenticity of the generated images, and unable to meet the requirements for micro-defects in high-precision detection.
[0049] Based on this, the embodiments of the present application provide an industrial defect image generation method, device, electronic device, and medium, which will be described below through embodiments.
[0050] Embodiment 1
[0051] To facilitate the understanding of this embodiment, a method for generating industrial defect images disclosed in the embodiments of the present application will be introduced in detail first. As Figure 1 shown in the flowchart of a method for generating industrial defect images, as Figure 2 shown in another schematic diagram of the process of a method for generating industrial defect images, a method for generating industrial defect images provided by the present application includes:
[0052] S101. Obtain the target input text and the normal industrial image corresponding to the target input text, and perform text inversion on the target input text to generate an abnormal mask; the abnormal mask includes a masked area;
[0053] S102. Input the target input text, the normal industrial image, and the abnormal mask into a defect imitation network model pre-established based on a diffusion model; the defect imitation network model is established based on the abnormal features transmitted by a defect reference network model; the defect reference network model is established based on real industrial defect images;
[0054] S103. Control the defect imitation network model to mask the target masked area corresponding to the masked area in the normal industrial image based on the abnormal mask, obtain a normal industrial image with the target masked area, and reconstruct the target masked area of the normal industrial image to obtain a target defect image;
[0055] S104. Calculate the similarity result between the target defect image and the target input text, and determine the target defect image as the industrial defect image corresponding to the target input text based on the similarity result.
[0056] In step S101, the present application obtains the target input text based on a variety of methods, which may be industrial standards, product specifications, operating manuals, or user-defined inputs. The target input text should describe in detail the features of a normal industrial image, such as shape, color, texture, size, and any other features. The target text obtained needs to correspond to the normal industrial image corresponding to the target input text, that is, the normal industrial image must correspond to the subject in the target input text. For example, if the target input text is "A Hazelnut with a hole", the corresponding normal industrial image must contain "hazelnut", and the target input text is converted into an abnormal mask by executing a text inversion technique through a pre-trained diffusion model, wherein the diffusion model can generate a large number of similar images that are similar but not completely the same according to the text content of the target input text and the instance image, and the abnormal mask can cover a rich variety of defect forms, thereby reducing the difficulty of subsequent tasks of generating industrial defect images and improving the stability of the overall generation process, wherein one abnormal mask corresponds to one defect form, and multiple abnormal masks correspond to multiple forms, wherein a masking area has also been set in the abnormal mask, and the masking area is a white area.
[0057] In step S102, after obtaining the abnormal mask, the target input text, the normal industrial image and the abnormal mask are input into a defect imitation network model that has been pre-established based on a diffusion model; so that the defect imitation network model processes the target input text, the normal industrial image and the abnormal mask, wherein the defect imitation network model is established based on the abnormal features transmitted by the defect reference network model, and the defect reference network model is established based on real industrial defect images; that is, the defect reference network model is based on a pre-trained diffusion model and takes real industrial defect images as input to extract the features of real industrial defects of real industrial defect images, thereby ensuring the accuracy and effectiveness of the industrial defect images obtained by the defect imitation network model.
[0058] In the specific implementation process of step S102, there is an embodiment in which the defect simulation network model is established based on the abnormal characteristics transmitted by the defect reference network model, including:
[0059] S1021. Building an initial defect imitation network model based on a pre-trained diffusion model, and receiving an abnormal reference feature combination transmitted by the defect reference network model;
[0060] S1022. The initial defect simulation network model is combined with the abnormal reference feature reference to obtain a defect simulation network model.
[0061] In steps S1021 - S1022, with a pre - trained diffusion model as the framework, initialize the initial defect imitation network model, and retain the basic structure and parameters of the diffusion model to utilize its learned image generation ability. Design a feature interface in the initial defect imitation network model to receive the externally input abnormal reference feature combination. The feature interface should be able to process feature data of different dimensions and types to adapt to diverse abnormal reference features. The defect reference network model can extract and combine the key features of defects to form an abnormal reference feature combination. The abnormal reference feature combination includes multiple cross - attention layers, various features, and fusion weight parameters. Transmit the abnormal reference feature combination generated by the defect reference network model to the initial defect imitation network model through the feature interface, and based on the feature fusion mechanism pre - designed in the initial defect imitation network model, combine the received abnormal reference feature combination with the internal generation process of the initial defect imitation network model. After fusing the abnormal reference feature combination, fine - tune the initial defect imitation network model. During the fine - tuning process, a small number of real defect images or simulated defect images can be used as supervision signals to optimize the accuracy and diversity of the model in generating defect images.
[0062] In a specific implementation process of step S1021, there is an embodiment as follows: The receiving of the abnormal reference feature combination transmitted by the defect reference network model includes:
[0063] S10211. Extract multiple abnormal reference features and fusion weight parameters corresponding to different layers of the multi - layer cross - attention layer based on the pre - trained defect reference network model;
[0064] S10212. Aggregate the multiple abnormal reference features and fusion weight parameters corresponding to different layers of the multi - layer cross - attention layer to obtain an abnormal reference feature combination, and transmit the abnormal reference feature combination to the initial defect imitation network model based on a preset transmission method.
[0065] In steps S10211 - S10212, the defect reference network model is trained with real industrial defect images and can capture and learn the key features of defects. That is, after receiving a real industrial defect image, data augmentation is performed on the real industrial defect image, and the data augmentation includes denoising or adding noise. The defect reference network model usually includes multiple convolutional layers, multiple cross - attention layers, etc., for extracting features at different levels. Among them, the multiple cross - attention layers can capture abnormal features at different levels. The abnormal features include Key features and Value features, providing rich information for subsequent abnormal reference feature combinations. And the multiple abnormal reference features and fusion weight parameters corresponding to the multiple cross - attention layers in the defect reference network model are summarized. That is, the defect reference network model has multiple cross - attention layers, each layer has corresponding Key features, Value features, and corresponding fusion weight parameters. The fusion weight parameters are used to adjust the fusion ratio between the Key features and the Value features, and the multiple abnormal reference features and fusion weight parameters corresponding to the multiple cross - attention layers are packed to obtain an abnormal reference feature combination. Then, the abnormal reference feature combination is transmitted to the initial defect imitation network model based on a preset transmission method. The preset transmission method is a cross - branch feature transmission mechanism, so as to transmit the abnormal reference feature combination composed of the multiple abnormal reference features and fusion weight parameters to the initial defect simulation network model, and then fuse them to obtain the defect imitation network model.
[0066] In the specific implementation process of step S1022, there is an embodiment: The fusion of the initial defect imitation network model and the abnormal reference feature combination to obtain the defect imitation network model includes:
[0067] S10221. Determine the target level for fusing the initial defect imitation network model, and call the sub - abnormal reference feature combination of the defect reference network model at the target level;
[0068] S10222. Process the sub - abnormal reference feature combination to obtain a sub - abnormal simulation feature combination, and obtain the defect imitation network model based on the sub - abnormal simulation feature combination.
[0069] In steps S10221 - S10222, when the initial defect imitation network model is fused with the abnormal reference feature combination, the initial defect imitation network model also has multiple cross - attention layers. First, determine the target layer for fusing the initial defect imitation network model, so as to call the sub - abnormal reference feature combination of the defect reference network model at the target layer in the received abnormal reference feature combination, and process the sub - abnormal reference feature combination based on a preset processing network to obtain a sub - abnormal simulation feature combination. That is, the sub - abnormal simulation feature combination is the Key feature, Value feature, and corresponding fusion weight parameters at the target layer. The processing network is shown in formulas (1) - (2):
[0070]
[0071] Wherein, and respectively represent the Key feature and Value feature extracted by the reference network at the l - th layer, and respectively represent the Key feature and Value feature α extracted by the imitation network at the l - th layer l is the fusion weight parameter learned at the l - th layer. Then, based on the processing network, the sub - abnormal simulation feature combination of each layer is obtained, and based on the sub - abnormal simulation feature combination of each layer, the defect imitation network model is fused based on the initial defect imitation network model.
[0072] In step S103, after the defect imitation network model receives the target input text, the normal industrial image, and the abnormal mask, a masking operation is performed on the normal industrial image based on the abnormal mask. The defect imitation network model determines the target masking area corresponding to the masking area in the normal industrial image to be masked based on the masking area set by the abnormal mask. Wherein the normal industrial image is a normal hazelnut. After masking the normal industrial image based on the abnormal mask, a normal industrial image with a target masking area is obtained. Before masking, data augmentation is also performed on the normal industrial image and the abnormal mask. The data augmentation includes adding noise, etc., so as to pre - process the normal industrial image and the abnormal mask to make the normal industrial image and the abnormal mask convenient for masking operation, and reconstruct the target masking area of the normal industrial image to obtain a target defect image. The target defect image can be a defect image of a hazelnut with a hole, that is, the defect imitation network model can obtain multiple target defect images for the abnormal mask, as Figure 3 shown.
[0073] In the specific implementation process of step S103, there is an embodiment: the abnormal mask masks the target mask area corresponding to the mask area in the normal industrial image to obtain a normal industrial image with the target mask area, including:
[0074] S10311. Traverse the normal industrial image to identify the target mask area corresponding to the mask area in the abnormal mask;
[0075] S10312. Generate the defect morphology included in the abnormal mask on the target mask area to mask the target mask area of the normal industrial image.
[0076] In steps S10311 - S10312, first determine the mask area on the abnormal mask, which is the area for generating the defect morphology, and traverse the normal industrial image based on the mask area, that is, search for the target mask area on the normal image corresponding to the mask area of the abnormal mask for subsequent generation of the defect morphology. Generate the same defect morphology as the defect morphology included in the abnormal mask on the target mask area to mask the target mask area of the normal industrial image. Usually, it involves accurately copying the features such as the shape, size, and color of the defect morphology to the target mask area of the normal industrial image. By applying the generated defect morphology, the corresponding target mask area of the normal industrial image is masked or covered to highlight potential defects, which can be achieved by replacing the pixel values of the defect area with specific color or intensity values, or by overlaying a semi - transparent layer, etc.
[0077] In the specific implementation process of step S103, there is another embodiment: reconstructing the target mask area of the normal industrial image to obtain a target defect image, including:
[0078] S10321. Call the sub - abnormal simulation feature combination corresponding to the multi - layer cross - attention layer of the defect imitation network model;
[0079] S10322. Control the fusion of the abnormal mask and the normal industrial image in the target mask area based on the sub - abnormal simulation feature combination to obtain a target defect image.
[0080] In steps S10321 - S10322, the defect imitation network model has corresponding sub - abnormal simulation feature combinations on multiple cross - attention layers. Among them, the multiple cross - attentions are used to capture and integrate key information in the image at multiple scales. These cross - attention layers can enhance the defect imitation network model's perception ability of image details, so as to more accurately simulate defect features. In the process of obtaining the target defect image corresponding to the target input text, the sub - abnormal simulation feature combinations control the defect imitation network model to remove the image before the target masking area. On this basis, defect morphologies corresponding to the abnormal mask are generated in the target masking area. During the generation process, fusion is performed according to each level of the target masking area, and during the fusion process, weighted fusion is performed using the fusion weight parameters, so as to obtain the icon defect image corresponding to the target input text, thus generating defect morphologies corresponding to the abnormal mask in the target masking area of the normal industrial image.
[0081] In step S104, after obtaining the target curve image, the defect imitation network model also needs to calculate the similarity result between the target defect image and the target input text. The similarity result includes two types: similar and dissimilar. When the similarity result is similar, it is determined that the target defect image is the industrial defect image corresponding to the target input text, and then the industrial defect image is added to a pre - set database for subsequent segmentation or classification tasks, or the industrial defect image is saved to expand the training set, so as to ensure the quality and consistency of the newly added data. If the similarity result is dissimilar, the defect imitation network model is controlled to regenerate the industrial defect image corresponding to the target input text.
[0082] In the specific implementation process of step S104, there is an embodiment as follows: Figure 4 As shown, calculating the similarity result between the target defect image and the target input text includes:
[0083] S1041: Encode the target defect image and the target input text respectively to obtain image encoding information and text encoding information, and extract the corresponding key features;
[0084] S1042: Calculate the similarity between the key features corresponding to the image encoding information and the text encoding information respectively, and generate a similarity result.
[0085] In steps S1041 - S1042, the target input text and the target defect image are respectively encoded based on the text encoder and the image encoder in the CLIP model to obtain text encoding information and image encoding information. Corresponding key features are respectively extracted from the text encoding information and the image encoding information, and the similarity between the key features corresponding to the image encoding information and the text encoding information is calculated. Based on the set similarity threshold between the target defect image and the target input text, if the similarity exceeds the similarity threshold, the generated similarity result is similar, that is, it meets the requirements of the target input text. If the similarity is lower than the similarity threshold, the generated similarity result is dissimilar.
[0086] Embodiment 2
[0087] The present application also provides an industrial defect image generation device, as Figure 5 shown in the block diagram of an industrial defect image generation device. The functions implemented by this industrial defect image generation device correspond to the steps of performing an industrial defect image generation method on a terminal device. This device can be understood as a component of a server including a processor. The industrial defect image generation device described in the present application, the device includes:
[0088] An acquisition module 501, configured to acquire a target input text and a normal industrial image corresponding to the target input text, and perform text inversion on the target input text to generate an abnormal mask; the abnormal mask includes a masked area;
[0089] An input module 502, configured to input the target input text, the normal industrial image, and the abnormal mask into a defect imitation network model established in advance based on a diffusion model; the defect imitation network model is established based on abnormal features transmitted by a defect reference network model; the defect reference network model is established based on real industrial defect images;
[0090] A reconstruction module 503, configured to control the defect imitation network model to mask a target masked area corresponding to the masked area in the normal industrial image based on the abnormal mask, obtain a normal industrial image with the target masked area, and reconstruct the target masked area of the normal industrial image to obtain a target defect image;
[0091] A calculation module 504, configured to calculate a similarity result between the target defect image and the target input text, and determine the target defect image as the industrial defect image corresponding to the target input text based on the similarity result.
[0092] In a feasible implementation manner, the input module includes:
[0093] A construction module, which is used to construct an initial defect imitation network model based on a pre-trained diffusion model and receive an abnormal reference feature combination transmitted by the defect reference network model;
[0094] A fusion module, which is used to fuse the initial defect imitation network model and the abnormal reference feature reference combination to obtain a defect imitation network model.
[0095] In a feasible implementation manner, the input module further includes:
[0096] A training module, which is used to extract multiple abnormal reference features and fusion weight parameters corresponding to multiple cross-attention layers respectively based on a pre-trained defect reference network model;
[0097] A summarization module, which is used to summarize multiple abnormal reference features and fusion weight parameters corresponding to multiple cross-attention layers respectively to obtain an abnormal reference feature combination, and transmit the abnormal reference feature combination to the initial defect imitation network model based on a preset transmission method.
[0098] In a feasible implementation manner, the input module also includes:
[0099] A first calling module, which is used to determine the target level for fusing the initial defect imitation network model, so as to call the sub-abnormal reference feature combination of the defect reference network model at the target level;
[0100] A processing module, which is used to process the sub-abnormal reference feature combination to obtain a sub-abnormal simulation feature combination, so as to obtain a defect imitation network model based on the sub-abnormal simulation feature combination.
[0101] In a feasible implementation manner, the reconstruction module includes:
[0102] A traversal module, which is used to traverse the normal industrial image and identify the target masking area corresponding to the masking area in the abnormal mask;
[0103] A masking module, which is used to generate the defect morphology included in the abnormal mask on the target masking area to mask the target masking area of the normal industrial image.
[0104] In a feasible implementation manner, the reconstruction module further includes:
[0105] A second calling module, which is used to call the sub-abnormal simulation feature combination corresponding to the defect imitation network model on multiple cross-attention layers;
[0106] A control module, which is used to control the fusion of the abnormal mask and the normal industrial image in the target masking area based on the sub-abnormal simulation feature combination to obtain a target defect image.
[0107] In a feasible implementation manner, the calculation module includes:
[0108] An encoding module for encoding the target defect image and the target input text respectively to obtain image encoding information and text encoding information, and extracting corresponding key features;
[0109] A generation module for calculating the similarity between the key features corresponding to the image encoding information and the text encoding information respectively, and generating a similarity result.
[0110] Embodiment 3
[0111] The present application also provides an electronic device, as Figure 6 shown, including: a processor 601, a memory 602, and a bus 603. The memory 602 stores machine-readable instructions executable by the processor 601. When the electronic device runs, the processor 601 communicates with the memory 602 through the bus 603. When the machine-readable instructions are executed by the processor 601, the steps of any one of the industrial defect image generation methods are executed.
[0112] Embodiment 4
[0113] The present application also provides a computer-readable storage medium. A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of any one of the industrial defect image generation methods are executed.
[0114] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems and devices can refer to the corresponding processes in the method embodiments, which will not be repeated in the present application. In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point, the displayed or discussed coupling or direct coupling or communication connection between each other can be through some communication interfaces. The indirect coupling or communication connection of the devices or modules can be in an electrical, mechanical, or other form.
[0115] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0116] In addition, in each embodiment of the present application, each functional unit may be integrated into one processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit.
[0117] If the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical discs that can store program codes.
[0118] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims
1. An industrial defect image generation method, characterized in that The method includes: Obtain the target input text and the corresponding normal industrial image of the target input text, and text-invert the target input text to generate an anomaly mask; the anomaly mask includes a masked area; Input the target input text, the normal industrial image, and the anomaly mask into a pre-established defect imitation network model based on a diffusion model; the defect imitation network model is established based on the abnormal features transmitted by a defect reference network model; the defect reference network model is established based on real industrial defect images; Control the defect imitation network model to mask the target masked area corresponding to the masked area in the normal industrial image based on the anomaly mask, obtain a normal industrial image with the target masked area, and reconstruct the target masked area of the normal industrial image to obtain a target defect image; Calculate the similarity result between the target defect image and the target input text, and determine the target defect image as the industrial defect image corresponding to the target input text based on the similarity result.
2. The method according to claim 1, characterized in that, The specific steps for establishing the defect imitation network model based on the abnormal features transmitted by the defect reference network model include: Construct an initial defect imitation network model based on a pre-trained diffusion model, and receive the abnormal reference feature combination transmitted by the defect reference network model; Fuse the initial defect imitation network model and the abnormal reference feature reference combination to obtain a defect imitation network model.
3. The method according to claim 2, wherein The receiving of the abnormal reference feature combination transmitted by the defect reference network model includes: Extract various abnormal reference features and fusion weight parameters corresponding to multiple cross-attention layers respectively based on a pre-trained defect reference network model; Summarize the various abnormal reference features and fusion weight parameters corresponding to multiple cross-attention layers respectively to obtain an abnormal reference feature combination, and transmit the abnormal reference feature combination to the initial defect imitation network model based on a preset transmission method.
4. The method according to claim 2, wherein The fusing of the initial defect imitation network model and the abnormal reference feature combination to obtain a defect imitation network model includes: Determine the target layer for fusing the initial defect imitation network model to call the sub-abnormal reference feature combination of the defect reference network model at the target layer; Process the sub-abnormal reference feature combination to obtain a sub-abnormal simulation feature combination, and obtain a defect imitation network model based on the sub-abnormal simulation feature combination.
5. The method according to claim 1, characterized in that, The anomaly mask masks the target masked area corresponding to the masked area in the normal industrial image to obtain a normal industrial image with the target masked area, including: Traverse the normal industrial image to identify the target masked area corresponding to the masked area in the anomaly mask; Generate the defect form included in the anomaly mask on the target masked area to mask the target masked area of the normal industrial image.
6. The method according to claim 1, characterized in that The reconstructing of the target masked area of the normal industrial image to obtain a target defect image includes: Call the sub-abnormal simulation feature combination corresponding to the defect imitation network model on multiple cross-attention layers; Based on the sub - anomaly simulation feature combination, control the fusion of the anomaly mask and the normal industrial image in the target masking area to obtain a target defect image.
7. The method according to claim 1, characterized in that, Calculating the similarity result between the target defect image and the target input text includes: Encoding the target defect image and the target input text respectively to obtain image encoding information and text encoding information, and extracting corresponding key features; Calculating the similarity between the key features corresponding to the image encoding information and the text encoding information respectively, and generating a similarity result.
8. An industrial defect image generation device, characterized in that, The device includes: An acquisition module, configured to acquire a target input text and the normal industrial image corresponding to the target input text, and perform text inversion on the target input text to generate an anomaly mask; the anomaly mask includes a masking area; An input module, configured to input the target input text, the normal industrial image, and the anomaly mask into a defect imitation network model established in advance based on a diffusion model; the defect imitation network model is established based on the anomaly features transmitted by a defect reference network model; the defect reference network model is established based on real industrial defect images; A reconstruction module, configured to control the defect imitation network model to mask the target masking area corresponding to the masking area in the normal industrial image based on the anomaly mask, to obtain a normal industrial image with the target masking area, and reconstruct the target masking area of the normal industrial image to obtain a target defect image; A calculation module, configured to calculate the similarity result between the target defect image and the target input text, and determine the target defect image as the industrial defect image corresponding to the target input text based on the similarity result.
9. An electronic device, characterized in that, Including: A processor, a memory, and a bus, where the memory stores machine - readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine - readable instructions are executed by the processor, the steps of an industrial defect image generation method according to any one of claims 1 to 7 are executed.
10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer - readable storage medium. When the computer program is run by the processor, the steps of an industrial defect image generation method according to any one of claims 1 to 7 are executed.
Citation Information
Cited By
Urban rail transit engineering construction potential safety hazard rectification rechecking method and system
CN121010782A