A method for generating surface defect data of thick plates based on stable diffusion model

Abstract

This application proposes a method for generating surface defect data for thick plates based on a stable diffusion model. This method belongs to the fields of thick plate production and computer vision technology. The method includes: obtaining a guide image of the defect location and a text description related to the defect type; extracting features from the guide image of the defect location using an image adapter; extracting features from the text description related to the defect type using a CLIP network; inputting the guide features and text prompt features into a stable diffusion model, optimizing the latent space features through multi-step iterations, and generating a defect image; inputting the guide features, the latent space features of the last iteration, and the text prompt features into a mask generator to generate a defect mask; and using the defect image and defect mask as samples of thick plate surface defect data. This method eliminates the tedious and time-consuming work of image acquisition and manual annotation, and can automatically generate defect images and their corresponding pixel-level annotation masks, effectively expanding the data set.

Description

Technical Field

[0001] The present invention belongs to the fields of thick plate production and computer vision technology, and particularly relates to a method for generating thick plate surface defect data based on a stable diffusion model. Background Art

[0002] As an important industrial base material, thick plates have been widely used in many fields due to their excellent mechanical properties and superior processing characteristics. In particular, thick plates are an indispensable core material in infrastructure construction, shipbuilding, defense industry, and heavy machinery. The production process of thick plates is complex and technically demanding. Its main process flow includes four key links: heating, rolling, cooling, and heat straightening. However, due to the complexity of the thick plate production process and the fine operation requirements of each link, some surface defects such as plaques, black lines, and edge cracks are often unavoidable in actual production. These surface defects not only affect the appearance quality of the thick plates, but may also reduce the fatigue resistance and corrosion resistance of the thick plates, thereby significantly affecting the performance and service life of the thick plates.

[0003] Timely and accurate surface defect detection is crucial for ensuring the quality of thick plate products, improving production efficiency, and ensuring safety. Traditional manual inspection methods rely on empirical judgment, making them inefficient and unable to meet the demands of modern large-scale production. Computer vision-based inspection methods offer significant advantages, including high efficiency, non-contact detection, and the ability to simultaneously identify multiple defect types. Defect segmentation methods, in particular, can precisely delineate defect regions at the pixel level, enabling fine-grained detection. This approach not only detects defect locations but also accurately quantifies their area, shape, and location, making it particularly well-suited for complex inspection scenarios. Currently, most high-performing defect segmentation methods are based on deep learning techniques, but these methods require large amounts of pixel-level annotated data for training. However, in real-world production environments, the distribution of defects on thick plate surfaces is often random due to the uncertainty and scarcity of defect occurrence, posing numerous challenges in acquiring and constructing datasets. Furthermore, pixel-level annotation typically requires significant human and material resources, particularly the time and technical expertise of specialized personnel, making the process cumbersome and time-consuming. These factors collectively hinder the collection and production of datasets for thick plate surface defect segmentation, limiting the effectiveness and reliability of existing deep learning models in practical applications. Therefore, developing efficient synthetic data to expand the training set has become an urgent need to improve the performance of defect detection systems. Summary of the Invention

[0004] To address the shortcomings of existing technologies, a method for generating thick plate surface defect data based on a stable diffusion model is proposed to solve the problem of insufficient datasets in thick plate surface defect segmentation. The method of this application can automatically generate thick plate surface defect images and their corresponding mask information.

[0005] In a first aspect, the present application proposes a method for generating thick plate surface defect data based on a stable diffusion model, comprising:

[0006] Obtain a guide image of the defect location and a text description of the defect type;

[0007] An image adapter is used to extract features from the guide image of the defect position to obtain a guide feature;

[0008] The CLIP network is used to extract features from text descriptions related to defect types to obtain features of text prompts;

[0009] The guidance features and text prompt features are input into the stable diffusion model, and the latent space features are optimized through multi-step iterations to generate defect images;

[0010] Input the guided features, the latent space features of the last iteration, and the features of the text prompt into the mask generator to generate the defect mask;

[0011] The defect image and the defect mask are used as samples of thick plate surface defect data.

[0012] The guide image of the defect position is a guide image with the same size as the target image;

[0013] The text description related to the defect type includes: the category of the defect in the target image and the corresponding text prompt.

[0014] The image adapter includes: an image downsampling module and a feature extraction module;

[0015] The image downsampling module is configured to perform image downsampling on the guide image of the defect position to obtain a first feature;

[0016] The feature extraction module is used to extract the first feature to obtain a second feature, and use the second feature as a guiding feature.

[0017] The image downsampling module includes: a residual block, an activation function, and a fully connected layer; the residual block includes: a first convolutional layer and a second convolutional layer, which are fused with the output residual features of the residual block through a jump connection, the residual block is connected to the activation function, and the activation function is connected to the fully connected layer.

[0018] The feature extraction module includes: a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer, and an eighth convolutional layer. The output of the fifth convolutional layer is used as the input of the sixth convolutional layer, the output of the sixth convolutional layer is used as the input of the seventh convolutional layer, and the output of the seventh convolutional layer is used as the input of the eighth convolutional layer. The output of the fifth convolutional layer, the output of the sixth convolutional layer, the output of the seventh convolutional layer, and the output of the eighth convolutional layer are used as the second feature. The calculation formula is as follows:

[0019]

[0020] Among them, G e For the second feature, is the output of the fifth convolutional layer, is the output of the sixth convolutional layer, is the output of the seventh convolutional layer, is the output of the eighth convolutional layer, ReLU is the activation function, and BN represents batch normalization. represents a convolution kernel with a step size of s and a convolution kernel size of k, G d The first feature.

[0021] The guide features and text prompt features are input into the stable diffusion model, and the latent space features are optimized through multi-step iterations to generate defect images, including:

[0022] Using the features of the guidance features and text prompts, the initial noise is denoised multiple times to obtain the latent space features;

[0023] The image autoencoder in the stable diffusion model is used to decode the latent space features to obtain the defect image.

[0024] The mask generator includes: a first cross attention module, a second cross attention module and a third cross attention module;

[0025] The first cross attention module includes: a plurality of parallel first attention units, configured to input the latent space features of the last iteration and the features of the text prompt into the plurality of parallel attention units respectively, to obtain a plurality of first attention outputs;

[0026] The second cross attention module includes: a splicing unit and a second cross attention unit, configured to input the latent space features of the last iteration into the splicing unit for splicing to obtain a splicing result, and input the splicing result and the plurality of first attention outputs into the second cross attention unit to obtain a second cross attention output;

[0027] The third cross-attention module includes: a third cross-attention unit, which is used to input the guiding feature and the second cross-attention output into the third cross-attention unit to obtain a defect mask.

[0028] In a second aspect, the present application proposes an electronic device comprising: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the method for generating thick plate surface defect data based on a stable diffusion model.

[0029] In a third aspect, the present application proposes a computer-readable storage medium storing executable instructions, which, when executed, enable a processor to execute the method for generating thick plate surface defect data based on a stable diffusion model.

[0030] In a fourth aspect, the present application proposes a computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the method for generating thick plate surface defect data based on a stable diffusion model.

[0031] Beneficial effects:

[0032] This application proposes a method for generating surface defect data for thick plates based on a stable diffusion model. This method eliminates the tedious and time-consuming task of image acquisition and manual annotation, automatically generating defect images and their corresponding pixel-level annotation masks, and effectively expanding the dataset. Compared to the existing stable diffusion model, which is limited to generating defect images only, this application innovatively proposes a mask generator that simultaneously generates defect images and their corresponding pixel-level annotation masks. Compared to existing dataset generation methods based on open vocabulary pixel-level annotation, this application utilizes an image adapter to transfer the natural image generation capabilities of the stable model to the generation of thick plate surface defect datasets, demonstrating significant advantages in practical industrial applications. Overall, this application, based on a stable diffusion model, can automatically generate surface defects and their corresponding masks for thick plates, significantly reducing the complexity of data collection and annotation, and effectively addressing the pain points of traditional methods that rely on extensive manual annotation. This method not only efficiently expands the dataset size but also significantly improves the quality and diversity of the data. Due to its efficient automation, this method is particularly suitable for industrial inspection scenarios with limited actual data, and has important engineering applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flow chart of a method for generating thick plate surface defect data based on a stable diffusion model according to an embodiment of the present application;

[0034] Figure 2 A schematic flow chart of a method for generating surface defect data of thick plates based on a stable diffusion model according to an embodiment of the present application;

[0035] Figure 3 The algorithm framework of the embodiment of this application;

[0036] Figure 4 The guiding images of the embodiment of the present application; wherein (a) is a guiding image of a black line, and (b) is a guiding image of an edge crack;

[0037] Figure 5 Two defect images generated in the embodiment of the present application; (a) is a generated black line image, and (b) is a generated edge crack image;

[0038] Figure 6 Images of two defects and corresponding masks generated in the embodiments of the present application; among them, (a) is the generated black line image; (b) is the mask corresponding to the generated black line image, (c) is the generated edge crack image, and (d) is the mask corresponding to the generated edge crack image. DETAILED DESCRIPTION

[0039] The specific implementation of the present application is further described in detail below with reference to the accompanying drawings and examples.

[0040] In the actual thick plate production process, due to factors such as uneven heating of the slab or insufficient dephosphorization, various defects such as plaques, black lines, and edge cracks often appear on the thick plate surface. These defects not only affect the quality of the thick plate, but also reduce its performance. Therefore, timely and accurate detection of thick plate surface defects is of great significance for optimizing the production process and performing defect treatment. Defect segmentation, as an effective detection method, can provide key information such as the detailed shape and area of ​​the defect, providing an important reference for subsequent thick plate processing. However, existing segmentation algorithms usually require a large amount of labeled training data. In an actual production environment, obtaining a pixel-level labeled thick plate surface defect dataset faces two major challenges: the occurrence of defects is characterized by uncertainty and scarcity; the pixel-level labeling process requires a large amount of manpower and financial resources.

[0041] In the thick plate production process, surface defect detection is a core component of product quality control. In recent years, deep learning-based image segmentation technology has provided innovative solutions for thick plate surface defect detection. However, training deep learning models typically requires a large amount of labeled data. In real-world production, existing thick plate surface defect datasets generally face the key challenge of insufficient sample size due to the scarcity of defect samples and the high cost of manual annotation.

[0042] To address these issues, this application proposes a method for generating surface defect data for thick plates based on a stable diffusion model. This method can automatically generate a large number of high-quality defect images and their corresponding pixel-level masks, effectively expanding the thick plate surface defect dataset for segmentation tasks. This method comprises two core modules: an ImageAdapter and a Mask Generator. The specific workflow is as follows: a guidance image of the defect location is fed into the ImageAdapter module to extract defect location information. Simultaneously, a textual description containing the defect type and image style is fed into the CLIP network to extract linguistic features. These two features are then fed into a stable diffusion model to control and generate the final defect image. During mask generation, the Mask Generator module generates the corresponding defect mask by combining the defect visual features from the stable diffusion model, the features of the textual prompt, and the guidance information extracted by the adapter. This application implements the algorithm encapsulation in a Python environment by integrating libraries such as transformers, diffusers, and accelerate. Experimental results demonstrate that this method can effectively expand defect segmentation datasets and significantly improve the accuracy of downstream defect segmentation tasks. This invention provides a new approach to addressing the data scarcity problem in industrial surface defect segmentation tasks and has important practical significance.

[0043] Example 1:

[0044] This embodiment proposes a method for generating thick plate surface defect data based on a stable diffusion model, such as Figure 1 、 Figure 2 Shown, including:

[0045] Step S1: Obtain a guide image of the defect location and a text description of the defect type;

[0046] In this embodiment, the implementation case comes from a large domestic steel company, which is equipped with advanced thick plate production lines. In actual application, this embodiment uses the thick plate production data set of the company for verification. Specifically, after the thick plate is rolled, it is transported to the quality inspection department via rollers. During this process, the industrial camera equipped with the production line shoots the surface of the thick plate in real time and collects images of the steel plate surface. For the two typical defects of edge cracks and black lines, this embodiment focuses on collecting and screening representative defect image samples to ensure the diversity and clarity of the samples. To facilitate subsequent processing, this embodiment uniformly crops the collected defect images, and the resolution of the cropped images is 320×320 pixels. Subsequently, the X-AnyLabeling software is used to perform pixel-level annotation on the images, and a real thick plate surface defect image dataset with segmentation labels is constructed. This dataset contains two types of defect samples, including 900 black line defect samples and 800 edge crack defect samples, which provides a solid data foundation for subsequent model training and performance verification.

[0047] Using the surface defect data of thick plates, this embodiment Figure 3 The generative model shown in the figure is trained. Based on the stable-diffusion-xl-base-1.0 stable diffusion model, the entire training process is divided into two phases. The first phase trains the image adapter for 8000 training iterations, using the AdamW optimizer, a learning rate of 1\times{10}^{-5}, and a batch size of 16. The second phase maintains the parameters of the image adapter module and trains the mask generator. This phase also trains for 8000 iterations, with a learning rate of 8\times{10}^{-6} and a batch size of 4. To ensure training efficiency and computational performance, all experiments were conducted on a high-performance computing platform equipped with NVIDIA V100 GPUs.

[0048] Based on the trained generative model, thick plate surface defect images and their corresponding labels are generated, and these generated sample pairs are used for the thick plate surface defect detection task.

[0049] In step S1, the defect type-related text description includes: the category of the defect in the target image and the corresponding text prompt.

[0050] Generate a text prompt T corresponding to the defect name. For example, if you want to generate an image of the "edge crack" defect, the generated prompt is:

[0051] "an image of[edge cracks]on a heavy-plate.";

[0052] The guide image of the defect position is a guide image with the same size as the target image;

[0053] In this embodiment, a guide image G with the same size as the target image is constructed to generate the approximate location of the defect. Specifically, since the generation position and size of each defect have certain rules, these rules can be followed to generate a rough defect guide image G based on the rules of the defect generation position. For example, black line defects on the surface of a thick plate usually occur at the left and right edges of the image, parallel to both sides of the plate surface, and run through the entire image. According to this rule, a through rectangular bounding box is randomly generated on a completely black 320×320 image. Then, a random walk algorithm is applied within the bounding box to form a random shape of the defect area. Through this process, a group of 1 pixels are eventually formed, representing the approximate area of ​​the defect, and the other values ​​represent the normal thick plate area (no defects). For example: select a random point (x, y) in the box, and freely walk in the eight directions of (x+1, y), (x, y+1), (x+1, y+1), (x+1, y-1), (x-1, y+1), (x-1, y), (x, y-1), (x-1, y-1), and finally form a random white connected area, indicating the approximate location of the black line defect. Figure 4 As shown, Figure 4 (a) is the guidance image of the black line, Figure 4 (b) is the guidance image of edge cracking.

[0054] Step S2: extracting features from the guide image of the defect location using an image adapter to obtain a guide feature;

[0055] The image adapter includes: an image downsampling module and a feature extraction module;

[0056] The image downsampling module is configured to perform image downsampling on the guide image of the defect position to obtain a first feature;

[0057] The feature extraction module is used to extract the first feature to obtain a second feature, and use the second feature as a guiding feature.

[0058] The image downsampling module includes: a residual block, an activation function, and a fully connected layer; the residual block includes: a first convolutional layer and a second convolutional layer, which are fused with the output residual features of the residual block through a jump connection, the residual block is connected to the activation function, and the activation function is connected to the fully connected layer.

[0059] Specifically, in this embodiment, G is input into the designed image adapter to extract the position feature G generated by the defect contained in G. d and G eThe image adapter aims to extract important features from the guidance image and feed these features into the stable diffusion model and mask generator. Figure 3 As shown in Figure 2, the image adapter consists of two core modules, including an image downsampling module and a feature extraction module. First, the guidance image G is input into the image downsampling module φ resnet In the above example, we extract the feature G d :

[0060] G d =φ resnet (G). (1)

[0061] Among them, G d The first feature.

[0062] Next, G d Input to the feature extraction module φ conv In the above example, we further extract the feature G e :

[0063] G e =φ conv (G). (2)

[0064] Among them, G e For the second feature, G d The feature of G is used in the mask generator to provide the location information of the defect; e Contains rich multi-scale features to guide the stable diffusion model to control the approximate location of defect generation. Specifically, φ resnet The module contains a residual block, which consists of two convolutional layers (with kernel sizes of 5 and 3 respectively) and fuses the input with the residual features through a skip connection:

[0065]

[0066] Then, it is activated by the ReLU activation function and outputs G through a fully connected layer. d :

[0067]

[0068] φ conv The module consists of four convolutional layers for extracting features of different scales. Specifically, the feature extraction module includes: the fifth convolutional layer, the sixth convolutional layer, the seventh convolutional layer and the eighth convolutional layer. The output of the fifth convolutional layer is used as the input of the sixth convolutional layer, the output of the sixth convolutional layer is used as the input of the seventh convolutional layer, and the output of the seventh convolutional layer is used as the input of the eighth convolutional layer. The output of the fifth convolutional layer, the output of the sixth convolutional layer, the output of the seventh convolutional layer and the output of the eighth convolutional layer are used as the second feature. The calculation formula is as follows:

[0069]

[0070] Among them, G e For the second feature, is the output of the fifth convolutional layer, is the output of the sixth convolutional layer, is the output of the seventh convolutional layer, is the output of the eighth convolutional layer, ReLU is the activation function, and BN represents batch normalization. represents a convolution kernel with a step size of s and a convolution kernel size of k, G d is the first feature. Through the above multi-scale feature extraction process, we finally get G e :

[0071]

[0072] Step S3: Using CLIP (Contrastive Language-Image Pre-training) network to extract features from text descriptions related to defect types to obtain features of text prompts;

[0073] In this example, the text prompt T is input into the CLIP network proposed by OpenAI. After being processed by the embedding layer, the text feature t is generated. As a pre-trained model with cross-modal alignment capabilities, the CLIP network can effectively extract semantic information from the text. This semantic information is used to control image generation and ensure that the generated image content is highly consistent with the text description. Figure 3 As shown, the text-related features t obtained in the above steps and the multi-scale features G generated by the control defects are combined e The information from the text and the guidance image is fed into a stable diffusion model, which controls the generation of defect images. The diffusion model starts with a noisy image and generates defects by gradually denoising it.

[0074] Step S4: Input the guiding features and text prompt features into the stable diffusion model, and generate defect images through multi-step iterative optimization of latent space features, including:

[0075] Step S4.1: Using the guidance features and the features of the text prompt, perform multiple iterations of denoising on the initial noise to obtain the latent space features;

[0076] Step S4.2: Use the image autoencoder in the stable diffusion model to decode the latent space features to obtain the defect image.

[0077] In this embodiment, the guide feature G e and text features t as control conditions, input into the stable diffusion model, the model from the noise image XM Let's start by generating a standard image by gradually removing noise:

[0078]

[0079] Among them, ∈ θ Represents the Unet network, ∈ θ (X m ,m,t,G e ) is the noise predicted by the model, m represents the number of steps of current denoising, α M-m represents the noise scheduling parameter, X M-m represents the image after denoising in the Mmth step, X m Represents the image after denoising in the mth step.

[0080] After M iterations of denoising, the generated latent map X0 becomes the latent representation of the final defect image. Then, the image autoencoder in the stable diffusion model is used to decode X0 into the final defect image:

[0081] I=f(X0), (11)

[0082] Where f represents the autoencoder.

[0083] Assume that the number of iterations of the diffusion model is M = 50, and the representation of the latent space of the defect is generated through 50 steps of iteration. Then, the image autoencoder in the stable diffusion model is used to decode the latent features of the defect into the final defect image I. The generated defect image is as follows Figure 5 As shown, Figure 5 (a) is the generated black line image, Figure 5 (b) is the generated edge crack image.

[0084] Step S5: input the guiding features, the latent space features of the last iteration, and the features of the text prompt into the mask generator to generate a defect mask;

[0085] The mask generator includes: a first cross attention module, a second cross attention module and a third cross attention module;

[0086] The first cross attention module includes: a plurality of parallel first attention units, configured to input the latent space features of the last iteration and the features of the text prompt into the plurality of parallel attention units respectively, to obtain a plurality of first attention outputs;

[0087] The second cross attention module includes: a splicing unit and a second cross attention unit, configured to input the latent space features of the last iteration into the splicing unit for splicing to obtain a splicing result, and input the splicing result and the plurality of first attention outputs into the second cross attention unit to obtain a second cross attention output;

[0088] The third cross-attention module includes: a third cross-attention unit, which is used to input the guiding feature and the second cross-attention output into the third cross-attention unit to obtain a defect mask.

[0089] In this embodiment, the mask generator is dedicated to integrating defect-related text features, visual features, and guidance features to generate accurate defect segmentation masks. Figure 3 As shown in the figure, the generator consists of three cross attention modules. First, the last step of the generation process ∈ θ The defect visual features extracted from V={V1,V2,…,V n} and text feature t are input into the first cross attention unit. This unit contains t parallel attention modules Used to generate visual-text features M TV :

[0090]

[0091] Next, a second cross attention unit C is used VM Processing V and M TV , to further extract key defect-related visual features M VM :

[0092]

[0093] Then, the guided feature G d and M VM Input to the third cross attention unit C GM , obtain the generated defect-related features

[0094]

[0095] Finally, The decoded image is the final defect mask image M. From a structural design perspective, the core architecture of the mask generator is based on the Transformer's decoding layer, which integrates three attention modules, each of which is implemented based on the MultiHeadAttention mechanism. This carefully designed structure can more fully integrate textual, visual, and guidance features to generate accurate defect segmentation masks.

[0096] Step S6: taking the defect image and the defect mask as samples of the thick plate surface defect data.

[0097] In this embodiment, there is no particular order between step S2 and step S3, and there is no particular order between step S4 and step S5.

[0098] In this embodiment, the text-related features t and the features G that guide the defect mask obtained in the above steps are d and the latent spatial features of defects generated during the diffusion process, and input them into Figure 3 In the mask generator shown, a mask M corresponding to the synthetic defect is generated. Figure 6 The image pairs of synthesized edge cracks and black line defects and their corresponding masks are shown, where Figure 6 (a) is the generated black line image; Figure 6 (b) The mask corresponding to the generated black line image, Figure 6 (c) is the generated edge crack image, Figure 6 (d) Mask corresponding to the generated edge crack image;

[0099] After generating 900 pairs of black lines and 800 pairs of edge crack image-mask pairs, the generated dataset was organized and fed into the downstream segmentation task to train the segmentation model SETR. The number of training iterations was set to 20,000, and the learning rate was set to 0.01. Table 1 shows the segmentation results obtained by training SETR on different data types.

[0100] Table 1 Segmentation effect of different types of data

[0101]

[0102] In summary, the present embodiment proposes a method for generating surface defect data of thick plates based on a stable diffusion model, which automatically generates defect images and masks using only a text description of one defect type and a guide image of the approximate location of the defect. The generated defect image has extremely high visual realism, and its corresponding segmentation mask and defect also have a high degree of consistency. The high-quality sample pairs generated by the present invention can effectively expand the scale of existing thick plate surface defect datasets and significantly improve the diversity of the dataset. The data pairs generated by the method of this embodiment provide richer and more reliable data support for the training of the segmentation model. Specifically, a training strategy combining a generated dataset with a real dataset is adopted: the segmentation algorithm is first pre-trained using the generated dataset, and then fine-tuned on the real dataset. Practical results show that this method can effectively improve the performance of the segmentation algorithm. Although only 900 groups of black lines and 800 groups of edge crack samples were generated, the segmentation mIoU still achieved a significant improvement of 0.4%.

[0103] Example 2:

[0104] This embodiment proposes an electronic device, comprising: one or more processors, and a memory, wherein the memory is used to store instructions. When the instructions are executed by the one or more processors, the one or more processors execute the method for generating thick plate surface defect data based on a stable diffusion model.

[0105] The electronic device can be a computer or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program that, when executed by the processor, implements a method for generating thick plate surface defect data based on a stable diffusion model as described in the embodiments. It is understood that the electronic device may also include an input / output (I / O) interface and a communication component.

[0106] The processor is configured to execute all or part of the steps of the method for generating thick plate surface defect data based on a stable diffusion model as described in the above embodiment. The memory is configured to store various types of data, such as instructions for any application or method in the electronic device, as well as data related to the application.

[0107] The processor can be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components, and is used to execute the method for generating thick plate surface defect data based on a stable diffusion model described in the above embodiment.

[0108] Example 3:

[0109] This embodiment provides a computer-readable storage medium storing executable instructions. When the instructions are executed, if they are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0110] The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of a method for generating thick plate surface defect data based on a stable diffusion model described in various embodiments of the present application.

[0111] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (for example, SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR abbreviation, memory data register) memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, CD, server, APP (Application, abbreviation of application software) application store and other media that can store program verification codes, on which a computer program is stored. When the computer program is executed by the processor, it can implement the above-mentioned various steps of the method for generating thick plate surface defect data based on the stable diffusion model.

[0112] Example 4:

[0113] This embodiment provides a computer program product, including a computer program or instructions. When the computer program or instructions are executed by a processor, the method for generating thick plate surface defect data based on a stable diffusion model is implemented.

[0114] Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution can be embodied in the form of a computer program product.

[0115] The various embodiments in this application are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0116] The scope of protection of this application is not limited to the above-described embodiments. Obviously, those skilled in the art may make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, the disclosure is intended to include such modifications and variations.

Claims

1. A method for generating surface defect data of thick plates based on a stable diffusion model, characterized in that: include: Obtain a guide image of the defect location and a text description of the defect type; An image adapter is used to extract features from the guide image of the defect position to obtain a guide feature; The CLIP network is used to extract features from text descriptions related to defect types to obtain features of text prompts; The guiding features and text prompt features are input into the stable diffusion model, and the latent space features are optimized through multi-step iterations to generate defect images; Input the guided features, the latent space features of the last iteration, and the features of the text prompt into the mask generator to generate the defect mask; The defect image and the defect mask are used as samples of thick plate surface defect data; The image adapter includes: an image downsampling module and a feature extraction module; The image downsampling module is configured to perform image downsampling on the guide image of the defect position to obtain a first feature; The feature extraction module is used to extract the first feature to obtain a second feature, and use the second feature as a guiding feature; The feature extraction module includes: a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer, and an eighth convolutional layer. The output of the fifth convolutional layer is used as the input of the sixth convolutional layer, the output of the sixth convolutional layer is used as the input of the seventh convolutional layer, and the output of the seventh convolutional layer is used as the input of the eighth convolutional layer. The output of the fifth convolutional layer, the output of the sixth convolutional layer, the output of the seventh convolutional layer, and the output of the eighth convolutional layer are used as the second feature. The calculation formula is as follows: Among them, G e For the second feature, is the output of the fifth convolutional layer, is the output of the sixth convolutional layer, is the output of the seventh convolutional layer, is the output of the eighth convolutional layer, ReLU is the activation function, and BN represents batch normalization. represents a convolution kernel with a step size of s and a convolution kernel size of k, G d is the first characteristic; The mask generator includes: a first cross attention module, a second cross attention module and a third cross attention module; The first cross attention module includes: a plurality of parallel first attention units, configured to input the latent space features of the last iteration and the features of the text prompt into the plurality of parallel attention units respectively, to obtain a plurality of first attention outputs; The second cross attention module includes: a splicing unit and a second cross attention unit, configured to input the latent space features of the last iteration into the splicing unit for splicing to obtain a splicing result, and input the splicing result and the plurality of first attention outputs into the second cross attention unit to obtain a second cross attention output; The third cross-attention module includes: a third cross-attention unit, which is used to input the guiding feature and the second cross-attention output into the third cross-attention unit to obtain a defect mask.

2. The method for generating thick plate surface defect data based on a stable diffusion model according to claim 1, characterized in that: The guide image of the defect position is a guide image with the same size as the target image; The text description related to the defect type includes: the category of the defect in the target image and the corresponding text prompt.

3. The method for generating thick plate surface defect data based on a stable diffusion model according to claim 1, characterized in that: The image downsampling module includes: a residual block, an activation function, and a fully connected layer; the residual block includes: a first convolutional layer and a second convolutional layer, which are fused with the output residual features of the residual block through a jump connection, the residual block is connected to the activation function, and the activation function is connected to the fully connected layer.

4. The method for generating thick plate surface defect data based on a stable diffusion model according to claim 1, characterized in that: The guide features and text prompt features are input into the stable diffusion model, and the latent space features are optimized through multi-step iterations to generate defect images, including: Using the features of the guidance features and text prompts, the initial noise is denoised multiple times to obtain the latent space features; The image autoencoder in the stable diffusion model is used to decode the latent space features to obtain the defect image.

5. An electronic device, characterized in that: include: One or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the method for generating thick plate surface defect data based on a stable diffusion model as described in any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that It stores executable instructions, which, when executed, enable a processor to execute a method for generating thick plate surface defect data based on a stable diffusion model as described in any one of claims 1 to 4.

7. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the method for generating thick plate surface defect data based on a stable diffusion model as described in any one of claims 1 to 4 is realized.

Images