A small sample precast concrete defect image generation method and system

Abstract

The application relates to a small sample prefabricated concrete defect image generation method and system, which comprises the following steps: acquiring a defect image and a non-defect image with different textures, processing the defect image, obtaining a small sample defect image training set and a normal image data set; based on the small sample defect image training set, appearance features and spatial features of the defect image are separated according to a stable diffusion model combined with a control net model, and an appearance feature embedding vector is obtained; general knowledge is smoothly migrated to the prefabricated concrete defect field according to a low-rank adaptive parameter fine-tuning algorithm, and a prefabricated concrete defect mask is generated; based on the appearance feature embedding vector, the generated defect appearance content is controlled according to a feature learning model, the normal image data and the defect mask are taken as input data in combination with the control net model, the position of the generated defect appearance is controlled, and a large number of high-quality defect images are generated. The application can generate high-quality prefabricated concrete defect images with diversity by using a small amount of samples.

Description

Technical Field

[0001] This application relates to the field of surface image dataset expansion technology, and in particular to a method and system for generating small sample precast concrete defect images. Background Technology

[0002] Precast concrete (PC) components are a core part of modern industrialized construction, and their surface quality directly affects the structural safety, durability, and aesthetics of buildings. Traditional PC component surface defect detection relies primarily on manual visual inspection, a method that is inefficient, highly subjective, and prone to missed or misjudged defects due to the experience and fatigue of the inspectors. In recent years, deep learning-based computer vision technology has provided new solutions for automated defect detection. However, these methods typically depend on large-scale, high-quality labeled datasets for model training. In PC component production scenarios, due to harsh production environments, high labor costs, and complex and varied defect patterns, acquiring and labeling large amounts of defect image data is extremely difficult, resulting in small datasets with insufficient diversity. This "data scarcity" problem has become a key bottleneck restricting the performance improvement of deep learning models.

[0003] To address the challenge of data scarcity, existing technologies have primarily explored three strategies. First, transfer learning and domain adaptation: this method fine-tunes models pre-trained on large, general datasets, but struggles to allow the model to fully learn the specific, fine-grained features of PC component defects. Second, unsupervised anomaly detection: this method trains using only defect-free "normal" samples; while it doesn't require defect labels, its detection performance significantly degrades for defects with complex shapes and subtle features. Third, defect image generation: this method uses generative models, such as GANs and diffusion models, to synthesize new defect images to expand the training set, and is considered a promising direction. However, existing methods have many shortcomings when generating PC component surface defects, such as insufficient realism, poor controllability of the generation process, poor performance in generating small-sized or elongated defects, and limited diversity, making it difficult to cover the various defect variations in the real world.

[0004] Therefore, there is an urgent need to provide a method and system for generating small-sample precast concrete defect images to solve the above-mentioned technical problems. Summary of the Invention

[0005] This application provides a method and system for generating small-sample precast concrete defect images. By constructing a high-fidelity defect generation pipeline, it is possible to generate high-quality precast concrete defect images that are highly consistent with real defects in appearance, morphology and spatial distribution and have diversity using a very small number of samples, thereby effectively supporting the training and performance improvement of precast concrete surface defect detection models.

[0006] This application provides a method for generating small-sample precast concrete defect images, comprising the following steps: S1, acquiring defect images and defect-free images with different textures, and processing the defect images by creating a binary mask for each defect to obtain a small-sample defect image training set and a normal image dataset; S2, constructing a feature learning model based on the small-sample defect image training set using a stable diffusion model combined with a control network model, separating the defect appearance features from the defect spatial features of the defect images to obtain appearance feature embedding vectors representing the defect appearance features in order to focus on learning the pure appearance features of the defects; S3, based on the stable diffusion model, according to a low-rank... An adaptive parameter efficient fine-tuning algorithm smoothly transfers general knowledge to the field of precast concrete defects, constructs a precast concrete defect mask generation model, and obtains a large number of naturally distributed defect masks. The general knowledge refers to basic visual features and concepts learned from large-scale, diverse, publicly available industrial visual datasets. S4: Based on the appearance feature embedding vector, the specific content of the generated defect appearance is controlled according to the feature learning model. Combined with the control network model, the normal image dataset and the defect mask are used as input data to control the position of the generated defect appearance, constructing a high-fidelity defect generation pipeline and generating a large number of high-quality defect images.

[0007] Optionally, S1 includes: acquiring defect images of various common defect types and defect-free images with different textures to obtain an original image; preprocessing the original image to obtain a standardized input image; performing fine pixel-level annotations on the defect images based on the input image, creating a binary mask for each defect to obtain a defect mask for the defect image; and based on the input image, selecting a preset number of specific representative defect images and their defect masks according to each defect category to construct a small sample defect image training set for model training, wherein the defect-free images constitute a normal image dataset.

[0008] Optionally, S2 includes: assigning a unique text placeholder for each type of defect in the defect image and initializing a learnable text embedding vector; wherein, the exclusive text prompt for each type of defect is represented by a text placeholder, and the text embedding vector is a mathematical representation connecting the text prompt and the internal structure of the stable diffusion model, the text placeholder corresponds to a unique, learnable text embedding vector, and the text embedding vector condenses the core appearance features of that type of defect; inputting the defect image into the stable diffusion model, and inputting the corresponding defect mask into the control network model for training to construct a feature learning model, wherein the defect mask provides accurate defect spatial features for the generation process; decoupling the learning task of the defect spatial features from the text embedding vector based on the defect spatial features to obtain an appearance feature embedding vector representing the appearance features of the defect; and focusing on learning the pure appearance features of the defect based on the appearance feature embedding vector according to the feature learning model.

[0009] Optionally, S2 further includes: focusing on learning the pure appearance features of the defect based on the appearance feature embedding vector, while fine-tuning the cross attention layer in the U-Net network of the stable diffusion model, and training and optimizing it together with the appearance feature embedding vector to obtain a high-precision defect appearance feature that has stripped away spatial features.

[0010] Optionally, S3 includes: employing a two-stage progressive domain adaptation strategy based on low-rank adaptive parameter efficient fine-tuning technology to fine-tune the stable diffusion model twice according to different data; performing preliminary fine-tuning of the stable diffusion model based on a low-rank adaptive parameter efficient fine-tuning algorithm according to a general industrial vision dataset, so that the stable diffusion model learns the general morphological knowledge of defect masks to obtain a general defect mask generation model; and performing secondary fine-tuning of the general defect mask generation model based on the low-rank adaptive parameter efficient fine-tuning algorithm according to the binary mask, so that the general defect mask generation model learns to generate defect masks that conform to the real physical laws of the concrete field to obtain a precast concrete defect mask generation model.

[0011] Optionally, the low-rank adaptive parameter-based efficient fine-tuning technique employs a two-stage progressive domain adaptive strategy. Based on different data, the stable diffusion model is fine-tuned twice. Specifically, during the training and construction of the general defect mask generation model and the precast concrete defect mask generation model, the parameters of the stable diffusion model are frozen, and only the injected, lightweight low-rank adaptive module is trained. This achieves two fine-tunings of the stable diffusion model, enabling the precast concrete defect mask generation model to master the ability to generate defect masks that conform to the real physical laws of the concrete domain.

[0012] Optionally, the low-rank adaptive parameter efficient fine-tuning algorithm performs secondary fine-tuning on the general defect mask generation model based on the binary mask, so that the general defect mask generation model learns to generate defect masks that conform to the real physical laws of the concrete field, thus obtaining a precast concrete defect mask generation model. The algorithm further includes: determining whether the area of ​​the connected region in the defect mask is less than a preset threshold; if the area of ​​the connected region is less than the preset threshold, then using a morphological dilation algorithm to expand the connected region to obtain a defect enhancement mask.

[0013] Optionally, step S4 includes: incorporating the appearance feature embedding vector into the text prompt as input data, and controlling the specific content of the generated defect appearance according to the feature learning model; inputting the normal image dataset and the defect enhancement mask into the control network model to control the shape and position of the generated defect appearance; according to the background preservation strategy, after each step of image generation, only the content generated within the mask area is retained, while the area outside the mask is covered with the original normal image content to ensure that the area outside the mask remains unchanged; and generating a large number of high-quality defect images through continuous sampling.

[0014] Optionally, the background preservation strategy, which retains only the content generated within the mask area after each image generation step while covering the area outside the mask with the original normal image content to ensure that the area outside the mask remains unchanged, includes: during the iterative denoising and image generation process, gradually predicting the denoised result of the entire image based on the feature learning model; retaining the newly generated image content within the mask area based on the enhanced mask, while forcibly covering all areas outside the mask with the content corresponding to the original normal image after adding noise; and using the forcibly covered image as input to enter the next step of denoising and image generation, repeatedly performing background preservation operations in the continuous iterative process to ensure that the background of the generated image is absolutely faithful.

[0015] Secondly, this application provides a small-sample precast concrete defect image generation system. The system includes: an acquisition module for acquiring defect images and defect-free images with different textures, processing the defect images, creating a binary mask for each defect, and obtaining a small-sample defect image training set and a normal image dataset; an appearance feature learning module for constructing a feature learning model based on the small-sample defect image training set using a stable diffusion model combined with a control network model, separating the defect appearance features from the defect spatial features of the defect images, and obtaining appearance feature embedding vectors representing the defect appearance features to focus on learning the pure appearance features of the defects; and a defect mask learning module for... The model uses a low-rank adaptive parameter efficient fine-tuning algorithm to smoothly transfer general knowledge to the precast concrete defect domain, constructing a precast concrete defect mask generation model to obtain a large number of naturally distributed defect masks. The general knowledge represents basic visual features and concepts learned from large-scale, diverse, publicly available industrial visual datasets. The defect image generation module, based on the appearance feature embedding vector and the feature learning model, controls the specific content of the generated defect appearance. It also combines the normal image dataset and the defect masks as input data with the control network model to control the position of the generated defect appearance, constructing a high-fidelity defect generation pipeline to generate a large number of high-quality defect images.

[0016] This application has at least the following advantages:

[0017] The above steps mainly involve acquiring small sample defect image data. First, a feature learning model that decouples appearance and space is constructed based on a stable diffusion model combined with a control network model to accurately learn defect appearance features. Then, based on a progressively adaptive domain defect mask generation framework, a large number of diverse and realistically distributed defect masks are generated using low-rank adaptive parameter fine-tuning technology. Based on the feature learning model and the control network model, a high-fidelity generation pipeline guided by both semantic and spatial conditions is constructed according to the generated defect appearance features and defect masks. This generates a large number of realistic, diverse defect images that are precisely aligned with the masks. Thus, it is possible to generate high-quality precast concrete defect images that are highly consistent with real defects in appearance, shape, and spatial distribution, and possess diversity, using a very small number of samples. This effectively supports the training and performance improvement of the precast concrete surface defect detection model. Attached Figure Description

[0018] Figure 1 This is an illustration of the application environment of a method for generating small sample precast concrete defect images in one embodiment.

[0019] Figure 2 This is a schematic flowchart illustrating the steps of a method for generating small sample precast concrete defect images in one embodiment.

[0020] Figure 3 This is a schematic diagram illustrating the construction of a feature learning model by decoupling defect space features from text embedding vectors in one embodiment;

[0021] Figure 4 This is a schematic diagram illustrating a precast concrete defect mask generation model obtained based on a low-rank adaptive parameter efficient fine-tuning technique in one embodiment.

[0022] Figure 5 This is a schematic diagram illustrating the generation of a large number of high-quality defect images by constructing a high-fidelity defect generation pipeline in one embodiment.

[0023] Figure 6 This is a structural block diagram showing a system for generating images of defects in small samples of precast concrete in one embodiment.

[0024] Figure 7 This is a schematic structural diagram of a computer device in one embodiment. Detailed Implementation

[0025] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the scope of the present application.

[0026] For ease of understanding, the system to which this application applies will first be described. This application provides a method for generating small-sample precast concrete defect images, which can be applied to, for example... Figure 1 The system architecture shown includes a user-space file server 103 and a terminal device 101. The terminal device 101 communicates with the user-space file server 103 via a network. The user-space file server 103 can be a file server based on the NFSv3 / v4 protocol, running in a Linux environment. NFS (Network File System) is a network abstraction on top of a file system, allowing remote clients running on the terminal device 101 to access the file system over the network in a manner similar to a local file system. The terminal device 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The user-space file server 103 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0027] Figure 2 This application provides a flowchart illustrating a method for generating small-sample precast concrete defect images, which may include the following steps:

[0028] S1. Obtain defective images and defect-free images with different textures, process the defective images, create a binary mask for each defect, and obtain a small sample defective image training set and a normal image dataset.

[0029] S2. Based on a small sample defect image training set, a feature learning model is constructed by combining a stable diffusion model with a control network model. The defect appearance features and defect spatial features of the defect image are separated to obtain the appearance feature embedding vector that represents the appearance features of the defect in order to focus on learning the pure appearance features of the defect.

[0030] S3. Based on a stable diffusion model, a low-rank adaptive parameter efficient fine-tuning algorithm is used to smoothly transfer general knowledge to the precast concrete defect domain, constructing a precast concrete defect mask generation model and obtaining a large number of naturally distributed defect masks; among them,

[0031] General knowledge represents the basic visual features and concepts learned from large-scale, diverse, publicly available industrial vision datasets;

[0032] S4. Based on the appearance feature embedding vector, the specific content of the generated defect appearance is controlled by the feature learning model. Combined with the control network model, the normal image dataset and the defect mask are used as input data to control the position of the generated defect appearance, thus constructing a high-fidelity defect generation pipeline and generating a large number of high-quality defect images.

[0033] In this embodiment, it should be noted that by acquiring a small sample of defect image data, a feature learning model that decouples appearance and space is first constructed based on a Stable Diffusion model combined with a ControlNet model to accurately learn the appearance features of defects. Based on a progressively adaptive domain defect mask generation framework, a large number of diverse and realistically distributed defect masks are generated through two fine-tuning techniques using low-rank adaptive (LoRA) parameters. Based on the feature learning model and the ControlNet model, a high-fidelity generation pipeline guided by both semantic and spatial conditions is constructed according to the generated defect appearance features and defect masks. This generates a large number of realistic, diverse defect images that are precisely aligned with the masks. Thus, it is possible to generate high-quality precast concrete defect images that are highly consistent with real defects in appearance, shape, and spatial distribution, and possess diversity, using a very small number of samples. This effectively supports the training and performance improvement of the precast concrete surface defect detection model.

[0034] The following is a detailed explanation of each step:

[0035] Please refer to Figure 2As shown, step S1 involves obtaining defective images and defect-free images with different textures, processing the defective images, creating a binary mask for each defect, and obtaining a small sample defective image training set and a normal image dataset.

[0036] In this embodiment, it should be noted that defect images of various common defect types and defect-free images with different textures are obtained to obtain the original image; the original image is preprocessed to obtain a standardized input image; based on the input image, the defect image is finely annotated at the pixel level, and a binary mask is made for each defect to obtain the defect mask of the defect image; based on the input image, a preset number of specific representative defect images and their defect masks are selected according to each defect category to construct a small sample defect image training set for model training, and the defect-free images constitute the normal image dataset.

[0037] Specifically, the original images can be collected on-site at multiple precast concrete component production plants. High-resolution cameras or smartphones are used to capture images of the surfaces of typical components such as beams, columns, and wall panels, ensuring that the images cover a variety of common defect types, such as holes, exposed reinforcement, dents, grout leakage, cracks, honeycombing, pitting, and stains, as well as defect-free images with different textures. Subsequently, the collected original images undergo preprocessing, such as uniformly cropping them to a fixed size to achieve standardized input. Finally, images containing defects are finely annotated at the pixel level. The open-source annotation tool LabelMe can be used to accurately delineate the contours of defects using polygon functionality, and then export and convert them into corresponding binary mask images, creating a binary mask for each defect. The defect mask is a black-and-white image of the exact same size as the original image, which precisely identifies which areas in the image are "defects" and which are "normal" in the simplest way. From the annotated dataset, six representative defect images and their defect masks are selected for each defect category to construct a small-sample defect training set for model training. The remaining defect-free images constitute the final normal image dataset.

[0038] Please continue to refer to Figure 2As shown, step S2 involves constructing a feature learning model based on a small sample defect image training set, combining a stable diffusion model with a control network model. This separates the defect appearance features from the defect spatial features of the defect image, obtaining appearance feature embedding vectors that represent the defect appearance features, thus focusing on learning the pure appearance features of the defect. Specifically, this includes: assigning a unique text placeholder for each type of defect in the defect image and initializing a learnable text embedding vector; inputting the defect image into the stable diffusion model and the corresponding defect mask into the control network model for training, thus constructing a feature learning model. The defect mask provides accurate defect spatial features for the generation process; decoupling the defect spatial feature learning task from the text embedding vector based on the defect spatial features, thus obtaining appearance feature embedding vectors that represent the defect appearance features; and focusing on learning the pure appearance features of the defect based on the appearance feature embedding vectors using the feature learning model.

[0039] In this embodiment, it should be noted that the Stable Diffusion model is an open-source text-to-image AI model based on latent diffusion technology. The ControlNet model is a framework that adds fine-grained control conditions to the diffusion model. By accepting additional image input, it precisely controls the composition, shape, and spatial structure of the generated result, greatly improving the controllability and practicality of AI-generated images.

[0040] like Figure 3 As shown, specifically in one example, a unique text placeholder is first specified for each type of defect. A learnable embedding vector is initialized. The specific text cue for each type of defect is provided using text placeholders. This indicates that the embedding vector is a mathematical representation connecting the text prompt and the stable diffusion model. Each text placeholder corresponds to a unique, learnable text embedding vector, which encapsulates the core appearance features of that type of defect. Here, a unique text placeholder is introduced for each defect category, such as exposed rebar. It possesses a corresponding, learnable text embedding vector. Then, based on a small sample defect image training set, a feature learning model is constructed using a stable diffusion model combined with a control network model. The stable diffusion model includes a variational autoencoder (VAE) encoder and a CLIP tokenizer. The VAE encoder is part of a variational autoencoder (VAE) and is responsible for transforming between the pixel space and the latent space, encoding the input small sample defect image into a latent variable. This latent variable is then progressively noise-added through a forward diffusion process, becoming the object that the model needs to denoise. This allows the entire generation process to be performed in the latent space, reducing computational cost. The CLIP word segmenter converts natural language (text prompts) into vectors that the model can understand—text embedding vectors. This connects the text prompts with the model's internal mathematical representation. Based on the variational autoencoder and the CLIP word segmenter, it performs encoding analysis and learning on the defect image. Combined with strong spatial priors (shape and position information) provided by the control network model, and utilizing defect masks, it provides accurate spatial features for the generation process, thus decoupling the spatial feature learning task from the text embedding vectors. This allows the text embedding vectors to focus on learning pure defect appearance features during loss function optimization, achieving high-fidelity modeling of defect appearance, and ultimately obtaining an appearance feature embedding vector representing the defect's appearance characteristics.

[0041] Additionally, please continue to refer to Figure 3 As shown, a feature learning model is constructed based on a small sample defect image training set, combining a stable diffusion model with a control network model. This model separates the defect appearance features from the defect spatial features, obtaining appearance feature embedding vectors that represent the defect appearance features. This allows for focused learning of the pure appearance features of the defect. Furthermore, while focusing on learning the pure appearance features of the defect based on the appearance feature embedding vectors, the model also fine-tunes the cross-attention layer in the U-Net network of the stable diffusion model, training and optimizing it together with the appearance feature embedding vectors to obtain high-precision defect appearance features that have separated the spatial features.

[0042] In this embodiment, it should be noted that while focusing on learning the pure appearance features of defects based on the appearance feature embedding vectors, the cross-attention layer in the U-Net network of the stable diffusion model is fine-tuned. U-Net is the core network for denoising in the diffusion model. Its cross-attention layer is a key mechanism for text conditional injection. It allows the U-Net network to focus on the most important parts of the cue word embeddings output by the text encoder at each step of the denoising process. Fine-tuning these layers can significantly improve the alignment accuracy between text semantics and generated pixels, resulting in high-precision defect appearance features that have stripped away spatial features.

[0043] Reference Figure 2 , Figure 4 As shown, in step S3, based on the stable diffusion model, general knowledge is smoothly transferred to the precast concrete defect domain using a low-rank adaptive parameter efficient fine-tuning algorithm to construct a precast concrete defect mask generation model, resulting in a large number of naturally distributed defect masks. This includes: using a two-stage progressive domain adaptation strategy based on low-rank adaptive parameter efficient fine-tuning technology, the stable diffusion model is fine-tuned twice according to different data; the stable diffusion model is initially fine-tuned based on a general industrial vision dataset using a low-rank adaptive parameter efficient fine-tuning algorithm, so that the stable diffusion model learns the general morphological knowledge of defect masks, resulting in a general defect mask generation model; and the general defect mask generation model is fine-tuned a second time based on a binary mask using a low-rank adaptive parameter efficient fine-tuning algorithm, so that the general defect mask generation model learns to generate defect masks that conform to the real physical laws of the concrete domain, resulting in a precast concrete defect mask generation model.

[0044] In this embodiment, it should be noted that the general knowledge representation is based on fundamental visual features and concepts learned from large-scale, diverse, publicly available industrial vision datasets. Low-rank adaptation (LoRA) is mainly used for efficient fine-tuning of large pre-trained models. It does not directly modify any weights of the original model, but learns the changes needed for fine-tuning by injecting a very lightweight "patch," i.e., a set of small matrices. It is used for training stylized models, object-specific generation, etc., and solves the computational and storage bottlenecks in fine-tuning large models through an extremely simple low-rank decomposition idea. Specifically, this application uses a low-rank adaptive parameter efficient fine-tuning technique to fine-tune a stable diffusion model according to a two-stage progressive domain adaptation strategy. In the first stage, a pre-trained stable diffusion model is selected as the basis, and training is performed on one or more general, publicly available industrial vision datasets, such as MVTec AD, which contain various defect images and their corresponding defect masks. To improve efficiency, this process employs a low-rank adaptive technique, freezing the parameters of the stable diffusion model and training only the injected, lightweight low-rank adaptive module. This allows the stable diffusion model to learn general morphological knowledge of defect masks, such as the linear features of cracks and the stripe features of scratches, making it a "general defect mask generator." In the second stage, based on the general defect mask generation model with its general knowledge, a small number of highly domain-specific precast concrete defect masks are used as new data for further training. This process again only updates the low-rank adaptive parameters, smoothly transferring and adapting the general knowledge already acquired by the general defect mask generation model to the specific domain of precast concrete. This allows it to learn to generate defect morphologies that conform to the real physical laws of this domain, generating defect masks that conform to the real physical laws of the concrete domain, thus obtaining the precast concrete defect mask generation model.

[0045] In addition, the low-rank adaptive parameter efficient fine-tuning algorithm performs secondary fine-tuning on the general defect mask generation model based on the binary mask, so that the general defect mask generation model learns to generate defect masks that conform to the real physical laws of the concrete field, thus obtaining the precast concrete defect mask generation model. It also includes: judging whether the area of ​​the connected region in the defect mask is less than a preset threshold; if the area of ​​the connected region is less than the preset threshold, the morphological dilation algorithm is used to expand the connected region to obtain the defect enhancement mask.

[0046] In this embodiment, it should be noted that the preset threshold is determined empirically based on the final generated effect. The morphological dilatation algorithm "enlarges" or "thickens" the white foreground region in the image, filling small holes and gaps in foreground objects, connecting adjacent or broken objects, and increasing the boundary size of objects. Specifically, the defect mask area on the defect mask generated by the precast concrete defect mask generation model is compared with the preset threshold. If the area of ​​the generated defect mask area is smaller than the preset threshold, the model may fail to generate a clear and effective defect appearance in subsequent image generation processes due to its weak features, leading to information loss or generation failure. Therefore, the morphological dilatation algorithm expands these small regions, thereby strengthening the spatial representation of small-sized defects and ensuring they are sufficiently "visible" in subsequent generation and can be successfully drawn.

[0047] Reference Figure 2 , Figure 5 As shown, in step S4, based on the appearance feature embedding vector, the specific content of the generated defect appearance is controlled by the feature learning model. Combined with the control network model, the normal image dataset and the defect mask are used as input data to control the position of the generated defect appearance, constructing a high-fidelity defect generation pipeline to generate a large number of high-quality defect images. This includes: incorporating the appearance feature embedding vector into text prompts as input data, and controlling the specific content of the generated defect appearance according to the feature learning model; inputting the normal image dataset and the defect enhancement mask into the control network model to control the shape and position of the generated defect appearance; according to the background preservation strategy, after each image generation step, only the content generated within the mask area is retained, while the area outside the mask is covered with the original normal image content to ensure that the area outside the mask remains unchanged; and generating a large number of high-quality defect images through continuous sampling.

[0048] In this embodiment, it should be noted that the high-fidelity defect generation pipeline refers to an automated and scalable process system. Utilizing a stable diffusion model, it takes defect-free product images and specific defect descriptions as input. Through fine-grained conditional control, it outputs a large amount of visually highly realistic, physically logically sound defect image data that can be effectively used to train and test industrial quality inspection AI models. Specifically, a dual-conditional guidance is applied to the iterative denoising and defect image generation process: first, semantic guidance is achieved by controlling the specific content of the generated defect appearance based on the appearance feature embedding vector according to the feature learning model; second, spatial guidance is achieved by combining a control network model with normal image datasets and defect enhancement masks as input data to control the position of the generated defect appearance. This constructs a high-fidelity defect generation pipeline. Finally, by using different random noise seeds to run the entire defect generation pipeline multiple times, a large number of high-quality, highly realistic defect images with varying appearances and shapes, all conforming to the control conditions, are generated in batches. This achieves the generation of a large number of high-quality precast concrete defect images that are highly consistent with real defects in appearance, shape, and spatial distribution, and possess diversity, using a small sample of defect images.

[0049] Furthermore, according to the background preservation strategy, only the content generated within the mask area is retained after each image generation step, while the area outside the mask is covered with the original normal image content to ensure that the area outside the mask remains unchanged. This includes: during the iterative denoising and image generation process, the denoised result of the entire image is gradually predicted based on the feature learning model; based on the enhancement mask, the newly generated image content within the mask area is retained, while all areas outside the mask are directly forcibly covered with the content corresponding to the original normal image after adding noise; the forcibly covered image is then used as input to enter the next step of denoising and image generation. In the continuous iterative process, the background preservation operation is repeatedly performed to ensure that the background of the generated image is absolutely faithful.

[0050] In this embodiment, it should be noted that by adopting a background preservation strategy, during the generation of defect images, only the corresponding precast concrete defects are generated based on the shape of the defect enhancement mask position, while other areas of the image remain unchanged. This maximizes the preservation of the integrity, consistency, and authenticity of the original background, reduces repetitive work, improves image generation efficiency, ensures the quality of generated images, provides space for experiments and iterations, and enhances flexibility.

[0051] The implementation principle of this embodiment is as follows: The above steps mainly involve acquiring small sample defect image data. First, based on the stable diffusion model combined with the control network model, a feature learning model that decouples appearance and space is constructed. The spatial encoding task is separated from the text embedding vector, so that the appearance feature embedding vector can focus on learning the pure appearance features of defects such as texture and color. While optimizing the embedding vector, the cross attention layer in the stable diffusion model U-Net network is fine-tuned to enhance the alignment accuracy between text semantics and pixel generation, thereby accurately learning the appearance features of defects. A defect mask generation framework based on a stable diffusion model and employing progressive domain adaptation utilizes a low-rank adaptive parameter fine-tuning technique to perform two fine-tuning operations on the generated defect mask. The first fine-tuning aims to learn the general morphological knowledge of the defect mask using a large-scale, general industrial vision dataset based on the original stable diffusion model. The second fine-tuning aims to achieve domain adaptation by using binary masks from a small sample defect image training set to train a precast concrete defect mask generation model based on the general defect mask generation model obtained after the first fine-tuning. After training, a large number of defect masks with diverse shapes and natural and realistic distributions are generated through repeated sampling. A morphological dilation algorithm is then used to expand the small regions of the defect masks to obtain a defect enhancement mask, ensuring that subsequent defect images can be successfully rendered. Then, based on the feature learning model and the control network model, a high-fidelity generation pipeline guided by both semantic and spatial conditions is constructed according to the generated defect appearance features and defect enhancement mask training. This generates a large number of realistic, diverse defect images that are precisely aligned with the mask, thereby enabling the generation of high-quality precast concrete defect images that are highly consistent with real defects in appearance, shape and spatial distribution, and have diversity, using a very small number of samples. This effectively supports the training and performance improvement of the precast concrete surface defect detection model.

[0052] Reference Figure 6 As shown in the figure, this application embodiment also provides a small sample precast concrete defect image generation system, which may include: an acquisition module 301, an appearance feature learning module 302, a defect mask learning module 303, and a defect image generation module 304; wherein the main functions of each component module are as follows:

[0053] The acquisition module 301 is used to acquire defective images and defect-free images with different textures, process the defective images, create a binary mask for each defect, and obtain a small sample defective image training set and a normal image dataset.

[0054] The appearance feature learning module 302 is used to construct a feature learning model based on a small sample defect image training set, according to a stable diffusion model combined with a control network model. It separates the defect appearance features and defect spatial features of the defect image to obtain the appearance feature embedding vector representing the defect appearance features in order to focus on learning the pure appearance features of the defect.

[0055] The defect mask learning module 303 is used to smoothly transfer general knowledge to the precast concrete defect domain based on a stable diffusion model and an efficient fine-tuning algorithm with low-rank adaptive parameters, thereby constructing a precast concrete defect mask generation model and obtaining a large number of naturally distributed defect masks. The general knowledge represents the basic visual features and concepts learned from large-scale, diverse, and publicly available industrial vision datasets.

[0056] The defect image generation module 304 is used to control the specific content of the generated defect appearance based on the appearance feature embedding vector according to the feature learning model, and combined with the control network model to take the normal image dataset and the defect mask as input data to control the position of the generated defect appearance, build a high-fidelity defect generation pipeline, and generate a large number of high-quality defect images.

[0057] like Figure 7 The diagram shown is a block diagram of a computer device according to an embodiment of this application. The term "computer device" is intended to represent various forms of digital computers or mobile devices. The digital computer may include a desktop computer, a portable computer, a workbench, a personal digital assistant, a server, a mainframe computer, and other suitable computers. The mobile device may include a tablet computer, a smartphone, a wearable device, etc.

[0058] like Figure 7 As shown, device 600 includes a computing unit 601, a ROM 602, a RAM 603, a bus 604, and an I / O interface 605. The computing unit 601, ROM 602, and RAM 603 are interconnected via the bus 604. The I / O interface 605 is also connected to the bus 604.

[0059] The computing unit 601 can execute various processes in the method embodiments of this application according to computer instructions stored in the read-only memory (ROM) 602 or computer instructions loaded from the storage unit 608 into the random access memory (RAM) 603. The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. The computing unit 601 can include, but is not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. In some embodiments, the methods provided in the embodiments of this application can be implemented as computer software programs, which are tangibly contained in a computer-readable storage medium, such as the storage unit 608.

[0060] RAM 603 can also store various programs and data required for the operation of device 600. Part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609.

[0061] The input unit 606, output unit 607, storage unit 608, and communication unit 609 in device 600 can be connected to I / O interface 605. The input unit 606 can be, for example, a keyboard, mouse, touchscreen, or microphone; the output unit 607 can be, for example, a display, speaker, or indicator light. Device 600 can exchange information and data with other devices through the communication unit 609.

[0062] It should be noted that the device may also include other components necessary for normal operation. It may also include only the components necessary for implementing the solution of this application, without necessarily including all the components shown in the figures.

[0063] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), payload programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof.

[0064] The computer instructions used to implement the methods of this application may be written in any combination of one or more programming languages. These computer instructions may be provided to the computing unit 601 such that when executed by the computing unit 601, such as a processor, the computer instructions cause the execution of the steps involved in the embodiments of the methods of this application.

[0065] The computer-readable storage medium provided in this application can be a tangible medium that can contain or store computer instructions for performing the steps involved in the method embodiments of this application. The computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, and other forms of storage media.

[0066] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for generating small sample precast concrete defect images, characterized in that, Includes the following steps: S1. Obtain defective images and defect-free images with different textures, process the defective images, create a binary mask for each defect, and obtain a small sample defective image training set and a normal image dataset. S2. Based on the small sample defect image training set, a feature learning model is constructed according to the stable diffusion model and the control network model. The defect appearance features and defect spatial features of the defect image are separated to obtain the appearance feature embedding vector that represents the defect appearance features in order to focus on learning the pure appearance features of the defect. S3. Based on the stable diffusion model, general knowledge is smoothly transferred to the precast concrete defect domain using a low-rank adaptive parameter efficient fine-tuning algorithm, constructing a precast concrete defect mask generation model to obtain a large number of naturally distributed defect masks; among which, The general knowledge represents the basic visual features and concepts learned from large-scale, diverse, publicly available industrial vision datasets; S4. Based on the appearance feature embedding vector, the specific content of the generated defect appearance is controlled by the feature learning model. Combined with the control network model, the normal image dataset and the defect mask are used as input data to control the position of the generated defect appearance, construct a high-fidelity defect generation pipeline, and generate a large number of high-quality defect images. Wherein, S2 includes: For each type of defect in the defect image, a unique text placeholder is assigned and a learnable text embedding vector is initialized; wherein, the exclusive text prompt for each type of defect is represented by the text placeholder, and the text embedding vector is a mathematical representation connecting the text prompt and the internal structure of the stable diffusion model. The text placeholder corresponds to a unique, learnable text embedding vector, and the text embedding vector condenses the core appearance features of that type of defect. The defect image is input into the stable diffusion model, and the corresponding defect mask is input into the control network model for training to construct a feature learning model. The defect mask provides accurate defect spatial features for the generation process. Based on the defect space features, the learning task of the defect space features is decoupled from the text embedding vector to obtain the appearance feature embedding vector that represents the appearance features of the defect. Based on the feature learning model, the pure appearance features of defects are learned by focusing on learning the appearance feature embedding vector.

2. The method according to claim 1, characterized in that, S1 includes: Obtain defect images of various common defect types and defect-free images of different textures to obtain the original image; The original image is preprocessed to obtain a standardized input image; Based on the input image, the defect image is annotated with fine pixel-level details, and a binary mask is created for each defect to obtain the defect mask of the defect image. Based on the input image, a preset number of specific representative defect images and their defect masks are selected according to each defect category to construct a small sample defect image training set for model training, and the defect-free images constitute a normal image dataset.

3. The method according to claim 1, characterized in that, The S2 further includes: While focusing on learning the pure appearance features of defects based on the appearance feature embedding vector, the cross attention layer in the U-Net network of the stable diffusion model is fine-tuned and trained and optimized together with the appearance feature embedding vector to obtain high-precision defect appearance features that have stripped away spatial features.

4. The method according to claim 3, characterized in that, The S3 includes: The low-rank adaptive parameter high-efficiency fine-tuning technique adopts a two-stage progressive domain adaptive strategy, and performs two fine-tunings on the stable diffusion model according to different data. The stable diffusion model is initially fine-tuned based on a low-rank adaptive parameter high-efficiency fine-tuning algorithm according to a general industrial vision dataset, so that the stable diffusion model learns the general morphological knowledge of defect masks and obtains a general defect mask generation model. Based on the efficient fine-tuning algorithm of low-rank adaptive parameters, the general defect mask generation model is fine-tuned twice according to the binary mask, so that the general defect mask generation model learns to generate defect masks that conform to the real physical laws of concrete, thus obtaining the precast concrete defect mask generation model.

5. The method according to claim 4, characterized in that, The low-rank adaptive parameter-based efficient fine-tuning technique employs a two-stage progressive domain adaptive strategy, performing two fine-tuning operations on the stable diffusion model based on different data, including: During the training and construction of the general defect mask generation model and the precast concrete defect mask generation model, the parameters of the stable diffusion model are frozen, and only the injected, lightweight low-rank adaptive module is trained to achieve two fine-tunings of the stable diffusion model, so that the precast concrete defect mask generation model can master the ability to generate defect masks that conform to the real physical laws of concrete.

6. The method according to claim 4, characterized in that, The low-rank adaptive parameter-based efficient fine-tuning algorithm performs secondary fine-tuning on the general defect mask generation model based on the binary mask, so that the general defect mask generation model learns to generate defect masks that conform to the real physical laws of concrete, thus obtaining a precast concrete defect mask generation model, and further includes: Determine whether the area of ​​the connected components in the defect mask is less than a preset threshold; If the area of ​​the connected region is less than the preset threshold, the connected region is expanded using a morphological dilation algorithm to obtain a defect enhancement mask.

7. The method according to claim 6, characterized in that, The S4 includes: The appearance feature embedding vector is incorporated into the text prompt as input data, and the specific content of the generated defect appearance is controlled according to the feature learning model. The normal image dataset and the defect enhancement mask are input together into the control network model to control the shape and position of the generated defect appearance; According to the background preservation strategy, only the content generated within the mask area is retained after each step of image generation, while the area outside the mask is covered with the original normal image content to ensure that the area outside the mask remains unchanged. A large number of high-quality defect images are generated through continuous sampling.

8. The method according to claim 7, characterized in that, According to the background preservation strategy, after each step of image generation, only the content generated within the mask area is retained, while the area outside the mask is covered with the original normal image content, ensuring that the area outside the mask remains unchanged. This includes: During the iterative denoising process to generate an image, the feature learning model gradually predicts the denoised result of the entire image. According to the enhancement mask, the newly generated image content within the mask area is retained, while all areas outside the mask are directly covered by the content corresponding to the original normal image after adding noise. The image after forced coverage is then used as input to enter the next step of the denoising and image generation process. In the continuous iterative process, the background preservation operation is repeatedly performed to ensure that the background of the generated image is absolutely faithful.

9. A system for generating images of defects in precast concrete with small sample sizes, characterized in that, include: The acquisition module is used to acquire defective images and defect-free images with different textures, and to process the defective images, create a binary mask for each defect, and obtain a small sample defective image training set and a normal image dataset. The appearance feature learning module is used to construct a feature learning model based on the small sample defect image training set according to the stable diffusion model and the control network model, to separate the defect appearance features and defect spatial features of the defect image, and obtain the appearance feature embedding vector representing the defect appearance features in order to focus on learning the pure appearance features of the defect. For each type of defect in the defect image, a unique text placeholder is assigned, and a learnable text embedding vector is initialized. Each type of defect has a unique text cue represented by a text placeholder. The text embedding vector is a mathematical representation connecting the text cue and the internal structure of the stable diffusion model. Each text placeholder corresponds to a unique, learnable text embedding vector, which encapsulates the core appearance features of that type of defect. The defect image is input into the stable diffusion model, and the corresponding defect mask is input into the control network model for training, constructing a feature learning model. The defect mask provides accurate defect spatial features for the generation process. Based on the defect spatial features, the learning task of the defect spatial features is decoupled from the text embedding vector to obtain an appearance feature embedding vector representing the appearance features of the defect. Based on the feature learning model, the appearance feature embedding vector is used to focus on learning the pure appearance features of the defect. The defect mask learning module is used to smoothly transfer general knowledge to the precast concrete defect domain based on the stable diffusion model and according to the low-rank adaptive parameter efficient fine-tuning algorithm, to construct a precast concrete defect mask generation model and obtain a large number of naturally distributed defect masks; wherein, the general knowledge represents basic visual features and concepts learned from large-scale, diverse, publicly available industrial visual datasets. The defect image generation module is used to control the specific content of the generated defect appearance based on the appearance feature embedding vector according to the feature learning model, and to control the position of the generated defect appearance by combining the normal image dataset and the defect mask as input data with the control network model, thereby constructing a high-fidelity defect generation pipeline and generating a large number of high-quality defect images.

Images