A stable diffusion method for few-shot meter defect image generation

Abstract

The application discloses a stable diffusion method for few-shot meter defect image generation, which embeds the structural features and defect knowledge of the transformer substation meter by fine-tuning the pre-trained model, and improves the similarity between the generated image and the actual meter. A crack feature modeling module is innovatively designed, which generates control images with geometric constraints by combining line drawings, crack masks and constraint graphs, and accurately expresses the shape and position of cracks. At the same time, a super network mechanism is introduced to dynamically adjust the weight distribution in the generation process, ensuring the consistency and diversity of the generated images in shape and position. This method can generate transformer substation meter defect images with specific crack features under the condition of a small number of samples, significantly improving the diversity and quality of the generated images. Through application in the downstream detection task, the generated data effectively improves the accuracy and robustness of the defect detection model, providing strong support for the safe and stable operation of the power system.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, and in particular to a stable diffusion method for generating defective images of meters with few samples. Background Technology

[0002] Substations are a crucial component of the power system, and the safe and stable operation of their equipment directly impacts the overall reliability of the power grid. Meters, as key monitoring devices in substations, are responsible for power dispatch and operation monitoring, and their normal operation is vital to ensuring grid security. However, due to long-term exposure to complex external environments, meters are susceptible to the effects of extreme weather conditions such as direct sunlight, strong winds, heavy rain, and blizzards, resulting in physical defects such as cracks and breakage. These defects can not only cause meter malfunctions, preventing normal monitoring or data recording, but may also lead to equipment damage or even systemic risks, seriously threatening the safety and stability of the power grid.

[0003] In recent years, with the development of computer vision technology, significant progress has been made in deep learning-based methods for detecting defects in power equipment. However, efficient training of deep learning models typically relies on large-scale and precisely labeled datasets. Substation meter defect data, due to its difficulty in collection and scarcity of samples, results in high costs and lengthy processing times for dataset construction. This data scarcity problem has become a bottleneck for further application, especially under conditions of few samples, where the model's generalization ability and detection accuracy are severely limited.

[0004] To alleviate the problem of data scarcity, data augmentation techniques have become an effective means. Traditional data augmentation methods expand the dataset through geometric transformations (such as flipping, rotating, cropping, and scaling) or adding noise, thereby improving the robustness of the model. However, these methods often ignore the specific semantic content of the images, have limited performance in handling complex scenes or fine-grained tasks, and are difficult to effectively improve the diversity and complexity of data distribution.

[0005] In recent years, Generative Adversarial Networks (GANs) and Diffusion Models have gradually become core technologies in the field of data augmentation. GANs generate high-quality images through adversarial training, significantly improving sample diversity when the sample size is limited. However, GANs suffer from problems such as mode collapse and training instability during the generation process. Especially under conditions of few samples, the generated images are prone to distortion and blurring, making it difficult to meet the needs of practical applications. Diffusion Models, with their significant advantages in detail control and generation quality, have gradually surpassed GANs and become the mainstream method for generating high-quality images. However, existing generation methods still face the following challenges in the task of generating defects in substation meters:

[0006] 1) There are significant differences in the scale of the dataset: Most existing generation methods are based on large-scale datasets (such as MS-COCO, CC12M). However, due to the difficulty of collecting substation meter defect data and the scarcity of samples, the performance of existing generation methods on small-sample datasets often fails to meet expectations, and the generated results are difficult to meet the actual application requirements.

[0007] 2) The generation process lacks effective control: Most existing generation methods rely on the probability mechanism in the denoising process, and the generated results are random, making it difficult to achieve quantitative control and meet specific requirements. While the editing methods with strong control capabilities rely too much on the original images and can only make mandatory modifications to the images, which limits the diversity of the generated images. Summary of the Invention

[0008] Therefore, in the above context, the present invention proposes a stable diffusion method for generating few-shot meter defect images, which solves the problem of insufficient substation meter defect data, expands the defect dataset, and at the same time helps to improve the accuracy of defect detection. That is, it solves the problems of insufficient substation meter defect data and the poor effect of downstream detection tasks caused by it. By introducing meter knowledge embedding, crack feature modeling, and a hypernetwork dynamic control mechanism, a novel method for generating meter defect images is designed, which can controllably generate high-quality and diverse meter defect images under few-shot conditions, solve the problem of insufficient substation meter defect data, and further improve the accuracy of meter defect detection.

[0009] To achieve the above object, the present invention provides the following solutions:

[0010] A stable diffusion method for generating few-shot meter defect images, comprising the following steps:

[0011] S1, construct a dataset containing normal substation meter images and meter images with crack defects, covering various meter types, including: ammeters, voltmeters, pressure gauges, wattmeters, etc. All images are preprocessed, uniformly cropped to a resolution of 512×512, and the defect categories and location information are annotated. The dataset is divided into a training set and a validation set according to the ratio of 8:2 to ensure the rationality of the data distribution;

[0012] S2, fine-tune the pre-trained Stable Diffusion model through the DreamBooth method, bind the unique Chinese identifier [meter] to the meter image, different from using English identifiers in traditional methods, to achieve the embedding of meter knowledge, and establish a strong association between the Chinese identifier and the meter image features. Design a dual-objective loss function to balance new feature learning and prior knowledge retention, and ensure that the model retains the general generation ability when generating meter defect images.

[0013] S3. Propose a crack feature modeling module. Extract the meter line drawing through an adaptive edge detection algorithm, retain the structural features of the dial contour and key components, and ensure the integrity and rationality of the generated image in terms of geometric structure. Based on SAM, perform high-precision dial area segmentation on the input image to generate a dynamic constraint map. Further fuse the crack mask images provided by expert hand-drawn and industrial datasets, and precisely limit the cracks to only distribute in the effective area of the dial through the constraint image, avoiding the abnormal phenomenon of cracks appearing in non-functional areas.

[0014] S4. Adopt a hypernetwork mechanism to dynamically adjust the weight distribution during the generation process, ensuring the consistency and diversity of the generated images in terms of morphology and position. Freeze the weights of the Stable Diffusion main model, construct a trainable hypernetwork branch, encode the control image into a latent representation, and inject it layer by layer into the diffusion process to constrain the crack morphology of the generated image.

[0015] Among them, S2 specifically includes:

[0016] First, by fine-tuning Stable Diffusion, endow the unique Chinese identifier [meter] with a new meaning and inject it as an input prompt into the training process, enabling the existing model to adapt to the characteristic knowledge of substation meters, thereby enhancing the model's ability to generate meter defect images.

[0017] Next, in order to prevent the model from forgetting the prior knowledge learned from large-scale data, an optimization objective containing the main body loss and prior preservation loss is used to achieve a balance between the learning of new concepts and the preservation of prior knowledge. The main body loss term L , 2 , pr , t , , t , t is used to optimize the model to generate images containing new features under specific conditions. By minimizing the gap between the model prediction and the target distribution, guide the model to learn specific meter defect features. The mathematical expression is as follows:

[0018]

[0019] Among them, x represents the real image, c is the text condition, ∈ is the random noise, t is the time step in the diffusion process, ω t is a weight term for a time step used to balance the loss contributions of different time steps. represents the predicted image of the model, α t and σ t are diffusion noise scheduling parameters, Perform the expectation calculation on x, c, ∈, t, that is, calculate the average over the data distribution. ‖·‖ 2 is the L2 norm, which measures the gap between the predicted image and the real image. The prior preservation loss term L prThe aim is to constrain the model's generated results to prevent them from deviating from the original data distribution, thereby avoiding a decline in the model's generalization ability due to overfitting with few samples. The loss function is expressed as follows:

[0020]

[0021] Where, x pr c represents the model in the previously generated image. pr Here, ∈′ represents the corresponding textual condition, ∈′ represents another random noise term, and t′ represents the time step in the diffusion process. By introducing generated samples from large-scale unlabeled data, the model effectively avoids forgetting its general generation capabilities.

[0022] Finally, the two loss functions are dynamically balanced by weighting, and the combined form is expressed as follows:

[0023] L = L sbj +λL pr (3)

[0024] Here, L is the overall loss function, and λ∈[0,1] are weight parameters used to adjust the tradeoff between the main loss and the prior preservation loss, flexibly optimizing the generation capability according to the task. Fine-tuning the base model improves the ability to generate specific images while retaining the generalization ability of the base model.

[0025] Specifically, S3 includes:

[0026] First, the defect-free original image I o ∈R H×W×3 (Where R indicates that the image data belongs to the real number space, meaning that the value of each pixel is a real number; H and W represent the height and width of the image, respectively; and 3 represents the RGB three channels.) After edge detection, image smoothing, and style enhancement preprocessing, the line art image I is extracted. l ∈R H×W×1 Subsequently, the line drawing image and crack mask I d ∈R H×W×1 Pixel-by-pixel fusion to generate control image I c ′∈R H×W×1 The fusion process is represented as:

[0027] I c ′(x,y)=max(I d (x,y),I l (x,y)) (4)

[0028] Among them, I c ′(x,y),I d (x,y) and I l(x,y) represent the gray values ​​at pixel (x,y) of the control image, crack image, and line drawing image, respectively, with the range limited to [0,255]. The max(·) operation represents taking the maximum gray value at each pixel.

[0029] However, due to the unique distribution of cracks, directly combining the line drawing image with the crack image may result in anomalies where crack features appear outside the dial area, making accurate modeling of the dial region impossible. Therefore, SAM is used to limit the modeling range of cracks and improve its realism. For the input original image I... o Based on the user-provided prompt condition c, SAM first uses the image encoder to process I... o The feature representation f is generated by mapping to the embedding space; simultaneously, the cue condition c is encoded by the cue encoder to generate cue features g; subsequently, features f and g are fused in the mask decoder to generate the dial constraint map I. r The formula is as follows:

[0030] I r =D(f,g),I r ∈R H×W×1 (5)

[0031] Where D is the mask decoder, which, after obtaining the dial constraints, constrains them to the pixel-by-pixel fusion process of the line drawing image and the mask image to generate the final control image. The fusion process can be updated as follows:

[0032]

[0033] Among them, I c (x,y) and I r (x, y) represent the gray values ​​of the final control image and constraint image at pixel (x, y), respectively. r The grayscale value of (x,y) takes only two values: 0 and 255. The former indicates a region where combination is allowed, while the latter indicates that the original grayscale value of the line drawing image is preserved. By introducing a constraint graph, more precise region control can be applied during crack feature modeling, which helps to control the image with higher modeling rationality.

[0034] Specifically, S4 includes:

[0035] First, the weights of the Stable Diffusion master model are frozen to preserve its original prior image knowledge. Based on this, a trainable copy of the model is constructed as a supernetwork, dynamically adjusting the generation process of the master model by sharing some feature representations.

[0036] Secondly, the input conditional control image is processed by the encoder to generate a corresponding latent representation. This latent representation, combined with the dynamic weights generated by the hypernetwork, effectively adjusts the overall contour and crack details of the generated image. Simultaneously, the basic generative model generates a preliminary image; the two work together to complete the generation of the final image, ensuring that the generated result strictly conforms to the set conditions and achieves precise control.

[0037] Finally, the constraint image is used as input to roughly outline the overall image and reveal the location and shape of the cracks. At this stage, a specific loss function is designed to optimize the generation process, defined as follows:

[0038]

[0039] Where x represents the original image, c, c', c+ are conditional information, ∈ represents standard normally distributed noise, and t is the time step in the diffusion process. This represents a neural network-parameterized denoising model used to recover the original image from noisy data, α t and σ t It is a time-dependent scaling factor that controls the degree of noise addition. Calculating the expectation of x,c,c',∈,t is equivalent to finding the average over the data distribution, ‖·‖ 2 It is the L2 norm, which measures the difference between the predicted image and the real image. During the diffusion generation process, control signals are injected into each feature layer of the model multiple times, affecting both the initial generation stage and providing structural constraints in the middle and later stages, thereby ensuring that the generated result meets the set conditions.

[0040] This includes: a crack mask generation method based on a combination of expert hand-drawing and existing industrial datasets, ensuring the integrity and accuracy of crack features. Specifically, this includes:

[0041] The crack masks are sourced from crack defect images in existing industrial datasets and crack images hand-drawn by industry experts. The expert-drawn crack masks provide a reliable supplement to crack modeling, especially in cases where SAM segmentation may fail. They effectively compensate for deficiencies in dial region segmentation, ensuring the integrity and accuracy of crack features, thereby reducing the risk of dependence on segmentation and further improving the robustness and generalization performance of the generative model. By combining expert hand-drawn images with existing industrial datasets, the generated crack masks can more accurately reflect the morphology and distribution characteristics of actual meter cracks, providing high-quality input data for subsequent crack feature modeling.

[0042] This application also provides a stable diffusion device for generating defective images of meters with few samples, the device comprising:

[0043] The S1 module is used to construct a dataset containing normal substation meter images and meter images with crack defects, covering various meter types, including: ammeters, voltmeters, pressure gauges, wattmeters, etc. All images are preprocessed, uniformly cropped to a resolution of 512×512, and the defect categories and location information are labeled. The dataset is divided into a training set and a validation set according to an 8:2 ratio to ensure the rationality of the data distribution.

[0044] The S2 module is used to fine-tune the pre-trained Stable Diffusion model through the DreamBooth method, bind the unique Chinese identifier [meter] to the meter image, different from using English identifiers in traditional methods, realize the embedding of meter knowledge, and establish a strong association between the Chinese identifier and the meter image features. Design a dual-objective loss function to balance new feature learning and prior knowledge retention, and ensure that the model retains general generation ability when generating meter defect images.

[0045] The S3 module is used to propose a crack feature modeling module. Extract the meter line drawing through an adaptive edge detection algorithm, retain the structural features of the dial contour and key components, and ensure the integrity and rationality of the generated image in terms of geometric structure. Based on SAM, perform high-precision dial area segmentation on the input image to generate a dynamic constraint map, and further fuse the crack mask images provided by expert hand-drawn and industrial datasets. Precisely limit the cracks to only distribute in the effective area of the dial through the constraint image, and avoid the abnormal phenomenon of cracks appearing in non-functional areas.

[0046] The S4 module adopts a hypernetwork mechanism to dynamically adjust the weight distribution during the generation process, ensuring the consistency and diversity of the generated images in terms of shape and position. Freeze the weights of the Stable Diffusion main model, construct a trainable hypernetwork branch, encode the control image as a latent representation, and inject it into the diffusion process layer by layer to constrain the crack morphology of the generated image.

[0047] This application also provides an electronic device, which includes: at least one memory and at least one processor; the memory stores a program, and the processor calls the program stored in the memory, and the program is used to implement the stable diffusion method for generating few-shot meter defect images.

[0048] This application also provides a storage medium, in which computer-executable instructions are stored, and the computer-executable instructions are used to execute the stable diffusion method for generating few-shot meter defect images.

[0049] The above technical solutions of the present invention can achieve the following technical effects:

[0050] A stable diffusion method, apparatus, electronic device, and storage medium for generating meter defect images with few samples are disclosed. Specifically, a meter knowledge embedding method based on DreamBooth is designed. By fine-tuning the pre-trained model, the structural features and defect knowledge of substation meters are injected into the generation model, significantly improving the similarity between the generated images and actual meters. A crack feature modeling module is proposed, combining line drawings, crack masks, and constraint maps to generate control images with geometric constraints, accurately representing the shape and location of cracks. A hypernetwork mechanism is introduced to dynamically adjust the weight distribution during the generation process, ensuring the consistency and diversity of generated images in terms of morphology and location. This meter defect generation method can controllably generate high-quality and diverse meter defect images under conditions of few samples, thereby improving the accuracy of meter defect detection networks from a data perspective. This invention applies the Stable Diffusion model to the generation of defect images of meters in substations with few samples. By combining meter knowledge embedding, crack feature modeling, and hypernetwork dynamic control, it effectively improves the quality and controllability of defect image generation, meets the data requirements of substation meter defect detection tasks under few sample conditions, and provides strong data support for the safe and stable operation of the power system. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of a stable diffusion method for generating defective images of meters with few samples according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the structure for crack feature modeling in an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of the structure generated by the hypernetwork control in an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the overall structure of an embodiment of the present invention;

[0056] Figure 5 The diagram illustrates the effect of generating meter defects in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Existing methods for generating meter defect images suffer from limitations such as insufficient generation quality, lack of effective control, and poor adaptability to small sample sizes. For example, traditional GANs (Generative Adversarial Networks) are prone to pattern collapse and training instability, resulting in blurry and significantly blurred images with significant ghosting, and excessive similarity to the original defect images, limiting diversity. Diffusion models rely on random denoising processes, making it difficult to constrain crack morphology and location, leading to generation results that deviate from actual requirements. Furthermore, existing methods rely on large-scale pre-training data, making it difficult to capture fine-grained features of meter defects under limited sample conditions, resulting in noisy or invalid defect images.

[0059] The purpose of this invention is to provide a stable diffusion method for generating meter defect images with few samples, addressing the problems of insufficient meter defect data in substations and the resulting poor performance of downstream detection tasks. By introducing meter knowledge embedding, crack feature modeling, and a hypernetwork dynamic control mechanism, a novel meter defect image generation method is designed. This method can controllably generate high-quality and diverse meter defect images under limited sample conditions, thereby improving the meter defect detection capability.

[0060] The method of this invention can significantly improve the generation quality and controllability of meter defect images under conditions of few samples. Its core advantages are as follows: First, the DreamBooth fine-tuning technology solves the problem of meter interpretation bias in the pre-trained Stable Diffusion model, and the generated image is highly consistent with the actual meter in terms of overall outline; Second, the crack feature modeling module uses the constraint map generated by SAM (Segment All Model) to limit the crack distribution area, realizing pixel-level precise control of crack morphology and location; Finally, the hypernetwork mechanism, without modifying the weights of the Stable Diffusion main model, dynamically controls the features of the control image to be injected into the diffusion process layer by layer, thus ensuring the consistency of crack generation.

[0061] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] like Figure 1 As shown, the present invention provides a stable diffusion method for generating defective images of meters with few samples, comprising the following steps:

[0063] S1. Construct a dataset containing normal substation meter images and meter images with crack defects, covering various meter types, including ammeters, voltmeters, pressure gauges, wattmeters, etc. All images are preprocessed, uniformly cropped to a resolution of 512×512, and the defect categories and location information are labeled. The dataset is divided into a training set and a validation set in an 8:2 ratio to ensure the rationality of data distribution.

[0064] S2. Fine-tune the pre-trained Stable Diffusion model using the DreamBooth method, bind the unique Chinese identifier [meter] to the meter image, different from using English identifiers in traditional methods, to achieve the embedding of meter knowledge and establish a strong association between the Chinese identifier and the meter image features. Design a dual-objective loss function to balance new feature learning and prior knowledge retention, ensuring that the model retains general generation ability when generating meter defect images.

[0065] S3. Propose a crack feature modeling module. Extract the meter line drawing through an adaptive edge detection algorithm, retain the structural features of the dial contour and key components, ensuring the integrity and rationality of the generated image in terms of geometric structure. Based on the Segment Anything Model (SAM), perform high-precision dial area segmentation on the input image to generate a dynamic constraint map, and further fuse the crack mask images provided by expert hand-drawn and industrial datasets. Precisely limit the cracks to only distribute in the effective area of the dial through the constraint image, avoiding the abnormal phenomenon of cracks appearing in non-functional areas.

[0066] S4. Adopt a hypernetwork mechanism to dynamically adjust the weight distribution during the generation process, ensuring the consistency and diversity of the generated images in terms of morphology and position. Freeze the weights of the Stable Diffusion main model, construct a trainable hypernetwork branch, encode the control image into a latent representation, and inject it layer by layer into the diffusion process to constrain the crack morphology of the generated image.

[0067] S5. Use the Stable Diffusion method for generating few-shot meter defect images to assist in improving the accuracy of the meter defect recognition network, and further verify the effectiveness of the few-shot substation meter defect generation method.

[0068] The basic flowchart of the present invention is as Figure 1 shown.

[0069] Considering that training a large model usually requires a large-scale labeled dataset and high computing resources, while the substation meter dataset is scarce, has complex features, and has a significant difference in data distribution from natural images, an effective method is needed to inject the feature knowledge of substation meters into existing large models. This method fine-tunes the pre-trained Stable Diffusion model through the DreamBooth method, binds the unique Chinese identifier [meter] to the meter image, differentiates from using English identifiers in traditional methods, realizes the embedding of meter knowledge, and establishes a strong association between the Chinese identifier and the meter image features. A dual-objective loss function is designed to balance new feature learning and prior knowledge retention, ensuring that the model retains general generation ability when generating meter defect images. The specific steps in step S2 include:

[0070] First, by fine-tuning Stable Diffusion, endow the unique Chinese identifier [meter] with a new meaning, inject it as an input prompt into the training process, and make the existing model adapt to the feature knowledge of substation meters, thereby enhancing the model's ability to generate meter defect images;

[0071] Next, in order to prevent the model from forgetting the prior knowledge learned from large-scale data, this method uses an optimization objective that includes a main loss and a prior preservation loss to achieve a balance between new concept learning and prior knowledge retention. The main loss term L sbj is used to optimize the model to generate images containing new features under specific conditions. By minimizing the gap between the model prediction and the target distribution, it guides the model to learn specific meter defect features. The mathematical expression is as follows:

[0072]

[0073] where x represents the real image, c is the text condition, ∈ is the random noise, t is the time step in the diffusion process, ω t is a weight term for a time step to balance the loss contributions of different time steps, represents the predicted image of the model, α t and σ t are diffusion noise scheduling parameters, performs an expectation calculation on x, c, ∈, t, that is, takes the average over the data distribution, and ‖·‖ 2 is the L2 norm, which measures the gap between the predicted image and the real image. The prior preservation loss term L pr aims to constrain the generation result of the model so that it does not deviate from the original data distribution, thereby avoiding the decline of the model's generalization ability caused by few-shot overfitting. The loss function is expressed as follows:

[0074]

[0075] where xpr c represents the model in the previously generated image. pr Here, ∈′ represents the corresponding textual condition, ∈′ represents another random noise term, and t′ represents the time step in the diffusion process. By introducing generated samples from large-scale unlabeled data, the model effectively avoids forgetting its general generation capabilities.

[0076] Finally, the two loss functions are dynamically balanced by weighting, and the combined form is expressed as follows:

[0077] L = L sbj +λL pr (3)

[0078] Here, L is the overall loss function, and λ∈[0,1] are weight parameters used to adjust the tradeoff between the main loss and the prior preservation loss, flexibly optimizing the generation capability according to the task. Fine-tuning the base model improves the ability to generate specific images while retaining the generalization ability of the base model.

[0079] In this invention, considering the complex morphology and irregular distribution of cracks in substation meters, it is necessary to closely integrate the geometric features of the cracks with the structural information of the meters. Furthermore, due to the diversity and randomness of crack distribution, a model-based method is required to achieve accurate modeling of crack features. This method proposes a crack feature modeling module, which generates a control image with geometric constraints by combining line drawings, crack masks, and constraint maps, ensuring accurate representation of crack morphology and location. The meter line drawings are extracted using edge detection and image smoothing techniques, and the constraint map generated by SAM (Simplified Image Modeling) limits the crack distribution range. Pixel-level fusion is then used to generate the control image, providing reliable geometric constraint support for subsequent generation tasks. Step S3 specifically includes:

[0080] First, the defect-free original image I o ∈R H×W×3 (Where R indicates that the image data belongs to the real number space, meaning that the value of each pixel is a real number; H and W represent the height and width of the image, respectively; and 3 represents the RGB three channels.) After edge detection, image smoothing, and style enhancement preprocessing, the line art image I is extracted. l ∈R H×W×1 Subsequently, the line drawing image and crack mask I d ∈R H×W×1 Pixel-by-pixel fusion to generate control image I c ′∈R H×W×1 The fusion process is represented as:

[0081] I c ′(x,y)=max(I d (x,y),I l (x,y)) (4)

[0082] Among them, I c ′(x,y),I d (x,y) and I l (x,y) represent the gray values ​​at pixel (x,y) of the control image, crack image, and line drawing image, respectively, with the range limited to [0,255]. The max(·) operation represents taking the maximum gray value at each pixel.

[0083] However, due to the unique distribution of cracks, directly combining the line drawing image with the crack image may result in anomalies where crack features appear outside the dial area, making accurate modeling of the dial region impossible. Therefore, SAM is used to limit the modeling range of cracks and improve its realism. For the input original image I... o Based on the user-provided prompt condition c, SAM first uses the image encoder to process I... o The feature representation f is generated by mapping to the embedding space; simultaneously, the cue condition c is encoded by the cue encoder to generate cue features g; subsequently, features f and g are fused in the mask decoder to generate the dial constraint map I. r The formula is as follows:

[0084] I r =D(f,g),I r ∈R H×W×1 (5)

[0085] Where D is the mask decoder, which, after obtaining the dial constraints, constrains them to the pixel-by-pixel fusion process of the line drawing image and the mask image to generate the final control image. The fusion process can be updated as follows:

[0086]

[0087] Among them, I c (x,y) and I r (x, y) represent the gray values ​​of the final control image and constraint image at pixel (x, y), respectively. r The grayscale value of (x,y) takes only two values: 0 and 255. The former indicates a region where combination is allowed, while the latter indicates that the original grayscale value of the line drawing image is preserved. By introducing a constraint graph, more precise region control can be applied during crack feature modeling, which helps to control the image with higher modeling rationality.

[0088] To address the generation of crack masks, this invention proposes a crack mask generation method based on a combination of expert hand-drawing and existing industrial datasets, ensuring the integrity and accuracy of crack features. Specifically, it includes:

[0089] The crack masks are sourced from crack defect images in existing industrial datasets and crack images hand-drawn by industry experts. The expert-drawn crack masks provide a reliable supplement to crack modeling, especially in cases where SAM segmentation may fail. They effectively compensate for deficiencies in dial region segmentation, ensuring the integrity and accuracy of crack features, thereby reducing the risk of dependence on segmentation and further improving the robustness and generalization performance of the generative model. By combining expert hand-drawn images with existing industrial datasets, the generated crack masks can more accurately reflect the morphology and distribution characteristics of actual meter cracks, providing high-quality input data for subsequent crack feature modeling.

[0090] The structural schematic diagram of crack feature modeling in this invention is shown below. Figure 2 As shown.

[0091] In this invention, considering the high-precision control requirements for crack morphology and location in the substation meter defect generation task, geometric constraints need to be closely integrated with the generation process. Furthermore, due to the limitations of traditional generation methods in control capabilities, a dynamic weight adjustment mechanism based on a hypernetwork is required. This method proposes a hypernetwork control generation module, which dynamically adjusts the weight distribution during the generation process by introducing trainable hypernetwork layers, ensuring that the generated image strictly adheres to preset crack morphology and location conditions. Specifically, in step S4, a hypernetwork mechanism is used to encode the control image into a latent representation, which is then injected layer by layer into the diffusion process to constrain the crack features of the generated image. This includes:

[0092] First, the weights of the Stable Diffusion master model are frozen to preserve its original prior image knowledge. Based on this, a trainable copy of the model is constructed as a supernetwork, dynamically adjusting the generation process of the master model by sharing some feature representations.

[0093] Secondly, the input conditional control image is processed by the encoder to generate a corresponding latent representation. This latent representation, combined with the dynamic weights generated by the hypernetwork, effectively adjusts the overall contour and crack details of the generated image. Simultaneously, the basic generative model generates a preliminary image; the two work together to complete the generation of the final image, ensuring that the generated result strictly conforms to the set conditions and achieves precise control.

[0094] Finally, the constraint image is used as input to roughly outline the overall image and reveal the location and shape of the cracks. At this stage, a specific loss function is designed to optimize the generation process, defined as follows:

[0095]

[0096] Where x represents the original image, c, c', c+ are conditional information, ∈ represents standard normally distributed noise, and t is the time step in the diffusion process. This represents a neural network-parameterized denoising model used to recover the original image from noisy data, α t and σ t It is a time-dependent scaling factor that controls the degree of noise addition. Calculating the expectation of x,c,c',∈,t is equivalent to finding the average over the data distribution, ‖·‖ 2 It is the L2 norm, which measures the difference between the predicted image and the real image. During the diffusion generation process, control signals are injected into each feature layer of the model multiple times, affecting both the initial generation stage and providing structural constraints in the middle and later stages, thereby ensuring that the generated result meets the set conditions.

[0097] The schematic diagram of the structure generated by the super network control in this invention is shown below. Figure 3 As shown.

[0098] In step S5, the accuracy of the meter defect identification network is improved by using a few-sample substation meter defect image generation method, further verifying the effectiveness of the few-sample substation meter defect generation method. Specifically, this includes:

[0099] To verify the effectiveness of this invention in generating defective images of substation meters with few samples, experiments were conducted on a defect detection task. The experimental procedure consisted of the following steps: First, a large number of substation meter images containing crack defects were generated using the method proposed in this invention; next, the generated defect images were mixed proportionally with a real defect dataset to construct an expanded training set; finally, the expanded dataset was used to train a defect detection model, and the model's detection performance was evaluated on an independent validation set. Experimental results show that the generated data significantly improved the accuracy and mAP50 index of the detection model, demonstrating the effectiveness of the proposed method and providing reliable data support for substation meter defect detection.

[0100] The present invention discloses a stable diffusion method for generating few-sample meter defect images. The network structure of this generation method is as follows: Figure 4 As shown.

[0101] The meter defect image generation effect of the method of the present invention is as follows: Figure 5As shown in the figure. The present invention designs a method for embedding meter knowledge based on DreamBooth. By fine-tuning the pre-trained model, the structural features and defect knowledge of substation meters are injected into the generation model, significantly improving the similarity between the generated images and the actual meters. A crack feature modeling module is proposed, which combines the line drawing, crack mask, and constraint map to generate a control image with geometric constraints, accurately expressing the shape and position of the crack. The hypernetwork mechanism is introduced to dynamically adjust the weight distribution during the generation process, ensuring the consistency and diversity of the generated images in terms of morphology and position. This meter defect generation method can controllably generate high-quality and diverse meter defect images under the condition of a small number of samples, assisting in improving the accuracy of the meter defect detection network from the data level. The present invention applies the StableDiffusion model to the generation of few-shot substation meter defect images. By combining meter knowledge embedding, crack feature modeling, and hypernetwork dynamic control, it effectively improves the quality and controllability of the defect image generation, meeting the data requirements of the substation meter defect detection task under the condition of few samples.

[0102] This application also provides a stable diffusion device for few-shot meter defect image generation, and the device includes:

[0103] Module S1, constructing a dataset containing normal substation meter images and meter images with crack defects, covering various meter types, including: ammeters, voltmeters, pressure gauges, wattmeters, etc. All images are pre-processed, uniformly cropped to a resolution of 512×512, and the defect category and position information are labeled. The dataset is divided into a training set and a validation set according to an 8:2 ratio to ensure the rationality of the data distribution.

[0104] Module S2, fine-tuning the pre-trained Stable Diffusion model by the DreamBooth method, binding the unique Chinese identifier [meter] to the meter image, different from using English identifiers in traditional methods, to achieve the embedding of meter knowledge and establish a strong association between the Chinese identifier and the meter image features. Design a dual-objective loss function to balance new feature learning and prior knowledge retention, ensuring that the model retains the general generation ability when generating meter defect images.

[0105] Module S3, proposing a crack feature modeling module, extracting the meter line drawing through an adaptive edge detection algorithm, retaining the structural features of the dial contour and key components, ensuring the integrity and rationality of the generated image in terms of geometric structure. Based on SAM, high-precision dial area segmentation is performed on the input image to generate a dynamic constraint map, and further fuse the crack mask images provided by expert hand-drawn and industrial datasets. The constraint image accurately restricts the crack to be distributed only in the effective area of the dial, avoiding the abnormal phenomenon of the crack appearing in the non-functional area.

[0106] The S4 module employs a hypernetwork mechanism to dynamically adjust the weight distribution during the generation process, ensuring consistency and diversity in the morphology and location of the generated images. It freezes the weights of the Stable Diffusion main model, constructs trainable hypernetwork branches, encodes the control image into a latent representation, and injects it layer by layer into the diffusion process, constraining the crack morphology of the generated images.

[0107] This application also provides an electronic device, the electronic device comprising: at least one memory and at least one processor; the memory stores a program, and the processor calls the program stored in the memory, the program being used to implement the stable diffusion method for generating defective images of few-sample meters.

[0108] This application also provides a storage medium storing computer-executable instructions for executing the stable diffusion method for generating images of defects in few-sample meters.

[0109] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

[0110] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0111] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that elements inherent to a process, method, article, or apparatus that comprises a list of elements, or elements inherent to such processes, methods, articles, or apparatus, are also included. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0112] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A stable diffusion method for few-shot metering defect image generation, characterized in that, The method comprises the following steps: S1, constructing a data set containing normal transformer meter image and meter image with crack defects, covering various meter types, all images are preprocessed, uniformly cropped to 512*512 resolution, and labeled with defect category and location information; S2, fine-tuning the pre-trained Stable Diffusion model through the DreamBooth method, binding the unique Chinese identifier [meter] with the meter image, which is different from the traditional method using English identifier, realizing the embedding of meter knowledge, establishing strong association between Chinese identifier and meter image features, designing a double-target loss function to balance new feature learning and prior knowledge reservation, and ensuring that the model retains general generation ability when generating meter defect images; S3, a crack feature modeling module is proposed, the meter line drawing is extracted through an adaptive edge detection algorithm, the structural features of the dial outline and key components are retained, the integrity and rationality of the generated image in the geometric structure are ensured, the input image is segmented based on SAM to generate a dynamic constraint image, and the crack mask image provided by the expert hand-drawing and industrial dataset is further fused, so that the crack is accurately limited to the effective area of the dial, and the abnormal phenomenon that the crack appears in the non-functional area is avoided; S4, an ultra-network mechanism is used to dynamically adjust the weight distribution in the generation process, ensure the consistency and diversity of the generated image in shape and position, freeze the Stable Diffusion main model weight, build a trainable ultra-network branch, encode the control image into a latent representation, and inject it into the diffusion process layer by layer to constrain the crack shape of the generated image; Among them, for the generation of crack mask, a crack mask generation method based on expert hand-drawing and existing industrial dataset is proposed to ensure the integrity and accuracy of crack features, which specifically includes: The source of the crack mask includes crack defect images in existing industrial datasets and crack images drawn by industry experts. The crack mask drawn by experts provides a reliable supplement for crack modeling, and can effectively make up for the deficiency of dial area segmentation in the case of SAM segmentation failure, ensure the integrity and accuracy of crack features, thereby reducing the risk of dependence on segmentation and further improving the robustness and generalization performance of the generation model. By combining expert hand-drawing and existing industrial datasets, the generated crack mask can more accurately reflect the shape and distribution characteristics of actual meter cracks, providing input data for subsequent crack feature modeling.

2. The stable diffusion method of claim 1, wherein, S2 specifically includes: First, by fine-tuning Stable Diffusion, a unique Chinese identifier [meter] is given a new meaning, which is injected into the training process as an input prompt, making the existing model adapt to the characteristics of transformer meters, to enhance the model's ability to generate meter defect images; Next, to prevent the model from forgetting the prior knowledge learned from large-scale data, an optimization objective containing a subject loss and a prior preservation loss is used to balance between new concept learning and prior knowledge preservation, the subject loss term For optimizing the model to generate images containing new features under specific conditions, the model is guided to learn specific meter defect features by minimizing the difference between the model prediction and the target distribution, and the mathematical expression is as follows: (1) in, Represents a real image. It is a text condition. It is random noise. It is the time step in the diffusion process. It is a weight term for a time step used to balance the loss contributions of different time steps. The image represents the model's prediction. and These are the parameters for scheduling diffused noise. right Calculating the expected value involves taking the average over the data distribution. It is the L2 norm, which measures the difference between the predicted image and the real image; it is a prior-preserving loss term. The aim is to constrain the model's generated results so that they do not deviate from the original data distribution, thereby avoiding a decrease in the model's generalization ability due to overfitting with few samples. The loss function is expressed as follows: (2) wherein, represents the model in the previously generated image, is the corresponding text condition, is another random noise term, is the time step in the diffusion process, by introducing the generated samples of large-scale unlabeled data, effectively avoiding the model forgetting the general generation ability; Finally, the two loss functions are dynamically balanced in a weighted manner, and the combination form is as follows: (3) wherein, is the overall loss function, is a weight parameter, which is used to adjust the trade-off between the main loss and the prior preservation loss, to flexibly optimize the generation ability according to the task, and to fine-tune the base model to improve the ability to generate specific images while preserving the generalization ability of the base model.

3. The stable diffusion method of claim 1, wherein, S3 specifically includes: First, the original image without defects After edge detection, image smoothing and style enhancement preprocessing, the line drawing image is extracted where, represents that the image data belongs to the real number space, that is, the value of each pixel is a real number, and respectively represent the height and width of the image, 3 represents the RGB three channels, and then the line drawing image and the crack mask are fused pixel by pixel to generate a control image The fusion process is represented as: (4) wherein, , and are the gray values of the control image, the crack image and the line drawing image at the pixel point , respectively, with the range limited to [0, 255], operation means taking the maximum gray value at each pixel point; However, due to the particularity of the crack distribution, directly combining the line drawing image with the crack image may cause abnormal phenomena that the crack features appear outside the dial, which cannot be accurately modeled for the dial area. Therefore, the SAM limits the modeling range of the crack to improve its authenticity. For the input original image and the prompt condition provided by the user , the SAM first maps the image to the embedding space through the image encoder to generate the feature representation ; at the same time, the prompt condition is encoded through the prompt encoder to generate the prompt feature ; then, the features and are fused in the mask decoder to generate the dial constraint map , which is as follows: (5) wherein, is a mask decoder, after getting the watch face constraints, it constrains the line drawing image and the mask image to a pixel-by-pixel fusion process to generate a final control image, and the fusion process can be updated as: (6) wherein, and respectively represent the gray value of the pixel point of the final control image and the constraint image, The gray value of the pixel point is only 0 and 255, the former represents the region allowed to be combined, and the latter represents the original gray value of the line drawing image. By introducing the constraint image, more accurate regional control can be applied in the crack feature modeling process, which helps to control the modeling rationality higher.

4. The stable diffusion method of claim 1, wherein, S4 specifically includes: Firstly, the weights of the Stable Diffusion main model are frozen to preserve its original image prior knowledge, and a trainable model copy is constructed as a super network to dynamically adjust the generation process of the main model by sharing part of the feature representation; Secondly, the input conditional control image is processed by the encoder to generate the corresponding latent representation, which is combined with the dynamic weights generated by the super network to effectively adjust the overall outline and crack details of the generated image. At the same time, the basic generation model generates a preliminary image, and the two work together to complete the final image generation, ensuring that the generated result strictly meets the set conditions and achieves precise control effect; Finally, the constraint image is used as the input condition to show the overall outline of the image and the position and shape of the crack. In this stage, a specific loss function is designed to optimize the generation process, defined as follows: (7) wherein, denotes the original image, is the conditional information, is the standard normal distributed noise, is the time step in the diffusion process, denotes a neural network parameterized denoising model for recovering the original image from the noisy data, and is a time-dependent scaling factor controlling the degree of noise injection, the expectation operation, i.e. the average over the data distribution, is the L2 norm measuring the difference between the predicted image and the true image, and is the L2 norm measuring the difference between the predicted image and the true image, and 5. A stable diffusion device for few-shot meter defect image generation, characterized in that, The device comprises: S1 module, for constructing a data set containing normal transformer station meter image and meter image with crack defects, covering various meter types, all images are preprocessed and cropped to 512x512 resolution, and defect category and position information are labeled; S2 module, for fine-tuning the pre-trained Stable Diffusion model through the DreamBooth method, binding the unique Chinese identifier [meter] with the meter image, which is different from the traditional method using English identifier, realizing the embedding of meter knowledge, establishing strong association between Chinese identifier and meter image features, designing a double-target loss function to balance new feature learning and prior knowledge preservation, ensuring that the model retains general generation ability when generating meter defect images; S3 module, proposes a crack feature modeling module, extracts meter line drawing through an adaptive edge detection algorithm, preserves the structural features of the dial outline and key components, ensures the integrity and rationality of the generated image in terms of geometric structure, performs high-precision dial area segmentation on the input image based on SAM, generates a dynamic constraint image, further combines the crack mask image provided by expert hand-drawing and industrial dataset, accurately limits the crack to be distributed only in the effective area of the dial, avoiding the abnormal phenomenon of crack appearing in the non-functional area; S4 module, adopts a super network mechanism to dynamically adjust the weight distribution in the generation process, ensuring the consistency and diversity of the generated image in shape and position, freezes the weights of the Stable Diffusion main model, constructs a trainable super network branch, encodes the control image into a latent representation, injects it into the diffusion process layer by layer, and constrains the crack shape of the generated image; Among them, for the generation of crack mask, a crack mask generation method based on expert hand-drawing and existing industrial dataset is proposed to ensure the integrity and accuracy of crack features, which includes: The sources of the crack masks include crack defect images in existing industrial data sets and crack images drawn by industry experts, wherein the crack masks drawn by experts provide a reliable supplement for crack modeling, can effectively make up for the deficiency of the dial area segmentation in the case of SAM segmentation failure, ensure the integrity and accuracy of the crack features, thereby reducing the risk of dependence on segmentation, further improving the robustness and generalization performance of the generation model, and through the combination of expert drawing and existing industrial data sets, the generated crack masks can more accurately reflect the morphology and distribution characteristics of the actual meter crack, providing input data for subsequent crack feature modeling.

6. An electronic device, comprising: The electronic device includes at least one memory and at least one processor; the memory stores a program, and the processor calls the program stored in the memory, and the program is used to realize the stable diffusion method for few-shot meter defect image generation in any one of claims 1-4.

7. A storage medium, characterized by The storage medium stores computer executable instructions, and the computer executable instructions are used to execute the stable diffusion method for few-shot meter defect image generation in any one of claims 1-4.

Images