Substation defect image generation method, system, device, and program product

Abstract

This application relates to the field of artificial intelligence image generation technology, and provides a method, system, device, and program product for generating substation defect images. The method includes: acquiring a standard image of defect-free equipment; performing semantic segmentation on the standard image of defect-free equipment to obtain its equipment contour information and key region information, wherein the equipment contour information and key region information constitute a description of equipment structural features; obtaining the correspondence between equipment categories and defect types; dividing the equipment structural region in the description of equipment structural features into a defect target generation region and a defect restriction generation region according to the correspondence; generating a binary mask; obtaining a binary space attention mask based on the binary mask; and inputting a pre-constructed domain semantic cue word and the binary space attention mask into a diffusion model to generate a substation defect image, thereby improving the realism and controllability of the generated image.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence image generation technology, and in particular relates to a method, system, device and program product for generating images of substation defects. Background Technology

[0002] With the development of smart grids, deep learning-based visual inspection technology plays a crucial role in substation equipment defect identification. However, the performance of this technology depends on sufficient training data. In actual operating environments, the probability of equipment defects is low, making it difficult to obtain defect image samples, which restricts the training effect and generalization ability of intelligent detection models.

[0003] To address the problem of data scarcity, existing technologies attempt to expand datasets using generative models. However, generative models trained on general datasets suffer from domain distribution differences between their training data and substation scenarios. This results in distorted main equipment structures or misaligned components in the generated images, defect textures that do not conform to material physical properties, unnatural integration of defects with the background, and a lack of consistent lighting and shadow relationships in realistic inspection images. Summary of the Invention

[0004] This application provides a method, system, device, and program product for generating substation defect images, aiming to solve the technical problems in the prior art where there is a difference in domain distribution between generative models trained on general datasets and substation scenes, resulting in distortion of the main structure of equipment, defect textures that do not conform to physical characteristics, and unnatural integration of defects with the background in the generated images.

[0005] In a first aspect, embodiments of this application provide a method for generating substation defect images, the method comprising: Obtain defect-free standard equipment images; The method involves semantic segmentation of the defect-free standard equipment image to extract the main equipment region and generate a binary spatial attention mask to indicate the defect target generation region and the defect restriction generation region. This includes: performing semantic segmentation on the defect-free standard equipment image to obtain equipment contour information and key region information; wherein the equipment contour information and the key region information constitute a description of the equipment structure features of the defect-free standard equipment image; obtaining a preset correspondence between equipment categories and defect types; dividing the equipment structure region in the equipment structure feature description into a defect target generation region and a defect restriction generation region based on the correspondence; generating a binary mask, assigning a first value to pixels within the defect target generation region and a second value to pixels within the defect restriction generation region; and obtaining the binary spatial attention mask based on the binary mask. The pre-built domain semantic cue words and the binary space attention mask are input into the diffusion model to generate substation defect images.

[0006] In one possible implementation of the first aspect, the step of inputting pre-constructed domain semantic cue words and the binary space attention mask into a diffusion model to generate a substation defect image includes: The domain semantic cue words and the binary space attention mask are input into the diffusion model; A reverse denoising process is performed based on the diffusion model, and a latent feature map is generated during the reverse denoising process. The latent feature map is subjected to spatial constraints pixel by pixel based on the binary spatial attention mask to constrain the generation of defects within the defect target generation area, thereby generating the substation defect image.

[0007] In one possible implementation of the first aspect, the method further includes: In the reverse denoising process, the device category and defect type of the generated image are constrained according to the domain semantic cue words.

[0008] In one possible implementation of the first aspect, the domain semantic prompts include device trigger words, device visual feature description information, and defect status description information.

[0009] In one possible implementation of the first aspect, the method further includes: Acquire a defect image dataset of substation equipment; the images in the defect image dataset include equipment morphological features and image detail information. Based on the device morphological characteristics, the image detail information, and the available computing resource constraints, the optimal training resolution is selected from the candidate training resolution set; The images in the defective image dataset are resized according to the optimal training resolution to construct a training dataset; the training dataset is used to train the diffusion model.

[0010] In one possible implementation of the first aspect, the method further includes: A low-rank matrix is ​​injected into the attention layer of the U-Net network in the diffusion model; Freeze the original parameters of the diffusion model; The parameters of the low-rank matrix are optimized using the training dataset and the domain semantic prompts to obtain low-rank adaptation parameters; The low-rank adaptation parameters are loaded into the diffusion model and superimposed on the original parameters of the diffusion model.

[0011] Secondly, embodiments of this application provide a substation defect image generation system, comprising: The acquisition module is used to acquire defect-free standard equipment images; An extraction module is used to perform semantic segmentation on the defect-free standard equipment image to obtain equipment contour information and key region information of the defect-free standard equipment image; wherein, the equipment contour information and the key region information constitute the equipment structural feature description of the defect-free standard equipment image; obtain a preset correspondence between equipment categories and defect types; according to the correspondence, divide the equipment structural region in the equipment structural feature description into a defect target generation region and a defect restriction generation region; generate a binary mask, assign a first value to the pixels in the defect target generation region, and assign a second value to the pixels in the defect restriction generation region; obtain the binary space attention mask based on the binary mask; The generation module is used to input pre-built domain semantic prompts and the binary space attention mask into the diffusion model to generate substation defect images.

[0012] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the substation defect image generation method described in any one of the first aspects above.

[0013] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the substation defect image generation method described in any one of the first aspects.

[0014] Fifthly, embodiments of this application provide a computer program product that, when run on a computer device, causes the computer device to execute the substation defect image generation method described in any one of the first aspects.

[0015] In this embodiment, semantic segmentation is performed on a standard image of defect-free equipment to obtain equipment outline information and key region information, which together constitute a description of the equipment structure features. Based on a preset correspondence between equipment categories and defect types, the equipment structure region in the description is divided into a defect target generation region and a defect restriction generation region. A binary mask is generated, assigning a first value to pixels within the defect target generation region and a second value to pixels within the defect restriction generation region. A binary space attention mask is obtained based on the binary mask. The domain semantic cue words and the binary space attention mask are input into a diffusion model to generate a substation defect image. This ensures that the generated defects are located within the defect target generation region, reducing the occurrence of false defects in the background region, and allows control over the equipment category and defect type of the generated image, improving the realism and controllability of the generated image.

[0016] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of a substation defect image generation method provided in an embodiment of this application; Figure 2 This is a structural block diagram of a substation defect image generation system provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0020] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0021] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0023] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] Figure 1 A schematic flowchart of a substation defect image generation method provided in an embodiment of this application is shown.

[0026] S101, Obtain a defect-free standard equipment image.

[0027] Among them, the defect-free standard equipment image refers to an image that shows the appearance of the target equipment in normal working condition. This image does not contain defective areas and is used as a reference for generating the spatial constraint mask.

[0028] S102, perform semantic segmentation on the defect-free standard equipment image, extract the main equipment region of the defect-free standard equipment image, and generate a binary space attention mask to indicate the defect target generation region and the defect restriction generation region.

[0029] Semantic segmentation refers to a computational technique that assigns a category label to each pixel in an image to distinguish between the main pixels of the device and the background pixels, thereby extracting the main device region.

[0030] The main device area refers to the pixel range occupied by the device itself in the image, excluding the background and irrelevant objects, and is used as a spatial reference for the subsequent defect generation location.

[0031] Semantic segmentation can be achieved using deep learning-based image segmentation networks, such as encoder-decoder structured segmentation networks. The output of such networks is a pixel-level category probability map of the same size as the input image. By taking the category with the highest probability as the predicted label for each pixel, the segmentation result of the main device region is obtained.

[0032] Binary attention mask refers to a binary mask with the same dimension as the latent feature map of the diffusion model. It contains two values ​​that correspond to the defect target generation region and the defect restriction generation region, respectively. It is used to indicate the allowed generation position of the defect pixel by pixel in the reverse denoising process of the diffusion model.

[0033] In this embodiment of the application, the pixel-level range of the main area of ​​the device is extracted by semantic segmentation, and the defects are constrained within the allowable area based on the prior knowledge of the device structure, so that the position of the defects in the generated image conforms to the physical laws.

[0034] Specifically, the step of semantically segmenting the defect-free standard equipment image, extracting the main equipment region of the defect-free standard equipment image, and generating a binary spatial attention mask to indicate the defect target generation region and the defect constraint generation region includes: Step a1: Perform semantic segmentation on the defect-free standard equipment image to obtain equipment outline information and key region information of the defect-free standard equipment image; wherein, the equipment outline information and the key region information constitute the equipment structural feature description of the defect-free standard equipment image.

[0035] Among them, equipment outline information refers to the spatial location data of the outer boundary and overall shape of the main body of the equipment, which is usually represented by a binary mask or a polygon coordinate sequence.

[0036] Critical area information refers to the location data of various functional or structural components on the equipment, such as the metal support area, the insulator area, and the conductive joint area.

[0037] Equipment structural feature description refers to a structured spatial representation of the equipment as a whole and its components, which is composed of equipment outline information and key area information.

[0038] In this embodiment of the application, semantic segmentation is used to transform the image into a structured description of the device's structural features, providing operable region boundaries and component category information for subsequent region division.

[0039] Step a2: Obtain the preset correspondence between equipment categories and defect types.

[0040] The correspondence between equipment categories and defect types refers to a set of rules predefined based on knowledge in the power industry, indicating the types of defects that may occur on various types of equipment. This correspondence is stored in the form of a lookup table or rule base. For example, for grounding down conductor equipment, the corresponding defect types include corrosion and breakage; for insulator equipment, the corresponding defect types include cracks and contamination.

[0041] In this embodiment of the application, by introducing the correspondence between equipment categories and defect types, physical laws are embedded into the generation process in the form of rules, so that defect generation is limited to the allowed area of ​​the equipment.

[0042] Step a3: Based on the correspondence, divide the equipment structure region in the equipment structure feature description into a defect target generation region and a defect restriction generation region.

[0043] The equipment structure area refers to the total number of all equipment components in the equipment structure feature description, including the main equipment area and various key areas.

[0044] The defect target generation area refers to the area in the equipment structure area where defects are allowed to be generated.

[0045] The defect restriction generation area refers to the area in the equipment structure area where defect generation is prohibited, as well as the background area.

[0046] In this embodiment of the application, the region is divided based on domain knowledge so that the defect generation region matches the physical structure of the device.

[0047] Step a4: Generate a binary mask, assign a first value to the pixels in the defect target generation area, and assign a second value to the pixels in the defect restriction generation area.

[0048] A binary mask is an image mask that contains only two pixel values, used to distinguish two spatial regions with different attributes with pixel-level precision. The first value and the second value are different numerical values ​​used to distinguish regions with two different attributes in the binary mask; the first value is 1, and the second value is 0.

[0049] Step a5: Based on the binary mask, perform dimension alignment processing to obtain the binary space attention mask.

[0050] Dimension alignment refers to adjusting the size of the binary mask to match the dimensions of the latent feature maps processed within the U-Net diffusion model, ensuring that the mask can participate in pixel-wise computations in the latent feature space. Dimension alignment is achieved through bilinear interpolation scaling.

[0051] In this embodiment, by dimensional alignment, the binary spatial attention mask can participate in pixel-wise multiplication operations inside the U-Net residual layer, thereby continuously applying spatial constraints in each step of reverse denoising.

[0052] S103, input the pre-constructed domain semantic prompts and the binary space attention mask into the diffusion model to generate a substation defect image.

[0053] Domain semantic prompts are structured text instructions used to semantically control the device category, appearance attributes, and defect status of images generated by the diffusion model. The final generated domain semantic prompts are derived from device trigger words. Equipment visual feature description information and defect status description information Combining the elements, its expression is: Where + indicates sequential connection. The device visual feature description information... From appearance attribute information Environmental background information and image style information Combining these elements, its expression is: , It refers to the visual appearance characteristics of the target substation equipment, such as its material, color, and shape. This refers to the scene characteristics of the environment in which the target substation equipment is located, including ground type, lighting conditions, and surrounding objects; It refers to the visual style of the generated image, such as the style of a realistic photograph.

[0054] In this embodiment of the application, the prompt word serves as the textual conditional input to the diffusion model, and the target defect image is generated by combining the trigger word, the visual features of the defect, and the degree of the defect.

[0055] In this context, the diffusion model refers to a generative model that generates new samples by learning to gradually recover the data distribution from noise. The diffusion model includes a forward diffusion process and a backward denoising process. The forward diffusion process gradually adds noise to the training data until it becomes pure noise, while the backward denoising process learns to gradually recover the original data from the noise. In this application, the diffusion model is a text-to-image diffusion model, which injects the textual semantic information of domain semantic cues into the image generation process through a cross-attention mechanism.

[0056] Substation defect images refer to the final generated defect sample images that contain specific equipment types, defect types and defect severity, and whose defect locations are constrained.

[0057] In an optional embodiment, step S103, which involves inputting pre-built domain semantic cue words and the binary space attention mask into a diffusion model to generate a substation defect image, includes: Step b1: Input the domain semantic prompts and the binary space attention mask into the diffusion model.

[0058] Step b2: Perform a reverse denoising process based on the diffusion model to generate a latent feature map during the reverse denoising process.

[0059] The reverse denoising process refers to the diffusion model starting from a standard Gaussian noise image and progressively removing noise at multiple time steps to recover the core image. Standard Gaussian noise refers to random noise where each element is independently sampled from a normal distribution with a mean of 0 and a variance of 1. Specifically, the reverse denoising process starts from the first time step... Starting at time step 0, the latent feature map is standard Gaussian noise. The reverse denoising operation is performed step by step until time step 0, resulting in clear image latent features.

[0060] Among them, the latent feature map refers to the feature representation that exists in the low-dimensional latent space at the intermediate time step in the reverse denoising process. It contains the semantic and structural information of the image being generated.

[0061] Step b3: Apply spatial constraints to the latent feature map pixel by pixel according to the binary spatial attention mask to constrain the generation of defects within the defect target generation area, thereby generating the substation defect image.

[0062] The spatial constraint refers to the selective preservation or reset operation performed on each pixel position of the latent feature map in each step of the inverse denoising process, based on the binary spatial attention mask. This spatial constraint is achieved through the following formula:

[0063] in, Indicates the first Step-by-step reverse denoising of the input latent feature map; This represents the binary space attention mask, whose dimension is... same; This represents the Hadamard pixel-wise product operation, which is the operation of multiplying corresponding elements of two matrices of the same dimension. Indicates and Standard Gaussian random noise of the same dimension, each element of which is independently sampled from a normal distribution with a mean of 0 and a variance of 1; This represents the attention mask in the binary space. Invert each pixel, that is, subtract 1 from each pixel position. The values ​​at the corresponding positions in the results show that areas with a value of 1 correspond to the defect restriction generation area, and areas with a value of 0 correspond to the defect target generation area. The output after spatial constraints represents the first... Step-diving feature map.

[0064] By employing the dual constraints of domain semantic cues and binary space attention masks, the generation process of defective images can be uniformly represented as follows: ,in, These are domain semantic prompts. For binary space attention mask, This represents the generator function of the diffusion model, which performs an inverse denoising process based on domain semantic cues and a binary space attention mask, outputting a generated substation defect image. .

[0065] In this embodiment, by continuously applying pixel-by-pixel constraints throughout the reverse denoising process, the diffusion of defect features to non-target regions is reduced, thereby achieving control over the defect generation location.

[0066] In an optional embodiment, the method further includes: In the reverse denoising process, the device category and defect type of the generated image are constrained according to the domain semantic cue words.

[0067] The device category and defect type of the generated image based on the domain semantic cue words are determined by injecting the textual semantic information of the domain semantic cue words into the generation process of the latent feature map through the cross-attention mechanism of the diffusion model, so as to control the device category, appearance attributes and defect status of the generated image.

[0068] In this embodiment, spatial constraints determine the location of the defect, while semantic constraints determine the content and category of the defect. These two constraints work together in the model's latent space to ensure that the generated image simultaneously satisfies preset conditions in both content and location.

[0069] In one optional embodiment, the domain semantic prompts include device trigger words, device visual feature description information, and defect status description information.

[0070] Among them, equipment trigger words are identifiers corresponding to target substation equipment, used to activate the visual features of specific equipment in the diffusion model. Equipment visual feature description information refers to natural language information describing the appearance attributes, environmental background, and image style of the target substation equipment. Defect status description information refers to natural language information describing the type and severity level of the target defect.

[0071] In this embodiment of the application, by structuring the domain semantic prompts into three independent dimensions, the generation task is decomposed into multiple controllable sub-tasks, thereby enabling control over the generated content.

[0072] In one optional embodiment, the construction of the device trigger word includes: Obtain the name of the target substation equipment; generate a unique identifier based on the name of the target substation equipment, and use the unique identifier as the equipment trigger word of the target substation equipment.

[0073] The unique identifier is a short identifier generated based on the name of the target substation equipment and bound to the visual features of the equipment during the diffusion model training process. For example, "ecjxh" corresponds to "secondary junction box", "jdyxx" corresponds to "grounding down conductor", and "ddjt" corresponds to "conductive connector".

[0074] In this embodiment of the application, by using a unique identifier instead of a natural language name, the ambiguity of the diffusion model's understanding of the device name is reduced, and the consistency when generating specific devices is improved.

[0075] In one optional embodiment, the construction of the device visual feature description information includes: Obtain the equipment appearance attribute information, equipment environmental background information, and image style information of the target substation equipment; combine the equipment appearance attribute information, equipment environmental background information, and image style information into the equipment visual feature description information.

[0076] Among these, equipment appearance attribute information refers to the visual appearance characteristics of the target substation equipment, such as its material, color, and shape. Equipment environmental background information refers to the scene characteristics of the environment in which the target substation equipment is located, such as ground type, lighting conditions, and surrounding objects. Image style information refers to the visual style of the generated image, such as a realistic photographic style.

[0077] In this embodiment of the application, by introducing three types of information—device appearance, environment, and style—the diffusion model learns the scene characteristics of the device while generating defects, and the generated image is visually close to the real inspection photo.

[0078] In an optional embodiment, the construction of the defect status description information includes: Obtain defect type information and defect severity level information; combine the defect type information and defect severity level information into the defect status description information.

[0079] The defect type information refers to the name of the type of defect, such as rust, cracks, condensation, and dirt. The defect severity level information refers to the graded description of the degree of defect development, such as minor, moderate, and severe. For example, for a blurry meter dial caused by moisture, a minor condition is described as "low coverage of water droplets," a moderate condition as "medium coverage of water droplets," and a severe condition as "high coverage of water droplets."

[0080] In this embodiment of the application, by expanding the defect category label to include a state description that includes severity, the diffusion model is able to generate defect samples at different stages of development.

[0081] In an optional embodiment, the method further includes: Step c1: Obtain a defect image dataset of substation equipment; the images in the defect image dataset include equipment morphological features and image detail information.

[0082] Among them, the defect image dataset refers to a collection of real-world images of substation equipment defects, which is used to train the diffusion model to learn the equipment structure and defect characteristics of a specific domain.

[0083] Equipment morphological characteristics refer to the spatial structural attributes of the main body of the equipment in the image, including aspect ratio. Image detail information refers to the fine-grained visual information of defects, textures, edge contours, and color distribution in the image.

[0084] Step c2: Select the optimal training resolution from the candidate training resolution set based on the device morphology features, the image detail information, and the available computing resource constraints.

[0085] The candidate training resolution set refers to a set of multiple possible training image size options.

[0086] Available computing resource constraints refer to the available hardware memory capacity during training.

[0087] Step c3: Adjust the size of the images in the defect image dataset according to the optimal training resolution to construct a training dataset; the training dataset is used to train the diffusion model.

[0088] The training dataset refers to the standardized image set obtained by uniformly adjusting the size of all images in the defective image dataset according to the optimal training resolution.

[0089] In this embodiment of the application, by selecting a candidate training resolution that matches the device form factor before training, the structural distortion and loss of detail caused by a fixed resolution are reduced.

[0090] In an optional embodiment, step c2 selects the optimal training resolution from the candidate training resolution set based on the device form features, the image detail information, and available computing resource constraints, including: Step d1: Calculate the average aspect ratio of all images in the defective image dataset.

[0091] Among them, the average width-to-height ratio The calculation formula is:

[0092] in, This represents the total number of images in the defective image dataset, and is a positive integer. Indicates the first The height of the image is a positive integer, in pixels. Indicates the first The width of the image is a positive integer, in pixels; Indicates the first The aspect ratio of the image is a positive real number.

[0093] Step d2: For each candidate training resolution, obtain the device form factor matching score of the candidate training resolution based on the degree of difference between the aspect ratio of the candidate training resolution and the average aspect ratio.

[0094] Among them, the device form matching score The calculation formula is:

[0095] in, Indicates the first One candidate training resolution, ; Indicates the first The height of each candidate training resolution is a positive integer in pixels. Indicates the first The width of each candidate training resolution is a positive integer in pixels. The average width-to-height ratio is represented by a positive real number. The range of values ​​is 0. 1.

[0096] This function measures the degree of fit between the candidate training resolution and the original device form factor. The aspect ratio of the candidate training resolution is compared to the average aspect ratio of the dataset. The closer they get, The larger the value, the better. For example, grounding down conductors typically have a distinct elongated structure, with an image aspect ratio of approximately 1:4, and the average aspect ratio of the dataset... Approximately 0.25. Aspect ratio of the candidate training resolution (1024×256). ,and Consistent Achieving a maximum value of 1 effectively maintains the device's elongated structure; while a fixed resolution (512×512) has an aspect ratio of 1, which is consistent with... Significant differences can lead to compression along the length of the equipment, resulting in structural proportion distortion.

[0097] Step d3: For each candidate training resolution, obtain the image detail preservation score of the candidate training resolution based on the total number of pixels of the candidate training resolution.

[0098] Among them, the image detail preservation score The calculation formula is:

[0099] in, The result of the natural logarithm (with base e) is given by the equation, and its value varies with... It increases monotonically.

[0100] This function measures the ability of candidate training resolutions to preserve defect texture and edge details. For example, in a dataset of rust defects in substation metal components, if the defect area is small (e.g., the rust area occupies about 5% of the original image), training at a resolution of (256×256) will result in fewer pixels after mapping the defect area, which can easily lead to the loss of rust texture. Training at a resolution of (768×768) can provide more effective pixels and improve the expressive power of defect edges and texture features.

[0101] Step d4: For each candidate training resolution, obtain the resource consumption score of the candidate training resolution based on the total number of images in the defect image dataset, the total number of pixels of the candidate training resolution, and the available computing resource constraint parameters.

[0102] Among them, the resource consumption score The calculation formula is:

[0103] in, This represents the total number of images in the defective image dataset, and is a positive integer. The height of the candidate training resolution is a positive integer in pixels. The width of the candidate training resolution is represented in pixels and is a positive integer. This represents the available computing resource constraint parameter, expressed in bytes of video memory, and is a positive real number. The range of values ​​is 0. +∞.

[0104] This function measures the training resource overhead under given dataset size and hardware conditions. The larger the value, the higher the proportion of computing resources consumed by that resolution. When the training resolution exceeds the video memory limit, the candidate training resolution will be automatically discarded. For example, when the number of training samples is 500 and 24GB of video memory is used for training, choosing a (768×768) resolution will not cause significant video memory pressure and can prioritize the ability to preserve image details; however, when the similar dataset is expanded to more than 5000 images, if (768×768) resolution is still used for training, it will significantly increase video memory usage and training time. In this case, it is preferable to choose a resolution with lower resource consumption such as (512×512) or (512×768).

[0105] Step d5: Obtain the comprehensive score for each candidate training resolution based on the device shape matching score, the image detail preservation score, and the suppression coefficient derived from the resource consumption score for each candidate training resolution.

[0106] Among them, the overall score The calculation formula is:

[0107] Where α represents the shape matching degree weight coefficient, β represents the detail preservation degree weight coefficient, and γ represents the resource consumption suppression weight coefficient. α, β, and γ are all positive real numbers, and their values ​​are all greater than 0 and less than 1, and α+β+γ=1. The inhibition coefficient is derived from the resource consumption score. It is a positive real number.

[0108] The three weighting coefficients can be dynamically adjusted according to the device form and hardware conditions. When the device images in the dataset have obvious elongated structures, the weight of α is increased; when the defect texture is delicate and more details need to be preserved, the weight of β is increased; when the number of input images is large or GPU memory resources are limited, the weight of γ is increased to suppress the resource consumption caused by high resolution.

[0109] Step d6: Select the candidate training resolution with the highest comprehensive score as the optimal training resolution.

[0110] Among them, the optimal training resolution The selection formula is:

[0111] in, The set of candidate training resolutions. Indicates the first One candidate training resolution, This represents the total number of candidate training resolutions, and is a positive integer. This represents the overall score.

[0112] In this embodiment of the application, a quantitative decision-making mechanism is established between device structure fidelity, defect detail clarity and training resource consumption through a comprehensive evaluation function.

[0113] In an optional embodiment, the method further includes: Step e1: Inject a low-rank matrix into the attention layer of the U-Net network of the diffusion model.

[0114] Among them, the U-Net network refers to the backbone neural network responsible for progressive denoising in the diffusion model. Its structure includes encoder path, decoder path and skip connections.

[0115] Attention layers refer to the layers in the U-Net network that employ attention mechanisms, including cross-attention layers and self-attention layers. Cross-attention layers use the feature representations of domain semantic cues as query vectors to perform attention calculations with latent image features, injecting textual conditions into the image generation process. Self-attention layers model inter-pixel correlations within the latent image features, capturing the spatial structural relationships of the image.

[0116] A low-rank matrix is ​​a trainable matrix with fewer parameters than the original weight matrix, used to capture incremental knowledge in a specific domain.

[0117] In this embodiment, a trainable adapter structure is established for subsequent incremental learning by injecting low-rank matrices into the cross-attention layer and self-attention layer of the U-Net network.

[0118] Step e2: Freeze the original parameters of the diffusion model.

[0119] Freezing refers to keeping the original parameters of the diffusion model unchanged during training.

[0120] In this embodiment of the application, the original parameters are frozen to prevent the destruction of the general image generation capabilities learned by the pre-trained model during domain adaptation.

[0121] Step e3: Optimize the parameters of the low-rank matrix using the training dataset and the domain semantic prompts to obtain the low-rank adaptation parameters.

[0122] The low-rank adaptation parameter refers to the incremental weight parameter obtained by optimizing the low-rank matrix, which is used to capture incremental knowledge in a specific domain.

[0123] During the training process, the images in the training dataset are... Domain semantic prompts Combined into input pairs The common input diffusion model enables the low-rank matrix to simultaneously learn the correspondence between device visual features, defect texture features, and text semantics.

[0124] In this embodiment, by optimizing only the low-rank adaptation parameters, the number of training parameters is reduced from billions to millions in the original model, which reduces the need for data and reduces overfitting in small sample scenarios.

[0125] Step e4: Load the low-rank adaptation parameters into the diffusion model and superimpose them with the original parameters of the diffusion model.

[0126] The formula for superimposing the low-rank adaptation parameter with the original parameters of the diffusion model is:

[0127] in, This represents the original parameter matrix of the pre-trained diffusion model; This represents the low-rank adaptation parameter matrix obtained during training. This represents the valid model parameters used during inference generation.

[0128] In this embodiment of the application, the trained low-rank adaptation parameters are used... Compared with the original parameters Overlaying achieves the goal of injecting domain knowledge into a general model without changing the structure of the original model.

[0129] Optionally, step e5 can also be performed to adjust the low-rank adaptation parameters. The original parameters of the diffusion model The domain semantic prompt words and the binary space attention mask The combined inputs are fed into the image generation module to generate a defect image.

[0130] Wherein, the combined input is represented as . This represents the original parameter matrix of the diffusion model; This represents the low-rank adaptation parameter matrix; The text embedding vector representing the semantic cue words of the domain; This represents the binary space attention mask; This indicates a combination operation of conditional inputs, which involves concatenating or aligning the low-rank adaptation parameters, the domain semantic cue words, and the binary space attention mask as conditional inputs to the diffusion model.

[0131] In this embodiment, by combining low-rank adaptation parameters, domain semantic cue words, and binary space attention mask as input, the diffusion model can simultaneously generate device structural features and defect texture features during image generation, and is subject to spatial constraints of defect locations, thus achieving dual controllable generation of content and location.

[0132] In an optional embodiment, the method further includes: Step f1: Train and save independent low-rank adaptation parameter files for different equipment types or different defect types.

[0133] Among them, the independent low-rank adaptation parameter file refers to a data file containing the low-rank adaptation parameters that is trained and saved for a specific device type or a specific defect type.

[0134] In this embodiment of the application, by saving independent low-rank adaptation parameter files for different devices or defect types, the knowledge of various tasks is encapsulated in independent pluggable modules, which facilitates subsequent on-demand loading and combination.

[0135] Step f2: When generating the image, load a low-rank adaptation parameter file corresponding to the target device type and the target defect type into the diffusion model.

[0136] In this embodiment, the user selects the corresponding low-rank adaptation parameter file to load into the basic diffusion model according to the actual generation needs, without having to switch or reload the entire model, thus realizing on-demand access to generation capabilities.

[0137] Step f3: The diffusion model generates an image corresponding to the device type or defect type based on the loaded low-rank adaptation parameter file.

[0138] In this embodiment, by loading different low-rank adaptation parameter files, the same basic diffusion model can generate images of different devices or different defect types, enabling rapid iteration and flexible deployment of functions.

[0139] This application sets a binary spatial attention mask in each residual layer of U-Net and continuously applies pixel-by-pixel spatial constraints in each step of inverse denoising, so that the defect features are continuously constrained within the device target area in all inverse denoising steps, thus avoiding the diffusion of defects into the background area.

[0140] To facilitate understanding of the solution, a specific embodiment of this application will be described using the generation of images of corrosion defects in grounding leads as an example. In this embodiment, a training phase is performed first.

[0141] The first step in the training phase is to build the training dataset.

[0142] A dataset containing 100 images of corrosion defects in grounding down conductors was obtained. Statistical analysis of the images in the dataset shows that the equipment in these images is elongated and has an average aspect ratio of [missing information]. Approximately 0.25 (i.e., the height-to-width ratio is approximately 1:4). Candidate training resolution set. Let the values ​​be (512×256), (768×256), (1024×256), (768×512), (512×512), and (768×768), where the first value in each tuple represents the height and the second value represents the width, in pixels. For elongated devices, to prioritize shape matching, set the shape matching weight coefficient α=0.6, the detail preservation weight coefficient β=0.25, and the resource consumption suppression weight coefficient γ=0.15.

[0143] Calculate the overall score for each candidate training resolution. For the candidate training resolution (1024×256), its aspect ratio is... ,and Consistent, therefore The maximum value is 1; the total number of pixels is 1024 × 256 = 262144. Assume available computing resource constraints. It is 24GB, which is 24×1024 3 byte, then Under these hardware conditions, this value is within the acceptable range. Calculations show that (1024×256) has the highest overall score and was selected as the optimal training resolution. All images in the dataset are uniformly adjusted to a size of (1024×256) pixels using bilinear interpolation to complete the construction of the training dataset.

[0144] The second step in the training phase is to construct domain semantic cues.

[0145] Generate device trigger words for grounding down conductor equipment For example, "jdyxx". Construct device visual feature description information. The description "a yellow-and-green painted metal strip on a concrete surface near grass, realistic photo" means: a yellow-and-green painted metal strip on a concrete surface near grass, in a realistic photo style. (This is part of a defect status description.) The phrase "medium coverage of rust" means moderate levels of rust. A complete domain semantic cue consists of the device trigger word TT and device visual feature description information. and defect status description information Assembled sequentially, for example: "jdyxx a yellow-and-green painted metal strip on a concrete surface near grass, realistic photo medium coverage of rust."

[0146] The third step in the training phase is to use a low-rank adaptation method for domain adaptation.

[0147] Low-rank matrices are injected into the attention layers of the pre-trained diffusion model's U-Net network, specifically into the query and key-value matrices of the cross-attention and self-attention layers. The original parameters of the diffusion model are then frozen. Using the constructed (1024×256) training dataset and domain semantic prompts, the parameters of the low-rank matrix are optimized to obtain the low-rank adaptation parameters. After training is completed, a low-rank adaptation parameter file containing the low-rank adaptation parameters is obtained.

[0148] After the training phase is completed, the generation phase is executed.

[0149] The first step in the generation phase is to generate a binary space attention mask.

[0150] Obtain a defect-free standard grounding lead image. Perform semantic segmentation on this image to extract the main metal strip region. Based on the physical laws governing corrosion defects on metal components, divide the metal strip region into a defect target generation region (pixel value assigned 1), and the background region into a defect constraint generation region (pixel value assigned 0). Adjust the size of the generated binary mask using bilinear interpolation to match the dimension of the latent feature map of the diffusion model, thus obtaining a binary space attention mask. .

[0151] The second step in the generation phase is to perform image generation under dual constraints.

[0152] Load the pre-trained diffusion model (original parameters) ) and a pre-trained low-rank adaptation parameter file, so that the effective model parameters are Input domain semantic cues and binary space attention mask .according to The diffusion model performs image generation under the dual constraints of domain semantic cue words and binary space attention mask.

[0153] The diffusion model initiates the inverse denoising process starting with the latent feature map of standard Gaussian noise. At each step of the inverse denoising process... ( From T down to 1), space constraint operations are performed. Among them, for Metal strip regions with a median value of 1, latent features Reserved; for The latent features of the background region with a median of 0 are reset to random noise ε. After T steps (e.g., T=50 steps) of reverse denoising iteration, a grounding down conductor defect image of size (1024×256 pixels) is generated. In this image, the grounding down conductor maintains its elongated strip structure, and the surface shows rust defects, which are located on the surface of the metal strip.

[0154] If the space constraint operation is cancelled during the generation process, that is, let If the latent feature map is not selectively preserved or reset, the diffusion model will diffuse the rust features to the background area at the edge of the lead-in line in the later stage of reverse denoising, resulting in defect overflow.

[0155] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0156] Corresponding to the substation defect image generation method described in the above embodiments, Figure 2The diagram shows a structural block diagram of a substation defect image generation system provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0157] Reference Figure 2 A substation defect image generation system, comprising: The acquisition module is used to acquire defect-free standard equipment images; An extraction module is used to perform semantic segmentation on the defect-free standard equipment image to obtain equipment contour information and key region information of the defect-free standard equipment image; wherein, the equipment contour information and the key region information constitute the equipment structural feature description of the defect-free standard equipment image; obtain a preset correspondence between equipment categories and defect types; according to the correspondence, divide the equipment structural region in the equipment structural feature description into a defect target generation region and a defect restriction generation region; generate a binary mask, assign a first value to the pixels in the defect target generation region, and assign a second value to the pixels in the defect restriction generation region; obtain the binary space attention mask based on the binary mask; The generation module is used to input pre-built domain semantic prompts and the binary space attention mask into the diffusion model to generate substation defect images.

[0158] It should be noted that the information interaction and execution process between the above modules or components are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0159] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0160] This application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor. When the processor executes the computer program, it implements the steps in any of the above method embodiments.

[0161] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0162] This application provides a computer program product that, when run on a computer device, enables the computer device to execute the steps described in the various method embodiments above.

[0163] Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 3 As shown, the computer device of this embodiment includes: at least one processor 30 ( Figure 3 Only one is shown in the diagram), memory 31, and computer program 32 stored in said memory 31 and executable on said at least one processor 30, wherein the processor 30 executes said computer program 32 to implement the steps in any of the above embodiments of the substation defect image generation method.

[0164] The computer device may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 3 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. They may include more or fewer components than shown in the illustration, or combinations of certain components, or different components. For example, they may also include input / output devices, network access devices, etc.

[0165] The processor 30 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0166] In some embodiments, the memory 31 may be an internal storage unit of the computer device, such as a hard drive or memory. In other embodiments, the memory 31 may be an external storage device of the computer device, such as a plug-in hard drive, smart media card (SMC), secure digital card (SD), flash card, etc. Furthermore, the memory 31 may include both internal and external storage units of the computer device. The memory 31 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0167] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or system capable of carrying computer program code to a system / computer device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0168] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0169] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0170] In the embodiments provided in this application, it should be understood that the disclosed systems / computer devices and methods can be implemented in other ways. For example, the system / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0171] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0172] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for generating images of substation defects, characterized in that, The method includes: Obtain defect-free standard equipment images; The method involves semantic segmentation of the defect-free standard equipment image to extract the main equipment region and generate a binary spatial attention mask to indicate the defect target generation region and the defect restriction generation region. This includes: performing semantic segmentation on the defect-free standard equipment image to obtain equipment contour information and key region information; wherein the equipment contour information and the key region information constitute a description of the equipment structure features of the defect-free standard equipment image; obtaining a preset correspondence between equipment categories and defect types; dividing the equipment structure region in the equipment structure feature description into a defect target generation region and a defect restriction generation region based on the correspondence; generating a binary mask, assigning a first value to pixels within the defect target generation region and a second value to pixels within the defect restriction generation region; and obtaining the binary spatial attention mask based on the binary mask. The pre-built domain semantic cue words and the binary space attention mask are input into the diffusion model to generate substation defect images.

2. The method according to claim 1, characterized in that, The step of inputting pre-constructed domain semantic cue words and the binary space attention mask into the diffusion model to generate substation defect images includes: The domain semantic cue words and the binary space attention mask are input into the diffusion model; A reverse denoising process is performed based on the diffusion model, and a latent feature map is generated during the reverse denoising process. The latent feature map is subjected to spatial constraints pixel by pixel based on the binary spatial attention mask to constrain the generation of defects within the defect target generation area, thereby generating the substation defect image.

3. The method according to claim 2, characterized in that, The method further includes: In the reverse denoising process, the device category and defect type of the generated image are constrained according to the domain semantic cue words.

4. The method according to claim 1, characterized in that, The domain semantic prompts include device trigger words, device visual feature description information, and defect status description information.

5. The method according to claim 1, characterized in that, The method further includes: Acquire a defect image dataset of substation equipment; the images in the defect image dataset include equipment morphological features and image detail information. Based on the device morphological characteristics, the image detail information, and the available computing resource constraints, the optimal training resolution is selected from the candidate training resolution set; The images in the defective image dataset are resized according to the optimal training resolution to construct a training dataset; the training dataset is used to train the diffusion model.

6. The method according to claim 5, characterized in that, The method further includes: A low-rank matrix is ​​injected into the attention layer of the U-Net network in the diffusion model; Freeze the original parameters of the diffusion model; The parameters of the low-rank matrix are optimized using the training dataset and the domain semantic prompts to obtain low-rank adaptation parameters; The low-rank adaptation parameters are loaded into the diffusion model and superimposed on the original parameters of the diffusion model.

7. A substation defect image generation system, characterized in that, include: The acquisition module is used to acquire defect-free standard equipment images; An extraction module is used to perform semantic segmentation on the defect-free standard equipment image to obtain equipment contour information and key region information of the defect-free standard equipment image; wherein, the equipment contour information and the key region information constitute the equipment structural feature description of the defect-free standard equipment image; obtain a preset correspondence between equipment categories and defect types; according to the correspondence, divide the equipment structural region in the equipment structural feature description into a defect target generation region and a defect restriction generation region; generate a binary mask, assign a first value to the pixels in the defect target generation region, and assign a second value to the pixels in the defect restriction generation region; obtain the binary space attention mask based on the binary mask; The generation module is used to input pre-built domain semantic prompts and the binary space attention mask into the diffusion model to generate substation defect images.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer program product, characterized in that, Includes a computer program that, when run, implements the method as described in any one of claims 1 to 6.

Images