Diversity and simulation degree controllable transformer substation defect image generation method

Abstract

The invention discloses a diversity and simulation degree controllable transformer substation defect image generation method, which comprises the following steps of: 1, constructing a defect sample set; a second step of extracting a defect sample; step 3, extracting features by a self-supervised learning encoder; 4, feature fusion is carried out; 5, representing the diffusion model to obtain new features; and 6, the image generation module outputs a generated image. According to the transformer substation defect image generation method with controllable diversity and simulation degree, vivid defects with diversified characteristics are generated based on existing defect samples, and expansion of a defect sample set is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of defect image production methods, and in particular to a method for generating defect images of a transformer substation with controllable diversity and realism. Background Art

[0002] The power system is a key infrastructure that affects the national economy, people's livelihood and national security. As the central link of "generation, transmission, transformation, distribution and use", the timely detection and treatment of substation defects is of great significance to maintaining the stable operation of the power grid. Affected by the natural environment, high-voltage loads and human factors, substations are prone to equipment defects such as foreign objects, breakage, oil leakage, and safety risks such as personnel smoking and not wearing safety helmets. For a long time, substation defects have relied on operation and maintenance personnel to discover through manual inspections. There are problems such as high labor intensity, low work efficiency, and many missed detections and false detections, which have brought hidden dangers to the safe operation of the power system. With the introduction of the concept of power vision and the development of the ubiquitous power Internet of Things, the automatic detection of substation defects based on computer vision technology and artificial intelligence technology has become a hot topic in research and application. That is, the surveillance cameras and inspection robots deployed in the substation are used to collect images of substation equipment, and then artificial intelligence algorithms are used to intelligently analyze and detect defects on the collected images. For example Figure 1 As shown, common substation defects include: broken meters, broken insulators, oil leakage, metal rust, broken respirators, abnormal door closure, oil stains on component surfaces, hanging suspended matter, bird nests, damaged doors, windows, walls and floors, broken covers, unlocked frame ladders, intrusion across the line, missing covers, not wearing safety helmets, not wearing work clothes, not wearing safety ropes, smoking, people falling to the ground, fire smoke, blurred dials, wall leakage, small animals intruding, abnormal meter readings, abnormal oil level of respirator oil seals, discoloration of silicone, water on the ground, roof leakage, etc.

[0003] In order to improve the generalization ability of the model, substation defect detection algorithms represented by deep learning require a large number of defect samples during model training. Due to the particularity of substation scenarios, the cost of collecting substation defects is high and there is a problem of scarcity of defect samples. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0005] To this end, the present invention proposes a method for generating defect images of substations with controllable diversity and realism, which generates realistic defects with diverse features based on existing defect samples to expand the defect sample set.

[0006] The method for generating defective substation images with controllable diversity and realism according to an embodiment of the present invention comprises the following steps: Step 1, constructing a defect sample set: annotating the acquired defective substation images, i.e. extracting defective area images and annotating their defect types to obtain a defect sample set; Step 2, extracting defective samples: randomly extracting defective samples from the defective sample set; Samples of the same type of defects ,in, is greater than or equal to A positive integer; Step 3, self-supervised learning encoder extracts features: each defect sample are all sent to the self-supervised learning encoder for encoding, and we get dimensional defect representation vector ;in, is greater than or equal to A positive integer, and ; Step 4, feature fusion: Defect representation vector Perform feature fusion to obtain a fusion representation with multiple defect features ; Step 5, the diffusion model is used to obtain new features: the fusion representation The representation diffusion model is sent to iterative denoising, fusion and denoising to obtain the denoised representation ; Step 6, the image generation module outputs the generated image: the denoised representation It is sent to the image generation module as a condition of the pixel generation model to guide the pixel generation of the defect image.

[0007] The beneficial effect of the present invention is that it can generate high-quality defect samples with certain color, texture and shape changes. Compared with conventional enhancement methods such as geometric transformation and color transformation, it has richer feature diversity, can effectively realize the expansion and enhancement of the sample set, and provide a rich training sample set for the deep learning model.

[0008] According to an embodiment of the present invention, in step 4, the specific process is: first, the defect representation Splice into one dimensional vector, concatenated dimensional vector as input and fed into the first fully connected layer And the second fully connected layer The network module is composed of Dimensional fusion features .

[0009] According to one embodiment of the present invention, the first fully connected layer The number of nodes is , the second fully connected layer The number of nodes is .

[0010] According to an embodiment of the present invention, in step 5, the representation diffusion model is implemented based on the RDM module.

[0011] According to one embodiment of the present invention, in the diffusion process of the representation diffusion model, the fusion representation Synchronous noise addition and random noise The intermediate results of iterative denoising are fused, that is, in the Step, the fusion expression Add noise to get , and then with the intermediate denoising result Perform weighted summation and update As the input of the next iterative denoising, is the realism control parameter, The preset Step noise level coefficient, The mean is , random Gaussian noise with variance equal to the identity matrix.

[0012] According to one embodiment of the present invention, the degree of realism is controlled by To achieve, when The smaller the value, the stronger the realism control.

[0013] According to one embodiment of the present invention, in step 6, the pixel generation model is implemented based on a mask generation encoder.

[0014] According to one embodiment of the present invention, the pixel generation model is based on a blank image and a feature representing the output of the diffusion model. As input, all tokens of the tokenizer are masked and the features representing the output of the diffusion model are As an additional category marker, it is combined with the ViT decoder to guide the iterative decoding.

[0015] According to one embodiment of the present invention, in each round of iteration, the pixel generation model first predicts the remaining mask marks, and calculates the number of mask marks that need to be replaced in this iteration according to the cosine function, and then performs mark sampling and mask mark replacement according to the predicted probability. After all mask marks are replaced, the iteration ends, and the pixel-level defect image is obtained after de-marking processing by the de-marker.

[0016] According to one embodiment of the present invention, the number of mask marks that need to be replaced in this iteration is calculated according to the cosine function, and the calculation formula is as follows:

[0017] in, The value is 20. yes The number of mask tokens to replace per iteration.

[0018] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description and the drawings.

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0021] Figure 1 This is a schematic diagram of common substation defects; Figure 2 It is a structural diagram of the substation defect image generation algorithm model; Figure 3 It is a schematic diagram of extracting defect area images from the original substation inspection image; Figure 4 It is a schematic diagram of the network structure of the feature fusion module; Figure 5 It is a schematic diagram showing the network structure of the diffusion model; Figure 6 This is a schematic diagram of the RDM module network structure; Figure 7 This is a schematic diagram of the substation defect image generation result; Figure 8 It is the overall flow chart of the substation defect image generation method. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] In recent years, generative artificial intelligence (AIGC) based on diffusion models has made breakthrough progress in generating high-quality images. It has surpassed traditional generative adversarial models in terms of generated diversity and realism, and provided new ideas for the acquisition of large-scale low-cost defect samples.

[0024] The following describes in detail a method for generating defect images of a substation with controllable diversity and realism according to an embodiment of the present invention with reference to the accompanying drawings.

[0025] The method for generating defect images of substations with controllable diversity and fidelity of the present invention comprises the following steps: Step 1, defect sample set construction: annotate the collected substation defect images, that is, extract defect area images and annotate their defect types to obtain a defect sample set; Step 2: Extract defect samples: Randomly extract Samples of the same type of defects ,in, is greater than or equal to A positive integer of ; Step 3: Extract features from the self-supervised learning encoder: are all sent to the self-supervised learning encoder for encoding, and we get dimensional defect representation vector ;in, is greater than or equal to A positive integer, and ; Step 4: Feature Fusion: Defect representation vector Perform feature fusion to obtain a fusion representation with multiple defect features ; In step 4, the specific process is: first, the defect is represented Splice into one dimensional vector, concatenated dimensional vector as input and fed into the first fully connected layer And the second fully connected layer The network module is composed of Dimensional fusion features The first fully connected layer The number of nodes is , the second fully connected layer The number of nodes is .

[0026] Step 5: Represent the diffusion model to obtain new features: fusion representation The representation diffusion model is sent to iterative denoising, fusion and denoising to obtain the denoised representation ; Step 6: Image generation module outputs the generated image: the denoised representation It is sent to the image generation module as a condition of the pixel generation model to guide the pixel generation of the defect image.

[0027] like Figure 2 As shown, the defect generation algorithm of the present invention consists of two parts: a defect representation generation module and an image generation module.

[0028] Among them, the defect representation generation module uses labeled defect samples to convert defects from image distribution to semantically rich representation distribution, thereby providing guidance conditions for the generation of defect images. At the same time, the conversion from pixel space to latent space can also improve the modeling efficiency of the diffusion model.

[0029] The specific process of the defect generation algorithm is: First, the acquired substation defect images are labeled, that is, defect area images are extracted and their defect types are labeled to obtain a defect sample set. Note that the defect area image here refers to the local image containing defects extracted from the original substation inspection image, such as Figure 3 As shown, therefore, the defect generation algorithm proposed in the present invention only generates a local defect image rather than a complete inspection image, because defects usually account for a small proportion in the inspection image, and the calculation cost of using the entire inspection image as the generation target is high, and the defect features are easily obliterated, resulting in the inability to generate defect samples.

[0030] 2. After completing the defect sample set construction, randomly select Defect samples of the same type ,like Figure 2 The suspended solids defect shown. For each defect sample (in, refers to Any one of the samples, ,and is a positive integer) are sent to the self-supervised learning encoder for encoding, and the dimensional defect representation vector Among them, the self-supervised learning encoder adopts the image encoder of Moco V3 (Paper: XinleiChen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers.In: Proceedings of the IEEE / CVF InternationalConference on Computer Vision, 9640-9649, 2021.), which performs model pre-training and fine-tuning on ImageNet and defect sample sets respectively by self-supervised contrastive learning.

[0031] 3. In order to improve the diversity of defect representation, the present invention proposes a feature fusion method to represent the defect Fusion is performed to obtain a fusion representation with multiple defect features It should be noted that the defect representation here refers to the defect characteristics. It is a defective sample The defect features are extracted. The feature fusion module is as follows Figure 4 As shown, first the defect is represented Splice into one dimensional vector (because each defect representation is dimensional vector), as input to a two-layer fully connected network, and we get Dimensional fusion features , that is, after splicing dimensional vector as input and sent to the network module consisting of two fully connected layers ( and ), and obtain the fusion feature In the two fully connected layers, The number of nodes in the layer (the first fully connected layer) is , The number of nodes in the layer (the second fully connected layer) is The diversity control of defect generation is achieved by the feature fusion module, which controls the number of defect representations involved in the fusion (i.e. The value of ), as well as the feature mapping realized by the two-layer fully connected network, can effectively control the diversity of the fused features (diversity means that the features of the defects are diverse. If the feature representation of the defects is diverse, the generated defect image will show as many changes in color, texture, shape, etc. as possible).

[0032] 4. Obtaining the Fusion Representation Afterwards, it is sent to the representation diffusion model for denoising to obtain the denoised representation ,like Figure 5 As shown. The representation diffusion model is implemented based on the RDM module (paper: Tianhong Li, Dina Katabi, Kaiming He. Self-conditioned Image Generation via Generating Representations. arXivpreprint arXiv:2312.03701, 2023. 2, 7). The structure of the RDM module is as follows Figure 6 As shown in the figure, each "Layer" layer consists of a LayerNorm layer, a SiLU layer and a linear layer. In the RDM module, the input noise representation is first feature mapped by a linear mapping layer composed of a fully connected layer, and then sent to 12 residual blocks for denoising, and finally outputs the denoised feature representation through a "Layer" layer. Figure 6 The time step embedding in the sentence refers to timestepembed, which converts the time step t into a vector, so that the network can predict the noise corresponding to the t step.

[0033] The original RDM module directly inputs the noise representation and performs denoising. In order to control the fidelity of the feature representation during the denoising process and avoid generation distortion (fidelity means that the generated feature representation and the input feature representation must have a certain degree of similarity, and there must not be serious feature deviation or erroneous feature representation), the RDM module is expanded here. Synchronous noise addition and random noise The intermediate results of iterative denoising are fused, that is, in the ( The value range is generally 0 to 1000) steps. Add noise to get , and then with the intermediate denoising result Perform weighted summation and update As the input of the next iterative denoising, is the realism control parameter, The preset Step noise level coefficient, The mean is , random Gaussian noise with variance equal to the identity matrix. That is, is the noise that satisfies the Gaussian distribution. The parameters of the Gaussian distribution are the mean and the variance. Here, the mean of the Gaussian distribution is , the variance is the identity matrix (that is, the diagonal is , others are The degree of realism is controlled by To achieve, when The smaller the value, the stronger the realism control.

[0034] 5. Get the denoised representation , and then the denoising expression The pixel generation model is fed into the image generation module as a conditional representation of the pixel generation model to guide the pixel generation of the defect image. The pixel generation model is implemented based on the Masked Generative Encoder (Paper: Tianhong Li, Huiwen Chang, Shlok Mishra, HanZhang,Dina Katabi, and Dilip Krishnan. Mage: Masked generative encoder to unify representation learning and image synthesis. In: Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2142-2152, 2023.), which is represented by a blank image and a conditional representation (representing the features output by the diffusion model). ) is input, all tokens of the tagger are masked, and the conditional representation (representing the features of the diffusion model output) is transformed into ) as an additional class token, combined with the ViT decoder, to guide iterative decoding. In each round of iteration, the model first predicts the remaining mask tokens, and calculates the number of mask tokens that need to be replaced in this iteration according to the cosine function (see Formula 1), and then performs token sampling and mask token replacement based on the predicted probability. After all mask tokens are replaced, the iteration ends, and the pixel-level defect image is obtained after detokenization by the detokenizer, as shown in Figure 7 As shown, examples of some defect images generated by the method of the present invention are shown. The first row is the generated images of suspended objects defects, the second row is the generated images of broken porcelain bottles defects, and the third row is the generated images of bird's nest defects. Obviously, the generated defect images have richer variations in color, texture, and shape, and retain the main features of the corresponding defect types.

[0035]

[0036] in, The value is 20. yes The number of mask tokens to replace per iteration.

[0037] The diversity and realism controllable substation defect image generation method of the present invention can generate high-quality defect samples with certain color, texture and shape changes. Compared with conventional enhancement methods such as geometric transformation and color transformation, it has richer feature diversity, can effectively realize the expansion and enhancement of the sample set, and provide a rich training sample set for the deep learning model.

[0038] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for generating defect images of substations with controllable diversity and fidelity, characterized in that: The following steps are involved: Step 1, defect sample set construction: annotate the collected substation defect images, that is, extract defect area images and annotate their defect types to obtain a defect sample set; Step 2: Extract defect samples: Randomly extract Samples of the same type of defects ,in, is greater than or equal to A positive integer of ; Step 3: Extract features from the self-supervised learning encoder: are all sent to the self-supervised learning encoder for encoding, and we get dimensional defect representation vector ;in, is greater than or equal to A positive integer, and ; Step 4: Feature Fusion: Defect representation vector Perform feature fusion to obtain a fusion representation with multiple defect features ; Step 5: Represent the diffusion model to obtain new features: fusion representation The representation diffusion model is sent to iterative denoising, fusion and denoising to obtain the denoised representation ; Step 6: Image generation module outputs the generated image: the denoised representation It is sent to the image generation module as a condition of the pixel generation model to guide the pixel generation of the defect image.

2. The method for generating substation defect images with controllable diversity and fidelity as claimed in claim 1, characterized in that: In step 4, the specific process is: first, the defect is represented Splice into one dimensional vector, concatenated dimensional vector as input and fed into the first fully connected layer And the second fully connected layer The network module is composed of Dimensional fusion features .

3. The method for generating defect images of substations with controllable diversity and fidelity as claimed in claim 2, characterized in that: The first fully connected layer The number of nodes is , the second fully connected layer The number of nodes is .

4. The method for generating defect images of substations with controllable diversity and fidelity as claimed in claim 1, characterized in that: In step 5, the representation diffusion model is implemented based on the RDM module.

5. The method for generating defect images of substations with controllable diversity and fidelity as claimed in claim 4, characterized in that: In the diffusion process of the representation diffusion model, the fusion representation Synchronous noise addition and random noise The intermediate results of iterative denoising are fused, that is, in the Step, the fusion expression Add noise to get , and then with the intermediate denoising result Perform weighted summation and update As the input of the next iterative denoising, is the realism control parameter, The preset Step noise level coefficient, The mean is , random Gaussian noise with variance equal to the identity matrix.

6. The method for generating defect images of substations with controllable diversity and fidelity as claimed in claim 5, characterized in that: The degree of realism is controlled by To achieve, when The smaller the value, the stronger the realism control.

7. The method for generating defect images of substations with controllable diversity and fidelity as claimed in claim 1, characterized in that: In step 6, the pixel generation model is implemented based on the mask generation encoder.

8. The method for generating defect images of substations with controllable diversity and fidelity as claimed in claim 7, characterized in that: The pixel generation model is based on a blank image and features representing the output of the diffusion model. As input, all tokens of the tokenizer are masked and the features representing the output of the diffusion model are As an additional category marker, it is combined with the ViT decoder to guide the iterative decoding.

9. The method for generating defect images of a substation with controllable diversity and fidelity as claimed in claim 8, characterized in that: In each round of iteration, the pixel generation model first predicts the remaining mask markers and calculates the number of mask markers that need to be replaced in this iteration according to the cosine function. Then, the marker sampling and mask marker replacement are performed according to the predicted probability. After all mask markers are replaced, the iteration ends and the pixel-level defect image is obtained after de-marking by the de-marker.

10. The method for generating defect images of a substation with controllable diversity and fidelity as claimed in claim 9, characterized in that: The number of mask markers that need to be replaced in this iteration is calculated according to the cosine function. The calculation formula is as follows: in, The value is 20. yes The number of mask tokens to replace per iteration.

Images