Power transmission line small sample abnormal data generation method and system based on diffusion model and feature migration

Abstract

The invention discloses a power transmission line small sample abnormal data generation method and system based on a diffusion model and feature migration, and belongs to the technical field of abnormal detection. The problem that stable and high-fidelity abnormal data can be generated only when the demand quantity of abnormal samples is much larger than that of actually obtained abnormal samples is solved. The method comprises the steps that an abnormal defect library and an equipment main body library are constructed based on defect reference images in a data set and corresponding labels, and the data set at least comprises one defect reference image of each type; training an abnormal defect identification baseline model based on the data set and verifying to obtain a low-quality image; comparing the abnormal defect library and the equipment main body library with low-quality images by using an image comparison technology to obtain an abnormal defect reference image and a base image to be generated; generating a to-be-migrated region mask based on the to-be-generated base map; and based on the to-be-generated base map and the to-be-migrated region mask, using a diffusion model to migrate the abnormal region features of the low-quality image to the to-be-generated base map for anomaly generation. The method is used for generating the small sample abnormal data of the power transmission line.

Description

Technical Field

[0001] A method and system for generating small sample anomaly data of transmission lines based on diffusion model and feature migration are used for generating small sample anomaly data of transmission lines, belonging to the technical fields of image generation, diffusion model, style migration, image comparison, power line monitoring, data enhancement, and anomaly detection. Background Art

[0002] Diffusion models generate data through a step-by-step denoising process and have been widely used in the field of image generation (such as Stable Diffusion) to generate high-fidelity images.

[0003] Image feature transfer technology achieves local feature transfer across images through semantic matching (such as CLIP) and style transfer (such as IP-Adapter).

[0004] Existing technologies rely on traditional image processing (such as edge detection) or supervised learning to detect anomalies in power equipment, which requires a large amount of labeled data.

[0005] As a core technology of generative artificial intelligence, diffusion models are widely used in image synthesis, data enhancement, cross-modal generation, and other fields. In power transmission line monitoring scenarios, due to the low probability of equipment anomalies (such as wire damage) and the difficulty in obtaining samples, traditional supervised learning models face a serious shortage of training data. In this case, it is urgent to expand the anomaly dataset through small sample generation technology so that the anomaly detection model can cover a more comprehensive range of defect types. Existing small sample anomaly data generation methods mainly include the following categories:

[0006] GAN-based small sample anomaly generation for transmission lines:

[0007] A GAN generator is trained using a small number of transmission line anomaly samples (e.g., 5-10 images of rust). Through adversarial training, images of wires with surface damage are generated. The generator learns the distribution of rust textures, while the discriminator distinguishes between real anomalies and generated samples. The following technical issues exist:

[0008] Severe pattern rigidity: GAN generates wire cracks in a single direction (such as only horizontal cracking) with a very small number of samples, which cannot simulate the multi-angle crack propagation characteristics in real scenarios.

[0009] Obvious texture distortion: Rust particles are blurred, and high-resolution wire material details (such as the microstructure of the aluminum alloy oxide layer) cannot be preserved.

[0010] Traditional style transfer methods (such as StyleTransfer):

[0011] The global style features of the reference image (such as rust color and background lighting) are transferred to the normal wire image, and texture fusion is achieved by optimizing the content-style loss function. The following technical problems exist:

[0012] The abnormality type is uncontrollable: only surface styles such as color and texture can be transferred, and specific forms of physical damage (such as the evolution of pitting rust into linear cracks) cannot be generated.

[0013] Risk of structural damage: Style transfer may change the geometric structure of the device, causing the generated image to fail to comply with safety regulations for power equipment.

[0014] Significant background interference: More than 30% of images will mistakenly transfer background anomalies to the wire surface, causing anomaly positioning errors.

[0015] Unsupervised Anomaly Generation:

[0016] GAN is used to extract unsupervised features of power transmission lines and generate abnormal areas. The following technical problems exist:

[0017] The generated anomalies lack physical plausibility (e.g., the rusted area does not conform to the material characteristics of the wire).

[0018] Anomaly generation based on LoRA fine-tuned diffusion model:

[0019] First, the Low Rank Adapter (LoRA) is fine-tuned for Stable Diffusion using 10-50 wire damage samples. Then, text prompts (such as "rusted transmission line") are used to control the generated defect type. Finally, an abnormal image that matches the text description is output. However, there are technical challenges: The sample requirement is large: at least 20 samples of the same damage type (e.g., different stages of corrosion) are required for stable fine-tuning, while in actual operations and maintenance, only 1-3 samples of the same defect type are typically available.

[0020] Strong semantic dependency: The generated abnormal morphology is limited by the generalization ability of the text prompts (e.g., the text cannot accurately describe the distribution details of rust particles).

[0021] The small sample image classification method based on the feature diffusion model disclosed in CN116863239A introduces the technology of feature migration and diffusion model, takes the deep convolution features of the sample as conditions, and denoises the deep features on this basis, so that the denoised features have a high similarity with the features of samples of the same category. A better feature extractor is learned under the base class data by using feature diffusion, extracts highly recognizable visual features, and generates more discriminative features. At the same time, compared with the original diffusion model algorithm, the amount of data required is small, easy to implement, and saves a lot of time and computing resource costs. In the test phase, the cosine distance is used to measure the cosine similarity between the query sample features and the denoised features, and then the query samples of the new class are classified, thereby improving the performance of small sample image classification. It is to optimize the classification performance by feature denoising, focus on small sample image classification, improve feature discriminability by feature denoising, rely on base class data to train the feature extractor, do not involve the single sample generation capability, and the generated denoised features are only used for classification tasks, and the adaptability of downstream tasks (such as classification, detection, and segmentation) is not verified.

[0022] In summary, the existing methods for generating small sample abnormal data have the following technical problems:

[0023] 1. In transmission line anomaly detection, the number of abnormal samples required is much greater than the actual number of abnormal samples obtained in order to generate stable and high-fidelity abnormal data.

[0024] 2. The geometric structure of the device (i.e., the guide wire structure) may be destroyed to generate heterogeneous data that is similar to the reference sample and has diversity, which may easily lead to poor results or generate a single abnormal data. Summary of the Invention

[0025] In response to the above research problems, the purpose of the present invention is to provide a method and system for generating small sample abnormal data of transmission lines based on diffusion model and feature migration, so as to solve the problem in the existing technology that the amount of abnormal samples required in transmission line anomaly detection is much greater than the actual number of abnormal samples obtained in order to generate stable and high-fidelity abnormal data.

[0026] In order to achieve the above object, the present invention adopts the following technical solutions:

[0027] A method for generating small sample abnormal data of transmission lines based on diffusion model and feature migration includes the following steps:

[0028] Step 1: Based on the defect reference images and corresponding labels in the dataset, build an abnormal defect library and a device main body library, where the dataset includes at least one defect reference image of each type;

[0029] Step 2: Train an abnormal defect recognition baseline model based on the dataset and verify the low-quality images obtained:

[0030] Step 3: For the abnormal defect library and the equipment main body library, use image comparison technology to compare with the low-quality image to obtain the abnormal defect reference image and the base image to be generated;

[0031] Step 4: Generate a mask of the area to be migrated based on the base map to be generated;

[0032] Step 5: Based on the base map to be generated and the mask of the area to be migrated, the diffusion model is used to migrate the abnormal area features of the low-quality image to the base map to be generated for anomaly generation.

[0033] Furthermore, the specific steps of step 1 are:

[0034] Step 1.1. Obtain a dataset, including a defect reference image and corresponding labels. The labels include a defect region label and a device body label, which respectively represent the bounding boxes and corresponding categories of the defect region and device body in the defect reference image framed in the detection task. The bounding box includes the x, y pixel coordinates of the upper left corner, upper right corner, lower left corner, and lower right corner, or represent the polygonal outline of the defect region and device body in the defect reference image framed in the image segmentation task. The polygonal outline includes the x, y pixel coordinates and corresponding category of each pixel. The defect region is the damaged area of ​​the wire, including rust and cracks, and the device body is the wire body area, i.e., the wire body.

[0035] Step 1.2: Based on the defect area labels in each defect reference image in the data set and the coordinates of the defect area and the coordinates of the device body area corresponding to the categories in the device body labels, image processing tools are used to deduct the defect images and device body images to obtain the defect library and device body library, wherein the image processing tools include OpenCV or Pillow.

[0036] Furthermore, the specific steps of step 2 are:

[0037] Step 2.1: Train an abnormal defect recognition baseline model based on the dataset and verify it to obtain a verification label. The abnormal defect recognition baseline model is an object detection model or an image segmentation model, specifically a YOL0v8 model.

[0038] Step 2.2: Under the detection task, the intersection-of-union ratio of the verification label and the true label is calculated. If the intersection-of-union ratio is lower than a given threshold, the defect reference image corresponding to the verification label is regarded as a low-quality image;

[0039] The calculation formula for the intersection-over-union ratio IoU is:

[0040]

[0041] A 预测框=(x2-x1)(y2-y1)

[0042]

[0043] A 并集 =A 预测框 ∩A GT框 =max(0, x 右 -x 左 )×max(0,y 下 -y 上 )

[0044]

[0045] A 并集 =A 预测框 +A GT框 -A 交集

[0046] Where A 预测框 represents the bounding box in the verification label predicted by the abnormal defect recognition baseline model under the detection task, (x1, y1) represents the coordinates of the upper left corner of the bounding box in the verification label, (x2, y2) represents the coordinates of the lower right corner of the bounding box in the verification label, and A GT框 represents the area of ​​the bounding box in the ground-truth label of the defect reference image predicted by the abnormal defect recognition baseline model, represents the coordinate of the upper left corner of the bounding box in the true label, Represents the lower right corner coordinate of the bounding box in the true label, A 预测框 ∩A GT框 represents the intersection area of ​​the bounding box in the verification label and the boundary in the true label, A 预测框 ∪A GT框 represents the union area of ​​the bounding box in the verification label and the bounding box in the true label;

[0047] In the image segmentation task, the intersection-of-union (IoU) ratio of pixels of each category in the bounding box of the verification label is calculated with the pixels of each category in the true label. If any IoU ratio is lower than a given threshold, the defective reference image corresponding to the verification label is regarded as a low-quality image.

[0048] Intersection over Union (IoU) j The calculation formula is:

[0049]

[0050] Where j represents the jth category in the verification label predicted by the abnormal defect recognition baseline model under the image segmentation task, j∈{1, 2, ...J}, TP j Indicates the number of pixels whose predicted pixel category and the actual category are both j, FPj FN represents the number of pixels whose predicted category is different from the true category. j Indicates that the predicted pixel category is not j, but the actual category is j pixels.

[0051] Furthermore, the specific steps of step 3 are:

[0052] Step 3.1: For the defect areas in the defect library and the low-quality image, obtain the top-K defect areas similar to the defect areas in the low-quality image based on the similarity search method, and use them as the abnormal defect reference image;

[0053] Step 3.2: For the device body library and the device body area of ​​the low-quality image, the top-K device bodies similar to the device body area of ​​the low-quality image are obtained according to the similarity search method as the base map to be generated.

[0054] Furthermore, the similarity search method in step 3.1 and step 3.2 is a similarity search method based on deep learning, and the specific steps are:

[0055] Use a pre-trained model to convert images and low-quality images in the defect library or the device library into high-dimensional feature vectors. The pre-trained model is a ResNet or CLIP image encoder. ResNet extracts spatial features of images through a residual network, which are high-dimensional feature vectors. The CLIP image encoder is based on a multimodal model of contrastive learning and directly extracts global semantic features from images, which are high-dimensional feature vectors. High-dimensional feature vectors have dimensions of 512 or more.

[0056] Perform L2 normalization on high-dimensional feature vectors;

[0057] The cosine similarity is used to measure the directional consistency of the high-dimensional feature vectors of the images in the defect library and the low-quality images or the images in the device body library and the low-quality images. The formula is:

[0058]

[0059] Among them, I1 represents the image in the defect library or the image in the device body library, I2 represents the low-quality image, and f image (I1) and f image (I2) represents the high-dimensional feature vector after the transformation of I1 or I2, "·" represents the vector dot product operation, ||f image (I1)|| and||f image (I2)|| respectively represent the L2 norm of the high-dimensional feature vector, and the calculation formula is f kIt represents the kth element in the high-dimensional feature vector, reflecting the strength or relevance of the image on a certain abstract feature. Abstract features include texture, shape or semantic category. ∑k represents the sum of all high-dimensional feature dimensions.

[0060] Furthermore, the specific steps of step 4 are:

[0061] Based on the base map to be generated, the repair area that needs to be modified is marked in a binary mask map of the same size as the base map to be generated of the defect, that is, a mask of the area to be migrated is generated, wherein the pixels of the repair area marked to be modified in the binary mask map are represented by 1, which is white, and the pixels of the retained area are represented by 0, which is black. The marking methods include drawing with a brush in a drawing software, marking with a labeling software, or drawing with an algorithm. The drawing software includes Photoshop, the labeling software includes labelme, and the algorithmic drawing includes the Monte Carlo rejection sampling method, the polar angle sorting algorithm, the convex hull algorithm, or the python tool Shapely.

[0062] Furthermore, the specific steps of step 5 are:

[0063] Step 5.1: The abnormal defect reference image corresponding to the repair area is passed into the clip visual encoder to extract the multi-scale feature vector of each abnormal defect reference image, including texture and color;

[0064] Step 5.2: Add noise to the repaired area based on the base image to be generated and the mask noise formula to obtain the initial noise image , the mask noise formula is:

[0065]

[0066] Where, represents the noisy image of the repaired area at time step t, represents the noise scheduling parameter, that is, the degree of noise cumulative attenuation, α i The noise attenuation coefficient controls the retention ratio of the original image during the noise addition process at the i-th time step, which is predefined by the basic diffusion model, i∈{1,2,...,T}, and the total time step is It is the predefined basic diffusion model, M represents the mask matrix, which is transformed from the binary mask map. represents the base map to be generated, ⊙ represents element-by-element multiplication, and ∈ represents standard Gaussian noise, which simulates random noise in the diffusion process and drives the generation of diversity;

[0067] Step 5.3: Based on IP-Adapter, the multi-scale feature vector is integrated into the basic diffusion model to analyze the initial noise image. The repaired area in the image is subjected to reverse diffusion denoising to generate a defect area that matches the features of the abnormal defect reference image, and an abnormal sample is generated by combining the non-defective area. The basic diffusion model is SD1.5, SDXL or Flux, and the reverse diffusion formula is:

[0068]

[0069] In scenarios where the original content of non-repaired areas needs to be preserved:

[0070]

[0071] In the scenario of completely regenerating the image:

[0072]

[0073] Where, X t Represents the global latent variable, which is the noise state of the binary mask at time step t, ∈ θ represents the noise predicted by the U-Net network, β t represents the noise attenuation coefficient predefined at time step t, α t =1-β t , c represents the multi-scale feature vector, z represents the standard normal distribution noise, σ t Represents the noise scaling factor, which controls the intensity of random noise and is usually Indicates the use of binary mask noise, Indicates that the global noise of the original image is retained outside the mask of the area to be migrated Indicates that the basemap will be repaired By adding noise to the time step t through the forward noise addition process of the diffusion model, the global noise is obtained. Indicates that standard Gaussian noise is retained outside the mask of the area to be migrated represents the initialized standard Gaussian noise, x t-1 represents the generated abnormal samples;

[0074] Step 5.4: If the abnormal sample meets the given requirements, the generation ends; otherwise, the execution is as follows:

[0075] If the similarity between the generated abnormal sample and the abnormal defect reference image is less than the given threshold, adjust the value of the IP-Adapter core parameter scale in step 5.3 and execute step 5.3 again; or go to step 5.1 and continue;

[0076] If the repair area is lower than the resolution of 50*50 or the number of generated images is insufficient, go to step 4 to redraw the binary mask and continue.

[0077] If the number of generated images is insufficient, re-acquire the base image to be generated, go to step 4 to redraw the binary mask image, go to step 5.3 to reselect the diffusion model random noise, or go to 5.3 to select a different basic diffusion model and continue.

[0078] A transmission line small sample abnormal data generation system based on diffusion model and feature migration, comprising:

[0079] Data acquisition module: Build an abnormal defect library and a device main body library based on the defect reference images and corresponding labels in the data set, where the data set includes at least one defect reference image of each type;

[0080] Low-quality image acquisition module: trains an abnormal defect recognition baseline model based on the dataset and verifies the low-quality images;

[0081] Comparison module: For the abnormal defect library and the equipment main body library, use image comparison technology to compare with low-quality images to obtain the abnormal defect reference image and the base image to be generated:

[0082] Mask module: Generates the mask of the area to be migrated based on the base map to be generated;

[0083] Abnormal sample generation module: Based on the base map to be generated and the mask of the area to be migrated, the diffusion model is used to migrate the abnormal area features of the low-quality image to the base map to be generated for anomaly generation.

[0084] Compared with the prior art, the present invention has the following beneficial effects:

[0085] The present invention can significantly reduce the dependence on the number of acquired abnormal samples, solve the technical problem of abnormal generation with only a single sample as a reference, and generate diverse abnormal samples similar to the reference sample without destroying the geometric structure of the device. Specifically, it is embodied in the following aspects:

[0086] 1. Breaking through the bottleneck of feature expression under small sample conditions

[0087] This paper constructs an independent device library and abnormal defect library, and adopts CLIP-based multi-scale feature extraction and IP-Adapter fine-grained feature injection technology. When only 1-3 reference images of defects of the same type are required, the method can realize the generation of abnormal attribute combinations through cross-sample local feature migration. For example, the rust particle texture of sample A is fused with the surface texture trend of sample B to finally generate an abnormal sample. By leveraging the strong generalization ability of the pre-trained CLIP model image encoder, the method decouples multi-dimensional features such as material, texture, and morphology from a single defect sample, overcoming the dependence of LoRA fine-tuning on the number of samples of the same type.

[0088] 2. Ensure the physical rationality of the device geometry (wire body structure)

[0089] By dually controlling the inpainting area constraints and the base map to be generated, during the feature migration phase, the IP-Adapter's multi-scale feature vectors are integrated into a basic diffusion model to perform back-diffusion denoising on the inpainted area in the initial noisy image. This strictly limits the multi-scale feature vectors to the device segmentation mask area, for example, only acting on the longitudinal region between 20% and 90% of the wire diameter. Similarity retrieval in the device database ensures that the base map to be generated has the same geometric topology as the target device body, such as being generated on wires of similar types.

[0090] 3. Achieving the Collaborative Generation of High-Fidelity Textures and Controllable Abnormal Morphologies

[0091] Using a diffusion model-based local remapping technique, during the denoising process, the cross-attention mechanism dynamically fuses normal wire structure features (from the device database) with abnormal texture features (from the defect database). Leveraging the multi-level feature injection of the IP-Adapter, this method preserves material details (such as the reflective properties of the aluminum alloy oxide layer) in the shallow network layer while controlling the macroscopic morphology of damage in the deep network layer.

[0092] 4. Eliminate background interference and abnormal positioning deviation

[0093] Through the physical separation mechanism of the abnormal defect library and the equipment main body library, in the feature retrieval stage, independent similarity calculations are performed on the defect area (containing only rust / cracks) and the equipment main body (excluding background). Dual-path feature input is used in the generation process: the defect reference image only provides abnormal features, and the equipment main body image only provides structural features. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 It is a schematic diagram of the process of the present invention;

[0095] Figure 2 Schematic diagram of the process of obtaining low-quality images in the present invention;

[0096] Figure 3 Schematic diagram of the process of generating abnormal samples in the present invention. DETAILED DESCRIPTION

[0097] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0098] A method for generating small sample abnormal data of transmission lines based on diffusion model and feature migration includes the following steps:

[0099] Step 1: Based on the defect reference images and corresponding labels in the dataset, build an abnormal defect library and a device main body library, where the dataset includes at least one defect reference image of each type;

[0100] The specific steps are:

[0101] Step 1.1. Obtain a dataset, including a defect reference image and corresponding labels. The labels include a defect region label and a device body label, which respectively represent the bounding boxes and corresponding categories of the defect region and device body in the defect reference image framed in the detection task. The bounding box includes the x, y pixel coordinates of the upper left corner, upper right corner, lower left corner, and lower right corner, or represent the polygonal outline of the defect region and device body in the defect reference image framed in the image segmentation task. The polygonal outline includes the x, y pixel coordinates and corresponding category of each pixel. The defect region is the damaged area of ​​the wire, including rust and cracks, and the device body is the wire body area, i.e., the wire body.

[0102] Step 1.2: Based on the defect area labels in each defect reference image in the data set and the coordinates of the defect area and the coordinates of the device body area corresponding to the categories in the device body labels, image processing tools are used to deduct the defect images and device body images to obtain the defect library and device body library, wherein the image processing tools include opencv or PiIow.

[0103] Step 2: Train an abnormal defect recognition baseline model based on the dataset and verify the low-quality images;

[0104] The specific steps are:

[0105] Step 2.1: Train an abnormal defect recognition baseline model based on the dataset and verify it to obtain a verification label. The abnormal defect recognition baseline model is an object detection model or an image segmentation model, specifically a Y0L0v8 model.

[0106] Step 2.2: Under the detection task, the intersection-of-union ratio of the verification label and the true label is calculated. If the intersection-of-union ratio is lower than a given threshold, the defect reference image corresponding to the verification label is regarded as a low-quality image;

[0107] The calculation formula for the intersection-over-union ratio IoU is:

[0108]

[0109] A 预测框 =(x2-x1)(y2-y1)

[0110]

[0111] A 并集 =A 预测框 ∩A GT框 =max(0, x 右 -x 左 )×max(0,y 下 -y上 )

[0112]

[0113] A 并集 =A 预测框 +A GT框 -A 交集

[0114] Where A 预测框 represents the bounding box in the verification label predicted by the abnormal defect recognition baseline model under the detection task, (x1, y1) represents the coordinates of the upper left corner of the bounding box in the verification label, (x2, y2) represents the coordinates of the lower right corner of the bounding box in the verification label, and A GT框 represents the area of ​​the bounding box in the ground-truth label of the defect reference image predicted by the abnormal defect recognition baseline model, represents the coordinate of the upper left corner of the bounding box in the true label, Represents the lower right corner coordinate of the bounding box in the true label, A 预测框 ∩A GT框 represents the intersection area of ​​the bounding box in the verification label and the boundary in the true label, A 预测框 ∪A GT框 represents the union area of ​​the bounding box in the verification label and the bounding box in the true label;

[0115] In the image segmentation task, the intersection-of-union (IoU) ratio of pixels of each category in the bounding box of the verification label is calculated with the pixels of each category in the true label. If any IoU ratio is lower than a given threshold, the defective reference image corresponding to the verification label is regarded as a low-quality image.

[0116] Intersection over Union (IoU) j The calculation formula is:

[0117]

[0118] Where j represents the jth category in the verification label predicted by the abnormal defect recognition baseline model under the image segmentation task, j∈{1, 2, ...J}, TP j Indicates the number of pixels whose predicted pixel category and the actual category are both j, FP j FN represents the number of pixels whose predicted category is different from the true category. j Indicates that the predicted pixel category is not j, but the actual category is j pixels.

[0119] Example: If the TP of defect category j j =800, FP j =200, FN j =400, then:

[0120]

[0121] If a certain category IoU j If the value is less than the threshold (such as 0.5), the segmentation of the category is determined to have failed, and the defective reference image corresponding to the verification label is regarded as a low-quality image.

[0122] Step 3: For the abnormal defect library and the equipment main body library, use image comparison technology to compare with the low-quality image to obtain the abnormal defect reference image and the base image to be generated;

[0123] The specific steps are:

[0124] Step 3.1: For the defect areas in the defect library and the low-quality image, obtain the top-K defect areas similar to the defect areas in the low-quality image based on the similarity search method, and use them as the abnormal defect reference image;

[0125] Step 3.2: For the device body library and the device body area of ​​the low-quality image, the top-K device bodies similar to the device body area of ​​the low-quality image are obtained according to the similarity search method as the base map to be generated.

[0126] The similarity search method in step 3.1 and step 3.2 is a similarity search method based on deep learning, and the specific steps are:

[0127] Use a pre-trained model to convert images and low-quality images in the defect library or the device library into high-dimensional feature vectors. The pre-trained model is a ResNet or CLIP image encoder. ResNet extracts spatial features of images through a residual network, which are high-dimensional feature vectors. The CLIP image encoder is based on a multimodal model of contrastive learning and directly extracts global semantic features from images, which are high-dimensional feature vectors. High-dimensional feature vectors have dimensions of 512 or more.

[0128] Perform L2 normalization on high-dimensional feature vectors;

[0129] The cosine similarity is used to measure the directional consistency of the high-dimensional feature vectors of the images in the defect library and the low-quality images or the images in the device body library and the low-quality images. The formula is:

[0130]

[0131] Among them, I1 represents the image in the defect library or the image in the device body library, I2 represents the low-quality image, and f image (I1) and f image (I2) represents the high-dimensional feature vector after the transformation of I1 or I2, "·" represents the vector dot product operation, ||f image(I1)|| and||f image (I2)|| respectively represent the L2 norm of the high-dimensional feature vector, and the calculation formula is f k It represents the kth element in the high-dimensional feature vector, reflecting the strength or relevance of the image on a certain abstract feature. Abstract features include texture, shape or semantic category. ∑k represents the sum of all high-dimensional feature dimensions.

[0132] Step 4: Generate a mask of the area to be migrated based on the base map to be generated;

[0133] The specific steps are:

[0134] Based on the base map to be generated, the repair area that needs to be modified is marked in a binary mask map of the same size as the base map to be generated of the defect, that is, a mask of the area to be migrated is generated, wherein the pixels of the repair area marked to be modified in the binary mask map are represented by 1, which is white, and the pixels of the retained area are represented by 0, which is black. The marking methods include drawing with a brush in a drawing software, marking with a labeling software, or drawing with an algorithm. The drawing software includes Photoshop, the labeling software includes labelme, and the algorithmic drawing includes the Monte Carlo rejection sampling method, the polar angle sorting algorithm, the convex hull algorithm, or the python tool Shapely.

[0135] Step 5: Based on the base map to be generated and the mask of the area to be migrated, the diffusion model is used to migrate the abnormal area features of the low-quality image to the base map to be generated for anomaly generation.

[0136] The specific steps are:

[0137] Step 5.1: The abnormal defect reference image corresponding to the repair area is passed into the clip visual encoder to extract the multi-scale feature vector of each abnormal defect reference image, including texture and color;

[0138] Step 5.2: Add noise to the repaired area based on the base image to be generated and the mask noise formula to obtain the initial noise image , the mask noise formula is:

[0139]

[0140] Where, represents the noisy image of the repaired area at time step t, represents the noise scheduling parameter, that is, the degree of noise cumulative attenuation, α i The noise attenuation coefficient controls the retention ratio of the original image during the noise addition process at the i-th time step, which is predefined by the basic diffusion model, i∈{1,2,...,T}, and the total time step is It is the predefined basic diffusion model, M represents the mask matrix, which is transformed from the binary mask map. represents the base map to be generated, ⊙ represents element-by-element multiplication, and ∈ represents standard Gaussian noise, which simulates random noise in the diffusion process and drives the generation of diversity;

[0141] Step 5.3: Based on IP-Adapter, the multi-scale feature vector is integrated into the basic diffusion model to analyze the initial noise image. The repaired area in the image is subjected to reverse diffusion denoising to generate a defect area that matches the features of the abnormal defect reference image, and an abnormal sample is generated by combining the non-defective area. The basic diffusion model is SD1.5, SDXL or Flux, and the reverse diffusion formula is:

[0142]

[0143] In scenarios where the original content of non-repaired areas needs to be preserved (for example, repairing local defects while preserving the background):

[0144]

[0145] In the case of completely regenerating the image (e.g. generating a completely new defect):

[0146]

[0147] Where, X t Represents the global latent variable, which is the noise state of the binary mask at time step t, ∈ θ represents the noise predicted by the U-Net network, β t represents the noise attenuation coefficient predefined at time step t, α t =1-β t , c represents the multi-scale feature vector, z represents the standard normal distribution noise, σ t Represents the noise scaling factor, which controls the intensity of random noise and is usually Indicates the use of binary mask The noise, that is, in the scenario where the original content of the non-repaired area needs to be preserved, represents the specific noise injected into the repaired area, and in the scenario where the image is completely regenerated, represents the guidance to generate specific defects. Indicates that the global noise of the original image is retained outside the mask of the area to be migrated Indicates that the basemap will be repaired By adding noise to the time step t through the forward noise addition process of the diffusion model, the global noise is obtained. Indicates that standard Gaussian noise is retained outside the mask of the area to be migrated represents the initialized standard Gaussian noise, x t-1represents the generated abnormal samples;

[0148] Step 5.4: If the abnormal sample meets the given requirements, the generation ends; otherwise, the execution is as follows:

[0149] If the similarity between the generated abnormal sample and the abnormal defect reference image is less than a given threshold, such as 0.7, adjust the value of the IP-Adapter core parameter scale in step 5.3 and execute step 5.3 again; or go to step 5.1 and continue.

[0150] If the repair area is lower than the resolution of 50*50 or the number of generated images is insufficient, go to step 4 to redraw the binary mask and continue.

[0151] If the number of generated images is insufficient, reacquire the base image to be generated, proceed to step 4 to redraw the binary mask image, proceed to step 5.3 to reselect the diffusion model random noise (adjust the number of noise steps of the diffusion model (e.g., 20-50 steps), and generate wire damage defect images with different degrees of corrosion in the mask area of ​​the same base image to be generated), or proceed to step 5.3 to select a different basic diffusion model, such as SDXL or FLUX, and continue.

[0152] The present invention can be applied to other power equipment and even to abnormal generation in other scenarios (such as insulator damage), simply by selecting different reference images and adjusting the mask generation logic.

[0153] The above are only representative embodiments of the present invention in many specific application scopes and do not constitute any limitation on the protection scope of the present invention. Any technical solutions formed by transformation or equivalent replacement fall within the scope of protection of the present invention.

Claims

1. A method for generating small sample abnormal data of transmission lines based on diffusion model and feature migration, characterized in that: The steps include: Step 1: Based on the defect reference images and corresponding labels in the dataset, build an abnormal defect library and a device main body library, where the dataset includes at least one defect reference image of each type; Step 2: Train an abnormal defect recognition baseline model based on the dataset and verify the low-quality images; Step 3: For the abnormal defect library and the equipment main body library, use image comparison technology to compare with the low-quality image to obtain the abnormal defect reference image and the base image to be generated; Step 4: Generate a mask of the area to be migrated based on the base map to be generated; Step 5: Based on the base map to be generated and the mask of the area to be migrated, the diffusion model is used to migrate the abnormal area features of the low-quality image to the base map to be generated for anomaly generation.

2. The method for generating small sample abnormal data of a power transmission line based on a diffusion model and feature migration according to claim 1, characterized in that: The specific steps of step 1 are: Step 1.

1. Obtain a dataset, including a defect reference image and corresponding labels. The labels include a defect region label and a device body label, which respectively represent the bounding boxes and corresponding categories of the defect region and device body in the defect reference image framed in the detection task. The bounding box includes the x, y pixel coordinates of the upper left corner, upper right corner, lower left corner, and lower right corner, or represent the polygonal outline of the defect region and device body in the defect reference image framed in the image segmentation task. The polygonal outline includes the x, y pixel coordinates and corresponding category of each pixel. The defect region is the damaged area of ​​the wire, including rust and cracks, and the device body is the wire body area, i.e., the wire body. Step 1.2: Based on the defect area labels in each defect reference image in the data set and the coordinates of the defect area and the coordinates of the device body area corresponding to the categories in the device body labels, image processing tools are used to deduct the defect images and device body images to obtain the defect library and device body library, wherein the image processing tools include OpenCV or Pillow.

3. The method for generating small sample abnormal data of a power transmission line based on a diffusion model and feature migration according to claim 1, characterized in that: The specific steps of step 2 are: Step 2.1: Train a baseline model for abnormal defect recognition based on the dataset and verify it to obtain a verification label. The baseline model for abnormal defect recognition is an object detection model or an image segmentation model, specifically a YOLOv8 model. Step 2.2: Under the detection task, the intersection-of-union ratio of the verification label and the true label is calculated. If the intersection-of-union ratio is lower than a given threshold, the defect reference image corresponding to the verification label is regarded as a low-quality image; The calculation formula for the intersection-over-union ratio IoU is: A 预测框 =(x2-x1)(y2-y1) TO 并集 =A 预测框 ∩A GT框 =max(0,x 右 -x 左 )×max(0,y 下 -and 上 ) A 并集 =A 预测框 +A GT框 -A 交集 Where A 预测框 represents the bounding box in the verification label predicted by the abnormal defect recognition baseline model under the detection task, (x1, y1) represents the coordinates of the upper left corner of the bounding box in the verification label, (x2, y2) represents the coordinates of the lower right corner of the bounding box in the verification label, and A GT框 represents the area of ​​the bounding box in the ground-truth label of the defect reference image predicted by the abnormal defect recognition baseline model, represents the upper left corner coordinate of the bounding box in the true label, Represents the lower right corner coordinate of the bounding box in the true label, A 预测框 ∩A GT框 represents the intersection area of ​​the bounding box in the verification label and the boundary in the true label, A 预测框 ∪A GT框 represents the union area of ​​the bounding box in the verification label and the bounding box in the true label; In the image segmentation task, the intersection-of-union (IoU) ratio of pixels of each category in the bounding box of the verification label is calculated with the pixels of each category in the true label. If any IoU ratio is lower than a given threshold, the defective reference image corresponding to the verification label is regarded as a low-quality image. Intersection over Union (IoU) j The calculation formula is: Where j represents the jth category in the verification label predicted by the abnormal defect recognition baseline model under the image segmentation task, j∈{1, 2, ...J}, TP j Indicates the number of pixels whose predicted pixel category and the actual category are both j, FP j FN represents the number of pixels whose predicted category is different from the true category. j Indicates that the predicted pixel category is not j, but the actual category is j pixels.

4. The method for generating small sample abnormal data of a power transmission line based on a diffusion model and feature migration according to claim 3, characterized in that: The specific steps of step 3 are: Step 3.1: For the defect areas in the defect library and the low-quality image, obtain the top-K defect areas similar to the defect areas in the low-quality image based on the similarity search method, and use them as the abnormal defect reference image; Step 3.2: For the device body library and the device body area of ​​the low-quality image, the top-K device bodies similar to the device body area of ​​the low-quality image are obtained according to the similarity search method as the base map to be generated.

5. The method for generating small sample abnormal data of a power transmission line based on a diffusion model and feature migration according to claim 4, characterized in that: The similarity search method in step 3.1 and step 3.2 is a similarity search method based on deep learning, and the specific steps are: Use a pre-trained model to convert images and low-quality images in the defect library or the device library into high-dimensional feature vectors. The pre-trained model is a ResNet or CLIP image encoder. ResNet extracts spatial features of images through a residual network, which are high-dimensional feature vectors. The CLIP image encoder is based on a multimodal model of contrastive learning and directly extracts global semantic features from images, which are high-dimensional feature vectors. High-dimensional feature vectors have dimensions of 512 or more. Perform L2 normalization on high-dimensional feature vectors; The cosine similarity is used to measure the directional consistency of the high-dimensional feature vectors of the images in the defect library and the low-quality images or the images in the device body library and the low-quality images. The formula is: Among them, I1 represents the image in the defect library or the image in the device body library, I2 represents the low-quality image, and f image (I1) and f image (I2) represents the high-dimensional feature vector after the transformation of I1 or I2, "·" represents the vector dot product operation, ||f image (l1)|| and||f image (I2)|| respectively represent the L2 norm of the high-dimensional feature vector, and the calculation formula is f k It represents the kth element in the high-dimensional feature vector, reflecting the strength or relevance of the image on a certain abstract feature. Abstract features include texture, shape or semantic category. ∑k represents the sum of all high-dimensional feature dimensions.

6. The method for generating small sample abnormal data of a power transmission line based on a diffusion model and feature migration according to claim 5, characterized in that: The specific steps of step 4 are: Based on the base map to be generated, the repair area that needs to be modified is marked in a binary mask map of the same size as the base map to be generated of the defect, that is, a mask of the area to be migrated is generated, wherein the pixels of the repair area marked to be modified in the binary mask map are represented by 1, which is white, and the pixels of the retained area are represented by 0, which is black. The marking methods include drawing with a brush in a drawing software, marking with a labeling software, or drawing with an algorithm. The drawing software includes Photoshop, the labeling software includes labelme, and the algorithmic drawing includes the Monte Carlo rejection sampling method, the polar angle sorting algorithm, the convex hull algorithm, or the python tool Shapely.

7. The method for generating small sample abnormal data of power transmission lines based on diffusion model and feature migration according to claim 6, characterized in that: The specific steps of step 5 are: Step 5.1: The abnormal defect reference image corresponding to the repair area is passed into the clip visual encoder to extract the multi-scale feature vector of each abnormal defect reference image, including texture and color; Step 5.2: Add noise to the repaired area based on the base image to be generated and the mask noise formula to obtain the initial noise image. The mask noise formula is: Where, represents the noisy image of the repaired area at time step t, represents the noise scheduling parameter, that is, the degree of noise cumulative attenuation, α i The noise attenuation coefficient controls the retention ratio of the original image during the noise addition process at the i-th time step, which is predefined by the basic diffusion model. i∈{1,2,...,T}, the total time step T is predefined by the basic diffusion model, and M represents the mask matrix, which is converted from the binary mask map. represents the base map to be generated, ⊙ represents element-by-element multiplication, and ∈ represents standard Gaussian noise, which simulates random noise in the diffusion process and drives the generation of diversity; Step 5.3: Based on IP-Adapter, the multi-scale feature vector is integrated into the basic diffusion model to analyze the initial noise image. The repaired area in the image is subjected to reverse diffusion denoising to generate a defect area that matches the features of the abnormal defect reference image, and an abnormal sample is generated by combining the non-defective area. The basic diffusion model is SD1.5, SDXL or Flux, and the reverse diffusion formula is: In scenarios where the original content of non-repaired areas needs to be preserved: In the scenario of completely regenerating the image: Where, X t Represents the global latent variable, which is the noise state of the binary mask at time step t, ∈ θ represents the noise predicted by the U-Net network, β t represents the noise attenuation coefficient predefined at time step t, α t =1-β t , c represents the multi-scale feature vector, z represents the standard normal distribution noise, σ t Represents the noise scaling factor, which controls the intensity of random noise and is usually Indicates the use of binary mask noise, Indicates that the global noise of the original image is retained outside the mask of the area to be migrated Indicates that the basemap will be repaired By adding noise to the time step t through the forward noise addition process of the diffusion model, the global noise is obtained. Indicates that standard Gaussian noise is retained outside the mask of the area to be migrated represents the initialized standard Gaussian noise, x t-1 represents the generated abnormal samples; Step 5.4: If the abnormal sample meets the given requirements, the generation ends; otherwise, the execution is as follows: If the similarity between the generated abnormal sample and the abnormal defect reference image is less than the given threshold, adjust the value of the IP-Adapter core parameter scale in step 5.3 and execute step 5.3 again; or go to step 5.1 and continue; If the repair area is lower than the resolution of 50*50 or the number of generated images is insufficient, go to step 4 to redraw the binary mask and continue. If the number of generated images is insufficient, re-acquire the base image to be generated, go to step 4 to redraw the binary mask image, go to step 5.3 to reselect the diffusion model random noise, or go to 5.3 to select a different basic diffusion model and continue.

8. A transmission line small sample abnormal data generation system based on diffusion model and feature migration, characterized by: include: Data acquisition module: Build an abnormal defect library and a device main body library based on the defect reference images and corresponding labels in the data set, where the data set includes at least one defect reference image of each type; Low-quality image acquisition module: trains an abnormal defect recognition baseline model based on the dataset and verifies the low-quality images; Comparison module: For the abnormal defect library and the equipment main body library, use image comparison technology to compare with low-quality images to obtain abnormal defect reference images and the base image to be generated; Mask module: Generates the mask of the area to be migrated based on the base map to be generated; Abnormal sample generation module: Based on the base map to be generated and the mask of the area to be migrated, the diffusion model is used to migrate the abnormal area features of the low-quality image to the base map to be generated for anomaly generation.

Images