An unmanned aerial vehicle inspection defect data generation method, system, device and medium

Abstract

The application discloses a kind of unmanned aerial vehicle inspection defect data generation method, system, equipment and medium, it is related to unmanned aerial vehicle inspection field, the method includes obtaining the defect-free and defect sample image of unmanned aerial vehicle in distribution line inspection collection;Defect sample image is injected into diffusion model, the semantic mask reflecting different defect types is extracted from defect sample image, to define embedding vector is iteratively updated with semantic mask as constraint, obtain target embedding vector;Based on semantic mask generates pseudo-defect mask, and constructs topological structure graph using device mask;The multiscale feature information extracted from pseudo-defect mask and topological structure graph is spliced with target embedding vector, and obtains composite embedding vector;With pseudo-defect mask as constraint, to pseudo-defect image is generated by executing mixed diffusion to defect-free sample image with composite embedding vector as condition, guarantee the quality of expansion dataset.

Description

Technical Field

[0001] This invention relates to the field of drone inspection, and in particular to a method, system, device and medium for generating defect data for drone inspection. Background Technology

[0002] In recent years, with the continuous deepening of the intelligentization and digitalization of power systems, drones have become the main tool for automated inspection in scenarios such as power distribution networks. Equipped with high-definition visible light or infrared cameras, drones can efficiently collect image data of equipment such as towers, conductors, insulators, and fittings, significantly reducing the risks and costs of manual tower climbing inspections. However, in actual inspection tasks, the problems of scarce defect samples and unbalanced categories are very prominent, resulting in insufficient generalization ability of detection models and seriously restricting their practical application value.

[0003] To expand the sample, traditional data augmentation methods can only achieve low-level geometric and lighting transformations such as rotation, cropping, flipping, and brightness adjustment. They are difficult to generate realistic and diverse defect morphologies at the semantic level, leading to frequent misjudgments by the detection model and failing to meet the needs of complex power distribution network inspection scenarios.

[0004] Therefore, how to effectively expand the defect sample and improve the accuracy of subsequent UAV defect detection has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This invention provides a method, system, device, and medium for generating defect data from unmanned aerial vehicle (UAV) inspections, which addresses how to decouple multiple types of features of defect images through a collaborative diffusion model and semantic masking, thereby achieving efficient and high-quality expansion of a small number of defect samples.

[0006] To address the aforementioned technical problems, embodiments of the present invention provide a method for generating defect data during UAV inspections, comprising: Acquire defect-free sample images and defect sample images collected by drones during power distribution line inspections; The defect sample image is injected into the diffusion model for noise prediction. During the prediction process, semantic masks reflecting different defect types are extracted from the defect sample image. The predefined embedding vector is iteratively updated using the semantic mask as the first constraint to obtain the target embedding vector; A pseudo-defect mask is generated based on the semantic mask, and a topology diagram is constructed using the device mask read from the defect-free sample image. The multi-scale feature information extracted from the pseudo-defect mask and the topology diagram is concatenated with the target embedding vector to obtain a composite embedding vector. Using the composite embedding vector as a condition and the pseudo-defect mask as a second constraint, a hybrid diffusion process is performed on the defect-free sample image to generate a pseudo-defect image.

[0007] Furthermore, the step of injecting the defective sample image into the diffusion model for noise prediction includes: In the diffusion model, the defect sample image is encoded into the latent space and noise is added to obtain noisy latent variables; Using the embedding vectors as guiding conditions, the noisy latent variables are input into the noise network of the diffusion model to perform noise prediction.

[0008] Furthermore, the step of iteratively updating the predefined embedding vector using the semantic mask as a region constraint to obtain the target embedding vector includes: Different loss calculation regions are defined using the semantic masks described above; Within each of the loss calculation regions, a loss function is introduced to perform iterative gradient updates on the corresponding embedding vector to obtain the target embedding vector.

[0009] Furthermore, the step of generating a pseudo-defect mask based on the semantic mask and constructing a topology map using the device mask read from the defect-free sample image includes: Perform a geometric transformation or manually draw the semantic mask to obtain the pseudo-defect mask; The skeleton and edge information of each transmission line device are extracted from the defect-free sample image using the device mask, and the topology diagram is generated using the skeleton and edge information.

[0010] Furthermore, the process of acquiring the multi-scale feature information includes: The pseudo-defect mask is input into a preset semantic branch encoder to extract the first defect feature, and the topology diagram is input into a topology branch encoder to extract the second defect feature. The first defect feature and the second defect feature are concatenated to obtain the multi-scale feature information.

[0011] Furthermore, the composite embedding vector is used as a condition, and the pseudo-defect mask is used as a second constraint to perform a hybrid diffusion on the defect-free sample image to generate a pseudo-defect image: The defective region is obtained by using the pseudo-defect mask to define a region on the defect-free sample image. Perform forward noise addition within the non-defect region, and, Within the defect region, a diffusion model is introduced conditioned on the composite embedding vector to perform inverse denoising, thereby obtaining the pseudo-defect image.

[0012] Furthermore, the step of performing inverse denoising within the defect region by introducing a diffusion model conditionally based on the composite embedding vector further includes: In the noise network of the diffusion model, the pixel difference and gradient difference between the denoised image of the current predicted output and the defect-free sample image are calculated; The pixel differences and gradient differences are weighted and analyzed to construct a fusion weight map; The weights of the attention layer in the noisy network are dynamically adjusted using the fused weight graph.

[0013] Another embodiment of the present invention provides a UAV inspection defect data generation system, comprising: The image acquisition module is used to acquire defect-free sample images and defect sample images collected by the UAV during power distribution line inspection. The noise prediction module is used to inject the defect sample image into the diffusion model for noise prediction. During the prediction process, semantic masks reflecting different defect types are extracted from the defect sample image. The vector update module is used to iteratively update the predefined embedding vector with the semantic mask as the first constraint to obtain the target embedding vector. A mask topology construction module is used to generate a pseudo-defect mask based on the semantic mask and construct a topology diagram using the device mask read from the defect-free sample image. The vector concatenation module is used to concatenate the multi-scale feature information extracted from the pseudo-defect mask and the topology diagram with the target embedding vector to obtain a composite embedding vector. The diffusion module is used to perform hybrid diffusion on the defect-free sample image with the composite embedding vector as a condition and the pseudo-defect mask as a second constraint to generate a pseudo-defect image.

[0014] Another embodiment of the present invention provides a computer device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the unmanned aerial vehicle (UAV) inspection defect data generation method as described above.

[0015] In another embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, wherein when the device containing the computer-readable storage medium executes the computer program, the method for generating defect data for UAV inspection as described above is implemented.

[0016] Compared with the prior art, the beneficial effects of the embodiments of the present invention are at least one of the following: This invention employs a collaborative diffusion model to invert and train UAV defect sample images and their corresponding semantic masks, obtaining embedding vectors reflecting different defect types. This achieves deep decoupling at the semantic level, ensuring the generation quality of specific equipment defects in complex backgrounds. By generating pseudo-defect masks and topology maps and concatenating their multi-scale features with the embedding vectors, it ensures that the generated defects conform to the physical orientation of the power equipment. Furthermore, the concatenated vectors are integrated into the generation conditions of the diffusion model, guaranteeing the physical rationality and structural authenticity of the generated defects. Finally, a hybrid diffusion strategy with masks as regional constraints is adopted to ensure that the defect regions of the generated images are highly controllable, improving the quality and usability of the generated data. This provides high-quality, highly diverse training samples for downstream defect detection models, enhancing the reliability of subsequent power system operation and maintenance. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the process for generating defect data during drone inspection in one embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a UAV inspection defect data generation system in one embodiment of the present invention; Figure 3 This is a structural block diagram of a preferred embodiment of a computer device provided by the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. The terms "vertical," "horizontal," "left," "right," "upper," "lower," and similar expressions used herein are for illustrative purposes only and do not indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0021] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0022] One embodiment of the present invention provides a method for generating defect data during UAV inspections. For details, please refer to [link to relevant documentation]. Figure 1 , Figure 1 The diagram shown is a flowchart of a method for generating defect data during UAV inspections, according to one embodiment of the present invention, including the following steps: S1. Acquire defect-free sample images and defective sample images collected by the UAV during power distribution line inspection.

[0023] In some implementations of this embodiment, for example: a 10kV power distribution line can be inspected by a drone equipped with a visible light camera, and 1,000 images of defect-free insulator strings (as normal defect-free samples) can be collected. The image resolution is set to 1920×1080 and the frame rate is 30fps. A small number of defect sample images containing real defects are collected and screened from the historical drone inspection sample library.

[0024] S2~S3. Inject the defect sample image into the diffusion model for noise prediction. During the prediction process, extract the semantic mask reflecting different defect types from the defect sample image. Use the semantic mask as the first constraint to iteratively update the predefined embedding vector to obtain the target embedding vector.

[0025] In this embodiment, instead of directly using a single defect appearance embedding, a "environment-device-defect" triple embedding is constructed to address the issue of defect errors occurring in the background. Specifically, firstly, semantic masks reflecting different defect types are extracted from defect sample images. For example, for a very small number of sample images containing real defects... Semantic segmentation is performed to obtain the background mask. Device mask Defect mask These semantic masks can reflect the defect characteristics of different transmission line areas.

[0026] Then, three independent text embedding vectors are trained using these masks, including: an environmental context embedding for capturing background textures such as the sky and ground. Equipment appearance embedding used to capture the material and structure of insulators, conductors, and other equipment. Embedded with defect characterization of textures used to capture defects such as cracks and rust. Specifically, the aforementioned embedding vector can be obtained by introducing a diffusion model for noise prediction. In this embodiment, the Latent Diffusion Model (LDM) is preferred. The core innovation of LDM lies in transferring the diffusion process from the pixel space to the latent space. It uses a pre-trained autoencoder to compress high-dimensional image data into a low-dimensional representation, and then performs diffusion modeling in this low-dimensional space. This embodiment aims to utilize the powerful prior capabilities of the LDM diffusion model to ensure that the generated defects are perfectly integrated with the background (defect-free sample image) in terms of texture, lighting, and shadow.

[0027] Based on this, defect sample images are injected into the diffusion model. During noise prediction in the diffusion model, the defect sample images are first encoded into the latent space using an autoencoder, and noise is added to obtain noisy latent variables. Then, using each embedding vector as a guiding condition, the noisy latent variables are input into the noise network of the diffusion model to perform noise prediction. For example, using the embedding vector... , as well as As a conditionally driven noise network for noise prediction, the three embedding vectors are trained independently during the prediction process.

[0028] During this training process, a loss function is used to optimize the three embeddings in parallel. This loss function... The definition is as follows: in, For semantic masking, For different semantic region categories (background, device, defect). This represents element-wise multiplication (Hadamard product). Is Standard Gaussian noise is added at all times. It is a noise network U-Net for the diffusion model. It is the input image Encoder via LDM Encode into latent space and add noise Noisy latent variables afterwards It is the time step of the diffusion process. It is the embedding vector of the corresponding category that is currently being optimized.

[0029] It can be seen that during the independent training of each embedding vector, these different semantic masks serve as a "region constraint," i.e., the first constraint, to limit the loss calculation region of the loss function. From the above... As can be seen from the definition formula, the loss function is used in calculation Predicted noise and actual noise Before the L2 distance between them, the corresponding semantic mask is used. Element-by-element multiplication was performed, thus achieving region limitation.

[0030] Therefore, within each loss calculation region, a loss function is introduced to iteratively update the gradient of the corresponding embedding vector, ultimately yielding the optimized target embedding vector. For example, The gradient update is determined only by the pixels in the background region. The gradient update is determined only by the device region, while The gradient update is determined only by the defect texture region.

[0031] S4~S5 First, a pseudo-defect mask is generated based on the semantic mask, and a topology diagram is constructed using the device mask read from the defect-free sample image.

[0032] In some embodiments of the present invention, a pseudo-defect mask can be obtained by performing geometric transformations or manually drawing the semantic mask. In another implementation of the present invention, the pseudo-defect mask can also be generated by training a dedicated diffusion model. Specifically, a mask embedding is learned using text inversion (…). ), the embedding Captured the real mask The distribution characteristics of are given by the objective function for training, which is expressed by the following formula: in, This represents the trained mask embedding. Once training is complete, it can be used. As a condition, a large number of new and diverse pseudo-defect masks are generated using the inverse denoising process. This pseudo-defect mask This will be used as input to the subsequent feature encoder and as a region guiding condition for the hybrid diffusion process.

[0033] To ensure that the generated defects visually conform to the physical structure and geometric constraints of the power transmission equipment, this embodiment further constructs a topology diagram. Specifically, the equipment mask is first read from a defect-free sample image. The skeleton and edge information of each power transmission line device are then extracted from the defect-free sample image using the equipment mask. Preferably, the equipment mask can be input into an edge detection algorithm to extract the skeleton and edge information. The extracted skeleton and edge information is then used to generate the topology diagram. This topology diagram describes the extension direction of the conductor or the arrangement curve of the insulator string, and is used to guide the subsequent growth of defects along the correct physical direction.

[0034] After obtaining the pseudo-defect mask and topology map, the multi-scale feature information extracted from the pseudo-defect mask and topology map is concatenated with the target embedding vector to obtain the composite embedding vector.

[0035] For the extraction of multi-scale feature information, this embodiment designs a dual-branch encoder, including a semantic branch encoder and a topology branch encoder. Specifically, the pseudo-defect mask is input into the preset semantic branch encoder to extract the first defect feature reflecting the spatial characteristics of the power transmission equipment, while the topology diagram is input into the topology branch encoder to extract the second defect feature reflecting the geometric topology of the power transmission line.

[0036] For example, this semantic branch uses a residual network (ResNet-50) as the backbone network, with the output of one residual block... Represented as: in, It is the input feature map of the residual block. The residual mapping function consists of multiple convolutional layers, batch normalization, and activation functions, and its trainable parameters are: , It's a "shortcut connection," which will be entered. Add it directly to the output of the residual mapping. The activation function is used. The backbone network extracts semantic features reflecting the "coverage" of defects, i.e., the first defect feature, by stacking multiple residual blocks with a feature pyramid network. .

[0037] Because electrical equipment (such as stranded wires and insulators) often exhibits slender or curved non-rigid structures, ordinary convolution is insufficient to capture their geometric deformations. Therefore, this embodiment introduces deformable convolution in the topology branch to capture these deformations, specifically expressed by the following formula: in, It is the current pixel position on the output feature map. That is, the output feature map at the location The response value at that point, i.e., the feature extracted after deformable convolution, These are the convolution kernel weights. It is the fixed offset of standard convolution. It is a learnable dynamic offset that is automatically predicted by the network based on input features, and K is the total number of sampling points of the convolution kernel. This allows the sampling points of the convolution kernel to adaptively shift, enabling feature extraction to closely follow the edges of curved wires or irregular insulators, generating feature maps that incorporate geometric constraints. This refers to the second defect feature.

[0038] The first defect feature and the second defect feature are concatenated and mapped to the final structured spatial embedding through a multilayer perceptron (MLP) to obtain multi-scale feature information. Specifically, it is expressed as: Multi-scale feature information With triple target embedding vector ( , , The tokens are concatenated to form a composite embedding vector. Specifically, it is expressed as follows: It should be understood that this vector It also contains four pieces of information: "background environment", "equipment structure", "defect texture" and "spatial location and geometric topology", which serve as the only strong constraint in the subsequent diffusion model's defect data generation process.

[0039] Based on this, in some embodiments of the present invention, a total loss function is designed. It can be expressed by the following formula: By minimizing this total loss function This enables the optimization of each embedding vector and the parameters of the dual-branch encoder.

[0040] S6. Using the composite embedding vector as a condition and the pseudo-defect mask as a second constraint, perform hybrid diffusion on the defect-free sample image to generate a pseudo-defect image.

[0041] This step generates the final defect image data. A diffusion model is still preferred, employing a hybrid diffusion mechanism to preserve the pseudo-defect mask. The original image content is excluded. The hybrid diffusion strategy specifically refers to performing inverse denoising only within the defect-free sample image region defined by the defect mask. Specifically, the pseudo-defect mask is used as a region-limiting constraint (second constraint) to limit the region on the defect-free sample image, resulting in a defect region. Within the defect region, a diffusion model is introduced with composite embedding vectors as conditions to perform inverse denoising; while forward denoising is performed in the undefined non-defect region, ultimately resulting in a pseudo-defect image.

[0042] In this embodiment, the iteration for each step t of the above-described mixing and diffusion process (from T to 1) follows the following formula: in, It is a reverse denoising process. yes Noisy latent variable at time -1 yes Noisy latent variables at time points, It is a forward noise addition process. It is a defect-free sample image. It is a defect-free sample image with noise added in the forward direction. Noisy latent variable at time -1 It is the mask area The reversal of represents a "non-defect area".

[0043] Based on this formula, it can be understood that in the defect region, i.e., the pseudo-defect mask... Within the defined area, a standard reverse denoising process is performed. This process is based on and composite embedding vector Given these conditions, defect features with multiple constraints including environment, equipment, defects, and topology are generated. In non-defect regions, i.e. Within the defined area, it will not be executed. Instead, it performs a forward noise addition process. The original defect-free sample image Add noise At that moment, we received .

[0044] This hybrid diffusion process ensures that the pseudo-defect mask is in place. In addition, the image content is strictly preserved as the original, defect-free image. In the mask Within, then according to The guidance generates new defective content.

[0045] Understandably, this is true when performing inverse denoising. In some cases, the standard diffusion model (especially its cross-attention layer) may not be able to perfectly fill the entire defect area with generated content. Especially when generating subtle linear defects (such as insulator cracks or broken conductor strands), the loss of high-frequency details often leads to blurred or poorly aligned defect edges. Therefore, after generating the pseudo-defect image, to avoid the loss of these subtle defects, this embodiment... In the cross-attention layer inside U-Net, an adaptive attention reweighting mechanism guided by edge gradients is introduced.

[0046] Among them, Inside U-Net, the cross-attention layer is accessed through... (query) (key) (value) vectors are used for information exchange. From noisy latent variables The middle representation , and From the composite conditional control vector They consist of learnable projection matrices. The generation process is specifically represented as follows: in, It is the first one inside the U-Net network Hierarchical identifiers for cross-attention layers, , , These are the learnable projection matrices used in this layer to generate query, key, and value vectors. It is a noisy latent variable Passed to the Intermediate feature representation at the layer level, It is the composite conditional control vector input to the attention layer, i.e., the aforementioned composite embedding vector. .

[0047] To achieve reweighting in the attention layer, the pixel differences and gradient differences between the denoised image and the defect-free sample image are simultaneously calculated during inverse denoising of the noisy network in the diffusion model. For example, at each step of inverse denoising... In the middle, calculation utilizes and Predict the currently estimated "clean" denoised image Compared with defect-free normal samples In the mask L2 distance of pixels within the region Specifically, it is represented by the following formula: At the same time, using the Sobel operator Extract separately and High-frequency edge features, calculate the structural gradient difference between the two. Specifically, it is represented by the following formula: The gradient operator is defined as follows: Then, a weighted analysis is performed on pixel differences and gradient differences to construct a fusion weight map. The specific construction process is shown in the following formula: in, To merge the weighted graphs, Balance coefficient (usually) (to emphasize edge constraints) To prevent extremely small values ​​from being divided by zero, this embodiment employs an adaptive scaling Softmax operation, mapping the difference values ​​to fusion weights. If the pixel differences in a certain region... or gradient difference If it is very small (i.e., the generated effect is weak, close to the defect-free original image), its reciprocal term will become very large, thus making... It receives extremely high weight in this region.

[0048] Furthermore, in standard cross-attention, the attention layer weights Depend on and The calculation results are shown in the following formula: Then, the attention layer weights in the noisy network are dynamically adjusted using the fused weight graph. The adjustment process is as follows: In the formula, To correct the attention layer weights, This is the intensity adjustment factor.

[0049] Based on this, the final output of the attention layer is represented as: In some embodiments of the present invention, the generated pseudo-defect images undergo a final authenticity assessment and quality screening, which can be achieved using methods such as FID scores and manual verification. Then, the selected pseudo-defect images and their corresponding pseudo-defect masks m are added to the original small set of defect sample images to form a large-scale, high-quality augmented dataset, which can be directly used for training downstream UAV inspection defect detection and localization models.

[0050] For example, in a scenario where it is necessary to identify insulator cracks in power line images transmitted back by drones for precise operation and maintenance of the power system, a new detection model (such as YOLO, Faster R-CNN, or Vision Transformer) can be trained using the generated extended dataset, thereby achieving efficient and high-precision defect identification.

[0051] In summary, this embodiment first acquires defect-free sample images and a small number of defective sample images from UAV inspections. Then, the defective sample images are input into a diffusion model. Through a noise prediction task and combined with the extracted semantic mask, embedding vectors representing different defect types are iteratively trained in a decoupled manner. Simultaneously, pseudo-defect masks and topology maps are obtained based on the semantic mask and the device mask on the defect-free samples, respectively. The semantic features of the pseudo-defect mask and the geometric features of the topology map are extracted and concatenated with the embedding vectors to form a composite vector. Finally, using this composite vector as a condition and the pseudo-defect mask, a region is delineated on the defect-free image, and hybrid diffusion is performed to generate pseudo-defect images, thus improving the quality of the expanded defect training dataset.

[0052] One embodiment of the present invention provides a UAV inspection defect data generation system. For details, please refer to [link / reference]. Figure 2 , Figure 2 The diagram shown is a schematic representation of a UAV inspection defect data generation system according to one embodiment of the present invention, comprising: The image acquisition module M1 is used to acquire defect-free sample images and defect sample images collected by the UAV during power distribution line inspection. The noise prediction module M2 is used to inject the defect sample image into the diffusion model for noise prediction. During the prediction process, semantic masks reflecting different defect types are extracted from the defect sample image. The vector update module M3 is used to iteratively update the predefined embedding vector with the semantic mask as the first constraint to obtain the target embedding vector. The mask topology construction module M4 is used to generate a pseudo-defect mask based on the semantic mask and construct a topology diagram using the device mask read from the defect-free sample image. The vector concatenation module M5 is used to concatenate the multi-scale feature information extracted from the pseudo-defect mask and the topology diagram with the target embedding vector to obtain a composite embedding vector. The diffusion module M6 is used to perform hybrid diffusion on the defect-free sample image with the composite embedding vector as a condition and the pseudo-defect mask as a second constraint to generate a pseudo-defect image.

[0053] like Figure 3 As shown, this embodiment of the invention also provides a computer device. Figure 3 This is a structural block diagram of a preferred embodiment of a computer device provided by the present invention. The computer device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the method described above.

[0054] Preferably, the computer program can be divided into one or more modules / units (such as computer program 1, computer program 2, ...), and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the computer device.

[0055] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor can be any conventional processor. The processor is the control center of the terminal device, connecting various parts of the terminal device through various interfaces and lines.

[0056] The memory mainly includes a program storage area and a data storage area. The program storage area can store the operating system, applications required for at least one function, etc., while the data storage area can store related data, etc. Furthermore, the memory can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard drive, a SmartMedia Card (SMC), a Secure Digital (SD) card, and a Flash Card, or other volatile solid-state storage devices.

[0057] It should be noted that the aforementioned terminal devices may include, but are not limited to, processors and memory, as will be understood by those skilled in the art. Figure 3 The structural block diagram is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or use different components. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0058] Accordingly, embodiments of the present invention provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the steps in the method of the above embodiments, for example... Figure 1 Steps S1 to S6 as described above.

[0059] The technical features and effects of the UAV inspection defect data generation system proposed in this embodiment are the same as those of the UAV inspection defect data generation method proposed in this embodiment, and will not be repeated here.

[0060] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for generating defect data during UAV inspection, characterized in that, include: Acquire defect-free sample images and defect sample images collected by drones during power distribution line inspections; The defect sample image is injected into the diffusion model for noise prediction. During the prediction process, semantic masks reflecting different defect types are extracted from the defect sample image. The semantic masks include background mask, device mask and defect mask. The predefined embedding vector is iteratively updated using the semantic mask as the first constraint to obtain the target embedding vector; A pseudo-defect mask is generated based on the semantic mask, and a topology diagram is constructed using the device mask read from the defect-free sample image. The multi-scale feature information extracted from the pseudo-defect mask and the topology diagram is concatenated with the target embedding vector to obtain a composite embedding vector. Using the composite embedding vector as a condition and the pseudo-defect mask as a second constraint, perform hybrid diffusion on the defect-free sample image to generate a pseudo-defect image; The step of iteratively updating the predefined embedding vector using the semantic mask as a region constraint to obtain the target embedding vector includes: Different loss calculation regions are defined using the background mask, the device mask, and the defect mask, respectively. Within each of the loss calculation regions, a loss function is introduced to perform iterative gradient updates on the corresponding embedding vector to obtain the target embedding vector; The process of acquiring the multi-scale feature information includes: The pseudo-defect mask is input into a preset semantic branch encoder to extract the first defect feature, and the topology diagram is input into a topology branch encoder to extract the second defect feature. The first defect feature and the second defect feature are concatenated to obtain the multi-scale feature information.

2. The method for generating defect data for UAV inspection as described in claim 1, characterized in that, The step of injecting the defective sample image into the diffusion model for noise prediction includes: In the diffusion model, the defect sample image is encoded into the latent space and noise is added to obtain noisy latent variables; Using the embedding vectors as guiding conditions, the noisy latent variables are input into the noise network of the diffusion model to perform noise prediction.

3. The method for generating defect data for UAV inspection as described in claim 1, characterized in that, The step of generating a pseudo-defect mask based on the semantic mask and constructing a topology map using the device mask read from the defect-free sample image includes: Perform a geometric transformation or manually draw the semantic mask to obtain the pseudo-defect mask; The skeleton and edge information of each transmission line device are extracted from the defect-free sample image using the device mask, and the topology diagram is generated using the skeleton and edge information.

4. The method for generating defect data for UAV inspection as described in claim 1, characterized in that, The process involves performing a hybrid diffusion operation on the defect-free sample image, using the composite embedding vector as a condition and the pseudo-defect mask as a second constraint, to generate a pseudo-defect image. The defective region is obtained by using the pseudo-defect mask to define a region on the defect-free sample image. Perform forward noise addition within the non-defect region, and, Within the defect region, a diffusion model is introduced conditioned on the composite embedding vector to perform inverse denoising, thereby obtaining the pseudo-defect image.

5. The method for generating defect data for UAV inspection as described in claim 4, characterized in that, The step of performing inverse denoising by introducing a diffusion model conditioned on the composite embedding vector within the defect region further includes: In the noise network of the diffusion model, the pixel difference and gradient difference between the denoised image of the current predicted output and the defect-free sample image are calculated; The pixel differences and gradient differences are weighted and analyzed to construct a fusion weight map; The weights of the attention layer in the noisy network are dynamically adjusted using the fused weight graph.

6. A system for generating defect data for unmanned aerial vehicle (UAV) inspections, characterized in that, include: The image acquisition module is used to acquire defect-free sample images and defect sample images collected by the UAV during power distribution line inspection. The noise prediction module is used to inject the defect sample image into the diffusion model for noise prediction. During the prediction process, semantic masks reflecting different defect types are extracted from the defect sample image. The semantic masks include background masks, device masks, and defect masks. The vector update module is used to iteratively update the predefined embedding vector with the semantic mask as the first constraint to obtain the target embedding vector. A mask topology construction module is used to generate a pseudo-defect mask based on the semantic mask and construct a topology diagram using the device mask read from the defect-free sample image. The vector concatenation module is used to concatenate the multi-scale feature information extracted from the pseudo-defect mask and the topology diagram with the target embedding vector to obtain a composite embedding vector. The diffusion module is used to perform hybrid diffusion on the defect-free sample image with the composite embedding vector as a condition and the pseudo-defect mask as a second constraint to generate a pseudo-defect image. The step of iteratively updating the predefined embedding vector using the semantic mask as a region constraint to obtain the target embedding vector includes: Different loss calculation regions are defined using the semantic masks described above; Within each of the loss calculation regions, a loss function is introduced to perform iterative gradient updates on the corresponding embedding vector to obtain the target embedding vector; The process of acquiring the multi-scale feature information includes: The pseudo-defect mask is input into a preset semantic branch encoder to extract the first defect feature, and the topology diagram is input into a topology branch encoder to extract the second defect feature. The first defect feature and the second defect feature are concatenated to obtain the multi-scale feature information.

7. A computer device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the UAV inspection defect data generation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the device containing the computer-readable storage medium executes the computer program, it implements the UAV inspection defect data generation method as described in any one of claims 1 to 5.

Images