Power inspection video diffusion super-resolution collaborative enhancement method for equipment defect identification

Abstract

This invention discloses a power inspection video diffusion and super-resolution collaborative enhancement method for equipment defect identification, comprising the following steps: Step 1, acquiring power inspection video data, compressing the power inspection video data, and sending the compressed power inspection video data to a decoding module; Step 2, the decoding module decodes the compressed power inspection data to obtain decoded data, reconstructs the details of the equipment defect through a diffusion model, and then optimizes the video information of the equipment defect through a super-resolution model; Step 3, inputting the video data output by the super-resolution model into a defect identification model to complete equipment defect detection and classification. This invention effectively avoids the identification failure problem caused by direct input of traditional compressed video, and significantly improves the stability and reliability of defect identification.

Description

Technical Field

[0001] This invention relates to the field of intelligent power system inspection technology, and in particular to a video diffusion super-resolution collaborative enhancement method for power system inspection aimed at equipment defect identification. Background Technology

[0002] With the intelligent upgrading of power systems, drone inspection has become the mainstream operation and maintenance method for key equipment such as high-voltage insulators, transmission line clamps, and transformer bushings. During the inspection, drones need to transmit massive amounts of video data in real time. However, due to limitations in wireless transmission bandwidth, the video must be highly compressed before transmission, which inevitably results in the loss of subtle features of equipment defects.

[0003] In video compression, current mainstream methods employ standard compression algorithms such as H.265 / HEVC and AV1, using techniques like intra-frame prediction, inter-frame motion compensation, and entropy coding to reduce bitrate. However, to meet transmission efficiency requirements, the compression ratio is typically ≥20:1, leading to distortions such as blockiness and texture smoothing, directly compromising the video's inherent characteristics.

[0004] Meanwhile, the backend AI defect recognition algorithm is highly dependent on the quality of the input video. If video details are lost or severely distorted, the AI ​​defect recognition algorithm will be unable to effectively extract defect features, leading to frequent false positives and false negatives. It not only fails to replace manual inspection but may also overlook potential safety hazards. Therefore, how to achieve accurate restoration of details and preservation of structural fidelity on compressed video has become the core bottleneck in overcoming the entire process from drone inspection to AI recognition.

[0005] In traditional image enhancement and super-resolution techniques, interpolation super-resolution and deep learning super-resolution are widely used. Interpolation super-resolution only enlarges images by interpolating pixels, without the ability to generate details. The enlarged video is still blurry and cannot recover lost defect features. Deep learning super-resolution, on the other hand, extracts image features and reconstructs high-resolution images based on deep learning. However, it is only suitable for single degradation scenes from low to high resolution and cannot handle the combined problems of blockiness caused by compression and environmental degradation. Furthermore, some GAN-based models are prone to generating false details, leading to distortion of defect structures.

[0006] In standalone applications of diffusion models, they learn data distribution through iterative processes of adding and removing noise, demonstrating excellent performance in image generation and restoration. Existing technologies have attempted to use diffusion models to restore compressed images, but these methods lack customized training for the defect features of power inspection equipment. Furthermore, when used alone, they lack structural constraints, easily generating details that do not conform to physical laws, making them unsuitable for direct defect identification. Summary of the Invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies, such as the loss of subtle features of equipment defects in power inspection videos due to high-intensity compression, the inability of traditional image enhancement and super-resolution algorithms to cope with the superposition of compression distortion and environmental degradation, and the lack of customized training and structural constraints for power inspection equipment defect features in standalone diffusion models. This results in high false detection and false negative rates in backend AI defect identification, making it difficult to achieve accurate restoration of details and structural fidelity of compressed videos. This invention provides a diffusion-super-resolution collaborative enhancement method for power inspection videos for equipment defect identification. By collecting power inspection video data, compressing and decoding it, the data is sequentially processed by a diffusion model to reconstruct equipment defect details and a super-resolution model to optimize defect-related video information. The output data is then input into a defect identification model, achieving the goal of accurately detecting and classifying equipment defects.

[0008] The objective of this invention is achieved through the following technical solution: A power inspection video diffusion super-resolution collaborative enhancement method for equipment defect identification includes the following steps: Step 1: Collect power inspection video data, compress the power inspection video data, and send the compressed power inspection video data to the decoding module; Step 2: The decoding module decodes the compressed power inspection data to obtain decoded data. The decoded data is then reconstructed using a diffusion model to reconstruct the details of the equipment defects. Finally, the video information of the equipment defects is optimized using a super-resolution model. Step 3: Input the video data output by the super-resolution model into the defect recognition model to complete the equipment defect detection and classification.

[0009] Preferably, the diffusion model is an improved DDPM (Denoising Diffusion Probability Model) network model built on the PyTorch framework, and the super-resolution model is an improved EDSR (Enhanced Deep Residual Network) network model built on the PyTorch framework.

[0010] Preferably, the construction process of the DDPM network model is as follows: Historical power inspection video frames are acquired, the video frames are compressed, and the first paired dataset of compressed frames and original frames is constructed. A defect feature attention branch is added to the DDPM network model to simultaneously locate the device defect area and supplement details during the noise estimation network denoising process of the DDPM network model. Construct the loss function of the DDPM network model, and use the first paired dataset as training data to train the diffusion model in the first iteration. During the iteration, reconstruct the details of the device defect and output the reconstructed frame containing the defect details.

[0011] Preferably, the first iteration training process is as follows: Set the number of iterations, fix the noise estimation network in the first few iterations, train the defect feature attention branch, and jointly train the noise estimation network and the defect feature attention branch in the remaining iterations.

[0012] Preferably, the loss function includes a perceptual loss function, which is based on features extracted from the VGG16 network and is used to measure visual perceptual consistency.

[0013] Preferably, the construction process of the EDSR network model is as follows: Obtain reconstructed frames containing defective details, and construct a second paired dataset of reconstructed frames and original frames; An edge detection branch is added to the EDSR network model to locate the edges of key structures of the device during the feature extraction process of the feature extraction network in the EDSR network model. An EDSR network model loss function is constructed. The super-resolution model is trained in the second iteration using the second paired dataset as training data through the EDSR network model loss function. During the iteration process, the resolution is reconstructed and enlarged, and high-definition enhanced frames are output.

[0014] Preferably, the second iteration training process is as follows: Set the number of iterations, and test the effect of reconstructing and scaling the resolution on the validation set after every certain number of rounds. Stop iterating when the edge loss on the validation set is less than a preset threshold.

[0015] Preferably, in step 3, the defect identification model is the YOLOv8 defect identification model, in which a device defect dataset is pre-trained.

[0016] Preferably, the diffusion model is the SVD (Stable Video Diffusion Model) network model.

[0017] Preferably, the super-resolution model is the RCAN (Residual Channel Attention Network) network model.

[0018] The beneficial effects of this invention are: by designing the entire process of video compression, decoding, diffusion model reconstruction of details, super-resolution model optimization, and defect identification in a collaborative manner, this invention effectively avoids the identification failure problem caused by direct input of traditional compressed video, and greatly improves the stability and reliability of defect identification.

[0019] The diffusion model is built upon an improved DDPM network and incorporates a defect feature attention branch. During the denoising process, it can accurately focus on the high-frequency defect areas of power inspection equipment, enabling targeted reconstruction of defect details. Compared to the indiscriminate recovery mode of traditional models, this design avoids the blindness of detail recovery, allowing lost subtle defect features to be accurately restored. Furthermore, the restored details are highly consistent with the morphology and location features of the equipment defects, providing effective support for defect judgment.

[0020] The super-resolution model introduces an edge detection branch to simultaneously locate the edges of key equipment structures during the feature extraction stage, creating a synergistic effect with the diffusion model for detail reconstruction and structural correction. The diffusion model is responsible for supplementing defect details, while the super-resolution model improves resolution while correcting potential structural distortions. This ensures that the output video has clear defect details and strictly conforms to the physical structure of the power inspection equipment, avoiding the structural distortion problem that is prone to occur in traditional enhancement algorithms and adapting to the defect identification needs in diverse environments of power inspection. Attached Figure Description

[0021] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0022] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0023] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0025] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0026] Example 1: A video diffusion super-resolution collaborative enhancement method for power line inspection aimed at equipment defect identification, such as... Figure 1 As shown, it includes the following steps: Step 1: Collect power inspection video data, compress the power inspection video data, and send the compressed power inspection video data to the decoding module; Step 2: The decoding module decodes the compressed power inspection data to obtain decoded data. The decoded data is then reconstructed using a diffusion model to reconstruct the details of the equipment defects. Finally, the video information of the equipment defects is optimized using a super-resolution model. Step 3: Input the video data output by the super-resolution model into the defect recognition model to complete the equipment defect detection and classification.

[0027] The diffusion model is an improved DDPM network model built on the PyTorch framework, and the super-resolution model is an improved EDSR network model built on the PyTorch framework.

[0028] The construction process of the DDPM network model is as follows: Historical power inspection video frames are acquired, the video frames are compressed, and the first paired dataset of compressed frames and original frames is constructed. A defect feature attention branch is added to the DDPM network model to simultaneously locate the device defect area and supplement details during the noise estimation network denoising process of the DDPM network model. A loss function for the DDPM network model is constructed. Using the first paired dataset as training data, the diffusion model is trained in its first iteration using this loss function. During the iteration, details of the device defects are reconstructed, and reconstructed frames containing these defect details are output. The loss function includes a perceptual loss function, which extracts features based on the VGG16 network and is used to measure visual perceptual consistency.

[0029] The first iteration training process is specifically as follows: Set the number of iterations, fix the noise estimation network in the first few iterations, train the defect feature attention branch, and jointly train the noise estimation network and the defect feature attention branch in the remaining iterations.

[0030] The core objective of the DDPM network model's loss function is to balance pixel-level reconstruction accuracy with visual perception consistency, addressing the loss of subtle defect features (such as cracks <0.2mm) after power line inspection video compression, and providing high-fidelity reconstructed frames for subsequent super-resolution enhancement and defect identification. Its overall structure employs a weighted fusion of "pixel-level loss + feature-level perceptual loss." The DDPM network model's loss function... The formula is as follows: ; in: This represents the pixel-level mean squared error loss, used to minimize the pixel error between the reconstructed frame and the original frame, ensuring the restoration of basic details; λ represents the perceptual loss, which is based on the VGG16 network to extract high-level visual features and measure the visual perceptual consistency between the reconstructed frame and the original frame, avoiding "correct pixels but distorted structure"; λ represents the perceptual loss weight coefficient, which is used to balance the contribution of the two types of loss. After experimental optimization, the value is set to 0.1 (taking into account both detail accuracy and structural fidelity).

[0031] The construction process of the EDSR network model is as follows: Obtain reconstructed frames containing defective details, and construct a second paired dataset of reconstructed frames and original frames; An edge detection branch is added to the EDSR network model to locate the edges of key structures of the device during the feature extraction process of the feature extraction network in the EDSR network model. An EDSR network model loss function is constructed. The super-resolution model is trained in the second iteration using the second paired dataset as training data through the EDSR network model loss function. During the iteration process, the resolution is reconstructed and enlarged, and high-definition enhanced frames are output.

[0032] The EDSR network model loss function balances pixel-level error correction with device structure edge fidelity, addressing potential structural distortion issues in the output frames of diffusion models while simultaneously improving resolution. The EDSR network model loss function... The specific formula is as follows: ; in, This represents pixel-level L1 loss, used to reduce pixel errors between reconstructed frames and original high-definition frames, and to suppress block artifacts remaining in compressed video. The edge loss is represented by extracting the key structural edges of the equipment (such as the outline of the insulator skirt and the connection edge of the clamp) based on Canny edge detection, and constraining the edge consistency between the reconstructed frame and the original frame to avoid structural distortion. This represents the edge loss weight coefficient, which was optimized to a value of 0.3 through experiments (prioritizing structural fidelity while not sacrificing pixel-level accuracy).

[0033] The second iteration training process is as follows: Set the number of iterations, and test the effect of reconstructing and scaling the resolution on the validation set after every certain number of rounds. Stop iterating when the edge loss on the validation set is less than a preset threshold.

[0034] In step 3, the defect identification model is the YOLOv8 defect identification model, which is pre-trained with a device defect dataset.

[0035] This embodiment also provides a power inspection video diffusion super-resolution collaborative enhancement device for equipment defect identification, including: The video acquisition and compression module is used to acquire power inspection video data, compress the power inspection video data, and output the compressed power inspection video data. The decoding module has its input end connected to the output end of the video acquisition and compression module, and is used to receive the compressed power inspection video data, decode it, and output the decoded data. The diffusion model module, whose input is connected to the output of the decoding module, is used to reconstruct the details of the device defect in the decoded data and output the reconstructed data containing the defect details. The super-resolution model module, whose input is connected to the output of the diffusion model module, is used to optimize the video information of the equipment defect part in the reconstructed data containing defect details, and output the optimized video data. The defect identification module, whose input is connected to the output of the super-resolution model module, is used to receive the optimized video data and complete the detection and classification of equipment defects.

[0036] Specifically, the technical solution of this embodiment is applied to the unmanned aerial vehicle (UAV) inspection of power equipment, especially high-voltage insulators, clamps, and transformer bushings, as follows: The overall framework process is as follows: 1. Encoding end: The industry standard H.265 encoding algorithm is used to perform high-intensity compression (bitrate 300-500kbps) on the original inspection video (1080P, 30fps) collected by the drone to ensure that the video can be transmitted in real time through the wireless link; 2. Decoding end: After decoding the compressed video, it is input into the diffusion and super-resolution collaborative enhancement module. First, the defect details are reconstructed through the diffusion model, and then the structure and resolution are optimized through the super-resolution model. 3. Recognition end: Input the enhanced high-definition video (1080P / 4K) into the YOLOv8 defect recognition model to complete the detection and classification of equipment defects.

[0037] The specific steps for drone inspection of high-voltage insulators, clamps, and transformer bushings are as follows: Step 1: Dataset Preparation Raw video capture: Using a DJI M300RTK drone equipped with a Zenmuse P1 camera, the high-voltage insulators (model XP-70) of a 220kV transmission line were inspected in three environments: sunny, cloudy, and light rain. 100 raw video clips were captured (each clip is 3 minutes long, 1080P resolution, 30fps), including 40 clips of "normal insulators", 30 clips of "cracked insulators", and 30 clips of "damaged insulators". Creating compressed video: Use FFmpeg to compress the original video using H.265 encoding, set the bitrate to 400kbps, and generate a "compressed video dataset"; Dataset partitioning: The "compressed video and original video" paired data were divided into a training set (70 segments), a validation set (20 segments), and a test set (10 segments) in a 7:2:1 ratio. The video frames were extracted into single-frame images (approximately 540,000 frames in total) for model training.

[0038] Step 2: Training the diffusion model submodule Network initialization: An improved DDPM network was built based on the PyTorch framework. The defect feature attention branch adopted a 3-layer convolution + Softmax attention mechanism. The input frame size was 540P (960×540). Training process: Rounds 1-50: Fix the noise estimation network parameters, train only the defect feature attention branch, and learn the location and morphological features of insulator defects (cracks, damage); Rounds 51-200: Jointly train the noise estimation network and the attention branch, using MSE and perceptual loss (based on VGG16 relu3_3 layer features). Every 10 rounds, test the reconstruction accuracy on the validation set. Training stops when the MSE loss of the validation set reconstruction is ≤0.005. Model saving: Save the weights of the trained diffusion model (named diffusion_insulator.pth).

[0039] Step 3: Training of Super-Resolution Model Sub-modules Network initialization: An improved EDSR network was built based on the PyTorch framework, with the addition of a Canny edge detection branch (edge ​​threshold set to 50 / 150). The input frame size was 540P, and the output frame size was 1080P (2x super-resolution). Training process: Input data: The compressed frames of the training set are input into the trained diffusion model to obtain preliminary reconstructed frames, which are then used as input to the super-resolution model; Training iterations: Using L1+ edge loss, the super-resolution performance is tested on the validation set every 15 rounds. Training stops when the edge loss on the validation set is ≤0.003 (for a total of 120 rounds). Model saving: Saves the weights of the super-resolution model after training.

[0040] Step 4: Enhanced Collaboration Take one segment of "compressed video of cracked insulator under light rain" (bitrate 400kbps, 540P) from the test set and decode it into a single frame image using an H.265 decoder; Input the decoded frame into the diffusion model, set the iterative denoising rounds to 150 rounds, and output the initial reconstructed frame; The initial reconstructed frame is input into the super-resolution model, super-resolution is performed by 2 times, and 1080P high-definition enhanced frame is output. The enhanced frames are stitched together in chronological order to form a high-definition video (30fps).

[0041] Step 5: AI Recognition Test The enhanced high-definition video was input into the YOLOv8 defect recognition model (pre-trained on the insulator defect dataset). Statistical identification results: The identification accuracy of cracked insulators was 92%, the false detection rate was 4.5%, and the missed detection rate was 7%. Compared with the unenhanced compressed video (identification accuracy was 65%, false detection rate was 22%, and missed detection rate was 18%), the performance was significantly improved.

[0042] Example 2: A video diffusion super-resolution collaborative enhancement method for power inspection aimed at equipment defect identification. Its principle and implementation method are basically the same as those of Example 1. The difference is that the diffusion model is the SVD network model and the super-resolution model is the RCAN network model.

[0043] Compared to Example 1, replacing the DDPM network model with the SVD network model supports inter-frame temporal consistency modeling and can further reduce inter-frame jitter after video enhancement. However, it requires significantly more training data, exceeding 150% compared to the DDPM network model. Replacing the EDSR network model with the RCAN network model, which focuses on key features through channel attention, further improves edge fidelity. However, it increases the number of model parameters by 30% and reduces inference speed by 15%.

[0044] Example 3: A collaborative enhancement method for power inspection video diffusion and super-resolution for equipment defect identification. The implementation method differs from the previous example in that the diffusion model and super-resolution model are deployed on a GPU cluster. Step 2 is replaced by: a decoding module decoding the compressed power inspection data to obtain decoded data, optimizing the video information of the equipment defect portion using a mocking model, and then reconstructing the details of the equipment defect portion using the diffusion module. Specifically, compared to Example 1, first using super-resolution to enlarge the compressed frame to 1080P, and then using the diffusion model to supplement details, reduces the input size of the diffusion model and improves inference speed.

[0045] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0046] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0047] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A video diffusion super-resolution collaborative enhancement method for power inspection aimed at equipment defect identification, characterized by: Includes the following steps: Step 1: Collect power inspection video data, compress the power inspection video data, and send the compressed power inspection video data to the decoding module; Step 2: The decoding module decodes the compressed power inspection data to obtain decoded data. The decoded data is then reconstructed using a diffusion model to reconstruct the details of the equipment defects. During the noise estimation network denoising process of the diffusion model, the defective areas of the equipment are simultaneously located and details are supplemented by an added defect feature attention branch. The diffusion model is trained using a first paired dataset consisting of compressed frames and original frames as training data for a first iteration. The first iteration training process specifically involves: setting the number of iterations, fixing the noise estimation network and training the defect feature attention branch in the first few iterations, and jointly training the noise estimation network and the defect feature attention branch in the remaining iterations. Then, the video information of the equipment defects is optimized using a super-resolution model. During the feature extraction process of the super-resolution model, the edges of key equipment structures are located by an added edge detection branch. The super-resolution model is trained using a second paired dataset consisting of reconstructed frames output by the diffusion model and original frames as training data for a second iteration. The second iteration training process specifically involves: setting the number of iterations, testing the effect of reconstructed and magnified resolution on a validation set after every few rounds, and stopping the iteration when the edge loss on the validation set is less than a preset threshold. Step 3: Input the video data output by the super-resolution model into the defect recognition model to complete the equipment defect detection and classification.

2. The power inspection video diffusion super-resolution collaborative enhancement method for device defect recognition according to claim 1, characterized in that, The diffusion model is an improved DDPM network model built on the PyTorch framework, and the super-resolution model is an improved EDSR network model built on the PyTorch framework.

3. The power inspection video diffusion super-resolution collaborative enhancement method for device defect recognition according to claim 2, characterized in that, The loss function of the DDPM network model includes a perceptual loss function, which is based on features extracted from the VGG16 network and is used to measure visual perceptual consistency.

4. The power inspection video diffusion super-resolution collaborative enhancement method for device defect recognition according to claim 1, characterized in that, In step 3, the defect identification model is the YOLOv8 defect identification model, which is pre-trained with a device defect dataset.

5. The power inspection video diffusion super-resolution collaborative enhancement method for equipment defect identification according to claim 1, characterized in that, The diffusion model described is the SVD network model.

6. The power inspection video diffusion super-resolution collaborative enhancement method for equipment defect identification according to claim 1, characterized in that, The super-resolution model described is the RCAN network model.

Images