Power adaptive inspection method and system based on improved YOLOv8 non-post-processing

Abstract

The invention discloses an improved YOLOv8 post-processing-free power adaptive inspection method and system, and the method comprises the steps: adding a gradient path branch on the basis of YOLOv8, and achieving feature fusion through cross-layer connection in combination with feature maps extracted from different resolution levels of an original backbone network; constructing a teacher-student model with aligned structures, calculating the output distribution of each structure between the teacher model and the student model by using a feature-based distillation method, and enabling the student model to achieve the performance close to that of the teacher model while compressing the parameter quantity; a double-prediction-head framework is introduced, one-to-one training and one-to-many training are simultaneously carried out during training, but only one-to-one training is started during reasoning. Compared with the prior art, through multi-scale gradient path branch optimization, structural consistency distillation and a double-prediction-head cooperative reasoning mechanism, a target detection system with high precision, low delay and no redundancy post-processing in a complex environment is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of power inspection, and in particular to a power adaptive inspection method and system based on improved YOLOv8 without post-processing. Background Art

[0002] As the core equipment of the national energy network, the safe and stable operation of the power system depends on the regular inspection of key facilities such as transmission lines and substations. Traditional power inspections are mostly performed manually or by drones and robots equipped with basic visual algorithms, but they still face severe challenges in complex scenarios. On the one hand, the small target defects in power equipment and the interference of complex environments result in insufficient detection accuracy of traditional algorithms, making it difficult to meet high reliability requirements; on the other hand, existing inspection equipment mostly relies on deep learning models, but their huge computing power makes it difficult to achieve real-time inference at the edge, and the delay often exceeds 100 milliseconds, which restricts the ability to quickly respond to sudden failures. In addition, the morphology of power equipment varies significantly, the generalization ability of traditional models is insufficient, and lightweight models are difficult to deploy directly due to the large loss of accuracy, which seriously hinders the implementation of large-scale intelligent inspections.

[0003] Early power inspection methods mainly relied on traditional image processing techniques such as threshold segmentation and template matching. Methods based on artificial feature design have poor robustness in dynamic environments and complex backgrounds, and are unable to adaptively learn target characteristics. With the development of deep learning, large models represented by Faster R-CNN and the YOLO series have significantly improved detection accuracy, but their high computational costs make it difficult to deploy the models on edge devices with limited computing power. In addition, the training process relies on massive amounts of labeled data, and the application cost remains high. In recent years, lightweight models have increased speed by compressing the number of parameters, but due to the lack of efficient feature distillation mechanisms and structural optimization, they cannot strike a balance between performance and efficiency. In addition, detection models usually use strategies such as non-maximum suppression and fixed thresholds for post-processing, which can easily lead to missed detection of dense targets or delayed detection results, making it difficult to meet the detection needs of scenarios with high-density distribution of power equipment. Summary of the Invention

[0004] Purpose of the invention: In response to the problems existing in the prior art, the present invention provides a power adaptive inspection method and system based on improved YOLOv8 without post-processing, aiming to realize a high-precision, low-latency, and non-redundant post-processing target detection system in complex environments through multi-scale gradient path branch optimization, structural consistency distillation framework and dual prediction head collaborative reasoning mechanism, so as to meet the reliability and efficiency requirements of intelligent power inspection.

[0005] Technical solution: The present invention provides a power adaptive inspection method based on improved YOLOv8 without post-processing, comprising the following steps:

[0006] Step 1: Obtain the power detection image to be processed and perform saliency processing;

[0007] Step 2: Build the target detection model. Based on the YOLOv8 framework, add a new gradient path branch. Combine the feature maps extracted at different resolution levels of the original YOLOv8 backbone network and achieve feature fusion through cross-layer connections.

[0008] Step 3: Build a structurally aligned Teacher-Student model. Use the object detection model built in step 2 as the Teacher model. Use a feature-based distillation method to calculate the output distribution of each structure between the Teacher model and the Student model. Adjust the network depth and width compression parameters to make the Student model achieve performance close to that of the Teacher model.

[0009] Step 4: Introduce a dual-prediction head architecture to replace the non-maximum suppression (NMS) post-processing to optimize the training process. The dual-prediction head architecture includes a one-to-one head and a one-to-many head. During training, both the one-to-one head and the one-to-many head are trained simultaneously, but only the one-to-one head is enabled during inference.

[0010] Step 5: Use the constructed and trained object detection model to detect the power detection image to be processed.

[0011] Furthermore, the gradient path branch includes multiple Conv convolution modules, multiple C2f modules, and multiple CBFuse modules, and is specifically implemented as follows:

[0012] The image after saliency processing first passes through two Conv convolution modules, the first C2f module, the Conv convolution module, the first CBFuse module, and the second C2f module to output the P3 feature layer. The feature maps output by the second to fourth C2f modules of the YOLOv8 backbone network pass through the CBLlinear module and are fused with the first CBFuse module.

[0013] After passing through the second C2f module of the Gradient Pathway Branch, it is input into the Conv convolution module, the second CBFuse module, and the third C2f module in sequence to output the P4 feature layer; the second CBFuse module also fuses the feature maps output by the third and fourth C2f modules of the YOLOv8 backbone network;

[0014] After passing through the third C2f module of the Gradient Pathway Branch, it is input into the Conv convolution module, the third CBFuse module, and the fourth C2f module in sequence, and then outputs the P5 feature layer;

[0015] The features of the P3, P4, and P5 feature layers are fused and input into the LabelAssigner module to assign labels and determine the location and category of the target.

[0016] Furthermore, the YOLOv8 basic framework also outputs P3 feature layer, P4 feature layer, and P5 feature layer after processing the saliency image. After feature fusion, it is input into the Task-AlignedAssigner module, and labels are assigned according to task alignment. The features are further processed for target detection.

[0017] Furthermore, the features output by the gradient path branch GradientPathway Branch and the original branch of the YOLOv8 basic framework are jointly input into the DualDet detection head module to perform the final detection task and output the detection results.

[0018] Furthermore, the Teacher-Student model constructed in step 3 is as follows:

[0019] Set the feature map F to be extracted at each key layer of Backbone and Neck t l and F s l , respectively represent the l-th layer feature maps of the Teacher model and the Student model, and perform channel normalization on the feature maps to eliminate dimensional differences;

[0020] The student model is forced to imitate the output of the teacher model. The intermediate feature distributions of the student and teacher are aligned through KL divergence. By simply adjusting the network depth and width, the student can learn the feature expression ability of the teacher, achieving lightweightness.

[0021] The KL divergence is used to force the normalized feature distribution of the Student corresponding layer to approach the Teacher, capturing the relative relationship between features:

[0022]

[0023] Among them, Loss distill is the distillation loss function, L is the number of distillation layers;

[0024] KL divergence formula expansion:

[0025]

[0026] in, Q=φ(F t l );

[0027] The difference between the output of the student model and the teacher model is quantified by the loss function to guide the learning direction of the student model. The loss function includes task loss and distillation loss:

[0028] Loss total =αLoss task +βLoss distill

[0029] Among them, Loss task is the task loss function, α and β are weight parameters;

[0030] The task loss function uses cross entropy loss to optimize the performance of the student model on the target task:

[0031]

[0032] Among them, y i is the true label, is the predicted probability of the student model.

[0033] Furthermore, during model training, the dual prediction head architecture in step 4 combines the category and position information between the two output prediction boxes to optimize the loss function:

[0034]

[0035] FL(p)=-(1-p) γ log(p)

[0036] in, is the predicted class probability and bounding box position and size, b gt is the actual bounding box position and size, α and β are two weight parameters between 0 and 1, express b gt The CIoU ratio between two predicted boxes.

[0037] The present invention also discloses an inspection system based on the above-mentioned power adaptive inspection method without post-processing based on the improved YOLOv8, comprising an image acquisition module, an image processing module and an image detection and recognition module;

[0038] The image acquisition module is used to acquire the power detection image to be processed;

[0039] The image processing module is used to perform saliency processing based on the power detection image to be processed;

[0040] The image detection and recognition module is used to build a target detection model. On the YOLOv8 basic framework, a new gradient path branch is added. The feature maps extracted at different resolution levels of the original YOLOv8 backbone network are combined to achieve feature fusion through cross-layer connections. A structurally aligned Teacher-Student model is constructed. A feature-based distillation method is used to calculate the output distribution of each structure between the Teacher model and the Student model. While compressing the number of parameters, the Student model achieves performance close to that of the Teacher model. A dual prediction head architecture is introduced. During training, both the one-to-one head and the one-to-many head are trained simultaneously, but only the one-to-one head is enabled during inference.

[0041] Beneficial effects:

[0042] 1. Based on the YOLOv8 framework, this paper adds a new Gradient Pathway Branch to detect objects in the input image at three different feature map resolution levels, enhancing the ability to detect multi-scale objects. At the same time, it calculates the objective function from the complete input, providing reliable gradient information for the backpropagation of the entire task, solving the problems of information loss and information bottlenecks.

[0043] 2. The general Teacher model and Student model share a consistent Backbone, Neck, and other structural designs. The Teacher model is a high-performance model based on an improved YOLOv8 (including gradient path branches) as a knowledge source, while the Student model achieves lightweightness by reducing the network depth and width within the same structural framework. Distillation loss (KL divergence) is used to force its performance to approach that of the Teacher model. Both the Teacher and Student models are based on networks built on an improved YOLOv8, with branches used to improve the basic YOLOv8 model. A feature-based distillation method is used to calculate the output distribution of each structure between the two, and the depth and width of the network are adjusted to enable the Student to achieve performance close to that of the Teacher.

[0044] 3. In the post-processing stage of the model output, the present invention designs a dual prediction head to solve the time-consuming NMS module. During training, the one-to-one head and the one-to-many head are trained simultaneously, but during inference, only the one-to-one head is relied upon, ensuring that each output corresponds to an actual target without any post-processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is the model structure diagram;

[0046] Figure 2 Schematic diagram of the Teacher-Student model structure;

[0047] Figure 3 Schematic diagram of the dual prediction head structure;

[0048] Figure 4 Statistics on the time consumption for inference and post-processing when four models of different specifications are running on the terminal. DETAILED DESCRIPTION

[0049] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0050] The present invention discloses a power adaptive inspection method and system based on improved YOLOv8 without post-processing, see Figures 1 to 3 The inspection system includes an image acquisition module, an image processing module, and an image detection and recognition module. The power adaptive inspection method based on the improved YOLOv8 without post-processing includes the following processes:

[0051] Step 1: Obtain the power detection image to be processed and perform saliency processing.

[0052] Step 2: Based on the YOLOv8 framework, a new Gradient Pathway Branch is added to solve the information loss and information bottleneck problems through multi-resolution feature fusion and gradient back propagation.

[0053] Image input is processed by YOLOv8 for saliency and then fed into both the Original Branch and the newly added Gradient Pathway Branch. The Original Branch retains YOLOv8's default feature extraction process, extracting multi-scale feature maps from different Backbone layers (high, medium, and low resolution), enhancing detail through cross-layer connections and dynamic fusion.

[0054] The feature maps extracted from the Backbone of the YOLOv8 framework are first inserted into the Gradient Pathway Branch through the CBLlinear module for cross-layer connections. After processing by modules such as convolution and C2f, the P3, P4, and P5 feature layers are gradually generated. The output of the branch enters the LabelAssigner module to assign labels and determine the location and category of the target. In the original branch, the feature maps undergo upsampling and concatenation to fuse features from different layers. After a series of C2f modules and convolution operations, the P3, P4, and P5 feature layers are generated. The fused features from the branch are then input into the Task-AlignedAssigner module, where labels are assigned based on task alignment and further processed for target detection.

[0055] The final processing results of the two branches will be input into the DualDet detection head module together to perform the final detection task and output the detection results, including information such as the target and position in the image.

[0056] In this embodiment, the Gradient Pathway Branch includes multiple Conv convolution modules, multiple C2f modules, and multiple CBFuse modules, and is specifically implemented as follows:

[0057] The image after saliency processing first passes through two Conv convolution modules, the first C2f module, the Conv convolution module, the first CBFuse module, and the second C2f module to output the P3 feature layer; the feature maps output by the second to fourth C2f modules of the YOLOv8 backbone network pass through the CBLlinear module respectively and are fused with the first CBFuse module.

[0058] After passing through the second C2f module of the Gradient Pathway Branch, it is input into the Conv convolution module, the second CBFuse module, and the third C2f module in sequence to output the P4 feature layer; the second CBFuse module also fuses the feature maps output by the third and fourth C2f modules of the YOLOv8 backbone network.

[0059] After passing through the third C2f module of the Gradient Pathway Branch, it is sequentially input into the Conv convolution module, the third CBFuse module, and the fourth C2f module before outputting the P5 feature layer.

[0060] The features of the P3, P4, and P5 feature layers are fused and input into the LabelAssigner module to assign labels and determine the location and category of the target.

[0061] Step 3: Build a structurally aligned Teacher-Student model and implement feature-based distillation to achieve lightweight and high-precision transfer. The Teacher model is a high-performance model based on an improved YOLOv8 (including gradient path branches) and serves as the knowledge source. The Student model, while maintaining the same structural framework, is lightweight by reducing the network depth and width. Distillation loss (KL divergence) is used to force its performance to approach that of the Teacher model. Both the Teacher and Student models are based on improved YOLOv8 networks, with branches used to improve the basic YOLOv8 model.

[0062] Based on the feature distillation strategy, the Teacher-Student framework is adopted to align the intermediate feature distribution of the structure:

[0063] Large model training and small model distillation: In the initial stage of model training, ensure that the core structures such as Backbone and Neck of the Teacher and Student models are fully aligned, providing an alignment basis for feature distillation and ensuring that the intermediate feature maps of the Teacher and Student are comparable in spatial dimensions and semantic levels.

[0064] Extract feature maps F at each key layer of Backbone and Neck t l (Teacher) and (Student), perform channel normalization on the feature map to eliminate dimensional differences:

[0065]

[0066] Among them, φ is the feature normalization function, μ(F l ),σ(F l ) are the mean and standard deviation of the feature map, and ε is a small constant to prevent division by zero;

[0067] The student model is forced to imitate the output of the teacher model, and the intermediate feature distributions of the student and teacher are aligned through the KL divergence (Kullback-Leibler Divergence). By only adjusting the network depth and width, the student can learn the feature expression ability of the teacher, achieving lightweight.

[0068] The KL divergence is used to force the normalized feature distribution of the Student corresponding layer to approach the Teacher, capturing the relative relationship between features:

[0069]

[0070] Among them, Loss distillis the distillation loss function, L is the number of distillation layers, F t l are the l-th layer feature maps of the student and teacher models respectively.

[0071] KL divergence formula expansion:

[0072]

[0073] in, Q=φ(F t l ).

[0074] The difference between the output of the student model and the teacher model is quantified by the loss function to guide the learning direction of the student model. The loss function includes task loss and distillation loss:

[0075] Loss total =αLoss task +βLoss distill

[0076] Among them, Loss task is the task loss function, and α and β are weight parameters.

[0077] The task loss function uses cross entropy loss to optimize the performance of the student model on the target task:

[0078]

[0079] Among them, y i is the true label, is the predicted probability of the student model.

[0080] See also Figure 2 , Figure 2 The following is a schematic diagram of the Teacher-Student model structure. Data represents the power inspection image data to be processed. After saliency processing in step 1, it is input into the model. Feature extraction and fusion are achieved using an improved YOLOv8 framework and gradient path branches. It also serves as a medium for knowledge transfer within the teacher-student model, ensuring that the lightweight student model inherits the detection capabilities of the teacher model. The intermediate features (Layer n) and logits of the teacher model guide the alignment of the intermediate features (Layer m) and logits of the student model through a distillation loss.

[0081] Step 4: Design a dual-prediction head architecture to jointly optimize the One-to-One Head and One-to-Many Head during training. During inference, only the One-to-One Head is used to directly output non-redundant detection results, completely eliminating the time-consuming NMS post-processing problem.

[0082] A dual detection head architecture is designed to replace non-maximum suppression (NMS) post-processing to optimize the training process and reduce the computational complexity in the inference and post-processing stages:

[0083] like Figure 4 As shown in the figure, the time consumption of the inference and post-processing phases when four models of different specifications are running on terminal devices is statistically analyzed. It is not difficult to find that when the model scale is large enough or the input size is relatively large, the overall model time consumption is almost entirely concentrated in the network inference phase, reaching a maximum proportion of over 92%. However, as the network scale gradually decreases, the proportion of time consumed by post-processing begins to increase, reaching a maximum of between 40% and 55%. During the entire model training process, post-processing accounts for nearly half of the total time. At the same time, another major factor affecting the deployment of detection models on terminals is that the input image size is generally close to 640. Due to the limited computing power of the device itself, it is necessary to improve the time consumption of NMS to improve the real-time performance of model detection.

[0084] In object detection tasks, NMS is used as a post-processing step to filter the model's prediction outputs and remove those that are duplicated or have insufficient confidence. If a model can be trained where each prediction output by the network uniquely corresponds to an actual object, the entire NMS module can be removed.

[0085] To achieve this goal, the network incorporates a dual prediction head design: one for one-to-one prediction and one for many. During training, both are trained simultaneously, but during inference, only the one-to-one head is used, ensuring that each output corresponds to a specific target without any post-processing. During model training, the loss function is optimized by combining the category and position information between the prediction boxes output by both heads:

[0086]

[0087] FL(p)=-(1-p) γ log(p)

[0088] in, is the predicted class probability and bounding box position and size, b gt is the actual bounding box position and size, α and β are two weight parameters between 0 and 1, express b gt The CIoU ratio between two predicted boxes.

[0089] In the power inspection scenario, this design enables the model to meet the high-density, low-latency industrial-grade detection requirements when used in terminal equipment.

[0090] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A power adaptive inspection method based on improved YOLOv8 without post-processing, characterized in that: The steps include: Step 1: Obtain the power detection image to be processed and perform saliency processing; Step 2: Build the target detection model. Based on the YOLOv8 framework, add a new gradient path branch. Combine the feature maps extracted at different resolution levels of the original YOLOv8 backbone network and achieve feature fusion through cross-layer connections. Step 3: Build a structurally aligned Teacher-Student model. Use the object detection model built in step 2 as the Teacher model. Use a feature-based distillation method to calculate the output distribution of each structure between the Teacher model and the Student model. Adjust the network depth and width compression parameters to make the Student model achieve performance close to that of the Teacher model. Step 4: Introduce a dual-prediction head architecture to replace the non-maximum suppression (NMS) post-processing to optimize the training process. The dual-prediction head architecture includes a one-to-one head and a one-to-many head. During training, both the one-to-one head and the one-to-many head are trained simultaneously, but only the one-to-one head is enabled during inference. Step 5: Use the constructed and trained object detection model to detect the power detection image to be processed.

2. The power adaptive inspection method based on improved YOLOv8 without post-processing according to claim 1 is characterized in that: The Gradient Pathway Branch includes multiple Conv convolution modules, multiple C2f modules, and multiple CBFuse modules, and is specifically implemented as follows: The image after saliency processing first passes through two Conv convolution modules, the first C2f module, the Conv convolution module, the first CBFuse module, and the second C2f module to output the P3 feature layer. The feature maps output by the second to fourth C2f modules of the YOLOv8 backbone network pass through the CBLlinear module and are fused with the first CBFuse module. After passing through the second C2f module of the Gradient Pathway Branch, it is input into the Conv convolution module, the second CBFuse module, and the third C2f module in sequence to output the P4 feature layer; the second CBFuse module also fuses the feature maps output by the third and fourth C2f modules of the YOLOv8 backbone network; After passing through the third C2f module of the Gradient Pathway Branch, it is input into the Conv convolution module, the third CBFuse module, and the fourth C2f module in sequence, and then outputs the P5 feature layer; The features of the P3, P4, and P5 feature layers are fused and input into the Label Assigner module to assign labels and determine the location and category of the target.

3. The power adaptive inspection method based on improved YOLOv8 without post-processing according to claim 1 is characterized in that: The YOLOv8 basic framework also outputs P3 feature layer, P4 feature layer, and P5 feature layer after processing the saliency image. After feature fusion, it is input into the Task-AlignedAssigner module, and labels are assigned according to task alignment. The features are further processed for target detection.

4. The power adaptive inspection method based on improved YOLOv8 without post-processing according to claim 2 or 3, characterized in that: The features output by the gradient path branch GradientPathway Branch and the original branch of the YOLOv8 basic framework are jointly input into the DualDet detection head module to perform the final detection task and output the detection results.

5. The power adaptive inspection method based on improved YOLOv8 without post-processing according to claim 1 is characterized in that: The Teacher-Student model constructed in step 3 is as follows: Set the feature map F to be extracted at each key layer of Backbone and Neck t l and F s l , respectively represent the l-th layer feature maps of the Teacher model and the Student model, and perform channel normalization on the feature maps to eliminate dimensional differences; The student model is forced to imitate the output of the teacher model. The intermediate feature distributions of the student and teacher are aligned through KL divergence. By simply adjusting the network depth and width, the student can learn the feature expression ability of the teacher, achieving lightweightness. The KL divergence is used to force the normalized feature distribution of the Student corresponding layer to approach the Teacher, capturing the relative relationship between features: Among them, Loss distill is the distillation loss function, L is the number of distillation layers; KL divergence formula expansion: in, The difference between the output of the student model and the teacher model is quantified by the loss function to guide the learning direction of the student model. The loss function includes task loss and distillation loss: Loss total =αLoss task +βLoss distill Among them, Loss task is the task loss function, α and β are weight parameters; The task loss function uses cross entropy loss to optimize the performance of the student model on the target task: Among them, y i is the true label, is the predicted probability of the student model.

6. The power adaptive inspection method based on improved YOLOv8 without post-processing according to claim 1 is characterized in that: In step 4, the dual prediction head architecture combines the category and position information between the two output prediction boxes to optimize the loss function during model training: FL(p)=-(1-p) γ log(p) in, is the predicted class probability and bounding box position and size, b gt is the actual bounding box position and size, α and β are two weight parameters between 0 and 1, express b gt The CIoU ratio between two predicted boxes.

7. An inspection system based on the improved YOLOv8-based power adaptive inspection method without post-processing according to any one of claims 1 to 6, characterized in that: It includes image acquisition module, image processing module and image detection and recognition module; The image acquisition module is used to acquire the power detection image to be processed; The image processing module is used to perform saliency processing based on the power detection image to be processed; The image detection and recognition module is used to build a target detection model. On the YOLOv8 basic framework, a new gradient path branch is added. The feature maps extracted at different resolution levels of the original YOLOv8 backbone network are combined to achieve feature fusion through cross-layer connections. A structurally aligned Teacher-Student model is constructed. A feature-based distillation method is used to calculate the output distribution of each structure between the Teacher model and the Student model. While compressing the number of parameters, the Student model achieves performance close to that of the Teacher model. A dual prediction head architecture is introduced. During training, both the one-to-one head and the one-to-many head are trained simultaneously, but only the one-to-one head is enabled during inference.

Images