Electric power operation image detection distillation method based on yov10

Abstract

The invention provides an electric power operation image detection and distillation method based on yov10. The method comprises the following steps: acquiring an electric power operation image data set; training a teacher model by using the power work image data set; and freezing parameters of the teacher model, and distilling knowledge of the teacher model to the student model. The teacher model YOLOv10-X similar to the student model YOLOv10-N in structure is used for knowledge distillation, it is ensured that probability distribution of the teacher model YOLOv10-X and the student model YOLOv10-N in the feature space has high similarity, and the student model can better learn and adapt to feature representation of the teacher model; according to the method, the similarity distillation loss between pixels is adopted to restrain the student model, so that the student model can learn the strong feature extraction capability of the teacher model; the student model not only can improve the capturing capability of local details of the image, but also can improve the robustness and precision in a complex scene, and is suitable for an application scene having requirements for the detection speed and accuracy in power production work.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a distillation method for power operation image detection based on yolov10. Background Art

[0002] Object detection is an important technology in the field of computer vision. Its task is to identify and locate specific objects in images or videos. Object detection requires the model to be able to identify all objects in the image and also to give the location of these objects, usually in the form of bounding boxes.

[0003] Convolutional neural network (CNN) is one of the core foundations of target detection. Compared with traditional machine learning algorithms, CNN can automatically extract hierarchical features of images, especially in edge detection, texture analysis and shape recognition. For example, LeNet, AlexNet, VGG, ResNet, etc. have become the backbone networks of target detection models, used to extract effective features in images.

[0004] In the field of target detection, common detection algorithms can be divided into single-stage and multi-stage detection methods according to the execution process. The single-stage detection method directly simplifies the target detection task into a regression problem. After extracting features from the image, it directly generates the category and bounding box of the target. The goal of this type of method is to improve the detection speed, so it is particularly suitable for real-time application scenarios, but the relative accuracy is low, especially for small objects or dense scenes. The detection performance is not as good as multi-stage detection. The multi-stage detection method divides target detection into multiple steps, and gradually improves the detection accuracy through candidate region generation, candidate region refinement, etc.; the multi-stage detection method usually first generates a set of candidate regions that may contain the target, and then performs fine detection on these candidate regions. Multi-stage detection can achieve higher detection accuracy by gradually refining the bounding box and category label, especially for the detection of small objects and complex backgrounds. However, since the detection process of the multi-stage detection method is divided into multiple steps, especially the need to generate candidate regions, the amount of calculation is large, and it is difficult to achieve real-time detection; at the same time, the multi-stage detection algorithm has a complex structure, high resource requirements during training and inference, and is not easy to deploy. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a distillation method for power operation image detection based on yolov10. The present invention "distills" the knowledge of the yolov10-X model to the yolov10-N model, thereby using the yolov10-N model and improving the accuracy and performance under the premise of extremely limited resources.

[0006] The technical solution of the present invention is: a distillation method for power operation image detection based on yolov10, comprising the following steps:

[0007] S1), obtaining a power operation image dataset;

[0008] S2), training teacher model;

[0009] S3), freeze the teacher model parameters and distill the teacher model’s knowledge to the student model.

[0010] Preferably, in step S2), the teacher model is trained as follows:

[0011] S21), extracting features of the homework image through the backbone network of the teacher model to obtain a feature map;

[0012] S22) Perform multi-scale feature fusion on the feature map through the head network of the teacher model, output the prediction result, and compare it with the real box, and constrain the teacher model training through loss.

[0013] Preferably, in step S3), the knowledge distillation specifically comprises the following steps:

[0014] S31), freeze the teacher model parameters, input the image into the backbone network of the teacher model and the student model for feature extraction, and obtain feature maps T respectively. t and feature map T s ;

[0015] S32), the feature map T t and feature map T s The head networks of the teacher model and the student model are input respectively for feature fusion, the prediction results are output, and the loss is calculated for the prediction results.

[0016] Preferably, in steps S2 and S3), the teacher model and the student model both adopt the yolov10 model, and the teacher model and the student model have different numbers of network layers and parameters.

[0017] Preferably, in step S2), the backbone network of the teacher model adopts an improved version of the cross-stage partial network CSPNet, and optimizes the transmission of the gradient flow through a cross-stage partial connection strategy.

[0018] Preferably, in step S2), during the training process of the teacher model, the head network of the teacher model uses one-to-many and one-to-one matching strategies to generate rich supervision signals.

[0019] Preferably, in step S2), the one-to-many matching strategy is implemented by assigning multiple prediction boxes to each real box.

[0020] Preferably, in step S2), the head network of the teacher model is composed of two sub-networks with the same structure but independent parameters, which are responsible for one-to-many matching and one-to-one matching respectively; during the training process, the two sub-networks participate in the optimization at the same time; in the inference stage, only the one-to-one matching network is used for prediction.

[0021] Preferably, in step S3), global distillation loss is used to optimize the feature maps of the teacher model and the student model.

[0022] Preferably, in step S3), the similarity distillation loss between pixels is used to constrain the student model.

[0023] The beneficial effects of the present invention are:

[0024] 1. The present invention "distills" the knowledge of the yolov10-X model to the yolov10-N model, thereby using the yolov10-N model and improving the accuracy under the premise of extremely limited resources;

[0025] 2. The present invention can achieve real-time detection and early warning with low latency while ensuring high accuracy, and is suitable for application scenarios in power production operations that require both detection speed and accuracy;

[0026] 3. The distilled YOLOv10-N model of the present invention can achieve detection results comparable to those of larger models without relying on large-scale computing resources, allowing it to be deployed in edge devices or embedded systems with limited resources, thus improving the feasibility of the model in practical applications;

[0027] 4. The present invention uses the teacher model YOLOv10-X, which has a similar structure to the student model YOLOv10-N, for knowledge distillation, thereby ensuring that the probability distributions of the two in the feature space have a high similarity, which helps the student model to better learn and adapt to the feature representation of the teacher model;

[0028] 5. The present invention uses the similarity distillation loss between pixels to constrain the student model, so that it can learn the powerful feature extraction ability of the teacher model; the student model can not only improve the ability to capture local details of the image, but also improve the robustness and accuracy in complex scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the process of the present invention;

[0030] Figure 2 It is a flow chart of the method of the present invention;

[0031] Figure 3 It is the framework structure diagram of yolov10 of the present invention;

[0032] Figure 4 Schematic diagram of the results of model recognition before and after distillation in an embodiment of the present invention;

[0033] Figure 5 It is a performance curve diagram of the model after distillation in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings:

[0035] like Figure 1 and 2 As shown, this embodiment provides a method for detecting and distilling electric power operation images based on yolov10, comprising the following steps:

[0036] S1), obtaining a power operation image dataset;

[0037] After acquiring the power work images, this embodiment extracts the label information corresponding to each image; and converts these labels into JSON format for subsequent processing. The generated JSON file is further converted into COCO dataset format labels, and during the label conversion process, the position information of the bounding box needs to be normalized; specifically, the coordinates of the bounding box are converted from pixel coordinates to relative coordinates; that is, the coordinate values ​​of the box are normalized relative to the image width and height. This helps the model better adapt to image inputs of different resolutions and improves the generalization ability of the model.

[0038] S2), training the teacher model; specifically includes the following steps:

[0039] S21), extracting features of the homework image through the backbone network of the teacher model to obtain a feature map;

[0040] S22) Perform multi-scale feature fusion on the feature map through the head network of the teacher model, output the prediction result, and compare it with the real box, and constrain the teacher model training through loss.

[0041] S3) Freeze the teacher model parameters and distill the teacher model knowledge to the student model; specifically includes the following steps:

[0042] S31), freeze the teacher model parameters, input the image into the backbone network of the teacher model and the student model for feature extraction, and obtain feature maps T respectively. t and feature map T s ;

[0043] S32), the feature map T t and feature map T sThe head networks of the teacher model and the student model are input respectively for feature fusion, the prediction results are output, and the loss is calculated for the prediction results.

[0044] As preferred in this embodiment, the teacher model and the student model both adopt the yolov10 model, and the teacher model and the student model have different network layers and parameter amounts, and the network layers and parameter amounts of the student model are significantly reduced compared to the teacher model. By reducing the model complexity, the student model can achieve higher reasoning efficiency and resource utilization while maintaining high accuracy.

[0045] As preferred in this embodiment, Figure 3 As shown, the backbone network of the teacher model adopts an improved version of the cross-stage partial network CSPNet.

[0046] The cross-stage partial network CSPNet includes multiple convolutional layers Conv, a C2f module, a SCDown module, a C2fCIB module, a SPPF module, and a PSA module.

[0047] Wherein, the C2f module is used for feature extraction and conversion to generate more expressive output;

[0048] The SCDown module reduces computational overhead and parameter quantity by optimizing the downsampling method;

[0049] The C2fCIB module replaces the convolution operation with group convolution to improve computational efficiency;

[0050] The SPPF module is an enhanced version of SPP, which optimizes the pooling operation, reduces the computational burden, and maintains the ability of multi-scale object detection;

[0051] The PSA module combines channel and spatial attention mechanisms to polarize and filter high-resolution information, reduce information loss and enhance the ability to handle nonlinear distributions.

[0052] The head network of the teacher model uses the integration of multi-level features to improve the accuracy of target detection and generate the final prediction results. The prediction output includes the location and category information of the bounding box, which will be compared with the true box to guide the learning of the model.

[0053] And during the training process of the teacher model, the head network uses one-to-many and one-to-one matching strategies to generate rich supervision signals. Among them, the one-to-many matching strategy can enhance the supervision strength by assigning multiple prediction boxes to each true box, which is beneficial to the feature learning and optimization process. The one-to-one matching strategy is used to avoid redundant operations in the non-maximum suppression (NMS) post-processing stage during prediction reasoning, thereby improving the prediction efficiency. Specifically, the head network consists of two sub-networks with the same structure but independent parameters, which are responsible for one-to-many matching and one-to-one matching respectively. During the training process, these two sub-networks participate in the optimization at the same time to improve the overall performance of the model; in the inference stage, only the one-to-one matching network is used for prediction, thereby simplifying the inference process and improving the inference speed.

[0054] As a preferred embodiment of this invention, in step S3), during the knowledge distillation process, the parameters of the teacher model are frozen, and the feature graphs T extracted by the backbone networks of the teacher model and the student model are t and feature map T s The same size represents the spatial representation of the input image by different models, and the global distillation loss is used to optimize the feature maps of the teacher model and the student model. Specifically:

[0055] By finding the correlation between pixels to improve the effect of the network, represents the similarity between the i-th pixel and the j-th pixel generated by the teacher network T, let Represents the similarity between the i-th pixel and the j-th pixel generated by the student network S. This embodiment uses the square difference form to construct the pairwise similarity distillation loss L pair ,Right now:

[0056]

[0057] Where W′ and H′ represent the width and height of the feature map;

[0058] The similarity between two pixels is a ij The calculation is performed using the features of two pixels, namely:

[0059]

[0060] In the formula, f i and f j Represent the features of the i-th pixel and the j-th pixel respectively, is the inner product of the eigenvectors, ||f i ||2 and ||f j ||2 are the L2 norms of the eigenvectors respectively; the superscript T indicates the transpose operation.

[0061] In the model distillation framework, the feature map T tand feature map T s The features are extracted and fused through the head networks of the teacher model and the student model respectively. Then, the output p generated by the teacher model t As a soft label, the prediction result p of the student model is compared with the cross entropy loss function. s Matching is performed to achieve soft supervision in the knowledge distillation process, namely:

[0062] L dist =CrossEntropy(p t ,p s )

[0063] Where, L dist represents the supervision loss of soft labels; CrossEntropy represents the cross entropy loss function.

[0064] At the same time, the student model will further optimize the model parameters based on the true label y through the standard supervised learning strategy during the training process, and balance the two losses by weighting them with the coefficient a, that is:

[0065] L label = CrossEntropy(y,p s )

[0066] L = a × L dist +(1-a)×L label

[0067] Where, L label represents the supervision loss of the true label, L represents the overall loss of the model; L dist represents the supervision loss of soft labels;

[0068] Through this weighting mechanism, the student model can effectively absorb knowledge from the teacher model while also maintaining strong supervision on the true labels, thus achieving the best balance for optimizing performance.

[0069] The qualitative evaluation results of the model trained by knowledge distillation in the target detection task on the power operation dataset in this embodiment show that the student model trained by knowledge distillation technology has significantly improved the performance in the power operation scenario, especially in the accuracy and robustness of target detection in complex scenarios. Compared with the student model without distillation, the distilled model can better capture the key targets in power operation, especially in the case of small target detection and occlusion. The performance of the distilled model is more stable and accurate, such as Figure 4 shown.

[0070] As shown in Table 1 and Figure 5As shown in the figure, the quantitative evaluation results of the model trained by knowledge distillation in the target detection task on the power operation dataset. The distillation process effectively improves the detection accuracy of the student model, and the success rate of target detection is significantly improved even with limited computing resources and a small number of parameters. Specifically, the mean average precision (mAP) and recall rate (Recall) of the student model after distillation are better than those of the model that has not been trained by distillation.

[0071] Table 1 Comparison of model performance before and after distillation

[0072]

[0073] The above embodiments and descriptions are only for illustrating the principles and best embodiments of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, all of which fall within the scope of the present invention to be protected.

Claims

1. A distillation method for power operation image detection based on yolov10, characterized in that: The steps include: S1), obtaining a power operation image dataset; S2), using the power operation image dataset to train the teacher model; S3), freeze the parameters of the teacher model and distill the knowledge of the teacher model to the student model.

2. The method for detecting and distilling electric power operation images based on yolov10 according to claim 1 is characterized in that: Train the teacher model as follows: S21), extracting features of the power operation image through the backbone network of the teacher model to obtain a feature map; S22) Perform multi-scale feature fusion on the feature map through the head network of the teacher model, output the prediction result, and compare it with the real box, and constrain the teacher model training through loss.

3. The method for detecting and distilling electric power operation images based on yolov10 according to claim 1 is characterized in that: In step S3), the knowledge distillation specifically includes the following steps: S31), freeze the parameters of the teacher model, input the power operation image into the backbone network of the teacher model and the student model respectively to extract features, and obtain feature maps T respectively. t and feature map T S ; S32), the feature map T t and feature map Y S The head networks of the teacher model and the student model are input respectively for feature fusion, the prediction results are output, and the loss is calculated for the prediction results.

4. The method for detecting and distilling electric power operation images based on yolov10 according to claim 1 is characterized in that: In steps S2 and S3), the teacher model and the student model both adopt the yolov10 model, and the number of network layers and the number of parameters of the teacher model and the student model are different.

5. The method for detecting and distilling electric power operation images based on yolov10 according to claim 4 is characterized in that: In step S2), the backbone network of the teacher model adopts an improved version of the cross-stage partial network CSPNet, which optimizes the transmission of gradient flow through the cross-stage partial connection strategy.

6. The method for detecting and distilling electric power operation images based on yolov10 according to claim 5 is characterized in that: In step S2), the head network of the teacher model is composed of two sub-networks with the same structure but independent parameters, which are responsible for one-to-many matching and one-to-one matching respectively; during the training process, the two sub-networks participate in the optimization at the same time; in the inference stage, only the one-to-one matching network is used for prediction.

7. The method for detecting and distilling electric power operation images based on yolov10 according to claim 3 is characterized in that: In step S3, the global distillation loss is used to optimize the feature maps of the teacher model and the student model.

8. The method for detecting and distilling electric power operation images based on yolov10 according to claim 7 is characterized in that: In step S3, the similarity distillation loss between pixels is used to constrain the student model.

9. The method for detecting and distilling electric power operation images based on yolov10 according to claim 8, characterized in that: Step S3), set represents the similarity between the i-th pixel and the j-th pixel generated by the teacher network T, let Represents the similarity between the i-th pixel and the j-th pixel generated by the student network S; the pairwise similarity distillation loss L is constructed in the form of squared difference pair ,Right now: Where W′ and H′ represent the width and height of the feature map; The similarity between two pixels is a ij The calculation is performed using the features of two pixels, namely: In the formula, f i and f j Represent the features of the i-th pixel and the j-th pixel respectively, is the inner product of the eigenvectors, ||f i ||2 and ||f j ||2 are the L2 norms of the eigenvectors; superscript Represents a transpose operation.

10. The method for detecting and distilling electric power operation images based on yolov10 according to claim 9 is characterized in that: Step S3), under the model distillation framework, the feature map T t and feature map T s Feature extraction and fusion are performed through the head networks of the teacher model and the student model respectively; Next, the output p generated by the teacher model t As a soft label, the prediction result p of the student model is compared with the cross entropy loss function. s Matching is performed to achieve soft supervision in the knowledge distillation process, namely: L dist =CrossEntropy(p t ,p s ) Where, L dist represents the soft label supervision loss; CrossEntropy represents the cross entropy loss function; At the same time, the student model further optimizes the model parameters based on the real label y during the training process through a standard supervised learning strategy, and balances the two losses by weighting with the coefficient a, namely: L label =CrossEntropy(y,p s ) L=a×L dist +(1-a)×L label Where, L label represents the real label supervision loss, L represents the overall loss; L dist represents the soft label supervision loss.

Images