A yolov10-based power operation image detection distillation method

Abstract

The application provides a power operation image detection distillation method based on yolov10, comprising: acquiring a power operation image data set; training a teacher model by using the power operation image data set; freezing parameters of the teacher model, and distilling knowledge of the teacher model to a student model. By using the teacher model YOLOv10-X similar to the student model YOLOv10-N structure to perform knowledge distillation, the application ensures that the probability distribution of the two in the feature space has high similarity, which helps the student model to better learn and adapt to the feature representation of the teacher model; the application uses a similarity distillation loss between pixels to constrain the student model, so that the student model can learn the powerful feature extraction capability of the teacher model; the student model can not only improve the capturing capability of local details of an image, but also improve the robustness and precision in a complex scene, and is suitable for application scenarios requiring detection speed and accuracy in power production operations.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a power operation image detection distillation method based on yolov10. BACKGROUND

[0002] Target detection is an important technology in the field of computer vision, which aims to identify and locate specific objects in images or videos. Target detection requires the model to identify all objects in the image and give their positions, usually in the form of a bounding box.

[0003] Convolutional Neural Networks (CNN) is one of the core foundations of target detection. Compared with traditional machine learning algorithms, CNN can automatically extract hierarchical features of images, especially in edge detection, texture analysis and shape recognition. For example, LeNet, AlexNet, VGG, ResNet, etc. have become the backbone network of target detection model, which is used to extract effective features in images.

[0004] In the field of target detection, common detection algorithms can be divided into single-stage and multi-stage detection methods according to the execution process. Single-stage detection method directly simplifies the target detection task into a regression problem, and directly generates the class and bounding box of the target after extracting the features from the image. The goal of this method is to improve the detection speed, so it is particularly suitable for real-time application scenarios, but the relative accuracy is lower, especially for small objects or dense scene detection. Multi-stage detection method divides the target detection into multiple steps, gradually improves the detection accuracy through candidate region generation and candidate region refinement. Multi-stage detection method usually generates a set of candidate regions that may contain targets, and then refines these candidate regions. Multi-stage detection can achieve higher detection accuracy by gradually refining the bounding box and class label, especially suitable for small object and complex background detection. However, due to the multi-step detection process of multi-stage detection method, especially the need to generate candidate regions, the calculation is large, and it is difficult to realize real-time detection; at the same time, the multi-stage detection algorithm has complex structure, high resource demand in training and inference, and is not easy to deploy. SUMMARY

[0005] In view of the shortcomings of the prior art, the present application provides a power operation image detection distillation method based on yolov10. The present application "distills" the knowledge of yolov10-X model to yolov10-N model, so as to use yolov10-N model and improve the accuracy and performance under the premise of extremely limited resources.

[0006] The technical scheme of the present application is as follows: a power operation image detection distillation method based on yolov10, comprising the following steps:

[0007] S1), obtaining power operation image data set;

[0008] S2), training a teacher model;

[0009] S3), freezing the teacher model parameters, and distilling the knowledge of the teacher model to the student model.

[0010] Preferably, in step S2), the teacher model is trained, specifically:

[0011] S21), feature extraction of the operation image is performed through the backbone network of the teacher model to obtain a feature map;

[0012] S22), multi-scale feature fusion is performed on the feature map through the head network of the teacher model, a prediction result is output, and the prediction result is compared with the real frame to constrain the teacher model training through loss.

[0013] Preferably, in step S3), the knowledge distillation specifically includes the following steps:

[0014] S31), freezing the teacher model parameters, inputting the image into the backbone networks of the teacher model and the student model to perform feature extraction respectively, and obtaining feature map T t and feature map T s respectively;

[0015] S32), feature map T t and feature map T s are input into the head networks of the teacher model and the student model respectively for feature fusion, a prediction result is output, and loss is performed on the prediction result.

[0016] Preferably, in steps S2 and S3), the teacher model and the student model both adopt yolov10 model, and the network layers and parameter amounts of the teacher model and the student model are different.

[0017] Preferably, in step S2), the backbone network of the teacher model adopts an improved version of the cross-stage partial network CSPNet, and the transmission of gradient flow is optimized through the cross-stage partial connection strategy.

[0018] Preferably, in step S2), in the training process of the teacher model, the head network of the teacher model uses a one-to-many and one-to-one matching strategy to generate rich supervision signals.

[0019] Preferably, in step S2), the one-to-many matching strategy assigns multiple prediction frames to each real frame.

[0020] As preferred, in step S2), the head network of the teacher model is composed of two sub-networks with the same structure but independent parameters, respectively responsible for one-to-many matching and one-to-one matching; in the training process, the two sub-networks participate in optimization at the same time; in the inference stage, only the one-to-one matching network is used for prediction.

[0021] As preferred, in step S3), a global distillation loss is used to optimize the feature maps of the teacher model and the student model.

[0022] As preferred, in step S3), a pixel similarity distillation loss is used to constrain the student model.

[0023] The beneficial effects of the present application are:

[0024] 1. The present application can use the yolov10-N model and improve the accuracy by "distilling" the knowledge of the yolov10-X model to the yolov10-N model under the premise of extremely limited resources.

[0025] 2. The present application can realize real-time detection and early warning with lower delay while ensuring high accuracy, which is suitable for application scenarios in power production operations that require detection speed and accuracy.

[0026] 3. The distilled YOLOv10-N model can achieve detection effects comparable to larger models without relying on large-scale computing resources, making it possible to be deployed in resource-limited edge devices or embedded systems, and improving the feasibility of the model in practical applications.

[0027] 4. The present application uses a teacher model YOLOv10-X similar in structure to the student model YOLOv10-N for knowledge distillation, ensuring that the probability distribution of the two in the feature space has high similarity, which helps the student model to better learn and adapt to the feature representation of the teacher model.

[0028] 5. The present application uses a pixel similarity distillation loss to constrain the student model, enabling it to learn the powerful feature extraction capability of the teacher model; the student model can not only improve the ability to capture local details of images, but also improve robustness and accuracy in complex scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The flowchart of the method of the present application is shown in the figure;

[0030] Figure 2 The flowchart of the method of the present application is shown in the figure;

[0031] Figure 3 The framework structure diagram of the present application yolov10 is shown in the figure;

[0032] Figure 4 The result diagram of model identification before and after distillation in the embodiment of the application is shown in the figure;

[0033] Figure 5 The performance curve of the model after distillation in the embodiment of the application is shown in the figure. DETAILED DESCRIPTION

[0034] The specific embodiments of the application will be further described in combination with the accompanying drawings:

[0035] As shown in the figures, Figure 1 and 2 The embodiment provides a power operation image detection distillation method based on yolov10, which comprises the following steps:

[0036] S1), obtaining a power operation image dataset;

[0037] After obtaining the power operation image in the embodiment, the label information corresponding to each image is extracted; and the labels are converted into JSON format for subsequent processing. The generated JSON file is further converted into a COCO dataset format label, and in the label conversion process, the position information of the bounding box needs to be normalized; specifically, the coordinates of the bounding box are converted from pixel coordinates to relative coordinates; that is, the coordinate values of the frame are normalized with respect to the image width and height. This helps the model to better adapt to different resolution image inputs and improves the generalization ability of the model.

[0038] S2), training a teacher model; specifically comprising the following steps:

[0039] S21), extracting features of the operation image through the backbone network of the teacher model to obtain a feature map;

[0040] S22), performing multi-scale feature fusion on the feature map through the head network of the teacher model to output a prediction result, and comparing the prediction result with a real frame to constrain the teacher model training through a loss.

[0041] S3), freezing the parameters of the teacher model and distilling the knowledge of the teacher model to a student model; specifically comprising the following steps:

[0042] S31), freezing the parameters of the teacher model, inputting the image into the backbone networks of the teacher model and the student model to respectively extract features and respectively obtain a feature map T t and a feature map T s ;

[0043] S32), inputting the feature map T t and the feature map T sThe head networks of the teacher model and the student model are input respectively to perform feature fusion, output a prediction result, and perform loss on the prediction result.

[0044] As preferred in the embodiment, the teacher model and the student model both adopt a yolov10 model, and the network layers and parameter quantity of the teacher model and the student model are different, and the network layers and parameter quantity of the student model are significantly reduced compared with the teacher model. By reducing the model complexity, the student model can realize higher inference efficiency and resource utilization rate under the premise of maintaining high accuracy.

[0045] As preferred in the embodiment, as shown in Figure 3 The backbone network of the teacher model adopts an improved cross-stage partial network CSPNet.

[0046] The cross-stage partial network CSPNet includes a plurality of convolution layers Conv, a C2f module, an SCDown module, a C2fCIB module, an SPPF module, and a PSA module.

[0047] The C2f module is used for feature extraction and conversion to generate more expressive output.

[0048] The SCDown module reduces the calculation overhead and parameter quantity by optimizing the downsampling method.

[0049] The C2fCIB module replaces the convolution operation with group convolution to improve the calculation efficiency.

[0050] The SPPF module is an enhanced version of SPP, which optimizes the pooling operation, reduces the calculation burden, and maintains the ability of multi-scale target detection.

[0051] The PSA module combines channel and spatial attention mechanisms, polarizes high-resolution information, reduces information loss, and enhances the processing ability of non-linear distribution.

[0052] The head network of the teacher model uses multi-level feature integration to improve the accuracy of target detection and generates the final prediction result. The prediction output includes the position and category information of the bounding box, which will be compared with the real box to guide the learning of the model.

[0053] And in the training process of the teacher model, the head network uses one-to-many and one-to-one matching strategies to generate rich supervision signals. The one-to-many matching strategy can enhance the supervision strength and facilitate feature learning and optimization process by assigning multiple predicted boxes to each real box. The one-to-one matching strategy is used to avoid redundant operations in the non-maximum suppression (NMS) post-processing stage during prediction inference, improving the prediction efficiency. Specifically, the head network consists of two sub-networks with the same structure but independent parameters, which are responsible for one-to-many matching and one-to-one matching, respectively. In the training process, the two sub-networks participate in optimization at the same time to improve the overall performance of the model; while in the inference stage, only the one-to-one matching network is used for prediction, thus simplifying the inference process and improving the inference speed.

[0054] As preferred in the embodiment, in step S3), in the knowledge distillation process, the parameters of the teacher model are frozen, and the feature maps T t and T s extracted by the backbone networks of the teacher model and the student model are of the same size, representing the spatial representations of the input image by different models, and global distillation loss is used to optimize the feature maps of the teacher model and the student model. Specifically:

[0055] The effect of the network is improved by finding the correlation between pixels, and represents the similarity between the i-th pixel and the j-th pixel generated by the teacher network T, and represents the similarity between the i-th pixel and the j-th pixel generated by the student network S. The embodiment adopts the square difference form to construct the pairwise similarity distillation loss L pair , that is:

[0056]

[0057] In the formula, W' and H' represent the width and height of the feature map;

[0058] The similarity a ij between the two pixels is calculated by the features of the two pixels, that is:

[0059]

[0060] In the formula, f i and f j represent the i-th pixel and the j-th pixel feature, is the inner product of the feature vector, ||f i ||2and ||f j ||2are the L2 norm of the feature vector; the superscript T represents the transpose operation.

[0061] Under the model distillation framework, the feature map T tand feature map T s Feature extraction and fusion are performed by the head network of the teacher model and the student model, respectively. Then, the output p t As a soft label, the cross-entropy loss function is used to match the prediction p s of the student model to achieve soft supervision in the knowledge distillation process, that is:

[0062] L dist = CrossEntropy(p t , p s )

[0063] In the formula, L dist represents the supervision loss of the soft label; CrossEntropy represents the cross-entropy loss function.

[0064] At the same time, the student model will further optimize the model parameters based on the real label y through the standard supervised learning strategy during the training process, and balance the two losses by weighting with the coefficient a, that is:

[0065] L label = CrossEntropy(y, p s )

[0066] L = a x L dist + (1-a) x L label

[0067] In the formula, L label represents the supervision loss of the real label, and L represents the overall loss of the model; L dist represents the supervision loss of the soft label.

[0068] Through this weighting mechanism, the student model can effectively absorb knowledge from the teacher model, while also maintaining strong supervision on the real label, thereby achieving the best balance of optimizing performance.

[0069] The qualitative evaluation results of the model trained by the knowledge distillation in the target detection task on the power operation dataset show that the student model trained by the knowledge distillation technology has significantly improved performance in the power operation scene, especially in the precision and robustness of target detection in complex scenes. Compared with the student model without distillation, the distilled model can better capture the key targets in power operation, especially in small target detection and occlusion, and the distilled model performs more stably and accurately, as shown in Figure 4 .

[0070] As shown in Table 1 and Figure 5The quantitative evaluation results of the target detection task on the electric power operation dataset of the model trained by the knowledge distillation are shown, and the distillation process effectively improves the detection accuracy of the student model. Even with limited computing resources and fewer parameters, the success rate of target detection is significantly improved. Specifically, the distillation student model has better performance in terms of mean average precision (mAP) and recall than the model without distillation training.

[0071] Table 1 Comparison of model performance before and after distillation

[0072]

[0073] The above examples and descriptions in the specification are only to illustrate the principles and best modes of the present application, and various changes and improvements can be made to the present application without departing from the spirit and scope of the present application. These changes and improvements are all within the scope of the claimed present application.

Claims

1. A method for distillation detection in power operation images based on YOLOv10, characterized in that, Includes the following steps: S1) Obtain a dataset of power operation images; S2) Train the teacher model using a power operation image dataset; S3) Freeze the parameters of the teacher model and distill the knowledge of the teacher model into the student model; The feature maps of the teacher and student models are optimized by using global distillation loss; the student model is constrained by pixel similarity distillation loss. set up Let the similarity between the i-th pixel and the j-th pixel generated by the teacher network T be denoted as follows: The similarity between the i-th pixel and the j-th pixel generated by the student network S is represented; the pairwise similarity distillation loss L is constructed using the squared difference form. pair ,Right now: In the formula, W′ and H′ represent the width and height of the feature map; Similarity a between two pixels ij The calculation is performed using the features of two pixels, namely: In the formula, f i and f j Let i and j represent the features of the i-th and j-th pixels, respectively. It is the inner product of the eigenvectors, ||f i ||2 and||f j ||2 represents the L2 norm of the eigenvectors; superscript Indicates the transpose operation; Within the model distillation framework, the feature map T t and feature map T s Feature extraction and fusion are performed using the head networks of the teacher model and the student model, respectively. Next, the output p generated by the teacher model t As a soft label, the prediction result p of the student model is compared with the cross-entropy loss function. s Matching is performed to achieve soft supervision in the knowledge distillation process, namely: L dist =CrossEntropy(p t ,p s ) In the formula, L dist The soft-label supervision loss is represented by CrossEntropy; Meanwhile, during training, the student model, based on the true label y, further optimizes the model parameters using a standard supervised learning strategy, and balances the two losses by weighting the coefficients a. L label =CrossEntropy(y,p s ) L=a×L dist +(1-a)×L label In the formula, L label L represents the true label supervision loss, and L represents the overall loss; dist This indicates the loss of soft label supervision.

2. The method for distillation of power operation image detection based on YOLOv10 according to claim 1, characterized in that, The teacher model is trained as follows: S21) Feature maps are obtained by extracting features from power operation images through the backbone network of the teacher model; S22) The feature maps are fused at multiple scales through the head network of the teacher model to output the prediction results, which are then compared with the ground truth boxes. The loss is used to constrain the training of the teacher model.

3. The method for distillation of power operation image detection based on YOLOv10 according to claim 1, characterized in that, In step S3), the knowledge distillation specifically includes the following steps: S31) Freeze the parameters of the teacher model, and input the power operation images into the backbone networks of the teacher model and the student model respectively for feature extraction, and obtain feature maps T respectively. t and feature map T s ; S32), the feature map T t and feature map T s The head networks of the teacher model and the student model are respectively input for feature fusion, the prediction results are output, and the loss is applied to the prediction results.

4. The method for distillation of power operation image detection based on YOLOv10 according to claim 1, characterized in that, In steps S2 and S3), both the teacher model and the student model use the YOLOv10 model, but the number of network layers and parameters of the teacher model and the student model are different.

5. The method for detecting and distilling power operation images based on YOLOv10 according to claim 4, characterized in that, In step S2), the backbone network of the teacher model adopts an improved cross-stage partial network CSPNet, which optimizes the propagation of gradient flow through a cross-stage partial connection strategy.

6. The method for detecting and distilling power operation images based on YOLOv10 according to claim 5, characterized in that, In step S2), the head network of the teacher model consists of two sub-networks with the same structure but independent parameters, which are responsible for one-to-many matching and one-to-one matching respectively; during training, the two sub-networks participate in optimization at the same time; during the inference phase, only the one-to-one matching network is used for prediction.

Images