An unmanned aerial vehicle image cloud edge collaborative identification method based on knowledge distillation

Abstract

The application discloses a kind of unmanned aerial vehicle image cloud edge collaborative identification method based on knowledge distillation, solve the deficiency of prior art, including step 1, construct teacher model, and teacher model is trained to be completely converged and have guiding ability;Step 2, construct student model, improve the loss function of student model so that student model can simultaneously accept from artificial label and soft label of teacher model;Step 3, teacher model and student model are jointly trained, student model simultaneously accepts artificial label and soft label of teacher model and iterates until student model is completely converged;Step 4, the student model of training completion is deployed to unmanned aerial vehicle terminal, so that student model is actually operated in edge end and detects identification effect;Step 5, if detection identification effect is judged as not up to standard, then the image of identification is transmitted to teacher model, and teacher model guides student model to carry out secondary training by cloud edge collaborative mode until identification effect is up to standard.

Description

Technical Field

[0001] This invention relates to the field of power Internet of Things in the power distribution domain, and in particular to a cloud-edge collaborative recognition method for UAV images based on knowledge distillation. Background Technology

[0002] Low-voltage distribution networks serve as the link between the power system and low-voltage electricity users; any failure in these networks can cause significant economic losses to the power system. Statistics show that the vast majority of power outages and electrical-related public safety incidents affecting electricity users occur on low-voltage distribution lines. However, the level of informatization regarding basic data such as line topology and routing in low-voltage distribution networks is far lower than that of high-voltage transmission networks and medium-voltage distribution networks. In particular, many early-built low-voltage distribution networks are in a state of significant gaps and chaotic opacity, posing substantial safety hazards and severely hindering routine maintenance, repairs, and power restoration after faults. Therefore, it is urgent to utilize information technology to supplement and improve basic data such as line routing, topology, and corresponding relationships in low-voltage distribution networks, and to conduct inspections of overhead lines based on this data.

[0003] In the context of the power Internet of Things (IoT) in the distribution sector, and considering that drone inspection has become an important part of high-voltage power line inspection, the application of drones in low-voltage distribution networks is likely. Drones have a wide field of vision and are less affected by terrain, effectively improving work efficiency compared to manual inspection. However, the level of intelligence of drones currently proposed for use in low-voltage distribution networks is generally low. Most rely on manual operation to capture videos, which are then imported into computers for manual hazard inspection after the drone returns. This method not only increases the workload of maintenance personnel but also leads to significant differences in inspection quality due to variations in the flight control capabilities of different personnel. Furthermore, during flight, drones are susceptible to wind interference, making flight attitude difficult to control. To ensure the safe operation of power lines, drones need to maintain a certain discharge gap distance from energized equipment (the State Grid Corporation of China requires a distance of at least 5 meters). Due to these limitations, drone images often suffer from incomplete video recordings due to unstable target recognition, difficulty in freely adjusting flight attitude, long shooting distances, and background interference, resulting in problems such as inaccurate equipment segmentation and low recognition rates in the images. Therefore, achieving highly reliable intelligent inspection of low-voltage overhead power distribution lines by drones has significant practical engineering implications.

[0004] However, drone terminals are typical low-power edge terminals with limited power consumption and computing resources. They typically deploy only lightweight, simple models, which generally have low generalization ability and recognition accuracy. Therefore, it is essential to enhance these lightweight models to ensure their reliability and usability. Summary of the Invention

[0005] The purpose of this invention is to overcome the problems of insufficient generalization ability and insufficient reliability in existing power drone image recognition methods, and to provide a drone image cloud-edge collaborative recognition method based on knowledge distillation.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A cloud-edge collaborative recognition method for UAV imagery based on knowledge distillation includes the following steps:

[0008] Step 1: Build a teacher model and train it until it fully converges and has the ability to provide guidance.

[0009] Step 2: Build a student model and improve the loss function of the student model so that the student model can accept soft labels from both human labels and teacher models.

[0010] Step 3: The teacher model and the student model are jointly trained using soft label distillation. The student model receives both manual labels and soft labels from the teacher model for iteration until the student model fully converges.

[0011] Step 4: Deploy the trained student model to the drone terminal so that the student model can actually run at the edge and test the recognition effect;

[0012] Step 5: If the recognition effect detected in Step 4 is deemed unsatisfactory, the recognized image is transmitted to the teacher model. The teacher model guides the student model to undergo secondary training through cloud-edge collaboration until the recognition effect meets the standard.

[0013] Preferably, the teacher model is a teacher model based on Faster R-CNN, and the student model is a student model based on YOLOtiny.

[0014] Preferably, the loss function of the student model consists of three parts: box regression loss, class classification loss, and confidence loss, specifically:

[0015]

[0016] Where K×K and M represent the dimensions of the input features, respectively, p i (c) represents the true category. Represents the predicted category; where p i (c) = sigmoid(z) i )

[0017] Among them, z i This represents the final output of the classification network;

[0018] Since knowledge distillation learning is required for the student model, in order to improve the probability distribution entropy of the sigmoid function output of both the teacher and student models, thereby softening the output classification label, this invention proposes a sigmoid function incorporating a temperature variable, as follows:

[0019]

[0020] Where T1 and T2 are parameters for controlling the temperature, with T1 controlling z i The displacement, T2 controls z i The scaling is adjusted to ensure that the classification labels are sufficiently softened. When T1 = 0 and T2 = 1, this formula is the ordinary sigmoid function.

[0021] To enable the student model to learn from both manually labeled ground truth labels and soft labels provided by the teacher model during image classification in object detection, the classification loss function of the student model is improved as follows:

[0022]

[0023] in, The soft label refers to the teacher's model under a temperature parameter T. This refers to the predicted soft label of the student model with temperature parameter T, where α and β are the weights of the true hard label and the teacher model soft label, respectively.

[0024] Preferably, in step 3, the teacher model and student model are jointly trained using soft label distillation. This requires aligning the soft labels of the teacher and student models. The alignment function is as follows:

[0025]

[0026] Where M and N represent the length and width of the input image for the student model, respectively, and M′ and N′ represent the length and width of the input image for the teacher model, respectively.

[0027] Preferably, in step 5, the method by which the teacher model guides the student model for secondary training through cloud-edge collaboration is as follows: Without human labels, a distillation learning mechanism is used to converge the student model towards the teacher model, fine-tuning the student model to achieve better results for the edge model. The calculation formula is as follows:

[0028]

[0029] The beneficial effects of this invention are:

[0030] This invention combines knowledge distillation technology with cloud-edge collaboration technology to propose a unique knowledge distillation technology suitable for the power Internet of Things. The big data-edge device architecture of cloud-edge collaboration technology can be effectively combined with the teacher-student model architecture of knowledge distillation technology, thereby generating greater technical benefits based on the power Internet of Things.

[0031] This invention applies knowledge distillation technology to the field of UAV image processing to solve the problem of insufficient generalization ability of UAV edge models due to insufficient computing power.

[0032] The knowledge distillation proposed in this invention is the first attempt to perform knowledge distillation on a two-stage object recognition model (Faster R-CNN) and a lightweight single-stage object detection model (YOLOtiny). Compared with two-stage-to-two-stage knowledge distillation, it has the advantage of being lightweight, and compared with single-stage-to-single-stage knowledge distillation, it has the advantage of performance. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of a process of the present invention;

[0034] Figure 2 This is a flowchart of the joint training of the teacher model and student model in this invention;

[0035] Figure 3 This is a flowchart of the cloud-edge collaborative hardware structure and algorithm of the present invention. Detailed Implementation

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] Example:

[0038] A cloud-edge collaborative recognition method for UAV imagery based on knowledge distillation, such as Figure 1 As shown, it includes the following steps:

[0039] Step 1: Build a teacher model and train it until it fully converges and has the ability to provide guidance.

[0040] Step 2: Build a student model and improve the loss function of the student model so that the student model can accept soft labels from both human labels and teacher models.

[0041] Step 3: The teacher model and the student model are jointly trained using soft label distillation. The student model receives both manual labels and soft labels from the teacher model for iteration until the student model fully converges.

[0042] Step 4: Deploy the trained student model to the drone terminal so that the student model can actually run at the edge and test the recognition effect;

[0043] Step 5: If the recognition effect detected in Step 4 is deemed unsatisfactory, the recognized image is transmitted to the teacher model. The teacher model guides the student model to undergo secondary training through cloud-edge collaboration until the recognition effect meets the standard.

[0044] The teacher model is based on Faster R-CNN, and the student model is based on YOLOtiny. The student model uses the lightweight YOLOv4tiny network as its basic framework. On this basis, its loss function is improved so that it can be guided by the teacher model. At the same time, spatial attention mechanism and channel attention mechanism are introduced to enhance the learning ability of the student model.

[0045] The loss function of the student model consists of three parts: bounding box regression loss, class classification loss, and confidence loss. This invention focuses more on classification accuracy; therefore, it only improves the class classification loss function while keeping the bounding box regression loss and confidence loss unchanged. The specific loss function is as follows:

[0046]

[0047] Where K×K and M represent the dimensions of the input features, respectively, p i (c) represents the true category. Indicates the predicted category; where,

[0048] p i (c) = sigmoid(z) i )

[0049] Among them, z i This represents the final output of the classification network;

[0050] Since knowledge distillation learning is required for the student model, in order to improve the probability distribution entropy of the sigmoid function output of both the teacher and student models, thereby softening the output classification label, this invention proposes a sigmoid function incorporating a temperature variable, as follows:

[0051]

[0052] Where T1 and T2 are parameters for controlling the temperature, with T1 controlling z i The displacement, T2 controls z i The scaling is adjusted to ensure that the classification labels are sufficiently softened. When T1 = 0 and T2 = 1, this formula is the ordinary sigmoid function.

[0053] To enable the student model to learn from both manually labeled ground truth labels and soft labels provided by the teacher model during image classification in object detection, the classification loss function of the student model is improved as follows:

[0054]

[0055] in, The soft label refers to the teacher's model under a temperature parameter T. This refers to the predicted soft label of the student model with temperature parameter T, where α and β are the weights of the true hard label and the teacher model soft label, respectively.

[0056] To enhance the learning ability of the student model, this invention introduces spatial attention and channel attention mechanisms into the backbone network of the student model.

[0057] The formula for calculating spatial attention mechanism is as follows:

[0058]

[0059] Where: f 7×7 This represents a convolution calculation of size 7×7; and These represent the global average pooling and maximum average pooling operations of the spatial attention mechanism, respectively.

[0060] The formula for calculating the channel attention mechanism is as follows:

[0061]

[0062] Where σ(·) represents the sigmoid function; MLP represents the shared network in the module, which consists of hidden layers and multilayer perceptrons. and These represent the global average pooling and maximum average pooling operations of the channel attention mechanism, respectively.

[0063] In step 3, the teacher and student models are jointly trained using soft label distillation. This requires aligning the soft labels of the teacher and student models. The alignment function is as follows:

[0064]

[0065] Where M and N represent the length and width of the input image for the student model, respectively, and M′ and N′ represent the length and width of the input image for the teacher model, respectively.

[0066] During joint training, this invention performs knowledge distillation only on the target classifier in object detection. On one hand, the teacher and student models output soft labels under the same temperature parameters T1 = t1 and T2 = t2, and the distillation loss is calculated using a given loss function. To improve the model's convergence speed, the teacher model's soft labels undergo non-maximum suppression. On the other hand, the student model's temperature parameters are set to T1 = 0 and T2 = 1. After outputting hard labels, the loss between the hard labels and the true labels is calculated. This effectively reduces the possibility of errors from the teacher model being propagated to the student model.

[0067] After joint training in the cloud, the student model is ported to the edge and deployed to the production environment. Due to limitations in the student model's search space and computing power, the edge algorithm may experience a drop in accuracy in completely unfamiliar work environments due to poor adaptability. However, the teacher model typically does not exhibit this issue. Therefore, at an appropriate time, an evaluation function can be used to assess the difference between the edge-based student algorithm and the server-side teacher algorithm. If the difference exceeds a predetermined threshold, it indicates that the edge-based student model is exhibiting insufficient adaptability.

[0068] In step 5, the method by which the teacher model guides the student model for secondary training through cloud-edge collaboration is as follows: Without human labels, a distillation learning mechanism is used to converge the student model towards the teacher model, fine-tuning the student model to achieve better results for the edge model. The calculation formula is as follows:

[0069]

[0070] like Figure 2 , Figure 3 As shown, during the training phase, this invention first fully trains the teacher model. Then, during the training of the student model, the teacher and student models are jointly trained. The teacher model's role is to improve soft labels. During joint training, this invention only performs knowledge distillation on the target classifier in target detection. On one hand, the teacher and student models output soft labels under the same temperature parameters T1 = t1 and T2 = t2, and the distillation loss is calculated using the given loss function. To improve the model's convergence speed, the teacher model's soft labels undergo non-maximum suppression. On the other hand, the student model's temperature parameters are set to T1 = 0 and T2 = 1, and after outputting hard labels, the loss compared to the true labels is calculated. This effectively reduces the possibility of errors from the teacher model being propagated to the student model. In practical applications, a cloud-edge collaborative self-learning mode can also be used to improve the performance of the edge model. That is, without manual labels, the distillation learning mechanism proposed in this invention allows the student model to converge towards the teacher model, enabling model fine-tuning and thus achieving better results for the edge model.

[0071] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Other variations and modifications are possible without departing from the technical solutions described in the claims.

Claims

1. A cloud-edge collaborative recognition method for UAV imagery based on knowledge distillation, characterized in that, Includes the following steps: Step 1: Build a teacher model and train it until it fully converges and has the ability to provide guidance. Step 2: Construct a student model and improve the loss function of the student model so that it can accept soft labels from both human labels and teacher models. Introduce spatial attention and channel attention mechanisms. Step 3: The teacher model and student model are jointly trained using soft label distillation. The student model receives both manual labels and soft labels from the teacher model for iteration until the student model fully converges. During the joint training process, knowledge distillation is only performed on the target classifier in target detection to construct a Sigmoid function containing two temperature parameters T1 and T2. Step 4: Deploy the trained student model to the drone terminal so that the student model can actually run at the edge and test the recognition effect; Step 5: If the recognition effect detected in Step 4 is deemed unsatisfactory, the recognized image is transmitted to the teacher model. The teacher model guides the student model to undergo secondary training through cloud-edge collaboration until the recognition effect meets the standard. The secondary training is carried out without human labels, and the student model is fine-tuned by using a distillation learning mechanism to converge to the teacher model.

2. The cloud-edge collaborative recognition method for UAV imagery based on knowledge distillation according to claim 1, characterized in that, The teacher model is a teacher model based on Faster R-CNN, and the student model is a student model based on YOLOtiny.

3. The cloud-edge collaborative recognition method for UAV imagery based on knowledge distillation according to claim 2, characterized in that, The loss function of the student model consists of three parts: box regression loss, class classification loss, and confidence loss, specifically: Where K×K and M represent the dimensions of the input features, Indicates the true category, Indicates the predicted category; where, in, This represents the final output of the classification network; Since knowledge distillation learning is required for the student model, and in order to improve the probability distribution entropy of the Sigmoid function output of both the teacher and student models, thereby softening the output classification label, a Sigmoid function incorporating a temperature variable is proposed, as follows: in, and It is a parameter for controlling temperature. control displacement, control Scaling to ensure the category labels are sufficiently softened, when When this is the case, the expression is the ordinary Sigmoid function; To enable the student model to learn from both manually labeled ground truth labels and soft labels provided by the teacher model during image classification in object detection, the classification loss function of the student model is improved as follows: in, The soft label refers to the teacher's model under a temperature parameter T. This refers to the soft labels predicted by the student model when the temperature parameter is T. These represent the weights of the real hard labels and the teacher model soft labels, respectively.

4. The cloud-edge collaborative recognition method for UAV imagery based on knowledge distillation according to claim 3, characterized in that, In step 3, the teacher and student models are jointly trained using soft label distillation. This requires aligning the soft labels of the teacher and student models. The alignment function is as follows: Where M and N represent the length and width of the input image for the student model, respectively. These represent the length and width of the input image for the teacher model, respectively.

5. The cloud-edge collaborative recognition method for UAV imagery based on knowledge distillation according to claim 4, characterized in that, In step 5, the method by which the teacher model guides the student model for secondary training through cloud-edge collaboration is as follows: Without human labels, a distillation learning mechanism is used to converge the student model towards the teacher model, fine-tuning the student model to achieve better results for the edge model. The calculation formula is as follows: 。

Images