Aerial Personnel Detection Method Based on Lightweight Distillation Network

Abstract

This invention relates to an aerial personnel detection method based on a lightweight distillation network. First, the collected aerial data containing personnel targets is input into a trained response-based teacher-assisted distillation detection network containing multiple low-precision quantization modules. Then, the first detection result of the personnel targets output by the teacher-assisted distillation detection network is input into a robust detection module to obtain a second detection result. Next, the IoU (Interchange of Interest) between the first and second detection results and the label of the previous frame is calculated. If the IoU between the first and second detection results and the label of the previous frame is greater than the IoU between the second and the label of the previous frame, the first detection result is taken as the final detection result; otherwise, the second detection result is taken as the final detection result. This invention uses a lightweight network model to detect personnel targets, reducing the number of network model parameters and floating-point operations, resulting in high detection accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to the field of small target detection technology, and in particular to an aerial personnel detection method based on a lightweight distillation network. Background Technology

[0002] Aerial Person Detection (APD) is crucial for Information Acquisition and Search and Rescue (SaR) missions, which aim to detect the precise location of people from images captured by unmanned aerial vehicles (UAVs). To effectively detect people, various detection methods have been developed in the prior art, including deep learning-based online detection methods, personnel detection methods in aerial images, and lightweight detection methods using model acceleration, all aimed at improving the accuracy and computational efficiency of detection systems. However, due to the unique characteristics of high-resolution aerial images, such as wide fields of view, significant angular variations, and target pixels smaller than 20x20 pixels, these unique features result in low accuracy and efficiency for the aforementioned detection methods when targeting aerial persons, which fall into the small target category.

[0003] With the rise of deep convolutional neural networks, visual object detection methods have made unprecedented progress, improving the accuracy and efficiency of object detection in natural images. However, most visual object detection methods are developed for medium to large-sized objects in natural images. In aerial data, where people are considered small targets, the features of these targets are often subtle and easily obscured by the background, leading to reduced detection accuracy and efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide an aerial personnel detection method based on a lightweight distillation network to address the above-mentioned technical problems. This method uses a lightweight network model to detect personnel targets, which reduces the number of network model parameters and floating-point operations, resulting in higher detection accuracy and efficiency.

[0005] This invention provides a method for aerial personnel detection based on a lightweight distillation network, comprising:

[0006] The collected aerial data containing personnel targets is input into a trained response-based teacher-assisted distillation detection network containing multiple low-precision quantization modules;

[0007] The first detection result of the personnel target output by the teacher-assisted distillation detection network is input into the robust detection module to obtain the second detection result;

[0008] Calculate the IoU between the first detection result and the second detection result and the label of the previous frame respectively. If the IoU between the first detection result and the label of the previous frame is greater than the IoU between the second detection result and the label of the previous frame, the first detection result is taken as the final detection result; otherwise, the second detection result is taken as the final detection result.

[0009] The teacher-assisted distillation detection network includes parallel teacher and student modules. Each network layer of the teacher and student modules is connected to a low-precision quantization module, which is used to convert the 32-bit floating-point number output by each network layer into an 8-bit integer.

[0010] In one embodiment, the teacher module or student module is one of YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x.

[0011] In one embodiment, the low-precision quantization module uses a dynamically constructed binary tree structure to convert floating-point numbers to integers.

[0012] In one embodiment, the robustness detection module is a plug-and-play tracking network.

[0013] In one embodiment, the total loss of the teacher-assisted distillation detection network is

[0014] L Total =L cls +L obj +L bbox +L KD (1)

[0015] In the formula, L cLs L represents the classification loss. obj L represents the confidence loss. bbox L represents the bounding box regression loss. KD This represents the distillation loss based on the response.

[0016] In one embodiment, the response-based distillation loss is

[0017] L KD =αL soft +βL hard (2)

[0018] In the formula, L soft L represents an indicator that measures the consistency between the soft labels generated by the teacher module and those generated by the student module. hard This represents a metric that measures the consistency between the student module prediction and the label of the previous frame.

[0019] In one embodiment, the aerial data is a sequence of multiple images or videos.

[0020] In one embodiment, both the first detection result and the second detection result include detection confidence, coordinate information, and classification result.

[0021] The beneficial effects of this invention are:

[0022] (1) Each network layer of the teacher module and student module of the present invention includes a low-precision quantization module. The low-precision quantization module can convert the 32-bit floating-point number output by each network layer into an 8-bit integer, which can effectively accelerate the inference speed and improve the detection efficiency.

[0023] (2) The teacher-assisted distillation detection network endows students with the logical reasoning ability of module learning and transfer similar to that of teacher modules, which can effectively reduce the computational complexity of the model and improve detection efficiency.

[0024] (3) The larger the IoU value, the closer the detection result is to the true value. By adopting a voting mechanism, the detection result with the larger IoU value is taken as the final detection result, which can screen out the detection result with high accuracy and improve the accuracy of the final detection result. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating an aerial personnel detection method based on a lightweight distillation network provided in an embodiment of the present invention.

[0026] Figure 2 This is one of the structural schematic diagrams of the teacher-assisted distillation detection network provided in this embodiment;

[0027] Figure 3 This is a schematic diagram of the structure of a low-precision quantization module provided in an embodiment of the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0029] In one embodiment, such as Figure 1 As shown, Figure 1 This is one of the flowcharts illustrating the aerial personnel detection method based on a lightweight distillation network provided in this invention. When applied to a computer device, this method includes the following steps:

[0030] S101. The collected aerial photography data containing personnel targets is input into a trained response-based teacher-assisted distillation detection network containing multiple low-precision quantization modules.

[0031] Specifically, aerial photography data consists of multiple image sequences or videos, which are multi-frame data and can be captured by devices such as drones.

[0032] like Figure 2 As shown, Figure 2 This is one of the structural diagrams of the teacher-assisted distillation detection network provided in this embodiment. The teacher-assisted distillation detection network in this embodiment includes parallel teacher modules and student modules, both of which are Yolov5 structures. The upper network represents the teacher module, and the lower module represents the student network. P3-P7 obtain rich feature information of the object from the backbone and neck part, and transfer knowledge from the teacher module to the student module.

[0033] The teacher-assisted distillation detection network endows students with the logical reasoning ability to learn and transfer modules similar to that of the teacher modules, which can effectively reduce the computational complexity of the model and improve detection efficiency.

[0034] In this embodiment, a low-precision quantization module is connected after each network layer of the teacher and student modules. The low-precision quantization module is used to convert the 32-bit floating-point number output by each network layer into an 8-bit integer, which can effectively accelerate the inference speed of the teacher-assisted distillation detection network.

[0035] S102. Input the first detection result of the personnel target output by the teacher-assisted distillation detection network into the robustness detection module to obtain the second detection result. Both the first and second detection results include detection confidence, coordinate information, and classification results.

[0036] S103. Calculate the IoU between the first detection result and the second detection result and the label of the previous frame respectively. If the IoU between the first detection result and the label of the previous frame is greater than the IoU between the second detection result and the label of the previous frame, the first detection result shall be used as the final detection result; otherwise, the second detection result shall be used as the final detection result.

[0037] It should be noted that the label of the previous frame is composed of the detection confidence, coordinate information and classification result generated in the previous frame.

[0038] In this step, the final prediction result of the first frame of aerial data is based on the second detection result output by the robustness detection module.

[0039] Specifically, the previous frame label is manually labeled. The larger the IoU value, the closer the detection result is to the true value and the higher the detection accuracy. The final detection result is determined by comparing the IoU of the first and second detection results with the previous frame label. This is a voting mechanism that can filter out the detection results with high accuracy and improve the accuracy of the final detection result.

[0040] In one embodiment, the teacher or student module is one of Yolov5s, Yolov5m, Yolov5l, or Yolov5x. Training data is divided into training, validation, and test sets. The relevant parameters used for training, such as learning rate, batch size, optimizer, and loss function, are consistent with the Yolov5 framework. Training is conducted iteratively, and the model and parameters can be adjusted promptly based on the performance on the validation set to achieve better results. Finally, the performance of the trained model is evaluated using the test set, including accuracy, recall, and precision.

[0041] In this embodiment, the overall network training, including the teacher-assisted distillation detection network and the robustness detection module, can be performed on a computer running Ubuntu 22.04.1LTS (64-bit) operating system, equipped with an Intel Core i9-13900KF processor (6 cores, 32GB of memory, clock speed of 5.80GHz). The GPU server also assists with an NVIDIA GeForce RTX 4090 GPU with 24GB of memory.

[0042] The dataset used for training preferably contains aerial photography data from a diverse range of scenes, including but not limited to urban areas, oceans, coastlines, and jungles.

[0043] During training, a stochastic gradient descent (SGD) optimizer was used for 100 epochs, with a momentum of 0.9 and a weight decay of 0.0001. The initial learning rate was 0.01, which was decayed at the 10th epoch.

[0044] Furthermore, this embodiment selects several training accuracy metrics, including Precision, Recall, mAP@0.5, and mAP@0.95, as well as several inference speed metrics, including parameters, GFLOPs, Latency, and FPS, as evaluation standards. Precision measures the ratio between correctly detected objects and the total number of detected objects. Recall measures the ratio between correctly detected objects and the total number of actual objects. mAP@0.5 represents the average accuracy when the IoU threshold is greater than 0.5. mAP@0.95 represents the average accuracy with a step size of 0.05 between the IoU thresholds of 0.5 and 0.95. Parameters correspond to the number of trainable parameters in the network, typically including weights (W) and biases (b). GFLOPs represent the number of floating-point operations in the network.

[0045] In one embodiment, the low-precision quantization module uses a dynamically constructed binary tree structure to convert floating-point numbers to integers. For example... Figure 3 As shown, Figure 3This is a schematic diagram of the structure of a low-precision quantization module provided in an embodiment of the present invention.

[0046] The truncation error and rounding error (zp) are balanced by setting upper (u) and lower (l) thresholds for the quantization grid. x When data is truncated to fit predefined grid boundaries, truncation errors often occur, leading to information loss and reduced accuracy of the resulting quantized representation. Conversely, rounding errors stem from inherent inaccuracies introduced during the rounding process.

[0047] Specifically, in this embodiment, the quantizer Q(x|b) of the low-precision quantization module is defined as:

[0048]

[0049] It is a function that maps floating-point numbers x∈R to the nearest quantized interval. This calculation introduces a quantization parameter (scaling, s). x ) and (zero point, zp x The calculation formula is as follows:

[0050]

[0051] Where b represents the quantization bit width. The symbols u (upper bound) and l (lower bound) are used to specify the limits of the quantization grid.

[0052] The method of converting floating-point numbers to integers using a dynamically constructed binary tree structure can improve both absolute and relative errors, quantizing the weights and biases of the backbone, neck, and head networks in the teacher or student modules. By selectively quantizing non-uniformly distributed parameters and isolating outliers (usually represented by maximum and minimum values), information loss is minimized during activation quantization, thereby ensuring accurate target detection results from the teacher-assisted distillation detection network.

[0053] Specifically, floating-point data is divided into four components: dynamic exponent and mantissa, such as... Figure 3 As shown, a binary tree dynamic tree quantization module is used for processing. The sign bit is assigned to the first bit, and the number of subsequent bits, in variable quantities, represents the magnitude of the exponent. By default, the fourth bit is set to 1, and the remaining bits undergo linear quantization. High-precision floating-point values ​​are successfully quantized to 8-bit integers by continuously shifting block indicator bits. All tensors are normalized to the range [-1, 1]. Quantizers are applied to each weight portion of the convolutional layers in the teacher and student modules respectively to ensure reasonable truncation of floating-point bits, and quantizers are also applied to the activation layers to keep the output weight / bias tail as low as possible.

[0054] The method of using a dynamically constructed binary tree structure to convert floating-point numbers to integers can map the floating-point values ​​of weights and activation functions to a dynamically adjusted range, thereby effectively reducing the computational complexity of the teacher-assisted distillation detection network.

[0055] In one embodiment, the robustness detection module is a plug-and-play tracking network. For example, plug-and-play tracking networks include DeepSort, Sort, and ByteTrack.

[0056] The use of a pluggable tracking module can estimate the potential location of a target by utilizing the target motion information of each frame sequence of aerial data, and continuously update the position and feature information of the object in each frame of data, namely the detection confidence of the robust detection module, the coordinate information of the target detection personnel, and the classification results.

[0057] In one embodiment, the total loss of the teacher-assisted distillation detection network is...

[0058] L Total =L cls +L obj +L bbox +L KD (1)

[0059] In the formula, L cLs L represents the classification loss. obj L represents the confidence loss. bbox L represents the bounding box regression loss. KD This represents the distillation loss based on the response.

[0060] Specifically, classification loss (L cLs The calculation is performed using the binary cross-entropy function, defined as follows:

[0061]

[0062] In the formula, y represents the label of the input sample, with a value of 1 indicating a positive sample and a value of 0 indicating a negative sample. Furthermore, p represents the predicted probability that the model will classify the input sample as a positive sample.

[0063] Confidence loss (L obj The calculation uses a binary cross-entropy function and includes weight coefficients based on the detection layer scale. The calculation method for confidence loss is consistent with that for classification loss. These coefficients are assigned values ​​from largest to smallest according to the detection layer scale, with default values ​​of 4.0, 1.0, and 0.4. Therefore, small objects receive higher weight coefficients in the loss on multi-scale feature maps, thus emphasizing the training focus on small objects.

[0064] Bounding box regression loss (L bboxTo address the discrepancy between the predicted bounding box and the ground truth, CloU also considers other factors such as aspect ratio. The specific calculation formula is as follows:

[0065]

[0066] In the formula, B = (x, y, w, h) represents the bounding box predicted by the teacher-assisted distillation detection network. gt =(x gt y gt w gt h gt The target box is represented by , and the parameter v represents the normalized difference in aspect ratio between the predicted and actual values. The parameter α ranges from 0 to π / 4, and is then multiplied by π / 4 to ensure it falls within the range of 0 to 1. Furthermore, parameter α acts as a balancing factor to weigh the relationship between the aspect ratio difference and the loss caused by the intersection-union (IoU).

[0067] In one embodiment, the response-based distillation loss is

[0068] L KD =αL soft +βL hard (2)

[0069] In the formula, α and β are soft thresholds based on the training stage of different model depths.

[0070] L soft The metric representing the consistency between the soft tags generated by the teacher module and those generated by the student module can be expressed as:

[0071]

[0072] Among them, v i and v k It's the teacher's logits, z i and z k L is the student's logits, and N is the number of input instances. hard The metric that measures the consistency between the student module prediction and the label of the previous frame can be expressed as:

[0073]

[0074] Here, c j Let c represent the tag of the previous frame of instance j, where c j ∈0,1 (positive label is 1, negative label is 0), z j These are the students' logits.

[0075] This invention presents an aerial personnel detection method based on a lightweight distillation network. In the teacher-assisted distillation detection network, the student model can learn and transfer logical reasoning abilities similar to the teacher model, effectively reducing the computational complexity of the network and achieving lightweight computation. Furthermore, the low-precision quantization module maps data to a dynamically constructed tree structure, using 8-bit integers to represent 32-bit floating-point numbers, effectively accelerating inference speed. Finally, a robust detection module is constructed, enabling it to utilize the position and motion information of objects in consecutive frames. This invention can effectively improve the accuracy of personnel location.

[0076] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for aerial personnel detection based on a lightweight distillation network, characterized in that, include: The collected aerial data containing personnel targets is input into a trained response-based teacher-assisted distillation detection network containing multiple low-precision quantization modules; The first detection result of the personnel target output by the teacher-assisted distillation detection network is input into the robust detection module to obtain the second detection result; Calculate the IoU between the first detection result and the second detection result and the label of the previous frame respectively. If the IoU between the first detection result and the label of the previous frame is greater than the IoU between the second detection result and the label of the previous frame, the first detection result is taken as the final detection result; otherwise, the second detection result is taken as the final detection result. The teacher-assisted distillation detection network includes parallel teacher and student modules. Each network layer of the teacher and student modules is connected to a low-precision quantization module, which is used to convert the 32-bit floating-point number output by each network layer into an 8-bit integer. The robustness detection module is a plug-and-play tracking network; The label of the previous frame consists of the detection confidence, coordinate information and classification result generated in the previous frame; The final prediction result of the first frame of aerial data is based on the second detection result output by the robustness detection module.

2. The aerial personnel detection method based on a lightweight distillation network according to claim 1, characterized in that, The teacher module or student module is one of YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x.

3. The aerial personnel detection method based on a lightweight distillation network according to claim 2, characterized in that, The low-precision quantization module uses a dynamically constructed binary tree structure to convert floating-point numbers to integers.

4. The aerial personnel detection method based on a lightweight distillation network according to claim 2, characterized in that, The total loss of the teacher-assisted distillation detection network is 　　　（1） In the formula, Represents classification loss. Indicates confidence loss. This represents the bounding box regression loss. This represents the distillation loss based on the response.

5. The aerial personnel detection method based on a lightweight distillation network according to claim 4, characterized in that, The response-based distillation loss is 　　　（2） In the formula, This represents an indicator that measures the consistency between the soft tags generated by the teacher module and those generated by the student module. This represents a metric that measures the consistency between the student module prediction and the label of the previous frame. α and β are soft thresholds based on the training stage of different model depths.

6. The aerial personnel detection method based on a lightweight distillation network according to claim 1, characterized in that, The aerial data consists of multiple image sequences or videos.

7. The aerial personnel detection method based on a lightweight distillation network according to claim 1, characterized in that, Both the first and second detection results include detection confidence, coordinate information, and classification results.

Images