A lightweight navigation marker detection method based on KDS R-CNN

Abstract

The application provides a kind of light-weight navigation marker detection method based on KDSR-CNN, comprising the following steps: S1, obtain the aerial video of the unmanned aerial vehicle visual angle in the cruising range;S2, prepare visual navigation dataset and carry out data enhancement;S3, construct light-weight KDSR-CNN network model;S4, knowledge distillation training light-weight network to improve model accuracy;S5, deploy on embedded platform for inference verification.The application significantly reduces the parameter quantity and calculation amount of the model, is more easily deployed on the embedded platform, and at the same time improves the detection accuracy of the navigation marker, so as to effectively improve the positioning accuracy of visual navigation, and effectively realize the real-time aerial target detection of the aircraft.

Description

Technical Field

[0001] This invention belongs to the field of application technology of aircraft visual navigation methods, specifically involving a lightweight navigation marker detection method based on KDS R-CNN. Background Technology

[0002] In recent years, autonomous visual navigation technology for aircraft has developed rapidly, meeting the needs of intelligent navigation and guidance, situational awareness, and autonomous decision-making for aircraft. It plays an irreplaceable role in typical airborne platform tasks such as target detection, identification, and tracking. However, due to the limited hardware computing resources of aircraft platforms, it is often difficult to deploy large-scale algorithm models with high detection accuracy, or the deployed models suffer from poor computational speed or accuracy, failing to meet the requirements of real-time detection. Therefore, there is a need to design a lightweight navigation marker detection method for embedded devices, which can effectively achieve real-time detection of navigation markers by aircraft, improving detection accuracy and operating speed, and has significant and broad application prospects. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a lightweight navigation marker detection method based on KDS R-CNN to address the shortcomings of the prior art. This method overcomes the problems of large model parameters, slow calculation speed and low detection accuracy in the existing methods. It can also be deployed on aircraft platforms with limited hardware computing resources, enabling aircraft platforms with limited computing resources to effectively achieve fast, high-precision and real-time navigation marker detection.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a lightweight navigation landmark detection method based on KDS R-CNN, characterized by comprising the following steps:

[0005] S1. Acquire aerial video from the drone's perspective within the cruising range;

[0006] S2. Prepare a visual navigation dataset and perform data augmentation;

[0007] S3. Construct a lightweight KDS R-CNN network model;

[0008] S4. Knowledge distillation training of lightweight networks improves model accuracy;

[0009] S5. Deploy the model on the embedded platform and perform inference verification.

[0010] Preferably, the specific operation method of S1 is as follows:

[0011] S101. Using the Autel Intelligent EVO II UAV, cruise along the planned path within the set visual navigation area at an altitude of 350m with a downward-looking perspective to capture aerial images of the navigation area. The video includes houses, roads, plants, etc.

[0012] S102. For navigation markers, the UAV flight curve is set as a spiral curve. The UAV flies around the navigation marker 5 times, and the flight altitude gradually decreases from 350 meters to 50 meters to obtain aerial images of different scales and perspectives.

[0013] Preferably, the specific operation of S2 is as follows: Since the amount of aerial image data obtained by UAV aerial photography is limited, images of navigation signs under different perspectives, lighting conditions, and weather conditions can be further obtained through methods such as web crawling and ground camera shooting. These images, together with the images obtained in S1, constitute a basic dataset. Then, an image enhancement algorithm combining geometric space and feature space is used to augment the basic dataset. Finally, a target detection algorithm and a KCF tracking algorithm are used to automatically label the navigation signs in the augmented dataset.

[0014] Preferably, the specific operation of S3 is as follows: MobileNet is used as the backbone network, depthwise separable convolution is used instead of standard convolution, sparse and learnable suggestion windows are used to make initial guesses on the potential locations of navigation landmarks, and then the real locations of landmarks in the image are continuously learned through training and parameter backpropagation. Dynamic instance interactive head is used to interact RoI features and suggestion features, invalid suggestion windows are filtered out and predicted target features are output. Finally, classification and regression prediction are performed to obtain the final localization and classification results. Iterative structure and self-attention module are used to further improve model performance.

[0015] Preferably, the specific operation of S4 is as follows: the lightweight KDS R-CNN model constructed in S3 is used as the student network, and the teacher network adopts the Sparse R-CNN model. According to the pre-trained weight file, the output result of the teacher network is used as prior information to guide the student network to learn continuously, while updating the parameters of the student network. Secondly, the label allocation strategy of Gaussian receptive field is used to transform the prior information provided by the teacher network into a Gaussian distribution. Then, the similarity between the prior information and the prediction result is directly measured according to the Gaussian receptive field distance. Through continuous fine-tuning and training of the student network, a high-precision, lightweight KDS R-CNN detection model is finally obtained.

[0016] Preferably, the specific operation of S5 is as follows: First, install the Ubuntu 18.04 system with the aarch64 architecture on Xavier NX and configure the environment according to the algorithm requirements; second, use TensorRT to accelerate the network by reorganizing and optimizing the vertical and horizontal structures of the network and eliminating useless network output layers; then, convert the pre-trained model into a WTS file and generate a deployment engine; finally, use the deployed lightweight model to perform inference testing and obtain the detection results.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] This invention constructs a navigation marker dataset, providing data support for aircraft visual navigation and positioning; it constructs a lightweight KDS R-CNN network model, uses knowledge distillation to train the model, reduces network parameters while improving detection accuracy and running speed; the resulting lightweight detection algorithm is deployed on embedded devices, and the network is accelerated and optimized, improving the inference speed of the model on embedded devices.

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall process of the present invention.

[0021] Figure 2 This is a detailed flowchart of the present invention.

[0022] Figure 3 This is a schematic diagram of the flight trajectory of the UAV in this invention.

[0023] Figure 4 This is a schematic diagram of the data set preparation method of the present invention.

[0024] Figure 5 This is a schematic diagram of data augmentation in this invention.

[0025] Figure 6 This is a structural diagram of the feature space-based data augmentation algorithm in this invention.

[0026] Figure 7 This is a structural diagram of the lightweight KDS R-CNN model in this invention.

[0027] Figure 8 This is a diagram of the MobileNet backbone network structure in this invention.

[0028] Figure 9 This is a schematic diagram of the Xavier NX embedded device interface in this invention.

[0029] Figure 10 This is a diagram showing the detection effect of navigation markers in this invention. Detailed Implementation

[0030] like Figures 1 to 8 As shown, the present invention includes the following steps:

[0031] S1. Acquire aerial video from the drone's perspective within the cruising range;

[0032] S2. Prepare a visual navigation dataset and perform data augmentation;

[0033] S3. Construct a lightweight KDS R-CNN network model;

[0034] S4. Knowledge distillation training of lightweight networks improves model accuracy;

[0035] S5. Deploy the model on the embedded platform and perform inference verification.

[0036] In this embodiment, S1 specifically includes the following steps:

[0037] S101. Using the Autel Intelligent EVO II UAV, cruise along the planned path within the designated visual navigation area at an altitude of 350m with a downward-facing viewpoint, capturing aerial images of the navigation area. The video includes houses, roads, plants, etc.

[0038] S102. For navigation markers, the UAV flight path is set to a spiral curve, such as... Figure 3 As shown, the drone flew around the navigation marker five times, gradually decreasing its altitude from 350 meters to 50 meters, obtaining aerial images of different scales and perspectives.

[0039] In this embodiment, S2 specifically includes the following steps:

[0040] S201 specifies the format of the navigation signage dataset, which mainly consists of three parts: Images, Classification, and Labels. The Images section stores the original images of the dataset, comprising the acquired baseline images and algorithm-enhanced extended images, including multiple categories and scenes. The Classification section contains TXT files that categorize the dataset according to different methods, summarizing the filenames of all images of the same category. The Objectclass section contains image sets categorized by target type, and the Scenesclass section contains image sets categorized by scene type. The Labels section stores XML files, derived from annotation methods. The XML files correspond one-to-one with the original images stored in the Images section, and their main function is to provide the computer with processable semantic information for the aerial and ground image target cognition dataset, thereby transforming low-level image features into high-level semantic information.

[0041] S202. Since the amount of aerial imagery data obtained by drone aerial photography is limited, images of navigation markers under different angles, lighting conditions, and weather conditions can be further obtained through methods such as web crawling and ground camera shooting. These images, together with the aerial images taken in S1, constitute the basic dataset.

[0042] S203. Augmenting the dataset using geometric space image enhancement algorithms. Traditional geometric space-based data augmentation methods mainly utilize rotation, flipping, translation, cropping, erasure, and image blending to expand the dataset. To increase rotation invariance, image rotation angles of 30°, 60°, 150°, and 210° were selected. Excessive random erasure would lead to significant loss of target information; therefore, erasure ratios of 0.1, 0.2, and 0.3 were chosen. Image blending methods can effectively superimpose image information from positive and negative samples; to focus primarily on the feature information of positive samples, the blending ratios were set to 0.5, 0.6, and 0.7. Geometric space-based data augmentation methods only modify the pixels of the image geometrically, preserving the original target's outline. Finally, the augmented images are selected, discarding those with excessively large target erasure ratios.

[0043] S204. The dataset is augmented using a feature space image augmentation algorithm. A pyramid multi-scale generative adversarial network is employed to learn the features of the current sample and generate similar features through feature fitting, thereby enriching the diversity of the samples and enhancing the target data in the feature space. The structure of the feature space-based data augmentation algorithm is as follows: Figure 6 As shown.

[0044] S205. Automatically label navigation markers in the dataset using target detection and KCF tracking algorithms. First, manually label the first frame image, and then use the KCF tracking algorithm to coarsely label the image sequence in subsequent frames. Second, extract the saliency map of the tracking box obtained by KCF based on image saliency. Finally, perform threshold segmentation on the obtained saliency map to obtain the location of the target, and further perform fine labeling, thereby realizing the automatic labeling of the time sequence of navigation marker images.

[0045] In this embodiment, step S3 specifically includes the following steps:

[0046] The S301 lightweight KDS R-CNN network model mainly consists of four parts: the backbone network, object proposal, dynamic instance interaction head, and detection head.

[0047] S302. The backbone network adopts MobileNet, which uses depthwise separable convolutions instead of standard convolutions, greatly reducing the number of parameters and computational cost of the backbone network. The MobileNet network first uses a 3×3 standard convolution, followed by stacked depthwise separable convolutions, some of which are downsampled with a stride of 2.

[0048] S303. Initially guess the potential location of navigation landmarks using sparse and learnable suggestion windows, and then continuously learn the true location of the landmarks in the image through training and parameter backpropagation. The number of suggestion windows is set to 100.

[0049] S304. For a given 100 suggestion windows, use the RoIAlign operation to extract the RoI features within each suggestion window.

[0050] S305. Utilize a set of learnable 256-dimensional latent vectors as proposed features to provide more detailed information such as the pose and shape of navigation markers, with the number set to 100.

[0051] S306. The final prediction is generated using a dynamic instance interactive head, and depthwise separable convolution is used for computation. The RoI features extracted in S304 are interacted with the suggestion features in S305 to filter out invalid suggestion windows and output the predicted target features.

[0052] S307. Use a multilayer perceptron for regression prediction and a linear projection layer for classification prediction to obtain the final localization and classification results.

[0053] S308. Further improve performance by utilizing the iterative structure, and use the newly generated target bounding box and target features in S307 as the suggested window and suggested features for the next stage in the iteration process.

[0054] S309. In the iterative structure, before dynamic instance interaction, the self-attention module is used to perform inference calculations on multiple target features to obtain the relationship between the predicted targets.

[0055] In this embodiment, S4 specifically includes the following steps:

[0056] S401. The teacher network uses ResNet50 as the backbone network and KDS R-CNN with dynamic convolution in the dynamic instance interactive head. It has a pre-trained weight file and uses the constructed lightweight KDS R-CNN model as the student network.

[0057] S402. The teacher's network output results serve as prior information to guide the student's network in continuous learning, while also updating the student's network parameters.

[0058] S403. Using the Gaussian receptive field label assignment strategy, the prior information provided by the teacher network is first transformed into a Gaussian distribution, and then the similarity between the prior information and the prediction result is directly measured based on the Gaussian receptive field distance.

[0059] S404. By continuously fine-tuning and training the student network, a high-precision, lightweight KDS R-CNN detection model can be obtained.

[0060] In this embodiment, S5 specifically includes the following steps:

[0061] Considering factors such as cost, power consumption, and computing power, the Nvidia Jetson Xavier NX embedded development platform was chosen to deploy the lightweight detection network. Its interface is as follows: Figure 9 As shown.

[0062] S502, Install an Ubuntu 18.04 system with aarch64 architecture on Xavier NX, and install cuda10.2, cudnn7.5.0 and OpenCV4.1.1 according to the algorithm requirements.

[0063] S503 utilizes TensorRT to accelerate the network, reorganizes and optimizes the vertical and horizontal structures of the network, and eliminates useless network output layers to reduce computational load.

[0064] S504. Convert the pre-trained model pt file into onnx, torchscript.pt, and mlmodel, and then generate the wts file.

[0065] S505: Generate a deployment engine from the .wts file, and finally use the deployed lightweight model for inference testing to obtain the detection results, such as... Figure 10 As shown.

[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Any simple modifications, alterations, and equivalent changes made to the above embodiments based on the inventive essence shall still fall within the protection scope of the present invention.

Claims

1. A lightweight navigation marker detection method based on KDS R-CNN, characterized in that, It comprises the following steps: S1, obtaining aerial video of the visual angle of the UAV within the cruising range; S2, preparing a visual navigation data set and performing data enhancement; S3, constructing a lightweight KDS R-CNN network model; S4, training the lightweight network through knowledge distillation to improve the accuracy of the model; S5, deploying the model on an embedded platform and verifying the inference.

2. The KDS R-CNN-based lightweight navigation marker detection method according to claim 1, characterized in that, The specific operation mode of S1 is: S101, making the UAV cruise in the set visual navigation area according to the planned path, and shooting aerial images of the navigation area; S102, the UAV flies around the navigation marker in a spiral curve to obtain aerial images of different scales and different angles.

3. The KDS R-CNN-based lightweight navigation marker detection method according to claim 1, characterized in that, The specific operation mode of S2 is: further obtain images of navigation markers under different angles, different lightings and different weathers through various expansion approaches, jointly constitute a basic data set with the images obtained in S1, then use a geometric space and feature space joint image enhancement algorithm to augment the basic data set, and finally use a target detection algorithm and a KCF tracking algorithm to automatically label the navigation markers in the augmented data set.

4. The KDS R-CNN-based lightweight navigation marker detection method according to claim 1, characterized in that, The specific operation mode of S3 is: using MobileNet as the backbone network, using depth separable convolution instead of standard convolution, using a sparse and learnable proposal window to make an initial guess of the potential position of the navigation marker, then through training and parameter back propagation, constantly learning the real position of the marker in the image, using a dynamic instance interactive head to interact the RoI feature and the proposal feature, filtering out invalid proposal windows and outputting predicted target features, finally performing classification and regression prediction to obtain the final positioning and classification results, and using an iterative structure and a self-attention module to further improve the performance of the model.

5. The KDS R-CNN-based lightweight navigation marker detection method according to claim 1, characterized in that, The specific operation mode of S4 is: taking the lightweight KDS R-CNN model constructed in S3 as the student network, using the Sparse R-CNN model as the teacher network, using the output result of the teacher network as prior information to guide the student network to constantly learn and update the parameters of the student network; secondly, using the label assignment strategy of the Gaussian receptive field to convert the prior information provided by the teacher network into a Gaussian distribution, and then directly measuring the similarity between the prior information and the prediction result according to the Gaussian receptive field distance; through constant fine-tuning training of the student network, a high-precision and lightweight KDS R-CNN detection model is finally obtained.

6. The KDS R-CNN-based lightweight navigation marker detection method according to claim 1, characterized in that, The specific operation mode of S5 is: first, install the Ubuntu 18.04 system of aarch64 architecture on the Xavier NX, and configure the environment according to the algorithm requirements; secondly, use TensorRT to accelerate the network, and reorganize and optimize the vertical and horizontal structures of the network to eliminate useless network output layers; then convert the pre-trained model to generate a wts file, generate a deployment engine, and finally use the deployed lightweight model for inference test to obtain the detection result.

Images