Embedded remote sensing target detection system

Abstract

The invention relates to an embedded remote sensing target detection system, and the system comprises a data collection module which is used for collecting a ground remote sensing image; the target detection module is used for constructing a remote sensing target detection model with a directional rotating frame; wherein the remote sensing target detection model is obtained through training of a training set, and the training set comprises remote sensing detection data; and deploying the remote sensing target detection model to a domestic embedded development board RK3588, and inputting the ground remote sensing image to the remote sensing target detection model to obtain a detection result. Based on a single-stage remote sensing target detection network, model parameters and calculation amount are reduced by adopting lightweight methods such as model quantification and pruning, and the model parameters and the calculation amount are deployed to a domestic RK3588 development board with NPU acceleration to form an embedded remote sensing target detection system.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to an embedded remote sensing target detection system. Background Art

[0002] High-resolution remote sensing earth observation technology uses remote sensing satellites or aircraft and other carriers to conduct high-resolution, all-weather, and large-scale observation and monitoring of the earth's surface to obtain information such as surface cover, topography, resources and environment, and climate change, and is widely used in many fields such as industrial and agricultural production, urban construction, resource management, and environmental protection. Thanks to the vigorous development of technologies such as artificial intelligence, big data, and cloud computing in recent years, remote sensing image processing and interpretation have also entered the intelligent stage. As a key link in the intelligent interpretation of remote sensing images, remote sensing image target detection technology automatically locates and identifies ground objects in remote sensing images through technologies such as computer vision and machine learning, and has broad application prospects in many major national and people's livelihood needs.

[0003] Remote sensing image target detection models based on deep learning usually have high computational complexity, while the computing resources of satellite-borne, airborne and other remote sensing platforms are very limited. For many detection tasks with rapid response requirements, complex models are difficult to deploy on edge terminals to achieve on-orbit real-time processing and target detection. Summary of the invention

[0004] The purpose of the present invention is to provide an embedded remote sensing target detection system, which is based on a single-stage remote sensing target detection network and adopts lightweight methods such as model quantization and pruning to reduce model parameters and calculation amount, and is deployed to the RK3588 development board with NPU acceleration to form an embedded remote sensing target detection system.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] An embedded remote sensing target detection system, comprising:

[0007] Data acquisition module, used to collect ground remote sensing images;

[0008] The target detection module is used to build a remote sensing target detection model with a directional rotation frame; wherein the remote sensing target detection model is obtained by training with a training set, and the training set includes: remote sensing detection data; the remote sensing target detection model is deployed to a development board, and the ground remote sensing image is input into the remote sensing target detection model to obtain a detection result.

[0009] Optionally, the data acquisition module includes:

[0010] The data acquisition submodule is used to collect ground remote sensing images through the camera in the drone.

[0011] Optionally, the target detection module includes:

[0012] A model building submodule is used to build a remote sensing target detection model with a directional rotating frame; wherein the remote sensing target detection model is obtained by training with a training set, and the training set includes: remote sensing detection data, and a rotating frame detection loss function is designed;

[0013] The target detection submodule is used to perform lightweight processing on the remote sensing target detection model, deploy the lightweight remote sensing target detection model to a development board, and deploy the development board to a drone payload, thereby inputting the ground remote sensing image into the lightweight remote sensing target detection model to obtain a detection result.

[0014] Optionally, the remote sensing target detection model with a directional rotating frame includes:

[0015] The backbone network is used to input the ground remote sensing image, and uses the structure reparameterization method through the RepVGG structure to aggregate the three branches into a 3×3 convolution branch to extract the image features of the ground remote sensing image;

[0016] Neck network, used to obtain information of different receptive fields;

[0017] The head network is used to add an angle regression branch based on the YOLOX detection head, adopt KLD loss as the loss function, and obtain the detection result based on the image features and the information of the different receptive fields.

[0018] Optionally, performing lightweight processing on the remote sensing target detection model includes: performing lightweight processing on the remote sensing target detection model by using model pruning.

[0019] Optionally, the target detection module further includes:

[0020] The storage submodule is used to store the detection result in a storage device on the development board.

[0021] Optionally, the target detection module further includes:

[0022] The data transmission submodule is used to transmit the detection result to the ground host.

[0023] Optionally, transmitting the detection result to a ground host includes: transmitting the detection result to a ground host based on wireless WiFi or Bluetooth.

[0024] The beneficial effects of the present invention are:

[0025] The present invention uses the embedded development board RK3588 equipped with NPU as hardware, deploys an anchor-free high-precision remote sensing target detection network based on the improved training of the YOLOX target detection algorithm on the board, and performs lightweight processing on the network to meet the requirements of limited computing resources of the embedded platform. Finally, the development board is mounted on the drone payload. During the flight of the drone, the camera is used to capture visible light remote sensing images of the ground in real time and perform target detection. The detection results can be stored on the board and transmitted to the ground server in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0027] Figure 1 Schematic diagram of an embedded remote sensing target detection system according to an embodiment of the present invention; wherein: 1. remote sensing target detection model; 2. lightweight design of the model; 3. drone payload; 4. camera; 5. RK3588 embedded development board; 6. storage device; 7. receiving display terminal;

[0028] Figure 2 The following is a flowchart of an embedded remote sensing target detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] like Figure 1 As shown, this embodiment discloses an embedded remote sensing target detection system, including: a data acquisition module, used to collect ground remote sensing images; a target detection module, used to build a remote sensing target detection model with a directional rotation frame; wherein the remote sensing target detection model is obtained by training with a training set, and the training set includes: remote sensing detection data; the remote sensing target detection model is deployed to the embedded development board RK3588, and the ground remote sensing image is input into the remote sensing target detection model to obtain the detection result;

[0032] The data acquisition module includes: a data acquisition submodule, which is used to collect ground remote sensing images through the camera 4 in the drone and transmit the video stream to the embedded development board;

[0033] The target detection module includes: a model building submodule, which is used to build a remote sensing target detection model 1 with a directional rotation frame; wherein the remote sensing target detection model 1 is obtained by training with a training set, and the training set includes: remote sensing detection data, and a rotation frame detection loss function is designed; a target detection submodule, which is used to perform lightweight processing on the remote sensing target detection model, which can reduce the amount of calculation and the amount of parameters of the remote sensing target detection model, and deploy the lightweight remote sensing target detection model 2 to the RK3588 embedded development board 5, and deploy the embedded development board RK3588 to the drone payload 3, and the drone payload: used to carry an embedded board, which is fixed at the bottom of the drone, so as to input the ground remote sensing image into the lightweight remote sensing target detection model 2 to obtain the detection result;

[0034] The remote sensing target detection model with oriented rotation frame 1 includes:

[0035] The backbone network is used to input ground remote sensing images. It uses the RepVGG structure to maintain high detection accuracy while having a faster reasoning speed. The RepVGG structure uses a structural reparameterization method. During training, each convolution block contains three branches and a BN layer: 3×3 convolution, 1×1 convolution, and a residual connection. During reasoning, the three branches are aggregated into a 3×3 convolution branch, so that high detection accuracy can be maintained while having a faster reasoning speed.

[0036] Neck network, used to obtain information of different receptive fields;

[0037] The head network is used to design a directional rotating frame remote sensing target detection network based on the YOLOX detection head. YOLOX adopts a decoupled head design, first using a 1x1 convolution to reduce the dimension, and using two 3x3 convolutions in the classification and regression branches to extract features, and finally outputting the classification head and regression head to obtain the classification and regression results respectively. An angle regression branch is added to the YOLOX detection head. The angle branch is connected to the regression feature extraction convolution, outputs an angle information, and uses KLD loss as the loss function to finally obtain the detection result.

[0038] The target detection module also includes: a storage submodule, which is used to store the detection results in a storage device 6 on the embedded development board RK3588;

[0039] The target detection module also includes: a data transmission submodule, which is used to transmit the detection results to the ground host terminal 7, specifically including:

[0040] like Figure 2 As shown in the figure, firstly, a remote sensing target detection model is designed and trained on the server side. Remote sensing image target detection often has problems such as low detection accuracy, poor robustness, and failure to guarantee real-time performance. In order to better improve the detection accuracy of targets in remote sensing images, rotating frame detection is generally used. Compared with horizontal frames, rotating frames can provide more accurate direction and scale information. According to the characteristics of remote sensing images themselves, a directional rotating frame remote sensing target detection network 1 is designed based on the anchor-free frame detection network YOLOX, a rotating frame detection loss function is designed, and a remote sensing target detection model is obtained by training using a remote sensing detection dataset.

[0041] The remote sensing target detection system is implemented on the UAV payload platform through the RK3588 development board 5. In order to infer the remote sensing target detection model on limited computing resources, the model is lightweight 2. For the designed remote sensing target detection model, lightweight methods such as model pruning and quantization are used to simplify the model structure, reduce the number of model parameters and calculation, and meet the computing requirements of subsequent embedded devices.

[0042] Remote sensing image target detection often has problems such as low detection accuracy, poor robustness, and inability to guarantee real-time performance. In order to better improve the detection accuracy of targets in remote sensing images, rotating frame detection is generally used. Compared with horizontal frames, rotating frames can provide more accurate direction and scale information. In view of the characteristics of remote sensing images, the backbone part of the remote sensing target detection network uses the RepVGG structure to maintain high detection accuracy while having a faster reasoning speed. The neck part uses PAN to obtain information from different receptive fields. The head part is based on the head of the anchor-free frame detection network YOLOX to design a directional rotating frame remote sensing target detection network. YOLOX adopts a decoupled head design, first using a 1x1 convolution for dimensionality reduction, then connecting the classification and regression branches. The classification and regression branches each use two 3x3 convolutions to extract features, and finally output the classification head and regression head to obtain the classification and regression results respectively. On this basis, angle loss is added after the classification branch, and Kullback-Leibler Divergence (KLD) is used as the regression loss to calculate the loss function between the detection frame and the target frame Gaussian distribution, and finally the directional rotation detection frame is output. A remote sensing image target detection network is built under the Pytorch framework, and the SGD gradient descent method is used to train 12 epochs on the DOTA1.0 dataset. After training, the model can acquire the ability to detect ground vehicles, ships, airplanes, stadiums and other targets.

[0043] The remote sensing target detection system is implemented on the UAV payload platform through the RK3588 development board 5. In order to infer the remote sensing target detection model on limited computing resources, the model is lightweight. First, the pruning method based on the geometric median is used to filter out the convolution kernels that are far from the geometric center. These convolution kernels usually contribute less to the network. Then, the knowledge distillation-driven quantization-aware training method is used to quantize the model, and the original floating-point model is used to supervise the training and correction of the quantized model. Through pruning and quantization methods, the number of model parameters and the amount of calculation can be reduced to meet the computing requirements of subsequent embedded devices.

[0044] The lightweight remote sensing target detection network is deployed on the RK3588 development board 5 and fixed on the drone payload 3. During the flight of the drone, the camera 1 takes real-time ground remote sensing images and performs target detection on the RK3588 development board 5. The detection results can be stored in the storage device 6 on the board or transmitted to the ground host 7 via wireless wifi or Bluetooth communication. All devices constitute an embedded remote sensing target detection system to meet the needs of real-time ground remote sensing image detection.

[0045] The above embodiments are only descriptions of the preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the protection scope determined by the claims of the present invention.

Claims

1. An embedded remote sensing target detection system, characterized in that: include: Data acquisition module, used to collect ground remote sensing images; The target detection module is used to build a remote sensing target detection model with a directional rotation frame; wherein the remote sensing target detection model is obtained by training with a training set, and the training set includes: remote sensing detection data; the remote sensing target detection model is deployed to a development board, and the ground remote sensing image is input into the remote sensing target detection model to obtain a detection result.

2. The embedded remote sensing target detection system according to claim 1, characterized in that: The data acquisition module comprises: The data acquisition submodule is used to collect ground remote sensing images through the camera in the drone.

3. The embedded remote sensing target detection system according to claim 1, characterized in that: The target detection module comprises: The model building submodule is used to build a remote sensing target detection model with a directional rotating frame; wherein the remote sensing target detection model is obtained by training with a training set, and the training set includes: remote sensing detection data, and a rotating frame detection loss function KLD is designed; The target detection submodule is used to perform lightweight processing on the remote sensing target detection model, deploy the lightweight remote sensing target detection model to the development board, and deploy the embedded development board RK3588 to the drone payload, so as to input the ground remote sensing image into the lightweight remote sensing target detection model to obtain the detection result.

4. The embedded remote sensing target detection system according to claim 3, characterized in that: The remote sensing target detection model with a directional rotating frame includes: The backbone network is used to input the ground remote sensing image, and uses the structure reparameterization method through the RepVGG structure to aggregate the three branches into a 3×3 convolution branch to extract the image features of the ground remote sensing image; Neck network, used to obtain information of different receptive fields; The head network is used to add an angle regression branch based on the YOLOX detection head, adopt KLD loss as the loss function, and obtain the detection result based on the image features and the information of the different receptive fields.

5. The embedded remote sensing target detection system according to claim 3, characterized in that: The lightweight processing of the remote sensing target detection model includes: lightweight processing of the remote sensing target detection model by using model pruning or model quantization.

6. The embedded remote sensing target detection system according to claim 1, characterized in that: The target detection module also includes: The storage submodule is used to store the detection result in a storage device on the development board.

7. The embedded remote sensing target detection system according to claim 1, characterized in that: The target detection module also includes: The data transmission submodule is used to transmit the detection result to the ground host.

8. The embedded remote sensing target detection system according to claim 7, characterized in that: Transmitting the detection result to the ground host includes: transmitting the detection result to the ground host based on wireless WiFi or Bluetooth.

Images