A lightweight target detection method and system based on unmanned aerial vehicle platform
By embedded lightweight object detection model on the drone platform, combining backbone network, path aggregation network and detection head, the problem of difficulty in taking into account accuracy and efficiency of target detection technology under the drone platform is solved, and small targets and Efficient detection of complex backgrounds improves detection accuracy and speed.
Patent Information
- Application Number
- CN202411073757.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-08-07
AI Technical Summary
The target detection technology under the existing drone platform is difficult to take into account both detection accuracy and efficiency. Especially when the flight altitude changes, fast speed, complex scenes and limited load capacity, it increases the difficulty of object detection tasks, and existing methods often have errors. Inspection and missed inspection.
Using a lightweight object detection method based on the drone platform, a lightweight object detection model completed and embedded in the drone platform is adopted, including backbone network, path aggregation network and detection head, through a 1*1 convolutional layer and The bidirectional connection layer reduces the number of feature map channels, introduces graph depth convolution network and spatial pyramid pooling layer, and combines the decoupling head structure to achieve cross-scale feature fusion and real-time detection.
Improve the extraction ability of small targets, reduce the calculation amount, enhance the adaptability to complex backgrounds and object size changes, ensure the real-time and accuracy of detection, reduce the amount of parameters and calculation amount, improve the detection accuracy and speed .
Smart Images

Figure CN118587622B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of target detection, and in particular relates to a lightweight target detection method and system based on an unmanned aerial vehicle platform. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] Due to their wide field of view and strong real-time performance, drones have gradually become an important tool for remote sensing and are widely used in various fields, including emergency rescue, disaster monitoring, and urban traffic management. In the field of drone-based object detection, the large altitude fluctuations and high speeds of aircraft often lead to blurred objects and a high proportion of small objects in the scene, which increases the difficulty of object detection. Furthermore, the limited payload and energy resources of drone platforms make it difficult to carry high-power, high-performance computing equipment for object detection. Current applications typically transmit raw images over a network to a ground-based processing terminal for processing, which cannot meet the requirements of real-time detection. Existing two-stage detection methods for drone-based object detection first generate candidate bounding boxes through end-to-end learning using a Region Proposal Network (RPN). Then, deep neural networks (such as ResNet and VGG) are used to extract features for each candidate box. Finally, a classifier and regressor are used to classify the candidate boxes and fine-tune the position of the bounding boxes for accurate fit. Representative algorithms include Faster R-CNN and SPPNet. Although this type of method has good detection performance, the duplication of candidate regions will lead to a large amount of calculation, which makes it difficult to meet the real-time detection requirements on low-computing power platforms.
[0004] Existing single-stage object detection methods for drone platforms, such as the YOLO series, SSD, and RetinaNet, do not rely on an explicit candidate bounding box generation stage and instead directly densely predict the location and category of objects on the input image. While these methods offer advantages over two-stage algorithms in terms of speed, they struggle to achieve high accuracy due to factors such as target diversity and complex backgrounds, resulting in frequent false detections and missed detections. Summary of the Invention
[0005] In order to solve the technical problem in the above-mentioned background technology that the current target detection technology under the UAV platform cannot take into account both the target detection accuracy and efficiency at the same time, the present invention provides a lightweight target detection method and system based on the UAV platform, which can effectively balance the detection accuracy and speed.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] The first aspect of the present invention provides a lightweight target detection method based on an unmanned aerial vehicle platform.
[0008] A lightweight target detection method based on an unmanned aerial vehicle platform, comprising:
[0009] Acquire drone inspection images;
[0010] Using a lightweight target detection model that has been trained and embedded in the UAV platform, the UAV inspection image is processed to obtain target detection results;
[0011] The lightweight target detection model includes a backbone network, a path aggregation network and a detection head;
[0012] The backbone network is used to extract spatial information and semantic information of several layers at different scales from the drone inspection image to obtain feature maps of several layers;
[0013] The path aggregation network includes a 1*1 convolutional layer and a bidirectional connection layer; the 1*1 convolutional layer is used to reduce the number of channels of the corresponding layer feature map extracted by the backbone network before transmitting it to the bidirectional connection layer; the bidirectional connection layer is used to connect the feature map output by the 1*1 convolutional layer and the corresponding feature map directly output by the backbone network, and on this basis, add a bottom-up cross-scale connection path to obtain multiple fused feature maps;
[0014] The detection head is used to detect each fusion feature map to obtain a corresponding result map containing the target category and position, and then these result maps are processed by the non-maximum suppression method to obtain the final target category and position coordinates.
[0015] As an implementation method, the path aggregation network is used to fuse the third-layer feature map extracted by the backbone network and the feature maps thereafter.
[0016] As an implementation method, the backbone network includes several feature extraction layers and spatial pyramid pooling layers; the feature extraction layers are connected in series, and the spatial pyramid pooling layers are connected in series to the back end of the series-connected feature extraction layers.
[0017] A graph depth convolutional network is introduced into the feature extraction layer, and the graph depth convolutional network processes the input original feature map to obtain a feature map containing global information and local spatial information;
[0018] The graph deep convolutional network includes two branches, one of which includes a local feature extraction network and a hybrid local channel attention mechanism module. The local feature extraction network is used to extract local features of the original feature map, and then generates a local feature map through the hybrid local channel attention mechanism module; the other branch directly transmits the original feature map to the sum calculation module, and the sum calculation module sums the original feature map and its corresponding local feature map to obtain a feature map containing global information and local spatial information.
[0019] As an embodiment, the spatial pyramid pooling layer includes a first lightweight depth-wise separable convolutional network, several pooling convolution kernels connected in series, a splicing layer and a second lightweight depth-wise separable convolutional network; the first lightweight depth-wise separable convolutional network is used to extract features from the input feature map, and the extracted features are divided into two paths, one path is pooled step by step through several pooling convolution kernels connected in series and transmitted to the splicing layer, and the other path is directly transmitted to the splicing layer; the splicing layer is used to splice all received features, and then transmit them to the second lightweight depth-wise separable convolutional network for further feature extraction.
[0020] As an embodiment, the detection head includes a decoupling head structure, and the input end of the decoupling head structure is further connected in series with a partial convolution module with an inverted residual structure;
[0021] The partial convolution module of the inverted residual structure is used to: use point multiplication convolution to expand the number of channels of the feature map, use the partial convolution network to perform calculations in high-dimensional space after normalization and activation function processing, and finally restore the number of channels through the inverse process of dimensionality increase, and send the obtained feature map to the decoupling head structure for processing.
[0022] As an implementation method, the partial convolutional network is used to perform a convolution operation on some channels of the feature map, and concatenate the calculated convolution results with the remaining channels of the feature map.
[0023] As an implementation method, the activation function is an H-Swish function.
[0024] A second aspect of the present invention provides a lightweight target detection system based on an unmanned aerial vehicle platform.
[0025] In one or more embodiments, a lightweight target detection system based on an unmanned aerial vehicle platform includes:
[0026] An image acquisition module, which is used to acquire drone inspection images;
[0027] A target detection module, which is used to process the UAV inspection images using a lightweight target detection model that has been trained and embedded in the UAV platform to obtain target detection results;
[0028] The lightweight target detection model includes a backbone network, a path aggregation network and a detection head;
[0029] The backbone network is used to extract spatial information and semantic information of several layers at different scales from the drone inspection image to obtain feature maps of several layers;
[0030] The path aggregation network includes a 1*1 convolutional layer and a bidirectional connection layer; the 1*1 convolutional layer is used to reduce the number of channels of the corresponding layer feature map extracted by the backbone network before transmitting it to the bidirectional connection layer; the bidirectional connection layer is used to connect the feature map output by the 1*1 convolutional layer and the corresponding feature map directly output by the backbone network, and on this basis, add a bottom-up cross-scale connection path to obtain multiple fused feature maps;
[0031] The detection head is used to detect each fusion feature map to obtain a corresponding result map containing the target category and position, and then these result maps are processed by the non-maximum suppression method to obtain the final target category and position coordinates.
[0032] In one or more embodiments, a lightweight target detection system based on an unmanned aerial vehicle platform includes:
[0033] Image acquisition module and image processing module;
[0034] The image acquisition module is mounted on the UAV platform; the image processing module is embedded in the UAV platform;
[0035] The image acquisition module is used to collect the UAV inspection images and transmit them to the image processing module;
[0036] The image processing module is deployed with a lightweight target detection model, which is used to process the UAV inspection image to obtain a target detection result;
[0037] The lightweight target detection model includes a backbone network, a path aggregation network and a detection head;
[0038] The backbone network is used to extract spatial information and semantic information of several layers at different scales from the drone inspection image to obtain feature maps of several layers;
[0039] The path aggregation network includes a 1*1 convolutional layer and a bidirectional connection layer; the 1*1 convolutional layer is used to reduce the number of channels of the corresponding layer feature map extracted by the backbone network before transmitting it to the bidirectional connection layer; the bidirectional connection layer is used to connect the feature map output by the 1*1 convolutional layer and the corresponding feature map directly output by the backbone network, and on this basis, add a bottom-up cross-scale connection path to obtain multiple fused feature maps;
[0040] The detection head is used to detect each fusion feature map to obtain a corresponding result map containing the target category and position, and then these result maps are processed by the non-maximum suppression method to obtain the final target category and position coordinates.
[0041] Compared with the prior art, the present invention has the following beneficial effects:
[0042] (1) The path aggregation network of the present invention uses a 1*1 convolutional layer to reduce the number of channels of the corresponding layer feature map extracted by the backbone network, and then uses a bidirectional connection layer to connect the feature map output by the 1*1 convolutional layer and the corresponding feature map directly output by the backbone network. On this basis, a bottom-up cross-scale connection path is added to fuse the feature maps, thereby realizing the fusion of spatial information in the shallow feature extraction map of the backbone network, improving the ability to extract small targets, and reducing the amount of calculation. The bottom-up cross-scale connection path introduced realizes the fusion of feature maps extracted by different-sized hole convolution kernels, reducing the impact of size differences.
[0043] (2) The feature extraction layer of the present invention introduces a graph deep convolutional network, which includes two branches, one of which includes a local feature extraction network and a hybrid local channel attention mechanism module, and the other branch directly transmits the original feature map to the sum calculation module. The graph deep convolutional network integrates the adaptability of variable convolution to the shape and size of objects, and utilizes the ability of the hybrid local channel attention mechanism to extract important information, effectively solving the problems of interference from complex backgrounds and large changes in object size.
[0044] (3) The input end of the decoupling head structure of the detection head of the present invention is also connected in series with a partial convolution module with an inverted residual structure. This ensures that while ensuring the detection accuracy brought by the decoupling head, the number of parameters and the amount of calculation are reduced, thus ensuring the real-time performance of the entire lightweight target detection model.
[0045] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0047] Figure 1 This is a flow chart of a lightweight target detection method based on a UAV platform according to an embodiment of the present invention;
[0048] Figure 2 This is a schematic diagram of the YOLOv8s model structure;
[0049] Figure 3 2 is a schematic diagram of the DGLFG module structure according to an embodiment of the present invention;
[0050] Figure 4 2 is a schematic diagram of the structure of the MLCA attention mechanism module according to an embodiment of the present invention;
[0051] Figure 5 2 is a schematic structural diagram of a spatial pyramid pooling layer according to an embodiment of the present invention;
[0052] Figure 6 is a schematic diagram of the path aggregation network structure according to an embodiment of the present invention;
[0053] Figure 7 A schematic structural diagram of a detection head according to an embodiment of the present invention;
[0054] Figure 8 It is a comparison chart of the detection effects of different target detection models;
[0055] Figure 9 Schematic diagram of the structure of a lightweight target detection system based on an unmanned aerial vehicle platform according to an embodiment of the present invention. DETAILED DESCRIPTION
[0056] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0057] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.
[0058] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.
[0059] Figure 1 This is a flow chart of a lightweight target detection method based on a UAV platform according to an embodiment of the present invention. Figure 1 The lightweight target detection method based on the UAV platform of the embodiment of the present invention includes:
[0060] S101: Acquire UAV inspection images;
[0061] S102: Using the trained lightweight target detection model embedded in the UAV platform, the UAV inspection image is processed to obtain a target detection result.
[0062] In the specific implementation process, the lightweight target detection model includes a backbone network, a path aggregation network and a detection head;
[0063] The backbone network is used to extract spatial information and semantic information of several layers at different scales from the drone inspection image to obtain feature maps of several layers;
[0064] The path aggregation network includes a 1*1 convolution layer and a bidirectional connection layer; the 1*1 convolution layer is used to reduce the number of channels of the corresponding layer feature map extracted by the backbone network, and then transmit it to the bidirectional connection layer; the bidirectional connection layer is used to connect the feature map output by the 1*1 convolution layer and the corresponding feature map directly output by the backbone network, and on this basis, add a bottom-up cross-scale connection path to obtain several fused feature maps, such as Figure 6 As shown;
[0065] The detection head is used to detect each fusion feature map to obtain a corresponding result map containing the target category and position, and then these result maps are processed by the non-maximum suppression method to obtain the final target category and position coordinates.
[0066] exist Figure 6 middle:
[0067] SiLU: Sigmoid Linear Unit, activation function;
[0068] P stands for Pyramid. Feature maps are usually named hierarchically, representing different resolutions or scales. P3-P7 are the feature maps of the third to seventh layers in the feature pyramid, respectively.
[0069] This embodiment achieves cross-scale connection by adding features in the backbone feature extraction network and the bottom-up path, retains shallow spatial information, and enhances the ability to extract small targets.
[0070] It should be noted here that the backbone network can be implemented using an existing neural network structure, such as the target detection model of the YOLO system. Those skilled in the art can make specific choices based on actual conditions.
[0071] Let's take YOLOv8s as an example. Figure 2 As shown in the figure, YOLOv8s consists of a backbone network, a neck network, and a detection head. The backbone network is used to extract features from the input drone inspection images to obtain spatial information and semantic information at different scales.
[0072] in, Figure 2 The English abbreviations in the meaning are:
[0073] C2f: Channel to pixel, channel to pixel;
[0074] CBS: C stands for Conv (i.e., convolution); B stands for BatchNorm2d (BatchNormalization); S stands for SiLU (Sigmoid Linear Unit, activation function);
[0075] SPPF: Spatial Pyramid Pooling Fast, spatial pyramid fast pooling.
[0076] Among them, spatial information refers to the pixel value information at different locations in the image, including geometric features such as edges, textures, and simple shapes;
[0077] Semantic information refers to the category and attribute information of each area of the image, which is represented by each pixel in the feature map. Specifically, it includes:
[0078] Category probability: the category distribution of the predicted target at each position;
[0079] Target confidence: predict whether there is a target at each position;
[0080] Bounding box regression: predict the position, width, and height of the object at each location.
[0081] In this embodiment, the path aggregation network is used to fuse the third-layer feature map extracted by the backbone network and subsequent feature maps. This can enhance the semantic expression and positioning capabilities at multiple scales and reduce the amount of data calculation.
[0082] In this embodiment, the backbone network includes several feature extraction layers and spatial pyramid pooling layers; the feature extraction layers are connected in series, and the spatial pyramid pooling layers are connected in series to the back end of the series-connected feature extraction layers.
[0083] A graph depth convolutional network is introduced into the feature extraction layer, and the graph depth convolutional network processes the input original feature map to obtain a feature map containing global information and local spatial information;
[0084] The graph deep convolutional network includes two branches, one of which includes a local feature extraction network and a hybrid local channel attention mechanism module. The local feature extraction network is used to extract local features of the original feature map, and then generates a local feature map through the hybrid local channel attention mechanism module; the other branch directly transmits the original feature map to the sum calculation module, and the sum calculation module sums the original feature map and its corresponding local feature map to obtain a feature map containing global information and local spatial information.
[0085] In this embodiment, the graph deep convolutional network is defined as a DGLFG (Deformable Global-Local Feature Guidance) module, such as Figure 3 As shown in Figure 2, it is used to replace the C2f module in the backbone network. The DGLFG module can meet the shape and multi-scale differences of the detected target.
[0086] exist Figure 3 middle:
[0087] CBS: C stands for Conv (i.e. Convolution); B stands for BatchNorm2d function, normalization processing function; S stands for SiLU function (Sigmoid Linear Unit, activation function);
[0088] DCBS is the abbreviation of a network consisting of DCNv3 function, batch normalization function and SiLU function connected in series;
[0089] DCNv3: Deformable Convolutional Networks Version 3, deformable convolutional network;
[0090] SiLU: Sigmoid Linear Unit, activation function;
[0091] MLCA: Mixed Local Channel Attention, mixed local channel attention.
[0092] The 3×3 convolutional network in the Darknet Bottleneck (deep learning bottleneck) part of the original C2f module is replaced with a deformable convolutional network. Compared with the original network, this convolutional network expands the shallow feature receptive field, enabling the network to adapt to the problem of large differences in target size. The feature map processed by the convolutional network is then input into the MLCA attention mechanism module. The module structure is as follows Figure 4 As shown in the figure, the feature map is first converted into C×d×d by local average pooling, where C is the number of channels and d is the length and width. This step can quickly reduce the amount of calculation while extracting local spatial features. The feature map is further restored to the original feature map size through steps such as global average pooling and 1×1 convolution. The processed feature map is added to the original feature map so that the resulting feature map contains global information and local spatial information.
[0093] In this embodiment, if Figure 5As shown in the figure, the spatial pyramid pooling fast (SPPF) layer includes a first lightweight depth-wise separable convolutional network, several pooling convolution kernels connected in series (such as 3×3, 5×5 and 9×9 pooling convolution kernels), a splicing layer and a second lightweight depth-wise separable convolutional network; wherein the output of each pooling is used as the input of the next pooling, expanding the receptive field of shallow features and enabling the network to adapt to the problem of large differences in target size.
[0094] The first lightweight depth-wise separable convolutional network of this embodiment is used to extract features from the input feature map, and divides the extracted features into two paths. One path is pooled step by step through several pooling convolution kernels connected in series and transmitted to the splicing layer, and the other path is directly transmitted to the splicing layer; the splicing layer is used to splice all the received features, and then transmit them to the second lightweight depth-wise separable convolutional network for further feature extraction.
[0095] Specifically, if Figure 7 As shown, the detection head includes a decoupling head structure, and the input end of the decoupling head structure is further connected in series with a partial convolution module with an inverted residual structure;
[0096] The partial convolution module of the inverted residual structure is used to: use point multiplication convolution to expand the number of channels of the feature map, use the partial convolution network to perform calculations in high-dimensional space after normalization and activation function processing, and finally restore the number of channels through the inverse process of dimensionality increase, and send the obtained feature map to the decoupling head structure for processing.
[0097] The English abbreviation of the partial convolution module of the inverted residual structure is: ISPP module, which is the Inverse-residualShared Parameter based on Partial convolution module.
[0098] The partial convolutional network is used to perform a convolution operation on some channels of the feature map (for example, 1 / 3 of the channels), and concatenate the calculated convolution results with the remaining channels of the feature map (for example, 2 / 3 of the channels).
[0099] The activation function is the H-Swish (Hard Swish) function. The expression of the H-Swish function is: ;
[0100] in represents the input factor, The ReLU activation function represents a value range of [0,6], which is designed to adapt to low-precision mobile devices.
[0101] exist Figure 7In
[15] , BN_HSwish is a combination of Batch Normalization (BN) and H-Swish activation function. h is the height of the input feature map; w is the width of the input feature map; Cp is the number of partial channels; and k is the height and width of the feature map processed by the filter.
[0102] The following is a detailed description of the training process of the lightweight target detection model:
[0103] Step 1: Make a dataset.
[0104] Step 1.1: Data Collection. We collected typical datasets from two drone platforms, VisDrone2019 and UAVDT, as sample data. The main scenarios involved include low light levels, occlusions, and small objects. The VisDrone2019 dataset includes data from ten categories, including people, bicycles, and cars.
[0105] Step 1.2: Data Partitioning. To demonstrate the effectiveness of the present invention, the data in this embodiment was randomly divided into a training set, a validation set, and a test set according to a ratio of 6:2:2. The training set and validation set were used to train the model parameters, and the test set was used to test the model performance, ensuring that the three data sets did not overlap.
[0106] Step 1.3: Data Preprocessing. To improve detection efficiency, the model input data is resampled to 640×640. Random cropping, flipping, mixup, and mosaic techniques are also used to expand the training dataset, reduce class imbalance, and improve the robustness of the lightweight object detection model.
[0107] Step 2: Train and validate the model. The training dataset from Step 1 is fed into the constructed lightweight object detection model for training. The training parameters are set as follows: input image size 640×640, batch size 16, SGD optimizer, initial learning rate 0.01, and 300 iterations. The trained lightweight object detection model is obtained.
[0108] Step 3: Model export. Export the trained lightweight object detection model's pt format weight file as a lightweight object detection model in ONNX format. After compilation by the toolchain, convert it into a lightweight object detection model in rknn format suitable for NPU devices.
[0109] Step 4: Deploy the trained lightweight object detection model to the drone platform. The drone platform is equipped with the RK3588 chip. The trained lightweight object detection model is deployed to the board equipped with the RK3588 chip. Inference is performed by accessing multiple video streams. After performing non-maximum suppression processing, the model returns the target coordinates and category confidence.
[0110] Table 1 compares the performance of the lightweight target detection model of the present invention with that of YOLOv8s. As shown in Table 1, the lightweight target detection method based on the UAV platform is simple in principle and easy to implement. Compared with YOLOv8s, the lightweight target detection model of the present invention achieves an increase in target detection accuracy from 37.4% to 40.6%, a reduction in the number of parameters from 11.1M to 5.8M, a reduction in computational overhead from 28.5GFLOPS to 20.6GFLOPS, and an increase in inference speed from 131FPS to 147FPS.
[0111] Table 1 Performance comparison between the lightweight object detection model of the present invention and YOLOv8s.
[0112]
[0113] Figure 8 This is a comparison chart of the detection effects of different target detection models, where the target detection models include YOLOv5s, YOLOv8s and the lightweight target detection model of the present invention. Figure 8 It can be seen that the detection accuracy and precision of the lightweight target detection model of the present invention are higher than those of YOLOv5s and YOLOv8s.
[0114] Figure 9 FIG is a schematic diagram of the structure of a lightweight target detection system based on a UAV platform according to an embodiment of the present invention. Figure 9 As shown, a lightweight target detection system based on an unmanned aerial vehicle platform according to an embodiment of the present invention includes:
[0115] Image acquisition module 201, which is used to acquire drone inspection images;
[0116] The target detection module 202 is used to process the UAV inspection image using a lightweight target detection model that has been trained and embedded in the UAV platform to obtain target detection results;
[0117] The lightweight target detection model includes a backbone network, a path aggregation network and a detection head;
[0118] The backbone network is used to extract spatial information and semantic information of several layers at different scales from the drone inspection image to obtain feature maps of several layers;
[0119] The path aggregation network includes a 1*1 convolutional layer and a bidirectional connection layer; the 1*1 convolutional layer is used to reduce the number of channels of the corresponding layer feature map extracted by the backbone network before transmitting it to the bidirectional connection layer; the bidirectional connection layer is used to connect the feature map output by the 1*1 convolutional layer and the corresponding feature map directly output by the backbone network, and on this basis, add a bottom-up cross-scale connection path to obtain multiple fused feature maps;
[0120] The detection head is used to detect each fusion feature map to obtain a corresponding result map containing the target category and position, and then these result maps are processed by the non-maximum suppression method to obtain the final target category and position coordinates.
[0121] It should be noted here that the various modules in the lightweight target detection system based on the drone platform in this embodiment correspond one-to-one to the various steps in the above-mentioned lightweight target detection method based on the drone platform, and their specific implementation processes are the same, which will not be described in detail here.
[0122] Moreover, the specific structure of the lightweight target detection model is the same as that in the above-mentioned lightweight target detection method based on the UAV platform, and will not be repeated here.
[0123] In one or more embodiments, a lightweight target detection system based on a UAV platform is also provided, which includes: an image acquisition module and an image processing module;
[0124] The image acquisition module is mounted on the UAV platform; the image processing module is embedded in the UAV platform;
[0125] The image acquisition module is used to collect the UAV inspection images and transmit them to the image processing module;
[0126] The image processing module is deployed with a lightweight target detection model, which is used to process the UAV inspection image to obtain a target detection result;
[0127] The lightweight target detection model includes a backbone network, a path aggregation network and a detection head;
[0128] The backbone network is used to extract spatial information and semantic information of several layers at different scales from the drone inspection image to obtain feature maps of several layers;
[0129] The path aggregation network includes a 1*1 convolutional layer and a bidirectional connection layer; the 1*1 convolutional layer is used to reduce the number of channels of the corresponding layer feature map extracted by the backbone network before transmitting it to the bidirectional connection layer; the bidirectional connection layer is used to connect the feature map output by the 1*1 convolutional layer and the corresponding feature map directly output by the backbone network, and on this basis, add a bottom-up cross-scale connection path to obtain multiple fused feature maps;
[0130] The detection head is used to detect each fusion feature map to obtain a corresponding result map containing the target category and position, and then these result maps are processed by the non-maximum suppression method to obtain the final target category and position coordinates.
[0131] It should be noted here that the specific structure of the lightweight target detection model is the same as that in the above-mentioned lightweight target detection method based on the UAV platform, and will not be repeated here.
[0132] In one or more embodiments, a computer-readable storage medium is further provided on which a computer program is stored. When the program is executed by a processor, the above-mentioned Figure 1 The steps in the lightweight target detection method based on UAV platform are shown.
[0133] In one or more embodiments, an electronic device is further provided, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, wherein when the processor executes the program, the above-mentioned Figure 1 The steps in the lightweight target detection method based on UAV platform are shown.
[0134] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products of the embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams and the combination of processes and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0135] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.
Claims
1. A lightweight target detection method based on an unmanned aerial vehicle platform, characterized in that: include: Obtain drone inspection images; Using a lightweight target detection model that has been trained and embedded in the UAV platform, the UAV inspection image is processed to obtain a target detection result; Wherein, the lightweight target detection model includes a backbone network, a path aggregation network and a detection head; The backbone network is used to extract spatial information and semantic information of several layers at different scales of the drone inspection image to obtain feature maps of several layers; The backbone network includes several feature extraction layers and spatial pyramid pooling layers; A graph deep convolutional network is introduced into the feature extraction layer, and the graph deep convolutional network processes the input original feature map to obtain a feature map containing global information and local spatial information; The graph deep convolution network includes two branches, one of which includes a local feature extraction network and a hybrid local channel attention mechanism module. The local feature extraction network is used to extract local features of the original feature map, and then generate a local feature map through the hybrid local channel attention mechanism module; the other branch directly transmits the original feature map to the sum calculation module, and the sum calculation module sums the original feature map and its corresponding local feature map to obtain a feature map containing global information and local spatial information; The graph deep convolutional network is used to replace the DarknetBottleneck part of the C2f module in the backbone network; The path aggregation network includes a 1*1 convolution layer and a bidirectional connection layer; the 1*1 convolution layer is used to reduce the number of channels of the corresponding layer feature map extracted by the backbone network, and then transmit it to the bidirectional connection layer; the bidirectional connection layer is used to connect the feature map output by the 1*1 convolution layer and the corresponding feature map directly output by the backbone network, and on this basis, add a bottom-up cross-scale connection path to obtain a number of fused feature maps; The detection head is used to detect each fusion feature map to obtain a corresponding result map containing the target category and position, and then these result maps are processed by a non-maximum suppression method to obtain the final target category and position coordinates; The detection head comprises a decoupling head structure, and an input end of the decoupling head structure is further connected in series with a partial convolution module having an inverted residual structure; The partial convolution module of the inverted residual structure is used to: expand the number of channels of the feature map by point multiplication convolution, use the partial convolution network to perform calculations in a high-dimensional space after normalization and activation function processing, and finally restore the number of channels through the inverse process of dimensionality increase, and send the obtained feature map to the decoupling head structure for processing; The partial convolutional network is used to perform a convolution operation on some channels of the feature map, and concatenate the calculated convolution results with the remaining channels of the feature map.
2. The lightweight target detection method based on an unmanned aerial vehicle platform according to claim 1, characterized in that: The path aggregation network is used to fuse the third-layer feature map extracted by the backbone network and the feature maps thereafter.
3. The lightweight target detection method based on an unmanned aerial vehicle platform as claimed in claim 1, characterized in that: The feature extraction layers are connected in series, and the spatial pyramid pooling layer is connected in series to the rear end of the feature extraction layers connected in series.
4. The lightweight target detection method based on an unmanned aerial vehicle platform as claimed in claim 3, characterized in that: The spatial pyramid pooling layer includes a first lightweight depth-wise separable convolutional network, a plurality of pooling convolution kernels connected in series, a splicing layer and a second lightweight depth-wise separable convolutional network; the first lightweight depth-wise separable convolutional network is used to extract features from an input feature map, and the extracted features are divided into two paths, one path is pooled step by step through a plurality of pooling convolution kernels connected in series and transmitted to the splicing layer, and the other path is directly transmitted to the splicing layer; the splicing layer is used to splice all received features, and then transmit them to the second lightweight depth-wise separable convolutional network for further feature extraction.
5. The lightweight target detection method based on an unmanned aerial vehicle platform as claimed in claim 1, characterized in that: The activation function is the H-Swish function.
6. A lightweight target detection system based on an unmanned aerial vehicle platform, characterized in that: include: An image acquisition module, which is used to acquire UAV inspection images; A target detection module, which is used to process the UAV inspection image using a lightweight target detection model that has been trained and embedded in the UAV platform to obtain a target detection result; Wherein, the lightweight target detection model includes a backbone network, a path aggregation network and a detection head; The backbone network is used to extract spatial information and semantic information of several layers at different scales of the drone inspection image to obtain feature maps of several layers; The backbone network includes several feature extraction layers and spatial pyramid pooling layers; A graph deep convolutional network is introduced into the feature extraction layer, and the graph deep convolutional network processes the input original feature map to obtain a feature map containing global information and local spatial information; The graph deep convolution network includes two branches, one of which includes a local feature extraction network and a hybrid local channel attention mechanism module. The local feature extraction network is used to extract local features of the original feature map, and then generate a local feature map through the hybrid local channel attention mechanism module; the other branch directly transmits the original feature map to the sum calculation module, and the sum calculation module sums the original feature map and its corresponding local feature map to obtain a feature map containing global information and local spatial information; The graph deep convolutional network is used to replace the DarknetBottleneck part of the C2f module in the backbone network; The path aggregation network includes a 1*1 convolution layer and a bidirectional connection layer; the 1*1 convolution layer is used to reduce the number of channels of the corresponding layer feature map extracted by the backbone network, and then transmit it to the bidirectional connection layer; the bidirectional connection layer is used to connect the feature map output by the 1*1 convolution layer and the corresponding feature map directly output by the backbone network, and on this basis, add a bottom-up cross-scale connection path to obtain a number of fused feature maps; The detection head is used to detect each fusion feature map to obtain a corresponding result map containing the target category and position, and then these result maps are processed by a non-maximum suppression method to obtain the final target category and position coordinates; The detection head comprises a decoupling head structure, and an input end of the decoupling head structure is further connected in series with a partial convolution module having an inverted residual structure; The partial convolution module of the inverted residual structure is used to: expand the number of channels of the feature map by point multiplication convolution, use the partial convolution network to perform calculations in a high-dimensional space after normalization and activation function processing, and finally restore the number of channels through the inverse process of dimensionality increase, and send the obtained feature map to the decoupling head structure for processing; The partial convolutional network is used to perform a convolution operation on some channels of the feature map, and concatenate the calculated convolution results with the remaining channels of the feature map.
7. A lightweight target detection system based on an unmanned aerial vehicle platform, characterized in that: include: Image acquisition module and image processing module; The image acquisition module is mounted on the UAV platform; the image processing module is embedded in the UAV platform; The image acquisition module is used to collect the inspection images of the UAV and transmit them to the image processing module; The image processing module is deployed with a lightweight target detection model, and the lightweight target detection model is used to process the UAV inspection image to obtain a target detection result; Wherein, the lightweight target detection model includes a backbone network, a path aggregation network and a detection head; The backbone network is used to extract spatial information and semantic information of several layers at different scales of the drone inspection image to obtain feature maps of several layers; The backbone network includes several feature extraction layers and spatial pyramid pooling layers; A graph deep convolutional network is introduced into the feature extraction layer, and the graph deep convolutional network processes the input original feature map to obtain a feature map containing global information and local spatial information; The graph deep convolution network includes two branches, one of which includes a local feature extraction network and a hybrid local channel attention mechanism module. The local feature extraction network is used to extract local features of the original feature map, and then generate a local feature map through the hybrid local channel attention mechanism module; the other branch directly transmits the original feature map to the sum calculation module, and the sum calculation module sums the original feature map and its corresponding local feature map to obtain a feature map containing global information and local spatial information; The graph deep convolutional network is used to replace the DarknetBottleneck part of the C2f module in the backbone network; The path aggregation network includes a 1*1 convolution layer and a bidirectional connection layer; the 1*1 convolution layer is used to reduce the number of channels of the corresponding layer feature map extracted by the backbone network, and then transmit it to the bidirectional connection layer; the bidirectional connection layer is used to connect the feature map output by the 1*1 convolution layer and the corresponding feature map directly output by the backbone network, and on this basis, add a bottom-up cross-scale connection path to obtain a number of fused feature maps; The detection head is used to detect each fusion feature map to obtain a corresponding result map containing the target category and position, and then these result maps are processed by a non-maximum suppression method to obtain the final target category and position coordinates; The detection head comprises a decoupling head structure, and an input end of the decoupling head structure is further connected in series with a partial convolution module having an inverted residual structure; The partial convolution module of the inverted residual structure is used to: expand the number of channels of the feature map by point multiplication convolution, use the partial convolution network to perform calculations in a high-dimensional space after normalization and activation function processing, and finally restore the number of channels through the inverse process of dimensionality increase, and send the obtained feature map to the decoupling head structure for processing; The partial convolutional network is used to perform a convolution operation on some channels of the feature map, and concatenate the calculated convolution results with the remaining channels of the feature map.
