Adaptive sparse convolution guided cross-space uav target detection method and system

Abstract

The application belongs to the technical field of unmanned aerial vehicle image processing, and relates to a cross-space unmanned aerial vehicle target detection method and system guided by adaptive sparse convolution, which comprises the following steps: acquiring an unmanned aerial vehicle target detection data set, performing multi-scale feature extraction and multi-scale feature fusion on the unmanned aerial vehicle target detection data set to obtain multi-scale features, and processing the multi-scale features through an adaptive sparse detection head to obtain classification features and regression features; in the adaptive sparse detection head, an adaptive pixel-level mask unit can adaptively extract a target region in an image, and the detection of the target region is realized in combination with the principle of sparse convolution, so that the calculation cost of target detection is significantly reduced and the detection speed is improved; and the application designs a plug-and-play target detection head which can be applied to current mainstream target detectors, reduces the calculation cost of mainstream algorithms and improves the detection efficiency.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of unmanned aerial vehicle image processing, and relates to a cross-space unmanned aerial vehicle target detection method and system guided by adaptive sparse convolution. BACKGROUND

[0002] Unmanned aerial vehicle target detection, as one of the key fields of remote sensing image application, has been the focus of intelligent remote sensing information processing research for a long time. Its core task is to identify the location and category of targets in the image through feature extraction and analysis. The rapid development and wide application of target detection technology based on unmanned aerial vehicle images have brought great changes and development opportunities to various industries, playing an indispensable and important role in the country's social economy and national security, and have been highly valued by countries around the world.

[0003] Unmanned aerial vehicle images and natural images have significant differences. The background of natural images is single, the target area accounts for a large proportion, and the detection task is relatively simple. While the unmanned aerial vehicle remote sensing image has a long acquisition distance, a large image size, a complex and high proportion of background, a small and sparse target area, which significantly increases the difficulty of target detection. Although deep neural networks (such as CNN and Transformer) perform well on natural image datasets (such as COCO), the target detection of unmanned aerial vehicle images is still challenging. Existing researches mostly design complex models to improve the detection accuracy of small targets in high-resolution images, but the rich background and large background area lead to high demand for computing resources. At the same time, the hardware resources of unmanned aerial vehicles are limited, and lightweight models are needed to realize fast inference and low latency.

[0004] Domestic and foreign scholars have carried out in-depth research on this problem. Lu Yongxi et al. proposed an adaptive target detection method, which intelligently allocates computing resources through adjacent prediction and scaling prediction, focuses on small object areas, and realizes efficient detection. Yang Fan et al. designed an aggregated region extraction network, which accurately estimates the target scale using ScaleNet, fuses the detection results of cropped images and global images, and improves the performance. Deng Sutao et al. combined global rough detection and local fine detection, adaptively cropped dense regions, used local super-resolution networks to enhance details, and realized accurate target detection through global-local detection networks.

[0005] Although these methods have shown good results in unmanned aerial vehicle target detection, the complexity in designing network details, such as aggregated region extraction and super-resolution reconstruction, increases the detection time and limits its application potential in multiple scenarios. SUMMARY

[0006] The present application aims to provide a cross-space unmanned aerial vehicle target detection method and system guided by adaptive sparse convolution, so as to solve the problem of high computational resource consumption and slow detection speed of the target detection algorithm caused by high target sparsity in the complex background of the unmanned aerial vehicle image in the prior art.

[0007] To achieve the above object, the present application adopts the following technical solutions:

[0008] In a first aspect, the present application discloses a cross-space unmanned aerial vehicle target detection method guided by adaptive sparse convolution, comprising:

[0009] Obtaining an unmanned aerial vehicle target detection dataset, sequentially performing multi-scale feature extraction and multi-scale feature fusion on the unmanned aerial vehicle target detection dataset to obtain multi-scale features;

[0010] Inputting the multi-scale features into a preset adaptive sparse detection head for processing to obtain classification features and regression features;

[0011] Obtaining a target detection result based on the classification features and the regression features.

[0012] Preferably, the obtaining of the unmanned aerial vehicle target detection dataset and the sequentially performing of the multi-scale feature extraction and the multi-scale feature fusion on the unmanned aerial vehicle target detection dataset to obtain the multi-scale features specifically comprises:

[0013] S101: Obtaining an unmanned aerial vehicle remote sensing dataset;

[0014] S102: Randomly cropping the unmanned aerial vehicle remote sensing dataset to a uniform size to obtain a detection dataset;

[0015] S103: Dividing the detection dataset into a training set, a validation set and a test set and converting them into a coco data format;

[0016] S104: Performing multi-scale feature extraction on the detection dataset to obtain basic features;

[0017] S105: Performing multi-scale feature fusion on the basic features to obtain multi-scale features.

[0018] Preferably, the inputting of the multi-scale features into the preset adaptive sparse detection head for processing to obtain the classification features and the regression features specifically comprises:

[0019] S201: Obtaining global features by performing global feature generation processing on the multi-scale features, and obtaining mask features by performing adaptive pixel-level mask processing on the multi-scale features;

[0020] S202: Simultaneously passing the multi-scale features, the global features and the mask features through a plurality of context-enhanced sparse convolutions to obtain expressive sparse feature maps;

[0021] S203: The expressiveness-enhanced sparse feature map is processed by several sparse convolutions respectively to obtain classification features and regression features.

[0022] Preferably, the S201 specifically comprises:

[0023] S2011: The multi-scale features generate global features at each scale through point-wise convolution;

[0024] S2012: The multi-scale features are input into a mask unit for processing to obtain a soft feature mask; the soft feature mask is converted into a pixel-level hard feature mask through a binary Gumbel-Softmax method at each element of the soft feature mask, and the pixel-level hard feature mask is output as a mask feature.

[0025] Preferably, the S202 specifically comprises:

[0026] S2021: The multi-scale features and the mask feature are processed by sparse convolution to obtain sparse features;

[0027] S2022: The sparse features and the global features are processed by context-enhanced group normalization to obtain a sparse feature map;

[0028] S2023: The sparse feature map and the global features are added to obtain an expressiveness-enhanced sparse feature map.

[0029] Preferably, the adaptive sparse detection head comprises:

[0030] a global feature generation module configured to process the multi-scale features through point-wise convolution to obtain global features;

[0031] an adaptive pixel-level mask unit configured to mask process the multi-scale features to obtain a mask feature;

[0032] a context-enhanced sparse convolution module configured to perform feature enhancement based on the multi-scale features, the global features and the mask feature to obtain an expressiveness-enhanced sparse feature map;

[0033] a sparse convolution module configured to optimize the expressiveness-enhanced sparse feature map to obtain classification features and regression features;

[0034] a loss function module configured to perform loss calculation on the classification features and the regression features, and obtain an adaptive sparse detection head with optimal weights after iterative training.

[0035] Preferably, the loss calculation in the loss function module is performed by the following formula:

[0036]

[0037]

[0038] classification loss is:

[0039]

[0040]

[0041] regression loss is:

[0042]

[0043] normalization loss is:

[0044]

[0045] mask loss is:

[0046]

[0047]

[0048] wherein, and are hyperparameters for balancing and ; is a predicted value of the i-th class; is a true value of the i-th class; represents an intersection over union of the predicted value and the true value; is a feature map obtained by performing convolution on a complete input feature map; is a sparse feature map with enhanced expressiveness; represents a mask feature; represents a number of positive samples; represents a mask true value; represents a predicted value of a pixel-level mask unit. In a second aspect, the present application discloses a cross-space unmanned aerial vehicle target detection system based on adaptive sparse convolution guidance, comprising:

[0049] an acquisition unit, configured to acquire an unmanned aerial vehicle target detection dataset, and sequentially perform multi-scale feature extraction and multi-scale feature fusion on the unmanned aerial vehicle target detection dataset to obtain multi-scale features;

[0050] a data processing unit, configured to input the multi-scale features into a preset adaptive sparse detection head for processing to obtain classification features and regression features;

[0051]

[0052] ​​An output unit is configured to obtain a target detection result according to the classification feature and the regression feature.

[0053] In a third aspect, the present application discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the adaptive sparse convolution guided cross-space UAV target detection method.

[0054] In a fourth aspect, the present application discloses a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the adaptive sparse convolution guided cross-space UAV target detection method.

[0055] Compared with the prior art, the present application has the following beneficial effects:

[0056] (1) The present application designs an adaptive pixel-level mask unit, which can adaptively extract a target region in an image, and realizes detection of the target region in combination with a sparse convolution principle, thereby significantly reducing the calculation cost of target detection and improving the detection speed.

[0057] (2) The present application designs a context-enhanced grouping normalization strategy based on a residual structure, which effectively enhances the context information of the target region and reduces the loss of effective feature information, so as to reduce the calculation cost without reducing the accuracy of target detection.

[0058] (3) The present application designs a plug-and-play target detection head, which can be applied to the current mainstream target detector, thereby reducing the calculation cost of the mainstream algorithm and improving the detection efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0060] Figure 1 The method flowchart of the present application;

[0061] Figure 2 The network structure diagram of the adaptive sparse convolution guided cross-space UAV target detection method in the present application;

[0062] Figure 3 The general sparse convolution network structure diagram in the present application;

[0063] Figure 4 A structure diagram of a self-adaptive pixel-level mask unit network in the present application;

[0064] Figure 5 A structure diagram of a context-enhanced sparse convolution network in the present application;

[0065] Figure 6 A structure diagram of a self-adaptive sparse convolution detection head network in the present application;

[0066] Figure 7 A detection result and a corresponding pixel mask visualization result after the present application is applied to a general algorithm. DETAILED DESCRIPTION

[0067] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0068] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0069] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0070] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the present application is used, which is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second" and the like are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0071] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly inclined. For example, "horizontal" only means that its direction is relatively more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly inclined.

[0072] In the description of the embodiments of the application, it should also be noted that, unless otherwise explicitly specified and limited, if the terms "arrange", "mount", "connect", "connect" appear, they should be understood in a broad sense, for example, they can be fixedly connected, or can be detachably connected, or integrally connected, can be mechanically connected, or can be electrically connected, can be directly connected, or indirectly connected through an intermediate medium, or the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0073] The application will be described in further detail below with reference to the drawings:

[0074] Reference Figure 1 An adaptive sparse convolution guided cross-space unmanned aerial vehicle target detection method, comprising:

[0075] S1: obtaining an unmanned aerial vehicle target detection data set, sequentially performing multi-scale feature extraction and multi-scale feature fusion on the unmanned aerial vehicle target detection data set, and obtaining multi-scale features;

[0076] S2: inputting the multi-scale features into a preset adaptive sparse detection head for processing, to obtain classification features and regression features; the preset plug-and-play adaptive sparse detection head is suitable for various target detection algorithms and significantly reduces the calculation cost.

[0077] S3: obtaining a target detection result based on the classification features and the regression features.

[0078] The application first obtains an unmanned aerial vehicle target detection data set and performs preprocessing. Then, an adaptive sparse detection head is designed. Then, the adaptive sparse detection head is transplanted into mainstream one-stage and two-stage target detection models. Then, the improved model is trained and verified, and the improved method is evaluated. Finally, the unmanned aerial vehicle image is input into the trained model to obtain the detection result. The application solves the problems of high target sparsity and slow detection speed of target detection algorithms caused by complex background of unmanned aerial vehicle images. The adaptive sparse detection head designed by the application can realize plug-and-play and is suitable for various target detection algorithms, and significantly reduces the calculation cost.

[0079] In some embodiments, the unmanned aerial vehicle target detection data set is obtained, and multi-scale feature extraction and multi-scale feature fusion are sequentially performed on the unmanned aerial vehicle target detection data set to obtain multi-scale features, specifically comprising:

[0080] S101: obtaining an unmanned aerial vehicle remote sensing data set;

[0081] S102: randomly cropping the unmanned aerial vehicle remote sensing data set to a uniform size to obtain a detection data set;

[0082] S103: Divide the detection dataset into a training set, a validation set, and a test set and convert it into a coco data format;

[0083] S104: Perform multi-scale feature extraction on the detection dataset to obtain basic features;

[0084] S105: Perform multi-scale feature fusion on the basic features to obtain multi-scale features.

[0085] In some embodiments, the multi-scale features are input into a preset adaptive sparse detection head for processing to obtain classification features and regression features, specifically including:

[0086] S201: The multi-scale features are processed by global feature generation to obtain global features, and by adaptive pixel-level mask processing to obtain mask features;

[0087] S202: The multi-scale features, global features, and mask features are simultaneously processed by a plurality of context-enhanced sparse convolutions to obtain expressive sparse feature maps;

[0088] S203: The expressive sparse feature maps are processed by a plurality of sparse convolutions respectively to obtain classification features and regression features.

[0089] In some embodiments, the multi-scale features are processed by global feature generation and adaptive pixel-level mask processing to obtain global features and mask features respectively, specifically including:

[0090] S2011: The multi-scale features are processed by point-by-point convolution to generate global features at each scale;

[0091] S2012: The multi-scale features are processed by adaptive pixel-level mask processing to generate mask features at each scale; that is, the multi-scale features are input into a mask unit for processing to obtain a soft feature mask; a binary Gumbel-Softmax method is used on each element of the soft feature mask to convert the soft feature mask into a pixel-level hard feature mask, which is output as the mask features.

[0092] In some embodiments, the multi-scale features, global features, and mask features are processed by context-enhanced sparse convolutions and loss function calculation to obtain expressive sparse feature maps, specifically including:

[0093] S2021: The multi-scale features and mask features are processed by sparse convolution to obtain sparse features;

[0094] S2022: The sparse features and global features are processed by context-enhanced grouped normalization to obtain sparse feature maps;

[0095] S2023: The sparse feature maps and global features are added to obtain expressive sparse feature maps.

[0096] In some embodiments, the adaptive sparse detection head comprises:

[0097] a global feature generation module for point-by-point convolution processing of multi-scale features to obtain global features;

[0098] an adaptive pixel-level mask unit for mask processing of the multi-scale features to obtain mask features;

[0099] a context-enhanced sparse convolution module for feature enhancement based on the multi-scale features, the global features and the mask features to obtain expressive enhanced sparse feature maps;

[0100] a sparse convolution module for optimizing the expressive enhanced sparse feature maps to obtain classification features and regression features;

[0101] a loss function module for loss calculation on the classification features and the regression features, and after iterative training, obtaining an adaptive sparse detection head with optimal weights.

[0102] The application designs a plug-and-play target detection head, which is suitable for various target detection algorithms and significantly reduces the computational cost. An adaptive pixel-level mask unit is designed through a task alignment strategy to generate a target pixel-level mask, and a sparse convolution based on a global context information enhancement strategy is used to realize cross-space detection of the target, focusing only on target area detection while excluding the interference of complex backgrounds, thereby ensuring detection accuracy, reducing computational complexity and improving detection efficiency.

[0103] In some embodiments, an adaptive sparse convolution guided cross-space unmanned aerial vehicle target detection method comprises the following steps:

[0104] Step 1: Obtain an unmanned aerial vehicle target detection dataset and perform preprocessing.

[0105] A common unmanned aerial vehicle remote sensing dataset, VisDrone dataset, is used; preprocessing includes random cropping to a uniform size, dividing the dataset into a training set, a validation set and a test set, and converting the dataset into a coco data format.

[0106] Step 2: Adaptive sparse detection head design, which includes: ordinary sparse convolution, global feature generation, adaptive pixel-level mask unit, context-enhanced sparse convolution module, and loss function design.

[0107] The adaptive sparse detection head design process includes the following sub-steps:

[0108] Step 2.1: For the input drone image, multi-scale feature extraction is performed by the backbone network in the basic algorithm, here ResNet18 is adopted, and is denoted as {C3, C4, C5}. Then multi-scale feature fusion is performed in the feature pyramid network to obtain multi-scale features {P3, P4, P5, P6, P7}. Further input into the adaptive sparse detection head designed in the application, sparse detection is performed.

[0109] Step 2.2: The general sparse convolution adopted is submanifold sparse convolution. Only when the position covered by the center of the sparse convolution kernel is in the active state, the convolution output will be calculated. The position of the mask value of 1 corresponding to the feature map is the active state.

[0110] Step 2.3: The global feature generation adopts point-by-point convolution to generate global features .

[0111] Step 2.4: The adaptive pixel-level mask unit defines the spatial position processed by convolution, and a small mask unit acts on multi-scale features , and outputs the corresponding soft feature mask . The mask unit adopts 1x1 convolution. The Gumbel-Softmax module is adopted to convert the soft feature mask into a pixel-level hard feature mask by applying the binary Gumbel-Softmax trick to each element of :

[0112]

[0113] The hard feature mask is the mask feature which will serve as the execution mask to indicate the operation in the active spatial position.

[0114] Step 2.5: The context-enhanced sparse convolution module takes the input feature , the mask feature , and the global feature as inputs, and performs context-enhanced sparse convolution operation on the input feature.

[0115] Step 2.6: The loss function design includes classification loss , regression loss , normalization loss , and mask loss , and the total loss function is as follows:

[0116]

[0117] wherein and are hyperparameters used to balance and .

[0118] Step 2.7: The designed adaptive sparse convolution detection head, for input features , after processing by the mask unit, generates mask features , while after point-by-point convolution processing, global features are generated. After 4 context-enhanced sparse convolution processing, two ordinary sparse convolutions are used to process respectively to obtain classification features and regression features.

[0119] In some embodiments, step 1: transplanting the preset adaptive sparse detection head into mainstream one-stage and two-stage target detection models. The mainstream one-stage and two-stage target detection models obtain the unmanned aerial vehicle target detection dataset, perform multi-scale feature extraction and multi-scale feature fusion on the unmanned aerial vehicle target detection dataset to obtain multi-scale features; the one-stage and two-stage target detection models include Faster R-CNN, RetinaNet, FSAF, GFL_V1, and TOOD. These five models not only include one-stage and two-stage target detection models, but also include anchor-based and anchor-free target detection models, and are representative. Transplanting the sparse detection head into these five models is named as Faster R-CNN-SC, RetinaNet-SC, FSAF-SC, GFL_V1-SC, and TOOD-SC.

[0120] Step 2: training and verifying the improved model, and evaluating the improved method.

[0121] The base network models Faster R-CNN, RetinaNet, FSAF, GFL V1, TOOD and the network models improved based on the sparse detection head, Faster R-CNN-SC, RetinaNet-SC, FSAF-SC, GFL_V1-SC, and TOOD-SC are respectively trained on the VisDrone dataset. The trained models are evaluated. The average precision AP[50:95] is used as the precision evaluation index, and the floating point operation number GFLOPs and the frame rate FPS are used as the two indexes to evaluate the calculation efficiency of the model.

[0122] Step 3: inputting the unmanned aerial vehicle image into the trained model to obtain the detection result.

[0123]

EMBODIMENT

[0124] The specific steps of the embodiment of the present application are as follows:

[0125] Step 1: Obtain the unmanned aerial vehicle target detection data set and perform preprocessing. Among them, the commonly used unmanned aerial vehicle remote sensing data set VisDrone data set is adopted, the data set used is reasonably divided into a training set, a verification set and a test set, which respectively contain 6,471 images for model training, 548 images for parameter adjustment and verification of the model, and 3,190 images for final performance evaluation and test analysis. The preprocessing includes random cropping to a uniform size, an input size of 1333x800, and conversion of the data set format to a coco data format.

[0126] Step 2: Adaptive sparse detection head design, including: ordinary sparse convolution, global feature generation, adaptive pixel-level mask unit, context-enhanced sparse convolution module, loss function design. Specifically, the following steps are included:

[0127] Step 2.1: Figure 2 The network structure diagram of the adaptive sparse convolution guided cross-space unmanned aerial vehicle target detection method. For the input unmanned aerial vehicle image, the multi-scale feature extraction is performed through the backbone network in the basic algorithm, here ResNet18 is adopted, and is denoted as {C3, C4, C5}. Then multi-scale feature fusion is performed in the feature pyramid network to obtain multi-scale features {P3, P4, P5, P6, P7}. Further input into the adaptive sparse detection head designed in the present application for sparse detection.

[0128] Step 2.2: The ordinary sparse convolution adopted is submanifold sparse convolution, as shown in Figure 3 , only when the position covered by the center of the sparse convolution kernel is in the active state, the convolution output will be calculated, and the position of the mask corresponding to the feature map with a value of 1 is in the active state.

[0129] Step 2.3: Global feature generation, based on multi-scale features {P3, P4, P5, P6, P7}, a global feature is generated at each scale through point-by-point convolution. Specifically, the convolution kernel of the point-by-point convolution generates the global feature .

[0130] Step 2.4: Adaptive pixel-level mask unit, the generated multi-scale features {P3, P4, P5, P6, P7} are input into the adaptive pixel-level mask unit designed in the present application, and a mask feature is generated at each scale. Please refer to Figure 4 . The specific principle is as follows:

[0131] Given the feature maps from the first i layer of the feature pyramid network , feature processing is performed by introducing a mask network. The mask network is composed of shared convolution kernels , where B , C , H , W represent batch size, channel number, height and width, respectively, and k is the kernel size. The invention adopts point-wise convolution, i.e. k = 1, to reduce computational complexity. Convolution is performed on to generate a soft feature , which is further converted into a hard feature mask matrix by using the Gumbel-Softmax trick, which is expressed as follows:

[0132]

[0133] where and are two random Gumbel noises, is a Sigmoid function, is the corresponding temperature parameter in Gumbel-Softmax.

[0134] Step 2.5: Context-enhanced sparse convolution module, which enhances features by combining focal information and global context information, thereby improving detection accuracy and enhancing the stability of subsequent calculations. Its network structure diagram is referred to in Figure 5 , and its principle is as follows:

[0135] As a key component in the context-enhanced sparse convolution module, global context information is embedded into the Sparse Convolution-GN-ReLU layer. Since sparse convolution only operates on part of the elements, gradually stabilizes after multiple rounds of sparse convolution processing, and this process hardly increases additional computational cost, ensuring the network's high efficiency and precision. The input of the context-enhanced sparse convolution module includes feature maps , sparse mask and global features , where j represents the j context-enhanced sparse convolution module. Unlike traditional sparse convolution, the context-enhanced sparse convolution module does not use activation elements to calculate the statistics of group normalization, but adopts​ The mean and standard deviation of the are normalized, aiming to make up for the missing context. It is assumed that is the output feature map after applying sparse convolution, then the feature is obtained through context-enhanced group normalization, which is as follows:

[0136]

[0137] where and represent the mean and standard deviation, respectively, and and are learnable parameters.

[0138] Finally, before the activation layer, in order to better preserve the context information, a residual structure is adopted to directly add the global context feature to the sparse feature map, that is, the operation of is realized.

[0139] Step 2.6: Loss function design. It includes classification loss , regression loss , normalization loss , and mask loss , specifically:

[0140] is used to measure the error between the predicted class of the bounding box and the true value class, and the predicted value of the class is , the corresponding true value is , is calculated as follows:

[0141]

[0142] is used to measure the error between the position of the predicted bounding box and the position of the true bounding box, the IoU loss is used to calculate:

[0143]

[0144] In order to effectively reduce the information loss that may occur during the sparse convolution process and ensure the stability of the training process, the invention introduces a regular dense convolution during training. Specifically, in addition to sparse convolution, a feature map is generated by performing convolution on the complete input feature map. This additional step not only helps to capture more detailed information, but also provides a reliable foundation for subsequent feature optimization. Subsequently, by optimizing the mean square error loss function, the feature map is aligned with the sparse feature map ​Combine, thereby enhancing the expressiveness of sparse feature map, as follows:

[0145]

[0146] wherein, L The number of outputs of the feature pyramid is represented.

[0147] Mask loss of the application Adopting binary cross-entropy loss function BCEWithLogitsLoss calculation, soft feature mask generated by adaptive pixel mask module Supervision, the corresponding mask truth is represented as Then, Can be expressed as:

[0148]

[0149] The total loss function is as follows:

[0150]

[0151] Wherein And Is a hyperparameter used to balance And In the application, And Set to 1 and 10.

[0152] Step 2.7: design of adaptive sparse convolution detection head, its network structure diagram as Figure 6 Shown, for multi-scale features as input features , after processing by mask unit to generate mask features , at the same time after point-by-point convolution processing to generate global features Further, , , As input, after 4 context-enhanced sparse convolution processing, respectively using two ordinary sparse convolution for processing, get classification features and regression features.

[0153] Step 3: transplant the adaptive sparse detection head into mainstream one-stage and two-stage target detection models. Specifically, the one-stage and two-stage target detection models used include Faster R-CNN, RetinaNet, FSAF, GFL_V1, and TOOD. These five models not only include one-stage and two-stage target detection models, but also include anchor-based and anchor-free target detection models, which are representative. Transplant the sparse detection head into these five models, named FasterR-CNN-SC, RetinaNet-SC, FSAF-SC, GFL_V1-SC, and TOOD-SC.

[0154] Step 4: train and validate the improved model and evaluate the improved method. Specifically, train the base network models Faster R-CNN, RetinaNet, FSAF, GFL V1, TOOD, and the network models improved based on the sparse detection head, Faster R-CNN-SC, RetinaNet-SC, FSAF-SC, GFL_V1-SC, and TOOD-SC, on the VisDrone dataset. Evaluate the trained model. Use the average precision AP[50:95] as the precision evaluation index, and use the floating-point operation number GFLOPs and the frame rate FPS as two indicators to evaluate the computational efficiency of the model. The quantitative evaluation index is shown in Table 1: The results show that after applying the method of the present application to the classical target detection model, the detection results on the key indicators of AP[50:95], FPS, GFLOPs, etc. all show obvious improvement, fully verifying the effectiveness and superiority of the method of the present application in the task of unmanned aerial vehicle remote sensing image target detection, and proving its significant advantages in sparse detection, computational efficiency, and detection accuracy.

[0155] Table 1. Quantitative experimental results of the base model and the improved model based on the method of the present application on the VisDrone dataset.

[0156]

[0157] Step 5: input the unmanned aerial vehicle image into the trained model to obtain the detection result. Specifically, use the trained model to perform adaptive sparse convolution guided cross-space detection on the unmanned aerial vehicle image for efficient detection. The detection result is as follows Figure 7As shown, through analysis of the detection results and pixel mask visualization results under different complex scenes, it can be seen that the four target detection methods after applying the method of the application can accurately detect the targets in the unmanned aerial vehicle remote sensing image, can adaptively generate a target region mask, and only identify the target region, thereby greatly reducing the model calculation amount. The detection results and pixel mask visualization results fully confirm the advantages of the method of the application in reducing the model calculation amount. Through the cross-space sparse detection strategy, the method of the application only focuses on target region detection while excluding the interference of complex backgrounds, thereby ensuring detection accuracy, reducing calculation amount, and improving detection efficiency.

[0158] The application also discloses a cross-space unmanned aerial vehicle target detection system guided by adaptive sparse convolution, comprising:

[0159] An acquisition unit is configured to acquire an unmanned aerial vehicle target detection dataset, sequentially perform multi-scale feature extraction and multi-scale feature fusion on the unmanned aerial vehicle target detection dataset, and obtain multi-scale features.

[0160] A data processing unit is configured to input the multi-scale features into a preset adaptive sparse detection head for processing, and obtain classification features and regression features.

[0161] An output unit is configured to obtain target detection results according to the classification features and the regression features.

[0162] The application discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the cross-space unmanned aerial vehicle target detection method guided by adaptive sparse convolution according to any one of the above.

[0163] The application discloses a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the cross-space unmanned aerial vehicle target detection method guided by adaptive sparse convolution according to any one of the above.

[0164] To sum up, the unmanned aerial vehicle target detection data set is firstly acquired and preprocessed in the application. Then, the adaptive sparse detection head is designed. Then, the adaptive sparse detection head is transplanted into the mainstream one-stage and two-stage target detection model. Then, the improved model is trained and verified, and the improved method is evaluated. Finally, the unmanned aerial vehicle image is input into the trained model to obtain the detection result. The application solves the problems of high target sparsity and slow detection speed of the target detection algorithm caused by the complex background of the unmanned aerial vehicle image. The adaptive sparse detection head designed in the application can be plug and play, suitable for various target detection algorithms, and significantly reduce the computing cost. The adaptive pixel-level mask unit is designed through the task alignment strategy, which can generate the target pixel-level mask. The sparse convolution based on the global context information enhancement strategy is used to realize the cross-space detection of the target, which only focuses on the target area detection while excluding the interference of the complex background, thereby ensuring the detection accuracy, reducing the calculation amount and improving the detection efficiency

[0165] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Accordingly, the application can be embodied in the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware aspects. Moreover, the application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) having computer usable program code embodied thereon.

[0166] The application is described with reference to flowcharts and / or block diagrams that illustrate the method, apparatus (system) and computer program product according to embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more functions specified in the flowchart or flows and / or blocks. Figure 1 an apparatus that implements one or more functions specified in the flowchart or flows and / or blocks.

[0167] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more functions specified in the flowchart or flows and / or blocks. Figure 1 an apparatus that implements one or more functions specified in the flowchart or flows and / or blocks.

[0168] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 one or more flowcharts and / or blocks

[0169] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the technical solutions of the present application. Although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and any modification or equivalent replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A method for cross-space UAV target detection using adaptive sparse convolution guidance, characterized in that, The method comprises the following steps: obtaining a UAV target detection data set, sequentially performing multi-scale feature extraction and multi-scale feature fusion on the UAV target detection data set, and obtaining multi-scale features; inputting the multi-scale features into a preset adaptive sparse detection head for processing, and obtaining classification features and regression features; obtaining a target detection result based on the classification features and the regression features; the adaptive sparse detection head comprises: a global feature generation module for point-by-point convolution processing of the multi-scale features to obtain global features; an adaptive pixel-level mask unit for mask processing of the multi-scale features to obtain mask features; a context-enhanced sparse convolution module for feature enhancement based on the multi-scale features, the global features and the mask features to obtain expressive enhanced sparse feature maps; a sparse convolution module for optimizing the expressive enhanced sparse feature maps to obtain the classification features and the regression features; a loss function module for loss calculation on the classification features and the regression features, and obtaining the adaptive sparse detection head with the best weight after iterative training. 2.The method of claim 1, wherein, The method comprises the following steps: S101: obtaining a UAV remote sensing data set; S102: randomly cropping the UAV remote sensing data set to a uniform size to obtain a detection data set; S103: dividing the detection data set into a training set, a validation set and a test set and converting them into a coco data format; S104: performing multi-scale feature extraction on the detection data set to obtain basic features; S105: performing multi-scale feature fusion on the basic features to obtain multi-scale features. 3.The method of claim 1, wherein, The method comprises the following steps: S201: obtaining global features by processing the multi-scale features through a global feature generation process, and obtaining mask features by processing the multi-scale features through an adaptive pixel-level mask process; S202: simultaneously passing the multi-scale features, the global features and the mask features through a plurality of context-enhanced sparse convolutions to obtain expressive enhanced sparse feature maps; S203: processing the expressive enhanced sparse feature maps through a plurality of sparse convolutions to obtain classification features and regression features.

4. The method of claim 3, wherein, The S201 specifically comprises: S2011: generating global features at each scale by point-by-point convolution of the multi-scale features; S2012: inputting the multi-scale features into a mask unit for processing to obtain a soft feature mask; and converting the soft feature mask into a pixel-level hard feature mask as mask features by using a binary Gumbel-Softmax method on each element of the soft feature mask.

5. The method of claim 3, wherein, The S202 specifically comprises: S2021: obtaining sparse features by processing the multi-scale features and the mask features through sparse convolution; S2022: obtaining a sparse feature map by processing the sparse features and the global features through context-enhanced grouped normalization; S2023: adding the sparse feature map and the global features to obtain expressive enhanced sparse feature maps.

6. The method of claim 1, wherein, The loss calculation in the loss function module is performed by the following formula: Classification loss is: Regression loss is: Normalized loss is: Mask loss Is: wherein, and are hyperparameters for balancing and ; is a predicted value of the first category; is a true value of the first category; denotes a intersection over union of the predicted value and the true value; is a feature map that is convolved on a full input feature map; is a sparse feature map with expressiveness enhancement; denotes a mask feature; denotes a number of positive samples; denotes a mask true value; denotes a predicted value of a pixel-level mask unit.

7. An adaptive sparse convolution guided cross-space UAV target detection system, characterized in that, The method comprises the following steps: An acquisition unit is configured to acquire a UAV target detection dataset, sequentially perform multi-scale feature extraction and multi-scale feature fusion on the UAV target detection dataset, and obtain multi-scale features. A data processing unit is configured to input the multi-scale features into a preset adaptive sparse detection head for processing, and obtain classification features and regression features. An output unit is configured to obtain a target detection result based on the classification features and the regression features. 8.An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the adaptive sparse convolution guided cross-space UAV target detection method according to any one of claims 1-6 when executing the computer program. 9.A computer readable storage medium, storing a computer program, wherein the computer program is executable on a processor to implement the steps of the adaptive sparse convolution guided cross-space UAV target detection method according to any one of claims 1-6.

Images