A scene-adaptive unmanned aerial vehicle target detection method and system

Abstract

The application discloses a scene-adaptive unmanned aerial vehicle target detection method and system, which comprises the following steps: inputting an image into a scene classification network to obtain scene features; inputting the image and the scene features into a multi-scene target detection network to obtain target recognition and positioning results; specifically, inputting the image into an image feature extraction module to obtain local features, inputting the local features and preset reference points into an offset attention module to obtain offset attention features, and inputting the offset attention features into a target query positioning module; inputting the scene features and a preset target query vector into a scene feature guiding module to obtain a scene target query vector, inputting the scene target query vector into the target query positioning module; and the target query positioning module performs target detection based on the offset attention features and the scene target query vector. The application realizes unified unmanned aerial vehicle target detection tasks in different scenes by using one model, and saves computing resources by using a lightweight detection algorithm, which is convenient for unmanned aerial vehicle deployment.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, and in particular relates to a scene-adaptive unmanned aerial vehicle (UAV) target detection method and system. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] In the field of computer vision, object detection is a crucial foundational task. It combines the concepts of object recognition and object localization, aiming to identify and locate typical objects in an image. It is commonly deployed in drones, enabling them to adaptively perceive their surroundings. Currently, drones are used in numerous scenarios, each with its own unique environmental characteristics and typical targets. Therefore, intelligently and efficiently extracting targeted features for each scenario is key to ensuring the robust application of drone object detection algorithms across various environments.

[0004] Current solutions for UAV object detection in multi-scene tasks mostly fall into two categories. The first is to train a separate object detection model for each scene, then apply different model parameters based on the scene type. While this allows for targeted feature extraction based on specific scenes, it fragments the original dataset during training, hindering model robustness. Furthermore, it requires storing a large number of model parameters during testing, which is detrimental to practical deployment. The second approach doesn't differentiate between scenes, using the same feature extraction mode to extract image features from different scenes. This method doesn't consider the scene's imaging characteristics or object distribution patterns during object detection, further hindering model robustness. Both methods struggle with cross-scene UAV object detection. Therefore, for multi-scene object detection tasks for UAVs, existing solutions cannot unify object detection across different scenes with a single model. Training data from multiple scenes cannot be fully utilized, hindering model generalization performance.

[0005] Furthermore, for UAV platforms with limited computing resources, while traditional anchor-based object detection algorithms offer high accuracy, they often generate numerous duplicate bounding boxes. At the detector's end, non-maximum suppression (NMS) is needed to filter out high-confidence bounding boxes, a computationally intensive process that is unsuitable for deployment in UAV mission scenarios. A more effective solution is the Transformer-based object detection algorithm (Detection Transformer, DETR). By utilizing the multi-head attention mechanism in the Transformer to model the correlation between different targets, it can directly predict target bounding boxes without pre-setting anchor points, completing end-to-end object detection and reducing computational cost in practical applications. However, traditional Transformer algorithms still involve intensive attention operations, resulting in high computational costs and slow inference processes, hindering practical deployment for UAVs. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, this invention provides a scene-adaptive UAV target detection method and system, which realizes the use of a single model to unify UAV target detection tasks in different scenarios, and lightweights the detection algorithm, saving computing resources and facilitating UAV deployment.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, the present invention provides a scene-adaptive unmanned aerial vehicle (UAV) target detection method, comprising:

[0009] Acquire images of the current drone application scenario;

[0010] The image is input into a pre-trained scene classification network for feature extraction to obtain scene features;

[0011] The image and scene features are input into a pre-trained multi-scene target detection network for target detection to obtain the target recognition and localization result in the current scene. The multi-scene target detection network includes an image feature extraction module, a shift attention module, a scene feature guidance module, and a target query localization module. Specifically, the image is input into the image feature extraction module for feature extraction to obtain local features, and the local features and preset reference points are input into the shift attention module for shift attention operation to obtain shift attention features. The shift attention features are then input into the target query localization module. The scene features and preset target query vectors are input into the scene feature guidance module to obtain scene target query vectors, and the scene target query vectors are then input into the target query localization module. The target query localization module receives the shift attention features and the scene target query vectors and performs target detection.

[0012] In a further technical solution, the scene classification network includes a lightweight scene understanding module and a global information association module connected in sequence. The lightweight scene understanding module receives an image and extracts local scene features from the image, and inputs the local scene features into the global information association module. The global information association module uses scene information association rules to perform spatial similarity association on the local scene features to obtain global scene features.

[0013] A further technical solution is that the lightweight scene understanding module includes spatial convolution and channel convolution connected in sequence. The input image first undergoes spatial convolution to extract the spatial features of the image in a channel-by-channel manner. Then, the spatial features are input into channel convolution to extract the channel features of the spatial features in a spatial position-by-spatial manner, thereby obtaining local scene features of modeling spatial information and channel information.

[0014] A further technical solution yields the following specific offset attention features:

[0015] The local features, the query vector, and the coordinates of the reference point corresponding to the query vector are input into the offset attention module.

[0016] A feature mapping operation is performed on the query vector using a linear mapping to generate the offset of each sampling point and the corresponding attention weight;

[0017] Based on the reference point and its offset, the sampling points corresponding to the local features are obtained, and feature sampling is performed on the sampling points to obtain the sampling features;

[0018] A feature mapping operation is performed on the local features using linear operations. Then, the mapped features are multiplied by the attention weights, and the sampled features are weighted and summed based on offset attention to obtain the offset attention features.

[0019] A further technical solution to obtain the scene target query vector is as follows: the spatial dimension of the scene features is compressed using average pooling, and then the features are fused with the preset target query vector to obtain the scene target query vector.

[0020] In a further technical solution, the scene target query vector in the target query localization module is first modeled by the self-attention module to model the correlation between scene target query vectors, and then inputted into the cross-attention module along with the offset attention features, using the scene target query vector to query typical targets in the offset attention features.

[0021] In a further technical solution, the self-attention module is a target-centered self-attention module, wherein the scene target query vector is independently linearly mapped to the latent feature space, and then the self-attention mechanism is used to construct the similarity association between vectors.

[0022] Secondly, the present invention provides a scene-adaptive unmanned aerial vehicle (UAV) target detection system, comprising:

[0023] The image acquisition module is configured to acquire images of the current drone application scenario.

[0024] The scene classification module is configured to: input the image into a pre-trained scene classification network for feature extraction to obtain scene features;

[0025] The target detection module is configured to: input the image and the scene features into a pre-trained multi-scene target detection network for target detection, and obtain the target recognition and localization result in the current scene; the multi-scene target detection network includes an image feature extraction module, a shift attention module, a scene feature guidance module, and a target query localization module; wherein, the image is input into the image feature extraction module for feature extraction to obtain local features, and the local features and preset reference points are input into the shift attention module for shift attention operation to obtain shift attention features, and the shift attention features are input into the target query localization module; the scene features and preset target query vectors are input into the scene feature guidance module to obtain scene target query vectors, and the scene target query vectors are input into the target query localization module; the target query localization module receives the shift attention features and the scene target query vectors and performs target detection.

[0026] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a scene-adaptive unmanned aerial vehicle target detection method as described in the first aspect.

[0027] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a scene-adaptive unmanned aerial vehicle target detection method as described in the first aspect.

[0028] The above one or more technical solutions have the following beneficial effects:

[0029] This invention extracts local and global features of a scene step-by-step through a scene classification network, achieving efficient representation of scene features. Furthermore, in a multi-scene object detection network, the algorithm combines the extracted scene features to specifically perform object recognition and localization operations within the current scene, thus enabling robust application of the object detection algorithm across different scenes. A lightweight offset attention module is also proposed, which effectively aggregates information within a local range by establishing similarity associations between image features and features near a reference point, reducing the overall computational cost of the algorithm and promoting its practical deployment.

[0030] This invention introduces a Scene Classification Network (SCN): by extracting local scene features and associating global scene information in a lightweight manner, it can effectively extract scene features from the images captured by the UAV, making it easier for subsequent target detection algorithms to perform targeted target recognition and localization operations based on scene features.

[0031] This invention proposes a Multi-Scene Object Detection Network (MODN): It introduces a Shiftable Attention Module (SAM) to improve traditional intensive attention operations into sparse attention operations, reducing the overall computational cost of the network. Simultaneously, it proposes a Scene Feature Guidance Module (SFGM), which integrates scene features into the target query vector, enabling targeted feature extraction and target recognition operations, thus improving the model's generalization ability in multi-scene applications. Attached Figure Description

[0032] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0033] Figure 1 This is a flowchart of the target detection method according to an embodiment of the present invention. Detailed Implementation

[0034] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0035] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0036] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0037] In multi-scene UAV object detection, the training and testing processes involve images from multiple scenes, each with different scene characteristics and typical target distributions. Unlike traditional single-scene object detection tasks, the key challenge of cross-scene object detection is how to accurately understand scene information and perform targeted feature extraction and target recognition based on scene features, thereby achieving robust application of the algorithm in multi-scene object detection tasks.

[0038] Example 1

[0039] like Figure 1 As shown in the figure, this embodiment discloses a scene-adaptive UAV target detection method, which includes the following steps:

[0040] S1: Acquire images of the current drone application scenario;

[0041] In this embodiment, corresponding images are acquired and target detection is performed under various drone application scenarios. After the acquired images are labeled, a dataset is constructed for training the scene classification network and the multi-scene target detection network.

[0042] The training dataset is constructed as follows: First, image data from multiple scenes are collected using drones, such as urban, suburban, airport, and marine scenes. Then, each image is assigned two types of labels. The first type is the scene type label, with different scene type numbers assigned to different images; for example, urban scenes are labeled as type 1, suburban scenes as type 2, airport scenes as type 3, and marine scenes as type 4. The second type is the target recognition and localization label, where existing annotation software (such as labelimg) is used to annotate the bounding boxes and categories of typical targets in each image. For example, pedestrians and vehicles in urban scenes, vehicles and buildings in suburban scenes, aircraft in airport scenes, and ships in marine scenes are annotated. Each training dataset consists of three parts: input image data, scene type labels, and bounding boxes and category labels for typical targets.

[0043] S2: Input the image into a pre-trained scene classification network for feature extraction to obtain scene features;

[0044] In this embodiment, the application scenarios of UAVs may involve multiple scenarios (such as urban scenarios, suburban scenarios, and marine scenarios). Different scenarios have different imaging characteristics and task requirements. Therefore, perception algorithms are needed to extract differentiated features and locate typical targets for different scenario types. This requires an algorithm module with multi-scenario classification capabilities to connect this process. First, it efficiently performs adaptive extraction of scene features. Then, it combines the extracted scene features with a target detection algorithm to complete the typical target recognition operation in a specific scenario. Based on this, the present invention proposes a new scene classification network.

[0045] The scene classification network consists of a Lightweight Scene Understanding (LSU) module and a Global Information Association (GIA) module connected in sequence. It extracts local scene features in a lightweight manner and performs global information association to obtain global scene features, which can effectively represent the scene information of the UAV in the current environment and achieve efficient feature extraction.

[0046] S201: The lightweight scene understanding module receives an image and extracts local scene features from the image, then inputs these local scene features into the global information association module. Specifically, the lightweight scene understanding module includes spatial convolution and channel convolution connected in sequence. The input image first undergoes spatial convolution to extract spatial features of the image in a channel-by-channel manner. Subsequently, the spatial features are input into channel convolution to extract channel features of the spatial features in a spatial position-by-spatial manner, thus obtaining local scene features that model spatial information and channel information.

[0047] In practical drone applications, the timeliness of the algorithm directly affects the model's actual application capability. For classification tasks, the computational load of the feature extraction part accounts for the main part of the model. In order to reduce the overall computational load of the algorithm, this invention decouples the traditional standard convolution operation and designs spatial convolution (SC) and channel convolution to realize feature extraction of drone scene images.

[0048] The input image (input feature map) is first subjected to spatial convolution, an operation that performs convolution only in the spatial dimension to extract spatial features of the image channel by channel. Assume the input image F... in The dimensions are H×W×C, and each input channel c corresponds to one convolutional kernel D. s If the kernel size is a×a, then the formula for calculating spatial convolution can be expressed as:

[0049]

[0050] Among them, F s To model the local scene features (i.e., spatial features) of spatial information, h, w, and c represent the height, width, and number of channels of the input image, respectively, and m and n are the indices of the spatial locations, F in For the input image, D s It is the convolution kernel.

[0051] Then the spatial feature F s In the input channel convolution, this operation only performs convolution operations along the channel dimension to extract spatial features F in a position-by-position manner. s The channel features. Feature vectors at all spatial locations correspond to the same convolution kernel D. c Assuming the number of output channels is o and the kernel size is 1×1×c×o, the formula for calculating channel convolution can be expressed as:

[0052]

[0053] Among them, F c To model the local scene features of spatial and channel information, k is the index of the channel position.

[0054] This invention replaces the traditional convolution operation with a cascaded combination of spatial convolution and channel convolution. By sequentially extracting the spatial and channel features of the input image, scene features with modeled spatial and channel information are obtained. This enables a lightweight understanding of scene features and allows for the efficient and rapid extraction of scene features from the current image, thereby better meeting the needs of UAVs for real-time data processing and high-precision decision-making in complex environments.

[0055] S202: The global information association module receives local scene features and uses scene information association rules to perform spatial similarity association on the local scene features to obtain global scene features.

[0056] When understanding scene information, it is necessary to fully consider the global information distribution of the entire image. A major advantage of convolutional operations is local perception, that is, feature extraction is performed only on local regions, thereby reducing computational cost. However, this localized feature extraction method is not conducive to building long-range dependencies between features, making it difficult to achieve global information perception. To address this, this invention designs a global information association module. By integrating local scene features obtained from a lightweight scene understanding module, this module can model the global-level spatial correlations among local scene features, thereby more accurately capturing long-range dependencies in the image.

[0057] Assume the feature representation output by the lightweight scene understanding module is F c The dimensions are H×W×C. This invention first merges the two-dimensional spatial dimensions into a one-dimensional sequence dimension, resulting in a feature size of HW×C, where HW represents the sequence dimension and C represents the feature dimension. Then, a general scene information association rule is defined:

[0058]

[0059] Among them, F gi Let G(F) represent the global scene features output at position i, where i represents the current pixel position, j represents all other pixel positions, f(·) represents the function that calculates the relationship between positions i and j, g(·) represents the function that calculates the signal representation at position j, and G(F) represents the function that calculates the signal representation at position j. c F is used for normalization. ci F represents the feature data of the current pixel position. cj Feature data representing the location of all other pixels.

[0060] For f(·), this invention uses feature encoding and element-wise dot product to achieve it. Specifically, first, the input feature F... ci and F cjFeature encoding is performed by transforming the feature space into an independent feature space using a linear mapping. Then, the correlation between the current pixel feature and other pixel features is calculated using element-wise dot product summation. Finally, the similarity between features is obtained through normalization. The calculation formula can be expressed as:

[0061]

[0062] Where, θ(F) ci ) = W θ F ci ,φ(F cj ) = W φ F cj θ and φ represent different feature mapping processes, W θ and W φ They represent the corresponding mapping parameters, T represents the matrix transpose operation, and G(F) represents the matrix transpose operation. c ) is used for normalization.

[0063] For g(·), a linear mapping is also used to implement the feature encoding operation, and the formula can be expressed as:

[0064] g(F cj ) = F cj W g (5)

[0065] Among them, W g This represents the mapping parameters in a linear mapping.

[0066] After the local scene features are processed by the global information association module, each pixel establishes a spatial similarity association with all other pixels, which enhances the model's ability to understand scene information and provides effective scene feature support for subsequent multi-scene object detection.

[0067] S3: The image and scene features are input into a pre-trained multi-scene target detection network for target detection to obtain the target recognition and localization result in the current scene. The multi-scene target detection network includes an image feature extraction module, a shift attention module, a scene feature guidance module, and a target query localization module. Specifically, the image is input into the image feature extraction module for feature extraction to obtain local features. The local features and a preset reference point are then input into the shift attention module for shift attention operations to obtain shift attention features. These shift attention features are then input into the target query localization module. Scene features and a preset target query vector are input into the scene feature guidance module to obtain a scene target query vector. This scene target query vector is then input into the target query localization module. The target query localization module receives the shift attention features and the scene target query vector and performs target detection.

[0068] In this embodiment, for UAV target detection tasks, effectively integrating scene features during the target detection process to achieve targeted feature extraction and target recognition and localization is crucial. To address this, this invention designs an end-to-end multi-scene target detection network, mainly comprising an Image Feature Extraction (IFE) module, a Shiftable Attention (SA) module, a Scene Feature Guidance (SFG) module, and a Target Query Localization (TQL) module.

[0069] S301: Input the image into the image feature extraction module for feature extraction to obtain local features (i.e., spatial features and semantic features).

[0070] In the multi-scene object detection network, the image captured by the drone is first fed into an image feature extraction module composed of a convolutional neural network to extract local features. This module mainly consists of multiple convolutional layers and pooling layers. The convolutional layers are used to perform local modeling of the image features captured by the drone, and the pooling layers are used to compress the spatial dimension of the feature map and improve the receptive field of the features. This module effectively and automatically extracts identifiable spatial features (such as edges, textures, and colors) and semantic features (such as the shape and category of objects, and the layout of the scene) from the original image. These features help the algorithm understand the target information in the image and provide strong support for the subsequent accurate localization of the target.

[0071] S302: Input the local features and preset reference points into the offset attention module to perform offset attention calculations and obtain offset attention features.

[0072] After obtaining the local features of the image, they are fed into the Transformer encoder to extract features. The traditional Transformer structure uses intensive attention operations, where each sequence feature must be associated with all sequence features globally in terms of similarity. The computational cost is quadratic with the sequence length, which is not conducive to practical deployment on UAV platforms. To address this, this invention constructs an offset attention operation module. By pre-setting a set of reference points for each feature and learning a set of offset coordinates, each feature is then associated with only a limited number of image features near the reference points under the guidance of the offset coordinates. This greatly reduces the computational complexity of the original module and facilitates real-time deployment of the algorithm.

[0073] For the offset attention module, assuming a given input image feature (local feature) x, a query vector q, and the coordinates q of the reference point corresponding to the query vector. rFirst, a feature mapping operation is performed on the query vector using a linear mapping to generate the offset and corresponding attention weight for each sampling point:

[0074] Δq u =qW Δ (6)

[0075] A u =softmax(qW) A (7)

[0076] Among them, W Δ and W A These represent the linear mapping parameters used to calculate the offset and attention weights, respectively, where u represents the sampling point index, and Δq... u This represents the offset of the u-th sampling point, which includes two dimensions: horizontal offset and vertical offset. A u This represents the attention weight between the u-th sampling point and the query vector q, with the weight ranging from [0,1]. Adding the reference point to the generated offset yields the sampling point x corresponding to each feature in the local feature set. S Specifically, it can be expressed as:

[0077] x Su =x(q) r +Δq u (8)

[0078] Where, x Su This represents the u-th reference point feature corresponding to feature x, since q r +Δq u Since the values ​​are not necessarily integers, the feature data cannot be sampled directly using translation operations. To address this, this invention employs bilinear interpolation to implement feature sampling operations based on floating-point coordinates.

[0079] After obtaining the sampled features, a feature mapping operation is performed on the feature data using linear operations. Then, the mapped features are multiplied by the attention weights, and a weighted sum based on offset attention is performed on the sampled features. Finally, the features are fed into the output layer to obtain the offset attention operation result. The process can be represented as follows:

[0080]

[0081] Where U represents the total number of sampling points, W m W represents the parameters used when performing a linear mapping on the sampled features. o F represents the linear mapping parameter of the output layer. att The generated offset attention features are represented by the following: The overall computation process of the offset attention module can be represented as follows:

[0082]

[0083] Where ShiftAtt() represents the global offset attention operation, W o The output layer's linear mapping parameters are represented by U, u, and W. Δ and W A W represents the linear mapping parameters used in calculating the offset and attention weights, respectively. m This represents the parameters used when performing a linear mapping on the sampled features.

[0084] Offset attention operation associates the query vector with U feature data near the reference point based on similarity. The computational complexity is linear with the sequence length, which effectively reduces the overall computational complexity of the attention module and facilitates the practical deployment and application of the algorithm.

[0085] In multi-scene object detection networks, this invention pre-defines a set of learnable positional codes, combines these codes with image features, and inputs them into the constructed offset attention module to perform self-attention operations. During the self-attention operation, the query vector is preset as the image's feature vector, and the reference point of each vector is the coordinate point of the corresponding image feature. This operation aims to correlate the feature data of each coordinate in the image feature with a similarity set of its neighboring feature data, thereby better modeling feature dependencies in the spatial dimension and promoting efficient understanding of scene features by the UAV. Furthermore, compared to the multi-head attention mechanism in traditional Transformers, the offset attention module constructed in this invention has a lower computational burden and can be efficiently applied to devices with limited computing resources, such as UAVs, facilitating practical application and deployment of the algorithm.

[0086] S303: Input the scene features (i.e., global scene features) and the preset target query vector into the scene feature guidance module to obtain the scene target query vector. Specifically, use average pooling to compress the spatial dimension of the scene features, and then perform feature fusion with the preset target query vector to obtain the scene target query vector.

[0087] In this embodiment, in order to enable the multi-scene target detection algorithm to perform target detection by referring to the image characteristics of the scene, and to achieve targeted feature extraction and target recognition and localization functions based on different scene features, the present invention designs a scene feature guidance module. A set of target query vectors is preset, and then the scene features extracted by the scene classification network are integrated into the query vectors. The target query vectors are given the imaging characteristics of the current scene, and the target recognition and localization operation under the current scene can be well realized in the subsequent target query process.

[0088] Specifically, firstly, a set of target query vectors Q∈R is pre-defined.N×J Where N represents the number of query vectors, which is not less than the maximum number of targets in a single image in the dataset, and J represents the feature dimension of the query vectors, which is generally the same as the number of image feature channels extracted by the feature extraction module. Subsequently, this invention extracts the global scene features F obtained from the global association module in the scene classification network. c The spatial dimension of scene features (global scene features) is compressed using average pooling, and then the spatial dimensions are merged (that is, the two-dimensional scene features are merged into a one-dimensional scene feature vector, and the spatial dimensions are merged into a sequence dimension) to generate a set of scene feature vectors F to represent the characteristics of the current scene. P Assuming the compressed scene feature dimension is H×W×C, the generated scene feature vector dimension is HW×C. After obtaining the target query vector Q and the scene feature vector F... P Subsequently, this invention utilizes linear mapping to map the target query vector and scene feature vector into the latent feature space, respectively:

[0089] Q′=QW Q ,F′ P =F P W P ,F P " = F P W′ P (11)

[0090] Among them, Q′ and W Q F′ represents the mapping features and corresponding mapping parameters of the target query vector, respectively. P and F P "" represents two mapping features of the scene feature vector, W P and W′ P These represent two sets of mapping parameters that perform a linear mapping on the scene feature vectors.

[0091] Next, the similarity matrix between them is calculated. A function is used to normalize the data similarity in the matrix to obtain the attention matrix, which is used to fuse the query vector and the scene feature vector. The fusion process can be represented as:

[0092]

[0093] Where, d k W is the similarity normalization parameter. P Q represents the parameters that perform a linear mapping on the scene feature vectors. P This invention refers to the target query vector that incorporates scene features as a Scene Object Query. By using this vector to query typical targets in image features, the purpose of scene-differentiated target detection can be effectively achieved.

[0094] S304: Input the offset attention features and scene target query vectors into the target query localization module. The scene target query vectors are first modeled by the self-attention module to model the correlation between scene target query vectors, and then input into the cross-attention module along with the offset attention features. The scene target query vectors are used to query typical targets in the offset attention features.

[0095] In this embodiment, after obtaining the offset attention features and the scene target query vector, the present invention inputs them together into the target query localization module. The scene target query vector first passes through the object-focused self-attention module to model the correlation between target query vectors, and then is input together with the offset attention features into the image-to-object cross-attention module. The scene target query vector is used to query typical targets in the offset attention features, such as vehicle targets in the city, ship targets in the ocean, and aircraft targets in the airport, to achieve the purpose of UAV target detection in cross-scene scenarios.

[0096] Specifically, in the target-centric self-attention module, the scene target query vector is independently mapped to the latent feature space via a linear mapping. Then, the self-attention mechanism is used to construct similarity associations between vectors. The specific process can be represented as follows:

[0097]

[0098] Q′ S =Attention(Q) S W Q Q S W K Q S W V (14)

[0099] Among them, W Q W K W V These are the parameters corresponding to different linear mappings, used to adaptively adjust Q. S and Q′ S Mapping to different feature spaces improves the attention efficiency of attention operations, Q S Let Q′ be the scene target query vector. S This is the scene target query vector after self-attention operation.

[0100] After self-attention processing, scene query vectors can communicate with each other, and different vectors can focus on different objects, avoiding duplicate attention.

[0101] After the target-centric self-attention module, this invention then inputs the scene query vector and offset attention features into the target cross-attention module. The query vector is used to query typical targets from the offset attention features. To achieve scene target querying in a lightweight manner, this invention uses an offset attention module to implement the cross-attention operation. Specifically, the scene target query vector Q′ after self-attention operation... S The size is N×C, where N is the number of possible targets, and each target corresponds to the bounding box coordinates q of the object to be queried. ob In this invention, the coordinates are used as reference coordinates for the query vector, and the cross-attention operation formula based on offset attention can be expressed as:

[0102] F obj =ShiftAtt(F att ,Q′ S ,q ob (15)

[0103] Among them, F obj The retrieved target features are then fed into a target detection head composed of linear mappings to predict the target's category and bounding box coordinates, thus achieving target localization.

[0104] y box =F obj W box y cls =F obj W cls

[0105] Among them, y box For the predicted bounding box coordinates of the scene target, y cls W represents the predicted target category score for the scene. box W is used to predict the parameters of a typical target bounding box. cls The parameters used to predict typical target categories.

[0106] The loss function of this invention mainly consists of two parts: scene classification loss and target recognition and localization loss. The scene classification loss is used to optimize the scene classification network, improving its ability to distinguish scene types and thus more accurately identifying scene types. Assuming the prediction result of the scene classification network is y... s The corresponding actual label results are This invention uses the cross-entropy loss function as the scene classification loss, and the formula of the function can be expressed as:

[0107]

[0108] Among them, L scn For scene classification loss, C scn denoted by z, where z is the number of scene categories and z is the category index.

[0109] The target recognition and localization loss is used to optimize the multi-scene target detection network. This invention first uses the Hungarian algorithm to match the network's predictions with the object labels, assigning a label to each object query result. Assume the corresponding bounding box and category label are respectively... and The target recognition and localization loss can be expressed as:

[0110]

[0111] Among them, L m L represents the total algorithm loss of the target recognition and localization module. cls For object classification loss, this invention employs the cross-entropy loss function, L. box For bounding box regression loss, this invention uses IoU loss, and the loss calculation formula can be expressed as follows:

[0112]

[0113] Where Area() represents the operation to calculate the area of ​​a specified region, y box The coordinates of the predicted scene target bounding box. The coordinates of the bounding box of the real scene target.

[0114] The total loss function of the algorithm designed in this invention can be expressed as:

[0115] L = L scn +λL m (20)

[0116] Here, λ is used to balance the relative magnitudes between scene classification loss and target recognition and localization loss.

[0117] In summary, this invention first feeds the image into a scene classification network to extract scene features. Then, the image and scene features are simultaneously fed into a multi-scene object detection network to perform scene-specific object detection operations, thereby specifically detecting typical targets in the current scene and completing the UAV target detection task across multiple scenes. Specifically, this invention uses a scene classification network (SCN) to automatically extract scene features from the current image, while utilizing a global information association module to achieve the association representation of global scene information, enabling more efficient adaptive perception of scene information. The scene classification network models the feature differences between different scenes, facilitating the subsequent multi-scene object detection network to achieve scene-differentiated typical target detection based on scene features. Subsequently, in the multi-scene object detection network (MODN), the original image data is sequentially fed into a convolutional neural network and a Transformer encoder composed of an offset attention module, completing image feature extraction and spatial feature association with low computational cost. Afterwards, a set of object query vectors is initialized, and a scene feature guidance module is used to fuse the scene features extracted by the scene classification network with the object query vectors, generating a target query vector specific to the current scene for targeted querying of typical targets in the current UAV scene. Finally, the obtained scene target query vector and image features are fed into the target query and localization module. Cross-attention is used to allow the query vector and image features to fully interact, query typical targets in a specific scene, and finally realize the cross-scene target detection task for UAVs.

[0118] This invention provides a novel cross-scene UAV target detection algorithm. By fusing scene features with target query vectors, the query vectors can be effectively imbued with scene characteristics, thereby enabling targeted identification and localization of typical targets in the current scene. This significantly improves the robustness of the UAV target detection algorithm in cross-scene applications. This invention proposes a novel scene classification network, mainly composed of a lightweight scene understanding module and a global information association module, achieving efficient representation of scene features with low computational cost. This invention also proposes a novel lightweight end-to-end target detection algorithm for UAV applications. Utilizing an offset attention module to construct feature associations within a local range, this effectively reduces the overall computational cost of the algorithm, improves the inference speed on UAV platforms, and facilitates practical deployment.

[0119] Example 2

[0120] This embodiment discloses a scene-adaptive unmanned aerial vehicle (UAV) target detection system, including:

[0121] The image acquisition module is configured to acquire images of the current drone application scenario.

[0122] The scene classification module is configured to: input the image into a pre-trained scene classification network for feature extraction to obtain scene features;

[0123] The target detection module is configured to: input the image and the scene features into a pre-trained multi-scene target detection network for target detection, and obtain the target recognition and localization result in the current scene; the multi-scene target detection network includes an image feature extraction module, a shift attention module, a scene feature guidance module, and a target query localization module; wherein, the image is input into the image feature extraction module for feature extraction to obtain local features, and the local features and preset reference points are input into the shift attention module for shift attention operation to obtain shift attention features, and the shift attention features are input into the target query localization module; the scene features and preset target query vectors are input into the scene feature guidance module to obtain scene target query vectors, and the scene target query vectors are input into the target query localization module; the target query localization module receives the shift attention features and the scene target query vectors and performs target detection.

[0124] Example 3

[0125] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method of Embodiment 1.

[0126] Example 4

[0127] The purpose of this embodiment is to provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the method of Embodiment 1.

[0128] The steps and methods involved in the apparatuses of Embodiments 3 and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0129] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0130] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0131] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A scene-adaptive UAV target detection method, characterized in that, The method comprises the following steps: acquiring an image in a current unmanned aerial vehicle application scenario; inputting the image into a pre-trained scene classification network to extract a scene feature; the scene classification network comprises a lightweight scene understanding module and a global information correlation module connected in sequence, the lightweight scene understanding module receives the image and extracts a local scene feature of the image, and inputs the local scene feature into the global information correlation module, the global information correlation module uses a scene information correlation rule to correlate the local scene feature in a spatial dimension to obtain a global scene feature; inputting the image and the scene feature into a pre-trained multi-scene target detection network to perform target detection, and obtaining a target recognition and positioning result in the current scene; the multi-scene target detection network comprises an image feature extraction module, an offset attention module, a scene feature guiding module, and a target query positioning module; wherein the image is input into the image feature extraction module to extract a local feature, and the local feature and a preset reference point are input into the offset attention module to perform offset attention operation to obtain an offset attention feature, and the offset attention feature is input into the target query positioning module; the scene feature and a preset target query vector are input into the scene feature guiding module to obtain a scene target query vector, and the scene target query vector is input into the target query positioning module; the target query positioning module receives the offset attention feature and the scene target query vector to perform target detection. 2.The scene-adaptive UAV target detection method of claim 1, wherein, The lightweight scene understanding module comprises a spatial convolution and a channel convolution connected in sequence, the input image is first subjected to the spatial convolution to extract spatial features of the image in a channel-by-channel manner, and then the spatial features are input into the channel convolution to extract channel features of the spatial features in a spatial position-by-spatial position manner to obtain local scene features modeling spatial information and channel information. 3.The scene-adaptive UAV target detection method of claim 1, wherein, The offset attention feature is obtained in the following manner: inputting the local feature, a query vector, and a reference point coordinate corresponding to the query vector into the offset attention module; performing a feature mapping operation on the query vector by using linear mapping to generate an offset and a corresponding attention weight for each sampling point; obtaining a sampling point corresponding to the local feature based on the reference point and the offset, and performing feature sampling on the sampling point to obtain a sampling feature; performing a feature mapping operation on the local feature by using linear operation, then multiplying the mapped feature and the attention weight, and performing weighted summation on the sampling feature based on offset attention to obtain the offset attention feature.

4. The scene-adaptive UAV target detection method of claim 1, wherein, The scene target query vector is obtained in the following manner: compressing the spatial dimension of the scene feature by using average pooling, and then performing feature fusion with a preset target query vector to obtain the scene target query vector. 5.The scene-adaptive UAV target detection method of claim 1, wherein, In the target query positioning module, the scene target query vector first passes through a self-attention module to model the correlation between the scene target query vectors, and then is input into a cross-attention module together with the offset attention feature, and the scene target query vector is used to query a typical target in the offset attention feature.

6. The scene-adaptive UAV target detection method of claim 5, wherein, The self-attention module is a target-centered self-attention module, wherein the scene target query vector is independently mapped into a latent feature space through linear mapping, and then the similarity correlation between vectors is constructed by using a self-attention mechanism.

7. A scene-adaptive unmanned aerial vehicle target detection system, characterized in that, The method comprises the following steps: an image acquisition module configured to acquire an image in a current unmanned aerial vehicle application scenario; a scene classification module configured to input the image into a pre-trained scene classification network to extract a scene feature; The scene classification network comprises a lightweight scene understanding module and a global information correlation module connected in sequence, the lightweight scene understanding module receives the image and extracts a local scene feature of the image, and inputs the local scene feature into the global information correlation module, the global information correlation module performs similarity correlation on the local scene feature in a spatial dimension by using a scene information correlation rule to obtain a global scene feature; a target detection module configured to input the image and the scene feature into a pre-trained multi-scene target detection network to perform target detection and obtain a target recognition and positioning result in the current scene; the multi-scene target detection network comprises an image feature extraction module, an offset attention module, a scene feature guiding module, and a target query positioning module; wherein the image is input into the image feature extraction module to extract a local feature, and the local feature and a preset reference point are input into the offset attention module to perform offset attention operation and obtain an offset attention feature, and the offset attention feature is input into the target query positioning module; the scene feature and a preset target query vector are input into the scene feature guiding module to obtain a scene target query vector, and the scene target query vector is input into the target query positioning module; The target query positioning module receives the offset attention feature and the scene target query vector to perform target detection.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the scene adaptive unmanned aerial vehicle target detection method in any one of claims 1-6.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the scene adaptive unmanned aerial vehicle target detection method in any one of claims 1-6.

Images