Scene-adaptive unmanned aerial vehicle target detection method and system

Abstract

The invention discloses a scene-adaptive unmanned aerial vehicle target detection method and system. The method comprises the following steps: inputting an image into a scene classification network to obtain scene features; inputting the image and the scene features into a multi-scene target detection network to obtain a target identification positioning result; specifically, an image is input to an image feature extraction module to obtain a local feature, the local feature and a preset reference point are input to an offset attention module to obtain an offset attention feature, and the offset attention feature is input to a target query positioning module; inputting the scene features and a preset target query vector into a scene feature guiding module to obtain a scene target query vector, and inputting the scene target query vector into a target query positioning module; the target query positioning module performs target detection based on the offset attention feature and the scene target query vector. The unmanned aerial vehicle target detection tasks of different scenes are unified by adopting one model, a detection algorithm is lightened, computing resources are saved, and unmanned aerial vehicle deployment is facilitated.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a scene-adaptive unmanned aerial vehicle target detection method and system. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] In the field of computer vision, object detection is an important basic task. It combines the concepts of object recognition and object localization, and aims to identify and locate typical targets in images. It is often deployed in drones, so that drones can adaptively perceive the surrounding environment. Currently, there are many application scenarios for drones, and the environmental characteristics and typical targets of each scenario are different. How to intelligently and efficiently extract targeted features for each scene is the key to ensuring the robust application of drone target detection algorithms in various scenarios.

[0004] At present, most of the algorithm solutions for UAV object detection in multi-scene tasks are divided into two categories. The first is to train a target detection model for each scene separately, and then call different model parameters according to different scene types. Although this method can perform targeted feature extraction operations according to specific scenes, this method will split the original data set during the training stage, which hinders the improvement of model robustness. At the same time, a large number of model parameters need to be stored during the test process, which is not conducive to actual deployment and application. The second is to extract image features of different scenes with the same feature extraction mode without distinguishing scenes. In the process of performing target detection, this method does not consider the imaging characteristics of the scene and the distribution law of objects to achieve targeted feature extraction operations, which hinders the improvement of model robustness. Both methods are difficult to achieve cross-scene UAV target detection tasks. Therefore, for multi-scene target detection tasks for drones, the existing solutions are difficult to unify target detection tasks of different scenes with one model, and the training data of multiple scenes cannot be fully utilized, which is not conducive to the improvement of model generalization performance.

[0005] In addition, for UAV platforms with limited computing resources, although the traditional anchor-based target detection algorithm has high detection accuracy, this type of model often obtains a large number of repeated detection frames. At the end of the detector, the non-maximum suppression (NMS) algorithm needs to be used to filter out bounding boxes with high confidence. This process consumes a lot of computation and is not conducive to deployment and application in UAV mission scenarios. A more effective solution is the Transformer-based target detection algorithm (Detection Transformer, DETR). By using the multi-head attention mechanism in the Transformer to model the correlation between different targets, it can directly predict the target bounding box without presetting the anchor point, complete the end-to-end target detection operation, and reduce the amount of computation in the actual application of the algorithm to a certain extent. However, since the traditional Transformer uses intensive attention operations, the amount of computation is large, and the algorithm's reasoning process is slow, which is not conducive to the actual deployment and application of UAVs. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a scene-adaptive UAV target detection method and system, which realizes the use of one model to unify UAV target detection tasks in different scenes, and lightweights the detection algorithm, saves computing resources, and facilitates UAV deployment.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, the present invention provides a scene-adaptive drone target detection method, comprising:

[0009] Get the image of the current drone application scenario;

[0010] Inputting the image into a pre-trained scene classification network for feature extraction to obtain scene features;

[0011] The image and the scene features are input into a pre-trained multi-scene target detection network for target detection to obtain a target recognition and positioning result in the current scene; the multi-scene target detection network includes an image feature extraction module, an offset attention module, a scene feature guidance module and a target query positioning module; wherein the image is input into the image feature extraction module for feature extraction to obtain local features, and the local features and the preset reference points are input into the offset attention module for offset attention calculation to obtain offset attention features, and the offset attention features are input into the target query positioning module; the scene features and the preset target query vector are input into the scene feature guidance module to obtain a scene target query vector, and the scene target query vector is input into the target query positioning module; the target query positioning module receives the offset attention features and the scene target query vector to perform target detection.

[0012] According to a further technical solution, the scene classification network includes a lightweight scene understanding module and a global information association module which are connected in sequence. The lightweight scene understanding module receives an image and extracts local scene features of the image, and inputs the local scene features into the global information association module. The global information association module uses scene information association rules to perform similarity association on the local scene features in the spatial dimension to obtain global scene features.

[0013] A further technical solution is that the lightweight scene understanding module includes spatial convolution and channel convolution connected in sequence. The input image first undergoes spatial convolution to extract the spatial features of the image in a channel-by-channel manner. Subsequently, the spatial features are input into the channel convolution to extract the channel features of the spatial features in a spatial position-by-spatial position manner to obtain local scene features that model spatial information and channel information.

[0014] A further technical solution is to obtain the offset attention feature as follows:

[0015] Inputting the local feature, the query vector, and the coordinates of the reference point corresponding to the query vector into an offset attention module;

[0016] Use linear mapping to perform feature mapping on the query vector to generate the offset and corresponding attention weight of each sampling point;

[0017] Based on the reference point and its offset, a sampling point corresponding to the local feature is obtained, and feature sampling is performed on the sampling point to obtain a sampling feature;

[0018] A feature mapping operation is performed on the local features using linear operations, and then the mapped features are multiplied by the attention weights, and the sampled features are weighted summed based on the offset attention to obtain the offset attention features.

[0019] A further technical solution to obtain the scene target query vector is as follows: the spatial dimension of the scene features is compressed using average pooling, and then the features are fused with the preset target query vector to obtain the scene target query vector.

[0020] According to a further technical solution, the scene target query vector in the target query positioning module first passes through a self-attention module to model the correlation between the scene target query vectors, and then is input into a cross-attention module together with the offset attention feature, and the scene target query vector is used to query the typical target in the offset attention feature.

[0021] A further technical solution is that the self-attention module is a target-centric self-attention module, in which the scene target query vector is independently linearly mapped to the latent feature space, and then the self-attention mechanism is used to construct the similarity association between the vectors.

[0022] In a second aspect, the present invention provides a scene-adaptive drone target detection system, comprising:

[0023] An image acquisition module is configured to: acquire images in the current drone application scenario;

[0024] A scene classification module is configured to: input the image into a pre-trained scene classification network for feature extraction to obtain scene features;

[0025] The target detection module is configured as follows: the image and the scene features are input into a pre-trained multi-scene target detection network for target detection to obtain a target recognition and positioning result in the current scene; the multi-scene target detection network includes an image feature extraction module, an offset attention module, a scene feature guidance module and a target query positioning module; wherein the image is input into the image feature extraction module for feature extraction to obtain local features, and the local features and the preset reference points are input into the offset attention module for offset attention calculation to obtain offset attention features, and the offset attention features are input into the target query positioning module; the scene features and the preset target query vector are input into the scene feature guidance module to obtain a scene target query vector, and the scene target query vector is input into the target query positioning module; the target query positioning module receives the offset attention features and the scene target query vector to perform target detection.

[0026] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in a scene-adaptive drone target detection method as described in the first aspect.

[0027] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the scene-adaptive drone target detection method as described in the first aspect are implemented.

[0028] One or more of the above technical solutions have the following beneficial effects:

[0029] The present invention can achieve efficient characterization of scene features by gradually extracting local and global features of the scene through a scene classification network. Furthermore, in a multi-scene target detection network, the algorithm can combine the extracted scene features to specifically implement target recognition and positioning operations in the current scene, thereby achieving robust application of target detection algorithms across scenes. A lightweight offset attention module is proposed, which can effectively achieve information aggregation in a local range by establishing similarity associations between image features and features near reference points, reducing the overall computational complexity of the algorithm, thereby promoting the actual deployment and application of the algorithm.

[0030] The present invention introduces a scene classification network (SCN): by extracting local scene features and associating global scene information in a lightweight manner, the scene features of the image captured by the current drone can be effectively extracted, making it easier for subsequent target detection algorithms to perform targeted target recognition and positioning operations based on the scene features.

[0031] The present invention proposes a Multi-Scene Object Detection Network (MODN): the shiftable attention module (SAM) is introduced to improve the traditional intensive attention operation into a sparse attention operation, reducing the overall computational complexity of the network. At the same time, a scene feature guidance module (SFGM) is proposed, which can achieve targeted feature extraction and target recognition operations by integrating scene features into the target query vector, thereby improving the generalization ability of the model in multi-scenario applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0033] Figure 1 It is a flow chart of a target detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0034] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0035] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0036] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0037] In Multi-Scene UAV Object Detection, the training and testing processes involve images from multiple scenes, each with different scene characteristics and typical target distributions. Different from traditional single-scene target detection tasks, the key challenge of cross-scene target detection is how to accurately understand scene information and perform targeted feature extraction and target recognition based on scene features, so as to achieve robust application of the algorithm in multi-scene target detection tasks.

[0038] Embodiment 1

[0039] like Figure 1 As shown, this embodiment discloses a scene-adaptive drone target detection method, which includes the following steps:

[0040] S1: Get the image of the current drone application scenario;

[0041] In this embodiment, in various drone application scenarios, corresponding images are acquired for target detection, and the acquired images are annotated to construct a data set for training a scene classification network and a multi-scenario target detection network.

[0042] The specific process of constructing the training data set is as follows: first, use drones to collect image data of multiple scenes, such as cities, suburbs, airports, and oceans, and then assign two types of labels to each image. The first type is the scene type label. Different scene images are assigned different scene type numbers. For example, city scene images are labeled as type 1, suburban scene images are labeled as type 2, airport scene images are labeled as type 3, and ocean scene images are labeled as type 4. The second type is the target recognition and positioning label. Use existing labeling software (such as labelimg software) to label the bounding box position and category of typical targets in each image. For example, pedestrians and vehicle targets in urban scenes, vehicles and building targets in suburban scenes, aircraft targets in airport scenes, and ship targets in ocean scenes. Each set of training data consists of three parts: input image data, scene type labeling, and bounding box and category labeling of typical targets.

[0043] S2: inputting the image into a pre-trained scene classification network for feature extraction to obtain scene features;

[0044] In this embodiment, in the application scenarios of drones, multiple application scenarios (such as urban scenarios, suburban scenarios, ocean scenarios, etc.) may be involved. Different scenarios have different imaging characteristics and task requirements, so perception algorithms are needed to realize the functions of differentiated feature extraction and typical target positioning for different scene types. This requires an algorithm module with multi-scene classification function to connect this process, first efficiently complete the adaptive extraction of scene features, and then combine the extracted scene features with the target detection algorithm to complete the typical target recognition operation in a specific scene. Based on this, the present invention proposes a new scene classification network.

[0045] The scene classification network includes a lightweight scene understanding (LSU) module and a global information association (GIA) module, which are connected in sequence. It extracts local scene features in a lightweight manner and associates information at the global level to obtain global scene features, which can effectively characterize the scene information in the current environment of the drone and realize efficient feature extraction operations.

[0046] S201: The lightweight scene understanding module receives an image and extracts local scene features of the image, and inputs the local scene features into the global information association module. Specifically, the lightweight scene understanding module includes spatial convolution and channel convolution connected in sequence. The input image first undergoes spatial convolution to extract the spatial features of the image in a channel-by-channel manner. Subsequently, the spatial features are input into the channel convolution to extract the channel features of the spatial features in a spatial position-by-spatial manner, thereby obtaining local scene features that model spatial information and channel information.

[0047] In actual UAV applications, the timeliness of the algorithm directly affects the practical application ability of the model. For classification tasks, the computational complexity of the feature extraction part occupies the main part of the model. In order to reduce the overall computational complexity of the algorithm, the present invention decouples the traditional standard convolution operation and designs spatial convolution (SC) and channel convolution (SC) to realize feature extraction of UAV scene images.

[0048] The input image (input feature map) first undergoes spatial convolution, which performs convolution operations only in the spatial dimension to extract the spatial features of the image in a channel-by-channel manner. Assume that the input image F in The size is H×W×C, and each input channel c corresponds to a convolution kernel D s , the convolution kernel size is a×a, then the calculation formula of spatial convolution can be expressed as:

[0049]

[0050] Among them, F s In order to model the local scene features of spatial information (i.e., spatial features), h, w, c represent the height, width, and number of channels of the input image, respectively, m, n are the indexes of the spatial position, respectively, and F in is the input image, D s is the convolution kernel.

[0051] Then the spatial feature F s The convolution operation is performed only on the channel dimension to extract spatial features F in a spatial position-by-space manner. s The feature vectors at all spatial positions correspond to the same convolution kernel D c , assuming that the number of output channels is o and the convolution kernel size is 1×1×c×o, the calculation formula for channel convolution can be expressed as:

[0052]

[0053] Among them, F c In order to model the local scene features of spatial information and channel information, k is the index of the channel position.

[0054] The present invention replaces the traditional convolution operation with a series combination of spatial convolution and channel convolution, and obtains scene features of modeling spatial information and channel information by extracting the spatial features and channel features of the input image in sequence. It can complete a lightweight understanding of the scene features and quickly extract the scene features in the current image in an efficient manner, thereby better meeting the needs of drones for real-time data processing and high-precision decision-making in complex environments.

[0055] S202: The global information association module receives local scene features, and uses scene information association rules to perform similarity association on the local scene features in the spatial dimension to obtain global scene features.

[0056] When understanding scene information, it is necessary to fully consider the global information distribution of the entire image. One of the major advantages of convolution operation is local perception, that is, only extracting features from local areas, thereby reducing the amount of calculation. However, this localized feature extraction method is not conducive to building long-range dependencies between features, and it is difficult to achieve global information perception. In this regard, the present invention designs a global information association module, which integrates the local scene features obtained by the lightweight scene understanding module. This module can model the spatial correlation at the global level in the local scene features, thereby more accurately capturing the long-range dependencies in the image.

[0057] Assume that the feature representation of the output of the lightweight scene understanding module is F c , the size is H×W×C. The present invention first merges the two-dimensional spatial dimension into a one-dimensional sequence dimension, and the feature size becomes HW×C, where HW represents the sequence dimension and C represents the feature dimension. Then a general scene information association rule is defined:

[0058]

[0059] Among them, F gi represents the global scene feature output at position i, i represents the current pixel position, j represents all other pixel positions, f(·) represents the function that calculates the relationship between position i and position j, g(·) represents the signal representation at position j, G(F c ) is used for normalization, F ci Indicates the feature data of the current pixel position, F cj Represents feature data for all other pixel locations.

[0060] For f(·), the present invention adopts feature encoding and element dot product to implement it. Specifically, the input feature F is first ci and F cjPerform feature encoding operation and use linear mapping to convert it into an independent feature space. Then, use the element-by-element dot product summation method to calculate the correlation between other pixel features and the current pixel feature. Finally, the similarity between the features is obtained through normalization operation. The calculation formula can be expressed as:

[0061]

[0062] Among them, θ(F ci )=W θ F ci ,φ(F cj )=W φ F cj , θ and φ represent different feature mapping processes, W θ and W φ They represent the corresponding mapping parameters, T represents the matrix transposition operation, G(F c ) is used for normalization.

[0063] For g(·), linear mapping is also used to implement feature encoding operation, and the formula can be expressed as:

[0064] g(F cj )=F cj W g (5)

[0065] Among them, W g Represents the mapping parameters in a linear mapping.

[0066] After the local scene features pass through the global information association module, each pixel point builds a similarity association with all other pixels in the spatial dimension, which enhances the model's ability to understand scene information and provides effective scene feature support for subsequent multi-scene target detection.

[0067] S3: Input the image and the scene features into a pre-trained multi-scene target detection network for target detection to obtain target recognition and positioning results in the current scene; the multi-scene target detection network includes an image feature extraction module, an offset attention module, a scene feature guidance module and a target query positioning module; wherein, the image is input into the image feature extraction module for feature extraction to obtain local features, and the local features and the preset reference points are input into the offset attention module for offset attention calculation to obtain offset attention features, and the offset attention features are input into the target query positioning module; the scene features and the preset target query vector are input into the scene feature guidance module to obtain a scene target query vector, and the scene target query vector is input into the target query positioning module; the target query positioning module receives the offset attention features and the scene target query vector to perform target detection.

[0068] In this embodiment, for the drone target detection task, how to effectively integrate scene features in the target detection process, so as to achieve targeted feature extraction and target recognition and positioning, is particularly critical. To this end, the present invention designs an end-to-end multi-scene target detection network, which mainly includes an image feature extraction (IFE) module, a shiftable attention (SA) module, a scene feature guidance (SFG) module, and a target query localization (TQL) module.

[0069] S301: Input the image into the image feature extraction module for feature extraction to obtain local features (ie, spatial features and semantic features).

[0070] In the multi-scene target detection network, the image captured by the drone is first passed to the image feature extraction module composed of a convolutional neural network to extract local features. This module is mainly composed of multiple convolutional layers and pooling layers. The convolutional layer is used to locally model the image features captured by the drone, and the pooling layer is used to compress the spatial dimension of the feature map and improve the receptive field of the feature. This module effectively and automatically extracts recognizable spatial features (such as edges, textures, colors, etc.) and semantic features (such as the shape, category, layout of the scene, etc. of the object) from the original image. These features can help the algorithm understand the target information in the image and provide strong support for the precise positioning of subsequent targets.

[0071] S302: Input the local features and the preset reference points into the offset attention module to perform offset attention calculation to obtain the offset attention features.

[0072] After obtaining the local features of the image, it is passed to the Transformer encoder to extract features. The traditional Transformer structure uses intensive attention operations. Each sequence feature must be associated with all global sequence features by similarity. The amount of calculation is in a square relationship with the sequence length, which is not conducive to actual deployment and application on the UAV platform. In this regard, the present invention constructs an offset attention operation module, which presets a set of reference points for each feature, learns a set of offset coordinates, and then, under the guidance of the offset coordinates, allows each feature to only associate similarity with limited image features near the reference point, which greatly reduces the complexity of the original module's calculation amount and facilitates the real-time deployment of the algorithm.

[0073] For the offset attention module, assuming that an input image feature (local feature) x, a query vector q, and the reference point coordinates q corresponding to the query vector are given rFirst, a linear mapping is used to perform feature mapping on the query vector to generate the offset and corresponding attention weight of each sampling point:

[0074] Δq u =qW Δ (6)

[0075] A u =softmax(qW A ) (7)

[0076] Among them, W Δ and W A They represent the linear mapping parameters used to calculate the offset and attention weight, u represents the sequence number of the sampling point, Δq u Indicates the offset of the u-th sampling point, which includes two dimensions: horizontal offset and vertical offset. u represents the attention weight between the u-th sampling point and the query vector q, and the weight range is [0,1]. Adding the reference point to the generated offset can obtain the sampling point x corresponding to each feature in the local feature S , which can be specifically expressed as:

[0077] x Su = x(q r +Δq u ) (8)

[0078] Among them, x Su Represents the u-th reference point feature corresponding to feature x. Since q r +Δq u It is not necessarily an integer, so the sampling of feature data cannot be completed directly by using translation operation. For this reason, the present invention adopts bilinear interpolation operation (BilinearInterpolation) to implement feature sampling operation based on floating-point coordinates.

[0079] After obtaining the sampled features, linear operations are used to perform feature mapping operations on the feature data. Then, the mapped features are multiplied by the attention weights, and the sampled features are weighted summed based on the offset attention. Finally, the features are passed to the output layer to obtain the offset attention operation results. The process can be expressed as:

[0080]

[0081] Among them, U represents the total number of sampling points, W m Represents the parameters used for linear mapping of sampling features, W o represents the linear mapping parameters of the output layer, F att represents the generated offset attention feature. The overall operation process of the offset attention module can be expressed as:

[0082]

[0083] Among them, ShiftAtt() represents the overall shift attention operation, W o represents the linear mapping parameter of the output layer, U represents the total number of sampling points, u represents the sequence number of the sampling points, and W Δ and W A Respectively represent the linear mapping parameters used to calculate the offset and attention weight, W m Represents the parameters used for linear mapping of sampled features.

[0084] The offset attention operation will make a similarity association between the query vector and U feature data near the reference point. The computational complexity is linearly related to the sequence length, which effectively reduces the overall computational complexity of the attention module and facilitates the actual deployment and application of the algorithm.

[0085] In a multi-scene target detection network, the present invention presets a set of learnable position codes, combines the position codes with image features, and passes them into the constructed offset attention module to perform self-attention operations. During the self-attention operation, the query vector is preset as the feature vector of the image, and the reference point of each vector is the coordinate point of the corresponding image feature. This operation aims to make the feature data of each coordinate in the image feature similar to a set of feature data, so as to better model the feature dependency in the spatial dimension and promote the efficient understanding of scene features by drones. At the same time, compared with the multi-head attention mechanism in the traditional Transformer, the offset attention module constructed by the present invention has a small computational burden and can be efficiently applied to devices with limited computing resources such as drones, which is convenient for the actual application and deployment of the algorithm.

[0086] S303: Input the scene feature (i.e., global scene feature) and the preset target query vector into the scene feature guidance module to obtain the scene target query vector. Specifically, the spatial dimension of the scene feature is compressed by average pooling, and then the scene target query vector is fused with the preset target query vector to obtain the scene target query vector.

[0087] In this embodiment, in order to allow the multi-scene target detection algorithm to refer to the image characteristics of the scene to perform target detection, and to achieve targeted feature extraction and target recognition and positioning functions according to different scene features, the present invention designs a scene feature guidance module, presets a set of target query vectors, and then integrates the scene features extracted by the scene classification network into the query vector, and assigns the target query vector the imaging characteristics of the current scene. In the subsequent process of executing the target query, the target recognition and positioning operation in the current scene can be well achieved.

[0088] Specifically, we first preset a set of target query vectors Q∈RN×J , where N represents the number of query vectors, which is not less than the maximum number of targets in a single image in the data set, and J represents the feature dimension of the query vector, which is generally the same as the number of image feature channels extracted by the feature extraction module. Then, the present invention extracts the global scene feature F obtained by the global association module in the scene classification network. c , use average pooling to compress the spatial dimension of scene features (global scene features), and then merge the spatial dimensions (that is, merge the two-dimensional scene features into a one-dimensional scene feature vector, and merge the spatial dimensions into a sequence dimension) to generate a set of scene feature vectors F used to characterize the characteristics of the current scene P , assuming that the dimension of the compressed scene feature is H×W×C, the dimension of the generated scene feature vector is HW×C. After obtaining the target query vector Q and the scene feature vector F P Afterwards, the present invention uses linear mapping to map the target query vector and the scene feature vector to the potential feature space respectively:

[0089] Q′=QW Q ,F′ P =F P W P ,F P =F P W′ P (11)

[0090] Among them, Q′ and W Q They represent the mapping features and corresponding mapping parameters of the target query vector, respectively, and F′ P and F P ″ respectively represent the two mapping features of the scene feature vector, W P and W′ P They respectively represent two sets of mapping parameters for linear mapping of scene feature vectors.

[0091] Then calculate the similarity matrix between them, and use the function to normalize the data similarity in the matrix to get the attention matrix, which is used to fuse the query vector and the scene feature vector. The fusion process can be expressed as:

[0092]

[0093] Among them, d k is the similarity normalization parameter, W P Represents the parameter for linear mapping of scene feature vector, Q P The object query vector that integrates scene features is called a scene object query vector (Scene ObjectQuery) in the present invention. By using this vector to query typical objects in image features, the purpose of scene differentiated object detection can be effectively achieved.

[0094] S304: The offset attention features and the scene target query vector are input into the target query positioning module. The scene target query vector first passes through the self-attention module to model the correlation between the scene target query vectors, and then is input into the cross-attention module together with the offset attention features. The scene target query vector is used to query the typical targets in the offset attention features.

[0095] In this embodiment, after obtaining the offset attention feature and the scene target query vector, the present invention transmits them together to the target query positioning module. The scene target query vector first passes through the object-focused self-attention module (Object-Focused Self-Attention) to model the correlation between the target query vectors, and then is transmitted to the image-to-object cross-attention module (Image-to-Object Cross-Attention) together with the offset attention feature. The scene target query vector is used to query typical targets in the offset attention feature, such as vehicle targets in the city, ship targets in the ocean, and aircraft targets in the airport, etc., to complete the purpose of drone target detection under cross-scenes.

[0096] Specifically, in the target-centric self-attention module, the scene target query vector is independently linearly mapped to the latent feature space, and then the self-attention mechanism is used to construct the similarity association between the vectors. The specific process can be expressed as:

[0097]

[0098] Q′ S =Attention(Q S W Q ,Q S W K ,Q S W V ) (14)

[0099] Among them, W Q , W K , W V are the parameters corresponding to different linear mappings, which are used to adaptively adjust Q S and Q′ S Mapped to different feature spaces to improve the attention effect of attention operation, Q S is the scene target query vector, Q′ S is the scene target query vector after self-attention operation.

[0100] After the self-attention operation, the scene query vectors can communicate with each other, and different vectors can focus on different objects, avoiding duplicate attention.

[0101] After the target-centered self-attention module, the present invention then passes the scene query vector and the offset attention feature into the cross-attention module of the image to the target, and uses the query vector to query the typical target in the offset attention feature. In order to realize the query of the scene target in a lightweight way, the present invention adopts the offset attention module to realize the cross-attention operation. Specifically, the scene target query vector Q′ after the self-attention operation S The size is N×C, where N is the number of possible targets to be queried, and each target corresponds to a bounding box coordinate q of the object to be queried ob , the present invention regards the coordinate as the reference coordinate of the query vector, and the cross attention operation formula based on offset attention can be expressed as:

[0102] F obj =ShiftAtt(F att ,Q′ S ,q ob ) (15)

[0103] Among them, F obj is the target feature queried, and finally the target feature is passed into the target detection head composed of linear mapping to predict the target category and bounding box coordinates to achieve target determination operation:

[0104] y box =F obj W box ,y cls =F obj W cls

[0105] Among them, y box is the predicted scene object bounding box coordinate, y cls is the predicted scene target category score. box To predict the parameters used for a typical target bounding box, W cls Parameters used to predict typical target classes.

[0106] The loss function of the present invention mainly includes two parts: scene classification loss and target recognition and positioning loss. The scene classification loss is used to optimize the scene classification network, improve the classification network's ability to distinguish scene types, and is used to more accurately identify scene types. Assuming that the prediction result of the scene classification network is y s , the corresponding real label result is The present invention uses the cross entropy loss function as the scene classification loss, and the function formula can be expressed as:

[0107]

[0108] Among them, L scn is the scene classification loss, C scn is the number of scene categories, and z is the category index.

[0109] The target recognition and positioning loss is used to optimize the multi-scene target detection network. The present invention first uses the Hungarian algorithm to match the network's prediction results with the object's label, and assigns a label to each object query result. Assume that the corresponding bounding box and category label are and The target recognition and positioning loss can be expressed as:

[0110]

[0111] Among them, L m is the total algorithm loss of the target recognition and positioning module, L cls is the object classification loss. In this invention, the cross entropy loss function is used. L box is the bounding box regression loss. In this invention, IoU loss is used. The loss calculation formula can be expressed as:

[0112]

[0113] Among them, Area() is an operation to calculate the area of ​​a specified area, y box are the predicted scene object bounding box coordinates, is the actual scene target bounding box coordinate.

[0114] The total loss function of the algorithm designed by the present invention can be expressed as:

[0115] L=L scn +λL m (20)

[0116] Among them, λ is used to balance the relative size between scene classification loss and target recognition and positioning loss.

[0117] In summary, the present invention first passes the image into the scene classification network to extract the scene features of the image, and then passes the image and scene features into the multi-scene target detection network at the same time to perform the target detection operation of the specific scene, so as to detect the typical targets in the current scene in a targeted manner and complete the drone target detection task in multiple scenes. Specifically, the present invention automatically extracts the scene features of the current image by using the scene classification network (SCN), and at the same time uses the global information association module to realize the association representation of the scene global information, so as to realize the adaptive perception of the scene information in a more efficient way. The scene classification network models the feature differences of different scenes, so that the subsequent multi-scene target detection network can realize the purpose of typical target detection of scene differentiation according to the scene features. Subsequently, in the multi-scene target detection network (MODN), the original image data is sequentially passed into the convolutional neural network and the Transformer encoder composed of the offset attention module, and the image feature extraction and spatial feature association are completed with a low amount of calculation. After that, a set of target query vectors (ObjectQuery) are initialized, and the scene feature guidance module is used to fuse the scene features extracted from the scene classification network with the target query vector to generate a target query vector specific to the current scene, which is used to query the typical targets in the current drone scene in a targeted manner. Finally, the obtained scene target query vector and image features are passed into the target query and positioning module together. Cross-attention is used to allow the query vector and image features to fully interact, query typical targets in specific scenes, and finally realize the cross-scene target detection task for drones.

[0118] The present invention provides a new cross-scene UAV target detection algorithm. By fusing scene features with target query vectors, the query vector can be effectively endowed with scene characteristics, thereby realizing targeted recognition and positioning operations of typical targets in the current scene, and effectively improving the robustness of the UAV target detection algorithm in cross-scene applications. The present invention proposes a new scene classification network, which is mainly composed of a lightweight scene understanding module and a global information association module, and realizes efficient characterization of scene features with low computational complexity. The present invention proposes a new lightweight end-to-end target detection algorithm for UAV applications, which uses an offset attention module to construct feature associations within a local range, effectively reducing the overall computational complexity of the algorithm, improving the algorithm's reasoning speed in the UAV platform, and facilitating actual deployment and application.

[0119] Embodiment 2

[0120] This embodiment discloses a scene-adaptive drone target detection system, including:

[0121] An image acquisition module is configured to: acquire images in the current drone application scenario;

[0122] A scene classification module is configured to: input the image into a pre-trained scene classification network for feature extraction to obtain scene features;

[0123] The target detection module is configured as follows: the image and the scene features are input into a pre-trained multi-scene target detection network for target detection to obtain a target recognition and positioning result in the current scene; the multi-scene target detection network includes an image feature extraction module, an offset attention module, a scene feature guidance module and a target query positioning module; wherein the image is input into the image feature extraction module for feature extraction to obtain local features, and the local features and the preset reference points are input into the offset attention module for offset attention calculation to obtain offset attention features, and the offset attention features are input into the target query positioning module; the scene features and the preset target query vector are input into the scene feature guidance module to obtain a scene target query vector, and the scene target query vector is input into the target query positioning module; the target query positioning module receives the offset attention features and the scene target query vector to perform target detection.

[0124] Embodiment 3

[0125] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of embodiment 1 when executing the program.

[0126] Embodiment 4

[0127] The purpose of this embodiment is to provide a computer-readable storage medium, a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method of embodiment 1 are performed.

[0128] The steps involved in the apparatus of the above embodiments 3 and 4 correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0129] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0130] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0131] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A scene-adaptive UAV target detection method, characterized in that: include: Get the image of the current drone application scenario; Inputting the image into a pre-trained scene classification network for feature extraction to obtain scene features; The image and the scene features are input into a pre-trained multi-scene target detection network for target detection to obtain a target recognition and positioning result in the current scene; the multi-scene target detection network includes an image feature extraction module, an offset attention module, a scene feature guidance module and a target query positioning module; wherein the image is input into the image feature extraction module for feature extraction to obtain local features, and the local features and the preset reference points are input into the offset attention module for offset attention calculation to obtain offset attention features, and the offset attention features are input into the target query positioning module; the scene features and the preset target query vector are input into the scene feature guidance module to obtain a scene target query vector, and the scene target query vector is input into the target query positioning module; The target query positioning module receives the offset attention feature and the scene target query vector to perform target detection.

2. A scene-adaptive UAV target detection method as claimed in claim 1, characterized in that: The scene classification network includes a lightweight scene understanding module and a global information association module connected in sequence. The lightweight scene understanding module receives an image and extracts local scene features of the image, and inputs the local scene features into the global information association module. The global information association module uses scene information association rules to perform similarity association on the local scene features in the spatial dimension to obtain global scene features.

3. A scene-adaptive drone target detection method as claimed in claim 2, characterized in that: The lightweight scene understanding module includes spatial convolution and channel convolution connected in sequence. The input image first undergoes spatial convolution to extract the spatial features of the image in a channel-by-channel manner. Subsequently, the spatial features are input into the channel convolution to extract the channel features of the spatial features in a spatial position-by-spatial position manner to obtain local scene features of modeling spatial information and channel information.

4. The scene-adaptive UAV target detection method according to claim 1, characterized in that: The specific offset attention features are: Inputting the local feature, the query vector, and the reference point coordinates corresponding to the query vector into an offset attention module; Use linear mapping to perform feature mapping on the query vector to generate the offset and corresponding attention weight of each sampling point; Based on the reference point and its offset, a sampling point corresponding to the local feature is obtained, and feature sampling is performed on the sampling point to obtain a sampling feature; A feature mapping operation is performed on the local features using linear operations, and then the mapped features are multiplied by the attention weights, and the sampled features are weighted summed based on the offset attention to obtain the offset attention features.

5. The scene-adaptive UAV target detection method according to claim 1, characterized in that: The scene target query vector is obtained by compressing the spatial dimension of the scene features using average pooling, and then performing feature fusion with the preset target query vector to obtain the scene target query vector.

6. The scene-adaptive UAV target detection method according to claim 1, characterized in that: The scene target query vector in the target query positioning module first passes through the self-attention module to model the correlation between the scene target query vectors, and then is input into the cross-attention module together with the offset attention feature, and the scene target query vector is used to query the typical target in the offset attention feature.

7. A scene-adaptive UAV target detection method as claimed in claim 6, characterized in that: The self-attention module is a target-centric self-attention module, in which the scene target query vector is independently linearly mapped to the latent feature space, and then the self-attention mechanism is used to construct the similarity association between the vectors.

8. A scene-adaptive drone target detection system, characterized in that: include: An image acquisition module is configured to: acquire images in the current drone application scenario; A scene classification module is configured to: input the image into a pre-trained scene classification network for feature extraction to obtain scene features; A target detection module, which is configured to: input the image and the scene features into a pre-trained multi-scene target detection network for target detection to obtain a target recognition and positioning result in the current scene; the multi-scene target detection network includes an image feature extraction module, an offset attention module, a scene feature guidance module and a target query positioning module; wherein the image is input into the image feature extraction module for feature extraction to obtain local features, and the local features and a preset reference point are input into the offset attention module for offset attention calculation to obtain offset attention features, and the offset attention features are input into the target query positioning module; the scene features and a preset target query vector are input into the scene feature guidance module to obtain a scene target query vector, and the scene target query vector is input into the target query positioning module; The target query positioning module receives the offset attention feature and the scene target query vector to perform target detection.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a scene-adaptive drone target detection method as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the scene-adaptive drone target detection method according to any one of claims 1 to 7 are implemented.

Images