Adaptive unmanned aerial vehicle target detection method and device based on flow guide feature aggregation

Through the flow-guided feature aggregation technology, multi-layer downsampling convolution and motion estimation alignment features are used, and the final feature map is generated in combination with hollow convolution, which solves the detection problem of small drones in visible light imaging, and improves recognition accuracy and tracking Stability, suitable for UAV monitoring tasks in complex environments.

CN119068366APending Publication Date: 2024-12-03CHINA ACADEMY OF ELECTRONICS AND INFORMATION TECHNOLOGY OF CHINA ELECTRONICS TECHNOLOGY GROUP CORPORATION
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411177316.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The prior art is difficult to accurately detect and track small drones with small size and fast movement speeds in visible light imaging, especially in maneuverable flights or constrained vision environments.

Method used

Adaptive UAV target detection method using flow-guided feature aggregation, extract spatial features through multi-layer downsampling convolution, feature alignment and aggregation based on motion estimation, combined with hollow convolutions of different expansion rates, and generate the final feature map to achieve Object detection.

Benefits of technology

It improves the identification accuracy and tracking stability of small drones, enhances detection capabilities in complex environments, reduces misjudgment and repeated tracking, and optimizes system performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068366A_ABST
    Figure CN119068366A_ABST
Patent Text Reader

Abstract

The invention provides an adaptive unmanned aerial vehicle target detection method and device based on stream-guided feature aggregation, and the method comprises the steps: obtaining a visible light video sequence of an unmanned aerial vehicle, and processing the video sequence into a single-frame image; extracting spatial features of a current frame and an adjacent context frame in a single-frame image by adopting multi-layer down-sampling convolution; performing feature alignment on the features of the context frame and the spatial features of the current frame; convolution processing is carried out through hole convolution with different expansion rates, so that the output feature map is matched with the convolution feature map, and pixel addition is carried out to obtain a final feature map; and detecting the unmanned aerial vehicle target in the video sequence. According to the method, the flow guide feature aggregation technology and the image target detection technology are combined, the estimated optical flow is used for guiding the features of the adjacent frames to be aggregated to the current frame, the features of the current frame and the features of the front and back frames are considered at the same time for each frame, weighting and alignment are carried out through the optical flow, time information and context information can be enriched, and the recognition accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a flow-guided feature aggregation adaptive unmanned aerial vehicle target detection method and device. Background Art

[0002] With the widespread application and rapid development of drone technology, the use of drones in military, security monitoring, border patrol and other fields is increasing. This growth has also brought about the need for effective drone monitoring and defense systems, especially in the protection of important facilities and security-sensitive areas. As an effective defense, drone-borne anti-drone reconnaissance is of great significance to ensure airspace security. In this context, visible light imaging technology provides higher resolution and richer color information, which is conducive to more accurate identification and tracking of targets. Visible light imaging technology has become an important tool for drone reconnaissance with its unique advantages. However, in the field of visible light imaging video target detection, especially for small, low-flying, and fast-moving drones, existing technologies face multiple challenges: First, the small size and fast movement speed of small drones make it more difficult to accurately detect and track them in visible light imaging. Second, in continuous monitoring tasks, especially when drones are maneuvering or in environments with limited line of sight, maintaining stable tracking of targets becomes a major problem. Summary of the invention

[0003] The technical problem to be solved by the present invention is how to achieve stable tracking of a small UAV with a small size and fast movement speed; in view of this, the present invention provides an adaptive UAV target detection method and device with flow-guided feature aggregation.

[0004] The technical solution adopted by the present invention is an adaptive UAV target detection method based on flow-guided feature aggregation, comprising:

[0005] Step S1, obtaining a visible light video sequence of a drone, and processing the video sequence into a single frame image;

[0006] Step S2, extracting spatial features from the current frame and adjacent context frames in the single frame image using multi-layer downsampling convolution;

[0007] Step S3, aligning the features of the context frame with the spatial features of the current frame based on motion estimation, combining the aligned features with the features of the current frame through an aggregation function to form an intermediate feature map, and performing convolution processing on the intermediate feature map using a convolution kernel to generate a convolution feature map;

[0008] Step S4, performing convolution processing on the convolution feature map through hole convolutions with different expansion rates, and performing convolution processing on the output feature maps corresponding to different hole convolutions, so that the output feature map matches the convolution feature map, and adding the output feature map of different channels to the convolution feature map pixels, and then performing pixel addition on the addition results of different channels to obtain the final feature map;

[0009] Step S5: Detect the target of the UAV in the video sequence based on the final feature map.

[0010] In one embodiment, step S2 includes:

[0011] Extract spatial features of the frame images within the current frame and the adjacent N frames;

[0012] Pass the current spatial features through at least two convolutional layers in sequence, where each convolutional layer is used to gradually halve the size of the feature map and double the number of channels to generate a corresponding set of feature maps;

[0013] The feature map representing the second layer in the feature map set is used as input, and is combined with the feature map representing the third layer by element-by-element addition through downsampling convolution processing, and is used as the input of the next layer, and the processing of the upper layer is repeated until the last layer, so as to generate a feature map F;

[0014] The feature map F is convolved to generate processed spatial features.

[0015] In one embodiment, step S3 includes:

[0016] Obtaining spatial features of a pair of consecutive frames as input to determine a context frame feature map;

[0017] Aggregating the context frame feature map with the current frame feature map by applying a motion estimation function to obtain an aligned context frame feature map;

[0018] Aggregating the currently aligned context feature map with the current frame feature map through an aggregation function to form the intermediate feature map;

[0019] The intermediate feature map is convolved using a 3×3 convolution kernel to generate a convolution feature map.

[0020] Another aspect of the present invention further provides an adaptive UAV target detection device with flow-guided feature aggregation, comprising:

[0021] A preprocessing unit is configured to obtain a visible light video sequence of the drone and process the video sequence into a single frame image;

[0022] A feature extraction unit configured to extract spatial features from a current frame and adjacent context frames in the single frame image using multi-layer downsampling convolution;

[0023] A feature alignment unit is configured to perform feature alignment on the features of the context frame and the spatial features of the current frame based on motion estimation, combine the aligned features with the features of the current frame through an aggregation function to form an intermediate feature map, and perform convolution processing on the intermediate feature map using a convolution kernel to generate a convolution feature map;

[0024] An output unit is configured to perform convolution processing on the convolution feature map through hole convolutions with different expansion rates, and the output feature maps corresponding to different hole convolutions are convolved to match the output feature map with the convolution feature map, and add the output feature map of different channels to the convolution feature map pixels, and then add the addition results of different channels pixel by pixel to obtain a final feature map;

[0025] The detection unit is configured to detect the target of the drone in the video sequence based on the final feature map.

[0026] In one embodiment, the feature extraction unit is further configured to:

[0027] Extract spatial features of the frame images within the current frame and the adjacent N frames;

[0028] Pass the current spatial features through at least two convolutional layers in sequence, where each convolutional layer is used to gradually halve the size of the feature map and double the number of channels to generate a corresponding set of feature maps;

[0029] The feature map representing the second layer in the feature map set is used as input, and is combined with the feature map representing the third layer by element-by-element addition through downsampling convolution processing, and is used as the input of the next layer, and the processing of the upper layer is repeated until the last layer, so as to generate a feature map F;

[0030] The feature map F is convolved to generate processed spatial features.

[0031] In one embodiment, the feature alignment unit is further configured to:

[0032] Obtaining spatial features of a pair of consecutive frames as input to determine a context frame feature map;

[0033] Aggregating the context frame feature map with the current frame feature map by applying a motion estimation function to obtain an aligned context frame feature map;

[0034] Aggregating the currently aligned context feature map with the current frame feature map through an aggregation function to form the intermediate feature map;

[0035] The intermediate feature map is convolved using a 3×3 convolution kernel to generate a convolution feature map.

[0036] Another aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the adaptive drone target detection method with flow-guided feature aggregation as described in any of the above items.

[0037] Another aspect of the present invention further provides a computer storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the adaptive UAV target detection method with flow-guided feature aggregation as described in any of the above items are implemented.

[0038] Compared with the prior art, the present invention has at least the following advantages:

[0039] The present invention combines flow-guided feature aggregation technology with image target detection technology, uses the estimated optical flow to guide the features of adjacent frames to aggregate to the current frame, and considers the features of the current frame and the previous and next frames for each frame. These features are weighted and aligned through optical flow, which can enrich temporal information and contextual information and improve recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of a flow chart of an adaptive UAV target detection method using flow-guided feature aggregation according to an embodiment of the present invention;

[0041] Figure 2 A logical diagram of an adaptive UAV target detection method using flow-guided feature aggregation according to an embodiment of the present invention;

[0042] Figure 3 A data feature extraction network structure diagram according to an embodiment of the present invention;

[0043] Figure 4 A schematic diagram of the structure of an adaptive UAV target detection device with flow-guided feature aggregation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined purpose, the present invention is described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0045] Unless otherwise defined, all terms (including technical terms and scientific terms) used in this article have the same meaning as commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that terms (such as terms defined in commonly used dictionaries) should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined in this article.

[0046] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0047] In an embodiment of the present invention, a method for adaptive UAV target detection based on flow-guided feature aggregation is provided. Figure 1 As shown, including:

[0048] Step S1, obtaining a visible light video sequence of a drone, and processing the video sequence into a single frame image;

[0049] Step S2, extracting spatial features from the current frame and adjacent context frames in the single frame image using multi-layer downsampling convolution;

[0050] Step S3, aligning the features of the context frame with the spatial features of the current frame based on motion estimation, combining the aligned features with the features of the current frame through an aggregation function to form an intermediate feature map, and performing convolution processing on the intermediate feature map using a convolution kernel to generate a convolution feature map;

[0051] Step S4, performing convolution processing on the convolution feature map through hole convolutions with different expansion rates, and performing convolution processing on the output feature maps corresponding to different hole convolutions, so that the output feature map matches the convolution feature map, and adding the output feature map of different channels to the convolution feature map pixels, and then performing pixel addition on the addition results of different channels to obtain the final feature map;

[0052] Step S5: Detect the target of the UAV in the video sequence based on the final feature map.

[0053] Specifically, the step S2 includes:

[0054] Extract spatial features of the frame images within the current frame and the adjacent N frames;

[0055] Pass the current spatial features through at least two convolutional layers in sequence, where each convolutional layer is used to gradually halve the size of the feature map and double the number of channels to generate a corresponding set of feature maps;

[0056] The feature map representing the second layer in the feature map set is used as input, and is combined with the feature map representing the third layer by element-by-element addition through downsampling convolution processing, and is used as the input of the next layer, and the processing of the upper layer is repeated until the last layer, so as to generate a feature map F;

[0057] The feature map F is convolved to generate processed spatial features.

[0058] Specifically, step S3 includes:

[0059] Obtaining spatial features of a pair of consecutive frames as input to determine a context frame feature map;

[0060] Aggregating the context frame feature map with the current frame feature map by applying a motion estimation function to obtain an aligned context frame feature map;

[0061] Aggregating the currently aligned context feature map with the current frame feature map through an aggregation function to form the intermediate feature map;

[0062] The intermediate feature map is convolved using a 3×3 convolution kernel to generate a convolution feature map.

[0063] For further reference Figures 2 to 3 , the method provided by the present invention will be described in detail below.

[0064] Step S1, obtaining a visible light video sequence of a drone, and processing the video sequence into a single frame image;

[0065] Specifically, the drone is equipped with a high-resolution visible light camera that can capture clear video images under various lighting conditions. During the flight mission, the camera continuously records the video stream on the ground or in the air to ensure that all points of interest in the monitoring area are covered and any possible drone intrusion is captured. The recorded video will undergo preliminary processing before target detection.

[0066] Step S2, using multi-layer downsampling convolution to extract spatial features from the current frame and adjacent context frames in the single-frame image.

[0067] Specifically, the multi-frame feature extraction module in this embodiment uses multi-layer downsampling convolution to extract spatial features from the current frame and adjacent context frames, and uses alternating mixed input convolution operations on input data.

[0068] S2.1: Detecting UAV intrusion in camera-captured images during UAV aerial operations. Due to the challenges posed by their size, motion, and possible occlusion to model detection, we consider using flow-guided feature aggregation techniques. A video sequence F = {f t-N ,…,ft ,…,f t+N}, where f t represents the current frame at time point t, N represents the number of past and future context frames, and we will t-N 、f t 、f t+N Multi-layer downsampling convolution is used to extract spatial features.

[0069] S2.2: {f t-N ,…,f t ,…,f t+N To solve the problem of feature extraction of} data, we adopted an architecture consisting of two feature pyramids.

[0070] First, the first part of the architecture contains four convolutional layers, each of which uses a 3×3 convolution kernel for downsampling, thereby halving the size of the feature map and doubling the number of channels to generate a series of feature maps {C1, C2, C3, C4}.

[0071] Then, the second part of the architecture, a feature pyramid consisting of three layers, starts taking C2 as input and processes it through downsampling convolutions, and then merges it with C3 through element-wise addition to generate F t And serve as input for the next layer of processing.

[0072] This process continues until the third layer is processed, producing the output feature map F. Next, a 1×1 convolution operation is applied to the feature map F to increase its channel number to 256, thereby generating the final output feature map S t The process can be described by the following expression, where Conv i Represents a convolution operation with a convolution kernel size of i:

[0073] S t =Conv1(Conv3(Conv3(C2)+C3)+C4)

[0074] is {f t-N ,…,f t-1 ,f t+1 …,f t+N}Data feature extraction and f t Exactly the same, extract the features from these adjacent context frames into S context The following is announced:

[0075]

[0076] Step S3, based on motion estimation, aligning the features of the context frame with the spatial features of the current frame, combining the aligned features with the features of the current frame through an aggregation function to form an intermediate feature map, and convolving the intermediate feature map using a convolution kernel to generate a convolution feature map.

[0077] In this embodiment, a feature fusion module based on time alignment uses a motion estimation function to align the features of the context frame with the features of the current frame. The aligned features are combined with the features of the current frame through an aggregation function to form a rich feature map, and then a 3x3 convolution kernel is used for convolution processing to integrate and refine the connected features to finally generate a feature map.

[0078] S3.1: First, it is necessary to effectively quantify and align the dynamic changes between consecutive frames in the video sequence, which requires the use of the motion estimation function M. Motion estimation functions play a core role in video analysis and processing, especially in tasks such as target tracking and recognition. In this method, the implementation of the motion estimation function M is based on the FlowNetSimple network, which is a classic and efficient deep learning model specifically designed to estimate and describe motion changes between video frames. By receiving a pair of consecutive frames as input, FlowNetSimple can output a detailed motion vector field, also known as an optical flow map. In this optical flow map, each vector not only represents the direction of movement of the corresponding pixel or feature point from one frame to another, but also accurately indicates the distance it has moved. This highly detailed motion information is crucial to understanding the dynamics of the scene. By applying the motion estimation function M, we can transform the context frame S context The feature map of the current frame is S t Perform precise alignment. The feature map after alignment is S aligned :

[0079] S aligned =M(S context ,S t )

[0080] This alignment operation takes into account the motion of each pixel, ensuring that the features of the context frame can accurately match the spatial position of the current frame even in dynamically changing scenes. Such alignment maintains a high degree of consistency not only in the spatial dimension (i.e., the width and height of the image) but also in the channel dimension (i.e., the depth of the feature), thus ensuring seamless connection between feature maps.

[0081] Furthermore, by aligning the feature map of the context frame with the feature map of the current frame, we are able to capture the continuity and variability of the scene at different time points, thus providing a solid foundation for feature aggregation. This temporal alignment not only enhances the model’s understanding and representation of the scene at the current moment, but also significantly improves the model’s detection and recognition capabilities in dynamic and complex environments by integrating historical and future information. This approach ensures that no matter how the scene changes, the model is able to capture the most relevant and accurate feature information, thereby achieving more stable and accurate target detection and tracking in a continuously changing environment.

[0082] S3.2: Aligned features are combined with S through aggregation function A t Aggregate to form a rich feature map G that captures both spatial and temporal characteristics:

[0083] G=A(S aligned ,S t ).

[0084] In our approach, the aggregation function A combines S using concatenation and convolution operations. aligned and S t This operation is achieved through the connection step, which is formulated as:

[0085] S stacked =Concat(S aligned ,S t ),

[0086] This operation will S aligned and S t The number of channels is merged, which effectively increases the dimension of the input features and provides a rich source of information for the subsequent convolution processing. Then, a 3×3 kernel convolution layer is used to process the stacked feature map S stacked The convolution step aims to deeply integrate and refine features from different time points while maintaining the original spatial dimension. This process can be expressed as:

[0087] G=Conv 3×3 (S stacked ),

[0088] Where G is the output feature map of dimension H×W×C, maintaining the original spatial dimension while encapsulating a rich representation of spatial and temporal information.

[0089] Through this series of operations, we obtain the output feature map G, which not only retains the spatial size H×W×C of the input features, but also integrates the rich spatial and temporal information obtained from different time points. Such feature maps provide a strong data foundation for downstream analysis tasks, especially in dynamically changing environments, which can significantly enhance the detection and adaptation capabilities of the model. Through this method, the model can capture and parse video content more accurately and achieve effective recognition and understanding of complex scenes.

[0090] Step S4, convolving the convolution feature map through dilated convolutions with different expansion rates, convolving the output feature maps corresponding to different dilated convolutions, so that the output feature map matches the convolution feature map, and adding the output feature maps of different channels to the convolution feature map pixels, and then adding the addition results of different channels pixel by pixel to obtain the final feature map.

[0091] In this embodiment, a multi-branch based recognition module is used to map receptive fields of different sizes on the feature map through hole convolutions with different dilation rates to solve the problem of large differences in target size in different images. Specifically, the operation of the module begins by copying the input feature map into three independent branches. The three branches will be processed in parallel, but each branch uses 3×3 hole convolutions with different dilation rates, corresponding to dilation rates of 3, 2, and 1, respectively. The purpose of this design is to enable each branch to perceive context areas of different sizes under the same size convolution window, where convolutions with larger dilation rates can cover a wider area and are suitable for the recognition of larger targets; while convolutions with smaller dilation rates are suitable for the recognition of smaller targets. After the respective hole convolutions are processed, the output feature map of each branch will be processed by a 1×1 convolution layer. This step is mainly to adjust the number of channels to match the original feature map, and it can also play a role in dimensionality reduction and feature fusion.

[0092] Furthermore, these processed feature maps will be added to the original feature maps at the pixel level. This step helps to retain the original features while incorporating contextual information of different scales.

[0093] To further enhance the generalization ability of the model, a weight sharing mechanism can be used between the three parallel branches. This means that although each branch processes dilated convolutions with different expansion rates, they use the same set of weights in the convolution layer. This design reduces the total number of parameters in the model and reduces the risk of overfitting, while also ensuring that the model can learn consistent feature representations on objects of different scales.

[0094] Finally, in order to obtain the final output that integrates all scale information, the output feature maps of the three branches will be added pixel by pixel to obtain the final feature map that integrates multi-scale target features.

[0095] Step S5: based on the final feature map, target detection is performed on the drone in the video sequence.

[0096] This multi-scale feature fusion method enables the model to have better recognition capabilities for objects of different sizes, greatly improving the adaptability and accuracy of the model in complex scenes. In this way, our model can effectively handle the multi-scale object problem in the image and improve the performance of object detection and recognition.

[0097] Compared with the prior art, this embodiment has at least the following effects:

[0098] 1) The present invention can be used for processing and analyzing visible light images. By effectively extracting image features, the accuracy and stability of target detection are enhanced, and it is particularly suitable for complex environmental conditions. Compared with traditional single video processing methods, the visible light image processing strategy of the present invention can more effectively combine the accuracy and speed of single-frame object detection into the video object detection framework, significantly improving recognition efficiency and the reliability of results. In addition, by integrating time alignment and aggregation technology, misjudgment and repeated tracking are reduced, and system performance is optimized. For drone systems that perform complex monitoring, search and rescue missions, this not only reduces the workload, but also improves the efficiency and accuracy of mission execution.

[0099] 2) The present invention also has an adaptive mechanism, and the model can automatically adjust the detection strategy according to the changes in real-time video content, thereby enhancing the application flexibility and efficiency of the model in different environments and conditions.

[0100] The second embodiment of the present invention corresponds to the first embodiment, referring to Figure 4 ,This embodiment introduces an adaptive UAV target detection device with flow-guided feature aggregation, including the following components:

[0101] A preprocessing unit is configured to obtain a visible light video sequence of the drone and process the video sequence into a single frame image;

[0102] A feature extraction unit configured to extract spatial features from a current frame and adjacent context frames in the single frame image using multi-layer downsampling convolution;

[0103] A feature alignment unit is configured to perform feature alignment on the features of the context frame and the spatial features of the current frame based on motion estimation, combine the aligned features with the features of the current frame through an aggregation function to form an intermediate feature map, and perform convolution processing on the intermediate feature map using a convolution kernel to generate a convolution feature map;

[0104] An output unit is configured to perform convolution processing on the convolution feature map through hole convolutions with different expansion rates, and the output feature maps corresponding to different hole convolutions are convolved to match the output feature map with the convolution feature map, and add the output feature map of different channels to the convolution feature map pixels, and then add the addition results of different channels pixel by pixel to obtain a final feature map;

[0105] The detection unit is configured to detect the target of the drone in the video sequence based on the final feature map.

[0106] In one embodiment, the feature extraction unit is further configured to:

[0107] Extract spatial features of the frame images within the current frame and the adjacent N frames;

[0108] Pass the current spatial features through at least two convolutional layers in sequence, where each convolutional layer is used to gradually halve the size of the feature map and double the number of channels to generate a corresponding set of feature maps;

[0109] The feature map representing the second layer in the feature map set is used as input, and is combined with the feature map representing the third layer by element-by-element addition through downsampling convolution processing, and is used as the input of the next layer, and the processing of the upper layer is repeated until the last layer, so as to generate a feature map F;

[0110] The feature map F is convolved to generate processed spatial features.

[0111] In one embodiment, the feature alignment unit is further configured to:

[0112] Obtaining spatial features of a pair of consecutive frames as input to determine a context frame feature map;

[0113] Aggregating the context frame feature map with the current frame feature map by applying a motion estimation function to obtain an aligned context frame feature map;

[0114] Aggregating the currently aligned context feature map with the current frame feature map through an aggregation function to form the intermediate feature map;

[0115] The intermediate feature map is convolved using a 3×3 convolution kernel to generate a convolution feature map.

[0116] A third embodiment of the present invention is an electronic device, which can be understood as a physical device, including a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the following operations are performed:

[0117] Step S1, obtaining a visible light video sequence of a drone, and processing the video sequence into a single frame image;

[0118] Step S2, extracting spatial features from the current frame and adjacent context frames in the single frame image using multi-layer downsampling convolution;

[0119] Step S3, aligning the features of the context frame with the spatial features of the current frame based on motion estimation, combining the aligned features with the features of the current frame through an aggregation function to form an intermediate feature map, and performing convolution processing on the intermediate feature map using a convolution kernel to generate a convolution feature map;

[0120] Step S4, performing convolution processing on the convolution feature map through hole convolutions with different expansion rates, and performing convolution processing on the output feature maps corresponding to different hole convolutions, so that the output feature map matches the convolution feature map, and adding the output feature map of different channels to the convolution feature map pixels, and then performing pixel addition on the addition results of different channels to obtain the final feature map;

[0121] Step S5: Detect the target of the UAV in the video sequence based on the final feature map.

[0122] The fourth embodiment of the present invention, the process of the adaptive UAV target detection method of flow-guided feature aggregation in this embodiment is the same as that of the first, second or third embodiment, the difference is that in engineering implementation, this embodiment can be implemented by means of software plus a necessary general hardware platform, of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the method of the present invention can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), including a number of instructions for a device to execute the method described in the embodiment of the present invention.

[0123] Through the description of the specific implementation methods, a deeper and more specific understanding of the technical means and effects adopted by the present invention to achieve the predetermined purpose should be obtained. However, the accompanying drawings are only for reference and illustration purposes and are not intended to limit the present invention.

Claims

1. An adaptive UAV target detection method based on flow-guided feature aggregation, characterized in that: include: Step S1, obtaining a visible light video sequence of a drone, and processing the video sequence into a single frame image; Step S2, extracting spatial features from the current frame and adjacent context frames in the single frame image using multi-layer downsampling convolution; Step S3, aligning the features of the context frame with the spatial features of the current frame based on motion estimation, combining the aligned features with the features of the current frame through an aggregation function to form an intermediate feature map, and performing convolution processing on the intermediate feature map using a convolution kernel to generate a convolution feature map; Step S4, performing convolution processing on the convolution feature map through hole convolutions with different expansion rates, and performing convolution processing on the output feature maps corresponding to different hole convolutions, so that the output feature map matches the convolution feature map, and adding the output feature map of different channels to the convolution feature map pixels, and then performing pixel addition on the addition results of different channels to obtain the final feature map; Step S5: Detect the target of the UAV in the video sequence based on the final feature map.

2. The adaptive UAV target detection method based on flow-guided feature aggregation according to claim 1 is characterized in that: The step S2 comprises: Extract spatial features of the frame images within the current frame and the adjacent N frames; Pass the current spatial features through at least two convolutional layers in sequence, where each convolutional layer is used to gradually halve the size of the feature map and double the number of channels to generate a corresponding set of feature maps; The feature map representing the second layer in the feature map set is used as input, and is combined with the feature map representing the third layer by element-by-element addition through downsampling convolution processing, and is used as the input of the next layer, and the processing of the upper layer is repeated until the last layer, so as to generate a feature map F; The feature map F is convolved to generate processed spatial features.

3. The adaptive UAV target detection method based on flow-guided feature aggregation according to claim 2 is characterized in that: The step S3 comprises: Obtaining spatial features of a pair of consecutive frames as input to determine a context frame feature map; Aggregating the context frame feature map with the current frame feature map by applying a motion estimation function to obtain an aligned context frame feature map; Aggregating the currently aligned context feature map with the current frame feature map through an aggregation function to form the intermediate feature map; The intermediate feature map is convolved using a 3×3 convolution kernel to generate a convolution feature map.

4. An adaptive UAV target detection device with flow-guided feature aggregation, characterized in that: include: A preprocessing unit is configured to obtain a visible light video sequence of the drone and process the video sequence into a single frame image; A feature extraction unit configured to extract spatial features from a current frame and adjacent context frames in the single frame image using multi-layer downsampling convolution; A feature alignment unit is configured to perform feature alignment on the features of the context frame and the spatial features of the current frame based on motion estimation, combine the aligned features with the features of the current frame through an aggregation function to form an intermediate feature map, and perform convolution processing on the intermediate feature map using a convolution kernel to generate a convolution feature map; An output unit is configured to perform convolution processing on the convolution feature map through hole convolutions with different expansion rates, and the output feature maps corresponding to different hole convolutions are convolved to match the output feature map with the convolution feature map, and add the output feature map of different channels to the convolution feature map pixels, and then add the addition results of different channels pixel by pixel to obtain a final feature map; The detection unit is configured to detect the target of the drone in the video sequence based on the final feature map.

5. The adaptive UAV target detection device based on flow-guided feature aggregation according to claim 4 is characterized in that: The feature extraction unit is further configured as follows: Extract spatial features of the frame images within the current frame and the adjacent N frames; Pass the current spatial features through at least two convolutional layers in sequence, where each convolutional layer is used to gradually halve the size of the feature map and double the number of channels to generate a corresponding set of feature maps; The feature map representing the second layer in the feature map set is used as input, and is combined with the feature map representing the third layer by element-by-element addition through downsampling convolution processing, and is used as the input of the next layer, and the processing of the upper layer is repeated until the last layer, so as to generate a feature map F; The feature map F is convolved to generate processed spatial features.

6. The adaptive UAV target detection device based on flow-guided feature aggregation according to claim 5 is characterized in that: The feature alignment unit is further configured as: Obtaining spatial features of a pair of consecutive frames as input to determine a context frame feature map; Aggregating the context frame feature map with the current frame feature map by applying a motion estimation function to obtain an aligned context frame feature map; Aggregating the currently aligned context feature map with the current frame feature map through an aggregation function to form the intermediate feature map; The intermediate feature map is convolved using a 3×3 convolution kernel to generate a convolution feature map.

7. An electronic device, characterized in that: The electronic device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the adaptive UAV target detection method with flow-guided feature aggregation as described in any one of claims 1 to 3.

8. A computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the adaptive UAV target detection method with flow-guided feature aggregation as described in any one of claims 1 to 3 are implemented.

Citation Information

Patent Citations

  • Method and system for detecting image target of unmanned aerial vehicle

    CN109389086A

  • Optical flow multilayer frame feature propagation and aggregation method for video target detection

    CN109993096A

  • Target detection method, storage medium and computer equipment

    CN110751134A

  • Video target detection method and device, equipment and medium

    CN115861891A

  • Adaptive scene monitoring video target detection method and system based on feature fusion

    CN117237867A