A method for unmanned aerial vehicle target detection based on historical trajectory generative attention

Abstract

The application provides a UAV target detection method based on historical trajectory generative attention, which comprises the following steps: constructing a historical attention mapping diagram according to historical trajectory information of a UAV target, elements in the historical attention mapping diagram correspond to pixels of a to-be-detected image one by one, and the element values in the mapping diagram represent the context attention degree of the pixels; constructing a UAV target detection model based on historical trajectory generative attention; training the constructed UAV target detection model based on historical trajectory generative attention; and performing target detection by using the trained improved UAV target detection model based on historical trajectory generative attention, thereby completing the UAV target detection based on historical trajectory generative attention. By combining historical trajectory information and an attention mechanism, the application improves the accuracy of small target UAV detection and ensures the real-time performance of the detection process.

Description

Technical Field

[0001] This invention relates to a method for detecting unmanned aerial vehicle (UAV) targets, and more particularly to a method for detecting UAV targets based on historical trajectory generative attention. Background Technology

[0002] With the rapid development and widespread application of drone technology, the demand for counter-drone technology has also emerged, such as the detection of unauthorized and indiscriminate drone flights in the civilian sector. Drone detection is the first and most important step in counter-drone technology. Currently, drone detection technologies can be divided into various forms, including radar detection, audio detection, radio detection, and visual detection. Due to the small reflective area of ​​some plastic drones, and the presence of significant ground clutter in urban areas and airports, radar is difficult to use effectively to detect drones. Audio detection technology is greatly affected by environmental noise, making it difficult to detect drones in noisy environments such as cities. Radio detection technology requires high sensitivity from both transmitting and receiving equipment, and drones in electromagnetically silent states are difficult to detect. Visual detection technology relies on inexpensive and readily implementable optical sensors, and offers advantages such as high accuracy, fast detection speed, and a large monitoring range. These characteristics make it one of the most important detection technologies in counter-drone equipment.

[0003] Visual detection methods can identify desired objects from images, but they pose a challenge for detecting small targets, such as drones at a considerable distance. Small targets occupy only a few pixels in an image, typically exhibiting low contrast, limited visual features, and are easily obscured by complex backgrounds or other objects. Furthermore, drones may exhibit different flight attitudes and high speeds during flight, resulting in varying shapes and postures in the image, further complicating the detection of small drone targets.

[0004] Deep learning methods, with their excellent multi-scale feature extraction capabilities and strong generalization ability, have become an important technique for object detection. By constructing targeted deep learning network models, features of small objects can be effectively extracted from images, improving the accuracy of object detection. However, the complex network structure has too many network parameters, resulting in slow network speed and failing to meet the requirements of real-time detection. Summary of the Invention

[0005] Purpose of the invention: The technical problem to be solved by the present invention is to provide a drone target detection method based on historical trajectory generative attention, which addresses the shortcomings of the existing technology.

[0006] To address the aforementioned technical problems, this invention discloses a UAV target detection method based on historical trajectory generative attention, comprising the following steps:

[0007] Step 1: Construct a historical attention map based on the historical trajectory information of the UAV target. The elements in the historical attention map correspond one-to-one with the pixels of the image to be detected, and the element values ​​in the map represent the contextual attention level of the pixel.

[0008] Step 2: Design a generative attention module. Its input is the historical attention map constructed in Step 1, i.e., the historical trajectory information of the UAV target; the output is the attention prediction map of the current image to be detected, i.e., trajectory attention.

[0009] Step 3: Design a trajectory attention fusion module to predict the region in the image to be detected where the UAV may appear using the historical trajectory information of the UAV target, and define it as the region of interest;

[0010] Step 4: Optimize the feature extraction part of the YOLOv8 deep learning model to obtain an improved feature extraction network. Design an improved total loss function for the improved feature extraction network.

[0011] Step 5: Construct a drone target detection model based on historical trajectory generative attention;

[0012] Step 6: Train the UAV target detection model based on historical trajectory generative attention constructed in Step 5;

[0013] Step 7: Use the improved UAV target detection model based on historical trajectory generative attention trained in Step 6 to perform target detection, thus completing the UAV target detection based on historical trajectory generative attention.

[0014] Furthermore, the construction of the historical attention map described in step 1 specifically includes the following steps:

[0015] Step 1-1: Given an image to be detected with a width of m pixels and a length of n pixels, construct a matrix of size m×n. Assign initial values ​​to each element in matrix C0;

[0016] Steps 1-2: Let the set of drones in the historical trajectory information of the drone target be D = {1,2,...,d}. Calculate the position region matrix L of drone d (size w×h pixels) at the p-th trajectory point from the current position. d,p The size of the location region matrix is ​​equal to the size of the image to be detected, and the element with coordinates (x, y) in the location region matrix takes the value a. i,j The following methods are used:

[0017]

[0018] Steps 1-3 involve summing the location region matrices of all UAV targets with the p-th trajectory point; the result is the historical trajectory set and the historical attention mapping map C. p Let p be the decay sum of up to p sets of historical trajectories, i.e.:

[0019]

[0020] Where μ is the attenuation coefficient.

[0021] Furthermore, the design of the generative attention module described in step 2 specifically includes:

[0022] The generative attention module consists of a feature context modeling layer and a trajectory attention layer; the input to this module is the historical trajectory information of the UAV target.

[0023] The context attention layer is processed by a 1×1 convolutional layer and the Softmax activation function. The result is multiplied by itself before the 1×1 convolution and used to model the features of the mapping map in context.

[0024] The trajectory attention layer sequentially performs 1×1 convolution, normalization, ReLU activation, and 1×1 convolution on the output of the context attention layer to capture the feature dependencies in the mapping graph.

[0025] Finally, the above results are processed using the Sigmoid function to make them non-linear outputs, which is the output of the generative attention module: trajectory attention.

[0026] Furthermore, the design trajectory focus fusion module mentioned in step 3 specifically includes:

[0027] Step 3-1: The image to be detected obtains three levels of image features at different scales through a multi-scale feature extraction network. The first-level image features and the trajectory attention output in step 2 are used as inputs to this module.

[0028] Step 3-2: Perform linear mapping and Tanh activation function processing on the first-level image features of the image to be detected to obtain the first processing result. Perform a click operation on the image features and trajectory attention, and perform linear mapping on the dot product operation result to obtain the second processing result. Add and merge the two processing results.

[0029] Step 3-3 divides the result of the addition and merging into two branches. The first branch does not undergo any additional processing, while the second branch is processed using a linear mapping layer and the Softmax activation function.

[0030] Steps 3-4 involve multiplying the first branch and the processed second branch element by element to form the image features of the additional region of interest.

[0031] Furthermore, the improved total loss function described in step 4 specifically includes:

[0032] The total loss function includes: improved cross-entropy loss L vfl Cross-combination loss L IoU and attention bias loss L att Among them, the improved cross-entropy loss L vfl as follows:

[0033] L vfl = -r(rlog(s)+(1-r)log(1-s))

[0034] Where r is the intersection-union ratio of the predicted box and the ground truth box, and s is the local perception classification score. When the classification result is the same as the ground truth result, the IoU value between the predicted box and the ground truth box is taken, and the value at other positions is 0.

[0035] Crossover and union ratio loss L IoU for:

[0036]

[0037] Among them, b and b gt ρ represents the center point of the predicted bounding box and the actual bounding box of the target, respectively; ρ represents the Euclidean distance between the two rectangles; c represents the diagonal distance of the closed regions of the two rectangles; v represents the consistency of the relative proportions of the two rectangles; and α is the weighting coefficient.

[0038] The improved total loss function L is:

[0039] L = L vfl +L IoU +L att .

[0040] Furthermore, the attention bias loss mentioned in step 4 specifically includes:

[0041] The attention bias loss is used to represent the deviation between the trajectory attention generated by the generative attention module described in step 2 and the actual location of the UAV. The calculation of the attention bias loss consists of the trajectory attention F generated in step 2, the region location matrix, and L0. The expression for the trajectory attention F is:

[0042] F=conv(ReLu(norm(conv((1+softmax(conv(C p )))C p ))))

[0043] The expressions for the region location matrix and L0 are:

[0044]

[0045] Where d is the drone target number in the detection result, L d,0 The region location matrix representing the UAV target labeled d is generated from the detection labels corresponding to the images in the training dataset. The detection labels include the position of the target in the corresponding image.

[0046] Pay attention to the deviation loss L att The calculation method is as follows:

[0047]

[0048] Where m is the number of horizontal pixels in the image to be detected, n is the number of vertical pixels in the image to be detected, F(i,j) represents the trajectory attention of the pixel with pixel coordinates (i,j), and L0(i,j) represents the region position matrix of the pixel with pixel coordinates (i,j).

[0049] Furthermore, the optimization of the feature extraction part of the YOLOv8 deep learning model described in step 4 involves the following improvements to the original feature extraction network in the YOLOv8 deep learning model:

[0050] Step 4-1: Replace the C2f convolutional feature extraction module in the original feature extraction network with the redesigned C2p convolutional feature extraction module;

[0051] Step 4-2: Adjust the structural branches in the original feature extraction network, parallelize the cascaded feature extraction operations, reduce the number of convolutional blocks, and optimize repetitive modules to remove feature redundancy and reduce the number of parameters in the feature extraction network.

[0052] Furthermore, the redesigned C2p convolutional feature extraction module described in step 4-1 specifically includes:

[0053] Step 4-1-1: In the C2p convolutional feature extraction module, the input features are processed by the convolution module and then split into n half features. The number of channels in each half feature is half that of the original image features.

[0054] Step 4-1-2: Extract the backbone for each half-feature in parallel using n Darknet features, meaning that each feature extraction backbone is independent of the others.

[0055] Step 4-1-3: For the n semi-features processed by the feature extraction backbone, concatenate them with the image features before splitting.

[0056] Furthermore, the improved feature extraction network described in step 5 is as follows:

[0057] The input image features are refined through a convolutional module to restore the number of feature channels to match the number of feature channels in the input image. After passing through two convolutional modules, the input image is processed by a C2p convolutional feature extraction module to generate the first-level derived features. The first-level features are further divided into two branches for feature extraction, with the processing method being the same in both branches. The first-level features are first processed by the convolutional module, then processed by the C2p convolutional feature extraction module, and the result is passed through the convolutional module again, followed by the C2p convolutional feature extraction module. This completes the branch feature processing. After obtaining the feature results from the two branches, the two features are concatenated, and the concatenated features are used as the second-level derived features. The second-level features are passed through a convolutional module and a C2p convolutional feature extraction module, and then through a fast spatial pyramid pooling layer to generate the third-level derived features.

[0058] Furthermore, the UAV target detection model based on historical trajectory generative attention described in step 6 specifically includes:

[0059] The image to be detected is sequentially passed through two 3x3 convolutional modules and a C2p convolutional feature extraction module. The historical trajectory of the UAV, which has passed through the attention generation module, is then input into the trajectory attention fusion module. Finally, the image is passed through an improved feature extraction network to obtain the final target detection result.

[0060] Beneficial effects:

[0061] This invention improves the accuracy of small target UAV detection and ensures the real-time nature of the detection process by combining historical trajectory information and attention mechanisms. Attached Figure Description

[0062] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.

[0063] Figure 1 This is a diagram of the overall architecture for unmanned aerial vehicle (UAV) target detection according to the present invention.

[0064] Figure 2 This is a structural diagram of the generative attention module.

[0065] Figure 3 The structure diagram of the trajectory focus fusion module.

[0066] Figure 4 The structure diagram of the improved feature extraction network.

[0067] Figure 5 This is a block structure diagram for C2p feature extraction. Detailed Implementation

[0068] To improve the accuracy of small-target UAV detection and ensure real-time performance, this invention provides a small-target UAV detection method based on historical trajectory generative attention. In this method, the historical trajectory information of the UAV target is calculated as a historical attention map. Elements in the historical attention map correspond one-to-one with pixels in the detected image, and the element value in the map represents the contextual attention level of that pixel. The historical attention map is input into the generative attention module, and its output is the predicted current image attention map, i.e., generative attention for the current image. This generative attention is used to guide the neural network's attention to the areas where the UAV may appear. An attention loss function is proposed and used as part of the total loss function for neural network training. Its function is to backpropagate the attention region bias in the network model, updating the network parameters to better predict the areas of interest in the image. To improve detection speed and ensure real-time performance while maintaining detection quality, this invention adjusts and optimizes the network structure based on the YOLOv8 deep learning model. The overall model structure diagram of this invention is attached. Figure 1 As shown.

[0069] The UAV target detection method based on historical trajectory generative attention of the present invention is mainly implemented by the following steps.

[0070] Step 1: Generation of Historical Attention Map

[0071] The historical trajectory information of a single UAV target can be described as a (x, y, w, h) quadruple sequence, i.e., (center point x-axis coordinate, center point y-axis coordinate, target width, target height), with a sequence length less than a predetermined maximum trajectory threshold. The historical trajectory information of multiple UAV targets, however, can be described as a sequence of the above above, with variable sequence lengths. To enable the effective use of UAV trajectory information in visual inspection tasks, it is necessary to transform the UAV trajectory information to match it with network processing elements, i.e., the historical attention mapping proposed in this invention.

[0072] Given an image with a width of m and a length of n, construct an m×n matrix C0 and assign initial values ​​to each element in the matrix.

[0073] Let D = {1,2,...,d} be the set of drones detected in historical trajectories. Calculate the position region matrix L of drone d at the p-th trajectory point from its current position. d,p The matrix size is equal to the image size, and the element value in the location region matrix is ​​a. i,j It conforms to the following formula:

[0074]

[0075] The historical trajectory set is obtained by summing the location region matrices of all UAVs with the p-th trajectory point, and the historical attention mapping map C is also shown. p Let the decay sum be the sum of up to p sets of trajectories, i.e.:

[0076]

[0077] Where μ is the attenuation coefficient.

[0078] Step 2: Generative attention module design, such as... Figure 2 As shown:

[0079] The historical attention map generated in step 1 can be a single-channel image of the degree of attention to the target location. The generative attention module takes the historical attention map as input and outputs trajectory attention results. The network in the module is trained simultaneously during model training to generate appropriate trajectory attention.

[0080] The historical attention map first generates map features through convolutional layers, ensuring the feature size matches the scale of the first feature in the detection image, a size that remains unchanged in subsequent processing. Then, the map features are processed through a 1×1 convolutional layer using the Softmax activation function. The result is multiplied by the original feature before the 1×1 convolution; this operation aims to model the context of the map features. The result is then normalized after a 1×1 convolution, and then re-convolved with a 1×1 convolution. This sequence of 1×1 convolution, normalization, ReLU activation, and 1×1 convolution aims to capture feature dependencies within the map. The result is then processed using the Sigmoid function to add non-linearity. The final result is the generated trajectory attention.

[0081] Step 3 focuses on the design of the trajectory fusion module, such as... Figure 3 As shown:

[0082] Trajectory attention fusion applies trajectory attention to the extracted features of the image to be detected. This process is designed after the first-level derived features of the image to be detected have been extracted.

[0083] The input to this module is the first-level derived features of the image to be detected and the generated trajectory attention, and the output is the first-level derived features after applying the attention.

[0084] In this module, the first-level derived features of the image to be detected are dot-producted with trajectory attention, and the dot-product result is used for feature mapping through a linear layer. In the obtained mapped features, the drone's historical trajectory information is used to predict the regions where the drone might appear in the current frame, and more attention is given to the predicted regions. However, focusing only on objects within the predicted regions may cause the detection network to overlook newly appearing drones in the image; therefore, attention reconciliation is needed between the original image features and the additional region-focused image features.

[0085] The trajectory attention fusion module employs a lightweight attention reconciliation method. Linear feature mapping is applied to the original image features, followed by Tanh activation; while image features for additional region attention are directly processed using Tanh activation. The results from both methods are summed and merged, resulting in two branches: one branch performs no additional processing, while the other branch uses a linear mapping layer and Softmax activation. The two results are then multiplied element-wise, and the final output of this module is used. This process matches and aligns the features from the two sources, ensuring a one-to-one correspondence between elements, and then merges the corresponding elements.

[0086] Step 4: Improve the design of the feature extraction network

[0087] The improved feature extraction network reduces the number of parameters, enabling the detection model to perform in real-time in more common scenarios while maintaining reliable feature extraction capabilities. The network structure has been optimized for small object detection scenarios. The input image is processed by two convolutional modules followed by a C2p module to generate the first-level derived features. These first-level features are then further divided into two branches for feature extraction, with identical processing methods in both branches. The first-level features are first processed by a convolutional module, then by a C2p module, and the result is again processed by a convolutional module, followed by another C2p module. This completes the branch feature processing. The feature results from both branches are then concatenated, and the concatenated features serve as the second-level derived features. The second-level features are then processed by a convolutional module and a C2p module, followed by a fast spatial pyramid pooling layer to generate the third-level derived features. This supports the feature extraction network in extracting image features. The improved feature extraction network structure is as follows: Figure 4 As shown in the figure. Among them, the C2p module is an improved feature extraction module, and its structural design is described below.

[0088] like Figure 5As shown, in the C2p module, the input features are processed by a convolutional module and then split into n half-features. Each feature contains half the number of channels of the original image features. For each image feature, a backbone is extracted in parallel using n Darknet feature extraction tools, meaning each feature extraction backbone is independent of the others. For the n features processed by the feature extraction backbone, these n adjusted half-features are concatenated with the original image features. The concatenated result contains features with a dimension that is 1 + 0.5n times that of the original image features. This feature is then refined using a convolutional module to restore its feature channel count to the same number as the input feature channels.

[0089] Step 5 focuses on the calculation of deviation loss.

[0090] Attention bias loss represents the deviation between the trajectory prediction attention result generated by the generative attention module in the network and the actual location of the UAV. The calculation of attention bias loss consists of the generated trajectory prediction attention F, the region location matrix generated by the current image detection label, and L0.

[0091] The expression for trajectory attention F is:

[0092] F=conv(ReLu(norm(conv((1+softmax(conv(C p )))C p ))))

[0093] The expressions for the region location matrix and L0 are:

[0094]

[0095] Where d is the drone target number in the detection result.

[0096] The formula for calculating the deviation loss is as follows:

[0097]

[0098] Where m is the number of horizontal pixels in the image, and n is the number of vertical pixels in the image.

[0099] Step 6: Design of the total loss function

[0100] In deep learning, loss functions are used to guide the model training process. By changing the weight parameters in the network to minimize the loss function, the model's parameters can be adjusted to better fit the training data, thereby improving the model's performance.

[0101] The loss function of this method consists of three parts: improved cross-entropy loss (VFL Loss), cross-union ratio loss (IoU), and attention bias loss.

[0102] The improved cross-entropy loss is defined as:

[0103] L vfl = -r(rlog(s)+(1-r)log(1-s))

[0104] Where r is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box, which is the intersection of the predicted bounding box and the ground truth bounding box divided by the union of the two boxes, and s is the local perception classification score. When the classification result is the same as the ground truth result, the IoU value between the predicted bounding box and the ground truth bounding box is taken, and the value at other positions is 0.

[0105] Crossover ratio loss is defined as:

[0106]

[0107] Among them, b and b gt ρ represents the center point of the predicted bounding box and the actual bounding box of the target, respectively. ρ represents the Euclidean distance between the two rectangles, c represents the diagonal distance of the closed regions of the two rectangles, v represents the consistency of the relative proportions of the two rectangles, and α is the weighting coefficient.

[0108] The total loss function is then expressed as:

[0109] L = L vfl +L IoU +L att

[0110] Example:

[0111] This invention provides a drone target detection method based on historical trajectory generative attention, which improves the accuracy of small target drone detection and ensures the real-time nature of the detection process.

[0112] The embodiments of the UAV target detection method based on historical trajectory generative attention of the present invention are mainly implemented by the following steps.

[0113] Step 1: Generation of Historical Attention Map

[0114] The historical trajectory information of a single UAV target can be described as a (x, y, w, h) quadruple sequence, i.e., (center point x-axis coordinate, center point y-axis coordinate, target width, target height), with a sequence length less than a predetermined maximum trajectory threshold. Specifically, the maximum historical trajectory sequence length threshold recorded by a single UAV is set to 5. The historical trajectory information of multiple UAV targets can be described as a sequence of the above-mentioned sequences, with variable sequence lengths. To enable the UAV trajectory information to be effectively utilized in visual inspection tasks, it needs to be transformed to match network processing elements, i.e., the historical attention mapping proposed in this invention.

[0115] Given an image with a width of m and a length of n, construct an m×n matrix C0 and assign initial values ​​to each element in the matrix.

[0116] Let D = {1,2,...,d} be the set of drones detected in historical trajectories. Calculate the position region matrix L of drone d at the p-th trajectory point from its current position. d,p The matrix size is equal to the image size, and the element value in the location region matrix is ​​a. i,j It conforms to the following formula:

[0117]

[0118] The historical trajectory set is obtained by summing the location region matrices of all UAVs with the p-th trajectory point, and the historical attention mapping map C is also shown. p Let the decay sum be the sum of up to p sets of trajectories, i.e.:

[0119]

[0120] Where μ is the attenuation coefficient, and the specific attenuation coefficient is 0.6.

[0121] Step 2: Generative Attention Module Design

[0122] The historical attention map generated in step 1 can be a single-channel image of the degree of attention to the target location. The generative attention module takes the historical attention map as input and outputs trajectory attention results. The network in the module is trained simultaneously during model training to generate appropriate trajectory attention.

[0123] The historical attention map first generates map features through convolutional layers, ensuring the feature size matches the scale of the first feature in the detection image, a size that remains unchanged in subsequent processing. Then, the map features are processed through a 1×1 convolutional layer using the Softmax activation function. The result is multiplied by the original feature before the 1×1 convolution; this operation aims to model the context of the map features. The result is then normalized after a 1×1 convolution, and then re-convolved with a 1×1 convolution. This sequence of 1×1 convolution, normalization, ReLU activation, and 1×1 convolution aims to capture feature dependencies within the map. The result is then processed using the Sigmoid function to add non-linearity. The final result is the generated trajectory attention.

[0124] A schematic diagram of the generative attention module is attached. Figure 2 As shown.

[0125] Step 3: Design of the Trajectory Focus Fusion Module

[0126] Trajectory attention fusion applies trajectory attention to the extracted features of the image to be detected. This process is designed after the first-level derived features of the image to be detected have been extracted.

[0127] The input to this module is the first-level derived features of the image to be detected and the generated trajectory attention, and the output is the first-level derived features after applying the attention.

[0128] In this module, the first-level derived features of the image to be detected are dot-producted with trajectory attention, and the dot-product result is used for feature mapping through a linear layer. In the obtained mapped features, the drone's historical trajectory information is used to predict the regions where the drone might appear in the current frame, and more attention is given to the predicted regions. However, focusing only on objects within the predicted regions may cause the detection network to overlook newly appearing drones in the image; therefore, attention reconciliation is needed between the original image features and the additional region-focused image features.

[0129] The trajectory attention fusion module employs a lightweight attention reconciliation method. Linear feature mapping is applied to the original image features, followed by Tanh activation; while image features for additional region attention are directly processed using Tanh activation. The results from both methods are summed and merged, resulting in two branches: one branch performs no additional processing, while the other branch uses a linear mapping layer and Softmax activation. The two results are then multiplied element-wise, and the final output of this module is used. This process matches and aligns the features from the two sources, ensuring a one-to-one correspondence between elements, and then merges the corresponding elements.

[0130] The structure diagram of the trajectory attention fusion module is attached. Figure 3 As shown.

[0131] Step 4: Improve the design of the feature extraction network

[0132] The improved feature extraction network reduces the number of parameters, enabling the detection model to perform in real-time in more common scenarios while maintaining reliable feature extraction capabilities. The network structure has been optimized for small object detection scenarios. The input image is processed by two convolutional modules followed by a C2p module to generate the first-level derived features. These first-level features are then further divided into two branches for feature extraction, with identical processing methods in both branches. The first-level features are first processed by a convolutional module, then by a C2p module, and the result is again processed by a convolutional module, followed by another C2p module. This completes the branch feature processing. The feature results from both branches are then concatenated, and the concatenated features serve as the second-level derived features. The second-level features are processed by a convolutional module and a C2p module, and then by a fast spatial pyramid pooling layer to generate the third-level derived features. This supports the feature extraction network in extracting image features. The improved feature extraction network structure is shown in the attached figure. Figure 4 As shown in the figure. Among them, the C2p module is an improved feature extraction module, and its structural design is described below.

[0133] In the C2p module, the input features are processed by a convolutional module and then split into n half-features. Each feature contains half the number of channels of the original image features. For each image feature, a backbone is extracted in parallel using n Darknet feature extraction tools, meaning each feature extraction backbone is independent of the others. For the n features resulting from the feature extraction backbone, these n adjusted half-features are concatenated with the original image features. The concatenated result contains a feature dimension that is 1 + 0.5n times that of the original image features. This feature is then refined using a convolutional module to restore its feature channel count to match the input feature channel count. The structure of the C2p module is shown in the attached figure. Figure 5 As shown.

[0134] Step 5 focuses on the calculation of deviation loss.

[0135] Attention bias loss represents the deviation between the trajectory prediction attention result generated by the generative attention module in the network and the actual location of the UAV. The calculation of attention bias loss consists of the generated trajectory prediction attention F, the region location matrix generated by the current image detection label, and L0.

[0136] The expression for trajectory attention F is:

[0137] F=conv(ReLu(norm(conv((1+softmax(conv(C p )))C p ))))

[0138] The expressions for the region location matrix and L0 are:

[0139]

[0140] Where d is the drone target number in the detection result.

[0141] The formula for calculating the deviation loss is as follows:

[0142]

[0143] Where m is the number of horizontal pixels in the image, and n is the number of vertical pixels in the image.

[0144] Step 6: Design of the total loss function

[0145] In deep learning, loss functions are used to guide the model training process. By changing the weight parameters in the network to minimize the loss function, the model's parameters can be adjusted to better fit the training data, thereby improving the model's performance.

[0146] The loss function of this method consists of three parts: improved cross-entropy loss (VFL Loss), cross-union ratio loss (IoU), and attention bias loss.

[0147] The improved cross-entropy loss is defined as:

[0148] L vfl = -r(rlog(s)+(1-r)log(1-s))

[0149] Where r is the intersection-union ratio (IoU) between the predicted bounding box and the ground truth bounding box, which is the intersection of the predicted bounding box and the ground truth bounding box divided by the union of the two boxes, and s is the local perception classification score. When the classification result is the same as the ground truth result, the IoU value between the predicted bounding box and the ground truth bounding box is taken, and the value at other positions is 0.

[0150] Crossover ratio loss is defined as:

[0151]

[0152] Among them, b and b gt ρ represents the center point of the predicted bounding box and the actual bounding box of the target, respectively. ρ represents the Euclidean distance between the two rectangles, c represents the diagonal distance of the closed regions of the two rectangles, c represents the consistency of the relative proportions of the two rectangles, and α is the weighting coefficient.

[0153] The total loss function is then expressed as:

[0154] L = L vfl +L Io x+L att

[0155] Step 7: Training the small target UAV detection model

[0156] To train a small-target UAV detection model, this embodiment selects several videos containing UAV targets and processes them. Specifically, each frame in the video is extracted as an image, and corresponding image annotation files are generated based on the video annotation file. The annotations of the previous 5 frames of the current image are aggregated, and the target box annotations are processed into the trajectory of the current image in the form of a list of four-tuples. The method for determining whether the annotation boxes in the historical annotations belong to the same trajectory point is the nearest Euclidean distance method. All the acquired videos containing UAV targets are processed using the above method, and the processing result forms a UAV target detection dataset containing historical trajectory annotations.

[0157] To complete the training of the drone detection model, the dataset was divided into a training set and a test set, with a specific ratio of 7:3. The training set was used to train the drone target detection model, while the test set was used to evaluate the performance of the drone target detection model.

[0158] This embodiment uses PyTorch as the deep learning framework, specifically version 1.11.0. Accordingly, Python 3.8 is used as the Python programming language interpreter to run the embodiment's code. The embodiment utilizes a GPU for hardware acceleration; the GPU used is an NVIDIA GeForce RTX 2080Ti.

[0159] The example model was trained without using a pre-trained model. The network model parameters were set to random values, the model training optimizer was Adam, and the initial learning rate was set to 0.0001. The number of training epochs was set to 200, using batch image input with 16 images per batch. The training results were saved in pt format.

[0160] Step 8: Input the image and historical trajectory of the small target UAV to be detected, and obtain the detection results.

[0161] To obtain the detection results for the image to be detected, the image to be detected and the corresponding historical trajectory information of the UAV are input into the model through the inference interface. The model returns the detection results, which can be visualized on the image to be detected.

[0162] To obtain the detection results of the video to be detected, only the video needs to be passed in, without the need to pass in historical trajectory information. The detection method will generate a trajectory based on the cumulative video frame image detection results.

[0163] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding a UAV target detection method based on historical trajectory generative attention, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0164] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MCU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0165] This invention provides an idea and method for UAV target detection based on historical trajectory generative attention. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.

Claims

1. A UAV target detection method based on historical trajectory generative attention, characterized in that, Includes the following steps: Step 1: Construct a historical attention map based on the historical trajectory information of the UAV target. The elements in the historical attention map correspond one-to-one with the pixels of the image to be detected, and the element values ​​in the map represent the contextual attention level of the pixel. Step 2: Design a generative attention module. Its input is the historical attention map constructed in Step 1, i.e., the historical trajectory information of the UAV target; the output is the attention prediction map of the current image to be detected, i.e., trajectory attention. Step 3: Design a trajectory attention fusion module to predict the region in the image to be detected where the UAV may appear using the historical trajectory information of the UAV target, and define it as the region of interest; Step 4: Optimize the feature extraction part of the YOLOv8 deep learning model to obtain an improved feature extraction network. Design an improved total loss function for the improved feature extraction network. Step 5: Construct a drone target detection model based on historical trajectory generative attention; Step 6: Train the UAV target detection model based on historical trajectory generative attention constructed in Step 5; Step 7: Use the improved UAV target detection model based on historical trajectory generative attention trained in Step 6 to perform target detection, thus completing the UAV target detection based on historical trajectory generative attention. The improved total loss function mentioned in step 4 includes: improved cross-entropy loss. Cross-combination loss and attention bias loss Among these, the focus is on bias loss, specifically including: The attention bias loss is used to represent the deviation between the trajectory attention generated by the generative attention module described in step 2 and the actual location of the UAV. The attention bias loss is calculated based on the trajectory attention generated in step 2. With the regional location matrix and Composition, including trajectory attention The expression is: ； Regional location matrix and The expression is: ； in, The drone target number in the detection results. Indicates the label is The region location matrix of the UAV target is generated by the detection labels corresponding to the images in the training dataset, and the detection labels include the position of the target in the corresponding image. Pay attention to deviation loss The calculation method is as follows: ； in, The number of horizontal pixels in the image to be detected. The vertical pixel count of the image to be detected. Indicates pixel coordinates Pixel trajectory attention, Indicates pixel coordinates The region position matrix of the pixels; The improved feature extraction network is as follows: The input image features are refined through a convolutional module to restore the number of feature channels to match the number of feature channels in the input image. After passing through two convolutional modules, the input image is processed by a C2p convolutional feature extraction module to generate the first-level derived features. The first-level features are further divided into two branches for feature extraction, with the processing method being the same in both branches. The first-level features are first processed by the convolutional module, then processed by the C2p convolutional feature extraction module, and the result is passed through the convolutional module again, followed by the C2p convolutional feature extraction module. This completes the branch feature processing. After obtaining the feature results from the two branches, the two features are concatenated, and the concatenated features are used as the second-level derived features. The second-level features are passed through a convolutional module and a C2p convolutional feature extraction module, and then through a fast spatial pyramid pooling layer to generate the third-level derived features.

2. The UAV target detection method based on historical trajectory generative attention according to claim 1, characterized in that, The construction of the historical attention map described in step 1 specifically includes the following steps: Step 1-1: Given an image to be detected with a width of m pixels and a length of n pixels, construct a matrix of size m×n. For the matrix Each element in the table is assigned an initial value; Steps 1-2: Define the set of drones in the historical trajectory information of the drone target. Calculate the size of the drone in w×h pixels. The distance from the current position in the trajectory Location region matrix of each trajectory point The size of the location region matrix is ​​equal to the size of the image to be detected, and the coordinates in the location region matrix are... element values The following methods are used: ； Steps 1-3, for those possessing the first The historical trajectory set and historical attention mapping map are obtained by summing the location region matrices of all UAV targets at each trajectory point. For at least The decay sum of a set of historical trajectories, i.e.: ； in, This is the attenuation coefficient.

3. The UAV target detection method based on historical trajectory generative attention according to claim 2, characterized in that, The design of the generative attention module described in step 2 specifically includes: The generative attention module consists of a feature context modeling layer and a trajectory attention layer; the input to this module is the historical trajectory information of the UAV target. The context attention layer is processed by a 1×1 convolutional layer and the Softmax activation function. The result is multiplied by itself before the 1×1 convolution and used to model the features of the mapping map in context. The trajectory attention layer sequentially performs 1×1 convolution, normalization, ReLU activation, and 1×1 convolution on the output of the context attention layer to capture the feature dependencies in the mapping graph. Finally, the above results are processed using the Sigmoid function to make them non-linear outputs, which is the output of the generative attention module: trajectory attention.

4. The UAV target detection method based on historical trajectory generative attention according to claim 3, characterized in that, The design trajectory focus fusion module mentioned in step 3 specifically includes: Step 3-1: The image to be detected obtains three levels of image features at different scales through a multi-scale feature extraction network. The first-level image features and the trajectory attention output in step 2 are used as inputs to this module. Step 3-2: Perform linear mapping and Tanh activation function processing on the first-level image features of the image to be detected to obtain the first processing result. Perform a click operation on the image features and trajectory attention, and perform linear mapping on the dot product operation result to obtain the second processing result. Add and merge the two processing results. Step 3-3 divides the result of the addition and merging into two branches. The first branch does not undergo any additional processing, while the second branch is processed using a linear mapping layer and the Softmax activation function. Steps 3-4 involve multiplying the first branch and the processed second branch element by element to form the image features of the additional region of interest.

5. The UAV target detection method based on historical trajectory generative attention according to claim 4, characterized in that, The improved total loss function described in step 4 specifically includes: The total loss function includes: improved cross-entropy loss. Cross-combination loss and attention bias loss Among them, the improved cross-entropy loss as follows: ； in, The intersection-union ratio (IUU) of the predicted bounding box and the ground truth bounding box. For local perception classification scores, when the classification result is the same as the true result, the IoU value between the predicted box and the true box is taken, and the value is 0 for other positions; Cross-combination loss for: ； in, and These represent the center points of the predicted bounding box and the actual bounding box of the target, respectively. This represents the Euclidean distance between two rectangles. This represents the diagonal distance between the closed regions of the two rectangles. This indicates the consistency of the relative proportions of the two rectangles. These are weighting coefficients; Improved total loss function for: 。 6. The UAV target detection method based on historical trajectory generative attention according to claim 5, characterized in that, The optimization of the feature extraction part of the YOLOv8 deep learning model mentioned in step 4 involves the following improvements to the original feature extraction network in the YOLOv8 deep learning model: Step 4-1: Replace the C2f convolutional feature extraction module in the original feature extraction network with a redesigned C2p convolutional feature extraction module; Step 4-2: Adjust the structural branches in the original feature extraction network, parallelize the cascaded feature extraction operations, reduce the number of convolutional blocks, and optimize repetitive modules to remove feature redundancy and reduce the number of parameters in the feature extraction network.

7. The UAV target detection method based on historical trajectory generative attention according to claim 6, characterized in that, The redesigned C2p convolutional feature extraction module described in step 4-1 specifically includes: Step 4-1-1: In the C2p convolutional feature extraction module, the input features are processed by the convolution module and then split into n half features. The number of channels in each half feature is half that of the original image features. Step 4-1-2: Extract the backbone for each half-feature in parallel using n Darknet features, meaning that each feature extraction backbone is independent of the others. Step 4-1-3: For the n semi-features processed by the feature extraction backbone, concatenate them with the image features before splitting.

8. The UAV target detection method based on historical trajectory generative attention according to claim 7, characterized in that, The UAV target detection model based on historical trajectory generative attention described in step 6 specifically includes: The image to be detected is sequentially passed through two 3x3 convolutional modules and a C2p convolutional feature extraction module. The historical trajectory of the UAV, which has passed through the attention generation module, is then input into the trajectory attention fusion module. Finally, the image is passed through an improved feature extraction network to obtain the final target detection result.

Images