A nighttime drone target tracking method integrating hybrid attention mechanism

Abstract

The invention discloses a nighttime unmanned aerial vehicle target tracking method integrating a hybrid attention mechanism. The method uses a night vision image enhancement model to enhance each night image frame in a night vision video sequence of the unmanned aerial vehicle, and then performs image target tracking and recognition, so as to improve the target tracking accuracy for nighttime images. The night vision image enhancement model regards the enhancement processing of the nighttime image frames of the night vision video sequence as a curve projection task, and changes the image quality by adjusting the red, green and blue three-channel values. With the help of a spatial channel hybrid attention mechanism, the spatial channel features can be adaptively learned, and the generalization ability of the model can be improved, so that the network model pays attention to more effective features, and a denoising processing module is used to remove the hidden noise in the nighttime image, thereby improving the target positioning accuracy and tracking performance of the unmanned aerial vehicle for nighttime scenes, and reducing the probability of target tracking failure of the unmanned aerial vehicle in nighttime scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision target tracking, and in particular to a nighttime unmanned aerial vehicle target tracking method integrating a hybrid attention mechanism. Background Art

[0002] In recent years, drones have been increasingly widely used in many fields such as civil, military and scientific fields due to their small size, flexible movements and easy operation. For example, they are of great research significance in disaster relief in extreme weather, collecting enemy information on the battlefield, intelligent agricultural monitoring, and target recognition and tracking from the perspective of drones. Drone night target tracking is a branch of general target tracking. Its task is to continuously track the target in subsequent frames after specifying the target in the first frame of the video sequence, that is, to calibrate the target using a bounding box (usually represented by a rectangular box) to achieve the positioning and scale estimation of the target.

[0003] Target tracking methods are mainly divided into two categories, one based on correlation filtering method and the other based on deep learning method.

[0004] The correlation filtering method was used in the field of computer vision as early as the 1980s. In 2010, Bolme et al. first introduced the correlation filtering into the field of target tracking. They used the Fast Fourier Transform (FFT) and the convolution theorem to propose the Minimum Output Sum of Squared Error (MOSSE) algorithm, which determines the target position by calculating the maximum response value in the candidate area. Since this method relies on manual design, it has great limitations when the target appearance is deformed.

[0005] With the rapid development of deep learning, deep neural networks have been widely used in the field of computer vision by learning image features from large-scale data. Twin networks have become a hot topic in the field of target tracking with their advantages of fewer parameters and faster calculation speed. SiamFC first introduced twin networks into the field of target tracking. Inspired by Faster RCNN, SiamRPN introduced the Region Proposal Network (RPN) into the twin network framework to improve the robustness of the tracker. SiamRPN++ introduced a deep residual network (Residual Network, ResNet), which greatly improved the tracking performance. In order to avoid the complex subsequent calculations in the anchor frame design, some scholars introduced the anchor-free idea. HiFT, SiamAPN++ and others proposed anchor-free methods to distinguish targets from backgrounds. Most of these algorithms track in daytime scenes. Due to the different distribution of night image features, the night drone tracking performance is greatly weakened. Summary of the invention

[0006] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a nighttime UAV target tracking method integrating a hybrid attention mechanism, so as to improve the positioning accuracy of nighttime UAV target tracking and reduce the probability of target tracking failure of UAV in night scenes.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0008] A nighttime UAV target tracking method integrating a hybrid attention mechanism is proposed. The night vision video sequence of the UAV is obtained, and the night image frames in the night vision video sequence are input into a pre-trained night vision image enhancement model to obtain the corresponding enhanced night image frames. Then, the enhanced night image frames are subjected to image target tracking and recognition to obtain target tracking and recognition results.

[0009] The night vision image enhancement model includes an encoder module, a spatial mixed attention module, a channel mixed attention module, a decoder module, a curve projection module and a denoising processing module;

[0010] The encoder module is used to extract an initial convolutional feature map of a nighttime image frame;

[0011] The spatial hybrid attention module is used to improve the feature space dimension attention of the initial convolution feature map to form a spatial attention feature map of the night image frame;

[0012] The channel mixed attention module is used to improve the attention of the feature channel dimension of the spatial attention feature map to form a mixed attention feature map of the night image frame;

[0013] The decoder module is used to convert the hybrid attention feature map into a curve estimation parameter map;

[0014] The curve projection module is used to map the curve estimation parameter map to the night image frame in a curve projection manner to form an intermediate feature image of the night image frame;

[0015] The denoising module is used to perform denoising on the intermediate feature image to obtain a corresponding enhanced nighttime image frame.

[0016] As a preferred solution, the encoder module includes four convolutional layers and four downsampling layers connected in series, which are used to perform convolution processing and pooling processing on the input night image frame in sequence to obtain an initial convolution feature map of the night image frame after convolution processing;

[0017] The process of extracting the initial convolutional feature map of the night image frame by the encoder module is expressed as:

[0018] F en1 =Pool(Conv(X) en1 ) en1 ;

[0019] F en2 =Pool(Conv(F en1 ) en2 ) en2 ;

[0020] F en3 =Pool(Conv(F en2 ) en3 ) en3 ;

[0021] F en =Pool(Conv(F en3 ) en4 ) en4 ;

[0022] Where X represents the input original night image frame, Conv(·) eni is the i-th convolutional layer operator in the encoder module, Pool(·) eni is the i-th downsampling layer operator in the encoder module, i = 1, 2, 3, 4, F en1 、F en2 、F en3 They represent the intermediate operation outputs in the encoder module, F en Represents the encoder module output initial convolutional feature map.

[0023] As a preferred solution, the spatial hybrid attention module includes a first normalization layer LayerNorm1, a spatial attention unit SAM, a first window multi-head self-attention unit W-MSA1, a second normalization layer LayerNorm2 and a first residual feedforward layer ResFFN1; the input feature map of the spatial hybrid attention module is transmitted to the first normalization layer LayerNorm1, the output of the first normalization layer LayerNorm1 is respectively connected to the input of the spatial attention unit SAM and the window multi-head self-attention unit W-MSA, and the output of the spatial attention unit SAM and the first window multi-head self-attention unit W-MSA1 are added to the input features of the spatial hybrid attention module to form an intermediate feature map; then the intermediate features are processed by the second normalization layer LayerNorm2 and the first residual feedforward layer ResFFN1 in turn, and then added to the intermediate feature self-map to obtain the spatial attention feature map as the output of the spatial hybrid attention module;

[0024] The processing process of the spatial hybrid attention module to extract the spatial attention feature map of the night image frame is expressed as:

[0025] F N =LN1(F en );

[0026] F M =λ1SAM(F N )+W-MSA1(F N )+F en ;

[0027] F SHA =ResFFN1(LN2(F M ))+F M ;

[0028] Among them, F en represents the initial convolutional feature map input to the spatial hybrid attention module, LN1(·) is the first normalization layer LayerNorm1 operator, and F N is the output of the first normalization layer LayerNorm1; SAM(·) and W-MSA1(·) are the spatial attention unit SAM operator and the first window multi-head self-attention unit W-MSA1 operator, respectively, λ1 is the spatial attention weight, F M represents the intermediate feature map obtained by adding the output of the spatial attention unit SAM operator and the first window multi-head self-attention unit W-MSA1 to the input feature map; LN2(·) and ResFFN1(·) are the second normalization layer LayerNorm2 operator and the first residual feed-forward layer ResFFN1 operator, respectively. SHA Represents the spatial attention feature map output by the spatial hybrid attention module.

[0029] As a preferred solution, the spatial attention unit SAM is used to input the feature map F N After the convolution operation and GELU activation function operation of the compression channel, the convolution operation of the recovery channel is performed, and then input to the spatial attention layer; in the spatial attention layer, the input feature map F is first N The maximum pooling and average pooling are performed respectively, and then the Concat operation is used to aggregate the vectors. Finally, a two-dimensional convolution operation and a sigmoid activation function operation are performed, and then the vectors are combined with the input feature map F N Multiply them to get the output result; the processing process of the spatial attention unit SAM is expressed as:

[0030] F o =Conv U (GELU[Conv D (F N )]);

[0031] SAM(F N)=sigmoid(Conv([AvgPool(F o );MaxPool(F o )]))×F o ;

[0032] Among them, F N and SAM(F N ) represent the input and output of the spatial assistant unit SAM, GELU(·) represents the GELU activation function operation, Conv D (·) and Conv U (·) are the convolution operations of the compression channel and the restoration channel respectively, and F o represents the feature map input to the spatial attention layer after the convolution operation of the recovery channel; AvgPool(·) and MaxPool(·) represent average pooling and maximum pooling operations respectively, and Conv(·) represents a two-dimensional convolution operation.

[0033] As a preferred solution, the channel hybrid attention module includes a third normalization layer LayerNorm3, a channel attention unit CAB, a second window multi-head self-attention unit W-MSA2, a fourth normalization layer LayerNorm4 and a second residual feedforward layer ResFFN2; the input feature map of the hybrid attention block HAB is transmitted to the third normalization layer LayerNorm3, the output of the third normalization layer LayerNorm3 is respectively connected to the input of the channel attention unit CAB and the second window multi-head self-attention unit W-MSA2, and the output of the channel attention unit CAB and the second window multi-head self-attention unit W-MSA2 is added to the input features of the channel hybrid attention module to form an intermediate feature map; then the intermediate features are processed by the fourth normalization layer LayerNorm4 and the second residual feedforward layer ResFFN2 in turn, and then added to the intermediate feature self-map to obtain the channel attention feature map as the output of the channel hybrid attention module;

[0034] The processing process of the channel mixed attention module to extract the mixed attention feature map of the night image frame is expressed as:

[0035] F P =LN3(F SHA );

[0036] F Q =λ2CAB(F P )+W-MSA2(F P )+F SHA ;

[0037] F CHA =ResFFN2(LN4(F Q ))+FQ ;

[0038] Among them, F SHA represents the spatial attention feature map input to the channel hybrid attention module, LN3(·) is the third normalization layer LayerNorm3 operator, F P is the output of the third normalization layer LayerNorm3; CAB(·) and W-MSA2(·) are the channel attention unit CAB operator and the second window multi-head self-attention unit W-MSA2 operator, respectively, λ2 is the channel attention weight, F Q represents the intermediate feature map obtained by adding the output of the channel attention unit CAB operator and the second window multi-head self-attention unit W-MSA2 to the input feature map; LN4(·) and ResFFN2(·) are the fourth normalization layer LayerNorm4 operator and the second residual feed-forward layer ResFFN2 operator, respectively. CHA Mixed attention feature map representing the output of the channel-wise mixed attention module.

[0039] As a preferred solution, the channel attention unit CAB is used to perform a convolution operation of a compressed channel and a GELU activation function operation on the input feature map, and then perform a convolution operation of a restored channel, and then input it to the channel attention layer; in the channel attention layer, the input feature map is first subjected to an adaptive average pooling process, and then subjected to a channel compression process and a ReLU activation function operation, and then subjected to a channel recovery process and a sigmoid function operation, and then multiplied with the feature map input to the channel attention layer to obtain an output result; the processing process of the channel attention unit CAB is expressed as follows:

[0040] F B =Conv U (GELU[Conv D (F P )]);

[0041] CAB(F P )=H GP (F B )·sigmoid[W U (δ(W D (F B )))]×F B ;

[0042] Among them, F P and CAB(F P ) represent the input and output of the channel attention unit CAB, GELU(·) represents the GELU activation function operation, Conv D (·) and Conv U(·) are the convolution operations of the compression channel and the restoration channel respectively, and F B H represents the feature map input to the channel attention layer after the convolution operation of the restored channel; GP (·) represents adaptive average pooling, sigmoid(·) and δ(·) represent the sigmoid activation function and ReLU activation function operations respectively, W D (·) and W U (·) represent channel compression processing and channel restoration processing respectively.

[0043] As a preferred solution, the decoder module includes 4 convolutional layers and 4 upsampling layers connected in series, which are used to perform convolution processing and upsampling deconvolution processing on the input mixed attention feature map in sequence, and then perform hyperbolic tangent transformation processing to obtain a curve estimation parameter map; the processing process of the decoder module is expressed as:

[0044] F de1 =Up(Conv(F CHA ) de1 ) de1 ;

[0045] F de2 =Up(Conv(F de1 ) de2 ) de2 ;

[0046] F de3 =Up(Conv(F de2 ) de3 ) de3 ;

[0047] F de4 =Up(Conv(F de3 ) de4 ) de4 ;

[0048] F de =tanh(Conv(F de4 ) de );

[0049] Among them, F CHA represents the mixed attention feature map input to the decoder module, Conv(·) dei is the i-th convolutional layer operator in the decoder module, Up(·) dei is the i-th upsampling deconvolution layer operator in the decoder module, i = 1, 2, 3, 4, F de1 、F de2 、F de3 、F de4They represent the intermediate operation outputs in the decoder module, Conv(·) de is the convolution operator during the conversion process in the decoder module, tanh(·) represents the hyperbolic tangent function tanh operation, and F de Graph representing the estimated parameters of the encoder module output curve.

[0050] As a preferred solution, the curve projection module uses the curve estimation parameter map to multiply the original night image frame pixel by pixel, and maps it to the night image frame in a curve projection manner, thereby forming an intermediate feature image;

[0051] The processing process of the curve projection module is expressed as follows:

[0052] X′=X+F de ⊙(1-X);

[0053] Among them, F de represents the curve estimation parameter map of the input encoder module output, X represents the night vision image frame, ⊙ represents the element-wise multiplication curve projection operator, and X′ represents the output intermediate feature image.

[0054] As a preferred solution, the denoising processing module includes 19 convolution activation units connected in series and a hyperbolic tangent branch unit, each convolution activation unit is composed of a two-dimensional convolution layer and a ReLU activation function layer; the processing process of the denoising processing module is expressed as follows:

[0055] X out =X′+tanh(W(X′) 19 );

[0056] Among them, X′ represents the intermediate feature image input to the denoising processing module, X out represents the enhanced night image frame after denoising by the denoising module; W(·) 19 represents 19 groups of two-dimensional convolution operations and ReLU activation function operations in series, and tanh(·) represents the hyperbolic tangent function tanh operation.

[0057] As a preferred solution, the night vision image enhancement model is trained in the following manner:

[0058] The pre-completed night vision images and daytime images are used as paired training samples to form a training sample set as the input of the night vision image enhancement model, the training loss is calculated according to the obtained enhanced night vision images and the daytime images of the corresponding training samples, and the parameters of the night vision image enhancement model are optimized based on the training loss until the night vision image enhancement model converges to obtain a trained night vision image enhancement model;

[0059] In the training of the night vision image enhancement model, the training loss is calculated by the following formula:

[0060]

[0061] Among them, φ i (·) represents the operation of extracting the i-th layer features from the AlexNet network, X out represents the enhanced night image frame obtained by enhancing the night image frame in the paired training sample through the night vision image enhancement model, Y represents the day image frame in the paired training sample, c j represents the number of channels of the j-th layer feature map extracted by the AlexNet network, w j Indicates the width of the j-th layer feature map extracted by the AlexNet network, h j represents the height of the j-th layer feature map extracted by the AlexNet network, and ||·||2 represents the L2 norm operation.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] 1. In the nighttime UAV target tracking method integrating the hybrid attention mechanism of the present invention, each nighttime image frame in the night vision video sequence of the UAV is enhanced by using the night vision image enhancement model, and then image target tracking and recognition are performed to improve the target tracking accuracy for the nighttime images; wherein, the night vision image enhancement model regards the enhancement processing of the nighttime image frames of the night vision video sequence as a curve projection task, and changes the image quality by adjusting the red, green and blue three-channel values. With the help of the spatial channel hybrid attention mechanism, the spatial channel features can be adaptively learned, and the generalization ability of the model is improved, so that the network model pays attention to more effective features, and the denoising processing module is referenced to remove the hidden noise in the nighttime image, thereby improving the target positioning accuracy and tracking performance of the UAV for the nighttime scene, and reducing the probability of target tracking failure of the UAV in the nighttime scene.

[0064] 2. The nighttime UAV target tracking method integrating the hybrid attention mechanism of the present invention can enhance the nighttime image frames of the night vision video sequence with the help of the night vision image enhancement model, and then be widely combined with various target tracking algorithms to perform nighttime UAV target tracking tasks, thereby significantly improving the UAV tracking performance in nighttime scenes, and has great technical application and promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is an overall flow chart of the nighttime UAV target tracking method integrating the hybrid attention mechanism of the present invention.

[0066] Figure 2 Schematic diagram of the architecture of the night vision image enhancement model SCHAB-Net in the method of the present invention.

[0067] Figure 3Schematic diagram of the processing flow of the spatial attention unit SAM in the night vision image enhancement model.

[0068] Figure 4 Schematic diagram of the processing flow of the channel attention unit CAB in the night vision image enhancement model.

[0069] Figure 5 It is a schematic diagram of the architecture of the denoising processing module Denoise in the method of the present invention.

[0070] Figure 6 : This is a comparison chart of target tracking results using different drone target tracking algorithms in the embodiments. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0072] The specific implementation modes of the present invention are further described in detail below in conjunction with specific examples.

[0073] In view of the poor positioning accuracy and performance of drones in performing target tracking tasks in night scenes in the prior art, the present invention provides a nighttime drone target tracking method integrating a hybrid attention mechanism. The overall processing flow of the method is as follows: Figure 1 As shown, by obtaining the night vision video sequence of the UAV, the night image frames in the night vision video sequence are input into the pre-trained night vision image enhancement model to obtain the corresponding enhanced night image frames, and then the enhanced night image frames are subjected to image target tracking and recognition to obtain the target tracking and recognition results.

[0074] The night vision image enhancement model of the present invention adopts a deep learning network model that integrates a hybrid attention mechanism, which can be referred to as SCHAB-Net. The model processing process is as follows: Figure 2As shown, the night vision image enhancement model includes an encoder module, a spatial mixed attention module, a channel mixed attention module, a decoder module, a curve projection module and a denoising processing module; wherein the encoder module is used to extract the initial convolution feature map of the night image frame; the spatial mixed attention module is used to enhance the feature space dimension attention of the initial convolution feature map to form the spatial attention feature map of the night image frame; the channel mixed attention module is used to enhance the feature channel dimension attention of the spatial attention feature map to form the mixed attention feature map of the night image frame; the decoder module is used to convert the mixed attention feature map into a curve estimation parameter map; the curve projection module is used to map the curve estimation parameter map to the night image frame in a curve projection manner to form an intermediate feature image of the night image frame; the denoising processing module is used to denoise the intermediate feature image to obtain the corresponding enhanced night image frame.

[0075] pass Figure 1 , Figure 2 As can be seen from the above process, the entire real-time nighttime drone tracking process of the method of the present invention (such as Figure 1 As shown in the figure, the whole tracking process is mainly divided into two stages: the SCHAB-Net night image enhancement processing stage, and the drone tracker performs the target tracking and identification task stage. In order to make the whole tracking process have more stable performance, before executing the first enhancement processing stage, the night image frames in the extracted drone night vision video sequence can be subjected to certain preprocessing operations, such as cropping the night image frames to remove some useless information, normalizing the image size, performing scale transformation to scale the image to a fixed size, etc. After these preprocessing operations, it is convenient to subsequently send the SCHAB-Net network model to perform night image enhancement processing, and finally combine the drone tracker to complete the target tracking and identification task. In the night drone target tracking method integrating the hybrid attention mechanism of the present invention, each night image frame in the night vision video sequence of the drone is enhanced by the night vision image enhancement model, and then image target tracking and identification are performed to improve the target tracking accuracy for night images.

[0076] The following is a detailed description of the night vision image enhancement model in the solution of the present invention.

[0077] Night Vision Image Enhancement Model SCHAB-Net

[0078] The night vision image enhancement model SCHAB-Net used in this invention is a deep learning network model that integrates a hybrid attention mechanism. It is mainly used to enhance the night image frames of the drone's night vision video sequence, thereby helping to improve the drone's target tracking performance for night scenes. The input night image is improved through the deep learning network. The network structure of the night vision image enhancement model SCHAB-Net is as follows: Figure 2 As shown, it includes an encoder module, a spatial mixed attention module, a channel mixed attention module, a decoder module, a curve projection module and a denoising processing module; the mixed attention module and the channel mixed attention module are introduced to extract the spatial attention features and channel attention feature information of the night image frames, and the denoising processing module is used to remove the hidden noise in the night image to achieve feature enhancement processing of the night image frames.

[0079] Encoder modules

[0080] The encoder module includes 4 convolutional layers and 4 downsampling layers connected in series, which are used to perform convolution processing and downsampling processing on the input night image frames in sequence to obtain the initial convolution feature map of the night image frames after convolution processing. In a specific implementation, the 4 convolutional layers and 4 downsampling layers in the encoder module can use 4 two-dimensional convolution units and 4 pooling layers. Figure 2 The layers marked as Encoder layers layer1 to 4 are used for preliminary night image frame feature extraction.

[0081] The process of extracting the initial convolutional feature map of the nighttime image frame by the encoder module is expressed as:

[0082] F en1 =Pool(Conv(X) en1 ) en1 ;

[0083] F en2 =Pool(Conv(F en1 ) en2 ) en2 ;

[0084] F en3 =Pool(Conv(F en2 ) en3 ) en3 ;

[0085] F en =Pool(Conv(F en3 ) en4 ) en4 ;

[0086] Where X represents the input original night image frame, Conv(·)eni is the i-th convolutional layer operator in the encoder module, Pool(·) eni is the i-th downsampling layer operator in the encoder module, i = 1, 2, 3, 4, F en1 、F en2 、F en3 They represent the intermediate operation outputs in the encoder module, F en Represents the encoder module output initial convolutional feature map.

[0087] Spatial Hybrid Attention layer

[0088] The spatial mixed attention module is Figure 2 The layer marked as Spatial Hybrid Attention layer in the figure has a module structure including the first normalization layer LayerNorm1, the spatial attention unit SAM, the first window multi-head self-attention unit W-MSA1, the second normalization layer LayerNorm2 and the first residual feedforward layer ResFFN1; the input feature map of the spatial hybrid attention module is transmitted to the first normalization layer LayerNorm1, and the output of the first normalization layer LayerNorm1 is connected to the input of the spatial attention unit SAM and the window multi-head self-attention unit W-MSA respectively, and the output of the spatial attention unit SAM and the first window multi-head self-attention unit W-MSA1 are added to the input features of the spatial hybrid attention module to form an intermediate feature map; then the intermediate features are processed by the second normalization layer LayerNorm2 and the first residual feedforward layer ResFFN1 in turn, and then added to the intermediate feature self-map to obtain the spatial attention feature map as the output of the spatial hybrid attention module.

[0089] The processing process of the Spatial Hybrid Attention layer to extract the spatial attention feature map of the night image frame can be expressed as:

[0090] F N =LN1(F en );

[0091] F M =λ1SAM(F N )+W-MSA1(F N )+F en ;

[0092] F SHA =ResFFN1(LN2(F M ))+F M ;

[0093] Among them, Fen represents the initial convolutional feature map input to the spatial hybrid attention module, LN1(·) is the first normalization layer LayerNorm1 operator, and F N is the output of the first normalization layer LayerNorm1; SAM(·) and W-MSA1(·) are the spatial attention unit SAM operator and the first window multi-head self-attention unit W-MSA1 operator respectively; λ1 is the spatial attention weight, which can be taken as λ1 = 0.01 in practical applications; F M represents the intermediate feature map obtained by adding the output of the spatial attention unit SAM operator and the first window multi-head self-attention unit W-MSA1 to the input feature map; LN2(·) and ResFFN1(·) are the second normalization layer LayerNorm2 operator and the first residual feed-forward layer ResFFN1 operator, respectively. SHA Represents the spatial attention feature map output by the spatial hybrid attention module.

[0094] The first difference feedforward layer ResFFN1 is a residual feedforward network, which is a common network unit in deep learning networks. Here we only give a brief introduction. Figure 2 As can be seen in the figure, the network processing process of the residual feedforward network ResFFN is that the input feature map undergoes the first linear transformation and is reshaped by the feature map, and then the residual operation of the convolution layer and the deep convolution layer (DWConv) is performed, and then the feature is flattened and the second linear transformation is performed to generate the output. Specifically, the residual feedforward network ResFFN is a common network that has been used in the prior art, and will not be described in detail again.

[0095] Spatial Attention Unit SAM

[0096] In the spatial hybrid attention module, the spatial attention unit SAM consists of a set of Conv-GELU-Conv layers and a spatial attention layer, such as Figure 3 The Conv-GELU-Conv layer is used to perform convolution operations on the input feature map to compress the channel and perform GELU activation function operations, and then perform convolution operations on the recovery channel, and then input it to the spatial attention layer; for example, for an input feature with C channels, the number of channels of the output feature after the first convolution layer is compressed to Then, the number of channels is adjusted to C through the second convolution layer and sent to the channel attention layer. In the spatial attention layer, two parallel maximum pooling MaxPool operations and average pooling AvgPool operations are first performed on the input feature map, and then the Concat operation is used to aggregate the vector. Finally, a two-dimensional convolution operation and a sigmoid activation function operation are performed, and then multiplied with the feature map input to the channel attention layer to obtain the output result.

[0097] The processing of the spatial attention unit SAM is expressed as:

[0098] F o =Conv U (GELU[Conv D (F N )]);

[0099] SAM(F N )=sigmoid(Conv([AvgPool(F o );MaxPool(F o )]))×F o ;

[0100] Among them, F N and SAM(F N ) represent the input and output of the spatial assistant unit SAM, GELU(·) represents the GELU activation function operation, Conv D (·) and Conv U (·) are the convolution operations of the compression channel and the restoration channel respectively, and F o represents the feature map input to the spatial attention layer after the convolution operation of the recovery channel; AvgPool(·) and MaxPool(·) represent average pooling and maximum pooling operations respectively, and Conv(·) represents a two-dimensional convolution operation.

[0101] Multi-head self-attention unit W-MSA

[0102] The multi-head self-attention unit W-MSA is a common network unit in deep learning networks. Here we only give a brief introduction. The multi-head self-attention unit W-MSA directly obtains the input of each single-head attention head by slicing the input feature map evenly, and then performs a Concat aggregation operation after each single-head attention head is processed.

[0103] Channel Hybrid Attention layer

[0104] The channel-mixed attention module is Figure 2The layer marked as Channel Hybrid Attention layer in the figure has a module structure including the third normalization layer LayerNorm3, the channel attention unit CAB, the second window multi-head self-attention unit W-MSA2, the fourth normalization layer LayerNorm4 and the second residual feedforward layer ResFFN2; the input feature map of the hybrid attention block HAB is transmitted to the third normalization layer LayerNorm3, and the output of the third normalization layer LayerNorm3 is respectively connected to the input of the channel attention unit CAB and the second window multi-head self-attention unit W-MSA2, and the output of the channel attention unit CAB and the second window multi-head self-attention unit W-MSA2 are added to form an intermediate feature map; then the intermediate features are processed by the fourth normalization layer LayerNorm4 and the second residual feedforward layer ResFFN2 in turn, and then added to the intermediate feature self-map to obtain the channel attention feature map as the output of the channel hybrid attention module.

[0105] The channel hybrid attention module annular Hybrid Attention layer extracts the hybrid attention feature map of the night image frame. The processing process can be expressed as:

[0106] F P =LN3(F SHA );

[0107] F Q =λ2CAB(F P )+W-MSA2(F P )+F SHA ;

[0108] F CHA =ResFFN2(LN4(F Q ))+F Q ;

[0109] Among them, F SHA represents the spatial attention feature map input to the channel hybrid attention module, LN3(·) is the third normalization layer LayerNorm3 operator, F P is the output of the third normalization layer LayerNorm3; CAB(·) and W-MSA2(·) are the channel attention unit CAB operator and the second window multi-head self-attention unit W-MSA2 operator respectively; λ2 is the channel attention weight, which can be taken as λ2=0.02 in practical applications; F Qrepresents the intermediate feature map obtained by adding the output of the channel attention unit CAB operator and the second window multi-head self-attention unit W-MSA2 to the input feature map; LN4(·) and ResFFN2(·) are the fourth normalization layer LayerNorm4 operator and the second residual feed-forward layer ResFFN2 operator, respectively. CHA Represents the mixed attention feature map output by the channel mixed attention module. Among them, the second difference feedforward layer ResFFN2 is also a residual feedforward network, which has been introduced before.

[0110] Channel Attention Unit CAB

[0111] In the channel mixed attention module, the channel attention unit CAB consists of a set of Conv-GELU-Conv layers and a channel attention layer, such as Figure 4 The Conv-GELU-Conv layer is used to perform convolution operations on the input feature map to compress the channel and perform GELU activation function operations, and then perform convolution operations on the recovery channel, and then input it to the channel attention layer; for example, for an input feature with C channels, the number of channels of the output feature after the first convolution layer is compressed to Then, the number of channels is adjusted to C through the second convolution layer and sent to the channel attention layer. In the channel attention layer, the input feature map is first processed by adaptive average pooling, then channel compression and ReLU activation function operation are performed, and then channel recovery and sigmoid function operation are performed, and then multiplied with the feature map input to the channel attention layer to obtain the output result; for example, the input feature map is a 1×1×C vector, W D and W U The number of compression channels is And restore the operation of channel number C, and then perform activation operation.

[0112] The processing process of the channel attention unit CAB can be expressed as:

[0113] F B =Conv U (GELU[Conv D (F P )]);

[0114] CAB(F P )=H GP (F B )·sigmoid[W U (δ(W D (F B )))]×F B ;

[0115] Among them, F P and CAB(FP ) represent the input and output of the channel attention unit CAB, GELU(·) represents the GELU activation function operation, Conv D (·) and Conv U (·) are the convolution operations of the compression channel and the restoration channel respectively, and F B H represents the feature map input to the channel attention layer after the convolution operation of the restored channel; GP (·) represents adaptive average pooling, sigmoid(·) and δ(·) represent the sigmoid activation function and ReLU activation function operations respectively, W D (·) and W U (·) represent channel compression processing and channel restoration processing respectively.

[0116] Decoder module

[0117] The decoder module consists of 4 convolutional layers and 4 upsampling layers connected in series. Figure 2 Layers 5 to 8 are marked in the figure, which are used to perform four rounds of convolution and upsampling deconvolution on the input mixed attention feature map, and then perform hyperbolic tangent conversion. Specifically, they perform one round of two-volume convolution and hyperbolic tangent activation function. Figure 2 It is marked as Layer9 in the figure, thus obtaining the curve estimation parameter diagram.

[0118] The processing of the decoder module is expressed as:

[0119] F de1 =Up(Conv(F CHA ) de1 ) de1 ;

[0120] F de2 =Up(Conv(F de1 ) de2 ) de2 ;

[0121] F de3 =Up(Conv(F de2 ) de3 ) de3 ;

[0122] F de4 =Up(Conv(F de3 ) de4 ) de4 ;

[0123] F de =tanh(Conv(F de4 ) de );

[0124] Among them, F CHA represents the mixed attention feature map input to the decoder module, Conv(·) dei is the i-th convolutional layer operator in the decoder module, Up(·) dei is the i-th upsampling deconvolution layer operator in the decoder module, i = 1, 2, 3, 4, F de1 、F de2 、F de3 、F de4 They represent the intermediate operation outputs in the decoder module, Conv(·) de is the convolution operator during the conversion process in the decoder module, tanh(·) represents the hyperbolic tangent function tanh operation, and F de Graph representing the estimated parameters of the encoder module output curve.

[0125] Curve projection module

[0126] The curve projection module uses the curve estimation parameter map to multiply the original night image frame pixel by pixel, and maps it to the night image frame in a curve projection manner to form an intermediate feature image.

[0127] The processing of the curve projection module is expressed as:

[0128] X′=X+F de ⊙(1-X);

[0129] Among them, F de represents the curve estimation parameter map of the input encoder module output, X represents the night vision image frame, ⊙ represents the element-wise multiplication curve projection operator, and X′ represents the output intermediate feature image.

[0130] Denoising module Denoise

[0131] The denoising processing module Denoise uses residual learning to accelerate the training process and improve the denoising performance. The structure of the denoising processing module Denoise is as follows: Figure 5 As shown, it includes 19 convolutional activation units connected in series and a hyperbolic tangent branch unit, and each convolutional activation unit consists of a two-dimensional convolution layer and a ReLU activation function layer.

[0132] The processing process of the denoising module is expressed as:

[0133] X out =X′+tanh(W(X′) 19 );

[0134] Among them, X′ represents the intermediate feature image input to the denoising processing module, X outrepresents the enhanced night image frame after denoising by the denoising module; W(·) 19 represents 19 groups of two-dimensional convolution operations and ReLU activation function operations in series, and tanh(·) represents the hyperbolic tangent function tanh operation. In a specific implementation, the two-dimensional convolution operation in the convolution activation unit can take kernel_size=3 and padding=1.

[0135] Image object tracking task

[0136] After the night vision image enhancement model enhances the night image frames in the night vision video sequence, the target tracking algorithm currently used by drones to perform image target tracking tasks can be used to perform night image target tracking and recognition to obtain target tracking and recognition results. During the nighttime aerial photography of drones, any target can be specified and continuously tracked, and the identified tracking target can be marked with a bounding box marker.

[0137] The entire image target tracking task process can be expressed as:

[0138] B=φ(Z;SCTHAB(X))

[0139] Where B represents the tracking target boundary predicted by the UAV tracking recognition, φ(·) represents the target tracking algorithm of the UAV, Z and X represent the target to be tracked and the subsequent frame search area, respectively, and SCTHAB(·) represents the enhancement processing of the night vision image enhancement model SCTHAB-Net.

[0140] Through the above introduction, it can be seen that in the nighttime UAV target tracking method integrating the hybrid attention mechanism of the present invention, the night vision image enhancement model is used to enhance the night image frames in the night vision video sequence of the UAV, and then the image target tracking and recognition is performed to improve the target tracking accuracy for the night image. Among them, the night vision image enhancement model regards the enhancement processing of the night image frames of the night vision video sequence as a curve projection task, and changes the image quality by adjusting the red, green and blue three-channel values. With the help of the spatial channel hybrid attention mechanism, the spatial channel features can be adaptively learned, the generalization ability of the model can be improved, and the network model pays attention to more effective features, and the denoising processing module is used to remove the hidden noise in the night image, thereby improving the target positioning accuracy and tracking performance of the UAV for the night scene, and reducing the probability of target tracking failure of the UAV in the night scene.

[0141] Next, the training of the night vision image enhancement model in the present invention is introduced.

[0142] Training of night vision image enhancement model

[0143] Specifically, the night vision image enhancement model can be trained in the following ways:

[0144] The pre-completed night vision images and daytime images are used as paired training samples to form a training sample set as the input of the night vision image enhancement model. The training loss is calculated according to the enhanced night vision images and the daytime images of the corresponding training samples. The parameters of the night vision image enhancement model are optimized based on the training loss until the night vision image enhancement model converges to obtain a trained night vision image enhancement model.

[0145] In the training of the night vision image enhancement model, the training loss is calculated by the following formula:

[0146]

[0147] Among them, φ i (·) represents the operation of extracting the i-th layer features from the AlexNet network, X out represents the enhanced night image frame obtained by enhancing the night image frame in the paired training sample through the night vision image enhancement model, Y represents the day image frame in the paired training sample, c j represents the number of channels of the j-th layer feature map extracted by the AlexNet network, w j Indicates the width of the j-th layer feature map extracted by the AlexNet network, h j represents the height of the j-th layer feature map extracted by the AlexNet network, and ||·||2 represents the L2 norm operation.

[0148] The present invention calculates the training loss through the above formula, and considers the training loss by comparing the difference between the 3rd to 5th layer feature maps extracted in the AlexNet network between the night image frames in the paired training samples after model enhancement processing and the day image frames. This can effectively ensure the training gradient and avoid overly complicated training processing procedures and comparison operation procedures, which is conducive to better ensuring the robustness and efficiency of model training.

[0149] The present invention is further described below by way of examples.

[0150] Example

[0151] In this embodiment, the SCHAB-Net night vision image enhancement model is trained with 485 normal / low-light image pairs in the LOLdataset data set, the public test UAVDark135 is used as the evaluation benchmark, and the algorithm performance is evaluated by Success and Precision. In this embodiment, the night vision image enhancement model SCTHAB-Net of the present invention is combined with SiamAPN++ and SiamRPN drone tracking algorithms, and the algorithm comparison results are shown in Table 1. SiamAPN++ is selected as the drone night tracker, and the night vision image enhancement model SCTHAB-Net of the present invention is used to compare target tracking and recognition with the SCT algorithm and DCE++ night enhancement algorithm in the prior art. The target tracking and recognition results are shown in Table 2, and the target tracking results of the drone are shown in Table 3. Figure 6 As shown; Figure 6 The GT group is the real target tracking result used as the reference benchmark.

[0152] Table 1

[0153]

[0154] Table 2

[0155]

[0156] The above experimental results show that, in this embodiment, the nighttime UAV target tracking method enhanced by the method of the present invention using the night vision image enhancement model has better target positioning accuracy and tracking performance, and the probability of target tracking failure is also significantly reduced.

[0157] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A nighttime UAV target tracking method integrating hybrid attention mechanism, characterized in that: Obtain a night vision video sequence of the UAV, input the night image frames in the night vision video sequence into a pre-trained night vision image enhancement model to obtain the corresponding enhanced night image frames, and then perform image target tracking and recognition on the enhanced night image frames to obtain target tracking and recognition results; The night vision image enhancement model includes an encoder module, a spatial mixed attention module, a channel mixed attention module, a decoder module, a curve projection module and a denoising processing module; The encoder module is used to extract an initial convolutional feature map of a nighttime image frame; The spatial hybrid attention module is used to improve the feature space dimension attention of the initial convolution feature map to form a spatial attention feature map of the night image frame; The channel mixed attention module is used to improve the attention of the feature channel dimension of the spatial attention feature map to form a mixed attention feature map of the night image frame; The decoder module is used to convert the hybrid attention feature map into a curve estimation parameter map; The curve projection module is used to map the curve estimation parameter map to the night image frame in a curve projection manner to form an intermediate feature image of the night image frame; The denoising processing module is used to perform denoising processing on the intermediate feature image to obtain a corresponding enhanced nighttime image frame; The decoder module includes 4 convolutional layers and 4 upsampling layers connected in series, which are used to perform convolution processing and upsampling deconvolution processing on the input mixed attention feature map in sequence, and then perform hyperbolic tangent transformation processing to obtain a curve estimation parameter map; the processing process of the decoder module is expressed as: F de1 =Up(Conv(F CHA ) de1 ) de1 ; F de2 =Up(Conv(F de1 ) de2 ) de2 ; F de3 =Up(Conv(F de2 ) de3 ) de3 ; F de4 =Up(Conv(F de3 ) de4 ) de4 ; F de =tanh(Conv(F de4 ) de ); Among them, F CHA represents the mixed attention feature map input to the decoder module, Conv(·) dei is the i-th convolutional layer operator in the decoder module, Up(·) dei is the i-th upsampling deconvolution layer operator in the decoder module, i = 1, 2, 3, 4, F de1 、F de2 、F de3 、F de4 They represent the intermediate operation outputs in the decoder module, Conv(·) de is the convolution operator during the conversion process in the decoder module, tanh(·) represents the hyperbolic tangent function tanh operation, and F de Graph representing the estimated parameters of the encoder module output curve.

2. The nighttime UAV target tracking method integrating hybrid attention mechanism according to claim 1 is characterized in that: The encoder module includes four convolutional layers and four downsampling layers connected in series, which are used to perform convolution processing and pooling processing on the input night image frame in sequence to obtain an initial convolution feature map of the night image frame after convolution processing; The process of extracting the initial convolutional feature map of the night image frame by the encoder module is expressed as: F en1 =Pool(Conv(X) en1 ) en1 ; F en2 =Pool(Conv(F en1 ) en2 ) en2 ; F en3 =Pool(Conv(F en2 ) en3 ) en3 ; F en =Pool(Conv(F en3 ) en4 ) en4 ; Where X represents the input original night image frame, Conv(·) eni is the i-th convolutional layer operator in the encoder module, Pool(·) eni is the i-th downsampling layer operator in the encoder module, i = 1, 2, 3, 4, F en1 、F en2 、F en3 They represent the intermediate operation outputs in the encoder module, F en Represents the encoder module output initial convolutional feature map.

3. The nighttime UAV target tracking method integrating hybrid attention mechanism according to claim 1 is characterized in that: The spatial hybrid attention module includes a first normalization layer LayerNorm1, a spatial attention unit SAM, a first window multi-head self-attention unit W-MSA1, a second normalization layer LayerNorm2 and a first residual feedforward layer ResFFN1; the input feature map of the spatial hybrid attention module is transmitted to the first normalization layer LayerNorm1, the output of the first normalization layer LayerNorm1 is respectively connected to the input of the spatial attention unit SAM and the window multi-head self-attention unit W-MSA, and the output of the spatial attention unit SAM and the first window multi-head self-attention unit W-MSA1 are added to the input features of the spatial hybrid attention module to form an intermediate feature map; then the intermediate features are processed by the second normalization layer LayerNorm2 and the first residual feedforward layer ResFFN1 in turn, and then added to the intermediate feature self-map to obtain the spatial attention feature map as the output of the spatial hybrid attention module; The processing process of the spatial hybrid attention module to extract the spatial attention feature map of the night image frame is expressed as: F N =LN1(F en ); F M =λ1SAM(F N )+W-MSA1(F N )+F en ; F SHA =ResFFN1(LN2(F M ))+F M ; Among them, F en represents the initial convolutional feature map input to the spatial hybrid attention module, LN1(·) is the first normalization layer LayerNorm1 operator, and F N is the output of the first normalization layer LayerNorm1; SAM(·) and W-MSA1(·) are the spatial attention unit SAM operator and the first window multi-head self-attention unit W-MSA1 operator, respectively, λ1 is the spatial attention weight, F M represents the intermediate feature map obtained by adding the output of the spatial attention unit SAM operator and the first window multi-head self-attention unit W-MSA1 to the input feature map; LN2(·) and ResFFN1(·) are the second normalization layer LayerNorm2 operator and the first residual feed-forward layer ResFFN1 operator, respectively. SHA Represents the spatial attention feature map output by the spatial hybrid attention module.

4. The nighttime UAV target tracking method integrating hybrid attention mechanism according to claim 3 is characterized in that: The spatial attention unit SAM is used to input the feature map F N After the convolution operation and GELU activation function operation of the compression channel, the convolution operation of the recovery channel is performed, and then input to the spatial attention layer; in the spatial attention layer, the input feature map F is first N The maximum pooling and average pooling are performed respectively, and then the Concat operation is used to aggregate the vectors. Finally, a two-dimensional convolution operation and a sigmoid activation function operation are performed, and then the vectors are combined with the input feature map F N Multiply them to get the output result; the processing process of the spatial attention unit SAM is expressed as: F o =Conv U (GELU[Conv D (F N )]); SAM(F N )=sigmoid(Conv([AvgPool(F o );MaxPool(F o )]))×F o ; Among them, F N and SAM(F N ) represent the input and output of the spatial assistant unit SAM, GELU(·) represents the GELU activation function operation, Conv D (·) and Conv U (·) are the convolution operations of the compression channel and the restoration channel respectively, and F o represents the feature map input to the spatial attention layer after the convolution operation of the recovery channel; AvgPool(·) and MaxPool(·) represent average pooling and maximum pooling operations respectively, and Conv(·) represents a two-dimensional convolution operation.

5. The nighttime UAV target tracking method integrating hybrid attention mechanism according to claim 1 is characterized in that: The channel hybrid attention module includes a third normalization layer LayerNorm3, a channel attention unit CAB, a second window multi-head self-attention unit W-MSA2, a fourth normalization layer LayerNorm4 and a second residual feedforward layer ResFFN2; the input feature map of the hybrid attention block HAB is transmitted to the third normalization layer LayerNorm3, the output of the third normalization layer LayerNorm3 is respectively connected to the input of the channel attention unit CAB and the second window multi-head self-attention unit W-MSA2, and the output of the channel attention unit CAB and the second window multi-head self-attention unit W-MSA2 are added to the input features of the channel hybrid attention module to form an intermediate feature map; then the intermediate features are processed by the fourth normalization layer LayerNorm4 and the second residual feedforward layer ResFFN2 in turn, and then added to the intermediate feature self-map to obtain the channel attention feature map as the output of the channel hybrid attention module; The processing process of the channel mixed attention module to extract the mixed attention feature map of the night image frame is expressed as: <h2 style=";text-align:left;direction:ltr">F<h2 style=";text-align:left;direction:ltr"> P <h2 style=";text-align:left;direction:ltr"> =LN3(F<h2 style=";text-align:left;direction:ltr"> SHA <h2 style=";text-align:left;direction:ltr"> ); <h2 style=";text-align:left;direction:ltr">F<h2 style=";text-align:left;direction:ltr"> Q <h2 style=";text-align:left;direction:ltr"> =λ2CAB(F<h2 style=";text-align:left;direction:ltr"> P <h2 style=";text-align:left;direction:ltr"> +W-MSA2(F)<h2 style=";text-align:left;direction:ltr"> P <h2 style=";text-align:left;direction:ltr"> )+F<h2 style=";text-align:left;direction:ltr"> SHA <h2 style=";text-align:left;direction:ltr"> ; F CHA =ResFFN2(LN4(F Q ))+F Q ; Among them, F SHA represents the spatial attention feature map input to the channel hybrid attention module, LN3(·) is the third normalization layer LayerNorm3 operator, F P is the output of the third normalization layer LayerNorm3; CAB(·) and W-MSA2(·) are the channel attention unit CAB operator and the second window multi-head self-attention unit W-MSA2 operator, respectively, λ2 is the channel attention weight, F Q represents the intermediate feature map obtained by adding the output of the channel attention unit CAB operator and the second window multi-head self-attention unit W-MSA2 to the input feature map; LN4(·) and ResFFN2(·) are the fourth normalization layer LayerNorm4 operator and the second residual feed-forward layer ResFFN2 operator, respectively. CHA Mixed attention feature map representing the output of the channel-wise mixed attention module.

6. The nighttime UAV target tracking method integrating hybrid attention mechanism according to claim 5 is characterized in that: The channel attention unit CAB is used to perform a convolution operation of the compressed channel and a GELU activation function operation on the input feature map, and then perform a convolution operation of the restored channel, and then input it to the channel attention layer; in the channel attention layer, the input feature map is firstly subjected to an adaptive average pooling process, and then subjected to a channel compression process and a ReLU activation function operation, and then subjected to a channel recovery process and a sigmoid function operation, and then multiplied with the feature map input to the channel attention layer to obtain an output result; the processing process of the channel attention unit CAB is expressed as follows: F B =Conv U (GELU[Conv D (F P )]); CAB(F P )=H GP (F B )·sigmoid[W U (δ(W D (F B )))]×F B ; Among them, F P and CAB(F P ) represent the input and output of the channel attention unit CAB, GELU(·) represents the GELU activation function operation, Conv D (·) and Conv U (·) are the convolution operations of the compression channel and the restoration channel respectively, and F B H represents the feature map input to the channel attention layer after the convolution operation of the restored channel; GP (·) represents adaptive average pooling, sigmoid(·) and δ(·) represent the sigmoid activation function and ReLU activation function operations respectively, W D (·) and W U (·) represent channel compression processing and channel restoration processing respectively.

7. The nighttime UAV target tracking method integrating hybrid attention mechanism according to claim 1 is characterized in that: The curve projection module uses the curve estimation parameter map to perform pixel-by-pixel multiplication with the original night image frame, and maps it to the night image frame in a curve projection manner, thereby forming an intermediate feature image; The processing process of the curve projection module is expressed as follows: X′=X+F de ⊙(1-X); Among them, F de represents the curve estimation parameter map of the input encoder module output, X represents the night vision image frame, ⊙ represents the element-wise multiplication curve projection operator, and X′ represents the output intermediate feature image.

8. The nighttime UAV target tracking method integrating hybrid attention mechanism according to claim 1 is characterized in that: The denoising processing module includes 19 convolution activation units connected in series and a hyperbolic tangent branch unit, each convolution activation unit is composed of a two-dimensional convolution layer and a ReLU activation function layer; the processing process of the denoising processing module is expressed as follows: X out =X′+tanh(W(X′) 19 ); Among them, X′ represents the intermediate feature image input to the denoising processing module, X out represents the enhanced night image frame after denoising by the denoising module; W(·) 19 represents 19 groups of two-dimensional convolution operations and ReLU activation function operations in series, and tanh(·) represents the hyperbolic tangent function tanh operation.

9. The nighttime UAV target tracking method integrating hybrid attention mechanism according to claim 1 is characterized in that: The night vision image enhancement model is trained in the following way: The pre-completed night vision images and daytime images are used as paired training samples to form a training sample set as the input of the night vision image enhancement model, the training loss is calculated according to the obtained enhanced night vision images and the daytime images of the corresponding training samples, and the parameters of the night vision image enhancement model are optimized based on the training loss until the night vision image enhancement model converges to obtain a trained night vision image enhancement model; In the training of the night vision image enhancement model, the training loss is calculated by the following formula: Among them, φ i (·) represents the operation of extracting the i-th layer features from the AlexNet network, X out represents the enhanced night image frame obtained by enhancing the night image frame in the paired training sample through the night vision image enhancement model, Y represents the day image frame in the paired training sample, c j represents the number of channels of the j-th layer feature map extracted by the AlexNet network, w j Indicates the width of the j-th layer feature map extracted by the AlexNet network, h j represents the height of the j-th layer feature map extracted by the AlexNet network, and ||·||2 represents the L2 norm operation.

Images