An infrared unmanned aerial vehicle target tracking method based on morphosemantic collaborative perception

Abstract

The application discloses an infrared unmanned aerial vehicle target tracking method based on morphology and semantic collaborative perception, and relates to the technical field of target tracking. The application comprises the following steps: preparing a training data set and a test data set, constructing an infrared unmanned aerial vehicle target tracking network, training the network using the training data set and saving the weight, and evaluating the network performance using the test data set. The application proposes a morphology and semantic collaborative perception module, which enhances the network's attention to the morphology and semantic information of the infrared unmanned aerial vehicle target by jointly using a morphology information perception module and a semantic information perception module. The application designs a morphology information perception module, extracts and aggregates multi-level context information. The application designs a semantic information perception module, efficiently aggregates channel feature semantic information. The application designs a global feature interaction module, enhances the network's attention to the position and boundary information of the infrared unmanned aerial vehicle target, and solves the problem of poor performance of the existing infrared unmanned aerial vehicle target tracking network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target tracking, and particularly relates to an infrared unmanned aerial vehicle target tracking method based on morphology and semantic collaborative perception. BACKGROUND

[0002] Infrared unmanned aerial vehicle target tracking is a key perception technology for unmanned aerial vehicle monitoring in anti-unmanned aerial vehicle systems, and needs to accurately and effectively track unmanned aerial vehicles. In actual scenarios, due to the influence of factors such as complex background, unmanned aerial vehicle target deformation, and camera movement, infrared unmanned aerial vehicle target tracking still has great challenges. Existing target tracking algorithms incorporate feature pyramids into their tracking networks, gradually fuse cross-layer feature maps in a top-down manner, and effectively improve the tracking performance of multi-scale targets. Before feature fusion, the feature pyramid uses convolution to reduce the channels of multi-scale feature maps in the bottom-up path to align the feature channels of different levels. However, this alignment operation will cause the loss of morphology and semantic information of the infrared unmanned aerial vehicle target, and seriously affect the performance of the tracking network. SUMMARY

[0003] The technical solution of the present application to solve the above technical problems is to provide an infrared unmanned aerial vehicle target tracking method based on morphology and semantic collaborative perception, comprising the following steps:

[0004] S1: preparing a training data set and a test data set;

[0005] S2: constructing an infrared unmanned aerial vehicle target tracking network, comprising:

[0006] Feature extraction network: extracting template feature maps and search feature maps;

[0007] First multi-scale feature fusion network: fusing multi-level template feature maps through a morphology and semantic collaborative perception module;

[0008] Second multi-scale feature fusion network: fusing multi-level search feature maps through a morphology and semantic collaborative perception module;

[0009] Feature interaction network: fusing template and search feature maps through a global feature interaction module;

[0010] Detection module: detecting target position;

[0011] S3: training the network using the training data set and saving the weights;

[0012] S4: evaluating the network performance using the test data set;

[0013] The morphological and semantic collaborative perception module includes: a morphological information perception module, which acquires target morphological information through axial embedding, multi-branch grouping perception, and morphological aggregation; a semantic information perception module, which enhances semantic features through channel interaction and covariance analysis; and convolutional layers and upsampling layers.

[0014] Furthermore, the morphological information sensing module includes:

[0015] Axial morphological information embedding module: global average pooling is performed along the horizontal and vertical directions to generate one-dimensional feature codes;

[0016] Multi-branch grouping information perception module: decomposes the encoding into sub-features and uses one-dimensional deep convolutional kernels of different sizes to extract multi-scale features;

[0017] Morphological information aggregation module: splices and decouples multi-scale features, and generates morphological feature maps through normalization and activation operations.

[0018] Furthermore, the axial morphology information embedding module performs the following:

[0019] Horizontal embedding: in, This represents the output encoded at height h and channel c.

[0020] Vertical embedding: in, This represents the output encoded at height w and channel c.

[0021] Furthermore, the multi-branch grouping information sensing module concatenates two one-dimensional encoded features along the channel direction. The channel concatenation formula is as follows:

[0022] f = [z h ,z w ];

[0023] Where [·,·] represents a connection operation along a spatial dimension. It is an intermediate feature map that encodes morphological information in the horizontal and vertical directions;

[0024] Decompose f along the channel dimension into K independent sub-features of equal size, and the number of channels corresponding to each sub-feature is... The eigenvalue decomposition formula is:

[0025] f i =Split(f);

[0026] Where Split(g) represents a specific channel separation operation. Let i represent the i-th sub-feature, where i takes values ​​in the range [1, K].

[0027] The K sub-features are subjected to multi-scale feature extraction using one-dimensional deep convolution kernels of different sizes, and the multi-scale extraction formula is:

[0028]

[0029] where g i represents a feature with i-level context information, represents a one-dimensional deep convolution kernel applied to the i-th sub-feature, k i represents the size of the convolution kernel, and k i = i x 2 + 1.

[0030] Further, the morphology information aggregation module concatenates the context features of different levels along the channel dimension to obtain

[0031] g = [g1, g2,..., g K ];

[0032] Decouple g along the spatial dimension into two independent tensors and and use a group normalization layer GN to normalize g h and g w respectively; use a Sigmoid function to nonlinearly excite the normalized g h and g w to obtain:

[0033]

[0034] where σ(g) represents the Sigmoid function; multiply and the input feature map element by element to obtain a feature map Y with target morphology information; the calculation method of the c-th channel, (i, j) spatial coordinate feature of the feature map Y is:

[0035]

[0036] Further, the semantic information perception module maps the features containing global spatial information to channel through a two-dimensional global average pooling operation. Use a 1x1 two-dimensional convolution and a strip matrix B to respectively interact the global and local channel information of U to obtain a global feature matrix u g and a local feature matrix u l ; the mathematical expression is:

[0037]

[0038] u g = Conv 1×1(U)

[0039]

[0040] where B = [b1, b2, b3,..., bn] is a band matrix, and K is defined as the number of adjacent elements in a band. K

[0041] The semantic information perception module uses homologous bilinear pooling to multiply u l and the transposed u g , and obtains the covariance channel feature matrix m l ; the physical meaning of the i-th row m l is the statistical dependence of the i-th channel of u l on all channels of u g ; the row convolution Conv row is used to adaptively learn the channel dependence in the horizontal direction, and the pixel-level multiplication with u l is performed through a Sigmoid function to obtain the local feature with global attention u u g After the operation, the global feature with local attention u

[0042] The sum of u and u is added element by element and multiplied by the input feature through a Sigmoid function to obtain the final channel attention feature map F output :

[0043]

[0044] Further, the global feature interaction module converts the multi-scale fusion features of the template branch and the search branch u and u into one-dimensional feature vectors, and performs regularization processing through layer normalization to obtain u and u for feature interaction, where k represents the k-th layer of multi-scale fusion features.

[0045] u is sent into two linear layers u and u to obtain value vectors (V) and key vectors (K), respectively; similarly, u is sent into a single linear layer u to obtain a query vector (Q).

[0046] The key-value pair information of all positions is aggregated to generate a global context, and the Q is weighted to obtain

[0047] ​The residual connection and the feedforward neural network are used to capture the internal feature dependence of the vector, and final double-branch interaction features Y are obtained; and the mathematical expression of the global feature interaction module is as follows:

[0048]

[0049] Wherein, sim(·) represents a cosine similarity function, LN is a layer normalization function, and FFN is a feedforward neural network using a multilayer perceptron.

[0050] Compared with the prior art, the infrared unmanned aerial vehicle target tracking method with morphology and semantic collaborative perception has the following beneficial effects:

[0051] 1. The morphology and semantic collaborative perception module is proposed, the morphology information perception module and the semantic information perception module are jointly used to enhance the attention of the network to the morphology and semantic information of the infrared unmanned aerial vehicle target, reduce the loss of important information of the low-level feature map, and improve the performance of the infrared unmanned aerial vehicle target tracking network.

[0052] 2. The morphology information perception module is designed, the axial morphology information embedding module, the multi-branch grouping information perception module and the morphology information aggregation module are used to extract and aggregate multi-level context information, so that the tracking network has the ability to capture the morphology information of the infrared unmanned aerial vehicle target.

[0053] 3. The semantic information perception module is designed, the global and local information of the feature channel are integrated to efficiently aggregate the channel feature semantic information; the module can highlight the semantic feature information of the infrared unmanned aerial vehicle target while suppressing irrelevant background information with interference, so as to realize reasonable weight distribution.

[0054] 4. The global feature interaction module is designed, the global cross attention is used to better utilize the template-search branch features to enhance the attention of the network to the position and boundary information of the infrared unmanned aerial vehicle target. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the drawings shown.

[0056] Figure 1 The structure diagram of the infrared unmanned aerial vehicle target tracking method with morphology and semantic collaborative perception is shown in the figure.

[0057] Figure 2 The structure diagram of the infrared unmanned aerial vehicle target tracking network is shown in the figure.

[0058] Figure 3 A schematic diagram of the morphology semantic collaborative perception module structure of the present application;

[0059] Figure 4 A schematic diagram of the morphology information perception module structure of the present application;

[0060] Figure 5 A schematic diagram of the semantic information perception module structure of the present application;

[0061] Figure 6 A schematic diagram of the global feature interaction module structure of the present application. DETAILED DESCRIPTION

[0062] The present application proposes an infrared unmanned aerial vehicle target tracking method based on morphology semantic collaborative perception, aiming to design an infrared unmanned aerial vehicle target tracking method with better tracking precision and accuracy.

[0063] The infrared unmanned aerial vehicle target tracking method based on morphology semantic collaborative perception proposed by the present application will be described below in specific embodiments:

[0064] Embodiment 1:

[0065] An infrared unmanned aerial vehicle target tracking method based on morphology semantic collaborative perception, as shown in Figure 1 , comprising the following steps:

[0066] S1: Prepare a training data set and a test data set;

[0067] S2: Construct an infrared unmanned aerial vehicle target tracking network, comprising:

[0068] Feature extraction network: extract template feature map and search feature map;

[0069] First multi-scale feature fusion network: fuse multi-level template feature maps through a morphology semantic collaborative perception module;

[0070] Second multi-scale feature fusion network: fuse multi-level search feature maps through a morphology semantic collaborative perception module;

[0071] Feature interaction network: fuse template and search feature maps through a global feature interaction module;

[0072] Detection module: detect target position;

[0073] S3: Train the network using the training data set and save the weights;

[0074] S4: Evaluate the network performance using the test data set;

[0075] The morphological semantic collaborative perception module comprises: a morphological information perception module, which obtains target morphological information through axial embedding, multi-branch grouping perception and morphological aggregation; a semantic information perception module, which enhances semantic features through channel interaction and covariance analysis; a convolution layer and an up-sampling layer.

[0076] Further, the morphological information perception module comprises:

[0077] An axial morphological information embedding module, which performs global average pooling along the horizontal and vertical directions to generate one-dimensional feature encoding;

[0078] A multi-branch grouping information perception module, which decomposes the encoding into sub-features and extracts multi-scale features using one-dimensional deep convolution kernels of different sizes;

[0079] A morphological information aggregation module, which concatenates and decouples the multi-scale features to generate a morphological feature map through normalization and excitation operations.

[0080] Further, the axial morphological information embedding module performs:

[0081] Horizontal direction embedding: wherein, represents the output of the encoding on the h height and c channel;

[0082] Vertical direction embedding: wherein, represents the output of the encoding on the w height and c channel.

[0083] Further, the multi-branch grouping information perception module concatenates the two one-dimensional encoding features along the channel direction, and the channel concatenation formula is:

[0084] f=[z h ,z w ];

[0085] wherein, [·,·] represents a connection operation along the spatial dimension, is an intermediate feature map for encoding morphological information in the horizontal and vertical directions;

[0086] f is decomposed into K independent sub-features of the same size along the channel dimension, and the number of channels corresponding to each sub-feature is The feature decomposition formula is:

[0087] f i =Spilt(f);

[0088] wherein, Spilt(g) represents a channel separation operation, represents the i-th sub-feature, and i takes a value in the range of [1, K];

[0089] The K sub-features are subjected to multi-scale feature extraction using one-dimensional deep convolution kernels of different sizes, and the multi-scale extraction formula is:

[0090]

[0091] where g i represents the feature with the i-th level of context information, represents a one-dimensional deep convolution kernel applied to the i-th sub-feature, k i represents the size of the convolution kernel, and k i = i x 2 + 1.

[0092] Further, the morphology information aggregation module concatenates the context features of different levels along the channel dimension to obtain

[0093] g = [g1, g2,..., g K ];

[0094] g is decoupled along the spatial dimension into two independent tensors and and the group normalization layer GN is used to normalize g h and g w ; the normalized g h and g w are subjected to nonlinear excitation using a Sigmoid function to obtain:

[0095]

[0096] where σ(g) represents the Sigmoid function; and g and the input feature map are multiplied element by element to obtain a feature map Y with target morphology information; the calculation method of the c-th channel and the (i, j) spatial coordinate feature of the feature map Y is:

[0097]

[0098] Further, the semantic information perception module maps the feature containing global spatial information to the channel by two-dimensional global average pooling operation uses a 1x1 two-dimensional convolution and a strip matrix B to respectively interact the global and local channel information of U to obtain a global feature matrix u g and a local feature matrix u l ; and the mathematical expression is:

[0099]

[0100] u g = Conv 1×1(U);

[0101]

[0102] where B = [b1, b2, b3, …, b K ] is a band matrix, K is defined as the number of adjacent channel elements;

[0103] The semantic information perception module uses homologous bilinear pooling to multiply u l and the transposed u g to obtain the covariance channel feature matrix m l ; the physical meaning of the i-th row m l is the statistical dependence of the i-th channel of u l on all channels of u g ; the row convolution Conv row is used to adaptively learn the channel dependence in the horizontal direction, and the Sigmoid function is used to perform pixel-level multiplication with u l to obtain local features with global attention u g After the operation, global features with local attention are obtained

[0104] The sum of and is added element by element and multiplied by an input feature through a Sigmoid function to obtain the final channel attention feature map F output :

[0105]

[0106]

[0107] Further, the global feature interaction module converts the multi-scale fusion features and of the template branch and the search branch into one-dimensional feature vectors, and performs regularization processing through layer normalization to obtain and for feature interaction, where k represents the k-th layer of multi-scale fusion features.

[0108] The is input into two linear layers and to obtain value vectors (V) and key vectors (K), respectively; similarly, the is input into a single linear layer to obtain a query vector (Q);

[0109] The key-value pair information of all positions is aggregated to generate global context, which is weighted by Q to obtain

[0110] The final two-branch interaction feature Y is obtained by capturing the intra-vector feature dependency using residual connection and feed-forward neural network; the mathematical expression of the global feature interaction module is as follows:

[0111]

[0112] Wherein, sim(·) represents the cosine similarity function, LN is the layer normalization function, and FFN is the feed-forward neural network using multi-layer perceptron.

[0113] Embodiment 2:

[0114] A morphological semantic collaborative perception infrared unmanned aerial vehicle target tracking method, comprising the following steps:

[0115] S1: preparing a data set, including a data set one for training and a data set two for testing;

[0116] Specifically, the prepared data set one is Anti-UAV data set, which is used for training the infrared unmanned aerial vehicle target tracking network; the prepared data set two is self-made data set, which adds label information to the infrared video sequence for different challenges in the actual scene, and is used for testing the final infrared unmanned aerial vehicle target tracking model;

[0117] S2: constructing an infrared unmanned aerial vehicle target tracking network, including a feature extraction network, a multi-scale feature fusion network one, a multi-scale feature fusion network two, a feature interaction network and a detection module; the feature extraction network is used for extracting template and search feature map; the multi-scale feature fusion network one fuses template feature maps of different levels through a morphological semantic collaborative perception module (SCAM); the multi-scale feature fusion network two fuses search feature maps of different levels through the (SCAM); the feature interaction network fuses template and search feature maps through a global feature interaction module (DFM); and the detection module is used for detecting the position of the infrared unmanned aerial vehicle target;

[0118] As Figure 2As shown, the feature extraction network extracts a template feature map by taking the initial frame of the infrared video sequence as a template image and extracts a search feature map by taking the current frame of the infrared video sequence as a search image in a manner of sharing weights; a first multi-scale feature fusion network takes the template feature map as input, gradually fuses low-level and high-level feature maps along a top-down path through three shape and semantic collaborative perception modules (SCAMs), and obtains template feature maps of different scales. A second multi-scale feature fusion network takes the search feature map as input, gradually fuses low-level and high-level feature maps along a top-down path through three shape and semantic collaborative perception modules (SCAMs), and obtains search feature maps of different scales. The feature interaction network takes the output template and search feature maps of the first and second multi-scale feature fusion networks as input, fuses the template and search feature maps of different scales through three global feature interaction modules (DFMs), and obtains fusion feature maps of different scales; the detection module performs bounding box detection on each fusion feature map, and obtains the final infrared unmanned aerial vehicle target detection result using a non-maximum suppression strategy;

[0119] As shown in Figure 3 , the shape and semantic collaborative perception module includes a shape information perception module, a semantic information perception module, a convolution layer and an up-sampling layer; the shape and semantic collaborative perception module enhances the attention of the network to the shape and semantic information of the infrared unmanned aerial vehicle target by jointly using the shape information perception module and the semantic information perception module, so as to reduce the loss of the shape and semantic information of the infrared unmanned aerial vehicle target;

[0120] Specifically, in the top-down feature fusion process, the shape and semantic collaborative perception module first obtains a feature map with target shape information by passing the low-level feature through the shape information perception module, then focuses on important channel semantic information by passing the feature map through the semantic information perception module, and then aligns the channel dimension by using the convolution layer and adds the high-level feature elements after up-sampling to obtain an output feature with discriminative information of the infrared unmanned aerial vehicle target.

[0121] As shown in Figure 4 , the shape information perception module includes an axial shape information embedding module, a multi-branch grouping information perception module and a shape information aggregation module; the shape information perception module first embeds position shape information in each channel of the input feature along the horizontal direction and the vertical direction respectively by using the axial shape information embedding module to obtain a pair of one-dimensional feature encodings with two-dimensional spatial information; then, the pair of one-dimensional feature encodings are sent to the multi-branch grouping information perception module to capture multi-level context information; finally, the multi-level context information is aggregated by using the shape information aggregation module to obtain a feature map with target shape information. Next, the axial shape information embedding module, the multi-branch grouping information perception module and the shape information aggregation module will be introduced in detail.

[0122] (1) Axial shape information embedding module

[0123] The axial morphology information embedding module embeds the axial morphology information of each channel of the input feature map along the horizontal (x-axis) and vertical (y-axis) directions respectively through a global average pooling kernel in two spatial dimensions, to obtain a pair of one-dimensional feature encodings with two-dimensional spatial information; the information embedding along the horizontal direction is represented as:

[0124]

[0125] wherein, represents the encoded output at the h height and c channel; the information embedding along the vertical direction is represented as:

[0126]

[0127] wherein, represents the encoded output at the w height and c channel;

[0128] (2) Multi-branch grouping information perception module

[0129] The multi-branch grouping information perception module first concatenates the two one-dimensional encoded features along the channel direction to obtain

[0130] f = [z h ,z w ];

[0131] wherein, [·,·] represents a connection operation along the spatial dimension, is an intermediate feature map that encodes morphology information in the horizontal and vertical directions; then, f is decomposed into K independent sub-features of the same size along the channel dimension, and each sub-feature corresponds to a number of channels The process of decomposing f into sub-features is as follows:

[0132] f i = Split(f);

[0133] wherein, Split(g) represents a channel separation operation, represents the i-th sub-feature, and i takes a value in the range of [1, K];

[0134] Next, different sizes of one-dimensional deep convolution kernels are used to perform multi-scale feature extraction on the K sub-features respectively, to capture multi-level context information, and the process is as follows:

[0135]

[0136] wherein, g i represents a feature with i-th level context information, represents a one-dimensional deep convolution kernel applied to the i-th sub-feature, k​i This represents the size of the convolution kernel, and k i = i × 2 + 1;

[0137] (3) Morphological information aggregation module

[0138] To aggregate multi-level contextual features without introducing batch processing statistical noise, the morphological information aggregation module first concatenates contextual features from different levels along the channel dimension to obtain...

[0139] g = [g1, g2, ..., g K ];

[0140] Secondly, decouple g along the spatial dimension into two independent tensors. and Use group normalization layer GN to g h and g w Normalization is performed separately for each feature to reduce semantic interference between sub-features; then, the Sigmoid function is used to normalize the g. h and g w Obtain by applying nonlinear excitation

[0141]

[0142] Where σ(g) represents the Sigmoid function; finally, and input feature map Element-wise multiplication yields a feature map Y containing target morphology information; the calculation method for the c-th channel and (i,j)-th spatial coordinate features of feature map Y is as follows:

[0143]

[0144] like Figure 5 As shown, the semantic information perception module can highlight the semantic features of the infrared UAV target while suppressing irrelevant background information that may cause interference, thereby achieving a reasonable weight allocation; the specific workflow is as follows:

[0145] The semantic information perception module first maps features containing global spatial information through a two-dimensional global average pooling operation. Converging into channel descriptors Then, a 1×1 two-dimensional convolution and a strip matrix B are used to perform global and local channel information interaction on U, respectively, to obtain the global feature matrix u. g and local feature matrix u l Its mathematical expression is as follows:

[0146]

[0147] u g=Conv 1×1 (U);

[0148]

[0149] Wherein, the banded matrix B = [b1,b2,b3,...,b K K is defined as the number of adjacent channel elements.

[0150] To focus on the global features of a channel within local features, the semantic information perception module employs homologous bilinear pooling to pool u l With the transposed u g Perform matrix multiplication to obtain the covariance channel characteristic matrix m. l The i-th line m l The physical meaning of u is l The i-th channel pair u g Statistical dependencies of all channels. Then, row convolution Conv... row Channel dependence is adaptively learned along the horizontal (x-axis) direction and correlated with u via the Sigmoid function. l Perform pixel-level multiplication to obtain local features of global interest. Similarly, u g After the above operations, we obtain global features with local interest. Finally, and Element-wise summation followed by multiplication with the input features using a sigmoid function yields the final channel attention feature map F. output :

[0151]

[0152] like Figure 6 As shown, the global feature interaction module enhances the network's attention to the infrared UAV target location and boundary information by better utilizing template-search branch features through global cross-attention. The specific workflow is as follows:

[0153] The global feature interaction module integrates multi-scale features from the template branch and the search branch. and The vector is converted into a one-dimensional feature vector and then regularized using layer normalization to obtain the feature interaction vector. and Where k represents the multi-scale fusion feature of the k-th layer;

[0154] The secondary computational complexity of traditional self-attention mechanisms increases the network's computational load, severely impacting inference speed. Therefore, the global feature interaction module replaces multi-head self-attention with linear attention to meet the real-time tracking requirements of infrared UAV targets. Specifically, it will... two linear layers and , respectively, to obtain value vector (V) and key vector (K); similarly, input into a single linear layer to obtain query vector (Q); then, aggregate the key-value pair information of all positions to generate global context, and weight by Q to obtain Finally, capture the internal feature dependency of the vector using residual connection and feedforward neural network to obtain the final two-branch interaction feature Y; the mathematical expression of the global feature interaction module is as follows:

[0155]

[0156] wherein, sim(·) represents the cosine similarity function, LN is the layer normalization function, and FFN is the feedforward neural network using multilayer perceptron;

[0157] The detection module adopts an anchor-free detection head to predict the position of the infrared unmanned aerial vehicle target. Specifically, the detection head is composed of three parallel branches, namely a classification branch, a confidence branch, and a regression branch. The first two branches are responsible for determining the area where the target may exist, and the third branch is responsible for predicting the boundary box regression. Given the coordinate position (c x ,c y ) in the feature map and the output (t x ,t y ,t w ,t h ) predicted by the detection head, the boundary box calculation formula is

[0158] b x =(2×σ(t x )-0.5)+c x ,

[0159] b y =(2×σ(t y )-0.5)+c y ,

[0160] ;

[0161] wherein, (b x ,b y ,b w ,b h ) represents the target center and target size, and σ represents the Sigmoid function.

[0162] S3: training the infrared unmanned aerial vehicle target tracking network; inputting the training data set prepared by S1 into the infrared unmanned aerial vehicle target tracking network constructed by S2 for training, and saving the trained weight; the related settings in the training stage are as follows:

[0163] The optimizer adopts SGD, and the momentum is set to 0.9 and the weight decay is set to 0.0001; the total training rounds and batches are 250 and 16 respectively; each pair of training data is composed of a template and a search frame from the same sequence, the random interval is less than 200, and data augmentation is performed by using random flipping; the network is trained from zero, and no pre-training parameters are loaded; in terms of hardware platform, a NVIDIA GeForce RTX 3090 GPU server accelerated by CUDA 11.1 is selected for network training; in terms of software platform, Python 3.8 and Pytorch1.8.1 are used for implementation; the loss function is defined as

[0164]

[0165] Wherein, N pos represents the number of positive samples, L cls and L obj are focal loss. L reg is GIoU loss. j defines the index of the selected sample, and are the class label and target position respectively, s j is the predicted classification score, t j is the predicted offset, o j is the predicted object confidence score, is an indicator function, which is 1 when , otherwise 0.

[0166] S4: testing the infrared unmanned aerial vehicle target tracking network; inputting the test data set two prepared by S1 into the infrared unmanned aerial vehicle target tracking network trained by S3 for testing; selecting the average state accuracy, success rate and precision as evaluation indexes to verify the effectiveness of the network;

[0167] The related index comparison of the prior art and the method proposed by the present application is shown in Table 1:

[0168] Table 1: Comparison of related indexes of prior art and method proposed by the present application

[0169]

[0170] From the table, it can be seen that the method proposed by the present application has higher average state accuracy, success rate and precision than the prior art, which further illustrates that the method proposed by the present application has better tracking accuracy and accuracy.

[0171] The above description is only preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A morphosemantic synergistic perception based infrared unmanned aerial vehicle target tracking method, characterized in that, The method comprises the following steps: S1: preparing a training data set and a test data set; S2: constructing an infrared unmanned aerial vehicle target tracking network, comprising: a feature extraction network: extracting a template feature map and a search feature map; a first multi-scale feature fusion network: fusing multi-level template feature maps through a morphological semantic collaborative perception module; a second multi-scale feature fusion network: fusing multi-level search feature maps through a morphological semantic collaborative perception module; a feature interaction network: fusing the template and search feature maps through a global feature interaction module; a detection module: detecting a target position; S3: training the network using the training data set and saving the weight; S4: evaluating the network performance using the test data set; The morphological semantic collaborative perception module comprises: a morphological information perception module: obtaining target morphological information through axial embedding, multi-branch grouping perception and morphological aggregation; a semantic information perception module: enhancing semantic features through channel interaction and covariance analysis; a convolution layer and an up-sampling layer; The morphological information perception module comprises: an axial morphological information embedding module: performing global average pooling along the horizontal and vertical directions to generate one-dimensional feature encoding; a multi-branch grouping information perception module: decomposing the encoding into sub-features, and extracting multi-scale features using one-dimensional deep convolution kernels of different sizes; a morphological information aggregation module: concatenating and decoupling the multi-scale features, and generating a morphological feature map through normalization and excitation operations; The global feature interaction module fuses the multi-scale features of the template branch and the search branch and are converted into one-dimensional feature vectors, and are normalized by layer normalization to obtain and ; wherein, represents the layer multi-scale fusion feature; the global feature interaction module uses linear attention to replace multi-head self-attention, and inputs into two linear layers and to obtain value vectors and key vectors ; inputs into a single linear layer to obtain a query vector ; the key-value pair information of all positions is aggregated to generate a global context, and is weighted by to obtain ; the internal feature dependency of the vector is captured by using a residual connection and a feedforward neural network to obtain the final double-branch interaction feature .

2. The method of claim 1, wherein, The axial morphological information embedding module performs: Horizontal embedding: ; wherein, denotes the height, the encoded output on the channel; Vertical embedding: ; wherein, indicates the height, the encoded output on the channel; The axial morphological information embedding module uses global average pooling kernels in two spatial dimensions to embed the input feature map along the horizontal and vertical directions, respectively. Each channel embeds axial morphology information.

3. The method of claim 2, wherein, The multi-branch grouping information perception module: concatenates the two one-dimensional encoding features along the channel direction, and the channel concatenation formula is: ； wherein, represents a connection operation along a spatial dimension, is an intermediate feature map that encodes modality information in the horizontal and vertical directions; Will be decomposed into independent sub-features of the same size along the channel dimension, and each sub-feature corresponds to channel numbers; the feature decomposition formula is: ； wherein, represents a special channel separation operation, represents the first sub-feature, has a value range of ; Using one-dimensional depthwise convolution kernels of different sizes to respectively Multi-scale feature extraction is performed on each sub-feature. The multi-scale extraction formula is as follows: ； wherein, represents a first feature having a first hierarchical context information, represents a one-dimensional depth convolution kernel applied to the first sub-feature, represents a size of the convolution kernel, and .

4. The method of claim 3, wherein, The morphological information aggregation module concatenates context features of different levels along the channel dimension to obtain ； will be decoupled into two independent tensors along spatial dimensions and and use group normalization layers for and respectively; use function to do nonlinear excitation on normalized and get: ； ； Wherein, Indicates Function; will , And input feature map Element-wise multiplication, get the feature map with target morphology information ; Feature map The first Channel, the first The calculation method of spatial coordinate feature is: 。 5. The method of claim 1, wherein, The semantic information perception module maps features containing global spatial information through a two-dimensional global average pooling operation. Converging into a channel ; using a Two-dimensional convolution and strip matrix To each By exchanging global and local channel information, a global feature matrix is ​​obtained. and local feature matrix Its mathematical expression is: ； ； ； wherein the band matrix , is defined as the number of adjacent channel elements; The semantic information perception module uses homologous bilinear pooling to... With transposed Perform matrix multiplication to obtain the covariance channel feature matrix. ;No. OK The physical meaning is The Each channel pair Statistical dependencies of all channels; utilizing row convolution Adaptively learn channel dependence along the horizontal direction, and through Functions and Perform pixel-level multiplication to obtain local features of global interest. ; After the operation, a global feature with local interest is obtained. ; will be described below. and add up by element and multiply by an function with input features to get the final channel attention feature map : ； ； 。 6. The method of claim 1, wherein, The mathematical expression of the global feature interaction module is as follows: ， ； ； ； ； wherein, denotes a cosine similarity function, is a layer normalization function, is a feed forward neural network using a multi-layer perceptron.

Images