Infrared unmanned aerial vehicle target tracking method based on morphological semantic collaborative perception

Abstract

The invention discloses an infrared unmanned aerial vehicle target tracking method based on morphological semantic collaborative awareness, which relates to the technical field of target tracking, and comprises the following steps: preparing a training data set and a test data set, constructing an infrared unmanned aerial vehicle target tracking network, training the network by using the training data set and storing the weight, and evaluating the network performance by using the test data set. According to the invention, a morphological semantic collaborative sensing module is provided, and the attention of a network on the target morphological and semantic information of the infrared unmanned aerial vehicle is enhanced by jointly using a morphological information sensing module and a semantic information sensing module; a morphological information sensing module is designed, and multi-level context information is extracted and aggregated; a semantic information perception module is designed, and channel feature semantic information is efficiently aggregated; a global feature interaction module is designed to enhance the attention of the network on the target position and boundary information of the infrared unmanned aerial vehicle; the problem that an existing infrared unmanned aerial vehicle target tracking network is poor in performance is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target tracking, and particularly relates to an infrared unmanned aerial vehicle target tracking method based on morphology and semantic collaborative perception. BACKGROUND

[0002] Infrared unmanned aerial vehicle target tracking is a key perception technology for unmanned aerial vehicle monitoring in anti-unmanned aerial vehicle systems, and needs to accurately and effectively track unmanned aerial vehicles. In actual scenarios, due to the influence of factors such as complex background, unmanned aerial vehicle target deformation, and camera movement, infrared unmanned aerial vehicle target tracking still has great challenges. Existing target tracking algorithms incorporate feature pyramids into their tracking networks, gradually fuse cross-layer feature maps in a top-down manner, and effectively improve the tracking performance of multi-scale targets. Before feature fusion, the feature pyramid uses convolution to reduce the channels of multi-scale feature maps in the bottom-up path to align the feature channels of different levels. However, this alignment operation will cause the loss of morphology and semantic information of the infrared unmanned aerial vehicle target, and seriously affect the performance of the tracking network. SUMMARY

[0003] The technical solution of the present application to solve the above technical problems is to provide an infrared unmanned aerial vehicle target tracking method based on morphology and semantic collaborative perception, comprising the following steps:

[0004] S1: preparing a training data set and a test data set;

[0005] S2: constructing an infrared unmanned aerial vehicle target tracking network, comprising:

[0006] Feature extraction network: extracting template feature maps and search feature maps;

[0007] First multi-scale feature fusion network: fusing multi-level template feature maps through a morphology and semantic collaborative perception module;

[0008] Second multi-scale feature fusion network: fusing multi-level search feature maps through a morphology and semantic collaborative perception module;

[0009] Feature interaction network: fusing template and search feature maps through a global feature interaction module;

[0010] Detection module: detecting target position;

[0011] S3: training the network using the training data set and saving the weights;

[0012] S4: evaluating the network performance using the test data set;

[0013] Among them, the morphological and semantic collaborative perception module includes: a morphological information perception module: obtaining target morphological information through axial embedding, multi-branch grouping perception and morphological aggregation; a semantic information perception module: enhancing semantic features through channel interaction and covariance analysis; a convolutional layer and an upsampling layer.

[0014] Furthermore, the morphological information perception module includes:

[0015] Axial morphological information embedding module: performs global average pooling along the horizontal and vertical directions to generate a one-dimensional feature code;

[0016] Multi-branch grouping information perception module: decomposes the code into sub-features and uses one-dimensional deep convolution kernels of different sizes to extract multi-scale features;

[0017] Morphological information aggregation module: splices and decouples multi-scale features, and generates morphological feature maps through normalization and excitation operations.

[0018] Furthermore, the axial morphology information embedding module performs:

[0019] Horizontal embedding: in, Represents the output of the encoding at the hth height and cth channel;

[0020] Vertical embedding: in, Represents the output of the encoding at the w-th height and c-th channel.

[0021] Furthermore, the multi-branch grouping information perception module splices the two one-dimensional coding features along the channel direction. The channel splicing formula is:

[0022] f=[z h ,z w ];

[0023] Among them, [·,·] represents the connection operation along the spatial dimension, It is an intermediate feature map that encodes morphological information in the horizontal and vertical directions;

[0024] Decompose f into K independent sub-features of the same size along the channel dimension, and the number of channels corresponding to each sub-feature is The characteristic decomposition formula is:

[0025] f i =Spilt(f);

[0026] Among them, Spilt(g) represents the special channel separation operation, Represents the i-th sub-feature, and the value range of i is [1,K];

[0027] The K sub-features are subjected to multi-scale feature extraction using one-dimensional deep convolution kernels of different sizes, and the multi-scale extraction formula is:

[0028]

[0029] where g i represents a feature with i-level context information, represents a one-dimensional deep convolution kernel applied to the i-th sub-feature, k i represents the size of the convolution kernel, and k i = i x 2 + 1.

[0030] Further, the morphology information aggregation module concatenates the context features of different levels along the channel dimension to obtain

[0031] g = [g1, g2,..., g K ];

[0032] Decouple g along the spatial dimension into two independent tensors and and use a group normalization layer GN to normalize g h and g w respectively; use a Sigmoid function to nonlinearly excite the normalized g h and g w to obtain:

[0033]

[0034] where σ(g) represents the Sigmoid function; multiply and the input feature map element by element to obtain a feature map Y with target morphology information; the calculation method of the c-th channel, (i, j) spatial coordinate feature of the feature map Y is:

[0035]

[0036] Further, the semantic information perception module maps the features containing global spatial information to channel through a two-dimensional global average pooling operation. Use a 1x1 two-dimensional convolution and a strip matrix B to respectively interact the global and local channel information of U to obtain global feature matrix u g and local feature matrix u l ; the mathematical expression is:

[0037]

[0038] u g = Conv 1×1(U)

[0039]

[0040] where B = [b1, b2, b3,..., bn] is a band matrix, and K is defined as the number of adjacent elements in a band. K

[0041] The semantic information perception module uses homologous bilinear pooling to multiply u l and the transposed u g , and obtains the covariance channel feature matrix m l ; the physical meaning of the ith row m l is the statistical dependence of the ith channel of u l on all channels of u g ; the row convolution Conv row is used to adaptively learn the channel dependence in the horizontal direction, and the pixel-level multiplication with u l is performed through a Sigmoid function to obtain the local feature with global attention u u g After the operation, the global feature with local attention u

[0042] The sum of u and u is added element by element and multiplied by the input feature through a Sigmoid function to obtain the final channel attention feature map F output :

[0043]

[0044] Further, the global feature interaction module converts the multi-scale fusion features of the template branch and the search branch u and u into one-dimensional feature vectors, and performs regularization processing through layer normalization to obtain u and u for feature interaction, where k represents the kth layer of multi-scale fusion features.

[0045] u is sent into two linear layers u and u to obtain value vectors (V) and key vectors (K), respectively; similarly, u is sent into a single linear layer u to obtain a query vector (Q).

[0046] The key-value pair information of all positions is aggregated to generate a global context, and the Q is weighted to obtain

[0047] ​The residual connection and the feedforward neural network are used to capture the internal feature dependence of the vector, and final double-branch interaction features Y are obtained; and the mathematical expression of the global feature interaction module is as follows:

[0048]

[0049] Wherein, sim(·) represents a cosine similarity function, LN is a layer normalization function, and FFN is a feedforward neural network using a multilayer perceptron.

[0050] Compared with the prior art, the infrared unmanned aerial vehicle target tracking method with morphology and semantic collaborative perception has the following beneficial effects:

[0051] 1. The morphology and semantic collaborative perception module is proposed, the morphology information perception module and the semantic information perception module are jointly used to enhance the attention of the network to the morphology and semantic information of the infrared unmanned aerial vehicle target, reduce the loss of important information of the low-level feature map, and improve the performance of the infrared unmanned aerial vehicle target tracking network.

[0052] 2. The morphology information perception module is designed, the axial morphology information embedding module, the multi-branch grouping information perception module and the morphology information aggregation module are used to extract and aggregate multi-level context information, so that the tracking network has the ability to capture the morphology information of the infrared unmanned aerial vehicle target.

[0053] 3. The semantic information perception module is designed, the global and local information of the feature channel are integrated to efficiently aggregate the channel feature semantic information; the module can highlight the semantic feature information of the infrared unmanned aerial vehicle target while suppressing irrelevant background information with interference, so as to realize reasonable weight distribution.

[0054] 4. The global feature interaction module is designed, the global cross attention is used to better utilize the template-search branch features to enhance the attention of the network to the position and boundary information of the infrared unmanned aerial vehicle target. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the drawings shown.

[0056] Figure 1 The structure diagram of the infrared unmanned aerial vehicle target tracking method with morphology and semantic collaborative perception is shown in the figure.

[0057] Figure 2 The structure diagram of the infrared unmanned aerial vehicle target tracking network is shown in the figure.

[0058] Figure 3 A schematic diagram of the morphology semantic collaborative perception module structure of the present application;

[0059] Figure 4 A schematic diagram of the morphology information perception module structure of the present application;

[0060] Figure 5 A schematic diagram of the semantic information perception module structure of the present application;

[0061] Figure 6 A schematic diagram of the global feature interaction module structure of the present application. DETAILED DESCRIPTION

[0062] The present application proposes an infrared unmanned aerial vehicle target tracking method based on morphology semantic collaborative perception, aiming to design an infrared unmanned aerial vehicle target tracking method with better tracking precision and accuracy.

[0063] The infrared unmanned aerial vehicle target tracking method based on morphology semantic collaborative perception proposed by the present application will be described below in specific embodiments:

[0064] Embodiment 1:

[0065] An infrared unmanned aerial vehicle target tracking method based on morphology semantic collaborative perception, as shown in Figure 1 , comprising the following steps:

[0066] S1: Prepare a training data set and a test data set;

[0067] S2: Construct an infrared unmanned aerial vehicle target tracking network, comprising:

[0068] Feature extraction network: extract template feature map and search feature map;

[0069] First multi-scale feature fusion network: fuse multi-level template feature maps through a morphology semantic collaborative perception module;

[0070] Second multi-scale feature fusion network: fuse multi-level search feature maps through a morphology semantic collaborative perception module;

[0071] Feature interaction network: fuse template and search feature maps through a global feature interaction module;

[0072] Detection module: detect target position;

[0073] S3: Train the network using the training data set and save the weights;

[0074] S4: Evaluate the network performance using the test data set;

[0075] The morphological semantic collaborative perception module comprises: a morphological information perception module, which obtains target morphological information through axial embedding, multi-branch grouping perception and morphological aggregation; a semantic information perception module, which enhances semantic features through channel interaction and covariance analysis; a convolution layer and an up-sampling layer.

[0076] Further, the morphological information perception module comprises:

[0077] An axial morphological information embedding module, which performs global average pooling along the horizontal and vertical directions to generate one-dimensional feature encoding;

[0078] A multi-branch grouping information perception module, which decomposes the encoding into sub-features and extracts multi-scale features using one-dimensional deep convolution kernels of different sizes;

[0079] A morphological information aggregation module, which concatenates and decouples the multi-scale features to generate a morphological feature map through normalization and excitation operations.

[0080] Further, the axial morphological information embedding module performs:

[0081] Horizontal direction embedding: wherein, represents the output of the encoding on the h height and c channel;

[0082] Vertical direction embedding: wherein, represents the output of the encoding on the w height and c channel.

[0083] Further, the multi-branch grouping information perception module concatenates the two one-dimensional encoding features along the channel direction, and the channel concatenation formula is:

[0084] f=[z h ,z w ];

[0085] wherein, [·,·] represents a connection operation along the spatial dimension, is an intermediate feature map for encoding morphological information in the horizontal and vertical directions;

[0086] f is decomposed into K independent sub-features of the same size along the channel dimension, and the number of channels corresponding to each sub-feature is The feature decomposition formula is:

[0087] f i =Spilt(f);

[0088] wherein, Spilt(g) represents a channel separation operation, represents the i-th sub-feature, and i takes a value in the range of [1, K];

[0089] The K sub-features are subjected to multi-scale feature extraction using one-dimensional deep convolution kernels of different sizes, and the multi-scale extraction formula is:

[0090]

[0091] where g i represents the feature with the i-th level context information, represents a one-dimensional deep convolution kernel applied to the i-th sub-feature, k i represents the size of the convolution kernel, and k i = i x 2 + 1.

[0092] Further, the morphology information aggregation module concatenates the context features of different levels along the channel dimension to obtain

[0093] g = [g1, g2,..., g K ];

[0094] g is decoupled along the spatial dimension into two independent tensors and and the group normalization layer GN is used to normalize g h and g w ; the normalized g h and g w are subjected to nonlinear excitation using the Sigmoid function to obtain:

[0095]

[0096] where σ(g) represents the Sigmoid function; and g and the input feature map are multiplied element by element to obtain a feature map Y with target morphology information; the calculation method of the c-th channel, (i, j) spatial coordinate feature of the feature map Y is:

[0097]

[0098] Further, the semantic information perception module maps the feature containing global spatial information to the channel by two-dimensional global average pooling operation uses a 1x1 two-dimensional convolution and a strip matrix B to respectively interact the global and local channel information of U to obtain a global feature matrix u g and a local feature matrix u l ; the mathematical expression is:

[0099]

[0100] u g = Conv 1×1(U);

[0101]

[0102] where B = [b1, b2, b3, …, b K ] is a band matrix, K is defined as the number of adjacent channel elements;

[0103] The semantic information perception module uses homologous bilinear pooling to multiply u l and the transposed u g to obtain the covariance channel feature matrix m l ; the physical meaning of the i-th row m l is the statistical dependence of the i-th channel of u l on all channels of u g ; the row convolution Conv row is used to adaptively learn the channel dependence in the horizontal direction, and the Sigmoid function is used to perform pixel-level multiplication with u l to obtain local features with global attention u g After the operation, global features with local attention are obtained

[0104] The sum of and is added element by element and multiplied by an input feature through a Sigmoid function to obtain the final channel attention feature map F output :

[0105]

[0106]

[0107] Further, the global feature interaction module converts the multi-scale fusion features and of the template branch and the search branch into one-dimensional feature vectors, and performs regularization processing through layer normalization to obtain and for feature interaction, where k represents the k-th layer of multi-scale fusion features.

[0108] The is input into two linear layers and to obtain value vectors (V) and key vectors (K), respectively; similarly, the is input into a single linear layer to obtain a query vector (Q);

[0109] The key-value pair information of all positions is aggregated to generate global context, which is weighted by Q to obtain

[0110] The final two-branch interaction feature Y is obtained by capturing the intra-vector feature dependency using residual connection and feed-forward neural network; the mathematical expression of the global feature interaction module is as follows:

[0111]

[0112] Wherein, sim(·) represents the cosine similarity function, LN is the layer normalization function, and FFN is the feed-forward neural network using multi-layer perceptron.

[0113] Embodiment 2:

[0114] A morphological semantic collaborative perception infrared unmanned aerial vehicle target tracking method, comprising the following steps:

[0115] S1: preparing a data set, including a data set one for training and a data set two for testing;

[0116] Specifically, the prepared data set one is Anti-UAV data set, which is used for training the infrared unmanned aerial vehicle target tracking network; the prepared data set two is self-made data set, which adds label information to the infrared video sequence for different challenges in the actual scene, and is used for testing the final infrared unmanned aerial vehicle target tracking model;

[0117] S2: constructing an infrared unmanned aerial vehicle target tracking network, including a feature extraction network, a multi-scale feature fusion network one, a multi-scale feature fusion network two, a feature interaction network and a detection module; the feature extraction network is used for extracting template and search feature map; the multi-scale feature fusion network one fuses template feature maps of different levels through a morphological semantic collaborative perception module (SCAM); the multi-scale feature fusion network two fuses search feature maps of different levels through the (SCAM); the feature interaction network fuses template and search feature maps through a global feature interaction module (DFM); and the detection module is used for detecting the position of the infrared unmanned aerial vehicle target;

[0118] As Figure 2As shown, the feature extraction network extracts a template feature map by taking the initial frame of the infrared video sequence as a template image and extracts a search feature map by taking the current frame of the infrared video sequence as a search image in a manner of sharing weights; a first multi-scale feature fusion network takes the template feature map as input, gradually fuses low-level and high-level feature maps along a top-down path through three shape and semantic collaborative perception modules (SCAMs), and obtains template feature maps of different scales. A second multi-scale feature fusion network takes the search feature map as input, gradually fuses low-level and high-level feature maps along a top-down path through three shape and semantic collaborative perception modules (SCAMs), and obtains search feature maps of different scales. The feature interaction network takes the output template and search feature maps of the first and second multi-scale feature fusion networks as input, fuses the template and search feature maps of different scales through three global feature interaction modules (DFMs), and obtains fusion feature maps of different scales; the detection module performs bounding box detection on each fusion feature map, and obtains the final infrared unmanned aerial vehicle target detection result using a non-maximum suppression strategy;

[0119] As shown in Figure 3 , the shape and semantic collaborative perception module includes a shape information perception module, a semantic information perception module, a convolution layer and an up-sampling layer; the shape and semantic collaborative perception module enhances the attention of the network to the shape and semantic information of the infrared unmanned aerial vehicle target by jointly using the shape information perception module and the semantic information perception module, so as to reduce the loss of the shape and semantic information of the infrared unmanned aerial vehicle target;

[0120] Specifically, in the top-down feature fusion process, the shape and semantic collaborative perception module first obtains a feature map with target shape information by passing the low-level feature through the shape information perception module, then focuses on important channel semantic information by passing the feature map through the semantic information perception module, and then aligns the channel dimension by using the convolution layer and adds the high-level feature elements after up-sampling to obtain an output feature with discriminative information of the infrared unmanned aerial vehicle target.

[0121] As shown in Figure 4 , the shape information perception module includes an axial shape information embedding module, a multi-branch grouping information perception module and a shape information aggregation module; the shape information perception module first embeds position shape information in each channel of the input feature along the horizontal direction and the vertical direction respectively by using the axial shape information embedding module to obtain a pair of one-dimensional feature encodings with two-dimensional spatial information; then, the pair of one-dimensional feature encodings are sent to the multi-branch grouping information perception module to capture multi-level context information; finally, the multi-level context information is aggregated by using the shape information aggregation module to obtain a feature map with target shape information. Next, the axial shape information embedding module, the multi-branch grouping information perception module and the shape information aggregation module will be introduced in detail.

[0122] (1) Axial shape information embedding module

[0123] The axial morphology information embedding module embeds the axial morphology information of each channel of the input feature map along the horizontal (x-axis) and vertical (y-axis) directions respectively through a global average pooling kernel in two spatial dimensions, to obtain a pair of one-dimensional feature encodings with two-dimensional spatial information; the information embedding along the horizontal direction is represented as:

[0124]

[0125] wherein, represents the encoded output at the h height and c channel; the information embedding along the vertical direction is represented as:

[0126]

[0127] wherein, represents the encoded output at the w height and c channel;

[0128] (2) Multi-branch grouping information perception module

[0129] The multi-branch grouping information perception module first concatenates the two one-dimensional encoding features along the channel direction to obtain

[0130] f = [z h ,z w ];

[0131] wherein, [·,·] represents a connection operation along the spatial dimension, is an intermediate feature map that encodes morphology information in the horizontal and vertical directions; then, f is decomposed into K independent sub-features of the same size along the channel dimension, and each sub-feature corresponds to a number of channels The process of decomposing f into sub-features is as follows:

[0132] f i = Split(f);

[0133] wherein, Split(g) represents a channel separation operation, represents the i-th sub-feature, and i takes a value in the range of [1, K];

[0134] Next, different sizes of one-dimensional deep convolution kernels are used to perform multi-scale feature extraction on the K sub-features respectively, to capture multi-level context information, and the process is as follows:

[0135]

[0136] wherein, g i represents a feature with i-th level context information, represents a one-dimensional deep convolution kernel applied to the i-th sub-feature, k​i represents the size of the convolution kernel, and k i =i×2+1;

[0137] (3) Morphological information aggregation module

[0138] In order to aggregate multi-level context features without introducing batch statistical noise, the morphological information aggregation module first concatenates the context features of different levels along the channel dimension to obtain

[0139] g=[g1,g2,...,g K ];

[0140] Second, decouple g along the spatial dimension into two independent tensors and And use the group normalization layer GN to g h and g w Normalization operations are performed separately to reduce the semantic interference between sub-features; then, the normalized g is normalized using the Sigmoid function. h and g w Perform nonlinear excitation to obtain

[0141]

[0142] Among them, σ(g) represents the Sigmoid function; finally, And the input feature map Multiply by elements to obtain the feature map Y with target morphological information; the calculation method of the c-th channel and (i, j)-th spatial coordinate feature of the feature map Y is as follows:

[0143]

[0144] like Figure 5 As shown in the figure, the semantic information perception module can highlight the semantic feature information of the infrared UAV target while suppressing the irrelevant background information with interference to achieve reasonable weight distribution; the specific workflow is as follows:

[0145] The semantic information perception module first uses a two-dimensional global average pooling operation to transform the feature map containing global spatial information into Aggregate into channel descriptors Then, a 1×1 two-dimensional convolution and a banded matrix B are used to perform global and local channel information interaction on U to obtain the global feature matrix u g and the local feature matrix u l ; Its mathematical expression is as follows:

[0146]

[0147] u g=Conv 1×1 (U);

[0148]

[0149] Wherein, the band matrix B=[b1,b2,b3,...,b K ], K is defined as the number of adjacent channel elements.

[0150] In order to focus on the global features of the channel in the local features, the semantic information perception module uses homologous bilinear pooling to transform u l and the transposed u g Perform matrix multiplication to obtain the covariance channel feature matrix m l . i-th row m l The physical meaning is u l The i-th channel pair u g The statistical dependencies of all channels. Then, using row convolution Conv row Adaptively learn channel dependency along the horizontal (x-axis) direction and integrate it with u through the Sigmoid function l Perform pixel-level multiplication to obtain local features with global attention Likewise, u g After the above operations, we get the global features with local attention Finally, and Add the elements and multiply them with the input features through a Sigmoid function to get the final channel attention feature map F output :

[0151]

[0152] like Figure 6 As shown in the figure, the global feature interaction module uses global cross attention to better utilize the template-search branch features to enhance the network's attention to the infrared drone's target location and boundary information. The specific workflow is as follows:

[0153] The global feature interaction module combines the multi-scale fusion features of the template branch and the search branch and Converted into a one-dimensional feature vector and regularized by layer normalization to obtain the feature interaction and Among them, k represents the k-th layer multi-scale fusion feature;

[0154] The quadratic computational complexity of the traditional self-attention mechanism leads to an increase in network computation, which seriously affects the network inference speed. To this end, the global feature interaction module uses linear attention to replace multi-head self-attention to meet the real-time tracking of infrared UAV targets. Specifically, two linear layers and , respectively, to obtain value vector (V) and key vector (K); similarly, input into a single linear layer to obtain query vector (Q); then, aggregate the key-value pair information of all positions to generate global context, and weight by Q to obtain Finally, use residual connection and feed-forward neural network to capture the internal feature dependency of the vector to obtain the final two-branch interaction feature Y; the mathematical expression of the global feature interaction module is as follows:

[0155]

[0156] wherein, sim(·) represents the cosine similarity function, LN is the layer normalization function, and FFN is the feed-forward neural network using multi-layer perceptron;

[0157] The detection module adopts an anchor-free detection head to predict the position of the infrared unmanned aerial vehicle target. Specifically, the detection head is composed of three parallel branches, namely a classification branch, a confidence branch, and a regression branch. The first two branches are responsible for determining the area where the target may exist, and the third branch is responsible for predicting the boundary box regression. Given the coordinate position (c x ,c y ) in the feature map and the output (t x ,t y ,t w ,t h ) predicted by the detection head, the boundary box calculation formula is

[0158] b x =(2×σ(t x )-0.5)+c x ,

[0159] b y =(2×σ(t y )-0.5)+c y ,

[0160]

[0161] wherein, (b x ,b y ,b w ,b h ) represents the target center and target size, and σ represents the Sigmoid function.

[0162] S3: training the infrared unmanned aerial vehicle target tracking network; inputting the training data set prepared by S1 into the infrared unmanned aerial vehicle target tracking network constructed by S2 for training, and saving the trained weight; the related settings in the training stage are as follows:

[0163] The optimizer adopts SGD, and the momentum is set to 0.9 and the weight decay is set to 0.0001; the total training rounds and batches are 250 and 16 respectively; each pair of training data is composed of a template and a search frame from the same sequence, the random interval is less than 200, and data augmentation is performed by using random flipping; the network is trained from zero, and no pre-training parameters are loaded; in terms of hardware platform, a NVIDIA GeForce RTX 3090 GPU server accelerated by CUDA 11.1 is selected for network training; in terms of software platform, Python 3.8 and Pytorch1.8.1 are used for implementation; the loss function is defined as

[0164]

[0165] Wherein, N pos represents the number of positive samples, L cls and L obj are focal loss. L reg is GIoU loss. j defines the index of the selected sample, and are the class label and target position respectively, s j is the predicted classification score, t j is the predicted offset, o j is the predicted object confidence score, is an indicator function, which is 1 when , otherwise 0.

[0166] S4: testing the infrared unmanned aerial vehicle target tracking network; inputting the test data set two prepared by S1 into the infrared unmanned aerial vehicle target tracking network trained by S3 for testing; selecting the average state accuracy, success rate and precision as evaluation indexes to verify the effectiveness of the network;

[0167] The related index comparison of the prior art and the method proposed by the present application is shown in Table 1:

[0168] Table 1: Comparison of related indexes of prior art and method proposed by the present application

[0169]

[0170] From the table, it can be seen that the method proposed by the present application has higher average state accuracy, success rate and precision than the prior art, which further illustrates that the method proposed by the present application has better tracking accuracy and accuracy.

[0171] The above description is only preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A morphological and semantic collaborative perception infrared UAV target tracking method, characterized by: The following steps are involved: S1: Prepare training dataset and test dataset; S2: Build an infrared drone target tracking network, including: Feature extraction network: extract template feature map and search feature map; The first multi-scale feature fusion network: fuses multi-level template feature maps through the morphological and semantic collaborative perception module; Second multi-scale feature fusion network: fuses multi-level search feature maps through the morphological and semantic collaborative perception module; Feature interaction network: Fusion of templates and search feature maps through a global feature interaction module; Detection module: detect target position; S3: Train the network using the training dataset and save the weights; S4: Evaluate network performance using the test dataset; Among them, the morphological and semantic collaborative perception module includes: a morphological information perception module: obtaining target morphological information through axial embedding, multi-branch grouping perception and morphological aggregation; a semantic information perception module: enhancing semantic features through channel interaction and covariance analysis; a convolutional layer and an upsampling layer.

2. The method according to claim 1, characterized in that The morphological information perception module includes: Axial morphological information embedding module: performs global average pooling along the horizontal and vertical directions to generate a one-dimensional feature code; Multi-branch grouping information perception module: decomposes the code into sub-features and uses one-dimensional deep convolution kernels of different sizes to extract multi-scale features; Morphological information aggregation module: splices and decouples multi-scale features, and generates morphological feature maps through normalization and excitation operations.

3. The method according to claim 2, characterized in that The axial morphology information embedding module performs: Horizontal embedding: in, Represents the output of the encoding at the hth height and cth channel; Vertical embedding: in, Represents the output of the encoding at the w-th height and c-th channel.

4. The method according to claim 2, characterized in that The multi-branch grouping information perception module: splices the two one-dimensional coding features along the channel direction. The channel splicing formula is: f=[z h ,With w ]; Among them, [·,·] represents the connection operation along the spatial dimension, It is an intermediate feature map that encodes morphological information in the horizontal and vertical directions; Decompose f into K independent sub-features of the same size along the channel dimension, and the number of channels corresponding to each sub-feature is The characteristic decomposition formula is: f i =Game(f); Among them, Spilt(g) represents the special channel separation operation, Represents the i-th sub-feature, and the value range of i is [1,K]; Use one-dimensional deep convolution kernels of different sizes to perform multi-scale feature extraction on K sub-features. The multi-scale extraction formula is: Among them, g i represents the feature with the context information of level i, represents the one-dimensional depth convolution kernel applied to the i-th sub-feature, k i represents the size of the convolution kernel, and k i =i×2+1.

5. The method according to claim 2, characterized in that The morphological information aggregation module stitches together the context features of different levels along the channel dimension to obtain g=[g1,g2,...,g K ]; Decouple g along the spatial dimension into two independent tensors and And use the group normalization layer GN to g h and g w Perform normalization operations respectively; use Sigmoid function to normalize g h and g w Perform nonlinear excitation to obtain: Among them, σ(g) represents the Sigmoid function; And the input feature map Multiply by elements to obtain the feature map Y with target morphological information; the calculation method of the c-th channel and (i, j)-th spatial coordinate feature of the feature map Y is:

6. The method according to claim 1, characterized in that The semantic information perception module uses a two-dimensional global average pooling operation to convert the feature map containing global spatial information into Converge into channels Use a 1×1 two-dimensional convolution and a banded matrix B to perform global and local channel information interaction on U, and obtain the global feature matrix u g and the local feature matrix u l ; Its mathematical expression is: u g =Conv 1×1 (U); Wherein, the band matrix B=[b1,b2,b3,...,b K ], K is defined as the number of adjacent channel elements; The semantic information perception module uses homologous bilinear pooling to transform u l and the transposed u g Perform matrix multiplication to obtain the covariance channel feature matrix m l ; row i m l The physical meaning is u l The i-th channel pair u g Statistical dependencies of all channels; using row convolution Conv row Adaptively learn channel dependency along the horizontal direction and integrate it with u through Sigmoid function l Perform pixel-level multiplication to obtain local features with global attention u g After the operation, we get global features with local attention Will and Add the elements and multiply them with the input features through a Sigmoid function to get the final channel attention feature map F output :

7. The method according to claim 1, characterized in that The global feature interaction module combines the multi-scale fusion features of the template branch and the search branch and Converted into a one-dimensional feature vector and regularized by layer normalization to obtain the feature interaction and Among them, k represents the k-th layer multi-scale fusion feature; Will Feed it into two linear layers and In , we get the value vector (V) and key vector (K) respectively; similarly, Feed into a single linear layer Get the query vector (Q) from Aggregate the key-value pair information of all positions to generate the global context, and weight it by Q to obtain The residual connection and feedforward neural network are used to capture the internal feature dependencies of the vector to obtain the final two-branch interaction feature Y; the mathematical expression of the global feature interaction module is as follows: Where sim(·) represents the cosine similarity function, LN is the layer normalization function, and FFN is a feedforward neural network using a multilayer perceptron.

Images