A target tracking method based on depth estimation network

Abstract

The present invention provides a target tracking method based on a depth estimation network, comprising: selecting a target stereo pair image at different perspectives under an infrared background, performing feature extraction and image reconstruction to obtain a disparity image; inputting the target stereo pair image into a disparity estimation network model, the disparity estimation network model outputting an image conversion matrix at different perspectives; warping and mapping the target stereo pair image based on the disparity image and the image conversion matrix in combination with an internal reference matrix of an image acquisition device to obtain a distorted image; calculating the error between the distorted image and the target stereo pair image to obtain an error result; continuously iterating the depth estimation network model and the disparity estimation network model to obtain the optimal image feature; and using the optimal image feature to track the target. The present invention uses target images at different angles to solve for the optimal target feature, overcomes the feature interference caused by the environmental background, effectively improves the stability of target tracking, and achieves stable tracking in a complex background interference environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of infrared target tracking, and in particular relates to a target tracking method based on a depth estimation network. Background Art

[0002] Reconnaissance equipment plays an increasingly important role in modern society. Especially with the continuous advancement of technology and the continuous expansion of application fields, reconnaissance equipment has been widely used in military, police, civilian and other fields.

[0003] In reconnaissance equipment, stable target tracking against complex infrared backgrounds is crucial for improving performance and has become a major research area in recent years. With the continuous development and deepening of infrared imaging technology, target tracking against complex infrared backgrounds is significantly affected by the background, often leading to unstable tracking and even target loss.

[0004] The tracking stability of targets in infrared backgrounds is affected by multiple factors, including infrared image quality, target characteristics, dynamic environmental changes, and tracking algorithm performance. Infrared image quality includes background complexity and noise interference. Background complexity includes interference factors such as clouds, fog, and ground reflections, which reduce the signal-to-noise ratio (SNR) of infrared images and increase the difficulty of target detection and tracking. Noise interference refers to electromagnetic and thermal noise that may be present in infrared images, affecting image clarity and, in turn, tracking stability. Target characteristics include size, intensity, and motion. Size and intensity refer to the small size, low SNR, and poor contrast of small infrared targets, making accurate identification and tracking difficult in complex backgrounds. Motion refers to changes in target speed, direction, and acceleration, which increase tracking difficulty. Therefore, tracking algorithms must possess strong adaptability and robustness. Dynamic environmental changes include lighting conditions and weather factors. Lighting conditions include cloud cover and nighttime conditions, which affect infrared radiation reception and thus affect image quality. Weather factors refer to adverse weather conditions such as rain, snow, fog, and haze, which significantly reduce infrared image clarity and increase tracking difficulties. The performance of tracking algorithms includes algorithm adaptability and computational efficiency. Algorithm adaptability means that the tracking process adapts to target characteristics and dynamic environmental changes to achieve stable tracking; computational efficiency means that the tracking process needs to have high computational efficiency to quickly respond to target changes.

[0005] Current measures to improve tracking stability include background suppression, target extraction, segmentation algorithms, tracking algorithms, and multi-source information fusion. How to further improve the tracking stability of targets under infrared backgrounds is a difficult problem that needs to be solved urgently in the field of target tracking. Summary of the Invention

[0006] The purpose of the present invention is to solve the shortcomings of the existing technology in the existing environment that infrared targets are affected by background interference, resulting in unstable tracking and target loss, and to provide a target tracking method based on a depth estimation network, which uses target images at different angles to solve the optimal target features, overcomes the feature interference caused by the environmental background, effectively improves the stability of target tracking, and achieves stable tracking in a complex background interference environment.

[0007] To achieve the above objectives, the technical solutions provided by the present invention are:

[0008] The present invention provides a target tracking method based on a depth estimation network, comprising:

[0009] Step 1: Select target stereo images of different perspectives under an infrared background, input the target stereo images into a depth estimation network model for feature extraction and image reconstruction, and obtain a disparity image;

[0010] Step 2: inputting the target stereo pair image into a disparity estimation network model, and the disparity estimation network model outputs image conversion matrices of different viewing angles;

[0011] Step 3: Based on the disparity image and the image conversion matrix, the target stereo pair image is distorted and mapped in combination with an intrinsic parameter matrix of an image acquisition device to obtain a distorted image;

[0012] Step 4: Calculate the error between the distorted image and the target stereo pair image to obtain an error result; continuously iterate the depth estimation network model and the disparity estimation network model to obtain the optimal image feature; and use the optimal image feature to track the target.

[0013] As a further limitation of the present invention, the step 1 comprises:

[0014] An image acquisition device acquires target images of different viewing angles under an infrared background, and selects stereo pair images of the same target with left and right parallax from the target images to form the target stereo pair images;

[0015] The target stereo pair image is input into an established depth estimation network model, and the depth estimation network model performs feature extraction and image reconstruction.

[0016] As a further limitation of the present invention, the established depth estimation network model includes:

[0017] The depth estimation network model includes a feature extraction network and a feature reconstruction network. The feature extraction network includes a convolutional neural network, a convolutional attention module, and a pooling layer. The convolutional neural network and the pooling layer extract features from the target stereo pair image. The convolutional attention module focuses on important features in the target stereo pair image and suppresses unimportant features.

[0018] The convolutional attention module includes a channel attention module and a spatial attention module. The convolutional attention module is given input features. The convolutional attention module calculates the one-dimensional channel attention and two-dimensional spatial attention The fused features are obtained by fusing them, and the convolutional attention module outputs the fused features.

[0019] As a further limitation of the present invention, the computational expression of the convolutional attention module is:

[0020]

[0021] Among them, F′ represents the channel attention calculation result value, F″ represents the final calculation value, Indicates element-by-element multiplication;

[0022]

[0023] Where σ represents the sigmoid function, W0 represents the first shared weight of MLP, W1 represents the second shared weight of MLP, f 7×7 Indicates a convolution operation with a convolution kernel of 7×7.

[0024] As a further limitation of the present invention, the disparity estimation network model in step 2 includes an encoder and a decoder, and the encoder includes a convolutional layer;

[0025] The convolution layer inputs the right-view image of the target stereo pair image; the convolution layer also inputs the left-view shallow features extracted by the depth estimation network model;

[0026] The decoder includes a deconvolution layer, which performs decoding mapping to obtain a transformation matrix.

[0027] As a further limitation of the present invention, in step 3, the process of obtaining the distorted image includes:

[0028] The left perspective image in the disparity image is D l , the right view image is D r ;

[0029] The target pixel in the image conversion matrix is ​​M, and the pixel point in the left perspective image of the target stereo pair image is M l , corresponding to M in the right view image l The pixel point is M r ;

[0030] Let M = D l K-1 M l , M=D r K -1 M r , the pixel point M in the left view image l Projected to the corresponding pixel point M in the right perspective image r , pixel M l and pixel M r Meet M r =KT l-r D l K -1 M l ;

[0031] Using the expression K -1 M r =T l-r D l K -1 M l , D r K -1 M r =D r T l-r D l K -1 M l Deformation, resulting in a distorted image; where: K represents the camera internal parameter, T l-r Represents the image transformation matrix.

[0032] As a further limitation of the present invention, in step 4, calculating the error between the distorted image and the target stereo pair image to obtain an error result includes:

[0033] The SSIM loss function is used as the loss function of the disparity estimation network model. The SSIM loss function measures the degree of distortion of each image and the degree of similarity between images. The difference value of the SSIM loss function is set in the range of -1 to 1. The more similar the two images are, the closer the difference value measured by the SSIM loss function is to 1.

[0034] In the step 4, the iterative network is trained by using the minimum SSIM loss between the distorted image pair and the original image pair during continuous iteration.

[0035] As a further limitation of the present invention, the expression of the SSIM loss function is:

[0036]

[0037] Among them, SSIM represents the structural loss metric function, I l Represents the left view image in the target stereo pair image, I′ k Represents the distorted left view image in the distorted image, Ir Represents the original right view image in the target stereo pair image, I′ r represents the distorted right view image in the target stereo pair image, and α represents the coefficient weight of SSIM.

[0038] As a further limitation of the present invention, in step 4, using the optimal image feature to track the target includes:

[0039] Calculating the error between the distorted image and the target stereo pair image using a binary cross entropy loss function, wherein: the binary cross entropy loss function is used as the objective function, and the value of the binary cross entropy loss function is made as small as possible by training the network. The more similar the image features are, the smaller the error function is, and continuous iteration is performed so that the feature extraction network extracts the most critical target;

[0040] For a distorted image pair obtained by mapping the image conversion matrix and the disparity image, calculating an error between the distorted image pair and the target stereo pair image to obtain an error result; backpropagating the error result to the disparity estimation network model, and continuously iterating and optimizing;

[0041] The error result is also reversely input into the depth estimation network model. The depth estimation network model and the disparity estimation network model are continuously iterated to finally obtain the optimal image feature in the depth estimation network model under network convergence, and the optimal image feature is used for target tracking.

[0042] As a further limitation of the present invention, the expression of the binary cross entropy loss function is:

[0043]

[0044] Among them, y i is the binary label value of the i-th sample 0 or 1, p(y i =1) is the model's predicted value for the i-th sample.

[0045] The advantages of the present invention are:

[0046] The present invention selects target stereo images from different perspectives, introduces a depth estimation network model and a disparity estimation network model, and sequentially performs target feature extraction, image reconstruction, error calculation, and continuous iteration and joint optimization. The optimal image features of the depth estimation network model can effectively separate the target object from the background. Using its features as the features of target tracking can effectively improve the stability of target tracking and achieve stable tracking in a complex background interference environment.

[0047] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0049] Figure 1 :Flowchart of a target tracking method based on a depth estimation network provided by the present invention;

[0050] Figure 2 :A flow chart of a target tracking method based on a depth estimation network provided by the present invention;

[0051] Figure 3 : A structural diagram of the depth estimation network model provided by the present invention;

[0052] Figure 4 : A structural diagram of the position difference estimation network model provided by the present invention;

[0053] Figure 5 : The mapping relationship diagram of the left and right perspective images provided by the present invention;

[0054] Figure 6 : A structural block diagram of the channel attention submodule provided by the present invention;

[0055] Figure 7 : Structural block diagram of the spatial attention submodule provided by the present invention. DETAILED DESCRIPTION

[0056] The following describes in detail embodiments of the present invention. The embodiments are exemplary and intended to explain the present invention, but are not to be construed as limiting the present invention.

[0057] See also Figure 1 , an embodiment of the present invention provides a target tracking method based on a depth estimation network, comprising the following steps:

[0058] Step 1. Select a pair of target stereo images at different perspectives under an infrared background, and input the target stereo images into a depth estimation network model for feature extraction and image reconstruction to obtain a disparity image. Specifically, the embodiment of the present invention inputs a pair of target images at left and right angles into a depth estimation network model, and uses a convolutional layer, a convolutional attention module, and a pooling layer network to extract features. The first two layers of the network use asymmetric convolution kernels to suppress image distortion problems caused by illumination perspective. The features extracted by the feature extraction network are also reconstructed using an upsampling layer and a deconvolution network, and a fully connected layer is used to connect to each deconvolution layer to fuse the features extracted by each layer, thereby improving the quality of the network-reconstructed image.

[0059] More specifically, the above-mentioned step one of the embodiment of the present invention includes: the image acquisition device obtains target images of different perspectives under an infrared background, selects stereo pair images of the same target with left and right parallax from the target images, and constitutes a target stereo pair image; inputs the target stereo pair image into an established depth estimation network model, and the depth estimation network model performs feature extraction and image reconstruction.

[0060] More specifically, the depth estimation network model established in the embodiment of the present invention includes:

[0061] The depth estimation network model includes a feature extraction network and a feature reconstruction network. The feature extraction network includes a convolutional neural network, a convolutional attention module, and a pooling layer. The convolutional neural network and the pooling layer extract features from the target stereo image. The convolutional attention module focuses on important features in the target stereo image and suppresses unimportant features.

[0062] The convolutional attention module includes a channel attention module and a spatial attention module. The convolutional attention module is given input features. The convolutional attention module calculates the one-dimensional channel attention and two-dimensional spatial attention The fused features are obtained by fusing them, and the convolutional attention module outputs the fused features.

[0063] More specifically, the calculation expression of the convolutional attention module in the embodiment of the present invention is:

[0064]

[0065] Among them, F′ represents the channel attention calculation result value, F″ represents the final calculation value, Indicates element-by-element multiplication;

[0066]

[0067] Where σ represents the sigmoid function, W0 represents the first shared weight of MLP, W1 represents the second shared weight of MLP, f 7×7 Indicates a convolution operation with a convolution kernel of 7×7.

[0068] See also Figure 2 and Figure 3 The depth estimation network model of the embodiment of the present invention includes a feature extraction network and a feature reconstruction network. The feature extraction network consists of a 4-layer convolutional network, a 4-layer pooling layer, a convolutional attention module, a 4-layer upsampling layer, and a 4-layer deconvolution layer. The input of the depth estimation network model is a left and right stereo pair image. The left and right stereo pair images are as follows: Figure 5As shown in the figure. The feature extraction network consists of a 4-layer convolutional neural network, a convolutional attention module and a 4-layer pooling layer. The target image is sequentially subjected to convolution and pooling operations for feature extraction. The attention mechanism is added to improve network performance. The convolutional attention module focuses on important features in the network and suppresses unimportant features, thereby improving the feature network’s performance in extracting target area image features. Among them, the convolutional attention submodule includes a channel attention submodule and a spatial attention submodule. The channel attention submodule is as follows: Figure 6 As shown in Figure 2, the spatial attention submodule is Figure 7 The channel attention submodule is used to determine the important features of the input vector. When calculating the channel attention, the spatial descriptor is generated by pooling. The channel attention submodule is generated by a shared network consisting of a multi-layer perceptron and a hidden layer. In actual operation, the activation size of the hidden layer is generally set to To reduce network parameters, r is the reduction ratio. The calculation of channel attention first performs maximum pooling and average pooling on the input features, inputs the two pooling results into the multi-layer perceptron, and then sums the outputs of the perceptron.

[0069] The spatial attention submodule makes up for the deficiency of channel attention. Based on the channel attention submodule, the spatial attention submodule applies the convolution layer operation to the channel attention calculation value to generate the spatial attention map M s (F)∈R H×W Specifically, the pooling operation generates the maximum pooling feature and average pooling feature on the channel. The maximum pooling feature and average pooling feature are input into the convolution layer for convolution operation to generate a two-dimensional spatial attention feature map. Spatial attention inputs the channel attention calculation result value F′ into the convolution layer for convolution, and then passes through the sigmoid activation layer to output the spatial attention value.

[0070] The feature reconstruction network of the embodiment of the present invention is composed of 4 layers of upsampling layers and 4 layers of deconvolution networks. The extracted target features are input into the feature reconstruction network, and then the target is reconstructed through upsampling deconvolution. The feature extraction network extracts target features under different perspectives, and the feature reconstruction network reconstructs the target image using the extracted features. The depth estimation network continuously iterates using the Adam gradient descent method. The Adam gradient descent method minimizes the error between the reconstructed target and the original target. The error between the reconstructed image and the original image is used as the network training target. Continuous iteration minimizes the error between the reconstructed image and the original image to ensure that the extracted features are key and important, thereby inferring the structural information in the image and predicting the depth information of the target object in the image. The reconstructed image at this time is more accurate, and the distorted image pair is obtained by mapping with the perspective conversion matrix, which is conducive to the convergence training of the disparity estimation network.

[0071] Step 2: Input the target stereo image pair into the disparity estimation network model, which outputs image conversion matrices for different perspectives. Specifically, in this embodiment of the present invention, the target image pair from left and right angles is input into the disparity estimation network model. The input of the disparity estimation network model includes the image from the right perspective and the shallow features of the left perspective extracted by the depth estimation network model, which enables the sharing of some features and increases the cascade of the two network models. The decoder uses a deconvolution layer to perform decoding mapping to obtain a conversion matrix.

[0072] More specifically, the disparity estimation network model of step two of the embodiment of the present invention includes an encoder and a decoder, the encoder includes a convolutional layer; the convolutional layer inputs the image of the right perspective in the target stereo pair image; the convolutional layer also inputs the shallow features of the left perspective extracted by the depth estimation network model; the decoder includes a deconvolution layer, and the deconvolution layer performs decoding mapping to obtain a transformation matrix.

[0073] The disparity estimation network model of the embodiment of the present invention belongs to the encoding and decoding network structure. Unlike the deep network, the disparity estimation network output is a one-dimensional vector, so the network consists of simple convolutional layers and deconvolutional layers. The encoding network consists of 4 convolutional layers, and the left and right perspective image encoding network adopts a weight sharing method to improve the network computing performance. The left and right perspective features extracted by the network are fused in a cascade manner, thereby improving the accuracy of the network model. The cascaded features are input into the disparity network decoder. The disparity decoding network consists of 4 deconvolution layers. The last layer of the decoder is a dual-branch structure used to predict the left and right perspective conversion matrix; the input of the entire disparity network is a stereo pair image, and the output is a conversion matrix of images with different perspectives. The perspective conversion matrix learned by the pose estimation network and the disparity image predicted by the depth estimation network are combined with the camera intrinsic parameter matrix to perform distorted mapping on the original image pair, thereby obtaining a strictly constructed distorted image.

[0074] Step 3: Based on the disparity image and the image conversion matrix, the target stereo pair image is distorted and mapped in combination with the intrinsic parameter matrix of the image acquisition device to obtain a distorted image.

[0075] More specifically, in step 3 of the embodiment of the present invention, the process of obtaining the distorted image includes:

[0076] The left view image in the disparity image is D l , the right view image is D r ;

[0077] The target pixel in the image conversion matrix is ​​M, and the pixel point in the left view image of the target stereo pair image is M l , corresponding to M in the right view image l The pixel point is M r ;

[0078] Let M = D l K-1 M l , M=D r K -1 M r , the pixel point M in the left view image l Projected to the corresponding pixel point M in the right perspective image r , pixel M l and pixel M r Meet M r =KT l-r D l K -1 M l ;

[0079] Using the expression K -1 M r =T l-r D l K -1 M l , D r K -1 M r =D r T l-r D l K -1 M l Deformation, resulting in a distorted image; where: K represents the camera internal parameter, T l-r Represents the image transformation matrix.

[0080] Step 4. Calculate the error between the distorted image and the target stereo pair image to obtain the error result; the depth estimation network model and the disparity estimation network model are continuously iterated to obtain the optimal image features; and the optimal image features are used to track the target. Specifically, the embodiment of the present invention uses the reconstructed stereo pair image, the distorted image pair and the original target stereo pair image to solve the error, and the error is reversely input into the depth estimation network model and the disparity estimation network model, and continuously iterates. The error loss function of the depth estimation network model adopts the binary cross entropy error function, and the disparity estimation network model adopts the SSIM (structural similarity) loss function. Finally, the target features extracted by the depth estimation network when the error minimum network converges are used to track the target.

[0081] More specifically, in step 4 of the embodiment of the present invention, calculating the error between the distorted image and the target stereo pair image to obtain the error result includes:

[0082] The SSIM loss function is used as the loss function of the disparity estimation network model. The SSIM loss function measures the degree of distortion of each image and the degree of similarity between images. The difference value of the SSIM loss function ranges from -1 to 1. The more similar the two images are, the closer the difference value measured by the SSIM loss function is to 1.

[0083] In step 4 of the embodiment of the present invention, the iterative network is trained by using the minimum SSIM loss between the distorted image pair and the original image pair during continuous iteration.

[0084] The expression of the SSIM loss function in the embodiment of the present invention is:

[0085]

[0086] Among them, SSIM represents the structural loss metric function, I l Represents the left view image in the target stereo pair image, I′ l Indicates the distorted left view image in the distorted image, I r Represents the original right view image in the target stereo pair image, I′ r represents the distorted right view image in the target stereo pair image, and α represents the coefficient weight of SSIM.

[0087] In the above step 4 of the embodiment of the present invention, target tracking is performed using the optimal image features, including:

[0088] (1) The error between the distorted image and the target stereo pair image is calculated by a binary cross entropy loss function, wherein: the binary cross entropy loss function is used as the objective function, and the value of the binary cross entropy loss function is made as small as possible by training the network. The more similar the image features are, the smaller the error function is. Continuous iteration enables the feature extraction network to extract the most critical target. The expression of the binary cross entropy loss function of the embodiment of the present invention is:

[0089]

[0090] Where y i is the binary label value of the i-th sample 0 or 1, p(y i =1) is the model's predicted value for the i-th sample.

[0091] (2) For the distorted image pair obtained by mapping the image transformation matrix and the disparity image, the error between the distorted image pair and the target stereo pair image is calculated to obtain the error result; the error result is back-propagated to the disparity estimation network model and continuously iterated and optimized.

[0092] (3) The error result is also reversely input into the depth estimation network model. The depth estimation network model and the disparity estimation network model are continuously iterated to finally obtain the optimal image features in the depth estimation network model under network convergence, and the optimal image features are used for target tracking.

[0093] The Adam gradient descent method update process of the embodiment of the present invention includes:

[0094] Setting parameters: α learning rate, β1,β2∈[0,1) moment estimate exponential decay rate, f(θ) random objective function with respect to θ, m t The first moment estimate of the gradient, v t The second moment estimate of the gradient, Bias correction of the first moment, Bias correction for the second moment, θ t Parameter update. m t =β1·m t-1 +(1-β1)·g t , Where m t is an estimate of the first moment of the gradient, v t It is an estimate of the second moment of the gradient (non-deterministic error).

[0095] In Adam, the deviation is offset by calculating the first-order moment estimate and the second-order moment estimate of the deviation correction. Get the parameter update formula: Among them: β1 is 0.9, β2 is 0.999, and ε is 10 -8 The β1 and β2 parameters use the algorithm's default values, perform parameter updates, and complete the gradient calculation. In neural networks, the learning rate parameter has a significant impact on network performance. A small learning rate results in slow convergence and a high time cost, while a large learning rate can easily lead to optimality traps. A learning rate of 0.01 yields the best network model performance.

[0096] The embodiment of the present invention first selects a target stereo pair of images under different perspectives, introduces a depth estimation network model and a disparity estimation network model, inputs the target stereo pair of images into the depth estimation network model for target feature extraction and image reconstruction, calculates the error between the reconstructed image and the original image, and then backpropagates the error result to the depth estimation network model for continuous iteration; the disparity estimation network model inputs the target stereo pair of images, and outputs the conversion relationship between the two perspectives, inputs the conversion relationship between the perspectives output by the disparity estimation network model into the depth estimation network model, and jointly optimizes the two networks through the error function. At this time, the optimal features of the depth estimation network model can effectively separate the target object from the background, and using its features as features for target tracking can effectively improve the stability of target tracking and achieve stable tracking in a complex background interference environment.

[0097] The above description is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with this technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present invention, and these modifications or replacements should all be included in the scope of protection of the present invention.

Claims

1. A target tracking method based on a depth estimation network, characterized in that: include: Step 1: Select target stereo images of different perspectives under an infrared background, input the target stereo images into a depth estimation network model for feature extraction and image reconstruction, and obtain a disparity image; Step 2: inputting the target stereo pair image into a disparity estimation network model, and the disparity estimation network model outputs image conversion matrices of different viewing angles; Step 3: Based on the disparity image and the image conversion matrix, the target stereo pair image is distorted and mapped in combination with an intrinsic parameter matrix of an image acquisition device to obtain a distorted image; Step 4: Calculate the error between the distorted image and the target stereo pair image to obtain an error result; The depth estimation network model and the disparity estimation network model are continuously iterated to obtain optimal image features; and the optimal image features are used to track the target.

2. The target tracking method based on a depth estimation network according to claim 1, characterized in that: The step one comprises: An image acquisition device acquires target images of different viewing angles under an infrared background, and selects stereo pair images of the same target with left and right parallax from the target images to form the target stereo pair images; The target stereo pair image is input into an established depth estimation network model, and the depth estimation network model performs feature extraction and image reconstruction.

3. The target tracking method based on a depth estimation network according to claim 2, characterized in that: Established depth estimation network models include: The depth estimation network model includes a feature extraction network and a feature reconstruction network. The feature extraction network includes a convolutional neural network, a convolutional attention module, and a pooling layer. The convolutional neural network and the pooling layer extract features from the target stereo pair image. The convolutional attention module focuses on important features in the target stereo pair image and suppresses unimportant features. The convolutional attention module includes a channel attention module and a spatial attention module. The convolutional attention module is given input features. The convolutional attention module calculates the one-dimensional channel attention and two-dimensional spatial attention The fused features are obtained by fusing them, and the convolutional attention module outputs the fused features.

4. The target tracking method based on a depth estimation network according to claim 1, characterized in that: The calculation expression of the convolutional attention module is: Among them, F′ represents the channel attention calculation result value, F″ represents the final calculation value, Indicates element-by-element multiplication; Where σ represents the sigmoid function, W0 represents the first shared weight of MLP, W1 represents the second shared weight of MLP, f 7×7 Indicates a convolution operation with a convolution kernel of 7×7.

5. The target tracking method based on a depth estimation network according to claim 1, characterized in that: The disparity estimation network model in step 2 includes an encoder and a decoder, and the encoder includes a convolutional layer; The convolutional layer inputs the right-view image of the target stereo pair image; The convolutional layer also inputs the left-view shallow features extracted by the depth estimation network model; The decoder includes a deconvolution layer, which performs decoding mapping to obtain a transformation matrix.

6. The target tracking method based on a depth estimation network according to claim 1, characterized in that: In step 3, the process of obtaining the distorted image includes: The left perspective image in the disparity image is D l , the right view image is D r ; The target pixel in the image conversion matrix is ​​M, and the pixel point in the left perspective image of the target stereo pair image is M l , corresponding to M in the right view image l The pixel point is M r ; Let M = D l K -1 M l , M=D r K -1 M r , the pixel point M in the left view image l Projected to the corresponding pixel point M in the right perspective image r , pixel M l and pixel M r Meet M r =KT l-r D l K -1 M l ; Using the expression K -1 M r =T l-r D l K -1 M l , D r K -1 M r =D r T l-r D l K -1 M l Deformation, resulting in a distorted image; where: K represents the camera internal parameter, T l-r Represents the image transformation matrix.

7. The target tracking method based on a depth estimation network according to claim 1, characterized in that: In the step 4, calculating the error between the distorted image and the target stereo pair image to obtain an error result includes: The SSIM loss function is used as the loss function of the disparity estimation network model. The SSIM loss function measures the degree of distortion of each image and the degree of similarity between images. The difference value of the SSIM loss function is set in the range of -1 to 1. The more similar the two images are, the closer the difference value measured by the SSIM loss function is to 1. In the step 4, the iterative network is trained by using the minimum SSIM loss between the distorted image pair and the original image pair during continuous iteration.

8. The target tracking method based on a depth estimation network according to claim 7, characterized in that: The expression of the SSIM loss function is: Among them, SSIM represents the structural loss metric function, I l Represents the left view image in the target stereo pair image, I′ l represents the distorted left view image in the distorted image, I r Represents the original right view image in the target stereo pair image, I′ r represents the distorted right view image in the target stereo pair image, and α represents the coefficient weight of SSIM.

9. The target tracking method based on a depth estimation network according to claim 1, characterized in that: In the fourth step, target tracking is performed using the optimal image features, including: Calculating the error between the distorted image and the target stereo pair image using a binary cross entropy loss function, wherein: the binary cross entropy loss function is used as the objective function, and the value of the binary cross entropy loss function is made as small as possible by training the network. The more similar the image features are, the smaller the error function is, and continuous iteration is performed so that the feature extraction network extracts the most critical target; For a distorted image pair obtained by mapping the image conversion matrix and the disparity image, calculating an error between the distorted image pair and the target stereo pair image to obtain an error result; backpropagating the error result to the disparity estimation network model, and continuously iterating and optimizing; The error result is also reversely input into the depth estimation network model. The depth estimation network model and the disparity estimation network model are continuously iterated to finally obtain the optimal image feature in the depth estimation network model under network convergence, and the optimal image feature is used for target tracking.

10. The target tracking method based on depth estimation network according to claim 9, characterized in that: The expression of the binary cross entropy loss function is: Among them, y i is the binary label value of the i-th sample 0 or 1, p(y i =1) is the model's predicted value for the i-th sample.

Images