Target tracking method based on depth estimation network

Abstract

The invention provides a target tracking method based on a depth estimation network, and the method comprises the steps: selecting target three-dimensional pair images of different visual angles under an infrared background, and carrying out the feature extraction and image reconstruction, and obtaining a parallax image; inputting the target three-dimensional pair image pair into a potential difference estimation network model, and outputting image conversion matrixes of different visual angles by the potential difference estimation network model; based on the parallax image and the image conversion matrix, performing distortion mapping on the target stereo pair image in combination with an internal reference matrix of the image acquisition equipment to obtain a distorted image; calculating an error between the distorted image and the target stereo pair image to obtain an error result; continuously iterating the depth estimation network model and the potential difference estimation network model to obtain an optimal image feature; and performing target tracking by using the optimal image features. According to the method, the optimal target features are solved by using the target images at different angles, feature interference caused by an environmental background is overcome, the stability of target tracking is effectively improved, and stable tracking is realized in a complex background interference environment.

Description

Technical Field

[0001] The invention belongs to the technical field of infrared target tracking, and in particular relates to a target tracking method based on a depth estimation network. Background Art

[0002] Reconnaissance equipment plays an increasingly important role in modern society. Especially with the continuous advancement of technology and the continuous expansion of application fields, reconnaissance equipment has been widely used in military, police, civilian and other fields.

[0003] In reconnaissance equipment, stable tracking of targets under complex infrared backgrounds is of great significance to the performance improvement of reconnaissance equipment, and has become a major research direction in the field of target tracking in recent years. With the continuous development of infrared imaging technology and the deepening of research, target tracking under complex infrared backgrounds is greatly affected by the background, and target tracking instability and target loss often occur.

[0004] The tracking stability of targets under infrared background is affected by many factors, including infrared image quality, target characteristics, dynamic environmental changes, and tracking algorithm performance. Among them, infrared image quality includes background complexity and noise interference. In the background complexity, interference factors such as clouds, fog, and ground reflection reduce the signal-to-noise ratio of infrared images and increase the difficulty of target detection and tracking; noise interference refers to the electromagnetic noise and thermal noise that may exist in infrared images, which affect the clarity of the image and thus affect the stability of tracking. Target characteristics include size and intensity, as well as motion state. Size and intensity refer to the small size, low signal-to-noise ratio, and poor contrast of small infrared targets, which make it difficult to accurately identify and track in complex backgrounds; motion state includes changes in the target's movement speed, direction, acceleration, etc., which increase the difficulty of tracking, so the tracking algorithm is required to have strong adaptability and robustness. Dynamic environmental changes include lighting conditions and weather factors. Lighting conditions refer to cloud cover, night environment, etc., which affect the reception of infrared radiation and thus affect image quality; weather factors refer to severe weather conditions such as rain, snow, fog and haze, which significantly reduce the clarity of infrared images and increase tracking difficulties. The performance of tracking algorithms includes algorithm adaptability and computational efficiency. Algorithm adaptability means that the tracking process adapts to target characteristics and dynamic environmental changes to achieve stable tracking; computational efficiency means that the tracking process needs to have high computational efficiency to quickly respond to target changes.

[0005] At present, the existing measures to improve tracking stability include background suppression, target extraction, segmentation algorithm, tracking algorithm and multi-source information fusion. How to further improve the tracking stability of targets under infrared background is a difficult problem that needs to be solved urgently in the current target tracking field. Summary of the invention

[0006] The purpose of the present invention is to solve the shortcomings of the prior art that infrared targets in the existing environment are affected by background interference, resulting in unstable tracking and target loss, and to provide a target tracking method based on a depth estimation network, which utilizes target images at different angles to solve the optimal target features, overcomes the feature interference caused by the environmental background, effectively improves the stability of target tracking, and achieves stable tracking in a complex background interference environment.

[0007] To achieve the above purpose, the technical solution provided by the present invention is:

[0008] The present invention provides a target tracking method based on a depth estimation network, comprising:

[0009] Step 1: Select target stereo images of different viewing angles under an infrared background, input the target stereo images into a depth estimation network model for feature extraction and image reconstruction, and obtain a disparity image;

[0010] Step 2: inputting the target stereo pair image into a disparity estimation network model, and the disparity estimation network model outputs image conversion matrices of different viewing angles;

[0011] Step 3: Based on the disparity image and the image conversion matrix, the target stereo pair image is distorted and mapped in combination with an intrinsic parameter matrix of an image acquisition device to obtain a distorted image;

[0012] Step 4: Calculate the error between the distorted image and the target stereo pair image to obtain an error result; continuously iterate the depth estimation network model and the disparity estimation network model to obtain the optimal image feature; and use the optimal image feature to track the target.

[0013] As a further limitation of the present invention, the step 1 comprises:

[0014] An image acquisition device acquires target images of different viewing angles under an infrared background, and selects stereo pair images of the same target with left and right parallax from the target images to form the target stereo pair images;

[0015] The target stereo pair image is input into an established depth estimation network model, and the depth estimation network model performs feature extraction and image reconstruction.

[0016] As a further limitation of the present invention, the established depth estimation network model includes:

[0017] The depth estimation network model includes a feature extraction network and a feature reconstruction network. The feature extraction network includes a convolutional neural network, a convolutional attention module and a pooling layer. The convolutional neural network and the pooling layer extract features from the target stereo pair image. The convolutional attention module focuses on important features in the target stereo pair image and suppresses unimportant features. Wherein:

[0018] The convolutional attention module includes a channel attention module and a spatial attention module. The convolutional attention module is given an input feature The convolutional attention module calculates the one-dimensional channel attention and 2D spatial attention And they are fused to obtain fused features, and the convolutional attention module outputs the fused features.

[0019] As a further limitation of the present invention, the computational expression of the convolutional attention module is:

[0020]

[0021] Among them, F′ represents the channel attention calculation result value, F″ represents the final calculation value, Indicates element-by-element multiplication;

[0022]

[0023] Where σ represents the sigmoid function, W0 represents the first shared weight of MLP, W1 represents the second shared weight of MLP, f 7×7 Indicates a convolution operation with a convolution kernel of 7×7.

[0024] As a further limitation of the present invention, the disparity estimation network model in step 2 includes an encoder and a decoder, and the encoder includes a convolutional layer;

[0025] The convolution layer inputs the right-view image in the target stereo pair image; the convolution layer also inputs the left-view shallow features extracted by the depth estimation network model;

[0026] The decoder includes a deconvolution layer, which performs decoding mapping to obtain a transformation matrix.

[0027] As a further limitation of the present invention, in step 3, the process of obtaining the distorted image includes:

[0028] The left view image in the disparity image is D l , the right view image is D r ;

[0029] The target pixel in the image conversion matrix is ​​M, and the pixel point in the left perspective image of the target stereo pair image is M l , corresponding to M in the right view image l The pixel point is M r ;

[0030] Let M = D l K-1 M l , M=D r K -1 M r , the pixel point M in the left view image l Projected to the corresponding pixel point M in the right perspective image r , pixel M l and pixel M r Meet M r =KT l-r D l K -1 M l ;

[0031] Using the expression K -1 M r =T l-r D l K -1 M l , D r K -1 M r =D r T l-r D l K -1 M l Deformation, get a distorted image; where: K represents the camera internal parameter, T l-r Represents the image transformation matrix.

[0032] As a further limitation of the present invention, in the step 4, calculating the error between the distorted image and the target stereo pair image to obtain the error result includes:

[0033] The SSIM loss function is used as the loss function of the disparity estimation network model. The SSIM loss function measures the degree of distortion of each image and the degree of similarity between images. The value range of the difference value of the SSIM loss function is set between -1 and 1. The more similar the two images are, the closer the difference value measured by the SSIM loss function is to 1.

[0034] In the step 4, the iterative network is trained by using the minimum SSIM loss between the distorted image pair and the original image pair during continuous iterations.

[0035] As a further limitation of the present invention, the expression of the SSIM loss function is:

[0036]

[0037] Among them, SSIM represents the structural loss metric function, I l Represents the left view image in the target stereo pair image, I′ k represents the distorted left-view image in the distorted image, Ir Represents the original right view image in the target stereo pair image, I′ r represents the distorted right view image in the target stereo pair image, and α represents the coefficient weight of SSIM.

[0038] As a further limitation of the present invention, in step 4, using the optimal image feature to track the target includes:

[0039] The error between the distorted image and the target stereo pair image is calculated by a binary cross entropy loss function, wherein: the binary cross entropy loss function is used as the objective function, and the value of the binary cross entropy loss function is made as small as possible by training the network, the more similar the image features are, the smaller the error function is, and continuous iteration is performed so that the feature extraction network extracts the most critical target;

[0040] For a distorted image pair obtained by mapping the image conversion matrix with the disparity image, an error is calculated between the distorted image pair and the target stereo pair image to obtain an error result; the error result is back-propagated to the disparity estimation network model, and continuously iterated and optimized;

[0041] The error result is also reversely input into the depth estimation network model, and the depth estimation network model and the disparity estimation network model are continuously iterated to finally obtain the optimal image feature in the depth estimation network model under network convergence, and the optimal image feature is used for target tracking.

[0042] As a further limitation of the present invention, the expression of the binary cross entropy loss function is:

[0043]

[0044] Among them, y i is the binary label value of the i-th sample 0 or 1, p(y i =1) is the model's predicted value for the i-th sample.

[0045] The advantages of the present invention are:

[0046] The present invention selects target stereo images under different viewing angles, introduces a depth estimation network model and a disparity estimation network model, and sequentially performs target feature extraction, image reconstruction, error calculation, and continuous iteration and joint optimization. The optimal image features of the depth estimation network model can effectively separate the target object from the background. Using its features as the features of target tracking can effectively improve the stability of target tracking and achieve stable tracking in a complex background interference environment.

[0047] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0049] Figure 1 :A flow chart of a target tracking method based on a depth estimation network provided by the present invention;

[0050] Figure 2 : A flow chart of a target tracking method based on a depth estimation network provided by the present invention;

[0051] Figure 3 : A structural diagram of the depth estimation network model provided by the present invention;

[0052] Figure 4 : A structural diagram of the position difference estimation network model provided by the present invention;

[0053] Figure 5 : A mapping relationship diagram of left and right viewing angle images provided by the present invention;

[0054] Figure 6 : A structural block diagram of the channel attention submodule provided by the present invention;

[0055] Figure 7 : Structural block diagram of the spatial attention submodule provided by the present invention. DETAILED DESCRIPTION

[0056] Embodiments of the present invention are described in detail below. The embodiments are exemplary and intended to be used to explain the present invention, but should not be construed as limiting the present invention.

[0057] See also Figure 1 , an embodiment of the present invention provides a target tracking method based on a depth estimation network, comprising the following steps:

[0058] Step 1. Select a pair of target stereo images at different viewing angles under an infrared background, and input the pair of target stereo images into a depth estimation network model for feature extraction and image reconstruction to obtain a disparity image. Specifically, the embodiment of the present invention inputs a pair of target images at left and right angles into a depth estimation network model, and uses a convolutional layer, a convolutional attention module, and a pooling layer network to extract features. The first two layers of the network use asymmetric convolution kernels to suppress image distortion problems caused by illumination perspective. The features extracted by the feature extraction network are also reconstructed using an upsampling layer and a deconvolution network, and a fully connected layer is used to connect to each deconvolution layer to fuse the features extracted by each layer, thereby improving the quality of the network reconstructed image.

[0059] More specifically, the above-mentioned step one of the embodiment of the present invention includes: an image acquisition device obtains target images of different perspectives under an infrared background, selects stereo pair images of the same target with left and right parallax from the target images, and forms a target stereo pair image; inputs the target stereo pair image into an established depth estimation network model, and the depth estimation network model performs feature extraction and image reconstruction.

[0060] More specifically, the established depth estimation network model in the embodiment of the present invention includes:

[0061] The depth estimation network model includes a feature extraction network and a feature reconstruction network. The feature extraction network includes a convolutional neural network, a convolutional attention module and a pooling layer. The convolutional neural network and the pooling layer extract features from the target stereo pair image. The convolutional attention module focuses on important features in the target stereo pair image and suppresses unimportant features. Among them:

[0062] The convolutional attention module includes a channel attention module and a spatial attention module. The convolutional attention module is given an input feature The convolutional attention module calculates the one-dimensional channel attention and 2D spatial attention The fused features are obtained by fusion, and the convolutional attention module outputs the fused features.

[0063] More specifically, the calculation expression of the convolutional attention module in the embodiment of the present invention is:

[0064]

[0065] Among them, F′ represents the channel attention calculation result value, F″ represents the final calculation value, Indicates element-by-element multiplication;

[0066]

[0067] Where σ represents the sigmoid function, W0 represents the first shared weight of MLP, W1 represents the second shared weight of MLP, f 7×7 Indicates a convolution operation with a convolution kernel of 7×7.

[0068] See also Figure 2 and Figure 3 The depth estimation network model of the embodiment of the present invention includes a feature extraction network and a feature reconstruction network. The feature extraction network is composed of a 4-layer convolutional network, a 4-layer pooling layer, a convolutional attention module, a 4-layer upsampling layer, and a 4-layer deconvolution layer. The input of the depth estimation network model is a left and right stereo pair image. The left and right stereo pair images are as follows: Figure 5The feature extraction network consists of a 4-layer convolutional neural network, a convolutional attention module, and a 4-layer pooling layer. The target image is sequentially subjected to convolution and pooling operations for feature extraction. The attention mechanism is added to improve network performance. The convolutional attention module focuses on important features in the network and suppresses unimportant features, thereby improving the feature network’s ability to extract image features in the target area. Among them, the convolutional attention submodule includes a channel attention submodule and a spatial attention submodule. The channel attention submodule is shown in Figure 6 As shown in Figure 2, the spatial attention submodule is Figure 7 The channel attention submodule is used to determine the important features of the input vector. When calculating the channel attention, the spatial descriptor is generated by pooling. The channel attention submodule is generated by a shared network containing a multi-layer perceptron and a hidden layer. In actual operation, the activation size of the hidden layer is generally set to To reduce network parameters, r is the reduction ratio. The calculation of channel attention first performs maximum pooling and average pooling on the input features, inputs the two pooling results into the multi-layer perceptron, and then sums the outputs of the perceptron.

[0069] The spatial attention submodule makes up for the deficiency of channel attention. Based on the channel attention submodule, the spatial attention submodule applies the convolutional layer operation to the channel attention calculation value to generate the spatial attention map M s (F)∈R H×W Specifically, the maximum pooling feature and average pooling feature on the channel are generated through pooling operation, and the maximum pooling feature and average pooling feature are input into the convolution layer for convolution operation to generate a two-dimensional spatial attention feature map. Spatial attention inputs the channel attention calculation result value F′ into the convolution layer for convolution, and then outputs the spatial attention value through the sigmoid activation layer.

[0070] The feature reconstruction network of the embodiment of the present invention is composed of 4 upsampling layers and 4 deconvolution networks. The extracted target features are input into the feature reconstruction network, and then the target is reconstructed by upsampling deconvolution. The feature extraction network extracts target features under different viewing angles, the feature reconstruction network reconstructs the target image using the extracted features, and the depth estimation network continuously iterates using the Adam gradient descent method. The Adam gradient descent method minimizes the error between the reconstructed target and the original target. The error between the reconstructed image and the original image is used as the network training target. It is continuously iterated to minimize the error between the reconstructed image and the original image to ensure that the extracted features are key and important, thereby inferring the structural information in the image and predicting the depth information of the target object in the image. The reconstructed image at this time is more accurate, and the distorted image pair is obtained by mapping with the viewing angle conversion matrix, which is conducive to the convergence training of the disparity estimation network.

[0071] Step 2: Input the target stereo image pair into the disparity estimation network model, and the disparity estimation network model outputs image conversion matrices of different viewing angles. Specifically, in the embodiment of the present invention, the target image pair at left and right angles is input into the disparity estimation network model. The input of the disparity estimation network model includes the image under the right viewing angle and the shallow features of the left viewing angle extracted by the depth estimation network model, thereby realizing the sharing of some features and increasing the cascade of the two network models; the decoder uses a deconvolution layer to perform decoding mapping to obtain a conversion matrix.

[0072] More specifically, the disparity estimation network model of step two of the embodiment of the present invention includes an encoder and a decoder, the encoder includes a convolutional layer; the convolutional layer inputs the image of the right perspective in the target stereo pair of images; the convolutional layer also inputs the shallow features of the left perspective extracted by the depth estimation network model; the decoder includes a deconvolution layer, and the deconvolution layer performs decoding mapping to obtain a transformation matrix.

[0073] The disparity estimation network model of the embodiment of the present invention belongs to the encoding and decoding network structure. Unlike the deep network, the disparity estimation network outputs a one-dimensional vector, so the network consists of simple convolutional layers and deconvolutional layers. The encoding network consists of 4 convolutional layers, and the left and right perspective image encoding network adopts a weight sharing method to improve the network computing performance. The left and right perspective features extracted by the network are fused in a cascade manner, thereby improving the accuracy of the network model. The cascaded features are input into the disparity network decoder. The disparity decoding network consists of 4 deconvolutional layers. The last layer of the decoder is a dual-branch structure used to predict the left and right perspective conversion matrix; the input of the entire disparity network is a stereo pair image, and the output is a conversion matrix of different perspective images. The perspective conversion matrix learned by the pose estimation network and the disparity image predicted by the depth estimation network are combined with the camera intrinsic parameter matrix to distort the original image pair to obtain a strictly constructed distorted image.

[0074] Step 3: Based on the disparity image and the image conversion matrix, the target stereo pair image is distorted and mapped in combination with the intrinsic parameter matrix of the image acquisition device to obtain a distorted image.

[0075] More specifically, in step 3 of the embodiment of the present invention, the process of obtaining the distorted image includes:

[0076] The left view image in the disparity image is D l , the right view image is D r ;

[0077] The target pixel in the image conversion matrix is ​​M, and the pixel in the left view image of the target stereo pair image is M l , corresponding to M in the right view image l The pixel point is M r ;

[0078] Let M = D l K-1 M l , M=D r K -1 M r , the pixel point M in the left view image l Projected to the corresponding pixel point M in the right perspective image r , pixel M l and pixel M r Meet M r =KT l-r D l K -1 M l ;

[0079] Using the expression K -1 M r =T l-r D l K -1 M l , D r K -1 M r =D r T l-r D l K -1 M l Deformation, get a distorted image; where: K represents the camera internal parameter, T l-r Represents the image transformation matrix.

[0080] Step 4, calculate the error between the distorted image and the target stereo pair image to obtain the error result; the depth estimation network model and the disparity estimation network model are continuously iterated to obtain the optimal image features; the optimal image features are used to track the target. Specifically, the embodiment of the present invention uses the reconstructed stereo pair image, the distorted image pair and the original target stereo pair image to solve the error, and the error is reversely input into the depth estimation network model and the disparity estimation network model, and iterates continuously. The error loss function of the depth estimation network model adopts the binary cross entropy error function, and the disparity estimation network model adopts the SSIM (structural similarity) loss function. Finally, the target features extracted by the depth estimation network when the error minimum network converges are used to track the target.

[0081] More specifically, in step 4 of the embodiment of the present invention, the above-mentioned calculation of the error between the distorted image and the target stereo pair image to obtain the error result includes:

[0082] The SSIM loss function is used as the loss function of the disparity estimation network model. The SSIM loss function measures the degree of distortion of each image and the degree of similarity between images. The difference value of the SSIM loss function ranges from -1 to 1. The more similar the two images are, the closer the difference value measured by the SSIM loss function is to 1.

[0083] In step 4 of the embodiment of the present invention, the iterative network is trained by using the minimum SSIM loss between the distorted image pair and the original image pair during continuous iterations.

[0084] The expression of the SSIM loss function in the embodiment of the present invention is:

[0085]

[0086] Among them, SSIM represents the structural loss metric function, I l Represents the left view image in the target stereo pair image, I′ l It represents the distorted left view image in the distorted image, I r Represents the original right view image in the target stereo pair image, I′ r represents the distorted right view image in the target stereo pair image, and α represents the coefficient weight of SSIM.

[0087] In the above step 4 of the embodiment of the present invention, target tracking is performed using the optimal image features, including:

[0088] (1) The error between the distorted image and the target stereo pair image is calculated by a binary cross entropy loss function, wherein: the binary cross entropy loss function is used as the objective function, and the value of the binary cross entropy loss function is made as small as possible by training the network, and the more similar the image features are, the smaller the error function is, and continuous iteration enables the feature extraction network to extract the most critical target; wherein, the expression of the binary cross entropy loss function of the embodiment of the present invention is:

[0089]

[0090] In the formula, y i is the binary label value of the i-th sample 0 or 1, p(y i =1) is the model's predicted value for the i-th sample.

[0091] (2) For the distorted image pair obtained by mapping the image transformation matrix and the disparity image, the error between the distorted image pair and the target stereo pair image is calculated to obtain the error result; the error result is back-propagated to the disparity estimation network model, and the optimization is continuously iterated.

[0092] (3) The error result is also input back into the depth estimation network model. The depth estimation network model and the disparity estimation network model are continuously iterated to finally obtain the optimal image features in the depth estimation network model under network convergence, and the optimal image features are used for target tracking.

[0093] The Adam gradient descent method update process of the embodiment of the present invention includes:

[0094] Setting parameters: α is the learning rate, β1,β2∈[0,1) is the exponential decay rate of the moment estimate, f(θ) is the random objective function with respect to θ, m t The first moment estimate of the gradient, v t The second moment estimate of the gradient, Bias correction of the first moment, Bias correction of the second-order moment, θ t Parameter update. t =β1·m t-1 +(1-β1)·g t , In the formula, m t is an estimate of the first moment of the gradient, v t It is an estimate of the second moment of the gradient (non-deterministic error).

[0095] In Adam, the bias is offset by calculating the first-order moment estimate and the second-order moment estimate of the bias correction. Get the parameter update formula: Among them: β1 is 0.9, β2 is 0.999, and ε is 10 -8 . The β1 and β2 parameters use the default values ​​of the algorithm, update the parameters, and complete the gradient calculation. In neural networks, the size of the learning rate parameter has a great influence on network performance. A small learning rate will slow down network convergence and increase the time cost. A large learning rate will easily lead to optimal traps. The network model with a learning rate of 0.01 has the best performance.

[0096] The embodiment of the present invention first selects a target stereo pair of images under different viewing angles, introduces a depth estimation network model and a disparity estimation network model, inputs the target stereo pair of images into the depth estimation network model for target feature extraction and image reconstruction, performs error calculation between the reconstructed image and the original image, and then back-propagates the error result to the depth estimation network model for continuous iteration; the disparity estimation network model inputs the target stereo pair of images, and outputs the conversion relationship between the two viewing angles, inputs the conversion relationship between the viewing angles output by the disparity estimation network model into the depth estimation network model, and jointly optimizes the two networks through the error function. At this time, the optimal features of the depth estimation network model can effectively separate the target object from the background, and using its features as features for target tracking can effectively improve the stability of target tracking and achieve stable tracking in a complex background interference environment.

[0097] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be included in the protection scope of the present invention.

Claims

1. A target tracking method based on a depth estimation network, characterized in that: include: Step 1: Select target stereo images of different viewing angles under an infrared background, input the target stereo images into a depth estimation network model for feature extraction and image reconstruction, and obtain a disparity image; Step 2: inputting the target stereo pair image into a disparity estimation network model, and the disparity estimation network model outputs image conversion matrices of different viewing angles; Step 3: Based on the disparity image and the image conversion matrix, the target stereo pair image is distorted and mapped in combination with an intrinsic parameter matrix of an image acquisition device to obtain a distorted image; Step 4: Calculate the error between the distorted image and the target stereo pair image to obtain an error result; The depth estimation network model and the disparity estimation network model are continuously iterated to obtain optimal image features; and the optimal image features are used to track the target.

2. The target tracking method based on the depth estimation network according to claim 1, characterized in that: The step one comprises: An image acquisition device acquires target images of different viewing angles under an infrared background, and selects stereo pair images of the same target with left and right parallax from the target images to form the target stereo pair images; The target stereo pair image is input into an established depth estimation network model, and the depth estimation network model performs feature extraction and image reconstruction.

3. The target tracking method based on the depth estimation network according to claim 2 is characterized in that: The established depth estimation network models include: The depth estimation network model includes a feature extraction network and a feature reconstruction network. The feature extraction network includes a convolutional neural network, a convolutional attention module and a pooling layer. The convolutional neural network and the pooling layer extract features from the target stereo pair image. The convolutional attention module focuses on important features in the target stereo pair image and suppresses unimportant features. Wherein: The convolutional attention module includes a channel attention module and a spatial attention module. The convolutional attention module is given an input feature The convolutional attention module calculates the one-dimensional channel attention and 2D spatial attention And they are fused to obtain fused features, and the convolutional attention module outputs the fused features.

4. The target tracking method based on a depth estimation network according to claim 1, characterized in that: The computational expression of the convolutional attention module is: Among them, F′ represents the channel attention calculation result value, F″ represents the final calculation value, Indicates element-by-element multiplication; Where σ represents the sigmoid function, W0 represents the first shared weight of MLP, W1 represents the second shared weight of MLP, f 7×7 Indicates a convolution operation with a convolution kernel of 7×7.

5. The target tracking method based on a depth estimation network according to claim 1, characterized in that: The disparity estimation network model in step 2 includes an encoder and a decoder, and the encoder includes a convolutional layer; The convolution layer inputs the right-view image of the target stereo pair image; The convolution layer also inputs the shallow features of the left view extracted by the depth estimation network model; The decoder includes a deconvolution layer, which performs decoding mapping to obtain a transformation matrix.

6. The target tracking method based on a depth estimation network according to claim 1, characterized in that: In the step 3, the process of obtaining the distorted image includes: The left view image in the disparity image is D l , the right view image is D r ; The target pixel in the image conversion matrix is ​​M, and the pixel point in the left perspective image of the target stereo pair image is M l , corresponding to M in the right view image l The pixel point is M r ; Let M = D l K -1 M l , M=D r K -1 M r , the pixel point M in the left view image l Projected to the corresponding pixel point M in the right perspective image r , pixel M l and pixel M r Meet M r =KT l-r D l K -1 M l ; Using the expression K -1 M r =T l-r D l K -1 M l , D r K -1 M r =D r T l-r D l K -1 M l Deformation, get a distorted image; where: K represents the camera internal parameter, T l-r Represents the image transformation matrix.

7. The target tracking method based on a depth estimation network according to claim 1, characterized in that: In the step 4, calculating the error between the distorted image and the target stereo pair image to obtain an error result includes: The SSIM loss function is used as the loss function of the disparity estimation network model. The SSIM loss function measures the degree of distortion of each image and the degree of similarity between images. The value range of the difference value of the SSIM loss function is set between -1 and 1. The more similar the two images are, the closer the difference value measured by the SSIM loss function is to 1. In the step 4, the iterative network is trained by using the minimum SSIM loss between the distorted image pair and the original image pair during continuous iterations.

8. The target tracking method based on the depth estimation network according to claim 7, characterized in that: The expression of SSIM loss function is: Among them, SSIM represents the structural loss metric function, I l Represents the left view image in the target stereo pair image, I′ l represents the distorted left-view image in the distorted image, I r Represents the original right view image in the target stereo pair image, I′ r represents the distorted right view image in the target stereo pair image, and α represents the coefficient weight of SSIM.

9. The target tracking method based on a depth estimation network according to claim 1, characterized in that: In the step 4, using the optimal image feature to track the target includes: The error between the distorted image and the target stereo pair image is calculated by a binary cross entropy loss function, wherein: the binary cross entropy loss function is used as the objective function, and the value of the binary cross entropy loss function is made as small as possible by training the network, the more similar the image features are, the smaller the error function is, and continuous iteration is performed so that the feature extraction network extracts the most critical target; For a distorted image pair obtained by mapping the image conversion matrix with the disparity image, an error is calculated between the distorted image pair and the target stereo pair image to obtain an error result; the error result is back-propagated to the disparity estimation network model, and continuously iterated and optimized; The error result is also reversely input into the depth estimation network model, and the depth estimation network model and the disparity estimation network model are continuously iterated to finally obtain the optimal image feature in the depth estimation network model under network convergence, and the optimal image feature is used for target tracking.

10. The target tracking method based on the depth estimation network according to claim 9, characterized in that: The expression of the binary cross entropy loss function is: Among them, y i is the binary label value of the i-th sample 0 or 1, p(y i =1) is the model's predicted value for the i-th sample.

Images