Self-supervision monocular depth estimation method for non-motor vehicle scene

Abstract

The invention relates to the technical field of depth estimation, and discloses a non-motor vehicle scene-oriented self-supervised monocular depth estimation method, and the training process of an estimation model comprises the following steps: a motion estimation step: carrying out camera motion track estimation and track smoothing according to a video frame shot by a camera in the driving process of a non-motor vehicle; a depth estimation step: extracting multi-scale local features of the video frames shot by the camera to realize depth map prediction; according to the method, camera motion estimation and motion smoothing technologies are introduced, and interference caused by camera shaking is effectively reduced by predicting the motion state of the camera between video frames and combining motion smoothing processing; according to the method, a lightweight and efficient convolutional network architecture is adopted, rapid and robust depth estimation is realized, and the method can be deployed in a non-motor vehicle scene with limited computer resources and high real-time performance requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of depth estimation, and in particular to a self-supervised monocular depth estimation method for non-motor vehicle scenes. Background Art

[0002] At present, the depth estimation method based on deep learning has made significant progress. Among them, the method based on supervised learning usually requires a large data set with real depth annotation to train the depth estimation model. For example, the patent document with application publication number CN118229750A discloses a monocular image depth estimation method based on supervised learning, which uses a mixed different source depth data set for training to obtain depth estimation information. However, the acquisition and annotation process of depth data is expensive and cumbersome, which greatly limits the application of depth estimation methods based on supervised learning. Unlike the method of directly using depth labels as supervisory signals, self-supervised methods use continuous video frames or binocular images to provide indirect supervisory signals for network training. In actual scenes, monocular cameras have the advantages of low price and easy deployment compared to stereo cameras or lidars. Therefore, the use of monocular images without depth annotation to generate scene depth information is the task of self-supervised monocular depth estimation.

[0003] In the task of self-supervised monocular depth estimation, a basic method is to predict the disparity map from the source view through the monocular depth estimation model, reconstruct the target view based on the predicted disparity map and the source view, and use the reconstruction error between the synthetic target view and the real target view as a constraint to optimize the training process of the depth estimation model, and finally use the camera parameters and the predicted disparity map to further calculate the corresponding depth map. For example, the patent document with application publication number CN116245927B discloses a self-supervised monocular depth estimation method for unstructured monocular video, generates an initial disparity map and a depth map through a depth module, and then predicts the rotation and translation matrix from two consecutive video frames through a pose network, and finally generates a reconstructed depth image. In non-motor vehicle scenes, self-supervised monocular depth estimation technology can be used to obtain the depth information of the surrounding environment, effectively identify and understand complex road scenes, and serve as an auxiliary system for the intelligent driving of non-motor vehicles. However, compared with motor vehicle scenes, non-motor vehicles will experience more bumps or instability during driving, resulting in blurred imaging of the front monocular camera, and continuous jitter will lead to the accumulation of depth estimation errors, further affecting the final depth prediction effect. Therefore, in the self-supervised monocular depth estimation method, it is crucial to study how to solve the lens stabilization problem in non-motor vehicle scenes. Summary of the invention

[0004] In order to solve the lens stability problem during the driving of non-motor vehicles, and further improve the prediction accuracy and robustness of the model in self-supervised monocular depth estimation, the present invention discloses a self-supervised monocular depth estimation method for non-motor vehicle scenes. First, the present invention uses a new motion estimation module to estimate the camera motion during the driving of non-motor vehicles and perform motion smoothing, aiming to reduce the interference of lens jitter on the generation of target depth images; secondly, the present invention uses a lightweight depth estimation module to obtain the global and local context information of the input video frame and generate a multi-scale depth image; finally, the present invention combines the smoothed motion estimation to reconstruct a stable target depth image, aiming to improve the accuracy and reliability of depth estimation.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0006] A self-supervised monocular depth estimation method for non-motor vehicle scenes, wherein the video captured by the camera is input into a trained estimation model, and the corresponding depth map is output; the training process of the estimation model includes:

[0007] Motion estimation step: The camera motion trajectory is estimated and the trajectory is smoothed according to the video captured by the camera during the non-motor vehicle driving process, which specifically includes:

[0008] Keypoint estimation, which is used to determine keypoints in video frames and calculate motion vectors between adjacent video frames;

[0009] Motion trajectory estimation, using a multi-homography estimation strategy to generate the initial camera motion trajectory based on the motion vectors between adjacent video frames;

[0010] Trajectory smoothing, which smoothes the initial camera motion trajectory to generate stable video frames;

[0011] Depth estimation step: Extract multi-scale local features from the video frames captured by the camera to achieve depth map prediction, including:

[0012] Deep coding, using a lightweight backbone network to extract multi-scale local features from input video frames;

[0013] Local-global feature encoding: randomly generated global features are encoded into local features through the self-attention mechanism to obtain multi-scale feature maps;

[0014] Depth decoding, generating a multi-scale depth map of the input video frame based on the multi-scale feature map;

[0015] The estimation model is trained based on the multi-scale depth map generated in the depth estimation step and a loss function consisting of reconstruction loss and adversarial loss; the reconstruction loss is used to ensure the similarity between the generated stable video frames and the original video frames; the adversarial loss is used to optimize the visual quality of the generated video.

[0016] Furthermore, the key point estimation specifically includes:

[0017] Use the PWCNet model to extract the optical flow between adjacent video frames i :

[0018] OpticalFlow i =PWCNet(f i ,f i+1 )

[0019] Among them, f i represents the i-th video frame of the input video, E represents the total number of video frames, PWCNet(·) represents the PWCNet model; OpticalFlow between adjacent video frames i Used for motion vector generation;

[0020] Use the pre-trained multi-scale shallow network RFNet to extract key points from video frames:

[0021] p ij =RFNet(fi i )

[0022] Among them, p ij represents the video frame f i The jth key point detected on L represents the total number of key points detected; the motion vector V between adjacent video frames is calculated by the optical flow algorithm. i→i+1 , V i→i+1 Describes the pixel displacement of key points between adjacent video frames:

[0023] V i→i+1 =OpticalFlow i (p ij ).

[0024] Furthermore, the motion trajectory estimation specifically includes:

[0025] Each video frame is evenly divided into a set of grids so that the motion vector of each key point can be transferred to the corresponding grid vertices around it, and multi-homography estimation is used to predict the initial camera motion trajectory:

[0026]

[0027] p ij represents the video frame f i The jth key point detected on , L represents the total number of key points detected, Represents the initial camera motion trajectory, and MultiHomo represents multi-homography estimation.

[0028] Furthermore, the trajectory is smoothed, specifically including:

[0029] The motion vector V between adjacent video frames is smoothed by a smoother composed of multiple 3D convolutional layers. i→i+1 Smoothing is performed to eliminate errors caused by noise and jitter to generate a smooth camera motion trajectory P i :

[0030] P i =P i-1 +α(V i→i+1 -β);

[0031] Here, α is a scaling factor and β is a smoothing parameter used to reduce noise.

[0032] Further, the depth coding includes four stages;

[0033] In the first stage of deep coding, the input video frame is processed by the convolutional backbone network, downsampled by using a 3×3 ConvNeXtv2 convolution layer, and image features are extracted by a 3×3 convolution with a stride of 1. The output size is Local features of

[0034] In the second stage of deep coding, the local features output by the first stage of deep coding are concatenated with the pooled three-channel video frame and further downsampled using a 3×3 ConvNeXtv2 convolutional layer with a stride of 2, and the output size is The local features of the image are further mined through continuous dilated convolution and local-global feature interaction modules.

[0035] The third and fourth stages of depth coding use the same processing methods as the first and second stages of depth coding, and obtain the sizes of The local features and local features.

[0036] Furthermore, the depth decoding specifically includes:

[0037] The size of the multi-scale feature map is gradually restored through three bilinear upsampling layers, and the local features output from the second, third and fourth stages of deep coding are connected in series using convolutional layers to achieve the fusion of features of different resolutions; the prediction head connected after each upsampling block outputs full resolution, Resolution and The low-resolution feature map is converted into a high-resolution depth map.

[0038] Furthermore, the local-global feature encoding specifically includes:

[0039]

[0040] Among them, Attention(Q,K,V)=V·Softmax(Q T ·K), Attenton(·) is the attention mechanism function, Linear G is a point-level convolution operation, Linear represents a linear transformation layer that performs linear mapping on the input data, LN is a normalization operation; Q is a query vector, which represents the features or information that need to be paid attention to at present; K is a key vector, which represents the feature representation of each element in the input sequence; V is a value vector, which contains the actual information corresponding to the key; X is an input feature, which represents the output or input sequence from the previous layer; is the updated feature matrix, which represents the output after being processed by the attention mechanism; Represents the features after normalization and weighting operations.

[0041] Furthermore, the loss function L is composed of the reconstruction loss L recon and adversarial loss L adv composition;

[0042] Reconstruction loss L recon Used to ensure the similarity between the generated stabilized video frames and the original video frames:

[0043]

[0044] in, is the stabilized video frame generated by the estimation model, and T represents the number of video frames;

[0045] Adversarial loss L adv Used to optimize the visual quality of generated videos:

[0046]

[0047] in, represents the expected value of the real sample, represents the expected value of the generated sample, D(x) is the discriminator, which can distinguish between real video frames and generated video frames;

[0048] The loss function L is:

[0049] L=L recon +λL adv

[0050] Here, λ is a weight parameter used to balance the reconstruction loss and the adversarial loss.

[0051] Compared with the prior art, the beneficial technical effects of the present invention are:

[0052] The present invention discloses a self-supervised monocular depth estimation method for non-motor vehicle scenes. First, during the training period, the method only needs to input video sequence frames, and automatically generates a stable target depth map through a self-supervised learning mechanism, which effectively avoids the complex and expensive data collection and annotation process required in traditional depth estimation methods, greatly reduces the implementation cost and time overhead, and is suitable for non-motor vehicle scenes where data is scarce and annotation is difficult. Secondly, the present invention introduces camera motion estimation and motion smoothing technology, which effectively reduces the interference caused by camera shake by predicting the motion state of the camera between video frames and combining motion smoothing processing. Finally, the present invention adopts a lightweight and efficient convolutional network architecture to achieve fast and robust depth estimation, which can be deployed in non-motor vehicle scenes with limited computer resources and high real-time performance requirements. The present invention has good performance in the depth estimation task of non-motor vehicle scenes and has strong practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of the process of the self-supervised monocular depth estimation method in the present invention;

[0054] Figure 2 Schematic diagram of the output result of the self-supervised monocular depth estimation method in the present invention. DETAILED DESCRIPTION

[0055] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0056] The self-supervised monocular depth estimation method for non-motor vehicle scenes in the present invention has a detailed framework and structure as follows: Figure 1As shown. The method aims to effectively deal with the problem of decreased depth estimation accuracy caused by camera shake during dynamic driving of non-motor vehicles, and has significant research value and practical application potential. Specifically, the present invention first introduces a new motion estimation module to estimate the camera motion trajectory during the driving process of non-motor vehicles and realize trajectory smoothing, so as to accurately model the image, reduce the impact of camera shake caused by bumpy roads or unstable driving on image quality, and improve the stability and reliability of the input data for subsequent depth estimation; then the present invention extracts feature information of different scales from the input video frame, captures rich depth features in a variety of scenes, and generates an initial multi-scale depth map; finally, combined with camera motion estimation, a stable target depth image is reconstructed, thereby improving the depth perception ability of non-motor vehicles under complex traffic conditions. The method comprises three parts: a motion estimation module, a depth estimation module, and a loss function.

[0057] The motion estimation module can estimate jitter and generate a smooth trajectory by calculating the motion vector between frames. By defining a square grid on the video frame to divide the video frame into multiple grid areas, the motion vector of each feature point can be effectively transferred to the corresponding grid vertex around it. This process ensures that a unique motion vector is generated for each grid vertex at each moment to accurately capture and analyze the camera's motion dynamics, thereby providing the necessary motion information for subsequent depth estimation and improving the stability and reliability of the depth estimation process. As time goes by, each vertex of the grid continuously collects motion vectors from different moments. After smoothing, these vectors finally achieve stable and accurate camera motion estimation. The motion estimation module includes three steps: key point estimation, motion estimation, and trajectory smoothing.

[0058] Furthermore, the key point estimation refers to analyzing the video frames and calculating the optical flow information of each key point to identify the key features. Optical flow is a vector field that describes the movement of objects in an image, indicating the change in pixel positions in adjacent image frames. Use PWCNet to extract the optical flow between adjacent frames. i :

[0059] OpticalFlow i =PWCNet(f i ,f i+1 )

[0060] Among them, f i represents the i-th frame of the input video, , E represents the number of frames. Extract the optical flow between adjacent frames i Used for motion vector generation.

[0061] The pre-trained multi-scale shallow network structure RFNet is used to efficiently and robustly select key points from high-resolution video frames, extract local and global features, and track the motion vectors of these features in consecutive frames. ij To represent the key point position:

[0062] p ij =RFNet(f i );

[0063] Among them, p ij represents the video frame f i The jth key point detected on L represents the number of key points detected. The motion vector V between frames is calculated by the optical flow algorithm. i→i+1 Describes the pixel displacement of key points between adjacent video frames, the motion vector V i→i+1 The expression is:

[0064] V i→i+1 =OpticalFlow i (p ij ).

[0065] Furthermore, the motion estimation refers to using a multi-homography estimation strategy based on the motion vector V of the key point. i→i+1 To generate an initial camera motion estimate. Each video frame is evenly divided into a set of fixed grids, so that the motion vector of each key point can be effectively transferred to the corresponding grid vertices around it, thereby predicting the motion based on the grid vertices. Use MultiHomo to predict the initial motion estimate:

[0066]

[0067] Furthermore, the trajectory smoothing refers to generating a smooth motion trajectory P based on the extracted features and motion information. i That is, the smoother composed of multiple 3D convolutional layers collects the key point-based motion vector V i→i+1 Smoothing is performed. This process eliminates errors caused by noise and jitter and improves the stability and accuracy of camera motion estimation. The stabilization process can be expressed as:

[0068] P i =P i-1 +α(V i→i+1 -β);

[0069] Where α is a scaling factor and β is a smoothing parameter used to reduce noise. This formula ensures the generation of smooth motion.

[0070] The depth estimation module extracts multi-scale local features from the input video frame sequence to achieve high-precision depth map prediction. First, a lightweight backbone network is used to extract rich multi-scale local features from the input video frame, and then the randomly generated global features are encoded into the local features through the self-attention mechanism, and finally a multi-scale depth map of the input video frame is generated. This process is mainly divided into three key steps: depth encoding, local-global feature encoding, and depth decoding.

[0071] Furthermore, the deep coding is subdivided into four stages to ensure that multi-scale feature information is extracted from the input video frame. In the first stage, after being processed by the convolutional backbone network, downsampling is performed using a 3×3 ConvNeXtv2 convolution layer and image features are extracted using a 3×3 convolution with a stride of 1. The size of the output feature is In the second stage, the extracted features are concatenated with the pooled three-channel input video frame and further downsampled using a 3×3 ConvNeXtv2 convolutional layer with a stride of 2. The size of the output features is The features are further mined through continuous dilated convolution and local-global feature interaction modules to extract more complex and diverse feature representations. The introduction of dilated convolution enables the network to expand the receptive field without reducing the resolution of the features, thereby better capturing spatial information. In the third and fourth stages, the input features are processed to obtain features of size and The design logic of these two stages is similar to that of the previous two stages. Through continuous downsampling and feature interaction, multi-level information is aggregated to ensure that the deep encoder can fully capture the multi-scale features of the input video frame.

[0072] Furthermore, the local-global feature encoding part calculates the attention across feature channels to further enhance the global information. The attention in the original Transformer is to perform autocorrelation calculations between each feature, and the complexity is proportional to the square of the number of features, which doubles the memory consumption. The present invention does not calculate the attention across tokens but calculates the cross-covariance attention across feature channels.

[0073]

[0074] Among them, Attention(Q,K,V)=V·Softmax(Q T K), Linear G It is a point-level convolution operation, and LN is a normalization operation. Compared with the original self-attention, it reduces the time complexity and space complexity.

[0075] Furthermore, the depth decoding part calculates inverse depth maps of different resolutions. The size of the multi-scale feature map is gradually restored through three bilinear upsampling layers to increase the spatial dimension and improve the decoding accuracy. The feature maps from the second, third and fourth stage deep encoders are connected in series using convolutional layers to achieve the fusion of features of different resolutions. The prediction head connected after each upsampling block outputs full resolution, Resolution and The inverse depth map of the resolution effectively converts the low-resolution feature map into a high-resolution depth map, ensuring that the final generated depth map has high resolution and high accuracy.

[0076] The loss function is composed of the reconstruction loss L recon and adversarial loss L adv The reconstruction loss L is a comprehensive loss function consisting of two parts. recon The expression used to ensure the similarity between the generated stable video frame and the original video frame is:

[0077]

[0078] Among them is the stabilized video frame generated by the model.

[0079] Adversarial loss L adv Used to optimize the visual quality of the generated video, the expression is:

[0080]

[0081] Here D(x) is the discriminator, which can distinguish between real video frames and generated video frames.

[0082] The loss function e can be expressed as:

[0083] L=L recon +λL adv ;

[0084] Here, λ is a weight parameter used to balance the reconstruction loss and the adversarial loss.

[0085] Example

[0086] The present invention uses a motion estimation module to perform camera motion estimation and motion smoothing to reduce the interference caused by lens jitter on the generation of target depth images, and then uses a depth estimation module to obtain the global and local context information of the input video frame and generate a multi-scale depth image, and finally combines the smoothed motion estimation to reconstruct a stable target depth image. Specifically, the following steps are included:

[0087] Step 1, prepare the data set. First, prepare the video data of the non-motor vehicle scene. The scene contains traffic elements such as motor vehicles, non-motor vehicles, pedestrians, and various obstacles. Select the front camera on the vehicle for video acquisition and record the camera internal parameters. Then, after the data acquisition is completed, use the OpenCV video processing tool to pre-process the recorded video and adjust the resolution of the input video frame according to the input requirements of the model. Finally, use the Metric3D depth estimation model to infer the processed image frame sequence and generate the corresponding high-quality depth image for subsequent evaluation indicators.

[0088] Step 2, build the network framework. Specifically, first, RFNet is used to select key points from the video frame and obtain the motion vectors of the key points in continuous frames. The multi-homography estimation strategy is used to generate the initial motion trajectory based on the motion vectors of the key points, and then the initial trajectory is smoothed to achieve stable and accurate camera motion estimation. Then, ConvNeXtv2 is used to form the downsampling layer of the depth estimation module to extract the image features of the video frame. The local-global feature interaction module further mines complex and diverse feature representations. Subsequent features are passed through three bilinear upsampling layers to restore the size of the multi-scale feature map and calculate inverse depth maps of different resolutions. Finally, the camera motion estimation is combined to reconstruct a stable target depth image.

[0089] Step 3, train the network model. First, initialize the weights of the backbone network model to accelerate convergence. Then, normalize the input video frames and input them into the model for training. In each training iteration, randomly extract a certain number of video frames from the training data set as a training batch. Finally, a weighted comprehensive loss is used, which consists of two parts: reconstruction loss and adversarial loss. Set SGD as the optimizer, the learning rate is 0.0001, and train for a total of 200 epochs.

[0090] Step 4: Test the network model. Prepare a test set containing real depth images, input the video frame sequence into the model in sequence, and obtain the predicted depth image. Use indicators such as root mean square error RMSE, mean absolute error MAE, and structural similarity index SSIM to evaluate the gap between the generated depth image and the real depth image to verify the model performance and improvement direction.

[0091] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any reference numerals in the claims should not be regarded as limiting the claims involved.

[0092] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A self-supervised monocular depth estimation method for non-motor vehicle scenes, characterized in that: Input the video captured by the camera into the trained estimation model and output the corresponding depth map; The training process of the estimation model includes: Motion estimation step: The camera motion trajectory is estimated and the trajectory is smoothed according to the video captured by the camera during the non-motor vehicle driving process, which specifically includes: Keypoint estimation, which is used to determine keypoints in video frames and calculate motion vectors between adjacent video frames; Motion trajectory estimation, using a multi-homography estimation strategy to generate the initial camera motion trajectory based on the motion vectors between adjacent video frames; Trajectory smoothing, which smoothes the initial camera motion trajectory to generate stable video frames; Depth estimation step: Extract multi-scale local features from the video frames captured by the camera to achieve depth map prediction, including: Deep coding, using a lightweight backbone network to extract multi-scale local features from input video frames; Local-global feature encoding: randomly generated global features are encoded into local features through the self-attention mechanism to obtain multi-scale feature maps; Depth decoding, generating a multi-scale depth map of the input video frame based on the multi-scale feature map; The estimation model is trained based on the multi-scale depth map generated in the depth estimation step and a loss function consisting of reconstruction loss and adversarial loss; the reconstruction loss is used to ensure the similarity between the generated stable video frames and the original video frames; the adversarial loss is used to optimize the visual quality of the generated video.

2. The self-supervised monocular depth estimation method for non-motor vehicle scenes according to claim 1, characterized in that: The key point estimation specifically includes: Use the PWCNet model to extract the optical flow between adjacent video frames i : OpticalFlow i =PWCNet(f i ,f i+1 ) Among them, f i represents the i-th video frame of the input video, E represents the total number of video frames, PWCNet(·) represents the PWCNet model; OpticalFlow between adjacent video frames i Used for motion vector generation; Use the pre-trained multi-scale shallow network RFNet to extract key points from video frames: p ij =RFNet(f i ) Among them, p ij represents the video frame f i The jth key point detected on L represents the total number of key points detected; the motion vector V between adjacent video frames is calculated by the optical flow algorithm. i→i+1 , V i→i+1 Describes the pixel displacement of key points between adjacent video frames: V i→i+1 =OpticalFlow i (p ij )。 3. The self-supervised monocular depth estimation method for non-motor vehicle scenes according to claim 1, characterized in that: The motion trajectory estimation specifically includes: Each video frame is evenly divided into a set of grids so that the motion vector of each key point can be transferred to the corresponding grid vertices around it, and multi-homography estimation is used to predict the initial camera motion trajectory: p ij represents the video frame f i The jth key point detected on L represents the total number of key points detected. Represents the initial camera motion trajectory, and MultiHomo represents multi-homography estimation.

4. The self-supervised monocular depth estimation method for non-motor vehicle scenes according to claim 1, characterized in that: The trajectory smoothing specifically includes: The motion vector V between adjacent video frames is smoothed by a smoother composed of multiple 3D convolutional layers. i→i+1 Smoothing is performed to eliminate errors caused by noise and jitter to generate a smooth camera motion trajectory P i : P i =P i-1 +α(V i→i+1 -b); Here, α is a scaling factor and β is a smoothing parameter used to reduce noise.

5. The self-supervised monocular depth estimation method for non-motor vehicle scenes according to claim 1, characterized in that: The depth coding includes four stages; In the first stage of deep coding, the input video frame is processed by the convolutional backbone network, downsampled by using a 3×3 ConvNeXtv2 convolution layer, and image features are extracted by a 3×3 convolution with a stride of 1. The output size is Local features of In the second stage of deep coding, the local features output by the first stage of deep coding are concatenated with the pooled three-channel video frame and further downsampled using a 3×3 ConvNeXtv2 convolutional layer with a stride of 2, and the output size is The local features of the image are further mined through continuous dilated convolution and local-global feature interaction modules. The third and fourth stages of depth coding use the same processing methods as the first and second stages of depth coding, and obtain the sizes of The local features and local features.

6. The self-supervised monocular depth estimation method for non-motor vehicle scenes according to claim 1, characterized in that: The depth decoding specifically includes: The size of the multi-scale feature map is gradually restored through three bilinear upsampling layers, and the local features output from the second, third and fourth stages of deep coding are connected in series using convolutional layers to achieve the fusion of features of different resolutions; the prediction head connected after each upsampling block outputs full resolution, Resolution and The low-resolution feature map is converted into a high-resolution depth map.

7. The self-supervised monocular depth estimation method for non-motor vehicle scenes according to claim 1 is characterized in that: The local-global feature encoding specifically includes: Among them, Attention(Q,K,V)=V·Softmax(Q T ·K), Attention(·) is the attention mechanism function, Linear G is a point-level convolution operation, Linear represents a linear transformation layer that performs linear mapping on the input data, LN is a normalization operation; Q is a query vector, which represents the features or information that need to be paid attention to at present; K is a key vector, which represents the feature representation of each element in the input sequence; V is a value vector, which contains the actual information corresponding to the key; X is an input feature, which represents the output or input sequence from the previous layer; is the updated feature matrix, which represents the output after being processed by the attention mechanism; Represents the features after normalization and weighting operations.

8. The self-supervised monocular depth estimation method for non-motor vehicle scenes according to claim 1, characterized in that: The loss function L is composed of the reconstruction loss L recon and adversarial loss L adv composition; Reconstruction loss L recon Used to ensure the similarity between the generated stabilized video frames and the original video frames: in, is the stabilized video frame generated by the estimation model, and T represents the number of video frames; Adversarial loss L adv Used to optimize the visual quality of generated videos: in, represents the expected value of the real sample, represents the expected value of the generated sample, D(x) is the discriminator, which can distinguish between real video frames and generated video frames; The loss function L is: L=L recon +λL adv Here, λ is a weight parameter used to balance the reconstruction loss and the adversarial loss.

Images