Self-supervised monocular depth estimation method for non-motorized vehicle scenarios

Abstract

This invention relates to the field of depth estimation technology and discloses a self-supervised monocular depth estimation method for non-motorized vehicle scenarios. The training process of the estimation model includes: a motion estimation step: estimating the camera motion trajectory based on video frames captured by the camera during the non-motorized vehicle's movement and performing trajectory smoothing; and a depth estimation step: extracting multi-scale local features from the video frames captured by the camera to achieve depth map prediction. This invention introduces camera motion estimation and motion smoothing techniques, effectively reducing interference caused by camera shake by predicting the camera's motion state between video frames and combining it with motion smoothing processing. This invention adopts a lightweight and efficient convolutional network architecture, achieving fast and robust depth estimation, and can be deployed in non-motorized vehicle scenarios with limited computing resources and high real-time performance requirements.

Description

Technical Field

[0001] This invention relates to the field of depth estimation technology, and specifically to a self-supervised monocular depth estimation method for non-motorized vehicle scenarios. Background Technology

[0002] Currently, significant progress has been made in depth estimation methods based on deep learning. Supervised learning methods typically require large datasets with real depth annotations to train the depth estimation model. For example, patent application CN118229750A discloses a supervised learning-based monocular image depth estimation method that uses a mixture of depth datasets from different sources for training to obtain depth estimation information. However, the acquisition and annotation of depth data is both expensive and cumbersome, which greatly limits the application of supervised learning-based depth estimation methods. Unlike methods that directly use depth labels as supervision signals, self-supervised methods utilize consecutive video frames or binocular images to provide indirect supervision signals for network training. In real-world scenarios, monocular cameras have advantages over stereo cameras or LiDAR, such as lower cost and easier deployment. Therefore, generating scene depth information using monocular images without depth annotations constitutes the self-supervised monocular depth estimation task.

[0003] In self-supervised monocular depth estimation tasks, a basic approach is to predict a disparity map from a source view using a monocular depth estimation model, reconstruct a target view based on the predicted disparity map and the source view, and use the reconstruction error between the synthesized target view and the real target view as a constraint to optimize the training process of the depth estimation model. Finally, the corresponding depth map is calculated using camera parameters and the predicted disparity map. For example, patent document CN116245927B discloses a self-supervised monocular depth estimation method for unstructured monocular videos. It generates an initial disparity map and depth map through a depth module, then predicts rotation and translation matrices from two consecutive video frames using a pose network, and finally generates a reconstructed depth image. In non-motorized vehicle scenarios, self-supervised monocular depth estimation technology can be used to obtain depth information of the surrounding environment, effectively identifying and understanding complex road scenes, and providing an auxiliary system for intelligent driving of non-motorized vehicles. However, compared to motorized vehicle scenarios, non-motorized vehicles experience more bumps or instability during travel, leading to blurred images from the front monocular camera. Continuous shaking causes the accumulation of depth estimation errors, further affecting the final depth prediction effect. Therefore, in self-supervised monocular depth estimation methods, it is crucial to study how to solve the camera stabilization problem in non-motorized vehicle scenes. Summary of the Invention

[0004] To address the issue of camera stabilization during non-motorized vehicle movement and thus improve the prediction accuracy and robustness of self-supervised monocular depth estimation models, this invention discloses a self-supervised monocular depth estimation method for non-motorized vehicle scenarios. First, this invention utilizes a novel motion estimation module to estimate camera motion and perform motion smoothing during non-motorized vehicle movement, aiming to reduce the interference caused by camera shake in the generation of the target depth image. Second, this invention employs a lightweight depth estimation module to acquire global and local contextual information from the input video frame and generate multi-scale depth images. Finally, this invention combines the smoothed motion estimation to reconstruct a stable target depth image, aiming to improve the accuracy and reliability of depth estimation.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] A self-supervised monocular depth estimation method for non-motorized vehicle scenarios inputs video captured by a camera into a trained estimation model, outputting the corresponding depth map. The training process of the estimation model includes:

[0007] Motion estimation steps: Based on the video captured by the camera during the non-motorized vehicle's movement, the camera's motion trajectory is estimated and smoothed. Specifically, this includes:

[0008] Keypoint estimation is used to identify keypoints in video frames and calculate motion vectors between adjacent video frames.

[0009] Motion trajectory estimation utilizes a multi-homography estimation strategy to generate initial camera motion trajectories based on motion vectors between adjacent video frames;

[0010] Trajectory smoothing smooths the initial camera motion trajectory to generate stable video frames;

[0011] Depth estimation steps: Extract multi-scale local features from video frames captured by the camera to achieve depth map prediction, specifically including:

[0012] Deep coding utilizes a lightweight backbone network to extract multi-scale local features from the input video frames;

[0013] Local-global feature encoding uses a self-attention mechanism to encode randomly generated global features into local features, resulting in multi-scale feature maps.

[0014] Depth decoding generates multi-scale depth maps of the input video frames based on multi-scale feature maps.

[0015] The estimation model is trained based on the multi-scale depth map generated in the depth estimation step and the loss function consisting of reconstruction loss and adversarial loss; wherein, reconstruction loss is used to ensure the similarity between the generated stable video frames and the original video frames; and adversarial loss is used to optimize the visual quality of the generated video.

[0016] Furthermore, the keypoint estimation specifically includes:

[0017] Optical flow between adjacent video frames is extracted using the PWCNet model. i :

[0018] OpticalFlow i =PWCNet(f i ,f i+1 )

[0019] Among them, f i This represents the i-th video frame of the input video. E represents the total number of frames in the video, PWCNet(·) represents the PWCNet model; Optical Flow between adjacent video frames. i Used for generating motion vectors;

[0020] Keypoints are selected from video frames using the pre-trained multi-scale shallow network RFNet:

[0021] p ij =RFNet(fi i )

[0022] Where, p ij This represents video frame f. i The j-th key point detected above, L represents the total number of detected keypoints; the motion vector V between adjacent video frames is calculated using an optical flow algorithm. i→i+1 V i→i+1 This describes the pixel displacement of key points between adjacent video frames:

[0023] V i→i+1 =OpticalFlow i (p ij ).

[0024] Furthermore, the motion trajectory estimation specifically includes:

[0025] Each video frame is uniformly divided into a set of grids, such that the motion vector of each keypoint can be transferred to the corresponding grid vertices around it. Multihomography estimation is used to predict the initial camera motion trajectory.

[0026]

[0027] p ij This represents video frame f. i The j-th key point detected above, L represents the total number of keypoints detected. This represents the initial camera motion trajectory, and MultiHomo represents the multihomography estimation.

[0028] Furthermore, the trajectory smoothing specifically includes:

[0029] A smoother composed of multiple 3D convolutional layers is used to smooth the motion vector V between adjacent video frames. i→i+1 Smoothing is performed to eliminate errors caused by noise and jitter, in order to generate a smooth camera motion trajectory P. i :

[0030] P i =P i-1 +α(V i→i+1 -β);

[0031] Where α is the scaling factor and β is the smoothing parameter used to reduce noise.

[0032] Furthermore, the depth encoding includes four stages;

[0033] In the first stage of deep encoding, the input video frames are processed by a convolutional backbone network, then downsampled using a 3×3 ConvNeXtv2 convolutional layer, and image features are extracted using a 3×3 convolution with a stride of 1. The output size is [size missing]. Local features;

[0034] In the second stage of deep encoding, the local features output from the first stage are concatenated with the pooled three-channel video frames, and further downsampled using a 3×3 ConvNeXtv2 convolutional layer with a stride of 2. The output size is [size missing]. The local features are further mined by continuous dilated convolutions and local-global feature interaction modules.

[0035] The third and fourth stages of depth coding employ the same processing methods as the first and second stages, respectively, yielding sizes of [size not specified]. Local features and Local features.

[0036] Furthermore, the depth decoding specifically includes:

[0037] The size of the multi-scale feature map is gradually restored through three bilinear upsampling layers. Convolutional layers are used to concatenate local features from the outputs of the second, third, and fourth stages of deep encoding to achieve feature fusion at different resolutions. The prediction head connected after each upsampling block outputs full-resolution, multi-resolution, and multi-resolution features respectively. resolution and The inverse depth map of the resolution transforms a low-resolution feature map into a high-resolution depth map.

[0038] Furthermore, the local-global feature encoding specifically includes:

[0039]

[0040] Where Attention(Q,K,V)=V·Softmax(Q T ·K), Attenton(·) is the attention mechanism function, Linear G It is a point-level convolution operation. Linear represents a linear transformation layer that performs a linear mapping on the input data, and LN is a normalization operation; Q is the query vector, representing the feature or information that needs to be focused on; K is the key vector, representing the feature representation of each element in the input sequence; V is the value vector, containing the actual information corresponding to the key; X is the input feature, representing the output or input sequence from the previous layer. It is the updated feature matrix, representing the output after processing by the attention mechanism; This represents the characteristics after normalization and weighting operations.

[0041] Furthermore, the loss function L is composed of the reconstruction loss Lreconstruction. recon and combat losses L adv composition;

[0042] Reconstruction loss L recon Used to ensure the similarity between the generated stable video frames and the original video frames:

[0043]

[0044] in, These are the stabilized video frames generated by the estimation model, where T represents the number of video frames;

[0045] Combat loss L adv Used to optimize the visual quality of generated videos:

[0046]

[0047] in, Represents the expected value of the real sample. Let D(x) represent the expected value of the generated sample, and D(x) be the discriminator that can distinguish between real video frames and generated video frames.

[0048] The loss function L is:

[0049] L = L recon +λL adv

[0050] Here, λ is a weighting parameter used to balance the reconstruction loss and the adversarial loss.

[0051] Compared with the prior art, the beneficial technical effects of the present invention are:

[0052] This invention discloses a self-supervised monocular depth estimation method for non-motorized vehicle scenarios. First, this method only requires video sequence frames as input during training. Through a self-supervised learning mechanism, it automatically generates a stable target depth map, effectively avoiding the complex and expensive data acquisition and annotation processes required in traditional depth estimation methods. This significantly reduces implementation costs and time overhead, making it suitable for non-motorized vehicle scenarios where data is scarce and annotation is difficult. Second, this invention introduces camera motion estimation and motion smoothing techniques. By predicting the camera's motion state between video frames and combining it with motion smoothing processing, it effectively reduces interference caused by camera shake. Finally, this invention employs a lightweight and efficient convolutional network architecture, achieving fast and robust depth estimation, and can be deployed in non-motorized vehicle scenarios where computing resources are limited and real-time performance requirements are high. This invention demonstrates excellent performance in depth estimation tasks for non-motorized vehicle scenarios and has strong practical value. Attached Figure Description

[0053] Figure 1 This is a flowchart illustrating the self-supervised monocular depth estimation method of the present invention;

[0054] Figure 2 This is a schematic diagram of the output results of the self-supervised monocular depth estimation method in this invention. Detailed Implementation

[0055] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0056] The self-supervised monocular depth estimation method for non-motorized vehicle scenarios in this invention has the following detailed framework and structure: Figure 1As shown, this method aims to effectively address the problem of decreased depth estimation accuracy caused by camera shake during dynamic driving of non-motorized vehicles, and has significant research value and practical application potential. Specifically, this invention first introduces a novel motion estimation module to estimate and smooth the camera motion trajectory during the non-motorized vehicle's movement, enabling accurate modeling and reducing the impact of camera shake caused by road bumps or unstable driving on image quality, thereby improving the stability and reliability of the input data for subsequent depth estimation. Then, this invention extracts feature information at different scales from the input video frames, capturing rich depth features in diverse scenes to generate an initial multi-scale depth map. Finally, combined with camera motion estimation, a stable target depth image is reconstructed, thereby improving the depth perception capability of non-motorized vehicles under complex traffic conditions. The method comprises three parts: a motion estimation module, a depth estimation module, and a loss function.

[0057] The motion estimation module estimates jitter and generates smooth trajectories by calculating motion vectors between frames. By defining a square grid on the video frame, dividing it into multiple grid regions, the motion vector of each feature point can be effectively transferred to its corresponding grid vertices. This process ensures that a unique motion vector is generated for each grid vertex at each moment to accurately capture and analyze the camera's motion dynamics, thus providing necessary motion information for subsequent depth estimation and improving the stability and reliability of the depth estimation process. Over time, each vertex of the grid continuously collects motion vectors from different moments. After smoothing, these vectors ultimately achieve stable and accurate camera motion estimation. The motion estimation module includes three steps: keypoint estimation, motion estimation, and trajectory smoothing.

[0058] Furthermore, the keypoint estimation refers to analyzing video frames and calculating the optical flow information of each keypoint to identify key features. Optical flow is a vector field describing the motion of objects in an image, representing the change in pixel position between adjacent image frames. PWCNet is used to extract the optical flow between adjacent frames. i :

[0059] OpticalFlow i =PWCNet(f i ,f i+1 )

[0060] Among them, f i This represents the i-th frame of the input video. E represents the frame number. Optical flow is extracted between adjacent frames. i Used for generating motion vectors.

[0061] The RFNet pre-trained multi-scale shallow network architecture efficiently and robustly selects keypoints from high-resolution video frames, extracts local and global features, and tracks the motion vectors of these features across consecutive frames. The notation p is used. ij To indicate the location of key points:

[0062] p ij =RFNet(f i );

[0063] Where, p ij This represents video frame f. i The j-th key point detected above, L represents the number of detected keypoints. The motion vector V between frames is calculated using an optical flow algorithm. i→i+1 This describes the pixel displacement of key points between adjacent video frames, and the motion vector V i→i+1 The expression is:

[0064] V i→i+1 =OpticalFlow i (p ij ).

[0065] Furthermore, the motion estimation mentioned refers to using a multi-homography estimation strategy based on the motion vector V of key points. i→i+1 To generate initial camera motion estimates, each video frame is uniformly divided into a fixed grid, allowing the motion vector of each keypoint to be effectively transferred to its corresponding grid vertices, thus predicting motion based on these grid vertices. MultiHomo, a multi-homography estimation algorithm, is used to predict the initial motion estimate.

[0066]

[0067] Furthermore, the trajectory smoothing refers to generating a smooth motion trajectory P based on the extracted features and motion information. i That is, a smoother composed of multiple 3D convolutional layers is used to smooth the collected keypoint-based motion vector V. i→i+1 A smoothing process is performed. This process eliminates errors caused by noise and jitter, improving the stability and accuracy of camera motion estimation. The stabilization process can be expressed as:

[0068] P i =P i-1 +α(V i→i+1 -β);

[0069] Where α is the scaling factor and β is the smoothing parameter used to reduce noise, this formula ensures the generation of smooth motion.

[0070] The depth estimation module extracts multi-scale local features from the input video frame sequence to achieve high-precision depth map prediction. First, a lightweight backbone network is used to extract rich multi-scale local features from the input video frames. Then, a self-attention mechanism encodes randomly generated global features into the local features, ultimately generating a multi-scale depth map of the input video frames. This process mainly consists of three key steps: depth encoding, local-global feature encoding, and depth decoding.

[0071] Furthermore, the depth encoding is subdivided into four stages to ensure the extraction of multi-scale feature information from the input video frames. In the first stage, after processing by a convolutional backbone network, downsampling is performed using 3×3 ConvNeXtv2 convolutional layers, and image features are extracted using 3×3 convolutions with a stride of 1. The size of the output features is... In the second stage, the extracted features are concatenated with the pooled three-channel input video frames, and further downsampled using a 3×3 ConvNeXtv2 convolutional layer with a stride of 2. The size of the output features is [size missing]. The features are processed through successive dilated convolutions and local-global feature interaction modules to further extract more complex and diverse feature representations. The introduction of dilated convolutions allows the network to expand the receptive field without reducing the feature resolution, thereby better capturing spatial information. The subsequent third and fourth stages process the input features, obtaining representations of size [size missing]. and The design logic of these two stages is similar to that of the previous two stages. By continuously downsampling and feature interaction, multi-level information is gathered to ensure that the depth encoder can fully capture the multi-scale features of the input video frames.

[0072] Furthermore, the local-global feature encoding part calculates attention across feature channels to further enhance global information. In the original Transformer, attention is calculated by performing autocorrelation calculations between each feature, the complexity of which is proportional to the square of the number of features, doubling memory consumption. This invention calculates cross-covariance attention across feature channels instead of cross-token attention.

[0073]

[0074] Where Attention(Q,K,V)=V·Softmax(Q T ·K), Linear G It is a point-level convolution operation, and LN is a normalization operation. Compared with the original self-attention, it reduces the time and space complexity.

[0075] Furthermore, the depth decoding section calculates inverse depth maps at different resolutions. The dimensions of the multi-scale feature maps are progressively recovered through three bilinear upsampling layers to increase spatial dimension and improve decoding accuracy. Convolutional layers are used to concatenate feature maps from the second, third, and fourth stages of the depth encoder, achieving feature fusion at different resolutions. The prediction head connected after each upsampling block outputs full-resolution, multi-resolution, and multi-resolution features respectively. resolution and The inverse depth map effectively converts low-resolution feature maps into high-resolution depth maps, ensuring that the final generated depth map has high resolution and high accuracy.

[0076] The loss function is composed of reconstruction loss L recon and combat losses L adv The combined loss function consists of two parts. Reconstruction loss L recon The expression used to ensure the similarity between the generated stable video frames and the original video frames is:

[0077]

[0078] Among them These are the stabilized video frames generated by the model.

[0079] Combat loss L adv The expression used to optimize the visual quality of the generated video is:

[0080]

[0081] D(x) is the discriminator, which can distinguish between real video frames and generated video frames.

[0082] The loss function e can be expressed as:

[0083] L = L recon +λL adv ;

[0084] Here, λ is a weighting parameter used to balance the reconstruction loss and the adversarial loss.

[0085] Example

[0086] This invention utilizes a motion estimation module to estimate camera motion and perform motion smoothing to reduce interference from lens shake in the generation of the target depth image. Then, a depth estimation module is used to acquire global and local contextual information of the input video frame and generate a multi-scale depth image. Finally, a stable target depth image is reconstructed by combining the smoothed motion estimation. Specifically, it includes the following steps:

[0087] Step 1: Prepare the dataset. First, prepare video data for non-motorized vehicle scenes. These scenes include traffic elements such as motorized vehicles, non-motorized vehicles, pedestrians, and various obstacles. A vehicle-mounted front-facing camera is used to capture the video, and its intrinsic parameters are recorded. Next, after data acquisition, the recorded video is preprocessed using OpenCV video processing tools, adjusting the resolution of the input video frames according to the model's input requirements. Finally, the Metric3D depth estimation model is used to infer the processed image frame sequence, generating corresponding high-quality depth images for subsequent evaluation metrics.

[0088] Step 2: Building the Network Framework. Specifically, firstly, keypoints are selected from video frames using RFNet, and their motion vectors in consecutive frames are obtained. A multi-homography estimation strategy is used to generate initial motion trajectories based on these keypoint motion vectors. These initial trajectories are then smoothed to achieve stable and accurate camera motion estimation. Next, ConvNeXtv2 is used to construct the downsampling layer of the depth estimation module to extract image features from the video frames. A local-global feature interaction module further mines complex and diverse feature representations. Subsequent features are then processed through three bilinear upsampling layers to restore the size of multi-scale feature maps, and inverse depth maps at different resolutions are calculated. Finally, camera motion estimation is combined to reconstruct a stable target depth image.

[0089] Step 3: Train the network model. First, initialize the weights of the backbone network model to accelerate convergence. Then, normalize the input video frames before inputting them into the model for training. During each training iteration, a certain number of video frames are randomly selected from the training dataset as a training batch. Finally, a weighted comprehensive loss is used, consisting of reconstruction loss and adversarial loss. SGD is set as the optimizer with a learning rate of 0.0001, and the model is trained for a total of 200 epochs.

[0090] Step 4: Test the network model. Prepare a test set containing real depth images. Input the video frame sequence into the model sequentially to obtain the predicted depth images. Use metrics such as Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Structural Similarity Index (SSIM) to evaluate the difference between the generated depth images and the real depth images, in order to verify the model performance and identify areas for improvement.

[0091] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0092] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A self-supervised monocular depth estimation method for non-motorized vehicle scenarios, characterized in that, The video captured by the camera is input into the trained estimation model, which outputs the corresponding depth map. The training process for the estimation model includes: Motion estimation steps: Based on the video captured by the camera during the non-motorized vehicle's movement, the camera's motion trajectory is estimated and smoothed. Specifically, this includes: Keypoint estimation is used to identify keypoints in video frames and calculate motion vectors between adjacent video frames. Motion trajectory estimation utilizes a multihoming estimation strategy to generate initial camera motion trajectories based on motion vectors between adjacent video frames. Specifically, this involves uniformly dividing each video frame into a grid such that the motion vector of each keypoint can be transferred to the corresponding surrounding grid vertices, and using multihoming estimation to predict the initial camera motion trajectory. ; This represents video frames. The first detected One key point, , This represents the total number of key points detected. This represents the initial camera motion trajectory. This indicates a multiple monotonicity estimate; These are the motion vectors between adjacent video frames calculated using the optical flow algorithm. Trajectory smoothing smooths the initial camera motion trajectory to generate stable video frames. Specifically, this involves smoothing the motion vectors between adjacent video frames using a smoother composed of multiple 3D convolutional layers. Smoothing is performed to eliminate errors caused by noise and jitter, resulting in a smooth camera motion trajectory. : ;in, It is a scaling factor. It is a smoothing parameter used to reduce noise; Depth estimation steps: Extract multi-scale local features from video frames captured by the camera to achieve depth map prediction, specifically including: Deep coding utilizes a lightweight backbone network to extract multi-scale local features from the input video frames; Local-global feature encoding uses a self-attention mechanism to encode randomly generated global features into local features, resulting in multi-scale feature maps. Specifically, this includes: ; ;in, , It is an attention mechanism function. It is a point-level convolution operation. This represents a linear transformation layer that performs a linear mapping on the input data. It is a normalization operation; It is a query vector, representing the features or information that need to be focused on at this time; It is a key vector, representing the feature representation of each element in the input sequence; It is a value vector that contains the actual information corresponding to the key; These are input features, representing the output or input sequence from the previous layer; It is the updated feature matrix, representing the output after processing by the attention mechanism; This represents the characteristics after normalization and weighting operations; Depth decoding generates multi-scale depth maps of the input video frames based on multi-scale feature maps. The estimation model is trained based on the multi-scale depth map generated in the depth estimation step and the loss function consisting of reconstruction loss and adversarial loss; wherein, reconstruction loss is used to ensure the similarity between the generated stable video frames and the original video frames; and adversarial loss is used to optimize the visual quality of the generated video.

2. The self-supervised monocular depth estimation method for non-motorized vehicle scenarios according to claim 1, characterized in that, The keypoint estimation specifically includes: use The model extracts the optical flow between adjacent video frames. : in, Indicates the first input video. One video frame, , This indicates the total number of frames in the video. express Model; Optical flow between adjacent video frames Used for generating motion vectors; Using pre-trained multi-scale shallow networks Selecting key points from video frames: in, This represents video frames. The first detected One key point, , This represents the total number of detected keypoints; motion vectors between adjacent video frames are calculated using an optical flow algorithm. , This describes the pixel displacement of key points between adjacent video frames: 。 3. The self-supervised monocular depth estimation method for non-motorized vehicle scenarios according to claim 1, characterized in that, The deep encoding includes four stages; In the first stage of deep encoding, the input video frames are processed by a convolutional backbone network, then downsampled using a 3×3 ConvNeXtv2 convolutional layer, and image features are extracted using a 3×3 convolution with a stride of 1. The output size is [size missing]. Local features; In the second stage of deep encoding, the local features output from the first stage are concatenated with the pooled three-channel video frames, and further downsampled using a 3×3 ConvNeXtv2 convolutional layer with a stride of 2. The output size is [size missing]. The local features are further mined by continuous dilated convolutions and local-global feature interaction modules. The third and fourth stages of depth coding employ the same processing methods as the first and second stages, respectively, yielding sizes of [size not specified]. Local features and Local features.

4. The self-supervised monocular depth estimation method for non-motorized vehicle scenarios according to claim 3, characterized in that, The depth decoding specifically includes: The size of the multi-scale feature map is gradually restored through three bilinear upsampling layers. Convolutional layers are used to concatenate local features from the outputs of the second, third, and fourth stages of deep encoding to achieve feature fusion at different resolutions. The prediction head connected after each upsampling block outputs full-resolution, multi-resolution, and multi-resolution features respectively. resolution and The inverse depth map of the resolution transforms a low-resolution feature map into a high-resolution depth map.

5. The self-supervised monocular depth estimation method for non-motorized vehicle scenarios according to claim 1, characterized in that, The loss function Losses from reconstruction and combat losses composition; Reconstruction losses Used to ensure the similarity between the generated stable video frames and the original video frames: ； in, These are the stabilized video frames generated by the estimation model. Indicates the number of video frames; Combating losses Used to optimize the visual quality of generated videos: ； in, Represents the expected value of the real sample. This represents the expected value of the generated sample. It is a discriminator that can distinguish between real video frames and generated video frames; loss function for: Among them, here It is a weighting parameter used to balance reconstruction loss and adversarial loss.

Images