Self-supervised monocular depth estimation method based on combination of CBAM module and geometric consistency loss function

Abstract

The invention provides a monocular depth estimation method based on the synergistic effect of a geometric consistency loss function and a CBAM module. According to the method, Lite-Mono is adopted as a reference model. Firstly, a CBAM module is introduced into a continuous expansion convolution module in a model, so that the model gives more attention weights to edges and important detail areas in a feature extraction stage, depth information of the areas is effectively reserved, and an initial depth map output by a deep network is clearer; after an initial depth map is obtained, multi-view space structure constraint is carried out on depth estimation through a geometric consistency loss function, and the space structure consistency between different views is improved by combining with the feature enhancement effect of CBAM. After the method is introduced, the model improves the problem that the image edge and the detail area are fuzzy, and the obtained depth image is clearer and more accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision, and in particular relates to a monocular depth estimation method using a geometric consistency loss function combined with a CBAM module. Background Art

[0002] With the rapid development of deep learning technology, research on monocular depth estimation has also received widespread attention and has become a hot research direction. By using monocular images to estimate scene geometry information, the perception and understanding of real three-dimensional scenes is improved, and it has a wide range of applications in viewpoint synthesis, scene reconstruction, virtual reality and other fields. At present, monocular depth estimation based on deep learning is divided into three methods: supervised monocular depth estimation, semi-supervised monocular depth estimation and self-supervised monocular depth estimation.

[0003] Supervised learning methods require a large amount of labeled data, which is usually time-consuming, labor-intensive and costly. Especially in the field of depth estimation, obtaining high-quality depth labeled data is a huge challenge. Self-supervised learning methods do not need to rely on manually labeled data, which can greatly reduce the cost and time of data preparation. This method not only improves the efficiency of data acquisition, but also makes better use of large-scale unlabeled data, thereby improving the generalization ability and robustness of the model.

[0004] Self-supervised monocular depth estimation aims to infer the depth information in a scene from a single image through deep learning technology. The core idea is to use the geometric constraints and physical laws in the image to train a deep neural network so that it can automatically learn the mapping relationship of restoring the three-dimensional depth map from a single-view image. This process can not only reduce the dependence on expensive hardware equipment, but also simplify the design and deployment of the system, and improve the convenience and economy of the system.

[0005] However, the current research on self-supervised monocular depth estimation still faces some challenges, the most prominent of which is the problem of information loss at the edges and detail areas of the image. Therefore, the research on efficient, economical and accurate self-supervised monocular depth estimation algorithms can not only promote the advancement of computer vision technology, but also be widely used in many fields such as robot navigation, autonomous driving, augmented reality, etc., which has important practical significance and application value. Summary of the invention

[0006] To solve the above problems, the present invention improves the monocular depth estimation model Lite-Mono by combining the geometric consistency loss function and the CBAM module.

[0007] The Lite-Mono model achieves efficient monocular depth estimation through a lightweight design, significantly reducing the amount of model computation and parameters, making it suitable for resource-constrained device applications while maintaining competitiveness in terms of accuracy. However, the current model lacks effective capture of edge details, and it is difficult to fully optimize the depth estimation accuracy of edge and detail areas by relying solely on basic loss functions, resulting in information loss in these areas of the depth map.

[0008] To this end, the technical solution steps taken by the present invention are as follows:

[0009] Step 1: Data preparation and preprocessing. First, divide the monocular depth estimation related dataset into two parts: training set and validation set. In the preprocessing stage, first adjust the input image to a suitable resolution, usually the size required for the model input, to ensure that the image maintains its original proportion during the scaling process. Finally, perform random flipping, color adjustment, noise addition and other operations on the image to enhance data diversity, avoid model overfitting, and adapt it to the model input requirements.

[0010] Step 2: Network structure design. The entire network structure is divided into the depth estimation network DepthNet and the posture network PoseNet.

[0011] In DepthNet, image I t It is the color image input to DepthNet, which is subjected to feature extraction and downsampling operations to extract deep features. The four stages correspond to feature extraction at different levels. Stage 1 contains the Conv Stem block, which is the initial feature extraction and is mainly responsible for extracting low-level features of the image (such as edges and textures). H and W represent the height and width of the image, respectively. At this time, the image height and width of this stage are and The number of channels is C1. AvgPool is average pooling to reduce the resolution. Concat passes the features of Stage 1 to Stage 2. Stage 2 contains the Downsample operation to reduce the spatial size of the input feature map. Stage 2 is the mid-level feature extraction, which is mainly responsible for capturing the basic shape and contour of the image. Stage 3 is the high-level feature extraction, which is mainly responsible for extracting more abstract and complex spatial relationships. Stage 4 is the deep feature extraction that generates global, abstract semantic information. The same modules contained in Stage 3 and Stage 4 have the same functions as above.

[0012] In the original model, Stage 2, Stage 3, and Stage 4 contain the continuous dilated convolution module CDC Block to further extract features.

[0013] The attention mechanism (Attention-Infused), namely the CBAM module, is introduced into this module to form the Attention-Infused CDC Block (AIC-Block). In the AIC-Block, 3×3DWConv initially extracts features, processes information of different scales through dilated convolution, and then performs normalization operation through Batch Norm. Then, 1×1Point-wise reorganizes feature channels to reduce the amount of calculation, and then the GELU activation function improves nonlinear expression ability. Finally, a point-wise convolution is performed to further optimize the combination of feature channels.

[0014] LGFI Block represents the local-global feature interaction module, which can fuse feature maps of different scales. Upsampling Block gradually restores the feature map to the original size through convolutional layers and upsampling operations. PredictionHead uses 1x1 convolutional layers and Sigmoid function to generate depth maps.

[0015] In PoseNet, the input image pair, I s ∈[-1,1] means normalizing the image pair. ResNet is used as the backbone network of PoseNet to extract the features of the image. PoseNet generates the camera's pose change matrix P, reconstructs the image in the reconstruction module, and finally optimizes the depth map, i.e., Optimization. This process includes calculating the gradient of the loss function and optimizing the model weights to minimize the error.

[0016] Step 3: Calculate the loss function. The loss function includes reprojection loss, smoothing loss, and geometric consistency loss.

[0017] The reprojection loss is defined as shown in formula (1):

[0018]

[0019] In the formula, min is the minimization operation, and s here represents all adjacent frames. Here, the frame with the smallest error is selected for loss calculation. t is the current frame image, I s are adjacent frame images, I t→s The reconstructed image is obtained by reprojecting the current frame to the adjacent frame. SSIM is the structural similarity index, which is used to measure the structural similarity of two images. t -I t→s ∥1 is the L1 loss between the current frame and the reconstructed image, which is used to measure the brightness difference, and θ is the balancing factor used to adjust the weight between SSIM and L1 losses.

[0020] The smoothing loss is defined as shown in formula (2):

[0021]

[0022] Where i and j represent the row index and column index of the pixel in the image, respectively. and is the gradient of the depth map in the x and y directions, and is the gradient of the input image in the x and y directions, and The weight coefficient used to weigh the depth gradient. Based on the gradient information of the image, the weight of the area with a large gradient (edge) will be reduced to keep the edge clear. ∑ means adding the weighted gradients in the horizontal and vertical directions and summing them for all pixels to obtain the final smoothing loss.

[0023] Since these two losses have some limitations in the optimization process, especially in processing geometric information and depth consistency, geometric consistency loss is introduced in the loss function calculation stage. Geometric consistency loss makes up for the shortcomings of reprojection and smoothing loss, effectively preventing the detail information improved by AIC-Block from being weakened by smoothing loss during the optimization process, ensuring that the depth estimation of edge objects is more accurate. At the same time, geometric consistency loss further enhances the consistency of spatial structure based on the important areas that AIC-Block focuses on, making the depth relationship of these key areas more reasonable, especially in complex edge areas. Input the depth map D obtained in step 2 t , the depth map D corresponding to adjacent frames t-1 And the camera pose change matrix generated by the pose estimation network. The depth map of the current frame is projected into the coordinate system of the adjacent frame through pose transformation, and then the difference between the reprojected depth and the actual depth of the adjacent frame is calculated as the geometric consistency loss. The definition of the geometric consistency loss function is shown in formula (3):

[0024]

[0025] Where N represents the number of valid pixels, which is used for normalization. x, y represent the coordinates of a pixel in the image. x′, y′ represent the corresponding pixel coordinates in the reference frame. t→t-1 represents the depth value projected from time frame t to time frame t-1. The camera pose change matrix generated by the pose estimation network can map the pixel (x, y) in the target frame to the corresponding position in the reference frame to obtain (x′, y′). The depth differences of all pixels are summed and divided by the total number of pixels N to obtain the average loss.

[0026] Finally, the geometric consistency loss function, the reprojection loss function, and the smoothing loss are combined to form the total loss function. total The definition is shown in formula (4):

[0027]

[0028] In the formula, α, β, and γ are hyperparameters used to balance the importance of each loss term. Among them, 0<α<1, 0<β<1, 0<γ<1, α+β+γ=1. In the depth estimation model, the three loss function terms are weighted and balanced. A coefficient less than 1 can prevent a certain loss from being too large, thus dominating the optimization process and weakening the impact of other losses. Ensuring that the sum of these coefficients is 1 can keep the overall loss stable, avoid disordered enlargement or reduction of the loss value, and enable the model to learn the characteristics of each loss term more balanced, which can improve the comprehensive optimization effect of the network on multiple aspects.

[0029] Step 4: Pass the training set divided in step 1 into the improved Lite-Mono model for training.

[0030] Step 5: Use the weights trained in step 4 to test the color image to obtain the final depth map.

[0031] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0032] 1. The Lite-Mono model suffers from information loss in feature extraction of edge and detail areas, resulting in inaccurate depth estimation in these areas. To address this problem, the present invention introduces the CBAM module into the original CDC Blocks to form an AIC-Block, which can give more attention weights to edge and important detail areas, effectively retain the depth information of these areas, and enhance the performance of the model in detail and edge areas.

[0033] 2. Between multiple perspectives or multiple frames, the depth estimation of the Lite-Mono model is weak in spatial consistency, which may lead to incoherent depth maps. The present invention uses geometric consistency loss to constrain the spatial structure of depth estimation from multiple perspectives, combined with the feature enhancement of AIC-Block, which significantly improves the spatial structure consistency between different perspectives and reduces the depth jump phenomenon between frames, thereby achieving higher training stability and effect and making the depth map more reasonable. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is the Lite-Mono network architecture diagram;

[0035] Figure 2 It is a schematic diagram of CDC Blocks;

[0036] Figure 3 It is the schematic diagram of AIC-Block;

[0037] Figure 4 is the loss function diagram;

[0038] Figure 5a is a color image;

[0039] Figure 5b It is the original model output depth map;

[0040] Figure 5c is the output depth map of the present invention;

[0041] Figure 6a is a color image;

[0042] Figure 6b is the original model output depth map;

[0043] Figure 6c is the output depth map of the present invention; DETAILED DESCRIPTION

[0044] The present invention is further described in detail below in conjunction with examples, but the embodiments of the present invention are not limited thereto.

[0045] This embodiment is implemented under the pytorch deep learning framework, and the computer configuration adopts: Intel i5-13500H processor, 16GB memory, NVIDIA GeForce RTX4060 graphics card, Linux operating system. The technical solution adopted by the present invention includes the following steps:

[0046] Step 1: Data preparation and preprocessing. First, the KITTI dataset is divided into two parts: training set and validation set. In the preprocessing stage, the resolution of the input image is first adjusted to 640x192. Ensure that the image maintains its original proportion during the scaling process. Finally, some KITTI images are randomly flipped, color adjusted, and noise added to enhance data diversity, avoid model overfitting, and adapt them to the model input requirements.

[0047] Step 2: Network structure design. The entire network structure is divided into a depth estimation network DepthNet and a posture network PoseNet. The Lite-Mono network architecture is as follows: Figure 1 shown.

[0048] In DepthNet, image I tIt is the color image input to DepthNet, which is subjected to feature extraction and downsampling operations to extract deep features. The four stages correspond to different levels of feature extraction. Stage 1 contains the Conv Stem block, which is the initial feature extraction and is mainly responsible for extracting low-level features of the image (such as edges and textures). At this time, the image height and width of this stage are 320 and 96, and the number of channels is 48. AvgPool is an average pooling to reduce the resolution. Concat passes the features of Stage 1 to Stage 2. Stage 2 is the middle-level feature extraction, which is mainly responsible for capturing the basic shape and contour of the image. Stage 2 contains the Downsample operation, which reduces the spatial size of the input feature map. Stage 3 is the high-level feature extraction, which is mainly responsible for extracting more abstract and complex spatial relationships. Stage 4 is the deep feature extraction to generate global and abstract semantic information. The same modules contained in Stage 3 and Stage 4 have the same functions as above. Taking deep feature extraction as an example, the input deep image is downsampled three times to obtain feature maps of three sizes: 160×48, 80×24, and 40×12.

[0049] In the original model, Stage 2, Stage 3, and Stage 4 contain a continuous dilated convolution module CDC Block, which is used to further extract features. The schematic diagram of CDC Block is shown in the figure. Figure 2 shown.

[0050] In this module, the attention mechanism (Attention-Infused), namely the CBAM module, is introduced to form the Attention-Infused CDC Block (AIC-Block). In the AIC-Block, 3×3DWConv initially extracts features, processes information of different scales through dilated convolution, and then performs normalization operation through Batch Norm, and then 1×1Point-wise reorganizes the feature channels to reduce the amount of calculation, and then the GELU activation function improves the nonlinear expression ability. Finally, a point-wise convolution is performed to further optimize the combination of feature channels. The schematic diagram of AIC-Block is shown in the figure below. Figure 3 shown.

[0051] LGFI Block represents the local-global feature interaction module, which can fuse feature maps of different scales. Upsampling Block gradually restores the feature map to the original size through convolutional layers and upsampling operations. PredictionHead uses 1x1 convolutional layers and Sigmoid function to generate depth maps.

[0052] In PoseNet, the input image pair, I s∈[-1,1] indicates that the image pair is normalized. The present invention uses ResNet-18 as the backbone network of PoseNet to extract the features of the image. PoseNet generates the camera's pose change matrix P, reconstructs the image in the reconstruction module, and finally optimizes the depth map, i.e., Optimization. This process includes calculating the gradient of the loss function and optimizing the model weights to minimize the error.

[0053] Step 3: Calculate the loss function. The loss function includes reprojection loss, smoothing loss, and geometric consistency loss.

[0054] The reprojection loss is defined as shown in formula (1):

[0055]

[0056] In the formula, min is the minimization operation, and s here represents all adjacent frames. Here, the frame with the smallest error is selected for loss calculation. t is the current frame image, I s are adjacent frame images, I t→s The reconstructed image is obtained by reprojecting the current frame to the adjacent frame. SSIM is the structural similarity index, which is used to measure the structural similarity of two images. t -I t→s ∥1 is the L1 loss between the current frame and the reconstructed image, which is used to measure the brightness difference, and θ is a balance factor, which is 0.5 in the present invention, and is used to adjust the weight between SSIM and L1 loss.

[0057] The smoothing loss is defined as shown in formula (2):

[0058]

[0059] Where i and j represent the row index and column index of the pixel in the image, respectively. and is the gradient of the depth map in the x and y directions, and is the gradient of the input image in the x and y directions, and The weight coefficient used to weigh the depth gradient. Based on the gradient information of the image, the weight of the area with a large gradient (edge) will be reduced to keep the edge clear. ∑ means adding the weighted gradients in the horizontal and vertical directions and summing them for all pixels to obtain the final smoothing loss.

[0060] Since these two losses have some limitations in the optimization process, especially in processing geometric information and depth consistency, geometric consistency loss is introduced in the loss function calculation stage. Geometric consistency loss makes up for the shortcomings of reprojection and smoothing loss, effectively preventing the detail information improved by AIC-Block from being weakened by smoothing loss during the optimization process, ensuring that the depth estimation of edge objects is more accurate. At the same time, geometric consistency loss further enhances the consistency of spatial structure based on the important areas that AIC-Block focuses on, making the depth relationship of these key areas more reasonable, especially in complex edge areas. Input the depth map D obtained in step 2 t , the depth map D corresponding to adjacent frames t-1 And the camera pose change matrix generated by the pose estimation network. The depth map of the current frame is projected into the coordinate system of the adjacent frame through pose transformation, and then the difference between the reprojected depth and the actual depth of the adjacent frame is calculated as the geometric consistency loss. The definition of the geometric consistency loss function is shown in formula (3):

[0061]

[0062] Where N represents the number of valid pixels, which is used for normalization. x, y represent the coordinates of a pixel in the image. x′, y′ represent the corresponding pixel coordinates in the reference frame. t→t-1 represents the depth value projected from time frame t to time frame t-1. The camera pose change matrix generated by the pose estimation network can map the pixel (x, y) in the target frame to the corresponding position in the reference frame to obtain (x′, y′). The depth differences of all pixels are summed and divided by the total number of pixels N to obtain the average loss.

[0063] Finally, the geometric consistency loss function, the reprojection loss function, and the smoothing loss are combined to form the total loss function. total The definition is shown in formula (4):

[0064]

[0065] In the formula, α, β, and γ are hyperparameters used to balance the importance of each loss term. Among them, 0<α<1, 0<β<1, 0<γ<1, α+β+γ=1. In the depth estimation model, the three loss function terms are weighted and balanced. A coefficient less than 1 can prevent a certain loss from being too large, thereby dominating the optimization process and weakening the impact of other losses. Ensuring that the sum of these coefficients is 1 can maintain the stability of the overall loss, avoid disordered enlargement or reduction of the loss value, and enable the model to learn the characteristics of each loss term more balanced, thereby improving the comprehensive optimization effect of the network on various aspects of characteristics. In the present invention, α=0.7, β=0.15, γ=0.15. The geometric consistency loss calculation and the total loss function calculation are as follows: Figure 4 shown.

[0066] Step 4: Pass the training set divided in step 1 into the improved Lite-Mono model for training.

[0067] Step 5: Use the weights trained in step 4 to test the color image to obtain the final depth map.

[0068] The comparison of the indicators of the original model and the present invention on the KITTI validation set is shown in Table 1. It can be seen that the absolute relative error Abs Rel, average relative error Sq Rel, root mean square error RMSE, root mean square logarithmic error RMSE log, and threshold accuracy of the present invention are further improved.

[0069] Table 1 Index comparison

[0070] method AbsRel SqRel RMSE RMSElog δ<1.25 <![CDATA[δ<1.25 2 ]]> <![CDATA[δ<1.25 3 ]]> Lite-Mono 0.107 0.765 4.561 0.183 0.886 0.963 0.983 The present invention 0.090 0.733 4.387 0.169 0.911 0.974 0.989

[0071] like Figure 5a and 6a is a color image, Figure 5b and 6b is the original model output depth map, Figure 5c and 6c This is the output depth map of the present invention. It can be seen that after improving the network, the present invention improves the model's processing of image edge details and improves the accuracy and consistency of depth estimation.

[0072] The above description is only a preferred embodiment of the present invention and is not intended to limit the use of the method of the present invention. Any improvements made within the spirit and principles of the present invention should be included in the protection scope of the method of the present invention.

Claims

1. A self-supervised monocular depth estimation method based on the CBAM module combined with a geometric consistency loss function, characterized in that: The following steps are involved: Step 1: Data preparation and preprocessing. First, divide the monocular depth estimation related dataset into two parts: training set and validation set. In the preprocessing stage, first adjust the input image to a suitable resolution, usually the size required by the model input, to ensure that the image maintains its original proportion during the scaling process. Finally, perform random flipping, color adjustment, noise addition and other operations on the image to enhance data diversity, avoid model overfitting, and adapt it to the model input requirements. Step 2: Network structure design. The entire network structure is divided into a depth estimation network DepthNet and a posture network PoseNet. In DepthNet, image It is the color image input to DepthNet, and deep features are extracted through feature extraction and downsampling operations. The four stages correspond to feature extraction at different levels. Stage 1 contains the Conv Stem block, which is the initial feature extraction and is mainly responsible for extracting low-level features of the image (such as edges and textures). H and W represent the height and width of the image, respectively. At this time, the image height and width of this stage are and The number of channels is C1; AvgPool is average pooling to reduce the resolution; Concat passes the features of Stage 1 to Stage 2; Stage 2 includes the Downsample operation to reduce the spatial size of the input feature map; Stage 2 is the middle-level feature extraction, which is mainly responsible for capturing the basic shape and contour of the image; Stage 3 is the high-level feature extraction, which is mainly responsible for extracting more abstract and complex spatial relationships, and Stage 4 is the deep feature extraction to generate global and abstract semantic information; the same modules included in Stage 3 and Stage 4 have the same functions as above; In the original model, the Stage 2, Stage 3 and Stage 4 contain the continuous dilated convolution module CDC Block, which is used to further extract features. The attention mechanism (Attention-Infused), namely the CBAM module, is introduced into this module to form the Attention-Infused CDC Block (AIC-Block). In the AIC-Block, 3×3DWConv initially extracts features, processes information of different scales through dilated convolution, and then performs normalization operation through Batch Norm. Then, 1×1Point-wise reorganizes feature channels to reduce the amount of calculation, and then the GELU activation function improves the nonlinear expression ability. Finally, a point-wise convolution is performed to further optimize the combination of feature channels. LGFI Block represents the local-global feature interaction module, which can fuse feature maps of different scales. UpsamplingBlock gradually restores the feature map to the original size through convolutional layers and upsampling operations. Prediction Head uses 1x1 convolutional layers and Sigmoid functions to generate depth maps. In PoseNet, an image pair is input, Is∈[-1,1] means the image pair is normalized, ResNet is the backbone network of PoseNet, it extracts the features of the image, PoseNet generates the camera's pose change matrix P, reconstructs the image in the reconstruction module, and finally optimizes the depth map, i.e., Optimization; this process includes calculating the gradient of the loss function and optimizing the model weights to minimize the error; Step 3: Loss function calculation, where the loss function includes reprojection loss, smoothing loss, and geometric consistency loss; The reprojection loss is defined as shown in formula (1): In the formula, min is the minimization operation, s here represents all adjacent frames, and the frame with the smallest error is selected for loss calculation; It is the current frame image, Is is the adjacent frame image, It→s is the reconstructed image obtained by reprojecting the current frame to the adjacent frame, SSIM is the structural similarity index, which is used to measure the structural similarity of two images, ∥I t -I t→s ∥1 is the L1 loss between the current frame and the reconstructed image, which is used to measure the brightness difference. θ is the balance factor, which is used to adjust the weight between SSIM and L1 losses. The smoothing loss is defined as shown in formula (2): Where i and j represent the row index and column index of the pixel in the image, respectively. and is the gradient of the depth map in the x and y directions, and is the gradient of the input image in the x and y directions, and The weight coefficient used to weigh the depth gradient. Based on the gradient information of the image, the weight of the area with a large gradient (edge) will be reduced to keep the edge clear. ∑ means adding the weighted gradients in the horizontal and vertical directions and summing them for all pixels to get the final smoothing loss. Since these two losses have some limitations in the optimization process, especially in processing geometric information and depth consistency, geometric consistency loss is introduced in the loss function calculation stage. Geometric consistency loss makes up for the shortcomings of reprojection and smoothing loss, effectively preventing the detail information improved by AIC-Block from being weakened by smoothing loss during the optimization process, ensuring that the depth estimation of edge objects is more accurate. At the same time, geometric consistency loss further enhances the consistency of spatial structure based on the important areas concerned by AIC-Block, making the depth relationship of these key areas more reasonable, especially in complex edge areas. Input the depth map Dt obtained in step 2, the depth map Dt-1 corresponding to the adjacent frame, and the camera pose change matrix generated by the pose estimation network. Project the depth map of the current frame to the coordinate system of the adjacent frame through pose transformation, and then calculate the difference between the reprojected depth and the actual depth of the adjacent frame as the geometric consistency loss. The definition of the geometric consistency loss function is shown in formula (3): Where N represents the number of valid pixels, which is used for normalization; x, y represent the coordinates of a pixel in the image. x′, y′ represent the corresponding pixel coordinates in the reference frame; t→t-1 represents the depth value projected from time frame t to time frame t-1. The camera pose change matrix generated by the pose estimation network can map the pixel (x, y) in the target frame to the corresponding position in the reference frame to obtain (x′, y′). The depth differences of all pixels are summed and divided by the total number of pixels N to obtain the average loss. Finally, the geometric consistency loss function, the reprojection loss function and the smoothing loss are combined to form the total loss function; the total loss function Ltotal is defined as shown in formula (4): Where α, β, and γ are hyperparameters used to balance the importance of each loss term; 0<α<1, 0<β<1, 0<γ<1, α+β+γ=1; In the depth estimation model, the three loss function terms are weighted and balanced; A coefficient less than 1 can prevent a certain loss from being too large, thus dominating the optimization process and weakening the impact of other losses; Ensuring that the sum of these coefficients is 1 can keep the overall loss stable, avoid disorderly amplification or reduction of the loss value, and enable the model to learn the characteristics of each loss term more balanced, which can improve the comprehensive optimization effect of the network on multiple aspects of characteristics; Step 4: Pass the training set divided in step 1 into the improved Lite-Mono model for training; Step 5: Use the weights trained in step 4 to test the color image and obtain the final depth map.

Images