A monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion
By using Multi-Path Vision Transformer and multi-scale feature fusion method in monocular depth prediction, the problem of combining local and global information is solved, achieving efficient and accurate depth map generation and reducing computational costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANHUI UNIV OF SCI & TECH
- Filing Date
- 2023-06-21
- Publication Date
- 2026-04-17
AI Technical Summary
Existing monocular depth prediction methods struggle to effectively combine local and global information when utilizing convolutional neural networks, resulting in low depth prediction accuracy, high computational costs, and an inability to generate high-quality dense depth maps.
We adopt Multi-Path Vision Transformer (MPViT) as the backbone network, combined with multi-scale patch embedding and multi-path Transformer modules. Through multi-path feature extraction and multi-scale feature fusion, we utilize self-attention mechanism and skip connection method, combining the advantages of CNN and Transformer to improve the efficiency of feature extraction and fusion.
While improving the accuracy of depth prediction, it reduces computational costs, generates higher-quality dense depth maps, and enhances model performance and training speed.
Smart Images
Figure CN116758130B_ABST
Abstract
Description
Technical fields:
[0001] This invention belongs to the field of computer vision image processing. Specifically, it proposes a monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion. Background technology:
[0002] The statements in this section only involve the technical background related to the invention and do not necessarily constitute prior art.
[0003] Monocular depth prediction is a challenging and fundamental topic in computer vision that has long been actively researched. Its task is to generate pixel-level dense depth maps from a given input RGB color image using a depth prediction model. These depth maps contain distance information of the scene in the image relative to the camera. Accurate pixel-level scene depth information is crucial for performing higher-level computer vision tasks, including 3D reconstruction, autonomous driving, 3D object detection, and underwater image restoration. Generally, predicting scene depth directly from a single image using monocular depth prediction algorithms is a more cost-effective approach compared to costly and technically demanding LiDAR camera equipment. However, due to the inherent ambiguity of the monocular depth prediction problem, traditional methods often result in low inference accuracy and poor generalization, making it a challenging ill-posed problem. In recent years, with the advent of Convolutional Neural Networks (CNNs), many CNN-based methods have been widely applied to depth prediction tasks. These methods typically employ encoder-decoder-based model architectures and significantly improve the accuracy of depth prediction results.
[0004] Due to the lack of depth cues, fully utilizing long-range relevance (i.e., distance relationships between objects) and local information (i.e., consistency within objects) is crucial for accurate depth prediction. Researchers typically design convolutional operators of different scales at the encoder stage to expand the receptive field, capturing multi-scale features from coarse to fine, or incorporate attention mechanisms to integrate global contextual information. However, convolutional operators with limited receptive fields still struggle to adequately capture long-range relevance, which has become a potential bottleneck for current CNN-based depth prediction methods. Compared to CNNs, VisionTransformer (ViT) has achieved great success in pixel-level image recognition, demonstrating advantages in depth prediction encoders and attracting widespread attention. Thanks to its self-attention mechanism, Transformer excels at capturing long-range relevance with its global receptive field, is considered to have stronger generalization ability than CNNs, and is more similar to human cognitive processes. However, in dense prediction tasks, pixels must also understand local information. Transformer-based encoders increase the network's receptive field, but may lack spatial inductive bias when extracting local feature information and lose some scene and object texture information, failing to produce satisfactory performance. Direct upsampling to higher resolution and simple fusion can lead to the loss of local information. Therefore, in the decoder stage, researchers have designed a skip connection method based on the self-attention mechanism in Transformers to fuse multi-scale feature information extracted from the encoder and decoder stages, considering and passing on global and local information to recover image details and generate a refined depth map. However, these frameworks increase computational costs during inference in the feature aggregation stage. Therefore, in practical applications, it is necessary to consider how to effectively utilize and combine the respective advantages of CNNs and Transformers to improve model performance while increasing training speed. Summary of the Invention:
[0005] To address the aforementioned issues, this invention proposes a monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion, and constructs a novel encoder and lightweight decoder structure. In the encoder stage, a Multi-Path Vision Transformer (MPViT) is used as the backbone network for feature extraction. MPViT consists of multi-scale patch embedding (MS-PatchEmbed) and multi-path Transformer (MP-Transformer) modules, constructing a four-stage hierarchical structure. The output features of each stage are used to generate feature maps at different scales. During each stage, drawing inspiration from CNN models, overlapping convolution operations are performed on the patch embedding using multi-scale convolutional kernels with different receptive fields to produce features with the same resolution. Then, tokens of different scales are passed in parallel and independently to multiple Transformer encoders, performing global self-attention and additional convolutional branches to preserve local information. Finally, the obtained features are aggregated to ensure semantically rich and spatially accurate representations at each stage. This fine-grained and coarse-grained multi-level feature extraction helps achieve more accurate predictions and improves model performance. In the decoder stage, a lightweight feature fusion module, EDFI, is proposed. This module iteratively upsamples feature maps and utilizes a self-attention mechanism to effectively encode and fuse long-range global information from the decoder into the multi-scale features extracted by the encoder. This process starts at the lowest resolution and moves towards higher resolutions. This feature fusion approach helps transfer global information to high resolution while preserving local information features, thereby improving both training speed and the model's performance and the accuracy of predicted depth maps.
[0006] 1. A monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion, characterized in that the method includes the following steps:
[0007] 1) This monocular depth prediction method uses a pre-trained Multi-Path VisionTransformer (MPViT) model on ImageNet to construct a multi-scale patch embedding and multi-path feature extraction structure, and initializes the backbone network of the encoder network.
[0008] 2) The output feature map of the last layer obtained by the MPViT backbone network is processed by Pyramid Scene Parsing Network (PSP, Spatial Pyramid Pooling Module) to obtain semantic information at different scales, which effectively enhances the globality and relevance of the initial input features of the decoder.
[0009] 3) Using a bottom-up approach, starting from the lowest resolution output feature map, it iterates upsampling operations, uses attention mechanisms and skip connection methods to fuse encoder and decoder feature maps of the corresponding scale, performs multi-scale local feature and global context information aggregation, and predicts more accurate depth maps.
[0010] 4) Train and optimize the network model using scale-invariant loss (SILog) to calculate the difference between the predicted depth value and the true depth value.
[0011] 2. The monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion according to claim 1, characterized in that: step 1) is specifically a method of:
[0012] 2.1) Download the open-source official datasets for monocular depth prediction: NYU Depth v2 and KITTI. NYU Depth v2 provides RGB images and depth maps of different indoor scenes captured at a resolution of 640×480 pixels, with the depth map ranging from 0 to 10m. KITTI is a dataset of stereo images of outdoor scenes captured by a device mounted on a mobile vehicle and the corresponding 3D laser scans, with the depth map having an upper limit of 80m.
[0013] 2.2) For the KITTI dataset, center cropping was used to crop a subset of 26K left-view images to produce sub-images of size 352×704 as samples for training and 697 test set images for testing the network. For the NYU Depth v2 dataset, random cropping was used to randomly crop sub-images of size 448×576 from the original images in 249 scenes containing 50K images to train the network, and it was tested on 654 images.
[0014] 2.3) Download and use the pre-trained weight file of MPViT-Base to initialize the MPViT backbone network in the encoder. This avoids the problem of low efficiency and poor quality of feature map extraction caused by training from scratch, and can speed up the training of the model, avoid overfitting, and improve the performance of the model.
[0015] 3. The monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion according to claim 1, characterized in that: step 2) is specifically a method of:
[0016] 3.1) The MPViT backbone network first extracts initial multi-scale features from the input RGB image. The generated features are represented as follows: The resolutions are respectively those of the original image. The number of channels are {224,368,480,480}.
[0017] 3.2) This will include the most important details in the scene, which are also at the coarsest resolution. The feature maps at different scales are fed as input into the Spatial Pyramid Pooling (PSP) module for further processing. The PSP module uses multi-scale adaptive pooling to extract semantic information at different scales, with scales of {1×1, 2×2, 3×3, 6×6}.
[0018] 3.3) To maintain the same dimensionality as the input features Figure 1 Therefore, after performing pooling operations at different scales, a 1×1 convolution is used to transform the dimension of the feature map into the dimension of the input feature map. Then, these feature maps are upsampled using bilinear interpolation to restore them to the same size as the input feature maps, and a Concat operation is performed to merge them together. This effectively aggregates the entire scene information and yields a result of size [size missing]. The feature map, where N c1 The dimensions are set to 512, where H and W are the height and width of the input feature map, respectively. Finally, to further process the feature map while maintaining the resolution, this invention uses a convolutional layer to adjust the dimension of the feature map to 128.
[0019] 4. The monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion according to claim 1, characterized in that: step 3) is specifically a method of:
[0020] 4.1) For dense prediction tasks, a pixel must know whether it lies on the boundary of an object or belongs to a group of pixels on a common surface in order to make an accurate prediction. Therefore, this method constructs a lightweight and efficient module, EDFI, for encoder-decoder feature interaction, adaptively selecting and fusing global contextual semantic information and fine-grained local information. This module starts with the lowest resolution feature map and moves towards higher resolutions. First, the encoder... and decoder The feature maps are all changed to 128 channels through 1×1 convolutional layers to facilitate dimension matching. Then, a dual-path cross-attention mechanism and skip connections are used to pass through a series of convolutional layers. This enhances the ability to learn rich global context and dense local edge features without using complex upsampling methods or adding extra attention modules.
[0021] 4.2) In the encoder-decoder feature interaction module EDFI, firstly, a dual-path cross-attention mechanism module is constructed to adaptively focus on important locations. Specifically,
[0022]
[0023]
[0024]
[0025]
[0026]
[0027] Where i∈{1,2,3,4}, W K W Q and W V It is a weight matrix. After convolution, the size is {H} i W i N c The encoder feature map of}, where N c It is 128. It corresponds to the scale {H} i W i N c The decoder feature map of} It is a feature obtained by concatenating two features along the channel dimension. The size is {H i W i ,2N c After one convolutional layer, the dimension is reduced to N. c .
[0028] Enhance feature mapping using cross-covariance attention mechanism and Specifically
[0029]
[0030]
[0031]
[0032] Where Attention(Q,K,V)=V·Softmax(Q T K).
[0033] Then, increase the non-linearity of the features, specifically,
[0034]
[0035] Where LN represents the layer normalization operation, ReLU represents the activation function, conv1 represents a 1×1 convolution, and DWConv rThis represents a 3×3 dilated convolution with an expansion rate of r.
[0036] 4.3) In the decoder, the initial inputs of the encoder and decoder feature interaction module EDFI are respectively the output features of the PSP module. and the output features of the last layer of the MPViT backbone network Then it is obtained by upsampling using UpSample. Specifically
[0037]
[0038] Then, by fusing encoder features at different scales Output features of the decoder This helps the initial feature mapping gradually enhance global contextual information and local continuity. Specifically,
[0039]
[0040] Here, i∈{1,2,3}.
[0041] Finally, the obtained decoder output features After convolution, ReLU activation function, convolution dimensionality reduction and Sigmoid activation function operations, the weights of each channel are normalized to between 0 and 1, and features are further extracted and fused to predict a depth map H×W×1. Then the depth map is multiplied by the maximum depth value (in meters) for scaling.
[0042] 5. The monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion according to claim 1, characterized in that: step 4) is specifically a method of:
[0043] This invention uses scale-invariant loss (SILog) to optimize and train the network model, and calculates the predicted depth d at pixel position i. i With effective true depth d i * The logarithmic distance between them. The SILog loss function uses the number of pixels surrounding each pixel as a scaling factor, which is better suited for smaller objects and reduces the error rate.
[0044]
[0045] Here, n represents the number of pixels in the image. λ = 0.85, α = 10. Attached Figure Description
[0046] Figure 1Flowchart of a monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion
[0047] Figure 2 MPViT backbone network architecture diagram
[0048] Figure 3 PSP module details
[0049] Figure 4 Detailed diagram of the encoder and decoder feature interaction module EDFI
[0050] Figure 5 Comparison table of results on the NYU Depth v2 dataset Detailed Implementation
[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Furthermore, the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0052] The process framework of this invention is as follows: Figure 1 The flowchart shown illustrates a monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion. The specific operations are as follows:
[0053] 1. This monocular depth prediction method uses a pre-trained Multi-Path VisionTransformer (MPViT) model on ImageNet to construct a multi-scale patch embedding and multi-path feature extraction structure, and initializes the backbone network of the encoder network.
[0054] 1.1 Download the open-source official datasets for monocular depth prediction: NYU Depth v2 and KITTI. NYU Depth v2 provides RGB images and depth maps of different indoor scenes captured at a resolution of 640×480 pixels, with the depth maps ranging from 0 to 10m. KITTI is a dataset of stereo images of outdoor scenes captured by a device mounted on a mobile vehicle and the corresponding 3D laser scans, with the depth map having an upper limit of 80m.
[0055] 1.2 For the KITTI dataset, center cropping was used to crop a subset of 26K left-view images to produce sub-images of size 352×704 as samples for training, and 697 test set images were used to test the network. For the NYU Depth v2 dataset, random cropping was used to randomly crop sub-images of size 448×576 from the original images in 249 scenes containing 50K images to train the network, and 654 images were used for testing.
[0056] 1.3 Download and use the pre-trained weight file from MPViT-Base to initialize the MPViT backbone network in the encoder. This avoids the problems of low efficiency and poor quality feature map extraction caused by training from scratch, and can accelerate the model training speed, avoid overfitting, and improve model performance. The overall network architecture is as follows: Figure 2 As shown.
[0057] 2. The final layer output feature map obtained from the MPViT backbone network is processed by the Pyramid Scene Parsing Network (PSP, Spatial Pyramid Pooling Module) to obtain semantic information at different scales, effectively enhancing the globality and relevance of the initial input features of the decoder. A detailed diagram of the PSP module is shown below. Figure 3 As shown.
[0058] 2.1 The MPViT backbone network first extracts initial multi-scale features from the input RGB image. The generated features are represented as follows: The resolutions are respectively those of the original image. The number of channels are {224,368,480,480}.
[0059] 2.2 will include the most important details in the scene, which are also at the coarsest resolution. The feature maps at different scales are fed as input into the Spatial Pyramid Pooling (PSP) module for further processing. The PSP module uses multi-scale adaptive pooling to extract semantic information at different scales, with scales of {1×1, 2×2, 3×3, 6×6}.
[0060] 2.3 To maintain the same dimensionality as the input features Figure 1 Therefore, after performing pooling operations at different scales, a 1×1 convolution is used to transform the dimension of the feature map into the dimension of the input feature map. Then, these feature maps are upsampled using bilinear interpolation to restore them to the same size as the input feature maps, and a Concat operation is performed to merge them together. This effectively aggregates the entire scene information and yields a result of size [size missing]. The feature map, where Nc1 The dimensions are set to 512, where H and W are the height and width of the input feature map, respectively. Finally, to further process the feature map while maintaining the resolution, this invention uses a convolutional layer to adjust the dimension of the feature map to 128.
[0061] 3. Using a bottom-up approach, starting with the lowest-resolution output feature map, it iteratively upsampling operations are performed. Attention mechanisms and skip connections are then used to fuse encoder and decoder feature maps at corresponding scales, aggregating multi-scale local features and global contextual information to predict a more accurate depth map. A detailed diagram of the encoder and decoder feature interaction module (EDFI) is shown below. Figure 4 As shown
[0062] 3.1 For dense prediction tasks, a pixel must know whether it lies on the boundary of an object or belongs to a group of pixels on a common surface in order to make an accurate prediction. Therefore, this method constructs a lightweight and efficient module, EDFI, for encoder-decoder feature interaction, adaptively selecting and fusing global contextual semantic information and fine-grained local information. This module starts with the lowest resolution feature map and moves towards higher resolutions. First, the encoder... and decoder The feature maps are all changed to 128 channels through 1×1 convolutional layers to facilitate dimension matching. Then, a dual-path cross-attention mechanism and skip connections are used to pass through a series of convolutional layers. This enhances the ability to learn rich global context and dense local edge features without using complex upsampling methods or adding extra attention modules.
[0063] 3.2 In the encoder-decoder feature interaction module EDFI, firstly, a dual-path cross-attention mechanism module is constructed to adaptively focus on important locations. Specifically,
[0064]
[0065]
[0066]
[0067]
[0068] Where i∈{1,2,3,4}, W K W Q and W V It is a weight matrix. After convolution, the size is {H} i W i N c The encoder feature mapping of} It corresponds to the scale {H}i W i N c The decoder feature map of} It is a feature obtained by concatenating two features along the channel dimension. The size is {H i W i ,2N c After one convolutional layer, the dimension is reduced to N. c .
[0069] Enhance feature mapping using cross-covariance attention mechanism and Specifically
[0070]
[0071]
[0072]
[0073] Where Attention(Q,K,V)=V·Softmax(Q T K).
[0074] Then, increase the non-linearity of the features, specifically,
[0075]
[0076] Where LN represents the layer normalization operation, ReLU represents the activation function, conv1 represents a 1×1 convolution, and DWConv r This represents a 3×3 dilated convolution with an expansion rate of r.
[0077] 3.3 In the decoder, the initial inputs of the encoder and decoder feature interaction module EDFI are respectively the output features of the PSP module. and the output features of the last layer of the MPViT backbone network Then it is obtained by upsampling using UpSample. Specifically
[0078]
[0079] Then, by fusing encoder features at different scales Output features of the decoder This helps the initial feature mapping gradually enhance global contextual information and local continuity. Specifically,
[0080]
[0081] Here, i∈{1,2,3}.
[0082] Finally, the obtained decoder output features After convolution, ReLU activation function, convolution dimensionality reduction and Sigmoid activation function operations, the weights of each channel are normalized to between 0 and 1, and features are further extracted and fused to predict a depth map H×W×1. Then the depth map is multiplied by the maximum depth value (in meters) for scaling.
[0083] 4. To optimize and train the network model using scale-invariant loss (SILog) in this invention, the predicted depth d at pixel position i is calculated. i With effective true depth d i * The logarithmic distance between them. The SILog loss function uses the number of pixels surrounding each pixel as a scaling factor, which is better suited for smaller objects and reduces the error rate.
[0084]
[0085] Here, n represents the number of pixels in the image. λ = 0.85, α = 10. The results of this method compared with other monocular depth prediction methods on the NYU Depth v2 dataset are shown in the table below. Figure 5 As shown.
[0086] The above description is for preferred embodiments of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. A monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion, characterized in that, The method includes the following steps: 1) This monocular depth prediction method utilizes the Multi-Path VisionTransformer (MPViT) model pre-trained on ImageNet to construct multi-scale patch embedding and multi-path feature extraction structures, and initializes the backbone network of the encoder network. 2) The output feature map of the last layer obtained by the MPViT backbone network is processed by the Pyramid Scene Parsing Network, namely the PSP spatial pyramid pooling module, to obtain semantic information at different scales, which effectively enhances the globality and relevance of the initial input features of the decoder. 3) Using a bottom-up approach, starting with the lowest resolution output feature map, it iteratively upsamples the encoder and decoder feature maps at corresponding scales using attention mechanisms and skip connections. This allows for multi-scale aggregation of local features and global contextual information, resulting in more accurate depth map predictions. Specific steps include: 3.1) For dense prediction tasks, a pixel must know whether it lies on the boundary of an object or belongs to a group of pixels on a common surface for accurate prediction. Therefore, a lightweight and efficient module, EDFI, is constructed to facilitate feature interaction between the encoder and decoder. This module adaptively selects and fuses global contextual semantic information and fine-grained local information. Starting with the lowest resolution feature map and moving towards higher resolutions, it first optimizes the encoder... and decoder Feature maps all passed The convolutional layer changes the number of channels to 128 to facilitate dimension matching; then, a dual-path cross-attention mechanism and skip connections are used through a series of convolutional layers to enhance the ability to learn rich global context and dense local edge features without using complex upsampling methods or adding additional attention modules. 3.2) In the encoder and decoder feature interaction module EDFI, a dual-path cross-attention mechanism module is first constructed to adaptively focus on important locations: (1) (2) (3) (4) (5) in, , , and It is a weight matrix. The size after convolution is encoder feature mapping, It is 128. It is the corresponding scale Decoder feature mapping, It is a feature obtained by concatenating two features along the channel dimension. The size is After one convolutional layer, the dimension is reduced to ; Enhance feature mapping using cross-covariance attention mechanism and , (6) (7) (8) in, ; Then, increase the nonlinearity of the feature. (9) Where LN represents the layer normalization operation, ReLU represents the activation function, and conv1 represents... convolution, Indicates a property with an expansion rate of r. Dilated convolution; 3.3) In the decoder, the initial inputs of the encoder and decoder feature interaction module EDFI are respectively the output features of the PSP module. and the output features of the last layer of the MPViT backbone network Then through Upsampling , (10) Then, by fusing encoder features at different scales Output features of the decoder This helps the initial feature mapping gradually enhance global contextual information and local continuity. (11) ； Finally, the obtained decoder output features After convolution, ReLU activation, convolutional dimensionality reduction, and Sigmoid activation, the weights of each channel are normalized to between 0 and 1. Features are then further extracted and fused to predict the depth map. Then the depth map is multiplied by the maximum depth value and scaled. 4) Use scale-invariant loss (SILog) to train and optimize the network model, and calculate the difference between the predicted depth value and the true depth value.
2. The monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion according to claim 1, characterized in that: The specific method for step 1) is as follows: 1.1) Download the open-source official datasets for monocular depth prediction: NYU Depth v2 and KITTI. NYU Depth v2 provides... RGB images and depth maps of different indoor scenes captured at pixel resolution, with depth maps ranging from 0 to 10m. KITTI is a dataset of stereo images of outdoor scenes and corresponding 3D laser scans captured by a device mounted on a mobile vehicle, with a depth map upper limit of 80m. 1.2) For the KITTI dataset, center cropping is used to crop a subset of left-view images from 26K to a size of [size missing]. Sub-images were used as samples for training and 697 test set images were used to test the network; for the NYUDepth v2 dataset, random cropping was used to randomly crop a portion of the original images from 249 scenes containing 50K images to a size of [missing information]. The network was trained on sub-images and tested on 654 images; 1.3) Download and use the pre-trained weight file of MPViT-Base to initialize the MPViT backbone network in the encoder. This avoids the problem of low efficiency and poor quality of feature map extraction caused by training from scratch, and can speed up the training of the model, avoid overfitting, and improve the performance of the model.
3. The monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion according to claim 1, characterized in that: The specific method for step 2) is as follows: 2.1) The MPViT backbone network first extracts initial multi-scale features from the input RGB image. The generated features are represented as follows: The resolutions are respectively those of the original image. The number of channels are respectively ; 2.2) This will include the most important details in the scene, which are also at the coarsest resolution. The feature maps at different scales are fed as input into the Spatial Pyramid Pooling (PSP) module for further processing. The PSP module uses multi-scale adaptive pooling to extract semantic information at different scales, with the scales being... ; 2.3) To maintain the same dimensionality between the output feature map and the input feature map, after performing pooling operations at different scales, use... Convolution transforms the dimension of the feature map into the dimension of the input feature map. Then, these feature maps are upsampled using bilinear interpolation to restore them to the same size as the input feature maps, and a Concat operation is performed to merge them together. This effectively aggregates the entire scene information and yields a result of size [size missing]. The feature map, where The dimensions are set to 512, where H and W are the height and width of the input feature map, respectively. Finally, to further process the feature map and maintain the resolution, a convolutional layer is used to adjust the dimensions of the feature map to 128.
4. The monocular depth prediction method based on multi-path feature extraction and multi-scale feature fusion according to claim 1, characterized in that: The specific method for step 4) is as follows: The predicted depth at pixel location i is calculated using scale-invariant loss (SILog) optimization and training. With effective face true depth The SILog loss function uses the logarithmic distance between pixels and the number of pixels surrounding each pixel as a scaling factor, which better handles smaller objects and reduces the error rate. The specific formula is as follows: (12) Here, n represents the number of pixels in the image. , , .