Monocular absolute depth estimation method based on BEiT relative depth mapping network

Abstract

The invention discloses a monocular absolute depth estimation method based on a BEiT relative depth mapping network. The method comprises the steps of 1, obtaining a training sample set B and a test sample set C; 2, constructing a BEiT-based relative depth mapping network model Sr which is used for predicting the relative depth of an input image; the input image adopts a single RGB image; 3, performing iterative training on the BEiT-based relative depth mapping model Sr based on the training sample set B; and 4, based on the test sample set C, obtaining a depth estimation result of the image. The depth estimation model constructed by the method needs fewer parameters. A relative depth estimation and absolute depth estimation combined method is used, an accurate and reliable absolute depth map and excellent generalization performance are obtained at the same time, and only a single RGB image is needed to serve as input.

Description

Technical Field

[0001] The present invention belongs to the technical field of depth estimation, and in particular relates to a monocular absolute depth estimation method based on a BEiT relative depth mapping network. Background Art

[0002] The field of depth estimation has two major branches: relative depth estimation (RelativeDepthEstimation) and absolute depth estimation (MetricDepthEstimation). Currently, most absolute depth estimation methods face common challenges: poor accuracy and poor generalization performance. Depth estimation models typically perform well on a single dataset, but this good performance often cannot be effectively transferred to other datasets. For example, a model trained in indoor scenes will experience severe accuracy degradation when used in outdoor scenes.

[0003] The goal of relative depth estimation is to restore the relative position relationship between pixels rather than the scale relationship. Since relative depth maps do not contain scale information, the application scenarios of relative depth estimation are limited. However, one of the unique advantages of relative depth estimation is that it does not require a complete fit of pixel depth scale information. Therefore, it can be trained in multiple domains, significantly improving the generalization ability of the model.

[0004] Patent application publication number CN119579666A, titled "Method for Depth Estimation from Event Cameras Based on Unsupervised Domain Adaptation," discloses an unsupervised depth estimation method applicable to the field of event camera depth estimation in computer vision. However, this method's training relies on specialized equipment, which places high demands on hardware and is difficult to extend to other application scenarios, resulting in limited practicality.

[0005] The patent application with application publication number CN119559326A and name “Monocular real-time intensive depth estimation method and system” provides a monocular real-time intensive depth estimation method and system; however, the invention needs to keep the training scene and the usage scene consistent, and it is difficult to produce accurate estimates for unseen scenes. Moreover, the accuracy of the estimation results of the invention requires the scene to have obvious collective characteristics, which limits the generalization performance of the invention.

[0006] Existing monocular absolute depth estimation methods suffer from poor generalization performance and lack reliable methods for improvement. Accurate monocular absolute depth estimation is crucial for addressing downstream tasks in 3D perception and modeling. However, the high accuracy of most current MMDE methods is limited to their training domain and cannot be effectively extended. Even with a moderate domain gap, these methods fail to generalize to unseen domains, limiting the practical application of monocular depth estimation.

[0007] The models that currently perform outstandingly in mainstream indoor scene datasets and outdoor scene datasets often have extremely large numbers of parameters. Summary of the Invention

[0008] To overcome the shortcomings of the above-mentioned prior art, the present invention aims to provide a monocular absolute depth estimation method based on the BEiT relative depth mapping network. The depth estimation model constructed by this method requires fewer parameters. By combining relative depth estimation with absolute depth estimation, an accurate and reliable absolute depth map and excellent generalization performance are obtained, while only requiring a single RGB image as input.

[0009] In order to achieve the above object, the technical solution adopted by the present invention is:

[0010] A monocular absolute depth estimation method based on the BEiT relative depth mapping network includes the following steps:

[0011] Step 1: Obtain training sample set B and test sample set C , where B and C It is a set of RGB images containing depth information;

[0012] Step 2: Build a relative depth mapping network model S based on BEiT (pre-trained Image Transformer (BEiT)) r : Used to predict the relative depth of the RGB input image; the input image uses a single RGB image (color image);

[0013] Step 3: Based on the training sample set B, the relative depth mapping model S based on BEiT r Perform iterative training and use the test sample set C Conduct effectiveness testing;

[0014] Step 4: Use the trained model to obtain the depth estimation result of the image.

[0015] The step 1 is specifically as follows:

[0016] Use the NYU-DepthV2 dataset as the reference image, use a portion of the reference images as the training set B, and the remaining reference images as the test set C .

[0017] The step 2 is specifically as follows:

[0018] Step 2.1: Construct a standard BEiT network as the first-stage network. The standard BEiT network consists of a linear mapping module and an encoder. The linear mapping module is used to process 2D image patches, and the encoder consists of an attention module and an MLP stack.

[0019] The first-stage network predicts the relative depth of the RGB input image and outputs a feature vector to the second-stage network;

[0020] The linear mapping module first divides the two-dimensional RGB input image into a series of image patches so that the standard Transformer can directly accept image data; formally, the RGB input image Reshape to N = HW / P 2 Image blocks Where C is the number of channels, (H, W) is the resolution of the input image, and (P, P) is the resolution of each image block;

[0021] Image Block After being flattened into vectors and projected through a linear mapping, the image patch retains the original pixels and is used as the input of BEiT;

[0022] The decoder composed of the attention module and MLP represents the RGB input image as a discrete tag sequence obtained by the "image encoder" (imagetokenizer), and the RGB input image Tokenized The vocabulary Contains discrete tag indices so that the subsequent attention mechanism module accepts them as input and extracts feature information to output feature vectors.

[0023] The step 2.1 is specifically as follows:

[0024] The image tagger learned by discrete variational autoencoder (dVAE) is used to adapt the standard Transformer structure to accept images as input. There are two modules in the visual tagging learning process, namely encoder and decoder. The encoder and decoder are composed of attention modules and MLP stacks to extract feature information and output feature vectors.

[0025] Encoder q φ (z|x) maps the input image x to discrete tags z according to the visual codebook (i.e., vocabulary), and the decoder p ψ (x|z) learns to reconstruct the input image x based on the discrete label z;

[0026] In order to make the originally non-differentiable target approximately differentiable, the target needs to be reconstructed. The reconstructed target is written as Gumbel-softmax relaxation is used to train the model parameters. In addition, during the training of discrete variational autoencoder (dVAE), q φA uniform prior is imposed.

[0027] Each input image to be predicted is tokenized into a 14×14 grid of visual markers, with the number of visual markers being the same as the number of image patches per image.

[0028] Based on the BEiT (BERT pre-trained Image Transformer) network construction method, the standard Transformer is used as the first-order backbone network of the model;

[0029] The input of the Transformer is a series of image blocks The image block is linearly projected to obtain the image block embedding in Add a special token [S] before the input sequence to embed the standard learnable one-dimensional position Added to the image patch embedding, the input vector is input into the Transformer;

[0030] The encoder consists of L layers of Transformer modules H l =Transformer(H l-1 ), where l=1,…,L. The output vector of the last layer is used as the encoding representation of the image block, where is the vector of the i-th image block.

[0031] Step 2.2, constructing a relative depth mapping module P as a second-stage network; the second-stage network predicts the absolute depth of the input image based on the relative depth and the feature vector;

[0032] The second-stage network consists of two modules: a feature projection module and a depth prediction module. The feature projection module projects the feature vectors of different dimensions output by the first-stage model, while maintaining the same dimension. A 1*1 convolutional layer is used as the implementation of the feature projection module.

[0033] The step 2.2 is specifically as follows:

[0034] The depth prediction module realizes the multi-scale refinement of the box by receiving the feature vector and generating a set of depth prediction vectors. The depth prediction vector is used to move left and right in the depth interval to realize the multi-scale refinement. The multi-scale features are used to refine the box in the first th At the decoder layer, a multi-layer perceptron (MLP) takes as input the relative depth features output by the relative depth map module P and predicts n for that pixel location. a Attraction points k:k=1,…,n a}, the adjusted predicted value is c′ i =c i +Δc i , the adjustment amount is given by the following formula:

[0035]

[0036] Among them, the hyperparameters α and γ determine the attraction strength. The present invention uses α = 300, γ = 2. This adjustment method is called the fractional adjustment method; it is used for indoor data;

[0037] A variation of the index adjustment method, with the following formula:

[0038]

[0039] For outdoor data.

[0040] Step 2.3: Build the depth prediction network Q, integrate the output of the relative depth prediction network P, and regress the relative depth estimated by P to the final absolute image depth estimate. The implementation process is as follows:

[0041] q=MLP(y)

[0042] Among them, q represents the final depth estimation value, MLP represents the multi-layer perceptron, and y represents the output relative image depth estimation value.

[0043] The first-stage network predicts the relative depth of the RGB input image and outputs a feature vector to the second-stage network. The second-stage network predicts the absolute depth of the input image based on the relative depth information and the feature vector.

[0044] The step 3 is specifically as follows:

[0045] Step 3.1, initialize the number of iterations to t, the maximum number of iterations to T, and the current relative depth mapping model based on BEiT is S t , and let t = 1, S t =S;

[0046] Step 3.2: Randomly select b training samples from the training sample set B without replacement as the relative depth mapping model S tAs the number of network layers increases, the standard BEiT network extracts global and local features of the image and fuses them, transferring the image's shallow feature information, including color and texture, geometric shape, and the feature information of the current layer layer by layer. Global features are insensitive to detailed information such as texture and material, while local features lack a global perspective. The fusion of global and local features can effectively overcome these shortcomings and enable more accurate inference. The depth value regression network regresses the fused features to obtain the depth prediction map corresponding to each training sample. The depth prediction map is compared with the true value of the GroundTruth sample, and then the backpropagation training process is carried out.

[0047] Step 3.3: First, calculate the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample as the model S t The loss value of the network model S is then optimized using the adaptive moment estimation algorithm. t Update the weight parameters of each layer;

[0048] The formula for calculating the scale-shift invariant loss function (ScaleandshiftInvariantloss) of the depth estimate corresponding to each training sample and the actual value corresponding to the training sample is

[0049]

[0050] Where T represents the total number of images in each batch of training data, y j and They represent the depth estimation value and actual depth value predicted by the model corresponding to the j-th image in the batch respectively;

[0051] Adaptive moment estimation is used as the optimization algorithm, and the learning rate is adjusted every s generations during the training process to speed up the convergence speed and improve the learning accuracy. The adjustment formula is as follows:

[0052]

[0053] Where t represents the current iteration number, s represents the learning rate update step size, d represents the learning rate attenuation coefficient, lr t Indicates the learning rate when the model iterates to t times;

[0054] Step 3.4: Determine whether the training sample set B has been traversed. If so, execute step 3.5; otherwise, execute steps 3.2 and 3.3.

[0055] Step 3.5, determine whether t=T is true. If so, obtain the trained BEiT-based relative depth mapping network model S * =S t, otherwise, let t = t + 1 and execute steps 3.3 and 3.4.

[0056] The step 4 is specifically as follows:

[0057] The test sample set C is used as the relative depth mapping model S of the trained base BEiT t The input is used for forward reasoning to obtain the quality prediction score of each test sample to verify the model effect.

[0058] Beneficial effects of the present invention:

[0059] The present invention achieves the mapping from relative depth estimation results to absolute depth estimation through a two-stage model based on BEiT. The first-stage network only performs relative depth estimation, thereby retaining the generalization performance of the relative depth estimation network. In the second-stage network, the relative depth is gradually adjusted to the absolute depth. This improves the generalization performance of the entire model and obtains more accurate absolute depth estimation. Moreover, since the effective interactive fusion of global and local image features in the BEiT architecture is fully considered, the shortcomings of traditional depth estimation models that are not sensitive enough to the detailed features of the input image are improved, making the prediction results more accurate. The present invention has the advantages of strong model generalization performance and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a schematic flow chart of the present invention. DETAILED DESCRIPTION

[0061] The present invention will be described in further detail below with reference to the accompanying drawings.

[0062] like Figure 1 As shown, the monocular absolute depth estimation method based on the BEiT relative depth mapping network includes the following steps:

[0063] Step 1: Get the training sample set B and the test sample set C:

[0064] In this example, the NYU-DepthV2 dataset is used, with 80% of the reference images used as the training set and the remaining 20% ​​as the test set. This ensures that there are no images with the same scene and content in the training and test sets.

[0065] Step 2: Build a relative depth mapping network model S based on BEiT r :

[0066] In step 2.1, a standard BEiT network is constructed as the first-stage network, which consists of a linear mapping for processing 2D image patches and an encoder consisting of an attention module and an MLP stack.

[0067] The two-dimensional image is divided into a series of image patches so that the standard Transformer can directly accept image data. Formally, we divide the image into a series of image patches so that the standard Transformer can directly accept image data. Reshape to N = HW / P 2 Image blocks Where C is the number of channels, (H, W) is the resolution of the input image, and (P, P) is the resolution of each image block. is flattened into a vector and linearly projected. The image patches retain their original pixels and serve as the input features for BEiT. In this example, we split each 224×224 image into a 14×14 grid of patches, where each patch is 14×14 in size.

[0068] This method represents an image as a sequence of discrete tokens obtained through an "image tokenizer" instead of raw pixels. Tokenized as z=[z1,…,z N ]∈V h×w , where the vocabulary V = {1,…,|V|} contains discrete token indices.

[0069] Use an image tagger learned through discrete Variational Autoencoder (dVAE). There are two modules in the visual tagging learning process, namely encoder and decoder. Encoder q φ (z|x) maps image pixels x to discrete tags z according to the visual codebook (i.e. vocabulary). Decoder p ψ (x|z) learns to reconstruct the input image x based on the visual mark z. The reconstruction objective can be written as Since the underlying visual labels are discrete, model training is non-differentiable. Gumbel-softmax relaxation is used to train the model parameters. In addition, during the training of the discrete Variational Autoencoder (dVAE), the q φ A uniform prior is imposed.

[0070] Each image is tokenized into a 14×14 grid of visual tokens. Note that the number of visual tokens is the same as the number of image patches per image. The vocabulary size is set to

[0071] Following the construction of the ViT (Vision Transformer) network, a standard Transformer is used as the backbone network. Therefore, the results can be directly compared with previous work in terms of network architecture.

[0072] The input of the Transformer is a series of image blocks These image patches are linearly projected to obtain the image patch embeddings in Add a special token [S] before the input sequence. Embed the standard learnable one-dimensional position Added to the image patch embedding. Input vector is input into the Transformer. The encoder contains L layers of Transformer modules H l =Transformer(H l-1 ), where l=1,…,L. The output vector of the last layer is used as the encoding representation of the image block, where is the vector of the i-th image block.

[0073] Step 2.2, construct the relative depth mapping module P as the second stage network;

[0074] The second-stage network consists of two modules: a feature projection module and a depth prediction module. The feature projection module projects feature blocks at different latitudes while maintaining the same latitude. We use a 1x1 convolutional layer to implement the feature projection module.

[0075] The feature projection module realizes multi-scale refinement of the box by generating a set of depth prediction vectors. The multi-scale refinement is realized by moving left and right on the depth interval through the depth prediction vector. th The decoder layer, a multi-layer perceptron (MLP), takes the features at the pixel as input and predicts n for that pixel location. a Attraction points k :k=1,…,n a The adjusted predicted value is c′ i =c i +Δc i , the adjustment amount is given by the following formula:

[0076]

[0077] Among them, the hyperparameters α and γ determine the attraction strength. This adjustment method is called the fractional adjustment method. Another exponential adjustment method variant of this application is as follows:

[0078]

[0079] Step 2.3: Build the depth prediction network Q and regress to the final image quality score. The implementation process is as follows:

[0080] q=MLP(y)

[0081] Among them, q represents the final depth estimation value, MLP represents the multi-layer perceptron, and y represents the final output image depth estimation value.

[0082] Step 3: Iteratively train the BEiT-based relative depth mapping model:

[0083] Step 3.1, initialize the number of iterations to t, the maximum number of iterations to T, and the current relative depth mapping model based on BEiT is S t , and let t = 1, S t =S;

[0084] In this embodiment, T=100;

[0085] Step 3.2: Randomly select b training samples from the training sample set B without replacement as the relative depth mapping model S t As the number of network layers increases, the standard BEiT network extracts global features and local features of the image for fusion, transfers the shallow feature information of the image and the feature information of the current layer layer by layer, and regresses the depth value regression network to obtain the depth prediction map corresponding to each training sample;

[0086] In this embodiment, b=4;

[0087] Step 3.3: First, calculate the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample as the model S t The loss value of the network model S is then optimized using the adaptive moment estimation algorithm. t Update the weight parameters of each layer;

[0088] In this embodiment, the formula for calculating the scale-shift invariant loss function (ScaleandshiftInvariantloss) of the depth estimate corresponding to each training sample and the actual value corresponding to the training sample is:

[0089]

[0090] Where T represents the total number of images in each batch of training data, y j and They represent the depth estimation value and actual depth value predicted by the model corresponding to the j-th image in the batch, respectively.

[0091] In this embodiment, adaptive moment estimation is used as the optimization algorithm, and the initial learning rate lr is set to 10 -4 , and adjust the learning rate every s generations during training to speed up convergence and improve learning accuracy. The adjustment formula is as follows:

[0092]

[0093] Where t represents the current iteration number, s represents the learning rate update step size, d represents the learning rate attenuation coefficient, lr t Indicates the learning rate when the model is iterated to t times. In addition, to alleviate the overfitting phenomenon of noise and abnormal samples in the training data during model training, weight decay weight_decay is added during the training process.

[0094] In this embodiment, s=10, d=0.9, weight_decay=10 -4 .

[0095] Step 3.4: Determine whether the training sample set B has been traversed. If so, execute step 3.5; otherwise, execute steps 3.2 and 3.3.

[0096] Step 3.5, determine whether t=T is true. If so, obtain the trained BEiT-based relative depth mapping network model S * =S t Otherwise, set t = t + 1 and execute steps 3.3 and 3.4;

[0097] Step 4: Get the depth estimation result of the image:

[0098] The test sample set C is used as the relative depth mapping model S of the trained base BEiT t The input is used for forward reasoning to obtain the quality prediction score of each test sample to verify the model effect.

[0099] The following is a description of the technical effects of the present invention in conjunction with simulation experiments:

[0100] 1. Simulation conditions and content:

[0101] The hardware platform of the simulation experiment of the present invention is: the processor is Intel (R) Core (TM) i9-7900X CPU, the main frequency is 3.30GH, the memory is 32GB, and the graphics card is NVIDIA GeForce RTX 2004.

[0102] The software platform for the simulation experiment of the present invention is: Ubuntu 16.04 operating system, Pytorch 1.6.0 framework, and Python 3.9.

[0103] 2. Simulation Experiment

[0104] The input images used in the simulation experiment of the present invention are from the image quality evaluation database NYU-DepthV2. The NYU-DepthV2 dataset is a depth estimation-image segmentation dataset proposed by Nathan Silberma et al. in “Indoor segmentation and support inference from RGBD images. 12th European Conference on Computer Vision.”

[0105] The simulation experiment of the present invention uses two indicators, RMSE (Root mean squared error) and ARE (Absolute relative error) of the Pearson scale, to judge the image quality evaluation effects of the present invention and the prior art respectively. Specifically, the prior art and the present invention respectively select n samples from the test sample set C for depth prediction, output the quality depth prediction value, and calculate the RMSE and ARE values ​​according to the quality prediction score of the sample and the quality label score corresponding to the sample. Among them, the two indicators are calculated according to the following formulas. The lower the value, the more accurately the evaluation result of the monocular depth estimation method being judged can restore the depth information of the image.

[0106]

[0107]

[0108] Where N is the total number of images, d i Represents the depth prediction value of the i-th image. Represents the depth prediction value of the i-th image.

[0109] The simulation results are shown in Table 1.

[0110] Table 1 Comparison of evaluation results of the present invention and prior art

[0111]

[0112] As can be seen from Table 1, compared with the existing technology, the root mean square error RMSE and scale relative error ARE of the evaluation results of the present invention on the NYU-DepthV2 dataset are lower, proving that the accuracy of the monocular depth estimation method is higher.

[0113] The present invention realizes the interactive fusion of relative depth and absolute depth of images by gradually transferring BEiT shallow network feature information to deep network feature information, and on this basis realizes monocular absolute depth estimation of images, and obtains depth value estimation through a single image.

Claims

1. A monocular absolute depth estimation method based on the BEiT relative depth mapping network, characterized by: The following steps are included: Step 1: Obtain a training sample set B and a test sample set C; where B and C are sets consisting of RGB images containing depth information; Step 2: Construct a relative depth mapping network model S based on BEiT r : Used to predict the relative depth of the RGB input image; the input image uses a single RGB image; Step 3: Based on the training sample set B, the relative depth mapping model S based on BEiT r Perform iterative training and conduct effect testing based on the test sample set C; Step 4: Use the trained model to obtain the depth estimation result of the image.

2. The monocular absolute depth estimation method based on the BEiT relative depth mapping network according to claim 1, characterized in that The step 1 is specifically as follows: The NYU-DepthV2 dataset is used as the reference image, a part of the reference images is used as the training set B, and the remaining reference images are used as the test set C.

3. The monocular absolute depth estimation method based on the BEiT relative depth mapping network according to claim 1, characterized in that The step 2 is specifically as follows: Step 2.1: Construct a standard BEiT network as the first-stage network. The standard BEiT network consists of a linear mapping module and an encoder. The linear mapping module is used to process 2D image patches, and the encoder consists of an attention module and an MLP stack. The first-stage network predicts the relative depth of the RGB input image and outputs a feature vector to the second-stage network; Step 2.2, constructing a relative depth mapping module P as a second-stage network; the second-stage network predicts the absolute depth of the input image based on the relative depth and the feature vector; Step 2.3: Build a depth prediction network Q, integrate the output of the relative depth prediction network P, and regress the relative depth estimated by P to the final absolute image depth estimate; The implementation process is as follows: q=MLP(y) Among them, q represents the final depth estimation value, MLP represents the multi-layer perceptron, and y represents the output relative image depth estimation value.

4. The monocular absolute depth estimation method based on the BEiT relative depth mapping network according to claim 3 is characterized in that The step 2.1 is specifically as follows: The linear mapping module first divides the two-dimensional RGB input image into a series of image blocks, so that the standard Transformer can directly accept image data; Formally, the RGB input image Reshape to N = HW / P 2 image blocks, Where C is the number of channels, (H, W) is the resolution of the input image, and (P, P) is the resolution of each image block; Image Block After being flattened into vectors and projected through a linear mapping, the image patch retains the original pixels and is used as the input of BEiT; The encoder composed of the attention module and MLP represents the RGB input image as a discrete tag sequence obtained by the "image encoder". Tokenized The vocabulary Contains discrete tag indices so that the subsequent attention mechanism module accepts them as input and extracts feature information to output feature vectors.

5. The monocular absolute depth estimation method based on the BEiT relative depth mapping network according to claim 4 is characterized in that: Using an image tagger learned through a discrete variational autoencoder, the standard Transformer architecture accepts images as input. There are two modules in the visual tagging learning process, namely the encoder and the decoder. The encoder and decoder are composed of an attention module and an MLP stack to extract feature information and output a feature vector. Encoder q φ (z|x) maps the input image x to discrete labels according to the visual codebook Decoder p ψ (x|z) learning based on discrete labels Reconstruct the input image x; Reconstruct the target and write the reconstructed target as Gumbel-softmax relaxation is used to train the model parameters. During the training of the discrete variational autoencoder, q φ Imposing a uniform prior; Tokenize each input image x to be predicted into a grid of visual markers, with the number of visual markers being the same as the number of image patches per image; According to the construction method of BEiT network, the standard Transformer is used as the first-order backbone network of the model; The input of the Transformer is a series of image blocks The image block is linearly projected to obtain the image block embedding in Add a special token [S] before the input sequence to embed the standard learnable one-dimensional position Added to the image patch embedding, the input vector is input into the Transformer; The encoder consists of L layers of Transformer modules H l =Transformer(H l-1 ), where l=1,…,L. The output vector of the last layer is used as the encoding representation of the image block, where is the vector of the i-th image block.

6. The monocular absolute depth estimation method based on the BEiT relative depth mapping network according to claim 3, characterized in that: The step 2.2 is specifically as follows: The second-stage network consists of two modules: a feature projection module and a depth prediction module. The feature projection module projects the feature vectors of different dimensions output by the first-stage model while maintaining the same dimension. A 1*1 convolutional layer is used to implement the feature projection module. The depth prediction module performs multi-scale refinement of boxes by receiving a feature vector and generating a set of depth prediction vectors.

7. The monocular absolute depth estimation method based on the BEiT relative depth mapping network according to claim 6, characterized in that: Through the depth prediction vector, we move left and right in the depth interval to achieve multi-scale refinement, and use multi-scale features to achieve multi-scale refinement in the first l th The decoder layer, a multi-layer perceptron takes the relative depth features output by the relative depth map module P as input and predicts n for that pixel location. a Attraction points k :k=1,…,n a }, the adjusted predicted value is c′ i =c i +Δc i , the adjustment amount is given by the following formula: Among them, the hyperparameters α and γ determine the attraction strength. This adjustment method is called the fractional adjustment method; it is used for indoor data; For outdoor data, a variant of the exponential adjustment method is used, with the following formula:

8. The monocular absolute depth estimation method based on the BEiT relative depth mapping network according to claim 1, characterized in that: The step 3 is specifically as follows: Step 3.1, initialize the number of iterations to t, the maximum number of iterations to T, and the current relative depth mapping model based on BEiT is S t , and let t = 1, S t =S; Step 3.2: Randomly select b training samples from the training sample set B without replacement as the relative depth mapping model S t The standard BEiT network extracts global and local features of the image and fuses them as the number of network layers increases. It transfers shallow feature information of the image, including color, texture, and geometric shape, as well as feature information of the current layer layer by layer. The depth value regression network regresses the fused features to obtain the depth prediction map corresponding to each training sample. The depth prediction map is compared with the true value of the GroundTruth sample, and then the backpropagation training process is carried out. Step 3.3: First, calculate the mean square error between the quality prediction score corresponding to each training sample and the quality score label corresponding to the training sample as the model S t The loss value of the network model S is then optimized using the adaptive moment estimation algorithm. t Update the weight parameters of each layer; The formula for calculating the scale-translation invariant loss function between the depth estimate corresponding to each training sample and the actual value corresponding to the training sample is: Where T represents the total number of images in each batch of training data, y j and They represent the depth estimation value and actual depth value predicted by the model corresponding to the j-th image in the batch respectively; Adaptive moment estimation is used as the optimization algorithm, and the learning rate is adjusted every s generations during the training process to speed up the convergence speed and improve the learning accuracy. The adjustment formula is as follows: Where t represents the current iteration number, s represents the learning rate update step size, d represents the learning rate attenuation coefficient, lr t Indicates the learning rate when the model iterates to t times; Step 3.4: Determine whether the training sample set B has been traversed. If so, execute step 3.5; otherwise, execute steps 3.2 and 3.

3. Step 3.5, determine whether t=T is true. If so, obtain the trained BEiT-based relative depth mapping network model S * =S t , otherwise, let t = t + 1 and execute steps 3.3 and 3.

4.

9. The monocular absolute depth estimation method based on the BEiT relative depth mapping network according to claim 1, characterized in that: The step 4 is specifically as follows: The test sample set C is used as the relative depth mapping model S of the trained base BEiT t The input is used for forward reasoning to obtain the quality prediction score of each test sample to verify the model effect.

Images