Monocular Depth Estimation Method, Device and Medium Based on Bidirectional State Space Model

Abstract

A monocular depth estimation method, device and medium based on a bidirectional state-space model, relating to an image processing technology, solves the problem that current self-supervised methods are insufficient in long sequence modeling, training speed, inference speed and capturing complex details of scenes, and the optimization effect of the model under sparse supervision is poor. The monocular depth estimation method of the present invention comprises the following steps: constructing an encoder-decoder structure, including a continuous dilated convolution module and a bidirectional state-space module, the encoder generates intermediate features of the training image, models long-distance intermediate feature information, and the decoder generates output features; constructing a posture estimation model, and jointly reconstructing the output features generated by the decoder with the depth estimation of the training image, modeling the learning target, and designing the loss function of the target image and the reconstructed image to optimize the network; using the public KITTI dataset to train the constructed model to obtain a weight model, and evaluating the accuracy of the trained model for monocular image depth estimation. The present invention is used for monocular depth estimation of images.

Description

Technical Field

[0001] The present invention relates to an image processing technology, in particular to a monocular depth estimation method, device and medium based on a bidirectional state space model. Background Art

[0002] Monocular depth estimation currently aims to infer the distance or depth of objects in a scene using videos or images captured by a single camera. This technology is crucial for fields such as robotics and autonomous driving. Despite the rapid progress of deep learning technology in computer vision in recent years, depth estimation technology based on encoder-decoder architectures still faces many challenges.

[0003] Traditional visual depth estimation methods typically rely on binocular or RGBD cameras to acquire depth information. However, with the development of deep learning technology, monocular depth estimation algorithms based on deep learning models have gained widespread application. These methods typically employ supervised learning, training and optimizing the models using large amounts of depth-labeled data. However, obtaining such large amounts of depth-labeled data presents significant challenges in practical applications, leading to limitations in the generalization capabilities of these methods.

[0004] To address this issue, self-supervised learning methods have gradually become a research hotspot. Through an "encoder-decoder" structure, these methods can use their own data to design optimization objectives without the need for real-world deep labels, breaking through the limitations of traditional supervised learning methods.

[0005] For example, document number CN115731280A discloses a self-supervised monocular depth estimation method based on Swin-Transformer and CNN parallel networks. The invention uses Swin-Transformer and CNN for feature extraction at the same time, and fuses the extracted features, which can enable the network to strike a balance between establishing long-range correlation and retaining spatial structure information, enhance the network's ability to learn features, and combine the scale-by-scale self-distillation loss proposed in the present invention to perform self-supervised training of the network, thereby improving the accuracy of self-supervised monocular depth estimation.

[0006] The prior art document CN116883476A discloses a monocular depth estimation method based on attention feature fusion and multi-level correction (CN202310783026.7). The method includes obtaining and preprocessing a dataset of an image and a corresponding real depth image; pre-training the constructed ST encoding network structure on an image classification dataset; segmenting the image and performing feature extraction; inputting the encoded features into a monocular depth estimation network model, and using an attention feature fusion module to progressively fuse adjacent resolution features; processing the decoded features using a correction network to correct the decoded features; summing the results using a loss function; continuously adjusting the parameters of the monocular depth estimation network model through an optimizer; and obtaining the depth value of each pixel. This method achieves accurate prediction of local detail information in the depth image.

[0007] However, current self-supervised methods have shortcomings in long sequence modeling, training speed, inference speed, and capturing complex details of scenes, and the optimization effect of the model under sparse supervision also needs to be improved. Summary of the Invention

[0008] The technical problem to be solved by the present invention is: to solve the problem that the current self-supervisory method has deficiencies in long sequence modeling, training speed, inference speed and capturing complex details of the scene, and the optimization effect of the model under sparse supervision is poor.

[0009] The present invention is implemented by the following technical solution: providing a monocular depth estimation method based on a bidirectional state space model, comprising the following steps:

[0010] Step S1: Construct an encoder-decoder structure, including a continuously dilated convolution module and a bidirectional state space module. The encoder generates intermediate features of the training image, models long-distance intermediate feature information, and the decoder generates output features.

[0011] Step S2: Build a pose estimation model and use the output features generated by the decoder and the depth estimation of the training image to jointly reconstruct the learning target, model the learning target, and design the loss function of the target image and the reconstructed image to optimize the network;

[0012] Step S3: Use the public KITTI dataset to train the constructed model to obtain a weighted model, and evaluate the accuracy of the trained model for monocular image depth estimation.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] 1. The monocular depth estimation method based on the bidirectional state-space model can effectively model long sequences of images. At the same time, this method does not incur the secondary computational complexity associated with the attention mechanism, thereby enhancing the model's ability to estimate depth from monocular images and extract fine-grained local features more precisely. It also reduces the difficulty of model training and improves the model's inference speed, thus effectively solving the problem that leads to poor performance of monocular depth estimation in images. Compared with existing technologies, it improves the accuracy of monocular depth estimation in images.

[0015] 2. The method of the present invention can effectively model long sequences of images, with fast training and inference speeds, accurately capturing complex scene details, and significantly improved optimization effects under sparse supervision. This method solves the problems of slow training speed, insufficient performance, and difficulty in capturing complex scene details in monocular depth estimation models, improves the model's generalization capabilities, and effectively reduces the difficulty of industrial deployment of monocular depth estimation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the present invention;

[0017] Figure 2 Network structure diagram of monocular depth estimation method based on bidirectional state space model;

[0018] Figure 3 Schematic diagram of the continuous dilation convolution module;

[0019] Figure 4 Schematic diagram of the bidirectional state space module;

[0020] Figure 5 This is a diagram of the visualization results on the KITTI dataset. DETAILED DESCRIPTION

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0022] See attached Figure 1 This embodiment provides a monocular depth estimation method based on a bidirectional state space model, comprising the following steps:

[0023] Step S1: Construct an encoder-decoder structure, including a continuously dilated convolution module and a bidirectional state space module. The encoder generates intermediate features of the training image, models long-distance intermediate feature information, and the decoder generates output features.

[0024] The step S1 constructs an encoder-decoder structure, wherein the encoder generates intermediate features of the training image, models the long-distance intermediate feature information, and the decoder generates output features. The steps include:

[0025] An encoder backbone is constructed, comprising a convolutional module (Conv Stem) and an intermediate feature extraction layer, wherein the intermediate extraction layer comprises a continuously dilated convolutional module and a bidirectional state space module. After a training image is processed by the convolutional module (Conv Stem), a feature matrix that can be processed by the bidirectional state space module is generated by the continuously dilated convolutional module. The output feature matrix is input to the bidirectional state space module for processing to model long-distance intermediate feature information. Based on the feature matrix output by the encoder, depth information of the training image is generated by decoding through a three-layer decoder.

[0026] See attached Figure 2 In this embodiment, the size and channel information of the training image are processed to obtain features of the shape [B, C, H, W]. These features are then passed through a convolution module and an average pooling module. After the convolution layer, the training image passes through an average pooling layer that averages the image area. This reduces the dimensionality of the feature vector output by the convolution layer, reduces information redundancy, speeds up model training, and improves the robustness of the extracted features. Where B represents the batch size, C represents the number of channels, H represents the height, and W represents the width.

[0027] Input training image [B, C, H in ,W in ], first, the output features of the convolution module and the average pooling module enter the intermediate feature extraction layer, which is composed of a continuous expansion convolution module and a bidirectional state space module.

[0028] The shape is [B,C,H in ,W in ] The training image input average pooling module outputs [B, C, H out ,W out ], the average pooling module can be expressed as:

[0029]

[0030] The stride represents the window movement step, which is the same as the kernel size Keep consistent and round down to get [B,C,H out ,W out ]output.

[0031] For details, see the attached Figure 3 , extract the image after the convolution module (Conv Stem) processing through depth-wise separable dilated convolution to generate a feature matrix with multi-scale local information. The steps include:

[0032] The image processed by the convolution module (Conv Stem) is extracted through depth-wise separable dilated convolution to generate a feature matrix with multi-scale local information; the batch normalization layer (BatchNorm) vertically normalizes the multi-scale local features to obtain normalized features; the normalized features are passed through a one-dimensional convolution kernel to generate a new feature matrix (FeatureMap); the new feature matrix is activated by a RELU activation function; the features activated by the RELU activation function are passed through a one-dimensional convolution kernel to generate a feature matrix that can be processed by a bidirectional state space module.

[0033] See attached Figure 4 The output feature matrix is input to the bidirectional state space module for processing, and the steps of modeling the long-distance intermediate feature information include:

[0034] Accept the continuous expansion convolution module to generate the dimension [B, C, H in ,W in ] feature matrix, where B is the batch size, C is the number of channels, H is the height of the feature map, and W is the width of the feature map. The feature map is processed by the convolution module to obtain the feature matrix output by the convolution layer; the feature matrix output by the convolution layer is respectively generated by the fully connected layer 1 and the fully connected layer 2 to generate feature branch 1 (x1) and feature branch 2 (x2), and the ReLU6 activation function is applied to branch x1 and multiplied by x2 to obtain a shape of [B, C, H in ,W in ]’s fusion feature matrix;

[0035] The fused feature matrix is further processed by deep convolution (without batch normalization) to obtain a new feature with a shape of [B, C, H, W]. The new feature map with a shape of [B, C, H, W] is transformed in dimension, and the matrix shape is changed from [B, C, H, W] to [B, H*W, C], obtaining a feature matrix with a shape of [B, H*W, C]. The feature matrix with a shape of [B, H*W, C] is input into the projection layer of the bidirectional state space module, and the received input features are linearly projected. Its feature dimension is expanded to twice the original dimension for subsequent feature processing. The features output by the projection layer are received and convolution operation is performed with a one-dimensional convolution kernel to capture local spatiotemporal dependencies.

[0036] The feature sequence after one-dimensional convolution is activated by the SiLU activation function; the activated feature sequence is projected into the state space model, including the time step (dt), the state matrix B and the state matrix C, which are used to update the various parameters in the state space, including the bidirectional state space matrix, the convolution block and the fully connected layer, which are used to process the input features from both the forward and reverse directions, model the long-distance intermediate feature information, and obtain the bidirectional state space model output; the input features are directly added to the bidirectional state space model output to form a residual connection to ensure stable gradient propagation; the output features of the bidirectional state space model are reprojected to the input dimension, and the output feature matrix shape is [B, C, H, W].

[0037] The batch normalization layer (BatchNorm) performs vertical normalization on the multi-scale local features to obtain normalized features. The specific steps are as follows:

[0038] a) Save the input X for residual connection operation;

[0039] b)DDWConv r (·) is the dilated convolutional layer, which can expand the receptive field without increasing the number of parameters. The output dimension remains unchanged, still [B, C, H, W];

[0040] c) The batch normalization layer (BatchNorm) performs vertical normalization on the multi-scale local features to obtain normalized features, which can be expressed as:

[0041]

[0042] in, is the sample mean of the mini-batch sample B, is the sample standard deviation of sample B. After applying normalization, the generated mini-batch has a mean of 0 and a unit variance of 1. In addition, the scaling parameter γ and the offset parameter β are defined, and the shape is the same as the input parameter x;

[0043] d)Linear G1 For point-by-point convolution layer 1, the feature dimension of each pixel position is expanded from C to 6C, which can improve the feature representation capability;

[0044] e) ReLU is the activation function, which can be expressed as ReLU(x)=max(0,x);

[0045] f)Linear G2 For the point-by-point convolution layer 2, the feature dimension of each pixel is restored from 6C to C, restoring the original number of channels;

[0046] g) Perform residual connection, add the input X to the regularized output X to form a skip connection, and finally get the output of the continuous expansion convolution module The shape is [B,C,H,W] and [B,C,H in ,W in ]Stay consistent.

[0047] High-dimensional and nonlinear feature transformations are crucial in deep learning. This necessity stems from the complexity of real-world monocular image depth data and the model's ability to represent such data. Typically, low-dimensional features are first mapped to a high-dimensional space through linear projection, and then nonlinear activation functions are used to introduce nonlinear structures to better represent the complexity of the data.

[0048] For example, the polynomial kernel function can be expressed as k(x1,x2)=(γx1x2+c) d The input features can be Projection to (n+1) d In the dimensional space, the Gaussian kernel function can transform k(x1,x2) into Projected into infinite-dimensional feature space.

[0049] This work employs a novel operation that can obtain high-dimensional and nonlinear space from low-dimensional input features. This operation has the unique ability to implicitly consider extremely high-dimensional features while performing calculations in low-dimensional space.

[0050] This significant advantage distinguishes star operation from other technologies in the field of efficient networks and makes it suitable for efficient network design. The specific process is as follows:

[0051] In a single-layer neural network, the new operation is represented by The features of two linear transformations are fused by element-wise multiplication, And the new operation can be simplified to

[0052] definition represents a set of C+1-dimensional real vectors, where C is the number of channels of the input feature. The star operation can be expressed as the following equation:

[0053]

[0054] in:

[0055]

[0056] Extended to a 3-layer network structure, as follows:

[0057]

[0058] This method can effectively and reasonably amplify the implicit dimension of the input features, allowing the network to better learn effective features. At the same time, it only uses a three-layer network, greatly saving the resources consumed by model training. At the same time, it solves the problem of poor depth estimation of monocular images and the poor performance of the model's depth estimation.

[0059] Accept the features generated by the continuous expansion convolution module with dimensions [B, C, H, W] The features are processed through the convolution module to obtain the feature matrix output by the convolution layer;

[0060] The feature matrix output by the convolutional layer is passed through the fully connected layer 1 and the fully connected layer 2 to generate two feature branches x1 and x2 respectively. The ReLU6 activation function is applied to the branch x1 and multiplied with x2 to obtain a fused feature matrix with the shape of [B, C, H, W].

[0061] The fused feature matrix is further processed by deep convolution (without batch normalization) to obtain a new feature with the shape of [B, C, H, W]

[0062] Will The dimension is resized to [B, C, H*W], that is, the height and width are flattened and merged into one dimension H*W, and then the feature tensor dimensions are rearranged, [B, C, H*W] is rearranged to [B, H*W, C], and the new The shape is [B,H*W,C], which is convenient for passing to the selected state space model for processing.

[0063] After adjusting the dimensions Projecting to the fully connected layer to obtain features B, U and △ can be expressed as:

[0064]

[0065] Obtain the state space model parameter matrix based on the characteristics B, C and △ It can be expressed as:

[0066]

[0067] For the state space model parameter matrix And feature U performs state space modeling operation, output That is, the input feature The output features after long sequence modeling by the state space model (SSM) can be expressed as:

[0068]

[0069] These vectors are processed in both forward and backward directions to simulate the behavior of a bidirectional neural network layer. In each direction, a one-dimensional convolution module and a Sigmoid Linear Unit (SiLU) activation function are applied to capture the contextual information of the image features.

[0070] When processing image block sequences, the state-space sequence model significantly improves its ability to capture the global contextual information of the image through a bidirectional information transfer mechanism. Specifically, this model utilizes information flow in both forward and backward directions to comprehensively integrate the information in the image.

[0071] In the bidirectional information transmission mechanism, in the forward process, the model starts from the starting position of the image sequence, gradually processes each feature image, and gradually transmits and integrates information. This processing method allows the model to form a comprehensive understanding of the current image block based on the content of the current feature image and the previously processed information; in the backward information transmission process, the model starts from the end of the sequence and processes the image block forward. This process can integrate the information from the end of the sequence to the current position, allowing the model to use the upcoming image context information to further enrich the representation of the current image block. This global perspective enables the model to capture the complex relationships in the image, including the relative positions of objects and background information.

[0072] The bidirectional processing method enables the model to fuse information from different directions of the training image, enhances the ability to model the dependencies between data, realizes long sequence modeling, and effectively improves the accuracy of the model in monocular depth estimation. At the same time, the model can also show greater stability when processing complex visual scenes.

[0073] The activated feature sequence is projected into the state space model, including the time step (dt), state matrix B and state matrix U, which are used to update various parameters in the state space. This includes a bidirectional state space matrix, convolution blocks and fully connected layers, which are used to process input features from both the forward and reverse directions, model long-distance intermediate feature information, and obtain the output of the bidirectional state space model.

[0074] The input features of the bidirectional state-space module and the output features after passing through the fully connected layer in the module are directly added to the output of the bidirectional state-space model to form a residual connection to ensure stable gradient propagation.

[0075] Step S2: Build a pose estimation model and use the output features generated by the decoder and the depth estimation of the training image to jointly reconstruct the learning target, model the learning target, and design the loss function of the target image and the reconstructed image to optimize the network;

[0076] The steps of constructing a pose estimation model in step S2, jointly reconstructing the depth estimation of the training image using the output features generated by the decoder, modeling the learning target, and designing a loss function for the target image and the reconstructed image to optimize the network include:

[0077] A pose estimation model (PoseNet) is constructed. The model inputs two adjacent frames of images from the training dataset and outputs the relative pose between the two frames. The network extracts high-dimensional features from the input RGB images; the high-dimensional features are generated into feature maps through the feature extraction layer, and the feature maps are processed through convolution operations to extract information about the camera motion in the image; the model ultimately outputs a pose vector containing six-dimensional data, which consists of a 3D rotation vector and a 3D translation vector, used to represent the movement of the camera between the two frames; the deep features generated by the training images after the encoder-decoder are fused with the features generated by the pose estimation model to construct a loss function optimization network model, where I1 can be expressed as:

[0078] I1=F(I S ,P,D P ,K)

[0079] The minimum photometric error loss function in the image reconstruction loss in this method is:

[0080] L s (I1,I2)=L s (F(I S ,P,D P ,K),I2)

[0081] Where I1 is the reconstructed target image, I2 is the target image, P is the pose estimation, and D P is the predicted depth of the target image, and K is the internal parameter of the camera.

[0082] PoseNet combines the internal parameters of the camera (also called the intrinsic parameter matrix). Generally, the form of the intrinsic parameter matrix is as follows:

[0083]

[0084] Where: f x and f y is the focal length of the camera in the x and y directions (in pixels), c x and c y These parameters can be obtained through camera calibration, especially when calculating 3D pose estimation, object tracking or other applications that require accurate spatial position information.

[0085] L s(I1, I2) is calculated by weighted summation using structural similarity and the L1 norm between I1 and I2, and can be expressed as:

[0086]

[0087] The total loss of the model can be expressed as:

[0088]

[0089] Among them L smooth It can be expressed as:

[0090]

[0091] in, represents the normalized inverse depth map, and They represent the steps of finding partial derivatives of the width and height of the image, e is the natural exponent, and I2 is the target image.

[0092] See attached Figure 5 ,Step S3: Use the public KITTI dataset to train the constructed model to obtain a weight model, and evaluate the accuracy of the trained model for monocular image depth estimation.

[0093] The model training uses the public KITTI dataset to train the constructed model to obtain a weighted model. The training dataset consists of 4424 images and the test dataset consists of 697 images.

[0094] The present invention trains the model on NVIDIA A100, sets the batch size (Batch Size) of training learning to 32, the training cycle (Epoch) to 60, and the initialization learning rate to 0.00005.

[0095] Because the monocular depth estimation method based on the bidirectional state-space model can effectively model long sequences of images and does not incur the secondary computational complexity associated with the attention mechanism, it enhances the model's ability to estimate depth from monocular images and more precisely extracts fine-grained local features. It also reduces the difficulty of model training and increases the model's inference speed, effectively solving the problem that leads to poor performance in monocular depth estimation. Compared with existing technologies, it improves the accuracy of monocular depth estimation from images.

[0096] An embodiment of the present application discloses a monocular depth estimation device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the monocular depth estimation method based on the bidirectional state-space model as described above.

[0097] An embodiment of the present application discloses a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the image classification method based on the deep learning model as described above are implemented.

[0098] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the invention. It should be understood that many modifications may be made to the illustrative embodiments, and that other arrangements may be devised, without departing from the spirit and scope of the invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in ways other than those described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in conjunction with other described embodiments.

Claims

1. A monocular depth estimation method based on a bidirectional state-space model, characterized by: The steps include: Step S1: Construct an encoder-decoder structure, including a continuously dilated convolution module and a bidirectional state space module. The encoder generates intermediate features of the training image, models long-distance intermediate feature information, and the decoder generates output features. Step S2: Build a pose estimation model and use the output features generated by the decoder and the depth estimation of the training image to jointly reconstruct the learning target, model the learning target, and design the loss function of the target image and the reconstructed image to optimize the network; Step S3: Use the public KITTI dataset to train the constructed model to obtain a weighted model, and evaluate the accuracy of the trained model for monocular image depth estimation; The step S1 constructs an encoder-decoder structure, wherein the encoder generates intermediate features of the training image, models the long-distance intermediate feature information, and the decoder generates output features. The steps include: Constructing an encoder backbone, the encoder backbone comprising a convolution module (Conv Stem) and an intermediate feature extraction layer, the intermediate feature extraction layer comprising a continuously dilated convolution module and a bidirectional state space module; after a training image is processed by the convolution module (Conv Stem), the continuously dilated convolution module generates a feature matrix that can be processed by the bidirectional state space module; the output feature matrix is input into the bidirectional state space module for processing to model long-range intermediate feature information; and based on the feature matrix output by the encoder, depth information of the training image is generated by decoding through a three-layer decoder; The step of generating a feature matrix that can be processed by the bidirectional state space module by the continuous dilation convolution module after the training image is processed by the convolution module Conv Stem includes: The image processed by the convolution module Conv Stem is extracted through depth-wise separable dilated convolution to generate a feature matrix with multi-scale local information; the batch normalization layer BatchNorm vertically normalizes the feature matrix of the multi-scale local information to obtain normalized features; the normalized features are passed through a one-dimensional convolution kernel to generate a new feature matrix FeatureMap; the new feature matrix is activated by a RELU activation function; the features activated by the RELU activation function are passed through a one-dimensional convolution kernel to generate a feature matrix that can be processed by a bidirectional state space module.

2. The monocular depth estimation method based on a bidirectional state-space model according to claim 1, characterized in that: The output feature matrix is input into the bidirectional state space module for processing, and the step of modeling the long-distance intermediate feature information includes: the bidirectional state space module receives the feature matrix of the dimension [B, C, H, W] generated by the continuous expansion convolution module, where B is the batch size, C is the number of channels, H is the height of the feature map, and W is the width of the feature map; the feature map is processed by the convolution module to obtain the feature matrix output by the convolution layer; the feature matrix output by the convolution layer is respectively passed through the fully connected layer 1 and the fully connected layer 2 to generate two feature branches x1 and x2 respectively, the ReLU6 activation function is applied to the branch x1, and the branch is multiplied by x2 to obtain a fusion feature matrix with a shape of [B, C, H, W]; The obtained fused feature matrix is further processed by deep convolution without batch normalization to obtain a new feature with a shape of [B, C, H, W]. The new feature map with a shape of [B, C, H, W] is transformed in dimension, and the matrix shape is changed from [B, C, H, W] to [B, H*W, C], obtaining a feature matrix with a shape of [B, H*W, C]. The feature matrix with a shape of [B, H*W, C] is input into the projection layer of the bidirectional state space module, and the received input features are linearly projected. Its feature dimension is expanded to twice the original dimension for subsequent feature processing. The features output by the projection layer are received and convolution operation is performed with a one-dimensional convolution kernel to capture local spatiotemporal dependencies. The feature sequence after one-dimensional convolution is activated by the SiLU activation function; the activated feature sequence is projected into the state space model, including the time step dt, the state matrix B and the state matrix C, which are used to update the various parameters in the state space, including the bidirectional state space matrix, the convolution block and the fully connected layer, which are used to process the input features from both the forward and reverse directions, model the long-distance intermediate feature information, and obtain the bidirectional state space model output; the input features are directly added to the bidirectional state space model output to form a residual connection to ensure stable gradient propagation; the output features of the bidirectional state space model are reprojected to the input dimension, and the output feature matrix shape is [B, C, H, W].

3. The monocular depth estimation method based on a bidirectional state-space model according to claim 2, wherein: The batch normalization layer BatchNorm performs vertical normalization on the multi-scale local features to obtain normalized features. The specific steps are as follows: a) Save the input X for residual connection operation; b)DDWConv r (·) is the dilated convolutional layer, which can expand the receptive field without increasing the number of parameters. The output dimension remains unchanged, still [B, C, H, W]; c) The batch normalization layer BatchNorm performs vertical normalization on the multi-scale local features to obtain normalized features, which are expressed as: in, is the sample mean of the mini-batch sample B, is the sample standard deviation of sample B. After applying normalization, the generated mini-batch has a mean of 0 and a unit variance of 1. The scaling parameter γ and the offset parameter β are defined, and the shape is the same as the input parameter X; d)Linear G1 For point-by-point convolution layer 1, the feature dimension of each pixel position is expanded from C to 6C, which can improve the feature representation capability; e) ReLU is the activation function, expressed as ReLU(x)=max(0,x); f)Linear G2 For the point-by-point convolution layer 2, the feature dimension of each pixel is restored from 6C to C, restoring the original number of channels; g) Perform residual connection, add the input X to the output after regularization to form a skip connection, and finally get the output of the continuous expansion convolution module The shape is [B,C,H,W].

4. The monocular depth estimation method based on a bidirectional state-space model according to claim 3, wherein: The steps of constructing a pose estimation model in step S2, jointly reconstructing the depth estimation of the training image using the output features generated by the decoder, modeling the learning target, and designing a loss function for the target image and the reconstructed image to optimize the network include: A pose estimation model, PoseNet, is constructed. The model inputs two adjacent frames of images from the training dataset and outputs the relative pose between the two frames. The network extracts high-dimensional features from the input RGB images. The high-dimensional features are used to generate feature maps through the feature extraction layer. The feature maps are processed through convolution operations to extract information about the camera motion in the image. The model ultimately outputs a pose vector containing six-dimensional data. This vector consists of a 3D rotation vector and a 3D translation vector, which is used to represent the movement of the camera between the two frames. The deep features generated by the training images after the encoder-decoder are fused with the features generated by the pose estimation model to construct a loss function optimization network model.

5. The monocular depth estimation method based on a bidirectional state-space model according to claim 4, characterized in that: The step of fusing the depth features generated by the training image through the encoder-decoder with the features generated by the posture estimation model to construct a loss function optimization network model includes: I1=F(I S ,P,D P ,K) The minimum photometric error loss function in image reconstruction loss is: L s (I1,I2)=L s (F(I S ,P,D P ,K),I2) Where I1 is the reconstructed target image which can be calculated by the F function, I2 is the target image, I S represents the original image of the training input, P is the pose estimation, D P is the predicted depth of the target image, K is the internal parameter of the camera, and α is a hyperparameter set to 0.85; L s (I1, I2) is calculated by weighted summation using structural similarity and the L1 norm between I1 and I2, expressed as: The total loss of the model is expressed as: Among them L smooth Expressed as: in, represents the normalized inverse depth map, and They represent the steps of finding partial derivatives of the width and height of the image, e is the natural exponent, and I2 is the target image.

6. The monocular depth estimation method based on a bidirectional state-space model according to claim 5, characterized in that: The high-dimensional features are obtained as follows: In a single-layer neural network, the new operation is expressed as The features of two linear transformations are fused by element-wise multiplication, And the new operation can be simplified to definition C is the number of channels of the input feature, the superscript T indicates the transposition of the matrix, i and j are used to index the corresponding channel number, and the star operation is expressed as the following equation: in: Extended to a 3-layer network structure, as follows: Accept the features generated by the continuous expansion convolution module with dimensions [B, C, H, W] The features are processed through the convolution module to obtain the feature matrix output by the convolution layer; The feature matrix output by the convolutional layer is passed through the fully connected layer 1 and the fully connected layer 2 to generate two feature branches x1 and x2 respectively. The ReLU6 activation function is applied to branch x1 and multiplied with x2 to obtain a fused feature matrix with a shape of [B, C, H, W]. The fused feature matrix is further processed by deep convolution without batch normalization to obtain a new feature with the shape of [B, C, H, W]. Will The dimensions are resized to [B, C, H*W], the height and width are flattened and merged into one dimension H*W, and then the feature tensor dimensions are rearranged, [B, C, H*W] is rearranged to [B, H*W, C], and the new The shape is [B, H*W, C], which is convenient for passing to the selected state space model for processing; After adjusting the dimensions Projecting to the fully connected layer to obtain features B, C and △, expressed as: Obtain the state space model parameter matrix based on the characteristics B, C and △ Expressed as: For the state space model parameter matrix And feature U performs state space modeling operation, output That is, the input feature The output features after long sequence modeling by the state space model SSM, The calculation process is expressed as:

7. A monocular depth estimation device, characterized by: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the monocular depth estimation method based on the bidirectional state space model according to any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the monocular depth estimation method based on the bidirectional state space model described in any one of claims 1 to 6 are implemented.

Images