A self-supervised monocular depth and pose estimation method

By constructing a lightweight, self-supervised monocular depth and pose estimation network and utilizing 2D convolutional modules and residual modules, the computational and storage requirements of the network are reduced, enabling real-time depth and pose estimation on edge devices.

CN114742871BActive Publication Date: 2024-11-22KUNSHAN IND TECH INST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210358881.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-04-07
Publication Date
2024-11-22
Estimated Expiration
2042-04-07

AI Technical Summary

Technical Problem

Existing monocular depth estimation networks have too many computational parameters, making them unable to run in real time on end devices.

Method used

A self-supervised monocular depth pose estimation method is adopted, and a lightweight network model is constructed through 2D convolutional modules, basic residual modules, Block_E modules and Block_M modules to reduce the network computation and model storage during feature extraction and feature decoding.

Benefits of technology

Real-time depth and pose estimation is achieved on edge devices with limited computing and storage resources, solving the problem that the algorithm cannot run in real time due to excessive computing and storage overhead.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114742871B_ABST
    Figure CN114742871B_ABST
Patent Text Reader

Abstract

The present invention provides a self-supervised monocular depth and pose estimation method, which extracts features by inputting the acquired original feature image into a pre-established self-supervised monocular depth estimation network model fD and a self-supervised monocular pose estimation network model fP, and outputs a feature map with depth and pose estimation results. The self-supervised monocular depth estimation network model fD and the self-supervised monocular pose estimation network model fP both include a 2D convolution module, a basic residual module, a Block_E module, and a Block_M module. Depth estimation and pose estimation are performed on the input image, and only the main features are used in the jump connection of the network model, which reduces the network calculation amount and model storage amount during feature extraction and feature decoding. At the same time, the model can be inferred and updated simultaneously in the terminal device with limited computing and storage resources, solving the problem that the algorithm calculation and storage overhead are too large to run in real time on the terminal side.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision depth estimation, and in particular relates to a self-supervised monocular depth-of-field pose estimation method. Background Art

[0002] Monocular depth estimation has great development potential in edge scenarios such as AR, autonomous driving, and smartphones due to its low application cost (compared to other sensors, monocular cameras are low in price and energy consumption). However, thanks to the development of neural networks, the existing monocular depth estimation has achieved good accuracy indicators. However, when performing depth estimation and pose estimation on various images, the existing network parameters require too much calculation and the platform computing power is too high, so it cannot run in real time on the end device. Summary of the invention

[0003] To address the defects in the prior art, the present invention discloses a self-supervised monocular depth and pose estimation method, which reduces the network calculation amount and model storage amount in the feature extraction and feature decoding process, and can simultaneously infer and update the model in terminal devices with limited computing and storage resources, solving the problem that the algorithm calculation and storage overhead are too large to run in real time on the terminal side.

[0004] A self-supervised monocular depth pose estimation method, comprising:

[0005] Get the original feature image;

[0006] The original feature image is input into the pre-established self-supervised monocular depth estimation network model fD and the self-supervised monocular pose estimation network model fP for feature extraction, and a feature image with depth and pose is output;

[0007] The self-supervised monocular depth estimation network model fD and the self-supervised monocular pose estimation network model fP both include a 2D convolution module, a basic residual module, a Block_E module and a Block_M module.

[0008] Furthermore, the 2D convolution module includes a 2D convolution layer, a normalization layer and an activation function layer connected in sequence.

[0009] Further, the Block_E module and the Block_M module both include a Block0 module;

[0010] The construction method of the Block0 module includes:

[0011] The input feature map with dimension C0*H0*W0 is passed through the 2D convolution module to obtain a feature map with dimension (C0 / 2)*H0*W0, which is recorded as feature map F;

[0012] The feature map F is sequentially passed through the depth-wise convolution layer and the normalization layer to obtain the feature map F1 of dimension (C0 / 4)*H0*W0; the feature map F is sequentially passed through the point-wise convolution layer and the normalization layer to obtain the feature map F2 of dimension (C0 / 4)*H0*W0;

[0013] Connect the feature map F, feature map F1 and feature map F2 in series along the channel direction to obtain a feature map F3 with a dimension of C0*H0*W0;

[0014] The feature map F3 is randomly shuffled along the channel direction to obtain an output feature map F4 with a dimension of C0*H0*W0.

[0015] Furthermore, the construction method of the Block_E module includes:

[0016] The input feature map E0 with dimension C1*H1*W1 is passed through the 2D convolution module to obtain the feature map E1 with dimension C2*H2*W2;

[0017] Input the feature map E1 into the Block0 to obtain a feature map E2 with a dimension of C2*H2*W2 and a main feature map E3 with a dimension of (C2 / 2)*H2*W2;

[0018] The input feature map E0 with dimension C1*H1*W1 is passed through a point-by-point convolution layer to obtain a feature map E4 with dimension C2*H2*W2;

[0019] Feature maps E3 and E4 are added pixel by pixel, and then passed through an activation function to obtain feature map E5 with a dimension of C2*H2*W2, which is used as the output feature map of module Block_E.

[0020] Furthermore, the construction method of the Block_M module includes:

[0021] The input feature map M0 with a dimension of C3*H3*W3 passes through the Block0, the activation function and the Block0 in sequence to obtain a feature map M1 with a dimension of C3*H3*W3;

[0022] The input feature map M0 and feature map M1 with a dimension of C3*H3*W3 are added pixel by pixel, and then passed through an activation function to obtain a feature map M2 with a dimension of C3*H3*W3, which is used as the output feature map of module Block_M.

[0023] Furthermore, the method for constructing the self-supervised monocular depth estimation network model fD includes:

[0024] S11: The input image passes through a 2D convolution module to obtain the feature map F main0, and then pass through a pooling layer to obtain the feature map EL0, where the number of channels, height and width of the input image are 3, H, and W respectively;

[0025] S12: The feature map EL0 passes through two basic residual modules of ResNet in sequence to obtain the feature map EL1, among which the main feature map F is obtained after passing through the first basic residual module. main1 ;

[0026] S13: The feature map EL1 passes through the Block_E module in sequence, and the number of channels of the obtained feature map becomes twice the number of channels of the input feature map, and the height and width are reduced to half and the Block_M module obtains the feature map EL2; the main feature map F is obtained in the Block_E module main2 ;

[0027] S14: feature map EL2 is used as input, and step S13 is repeated twice to obtain feature maps EL3 and EL4. The main feature maps obtained are denoted as F main3 、F main4 ;

[0028] S15: The feature map EL4 passes through a 2D convolution module and is upsampled to double the height and width to obtain the feature map CL4. The feature map CL4 is consistent with the F main3 The layers are connected in series along the channel direction and passed through a 2D convolution module to obtain a feature map DL4. DL4 is passed through a 2D convolution layer and a sigmoid function to obtain a feature map DL4' with a channel number of 1. The feature map DL4' is upsampled to double its height and width, which is recorded as feature map D4.

[0029] S16: The feature map DL4 is passed through a 2D convolution module and upsampled to double the height and width; a feature map CL3 is obtained, and the feature map CL3 is consistent with the F main2 And D4 are connected in series along the channel direction, and the feature map DL3 is obtained through a 2D convolution module; DL3 is further passed through a 2D convolution layer and a sigmoid function to obtain a feature map DL3' with a channel number of 1, and the feature map DL3' is upsampled to double the height and width, which is recorded as feature map D3;

[0030] S17: Repeat step S16 twice to obtain feature maps DL2 and DL1, and upsampled feature maps D2 and D1 respectively;

[0031] S18: The feature map DL1 passes through a 2D convolution module and is upsampled. The obtained feature map CL0 is connected in series with the feature map D1 along the channel direction, and then passes through another 2D convolution module to obtain the feature map DL0.

[0032] S19: The feature map DL0 passes through a 2D convolution layer and a sigmoid function to obtain the final network output of the depth estimation network model. The number of channels of the output is 1 and the resolution is the same as the input image.

[0033] Furthermore, the method for constructing the self-supervised monocular pose estimation network model fD includes:

[0034] S21: The input images used to estimate the pose are spliced ​​together along the channel direction, with the number of channels, height and width being C, H and W respectively; the input images are sequentially passed through a 2D convolution module, a pooling layer, and two basic residual modules to obtain the feature map W1;

[0035] S22: the feature map W1 passes through the Block_E and Block_M modules in sequence, and this process is repeated twice to obtain the feature map W2;

[0036] S23: The feature map W2 passes through 4 2D convolution modules and a point-by-point convolution layer to obtain the feature map W3. The number of channels of the feature map W3 is the number of input images minus 1, multiplied by 6. The average of the obtained feature map along the height and width directions is taken, and a 6-dimensional vector with the pose estimation result is output.

[0037] Furthermore, the 6-dimensional vector includes a feature map with 3 rotation Euler angles and 3 translation amounts for each pixel point.

[0038] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0039] The present invention constructs a self-supervised monocular depth estimation network model fD and a self-supervised monocular pose estimation network model fP through a 2D convolution module, a basic residual module, a Block_E module and a Block_M module, performs depth estimation and pose estimation on the input image, and only uses the main features in the jump connection of the network model, thereby reducing the network calculation amount and model storage amount during feature extraction and feature decoding. At the same time, the model can be inferred and updated simultaneously in the terminal device with limited computing and storage resources, thereby solving the problem that the algorithm calculation and storage overhead are too large to run in real time on the terminal side. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of the self-supervised monocular depth and pose estimation method in Example 1;

[0041] Figure 2 2D convolution module in Example 1;

[0042] Figure 3 This is a structural diagram of the Block0 module in Example 1;

[0043] Figure 4It is a structural diagram of the Block_E module in Example 1;

[0044] Figure 5 It is a structural diagram of the Block_M module in Example 1;

[0045] Figure 6 : is a structural diagram of the self-supervised monocular depth estimation network model fD in Example 1;

[0046] Figure 7 It is a structural diagram of the self-supervised monocular pose estimation network model fP in Example 1. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0048] This embodiment provides a self-supervised monocular depth and pose estimation method for lightweight real-time self-supervised monocular RGB image depth and pose estimation. The method is based on two network models: a lightweight self-supervised monocular depth estimation network structure fD and a lightweight self-supervised monocular pose estimation network structure fP. Depth estimation and pose estimation are performed on input images. The self-supervised lightweight monocular depth estimation network can be executed in real time on a terminal device with limited computing and storage resources, solving the problem that the algorithm calculation and storage overhead are too large to run in real time on the terminal device.

[0049] Example 1

[0050] The lightweight self-supervised monocular depth estimation network model fD and the lightweight self-supervised monocular pose estimation network model fP described in this embodiment both include a 2D convolution module, a basic residual module, a Block_E module and a Block_M module.

[0051] 1. Build the basic network structure

[0052] like Figure 2 As shown, the 2D convolution module includes a 2D convolution layer, a normalization layer, and an activation function layer connected in sequence.

[0053] like Figure 3 As shown, the Block0 module includes a 2D convolution module for extracting basic features of the input feature map. The construction method of the Block0 module is:

[0054] The input feature map with dimension C0*H0*W0 is passed through the 2D convolution module to obtain a feature map with dimension (C0 / 2)*H0*W0, which is recorded as feature map F;

[0055] The feature map F is sequentially passed through the depth-wise convolution layer and the normalization layer to obtain the feature map F1 of dimension (C0 / 4)*H0*W0; the feature map F is sequentially passed through the point-wise convolution layer and the normalization layer to obtain the feature map F2 of dimension (C0 / 4)*H0*W0;

[0056] Connect the feature map F, feature map F1 and feature map F2 in series along the channel direction to obtain a feature map F3 with a dimension of C0*H0*W0;

[0057] The feature map F3 is randomly shuffled along the channel direction to obtain an output feature map F4 with a dimension of C0*H0*W0.

[0058] like Figure 4 As shown in the figure, the Block_E module includes a 2D convolution module and a Block0 module. The Block_E module extracts the features of the input image, expands the extraction channel, and can obtain more feature maps for feature extraction. The construction method of the Block_E module is:

[0059] The input feature map E0 with dimension C1*H1*W1 is passed through the 2D convolution module to obtain the feature map E1 with dimension C2*H2*W2;

[0060] Input the feature map E1 into the Block0 to obtain a feature map E2 with a dimension of C2*H2*W2 and a main feature map E3 with a dimension of (C2 / 2)*H2*W2;

[0061] The input feature map E0 with dimension C1*H1*W1 is passed through a point-by-point convolution layer to obtain a feature map E4 with dimension C2*H2*W2;

[0062] Feature maps E3 and E4 are added pixel by pixel, and then passed through an activation function to obtain feature map E5 with a dimension of C2*H2*W2, which is used as the output feature map of module Block_E.

[0063] like Figure 5 As shown in the figure, the Block_M module includes a 2D convolution module and a Block0 module. The Block_M module further encodes the features after the Block_E expansion channel. While maintaining the original number of channels, the computational complexity of feature extraction and feature decoding is reduced, and the accuracy of self-supervised monocular depth of field or pose estimation is improved. The construction method of the Block_M module is:

[0064] The input feature map M0 with a dimension of C3*H3*W3 passes through the Block0, the activation function and the Block0 in sequence to obtain a feature map M1 with a dimension of C3*H3*W3;

[0065] The input feature map M0 and feature map M1 with a dimension of C3*H3*W3 are added pixel by pixel, and then passed through an activation function to obtain a feature map M2 with a dimension of C3*H3*W3, which is used as the output feature map of module Block_M.

[0066] 2. Constructing a self-supervised monocular depth estimation network model fD

[0067] like Figure 6 As shown, the construction method of the self-supervised monocular depth estimation network model fD includes:

[0068] S11: The input image passes through a 2D convolution module to obtain the feature map F main0 , and then pass through a pooling layer to obtain the feature map EL0, where the number of channels, height and width of the input image are 3, H, and W respectively;

[0069] S12: The feature map EL0 passes through two basic residual modules of ResNet in sequence to obtain the feature map EL1, among which the main feature map F is obtained after passing through the first basic residual module. main1 ;

[0070] S13: The feature map EL1 passes through the Block_E module in sequence, and the number of channels of the obtained feature map becomes twice the number of channels of the input feature map, and the height and width are reduced to half and the Block_M module obtains the feature map EL2; the main feature map F is obtained in the Block_E module main2 ;

[0071] S14: feature map EL2 is used as input, and step S13 is repeated twice to obtain feature maps EL3 and EL4. The main feature maps obtained are denoted as F main3 、F main4 ;

[0072] S15: The feature map EL4 passes through a 2D convolution module and is upsampled to double the height and width to obtain the feature map CL4. The feature map CL4 is consistent with the F main3 The layers are connected in series along the channel direction and passed through a 2D convolution module to obtain a feature map DL4. DL4 is passed through a 2D convolution layer and a sigmoid function to obtain a feature map DL4' with a channel number of 1. The feature map DL4' is upsampled to double its height and width, which is recorded as feature map D4.

[0073] S16: The feature map DL4 is passed through a 2D convolution module and upsampled to double the height and width; a feature map CL3 is obtained, and the feature map CL3 is consistent with the F main2And D4 are connected in series along the channel direction, and the feature map DL3 is obtained through a 2D convolution module; DL3 is further passed through a 2D convolution layer and a sigmoid function to obtain a feature map DL3' with a channel number of 1, and the feature map DL3' is upsampled to double the height and width, which is recorded as feature map D3;

[0074] S17: Repeat step S16 twice to obtain feature maps DL2 and DL1, and upsampled feature maps D2 and D1 respectively;

[0075] S18: The feature map DL1 passes through a 2D convolution module and is upsampled. The obtained feature map CL0 is connected in series with the feature map D1 along the channel direction, and then passes through another 2D convolution module to obtain the feature map DL0.

[0076] S19: The feature map DL0 passes through a 2D convolution layer and a sigmoid function to output a feature map with the depth of field estimation result.

[0077] 3. Constructing the self-supervised monocular pose estimation network model fP

[0078] like Figure 7 As shown, the construction method of the self-supervised monocular pose estimation network model fP includes:

[0079] S21: The input images used to estimate the pose are spliced ​​together along the channel direction, with the number of channels, height and width being C, H and W respectively; the input images are sequentially passed through a 2D convolution module, a pooling layer, and two basic residual modules to obtain the feature map W1;

[0080] S22: The feature map W1 passes through the Block_E and Block_M modules in sequence, and the step S22 is repeated twice to obtain the feature map W2;

[0081] S23: The feature map W2 passes through 4 2D convolution modules and a point-by-point convolution layer to obtain the feature map W3. The number of channels of the feature map W3 is the number of input images minus 1, multiplied by 6. The average of the obtained feature map along the height and width directions is taken, and a 6-dimensional vector with the pose estimation result is output.

[0082] The 6-dimensional vector includes a feature map with 3 rotation Euler angles and 3 translation amounts for each pixel point.

[0083] 4. Process the depth of field and posture of the picture

[0084] like Figure 1 As shown, a self-supervised monocular depth pose estimation method comprises the following specific steps:

[0085] Get the original feature image;

[0086] The original feature image is input into the pre-built self-supervised monocular depth estimation network model fD and self-supervised monocular pose estimation network model fP for feature extraction, and a feature map with depth and pose estimation results is output.

[0087] The present invention constructs a self-supervised monocular depth estimation network model fD and a self-supervised monocular pose estimation network model fP through a 2D convolution module, a basic residual module, a Block_E module and a Block_M module, performs depth estimation and pose estimation on the input image, and only uses the main features in the jump connection of the network model, thereby reducing the network calculation amount and model storage amount during feature extraction and feature decoding. At the same time, the model can be inferred and updated simultaneously in the terminal device with limited computing and storage resources, thereby solving the problem that the algorithm calculation and storage overhead are too large to run in real time on the terminal side.

[0088] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0089] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0092] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A self-supervised monocular depth pose estimation method, It is characterized in that include: Get the original feature image; The original feature image is input into the pre-established self-supervised monocular depth estimation network model fD and the self-supervised monocular pose estimation network model fP for feature extraction, and a feature map with depth and pose estimation results is output; The self-supervised monocular depth estimation network model fD and the self-supervised monocular pose estimation network model fP both include a 2D convolution module, a basic residual module, a Block_E module and a Block_M module; The Block_E module and the Block_M module both include a Block0 module; The construction method of the Block0 module includes: The input feature map with dimension C0*H0*W0 is passed through the 2D convolution module to obtain a feature map with dimension (C0 / 2)*H0*W0, which is recorded as feature map F; The feature map F is sequentially passed through the depth-wise convolution layer and the normalization layer to obtain the feature map F of dimension (C0 / 4)*H0*W0 1 ; The feature map F is passed through the point-by-point convolution layer and the normalization layer in turn to obtain the feature map F of dimension (C0 / 4)*H0*W0 2 ; The feature map F and feature map F 1 And the feature map F 2 ; Connect in series along the channel direction to obtain a feature map F with a dimension of C0*H0*W0 3 ; The feature map F 3 Perform a random shuffle along the channel direction to obtain an output feature map F with a dimension of C0*H0*W0 4 ; The construction method of the Block_E module includes: Input feature map E with dimension C1*H1*W1 0 After the 2D convolution module, we get a feature map E with a dimension of C2*H2*W2. 1 ; The feature map E 1 Input the Block0 to obtain the feature map E with dimension C2*H2*W2 2 The main feature graph E with dimension (C2 / 2)*H2*W2 3 ; The input feature map E with dimension C1*H1*W1 0 After the point-by-point convolution layer, we get a feature map E with a dimension of C2*H2*W2. 4 ; The feature map E 3 And feature map E 4 Add pixel by pixel, and then pass an activation function to get the feature map E with dimension C2*H2*W2. 5 , as the output feature map of module Block_E; The construction method of the Block_M module includes: Input feature map M with dimension C3*H3*W3 0 After passing through the Block0, the activation function and the Block0 in sequence, a feature map M with a dimension of C3*H3*W3 is obtained. 1 ; The input feature map M with dimension C3*H3*W3 0 and feature map M 1 Add pixel by pixel, and then pass an activation function to get the feature map M with dimension C3*H3*W3. 2 , as the output feature map of module Block_M.

2. The self-supervised monocular depth pose estimation method according to claim 1, It is characterized in that The 2D convolution module includes a 2D convolution layer, a normalization layer and an activation function layer connected in sequence.

3. The self-supervised monocular depth pose estimation method according to claim 1, It is characterized in that The basic residual module includes a ResNet module.

4. The self-supervised monocular depth pose estimation method according to any one of claims 1 to 3, It is characterized in that The method for constructing the self-supervised monocular depth estimation network model fD includes: S11: The input image passes through a 2D convolution module to obtain the feature map F main0 , and then pass through a pooling layer to obtain the feature map EL 0 , where the number of channels, height and width of the input image are 3, H, and W respectively; S12: Feature Map EL 0 After passing through two ResNet basic residual modules in sequence, the feature map EL is obtained. 1 , where the main feature map F is obtained after the first basic residual module main1 ; S13: Feature Map EL 1 After passing through the Block_E module in sequence, the number of channels of the obtained feature map becomes twice the number of channels of the input feature map, and the height and width are reduced to half and the Block_M module obtains the feature map EL 2 ; The main feature map F is obtained in the Block_E module main2 ; S14: Feature Map EL 2 As input, repeat step S13 twice to get the feature map EL 3 ,EL 4 The main feature maps are denoted as F main3 、F main4 ; S15: Feature Map EL 4 After a 2D convolution module and upsampling, the height and width are doubled to obtain the feature map CL 4 , feature map CL 4 With the F main3 Connect in series along the channel direction and pass through the 2D convolution module to obtain the feature map DL 4 DL 4 After another 2D convolution layer and sigmoid function, we get a feature map DL with a channel number of 1. 4 ', for the feature map DL 4 'Upsample to double the height and width, recorded as feature map D 4 ; S16: the feature map DL 4 After a 2D convolution module and upsampling, the height and width are doubled; the feature map CL is obtained 3 , feature map CL 3 With the F main2 and D 4 Connect in series along the channel direction and pass through the 2D convolution module to obtain the feature map DL 3 DL 3 After another 2D convolution layer and sigmoid function, we get a feature map DL with a channel number of 1. 3 ', for the feature map DL 3 'Upsample to double the height and width, recorded as feature map D 3 ; S17: Repeat step S16 twice to obtain feature maps DL 2 and DL 1 , and the upsampled feature map D 2 and D 1 ; S18: Feature Map DL 1 After a 2D convolution module and upsampling, the resulting feature map CL 0 With feature map D 1 The concatenation is done along the channel direction, and then passes through a 2D convolution module to obtain the feature map DL 0 ; S19: Feature Map DL 0 After a 2D convolution layer and a sigmoid function, a feature map with depth estimation results is output.

5. The self-supervised monocular depth pose estimation method according to any one of claims 1 to 3, It is characterized in that The method for constructing the self-supervised monocular pose estimation network model fP includes: S21: The input images used to estimate the pose are spliced ​​together along the channel direction, with the number of channels, height and width being C, H and W respectively; the input images are sequentially passed through a 2D convolution module, a pooling layer, and two basic residual modules to obtain the feature map W 1 ; S22: Feature map W 1 Pass through the Block_E and Block_M modules in sequence, and repeat step S22 twice to obtain the feature map W 2 ; S23: Feature map W 2 After four 2D convolution modules and a point-by-point convolution layer, the feature map W is obtained. 3 , feature map W 3 The number of channels is the number of input images minus 1, multiplied by 6. The obtained feature map is averaged along the height and width directions, and a 6-dimensional vector with the pose estimation result is output.

6. The self-supervised monocular depth pose estimation method according to claim 5, It is characterized in that The 6-dimensional vector includes a feature map with 3 rotation Euler angles and 3 translation amounts for each pixel.