A spatial structure perception based self-supervised depth estimation method

Abstract

The application provides a self-supervised depth estimation method based on space structure perception. The method adopts different optimization strategies for structured regions and unstructured regions. Specifically, the scene is divided into structured (or planar) regions and unstructured (or non-planar) regions, and a time sequence geometry structure alignment module and a space geometry uncertainty guidance module are respectively designed. For the structured region, the time sequence structure alignment module introduces a novel selective plane alignment constraint, which promotes the network to learn accurate spatial geometric properties. For the unstructured region, the space geometry uncertainty guidance module models the depth uncertainty based on the structure complexity, and uses the uncertainty map to suppress the photometric loss in the unstructured region, thereby enhancing the adaptability of the network to complex outdoor scenes. Therefore, the application has good application prospect in the automatic driving scene.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving depth estimation, and in particular relates to an unsupervised monocular depth estimation method based on spatial structure perception. Background Art

[0002] Monocular depth estimation (MDE) aims to infer the depth of each pixel from a single image. It is widely used in various applications such as autonomous driving, robot navigation, and augmented reality. Monocular depth estimation provides an effective way for these applications, enabling intelligent systems to perceive the environment more accurately, thereby improving the safety and efficiency of navigation. In virtual reality and augmented reality applications, depth information plays an important role in realism and interactivity. Through monocular depth estimation, virtual objects can be more accurately embedded in real scenes, improving the user's immersion and experience quality. However, it is extremely difficult to obtain a large amount of dense depth annotations from the real world. To alleviate this limitation, researchers have tried to infer depth from a large amount of unlabeled monocular videos through self-supervision.

[0003] Self-supervised depth estimation transforms the depth prediction task into an image synthesis problem, using the photometric loss between the synthesized image and the target image as a supervisory signal, thus eliminating the need for large-scale annotated data. Early work focused on improving the quality of depth maps by designing better network structures and more reasonable loss functions. While these methods achieved good results, the lack of geometric constraints resulted in poor geometric detail and significant inconsistency in depth between frames. Therefore, exploring geometric cues from the scene has become a key focus of recent self-supervised monocular depth estimation research.

[0004] One class of methods directly establishes consistency constraints between predicted depths from adjacent frames, thereby leveraging temporal geometric cues. While these methods mitigate the scale inconsistency problem between samples, they ignore inter-pixel correlations and model depth independently on a pixel-by-pixel basis, resulting in limited improvement in scene structure accuracy. Furthermore, their consistency constraints are fragile; even small mismatches can lead to significant depth errors, particularly in regions with complex structures.

[0005] Another class of methods introduces the concept of piecewise planes, exploiting the geometric relationship between depth and plane normals to mine spatial geometric clues. Although these methods provide additional geometric constraints in structured (or planar) regions, they ignore the temporal correlation of geometric properties and fail to distinguish between structured and unstructured regions, which can lead to significant noise entering the constraint process and thus affecting the accuracy of the results. Existing methods generally treat all parts of the scene equally, ignoring the structural differences between them, making the network easily affected by noise. Summary of the Invention

[0006] The technical solution adopted by the present invention to solve the above technical problems is: a self-supervised depth estimation method based on spatial structure perception, comprising the following steps:

[0007] Step S1: Collect training image data and preprocess;

[0008] Step S2: Construct an end-to-end depth prediction network and pose prediction network, and build a self-supervised training framework based on spatial structure perception; the depth prediction network includes an encoder and a decoder; the associated source frame and target frame images are respectively used as input to obtain depth features through the encoder; the depth features are input to the decoder to output the target frame and source frame depth maps, and the normal map and distance map of the spatial plane are constructed based on the depth map; the pose prediction network includes an encoder, which takes the source frame and target frame images as input and predicts the relative camera pose from the target frame to the source frame; the depth map and camera pose are combined to perform image reconstruction, and the photometric error between the reconstructed image and the target image is used as a supervision signal to achieve self-supervised training;

[0009] Step S3: extracting large planar regions using the normal map and the input image, thereby dividing the large planar regions in the scene into structured regions and other regions into unstructured regions;

[0010] Step S4: For the structured region, the relative camera pose from the target frame to the source frame is obtained by the pose prediction network, and the geometric relationship between the normal map and the distance map of the target frame and the source frame is established. The normal map and the distance map of the target frame are reconstructed based on the normal map and the distance map of the source frame, and the inter-frame plane alignment constraint is further constructed. The structured region is then combined to obtain the selective plane alignment constraint.

[0011] Step S5: for the unstructured area, model the depth uncertainty, apply the minimum photometric error guided by the spatial geometric structure, and suppress the minimum photometric error in the unstructured area;

[0012] Step S6: Perform self-supervised training on the depth prediction network and the pose prediction network to obtain a trained depth prediction network for subsequent monocular depth estimation.

[0013] The step S3 is specifically as follows:

[0014] Assuming that the shooting height of all collected images is fixed, the y-axis of the camera coordinate system points downward, the x-axis points to the right, and the z-axis points forward, the ground normal in the camera coordinate system is obtained is (0,-1,0) T , T represents transpose; for outdoor images of size H×W, a binary mask M is used g ∈R H×WTo mark pixels belonging to the ground area in the image:

[0015]

[0016] Among them, λ1 is the similarity threshold, [·] is the Iverson bracket, if the normal at the pixel point p is the same as the ground normal The cosine similarity of is less than λ1, then in M g Marked as 1, otherwise marked as 0;

[0017] Then the low-frequency region is extracted as the approximation of the non-ground plane region, and the target image I is calculated by the following formula t The spatial frequency freq∈R H×W :

[0018] freq=||σ x (I t ),σ y (I t )||

[0019] where σ x and σ y are the gradients in the horizontal and vertical directions respectively; extract the low-frequency component area as an approximation of the non-ground plane and use M f express:

[0020] M f =[freq<λ2]

[0021] Where λ2 is an adjustable threshold, combined with the ground area mask M g , get the mask M plane , representing a large planar region with a simple structure:

[0022]

[0023] in, Representation and operation; according to M plane , the scene can be divided into structured and unstructured areas.

[0024] The step S4 is specifically as follows:

[0025] The pose prediction network predicts the camera pose T from the target view to the source view t→s By the rotation matrix r t→s and the translation vector t t→s Composition, a 3D point P in the target perspective t Convert to the 3D point P corresponding to the source perspective through the following formula s :

[0026] P s =Tt→s P t =r t→s P t +t t→s

[0027] The normal-distance constraint relationship between the source coordinate system and the target coordinate system is as follows:

[0028]

[0029] P s and P t Corresponding points in the source coordinate system and the target coordinate system, N s and d s P s Normal and distance to the plane, N t and d t P t The normal and distance of the plane, the superscript T indicates transposition; the association relationship between the geometric attributes of adjacent frames is constructed:

[0030]

[0031] Then perform perspective alignment through the camera pose T t→s =[r t→s ,t t→s ], the plane normal N in the source camera coordinate system s and the plane distance d s Transform to the target camera coordinate system:

[0032]

[0033] in, Indicates the plane distance after alignment, Represents the aligned plane normals, and then a warping operation is performed to restore the normal map and distance map of the target frame:

[0034]

[0035] Among them, D t is the depth map of the target frame, T t→s is the relative pose of the camera, K is the camera intrinsic parameter, <·> is the differentiable bilinear interpolation operation, and proj(·) is the coordinate projection operation, which projects the coordinates of the target frame to the corresponding points of the source frame;

[0036] Using the aligned planar geometric properties of the target frame, the depth map D of the target frame is reconstructed using the following formula: s→t :

[0037]

[0038] Among them, p is a coordinate of the target frame, is the corresponding homogeneous coordinate; then synthesize the target image

[0039]

[0040] The planar alignment constraint is defined as:

[0041]

[0042] Among them, L pe is the minimum photometric error;

[0043] The mask M plane Apply to L pa , we get the selective plane alignment constraint L spa :

[0044] L spa =M plane ·L pa .

[0045] The step S5 is specifically as follows:

[0046] First, we model the depth uncertainty in 3D space and obtain the uncertainty map U:

[0047] U=||σ x (N),σ y (N)|| x

[0048] Where N is the normal map derived from the depth map, σ x and σ y are the horizontal and vertical gradients,

[0049] Complex structure region mask M us Expressed as:

[0050] M us =1-M plane

[0051] Based on the unstructured region mask M us and uncertainty map U, adjust the minimum photometric error, and improve the photometric loss for:

[0052]

[0053] Among them, e is a natural constant and λ3 is an adjustable parameter.

[0054] The present invention proposes an optimization strategy that combines structured and unstructured regions for complex outdoor autonomous driving scenarios, effectively improving the ability to recover depth information in video sequences. Specifically, in autonomous driving scenarios, structured regions such as roads and buildings typically have clear geometric features, while unstructured regions such as trees and pedestrians often have complex features and are noisy. This method, by designing separate optimization schemes for structured and unstructured regions, fully exploits the temporal information and spatial geometric cues between video frames, significantly reducing noise interference. In practical applications, this method leverages the consistent geometric properties of pixels on the same plane to impose geometric constraints on regions with a clear collective structure, such as road surfaces or walls. This enhances the accuracy of the geometric structure and improves the robustness of the geometric constraints to noise. Consequently, in autonomous driving image perception tasks, this method not only effectively recovers depth information from the road surface but also more accurately estimates the depth of unstructured regions. Furthermore, this method ensures the accuracy of absolute depth estimation based on geometric properties, enabling self-supervised depth estimation methods to meet the application requirements of practical autonomous driving systems. This innovative design enables the present invention to perform well in complex outdoor scenes, providing solid technical support for accurate depth perception in the autonomous driving field. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Flowchart of the self-supervised depth estimation method based on spatial structure perception. DETAILED DESCRIPTION

[0056] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a self-supervised depth estimation method based on spatial structure perception of the present invention in conjunction with the accompanying drawings.

[0057] This embodiment specifically improves the ability to recover depth information in video sequences from two aspects: the division of spatial structure regions and regionalized response strategies.

[0058] 1) Spatial structure region partitioning. Assume that a 3D scene consists of piecewise planes, parameterized by plane normals and plane distances in the camera coordinate system. In the self-supervised depth estimation framework, a depth prediction network predicts a depth map. Projecting the depth map into 3D space yields a point cloud, from which a normal map is derived. Combining the normal map and depth map yields a distance map. The normal map and input image are then used to extract large planar regions, thereby partitioning the scene into structured regions (large planar regions) and unstructured regions (regions with complex structures).

[0059] 2) Regionalized response strategy. For structured regions, the temporal structure alignment module establishes the geometric relationship between adjacent pixels. Subsequently, the geometric properties are aligned across frames, and a novel selective plane alignment constraint is introduced to ensure that the network can learn the correct geometric properties. For unstructured regions, the depth uncertainty is explicitly modeled based on the complexity of the geometric structure to generate an uncertainty map. This map can characterize the complexity of the local spatial structure. Guided by the uncertainty map, the photometric loss of the unstructured region is weighted, making the existing photometric loss more suitable for the spatial structure of complex scenes.

[0060] This embodiment provides a flowchart of an end-to-end self-supervised depth estimation method based on deep learning. The specific steps of this method are as follows:

[0061] Step S1: Acquire and preprocess training data. The training data consists of a collection of continuous frames captured by a highly fixed monocular camera. In this paper, the KITTI open-source dataset is used for unsupervised training. This dataset consists of 200 street scenes. Static frames are removed during preprocessing before training. The training and validation sets contain 39,810 and 4,424 frames, respectively, with 697 frames used for testing.

[0062] Step S2: Build an end-to-end depth prediction network and pose prediction network, and build a self-supervised training framework based on spatial structure perception; Figure 1 The self-supervised depth estimation framework shown in the figure, the depth prediction network includes an encoder and a decoder; the source frame I of the continuous frame s and target frame image I t The encoder takes them as input to get the depth features; the depth features are input to the decoder to output the depth map D of the target frame and the source frame. t 、D s , construct the normal map and distance map of the spatial plane according to the depth map; the pose prediction network includes an encoder, which takes the source frame I s and target frame image I t As input, predict the relative camera pose T from the target frame to the source frame t→s ; Combined with the depth map D t and camera pose T t→s Perform image reconstruction and use the reconstructed image I s→t and target image I t The photometric error is used as a supervisory signal to achieve self-supervised training.

[0063] Step S3: Large planar regions are extracted using the normal map and the input image, thereby dividing the scene into structured regions (large planar regions) and unstructured regions (regions with complex structures).

[0064] Specifically, in the autonomous driving scenario, all images are captured by a camera mounted on the vehicle, and the camera is at a fixed height. The y-axis of the camera coordinate system points downward, the x-axis points to the right, and the z-axis points forward. Since the vehicle's motion is always parallel to the ground, the ground normal in the camera coordinate system is It can be approximated as (0,-1,0) T , T represents transposition; the pixel-level plane normal map N is obtained from the depth map, so by calculating the normal map N and the ground normal The cosine similarity between them can identify the area where the normal is close to the ground normal; for outdoor images of size H×W, a binary mask M is used g ∈R H×W To mark the pixels belonging to the ground area in the image, R represents a real number, and H×W is the image size:

[0065]

[0066] Among them, λ1 is the similarity threshold, [·] is the Iverson bracket, if the normal at the pixel point p is the same as the ground normal The cosine similarity of is less than λ1, then in M g It is marked as 1, otherwise it is marked as 0.

[0067] Then the low-frequency region is extracted as the approximation of the non-ground plane region, and the target image I is calculated by the following formula t The spatial frequency freq∈R H×W :

[0068] freq=||σ x (I t ),σ y (I t )||2

[0069] where σ x and σ y are the gradients in the horizontal and vertical directions respectively, and ‖·‖2 is the L2 norm. It can be found that large plane areas (such as walls) in non-ground areas usually correspond to low-frequency components. Therefore, the low-frequency component area is extracted as an approximation of the non-ground plane and M is used. f Indicates these areas:

[0070] M f =[freq<λ2]

[0071] Where λ2 is an adjustable threshold, combined with the ground area mask M g , we can get the mask M plane , representing a large planar region with a simple structure:

[0072]

[0073] in, Representation and operation. According to M plane , the scene can be divided into structured areas (large planar areas) and unstructured areas (areas with complex structures), and different optimization methods can be used according to the spatial geometric structure.

[0074] Step S4: For the structured area, the relative camera pose from the target frame to the source frame is obtained by the pose prediction network, and the geometric relationship between the normal map and the distance map of the target frame and the source frame is established. The true normal map and distance map of the target are reconstructed based on the normal map and distance map of the source frame, and the inter-frame plane alignment constraint is further constructed.

[0075] Specifically, given the estimated depth maps of the target and source perspectives, the plane normals and plane distances of the target and source perspectives are derived, and the pose prediction network predicts the camera pose T from the target perspective to the source perspective. t→s By the rotation matrix r t→s and the translation vector t t→s Composition, and a 3D point P in the target perspective t , and the corresponding point P under the original viewing angle s There is a clear conversion relationship, which can be converted to the source perspective through the following formula:

[0076] P s =T t→s P t =r t→s P t +t t→s

[0077] The normal-distance constraint relationship between the source coordinate system and the target coordinate system is as follows:

[0078]

[0079] P s and P t Corresponding points in the source coordinate system and the target coordinate system, N s and d s P s Normal and distance to the plane, N t and d t P t The normal and distance of the plane, combined with the perspective conversion formula and the target coordinate system normal distance constraint, yield the following formula:

[0080]

[0081] This formula is another form of the normal-distance constraint in the source view, so it can build the correlation relationship between the geometric properties of adjacent frames:

[0082]

[0083] The geometric properties of the plane will change with the change of the camera coordinate system; the rotation of the camera only affects the plane normal, and the translation of the camera only affects the plane distance. Therefore, in order to reconstruct the plane normal and plane distance map of the target perspective from the source perspective, the perspective is first aligned, and the camera pose T is used to calculate the plane normal and plane distance map. t→s =[r t→s ,t t→s ], the plane normal N in the source camera coordinate system s and the plane distance d s Transform to the target camera coordinate system:

[0084]

[0085] Indicates the plane distance after alignment, Representing the aligned plane normal, we then recover the normal map and distance map of the target frame through a warp operation:

[0086]

[0087] Among them, D t is the depth map of the target frame, T t→s is the relative pose of the camera, K is the camera intrinsic parameter, <·> is the differentiable bilinear interpolation operation, and proj(·) is the coordinate projection operation, which projects the coordinates of the target frame to the corresponding points of the source frame.

[0088] In order to ensure that the correct geometric properties are learned by the network, a plane alignment constraint is proposed. Using the plane geometric properties of the aligned target frame, the depth map of the target frame is first reconstructed using the following formula:

[0089]

[0090] Among them, p is a coordinate of the target frame, is the corresponding homogeneous coordinate. Then, using the source image I s , the reconstructed depth map D s→t , camera relative pose T t→s , camera internal parameter K synthesizes the target image

[0091]

[0092] The planar alignment constraint is defined as:

[0093]

[0094] Among them, L pe is the minimum photometric error, I t is the target image, is the target image synthesized by plane alignment. The minimum photometric error is calculated as follows:

[0095]

[0096] Here, α = 0.85, SSIM(·,·) is the structural similarity, which is an indicator to measure the similarity between two images, and ‖·‖1 is the L1 norm.

[0097] However, the geometric properties of the planes generated from the depth are not always accurate in structureless regions, which can give L pa For large planar regions, even if the sampling points are slightly offset, they are still located on the same plane, thus generating the same or similar planar geometric properties. To this end, a selective plane alignment constraint is further proposed, which is selectively applied to large planar regions in the scene. The previous step has extracted the mask M plane To represent a large planar region with a simple structure, the mask M plane Apply to L pa , we get the selective plane alignment constraint L spa :

[0098] L spa =M plane ·L pa

[0099] This constraint selectively applies the plane alignment constraint to structured regions in corresponding scenes between different images, thus avoiding the influence of unstructured regions. Rather than directly reconstructing the target depth map, the planar geometric properties are first recovered from adjacent frames and then recombined to reconstruct the depth of the target frame. In this process, as long as the geometric properties are accurate, even if there are matching deviations, combining the geometric properties into depth will not lead to significant depth errors, making this method more robust.

[0100] Step S5: For unstructured areas, apply the photometric loss guided by spatial geometry structure, use the spatial geometric attributes predicted by the network to detect unstructured areas, and suppress the photometric loss of these areas to reduce the impact of errors in unstructured areas on global training.

[0101] Specifically, the depth uncertainty is modeled in unstructured areas, complex structure areas are assigned larger uncertainties, while areas with simple geometric structures are assigned smaller uncertainties; based on the uncertainty, a balanced photometric loss is obtained. First, the depth uncertainty in 3D space is modeled to obtain the uncertainty map U:

[0102] U=||σ x (N),σ y (N)||2

[0103] Where N is the normal map derived from the depth map, σ x and σ y are the gradients in the horizontal and vertical directions, respectively, which can represent the complexity of the local structure of the space, and ‖·‖2 is the L2 norm. The higher the gradient of the normal map, the more complex the structure, and the smaller the gradient, the simpler the structure. Under the guidance of the uncertainty map, the complex structure area is assigned a larger uncertainty, while the area with simple geometric structure is assigned a smaller uncertainty. plane represents a large planar region with simple structures, so the complex structure region mask M us It can be expressed as:

[0104] M us =1-M plane

[0105] Based on the unstructured region mask M us And the uncertainty map U, adjust the luminosity loss, balance the luminosity loss in high and low frequency areas, the improved luminosity loss for:

[0106]

[0107] Among them, e is a natural constant, λ3 is an adjustable parameter, and L pe is the original minimum photometric error.

[0108] Step S6: Perform self-supervised training on the depth prediction network and the pose prediction network to obtain a trained depth prediction network for subsequent monocular depth estimation.

[0109] Edge-aware smoothing loss is used to promote the smoothness of local depth maps while maintaining image edges:

[0110]

[0111] where σ x and σ y are the horizontal and vertical gradients, is the normalized depth to avoid the reduction of predicted depth. The final loss function is as follows:

[0112]

[0113] β, γ, It is a hyperparameter used to control the trade-off between various losses. is the photometric loss caused by geometric uncertainty, L spa is the inter-frame plane alignment loss.

[0114] The model was trained using the Adam optimizer for 20 epochs. The learning rate was 1e-4 for the first 15 epochs and 1e-5 for the last 5 epochs. The batch size was set to 12.

[0115] The deep prediction network trained using this method demonstrates significant improvements in prediction accuracy and geometric structure recovery. In complex autonomous driving scenarios, the model is able to adapt to diverse driving environments, such as urban roads, highways, and rural roads, demonstrating strong environmental adaptability. The trained model is able to more accurately recover the scene's 3D geometric structure in complex environments.

[0116] It will be understood that the present invention is described by way of some embodiments, and it will be appreciated by those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are intended to be protected by the present invention.

Claims

1. A self-supervised depth estimation method based on spatial structure perception, characterized in that: The following steps are involved: Step S1: collecting and preprocessing training image data; the image data is specifically continuous frame image data taken by a camera with a fixed height; Step S2: Construct an end-to-end depth prediction network and pose prediction network, and build a self-supervised training framework based on spatial structure perception; the depth prediction network includes an encoder and a decoder; the associated source frame and target frame images are respectively used as input to obtain depth features through the encoder; The deep feature input decoder outputs depth maps of the target frame and the source frame, and constructs a normal map and a distance map of the spatial plane based on the depth map. The pose prediction network includes an encoder that takes the source frame and the target frame as input and predicts the relative camera pose from the target frame to the source frame. The image is reconstructed by combining the depth map and the camera pose, and the photometric error between the reconstructed image and the target image is used as a supervision signal to achieve self-supervised training. Step S3: extracting large planar regions using the normal map and the input image, thereby dividing the large planar regions in the scene into structured regions and other regions into unstructured regions; Step S4: For the structured region, the relative camera pose from the target frame to the source frame obtained by the pose prediction network is used to establish the geometric relationship between the normal map and the distance map of the target frame and the source frame, and the normal map and distance map of the target frame are reconstructed based on the normal map and distance map of the source frame, and the inter-frame plane alignment constraint is further constructed; Combined with the structured region, we obtain the selective plane alignment constraint; Step S5: for the unstructured area, model the depth uncertainty, apply the minimum photometric error guided by the spatial geometric structure, and suppress the minimum photometric error in the unstructured area; Step S6: Perform self-supervised training on the depth prediction network and the pose prediction network to obtain a trained depth prediction network for subsequent monocular depth estimation.

2. The self-supervised depth estimation method based on spatial structure perception according to claim 1, characterized in that The step S3 is specifically as follows: The shooting height of all collected images is fixed, the y-axis of the camera coordinate system points downward, the x-axis points to the right, and the z-axis points forward, and the ground normal in the camera coordinate system is obtained. is (0,-1,0) T , T represents transpose; for outdoor images of size H×W, a binary mask M is used g ∈R H×W Mark pixels in an image that belong to the ground: Among them, λ1 is the similarity threshold, [·] is the Iverson bracket, if the normal at the pixel point p is the same as the ground normal The cosine similarity of is less than λ1, then in M g Marked as 1, otherwise marked as 0; Then the low-frequency region is extracted as the approximation of the non-ground plane region, and the target image I is calculated by the following formula t The spatial frequency freq∈R H×W : freq=||σ x (I t ),s y (I t )||2 where σ x and σ y are the gradients in the horizontal and vertical directions respectively; extract the low-frequency component area as an approximation of the non-ground plane and use M f express: M f =[freq<λ2] Where λ2 is an adjustable threshold, combined with the ground area mask M g , get the mask M plane , representing a large planar region with a simple structure: in, Representation and operation; according to M plane ,dividing the scene into structured and unstructured areas.

3. The self-supervised depth estimation method based on spatial structure perception according to claim 2, characterized in that The step S4 is specifically as follows: The pose prediction network predicts the camera pose T from the target view to the source view t→s By the rotation matrix r t→s and the translation vector t t→s Composition, a 3D point P in the target perspective t Convert to the 3D point P corresponding to the source perspective through the following formula s : P s =T t→s P t =r t→s P t +t t→s The normal-distance constraint relationship between the source coordinate system and the target coordinate system is as follows: P s and P t Corresponding points in the source coordinate system and the target coordinate system, N s and d s They are P s Normal and distance to the plane, N t and d t They are P t The normal and distance of the plane, the superscript T indicates transposition; the association relationship between the geometric attributes of adjacent frames is constructed: Then perform perspective alignment through the camera pose T t→s =[r t→s ,t t→s ], the plane normal N in the source camera coordinate system s and the plane distance d s Transform to the target camera coordinate system: in, Indicates the plane distance after alignment, Represents the aligned plane normals, and then a warping operation is performed to restore the normal map and distance map of the target frame: Among them, D t is the depth map of the target frame, T t→s is the relative pose of the camera, K is the camera intrinsic parameter, <·> is the differentiable bilinear interpolation operation, and proj(·) is the coordinate projection operation, which projects the coordinates of the target frame to the corresponding points of the source frame; Using the aligned planar geometric properties of the target frame, the depth map D of the target frame is reconstructed using the following formula: s→t : Among them, p is a coordinate of the target frame, is the corresponding homogeneous coordinate; then synthesize the target image The planar alignment constraint is defined as: Among them, L pe is the minimum photometric error; The mask M plane Apply to L pa , we get the selective plane alignment constraint L spa : L spa =M plane ·L pa 。 4. The self-supervised depth estimation method based on spatial structure perception according to claim 3, characterized in that The step S5 is specifically as follows: First, we model the depth uncertainty in 3D space and obtain the uncertainty map U: U=||σ x (N),s y (N)||2 Where N is the normal map derived from the depth map, σ x and σ y are the gradients in the horizontal and vertical directions respectively, ‖·‖2 is the L2 norm; Complex structure region mask M us Expressed as: M us =1-M plane Based on the unstructured region mask M us and uncertainty map U, adjust the minimum photometric error, and improve the photometric loss for: Among them, e is a natural constant and λ3 is an adjustable parameter.

5. The self-supervised depth estimation method based on spatial structure perception according to claim 4, characterized in that: The minimum photometric error calculation formula is as follows: Among them, α is a parameter, SSIM(·,·) is the structural similarity, and ‖·‖1 is the L1 norm.

6. The self-supervised depth estimation method based on spatial structure perception according to claim 5, characterized in that: When performing the self-supervised training, edge-aware smoothing loss is used as the loss function: Among them, L sm represents the edge-aware smoothing loss, is the normalized depth map of the target frame; the final loss function is as follows: Among them, β, γ, is a hyperparameter that controls the trade-off between various losses, is the improved luminosity loss L spa is a selective planar alignment constraint.

Images