Self-supervised depth estimation method based on spatial structure perception

Abstract

The invention provides a self-supervision depth estimation method based on spatial structure perception. According to the method, different optimization strategies are adopted for a structured area and an unstructured area, specifically, a scene is divided into the structured (or planar) area and the unstructured (or non-planar) area, and a time sequence geometric structure alignment module and a space geometric uncertainty guiding module are designed respectively. For a structured region, a novel selective plane alignment constraint is introduced into a time sequence structure alignment module, and the network is promoted to learn accurate space geometric attributes. For an unstructured region, a space geometry uncertainty guiding module models depth uncertainty based on structural complexity, and uses an uncertainty graph to suppress luminosity loss in the unstructured region, thereby enhancing the adaptive capacity of the network to a complex outdoor scene. Therefore, the method has a good application prospect in an automatic driving scene.

Description

Technical Field

[0001] The present invention belongs to the field of autonomous driving depth estimation, and in particular relates to an unsupervised monocular depth estimation method based on spatial structure perception. Background Art

[0002] Monocular depth estimation (MDE) aims to infer the depth of each pixel from a single image. It is widely used in many applications such as autonomous driving, robot navigation, augmented reality, etc. Monocular depth estimation provides an effective way for these applications, enabling intelligent systems to perceive the environment more accurately, thereby improving the safety and efficiency of navigation. In virtual reality and augmented reality applications, depth information plays an important role in realism and interactivity. Through monocular depth estimation, virtual objects can be more accurately embedded into real scenes, improving the user's immersion and experience quality. However, it is extremely difficult to obtain a large number of dense depth annotations from the real world. To alleviate this limitation, researchers have tried to infer depth from a large number of unlabeled monocular videos through self-supervision.

[0003] Self-supervised depth estimation converts the depth prediction task into an image synthesis problem, using the photometric loss between the synthesized image and the target image as a supervisory signal, thus getting rid of the reliance on large-scale annotated data. Early work mainly improved the quality of depth maps by designing better network structures and more reasonable loss functions. Although these methods achieved good results, the lack of geometric constraints resulted in poor geometric details and significant inconsistency in depth between frames. Therefore, exploring geometric cues from the scene has become a key focus of recent self-supervised monocular depth estimation research.

[0004] There is a class of methods that directly establish consistency constraints between the predicted depths of adjacent frames, thereby exploiting temporal geometric cues. Although these methods alleviate the scale inconsistency problem between different samples, they ignore the correlation between pixels and model the depth independently pixel by pixel, which has limited effect on improving the accuracy of scene structure. In addition, their consistency constraints are relatively fragile, and even small matching deviations may lead to significant depth errors, especially in areas with complex structures.

[0005] Another class of methods introduces the concept of piecewise planes, exploiting the geometric relationship between depth and plane normal vectors to mine spatial geometric clues. Although these methods provide additional geometric constraints in structured (or planar) regions, they ignore the temporal correlation of geometric properties and fail to distinguish between structured and unstructured regions, which may lead to significant noise entering the constraint process, thus affecting the accuracy of the results. Existing methods usually treat all parts of the scene equally, ignoring the structural differences between them, making the network easily affected by noise. Summary of the invention

[0006] The technical solution adopted by the present invention to solve the above technical problems is: a self-supervised depth estimation method based on spatial structure perception, comprising the following steps:

[0007] Step S1: Collect training image data and preprocess;

[0008] Step S2: construct an end-to-end depth prediction network and pose prediction network, and build a self-supervised training framework based on spatial structure perception; the depth prediction network includes an encoder and a decoder; the associated source frame and target frame images are respectively used as input to obtain depth features through the encoder; the depth features are input to the decoder to output the target frame and source frame depth map, and the normal map and distance map of the spatial plane are constructed according to the depth map; the pose prediction network includes an encoder, which takes the source frame and target frame images as input to predict the relative camera pose from the target frame to the source frame; the image is reconstructed by combining the depth map and the camera pose, and the photometric error between the reconstructed image and the target image is used as a supervision signal, thereby realizing self-supervised training;

[0009] Step S3: extracting a large plane area using the normal map and the input image, thereby dividing the large plane area in the scene into structured areas, and dividing other areas into unstructured areas;

[0010] Step S4: For the structured region, the relative camera pose from the target frame to the source frame obtained by the pose prediction network is used to establish the geometric relationship between the normal map and the distance map of the target frame and the source frame, and the normal map and the distance map of the target frame are reconstructed according to the normal map and the distance map of the source frame, and the inter-frame plane alignment constraint is further constructed; and then combined with the structured region, the selective plane alignment constraint is obtained;

[0011] Step S5: for the unstructured area, model the depth uncertainty, apply the minimum photometric error guided by the spatial geometric structure, and suppress the minimum photometric error in the unstructured area;

[0012] Step S6: Perform self-supervised training on the depth prediction network and the pose prediction network to obtain a trained depth prediction network for subsequent monocular depth estimation.

[0013] The step S3 is specifically as follows:

[0014] Assuming that the shooting height of all collected images is fixed, the y-axis of the camera coordinate system points downward, the x-axis points to the right, and the z-axis points forward, the ground normal in the camera coordinate system is obtained is (0,-1,0) T , T represents transpose; for outdoor images of size H×W, a binary mask M is used g ∈R H×WTo mark pixels in an image that belong to the ground area:

[0015]

[0016] Among them, λ1 is the similarity threshold, [·] is the Iverson bracket, if the normal at the pixel point p is the same as the ground normal The cosine similarity of is less than λ1, then in M g It is marked as 1, otherwise it is marked as 0;

[0017] Then the low-frequency region is extracted as an approximation of the non-ground plane region, and the target image I is calculated by the following formula t The spatial frequency freq∈R H×W :

[0018] freq=||σ x (I t ),σ y (I t )||

[0019] where σ x and σ y are the gradients in the horizontal and vertical directions respectively; extract the low-frequency component area as an approximation of the non-ground plane and use M f express:

[0020] M f =[freq<λ2]

[0021] Where λ2 is an adjustable threshold, combined with the ground area mask M g , get the mask M plane , representing a large planar region with a simple structure:

[0022]

[0023] in, Representation and operation; according to M plane , the scene can be divided into structured and unstructured areas.

[0024] The step S4 is specifically as follows:

[0025] The pose prediction network predicts the camera pose T from the target view to the source view t→s By the rotation matrix r t→s and the translation vector t t→s Composition, a 3D point P from the target perspective t Convert to the 3D point P corresponding to the source perspective through the following formula s :

[0026] P s =Tt→s P t =r t→s P t +t t→s

[0027] The normal-distance constraint relationship between the source coordinate system and the target coordinate system is as follows:

[0028]

[0029] P s and P t Corresponding points in the source coordinate system and the target coordinate system, N s and d s They are P s Normal and distance to the plane, N t and d t They are P t The normal and distance of the plane, the superscript T indicates transposition; construct the association relationship between the geometric attributes of adjacent frames:

[0030]

[0031] Then the perspective alignment is performed through the camera pose T t→s =[r t→s ,t t→s ], the plane normal N in the source camera coordinate system s The distance d from the plane s Transform to the target camera coordinate system:

[0032]

[0033] in, represents the plane distance after alignment, Represents the aligned plane normals, and then a warping operation is performed to restore the normal map and distance map of the target frame:

[0034]

[0035] Among them, D t is the depth map of the target frame, T t→s is the relative position of the camera, K is the camera intrinsic parameter, <·> is a differentiable bilinear interpolation operation, and proj(·) is a coordinate projection operation that projects the coordinates of the target frame onto the corresponding points of the source frame;

[0036] Using the aligned planar geometric properties of the target frame, the depth map D of the target frame is reconstructed by the following formula: s→t :

[0037]

[0038] Among them, p is a coordinate of the target frame, is the corresponding homogeneous coordinate; then synthesize the target image

[0039]

[0040] The planar alignment constraint is defined as:

[0041]

[0042] Among them, L pe is the minimum photometric error;

[0043] The mask M plane Apply to L pa , we get the selective plane alignment constraint L spa :

[0044] L spa =M plane ·L pa .

[0045] The step S5 is specifically as follows:

[0046] First, we model the depth uncertainty in 3D space and obtain the uncertainty map U:

[0047] U=||σ x (N),σ y (N)|| x

[0048] Where N is the normal map derived from the depth map, σ x and σ y are the horizontal and vertical gradients, respectively.

[0049] Complex structure region mask M us It is expressed as:

[0050] M us =1-M plane

[0051] Based on the structureless region mask M us and uncertainty map U, adjust the minimum photometric error, and improve the photometric loss for:

[0052]

[0053] Among them, e is a natural constant and λ3 is an adjustable parameter.

[0054] The beneficial effect of the present invention is that, for complex outdoor autonomous driving scenes, an optimization strategy combining structured areas and unstructured areas is proposed, which can effectively improve the recovery ability of depth information in video sequences. Specifically, in the autonomous driving scene, structured areas such as roads and buildings usually have clear geometric features, while unstructured areas such as trees and pedestrians often have complex features and more noise. The method fully exploits the temporal information and spatial geometric clues between video frames by designing optimization schemes suitable for structured areas and unstructured areas respectively, and significantly reduces noise interference. In the application scenario, the method uses the characteristics of consistent pixel geometric properties on the same plane to geometrically constrain areas with clear set structures such as road surfaces or walls, thereby enhancing the correctness of the geometric structure and improving the robustness of geometric constraints to noise. Therefore, in the image perception task of autonomous driving, it can not only effectively process the recovery of road plane depth information, but also more accurately estimate the depth of unstructured areas. In addition, the method ensures the accuracy of absolute depth estimation based on geometric properties, so that the self-supervised depth estimation method can meet the application requirements in actual autonomous driving systems. This innovative design enables the invention to perform well in complex outdoor scenes and provides solid technical support for accurate depth perception in the field of autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Flowchart of the self-supervised depth estimation method based on spatial structure perception. DETAILED DESCRIPTION

[0056] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a self-supervised depth estimation method based on spatial structure perception of the present invention in conjunction with the accompanying drawings.

[0057] This embodiment specifically improves the ability to restore depth information in video sequences from two aspects: the division of spatial structure regions and regionalized response strategies;

[0058] 1) Division of spatial structured regions. Assume that the 3D scene consists of segmented planes, which are parameterized by plane normal vectors and plane distances in the camera coordinate system. In the self-supervised depth estimation framework, the depth prediction network predicts a depth map. Projecting the depth map into 3D space can obtain a point cloud, from which a normal map is derived. Combining the normal map and the depth map, a distance map is derived. The normal map and the input image are used to extract large plane regions, thereby dividing the scene into structured regions (large plane regions) and unstructured regions (regions with complex structures).

[0059] 2) Regionalized response strategy. For structured areas, the temporal structure alignment module establishes the geometric relationship between adjacent pixels, then aligns the geometric properties between frames and introduces a novel selective plane alignment constraint to ensure that the network can learn the correct geometric properties. For unstructured areas, the depth uncertainty is explicitly modeled based on the complexity of the geometric structure to generate an uncertainty map. This map can characterize the complexity of the local spatial structure. Under the guidance of the uncertainty map, the photometric loss of the unstructured area is weighted so that the existing photometric loss is more suitable for the spatial structure of complex scenes.

[0060] This embodiment provides a flowchart of an end-to-end self-supervised depth estimation method based on deep learning. The specific steps of this method are as follows:

[0061] Step S1: Obtain training data and preprocess; the training data is specifically a set of continuous frame image data taken by a highly fixed monocular camera. In the present invention, the KITTI open source dataset is used for unsupervised training. The dataset consists of street scenes in 200 scenes. Preprocessing is used to remove static frames before training. The training set and validation set have 39810 and 4424 frames of images respectively, and 697 frames of images are used for testing.

[0062] Step S2: Construct an end-to-end depth prediction network and pose prediction network, and build a self-supervised training framework based on spatial structure perception; Figure 1 The self-supervised depth estimation framework shown in the figure, the depth prediction network includes an encoder and a decoder; the source frame I of the continuous frame s and target frame image I t The encoder takes them as input to get the depth features; the depth features are input to the decoder to output the depth map D of the target frame and the source frame t , D s , construct the normal map and distance map of the spatial plane according to the depth map; the pose prediction network includes an encoder, based on the source frame I s and target frame image I t As input, predict the relative camera pose T from the target frame to the source frame t→s ; Combined with the depth map D t and camera pose T t→s Perform image reconstruction and use the reconstructed image I s→t and the target image I t The photometric error is used as a supervisory signal to achieve self-supervised training.

[0063] Step S3: extracting large planar areas using the normal map and the input image, thereby dividing the scene into structured areas (large planar areas) and unstructured areas (areas with complex structures).

[0064] Specifically, in the autonomous driving scenario, all images are taken by a camera mounted on the vehicle, and the camera is at a fixed height. The y-axis of the camera coordinate system points downward, the x-axis points to the right, and the z-axis points forward. Since the vehicle's motion is always parallel to the ground, the ground normal in the camera coordinate system is It can be approximated as (0,-1,0) T , T represents transpose; the pixel-level plane normal map N is obtained from the depth map, so by calculating the normal map N and the ground normal The cosine similarity between them can identify the area where the normal is close to the ground normal; for outdoor images of size H×W, a binary mask M is used g ∈R H×W To mark the pixels belonging to the ground area in the image, R represents a real number, and H×W is the image size:

[0065]

[0066] Among them, λ1 is the similarity threshold, [·] is the Iverson bracket, if the normal at the pixel point p is the same as the ground normal The cosine similarity of is less than λ1, then in M g is marked as 1, otherwise it is marked as 0.

[0067] Then the low-frequency region is extracted as an approximation of the non-ground plane region, and the target image I is calculated by the following formula t The spatial frequency freq∈R H×W :

[0068] freq=||σ x (I t ),σ y (I t )||2

[0069] where σ x and σ y are the gradients in the horizontal and vertical directions, respectively, and ‖·‖2 is the L2 norm. It can be found that large plane areas (such as walls) in non-ground areas usually correspond to low-frequency components. Therefore, the low-frequency component area is extracted as an approximation of the non-ground plane and M is used. f These areas are represented by:

[0070] M f =[freq<λ2]

[0071] Where λ2 is an adjustable threshold, combined with the ground area mask M g , we can get the mask M plane , representing a large planar region with a simple structure:

[0072]

[0073] in, Representation and operation. According to M plane , the scene can be divided into structured areas (large planar areas) and unstructured areas (areas with complex structures), and different optimization methods can be used according to the spatial geometric structure.

[0074] Step S4: For the structured area, the relative camera pose from the target frame to the source frame is obtained by the pose prediction network, and the geometric relationship between the normal map and the distance map of the target frame and the source frame is established. The true normal map and distance map of the target are reconstructed according to the normal map and distance map of the source frame, and the plane alignment constraints between frames are further constructed.

[0075] Specifically, given the estimated depth maps of the target view and the source view, the plane normal and plane distance of the target view and the source view are derived, and the pose prediction network predicts the camera pose T from the target view to the source view t→s By the rotation matrix r t→s and the translation vector t t→s The target perspective is a 3D point P t , and the corresponding point P under the original viewing angle s There is a clear conversion relationship, which can be converted to the source perspective by the following formula:

[0076] P s =T t→s P t =r t→s P t +t t→s

[0077] The normal-distance constraint relationship between the source coordinate system and the target coordinate system is as follows:

[0078]

[0079] P s and P t Corresponding points in the source coordinate system and the target coordinate system, N s and d s They are P s Normal and distance to the plane, N t and d t They are P t The normal and distance of the plane, combined with the perspective conversion formula and the target coordinate system normal distance constraint relationship, get the following formula:

[0080]

[0081] This formula is another expression of the normal-distance constraint in the source view, so it is possible to construct the relationship between the geometric properties of adjacent frames:

[0082]

[0083] The geometric properties of the plane will change with the change of the camera coordinate system; the rotation of the camera only affects the plane normal, and the translation of the camera only affects the plane distance. Therefore, in order to reconstruct the plane normal and plane distance map of the target perspective from the source perspective, the perspective is first aligned, and the camera pose T is used to t→s =[r t→s ,t t→s ], the plane normal N in the source camera coordinate system s The distance d from the plane s Transform to the target camera coordinate system:

[0084]

[0085] represents the plane distance after alignment, Representing the aligned plane normal, we then recover the normal map and distance map of the target frame through a warp operation:

[0086]

[0087] Among them, D t is the depth map of the target frame, T t→s is the relative position of the camera, K is the camera intrinsic parameter, <·> is a differentiable bilinear interpolation operation, and proj(·) is a coordinate projection operation that projects the coordinates of the target frame onto the corresponding points of the source frame.

[0088] In order to ensure that the correct geometric properties are learned by the network, a plane alignment constraint is proposed. Using the plane geometric properties of the aligned target frame, the depth map of the target frame is first reconstructed using the following formula:

[0089]

[0090] Among them, p is a coordinate of the target frame, is the corresponding homogeneous coordinate. Then, using the source image I s , the reconstructed depth map D s→t , the relative position of the camera is T t→s , camera internal parameter K synthesizes the target image

[0091]

[0092] The planar alignment constraint is defined as:

[0093]

[0094] Among them, L pe is the minimum photometric error, I t is the target image, is the target image synthesized by plane alignment. The minimum photometric error is calculated as follows:

[0095]

[0096] Among them, α = 0.85, SSIM(·,·) is the structural similarity, which is an indicator to measure the similarity between two images, and ‖·‖1 is the L1 norm.

[0097] However, the geometric properties of the planes generated from the depth are not always accurate in structureless regions, which can give L pa For large plane areas, even if the sampling points are slightly offset, they are still located on the same plane, thus generating the same or similar plane geometric properties. To this end, a selective plane alignment constraint is further proposed, which is selectively applied to large plane areas in the scene. The previous step has extracted the mask M plane To represent large planar regions with simple structures, we transform the mask M plane Apply to L pa , we get the selective plane alignment constraint L spa :

[0098] L spa =M plane ·L pa

[0099] This constraint selectively applies the plane alignment constraint to structured regions in corresponding scenes of different images, thereby avoiding the influence of unstructured regions. Since the target depth map is not reconstructed directly, the plane geometric properties are first recovered from adjacent frames, and then these geometric properties are recombined to reconstruct the depth of the target frame. In this process, as long as the geometric properties are accurate, even if there is a matching deviation, combining the geometric properties into depth will not lead to obvious depth errors, making this method more robust.

[0100] Step S5: For unstructured areas, apply the photometric loss guided by the spatial geometric structure, use the spatial geometric attributes predicted by the network to detect the unstructured areas, and suppress the photometric loss of these areas to reduce the impact of the errors in the unstructured areas on the global training.

[0101] Specifically, the depth uncertainty is modeled in unstructured areas, complex structure areas are assigned larger uncertainties, and areas with simple geometric structures are assigned smaller uncertainties; based on the uncertainty, a balanced photometric loss is obtained. First, the depth uncertainty in 3D space is modeled to obtain the uncertainty map U:

[0102] U=||σ x (N),σ y (N)||2

[0103] Where N is the normal map derived from the depth map, σ x and σ y are the gradients in the horizontal and vertical directions, respectively, which can represent the complexity of the local structure of the space, and ‖·‖2 is the L2 norm. The higher the gradient of the normal map, the more complex the structure, and the smaller the gradient, the simpler the structure. Under the guidance of the uncertainty map, the complex structure area is assigned a larger uncertainty, while the area with a simple geometric structure is assigned a smaller uncertainty. M plane represents a large planar region with simple structures, so the complex structure region mask M us It can be expressed as:

[0104] M us =1-M plane

[0105] Based on the structureless region mask M us And uncertainty map U, adjust the luminosity loss, balance the luminosity loss in high and low frequency areas, the improved luminosity loss for:

[0106]

[0107] Among them, e is a natural constant, λ3 is an adjustable parameter, and L pe is the original minimum photometric error.

[0108] Step S6: Perform self-supervised training on the depth prediction network and the pose prediction network to obtain a trained depth prediction network for subsequent monocular depth estimation.

[0109] The edge-aware smoothness loss is used to promote the smoothness of the local depth map while maintaining the image edges:

[0110]

[0111] where σ x and σ y are the horizontal and vertical gradients, respectively. is the normalized depth to avoid the reduction of predicted depth. The final loss function is as follows:

[0112]

[0113] β, γ, It is a hyperparameter used to control the trade-off between various losses. is the photometric loss caused by geometric uncertainty, L spa is the inter-frame plane alignment loss.

[0114] During the training process of the model, the Adam optimizer was used for optimization, and a total of 20 rounds of training were performed. The learning rate was 1e-4 for the first 15 rounds, and 1e-5 for the last 5 rounds. The batch size was set to 12.

[0115] The deep prediction network trained by the above method shows significant improvement in prediction accuracy and geometric structure recovery. In complex autonomous driving scenarios, the model can adapt to different driving environments, such as urban roads, highways, rural roads, etc., showing strong environmental adaptability. The trained model can more accurately restore the three-dimensional geometric structure of the scene in complex environments.

[0116] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A self-supervised depth estimation method based on spatial structure perception, characterized in that: The following steps are involved: Step S1: collecting and preprocessing training image data; the image data is specifically continuous frame image data taken by a camera with a fixed height; Step S2: construct an end-to-end depth prediction network and pose prediction network, and build a self-supervised training framework based on spatial structure perception; the depth prediction network includes an encoder and a decoder; the associated source frame and target frame images are respectively used as inputs to obtain depth features through the encoder; The deep feature input decoder outputs the target frame and the source frame depth map, and constructs the normal map and distance map of the spatial plane based on the depth map; the pose prediction network includes an encoder, which takes the source frame and the target frame image as input and predicts the relative camera pose from the target frame to the source frame; the image is reconstructed by combining the depth map and the camera pose, and the photometric error between the reconstructed image and the target image is used as a supervision signal to achieve self-supervised training; Step S3: extracting a large plane area using the normal map and the input image, thereby dividing the large plane area in the scene into structured areas, and dividing other areas into unstructured areas; Step S4: For the structured area, the relative camera pose from the target frame to the source frame obtained by the pose prediction network is used to establish the geometric relationship between the normal map and the distance map of the target frame and the source frame, and the normal map and the distance map of the target frame are reconstructed according to the normal map and the distance map of the source frame, and the plane alignment constraint between frames is further constructed; Combined with the structured region, a selective plane alignment constraint is obtained; Step S5: for the unstructured area, model the depth uncertainty, apply the minimum photometric error guided by the spatial geometric structure, and suppress the minimum photometric error in the unstructured area; Step S6: Perform self-supervised training on the depth prediction network and the pose prediction network to obtain a trained depth prediction network for subsequent monocular depth estimation.

2. The method for self-supervised depth estimation based on spatial structure perception according to claim 1, characterized in that: The step S3 is specifically as follows: The shooting height of all collected images is fixed, the y-axis of the camera coordinate system points downward, the x-axis points to the right, and the z-axis points forward, and the ground normal in the camera coordinate system is obtained. is (0,-1,0) T , T represents transpose; for outdoor images of size H×W, a binary mask M is used g ∈R H×W Mark pixels in an image that belong to the ground: Among them, λ1 is the similarity threshold, [·] is the Iverson bracket, if the normal at the pixel point p is the same as the ground normal The cosine similarity of is less than λ1, then in M g It is marked as 1, otherwise it is marked as 0; Then the low-frequency region is extracted as an approximation of the non-ground plane region, and the target image I is calculated by the following formula t The spatial frequency freq∈R H×W : freq=||σ x (I t ),s y (I t )||2 where σ x and σ y are the gradients in the horizontal and vertical directions respectively; extract the low-frequency component area as an approximation of the non-ground plane and use M f express: M f =[freq<λ2] Where λ2 is an adjustable threshold, combined with the ground area mask M g , get the mask M plane , representing a large planar region with a simple structure: in, Representation and operation; according to M plane , divide the scene into structured and unstructured areas.

3. The method for self-supervised depth estimation based on spatial structure perception according to claim 2, characterized in that: The step S4 is specifically as follows: The pose prediction network predicts the camera pose T from the target view to the source view t→s By the rotation matrix r t→s and the translation vector t t→s Composition, a 3D point P from the target perspective t Convert to the 3D point P corresponding to the source perspective through the following formula s : P s =T t→s P t =r t→s P t +t t→s The normal-distance constraint relationship between the source coordinate system and the target coordinate system is as follows: P s and P t Corresponding points in the source coordinate system and the target coordinate system, N s and d s They are P s Normal and distance to the plane, N t and d t They are P t The normal and distance of the plane, the superscript T indicates transposition; construct the association relationship between the geometric attributes of adjacent frames: Then the perspective alignment is performed through the camera pose T t→s =[r t→s ,t t→s ], the plane normal N in the source camera coordinate system s The distance d from the plane s Transform to the target camera coordinate system: in, represents the plane distance after alignment, Represents the aligned plane normals, and then a warping operation is performed to restore the normal map and distance map of the target frame: Among them, D t is the depth map of the target frame, T t→s is the relative position of the camera, K is the camera intrinsic parameter, <·> is a differentiable bilinear interpolation operation, and proj(·) is a coordinate projection operation that projects the coordinates of the target frame onto the corresponding points of the source frame; Using the aligned planar geometric properties of the target frame, the depth map D of the target frame is reconstructed by the following formula: s→t : Among them, p is a coordinate of the target frame, is the corresponding homogeneous coordinate; then synthesize the target image The planar alignment constraint is defined as: Among them, L pe is the minimum photometric error; The mask M plane Apply to L pa , we get the selective plane alignment constraint L spa : L spa =M plane ·L pa 。 4. The method for self-supervised depth estimation based on spatial structure perception according to claim 3, characterized in that: The step S5 is specifically as follows: First, we model the depth uncertainty in 3D space and obtain the uncertainty map U: U=||σ x (N),s y (N)||2 Where N is the normal map derived from the depth map, σ x and σ y are the gradients in the horizontal and vertical directions respectively, ‖·‖2 is the L2 norm; Complex structure region mask M us It is expressed as: M us =1-M plane Based on the structureless region mask M us and uncertainty map U, adjust the minimum photometric error, and improve the photometric loss for: Among them, e is a natural constant and λ3 is an adjustable parameter.

5. The method for self-supervised depth estimation based on spatial structure perception according to claim 4, characterized in that: The minimum photometric error calculation formula is as follows: Among them, α is a parameter, SSIM(·,·) is the structural similarity, and ‖·‖1 is the L1 norm.

6. The method for self-supervised depth estimation based on spatial structure perception according to claim 5, characterized in that: When performing the self-supervised training, the edge-aware smoothing loss is used as the loss function: Among them, L sm represents the edge-aware smoothing loss, is the normalized depth map of the target frame; the final loss function is as follows: Among them, β, γ, is a hyperparameter that controls the trade-off between various losses. is the improved luminosity loss L spa is a selective planar alignment constraint.

Images