Monocular depth map alignment method and device combining monocular depth estimation and normal

Abstract

The invention discloses a monocular depth map alignment method and device combining monocular depth estimation and a normal. Firstly, a normal vector correction monocular depth is generated through a normal vector basic model, and the scale and translation of a current frame are obtained. And then, through a depth alignment processing technology, the problem of inconsistency of multiple frames of depth maps in a video sequence in scale transformation and translation processes is solved, and the final scaling value of the current frame is determined. And finally, based on the final scaling value of the current frame and the scale and translation of the current frame, calculating the depth of the current frame to realize alignment with the reference frame. According to the method, depth estimation and normal calculation are combined, and more geometric information is introduced in the depth map alignment process, so that the alignment accuracy and robustness are improved; by optimizing the scale and translation parameters of the camera and combining the information of the depth image and the normal image, the relative motion of the camera can be estimated more accurately.

Description

Technical Field

[0001] This invention belongs to the field of monocular depth estimation and 3D reconstruction technology, specifically relating to a monocular depth map alignment method and apparatus that combines monocular depth estimation and normals. Background Technology

[0002] In monocular depth estimation and 3D reconstruction workflows, accurate alignment of depth maps is a crucial step in obtaining high-quality reconstruction results. Traditional alignment methods mainly rely on photometric or geometric consistency constraints, but they often fail in complex scenes due to changes in lighting and missing textures, leading to a significant increase in alignment errors.

[0003] In recent years, research on camera pose tracking and optimization based on deep learning models such as DROID-SLAM has made some progress. However, most of these methods are geared towards static scenes, and when directly applied to the geometric reconstruction of monocular depth maps, they struggle to handle depth jumps caused by dynamic objects. To address this limitation, this invention proposes a depth tracking method for dynamic scenes. Figure 1 The consistency correction scheme effectively suppresses abrupt changes in depth caused by moving objects, thereby significantly improving the accuracy of 3D reconstruction of dynamic regions. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and apparatus for monocular depth map alignment that combines monocular depth estimation and normal mapping. This invention relates to the related operations of depth estimation and normal calculation, and better adapts to different camera motion scenarios by optimizing the camera's scale and shift. Through optimized calculations of the depth image and normal image, the apparatus can estimate the relative motion (scale and displacement) of the camera and adjust subsequent image alignment and depth map generation processes accordingly.

[0005] In a first aspect, embodiments of this application provide a method for aligning monocular depth maps by combining monocular depth estimation and normals, the specific steps of which are as follows:

[0006] First, normal vectors are generated using the basic normal vector model to correct the monocular depth, thus obtaining the scale and translation of the current frame.

[0007] Then, depth alignment processing technology is used to solve the inconsistency problem of depth maps in multiple frames in a video sequence during scaling and translation, and to determine the final scaling value of the current frame. The depth alignment processing technology ensures the spatiotemporal consistency of the depth sequence through a triple mechanism of static region constraints, reference frame alignment, and outlier suppression.

[0008] Finally, based on the final scaling value of the current frame, as well as the scale and translation of the current frame, the depth of the current frame is calculated to achieve alignment with the reference frame.

[0009] In one possible implementation, monocular depth is corrected, specifically as follows:

[0010] The RGB image captured by the monocular camera is input into the base normal vector model. After generating point cloud normal vectors from the base normal vector model and the depth map output by the depth model, loss is calculated to obtain the scale (s) and translation (t) parameters for reconstructing the depth map geometry. The specific method is as follows:

[0011] First, initialize the Adam optimizer, setting the loss threshold, learning rate, and number of iterations. Calculate the cosine similarity between the point cloud normal vector generated from the current frame depth map output by the deep model and the normal vector generated by the base model. Use (1 - similarity) to form the loss. Backpropagate the current loss. After training, the Adam optimizer outputs the scale of the current frame k-depth map. ) and translation ( ).

[0012] In one possible implementation, the depth alignment processing technique specifically operates as follows:

[0013] When a dynamic frame first appears, its depth map is stored in a global cache as a reference base to establish a stable spatiotemporal reference system, ensuring that all subsequent frames are aligned to the same reference base. The specific operations are as follows: Extract the initial frame for mapping from the keyframes, using... This represents the monocular depth map corresponding to the initial frame. Add to the initialized list In the process, the scale of the initial frame depth map is calculated by the trained Adam optimizer. Peaceful relocation The negative translation is amplified by 1.1 times using scale compensation logic to form a reference translation for calculating the optimal scaling. To enhance the alignment of depth map edge regions, the specific formula is as follows:

[0014]

[0015]

[0016] in, This represents the reference depth after scaling and translation.

[0017] Using cumulative mask Indicates reference depth The static regions are used to form a static region depth map of the reference frame. .in This represents the mask of the current frame k, where This represents the mask of the reference frame.

[0018] After the static reference depth is calculated, the scale of the current frame depth map is calculated using the trained Adam optimizer. Peaceful relocation ,pass Calculate the pseudo-translation of the current frame, and then calculate the depth value of the current frame. Finally, the mask accumulated in the current frame. To output the static depth map of the static region in the current frame. .

[0019] Based on the static region depth map of the reference frame And the static region depth map of the current frame A linear regression model is established to solve for the optimal scaling value. The specific formula is as follows:

[0020]

[0021] in Find the optimal scaling value for the current frame k, and then combine the current frame k with the corresponding... The value is stored in a dictionary sn, and then the final scaling value is calculated using the following formula:

[0022]

[0023] in The optimal scaling for the reference frame is achieved, where transient interference is effectively filtered out by a 13.5% mutation threshold.

[0024] In one possible implementation, alignment with the reference frame is performed as follows:

[0025] First, based on the scale of the current frame. Peaceful relocation Obtain the depth data after preliminary transformation The calculation formula is as follows:

[0026]

[0027]

[0028] Check if the current frame k exists in the dictionary sn. If it does, further multiply the initially transformed depth data by the scale of the current frame. Peaceful relocation To achieve alignment with the reference frame, thus ensuring that different video frames are at the same relative scale, the calculation formula is as follows:

[0029] ,

[0030] in It is the final depth map after alignment.

[0031] Secondly, embodiments of this application provide a monocular depth map alignment device that combines monocular depth estimation and normals, comprising the following modules:

[0032] Monocular Depth Correction Module: Generates normal vectors based on the normal vector model to correct the monocular depth, thus obtaining the scale and translation of the current frame.

[0033] Scaling value calculation module: This module uses depth alignment technology to address inconsistencies in depth maps across multiple frames during scaling and translation in a video sequence, determining the final scaling value for the current frame. The depth alignment technology employs a triple mechanism of static region constraints, reference frame alignment, and outlier suppression to ensure the spatiotemporal consistency of the depth sequence.

[0034] Depth Alignment Module: Based on the final scaling value of the current frame obtained by the scaling value calculation module and the scale and translation of the current frame obtained by the monocular depth correction module, the depth of the current frame is calculated to achieve alignment with the reference frame.

[0035] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory;

[0036] The memory is used to store computer programs.

[0037] When the processor executes the program stored in the memory, it implements any of the monocular depth map alignment methods described in this application.

[0038] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the monocular depth map alignment methods described in this application.

[0039] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to execute any of the monocular depth map alignment methods described in this application.

[0040] The beneficial effects of this invention are as follows:

[0041] This invention utilizes existing consumer-grade RGB cameras, using color images as input, to design an effective depth alignment method under rapid camera movement. By combining normal vectors and depth information, the normal information can provide geometric constraints on the object surface, helping to better understand the 3D structure of the scene. By combining depth estimation with normal calculation, more geometric information can be introduced into the depth map alignment process, thereby improving the accuracy and robustness of the alignment.

[0042] By optimizing the camera's scale and shift parameters and combining information from the depth and normal images, the relative motion of the camera can be estimated more accurately. This method not only solves the scale uncertainty problem in monocular depth estimation but also utilizes normal information to make finer adjustments to the depth map, thereby achieving more accurate image alignment and depth map generation.

[0043] Furthermore, combining depth and normal information can reduce reliance on texture information to some extent, improving alignment performance in texture-poor scenes. By optimizing the matching relationship between depth and normal images, issues such as lighting variations and shadows can be better handled, further improving the accuracy and robustness of 3D reconstruction.

[0044] By optimizing the camera's scale and translation parameters and combining information from the depth image and normal image, the alignment accuracy of the depth map can be effectively improved, enabling more accurate 3D dynamic region reconstruction under a monocular camera. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0046] Figure 1 This shows the point cloud conditions of various depth maps in the embodiments of the present invention.

[0047] Figure 2 This is a flowchart of the method according to an embodiment of the present invention.

[0048] Figure 3 This is a schematic diagram illustrating the calculation process of scale and translation parameters for a single depth map according to an embodiment of the present invention.

[0049] Figure 4 This is a comparison chart of the dynamic scene experimental results of this invention and other methods. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0051] This application provides a method for aligning monocular depth maps by combining monocular depth estimation and normals, such as... Figure 2 As shown, the specific steps are as follows:

[0052] Step 1: Generate normal vectors from the basic normal vector model to correct the monocular depth.

[0053] In this embodiment, the basic model for normal vectors adopts the geowizard diffusion model, and the depth model adopts the depth_anythingv2 model. The RGB image captured by the monocular camera is input into the geowizard diffusion model. After generating point cloud normal vectors through the normal vectors generated by the geowizard diffusion model and the depth map output by the depth_anythingv2 model, loss is calculated to obtain the scale (s) and translation (t) parameters for restoring the geometry of the depth map, so that the depth image can better match the shape changes caused by camera motion. The specific method is as follows: Initialize the Adam optimizer using 'params':s,'lr':1e-3 and 'params':t,'lr':1e-3, set the loss threshold to 100000, set the learning rate to 0.001, and iterate a maximum of 500 times. Calculate the cosine similarity between the point cloud normal vector generated from the current frame depth map output by the depth_anythingv2 model and the normal vector generated by the GeoWizard diffusion model. Form the loss using (1 - similarity). Perform backpropagation on the current loss, checking the loss change every 40 steps: |loss - loss threshold|. Terminate early when the loss change is less than 1e-5, and output the scale of the current frame's k-depth (…). ) and translation ( ).

[0054] First, initialize the Adam optimizer, setting the loss threshold, learning rate, and number of iterations. Calculate the cosine similarity between the point cloud normal vectors generated from the depth map output by the current frame's depth model and the normal vectors generated by the base model, and use (1 - similarity) to form the loss. Perform backpropagation on the current loss. After training, output the scale of the k-th depth map of the current frame through the Adam optimizer. ) and translation ( ).

[0055] Step 2: By employing a designed depth alignment processing technique, the inconsistency problem of depth maps across multiple frames in a video sequence during scale transformation and translation is resolved. This technique ensures the spatiotemporal consistency of the depth sequence through a triple mechanism of static region constraints, reference frame alignment, and outlier suppression.

[0056] When a dynamic frame first appears, its depth map is stored in a global cache as a reference base to establish a stable spatiotemporal reference system, ensuring that all subsequent frames are aligned to the same reference base. The specific operations are as follows: Extract the initial frame for mapping from the keyframes, using... This represents the monocular depth map corresponding to the initial frame. Add to the initialized list In the middle, take later When using depth as a reference, the list needs to be... The first element is extracted, and the scale of the initial frame depth map is calculated by the trained Adam optimizer. Peaceful relocation The negative translation is amplified by 1.1 times using scale compensation logic to form a reference translation for calculating the optimal scaling. To enhance the alignment of depth map edge regions, the specific formula is as follows:

[0057]

[0058]

[0059] in, This represents the reference depth after scaling and translation.

[0060] Using cumulative mask Indicates reference depth The static regions are used to form a static region depth map of the reference frame. .in This represents the mask of the current frame k, where This represents the mask of the reference frame. Note. This is a quantity that needs to be calculated, and it will affect the scaling calculation of the depth map.

[0061] After the static reference depth is calculated, the current frame scale is calculated using the trained Adam optimizer. Peaceful relocation ,pass Calculate the pseudo-translation of the current frame, and then calculate the depth value of the current frame. Finally, the mask accumulated in the current frame. To output the static depth map of the static region in the current frame. .

[0062] Based on the static region depth map of the reference frame And the static region depth map of the current frame A linear regression model is established to solve for the optimal scaling value. The specific formula is as follows:

[0063]

[0064] in Find the optimal scaling value for the current frame k, and then combine the current frame k with the corresponding... The value is stored in a dictionary sn, and then the final scaling value is calculated using the following formula:

[0065]

[0066] in The optimal scaling for the reference frame is achieved, where transient interference is effectively filtered out by a 13.5% mutation threshold.

[0067] Step 3. Final scaling value based on the current frame. and the scale of the current frame Peaceful relocation Calculate the depth of the current frame.

[0068] First, based on the scale of the current frame. Peaceful relocation Obtain the depth data after preliminary transformation The calculation formula is as follows:

[0069]

[0070]

[0071] Check if the current frame k exists in the dictionary sn. If it does, further multiply the initially transformed depth data by the scale of the current frame. Peaceful relocation To achieve alignment with the reference frame, thus ensuring that different video frames are at the same relative scale, the calculation formula is as follows:

[0072] ,

[0073] in It is the final depth map after alignment.

[0074] Figure 4 This is a comparison chart of the experimental results of this invention with other methods in dynamic scenes. This invention upgrades "monocular depth estimation" from "relying solely on appearance" to a framework of "geometric prior + spatiotemporal consistency," achieving higher accuracy, stronger robustness, and cross-frame unified scale depth map alignment in fast-moving, dynamic objects, and weakly textured scenes. Specifically, it is demonstrated as follows:

[0075] 1. This invention introduces a "deep-normal joint prior" for scale-translation optimization. Existing methods, which rely solely on photometric or geometric consistency, have poor robustness to dynamic and weakly textured regions. This invention first generates reliable normals using the GeoWizard diffusion model, then uses normal-depth cosine similarity as a geometric constraint, and simultaneously solves for the scale s of each frame using the Adam optimizer. k pacing k .

[0076] 2. Triple Spatiotemporal Consistency Mechanism • Static Region Mask ▲M: Only static pixels participate in alignment, avoiding interference from dynamic objects. • Reference Frame Accumulation: Accumulate the depth d of the first frame. 11 As a global benchmark, subsequent frames use it as a standard, achieving a unified scale across frames. • 13.5% mutation threshold: at β k The estimation automatically removes transient anomalies to prevent drift.

[0077] 3. Negative translation enhancement strategy • For t k Frames with values ​​less than 0 are compensated with a 1.1x magnification to ensure that edge areas remain sharp after scaling and to reduce "point cloud streaking".

[0078] This application embodiment also provides a monocular depth map alignment device that combines monocular depth estimation and normals, including the following modules:

[0079] Monocular Depth Correction Module: Generates normal vectors based on the normal vector model to correct the monocular depth, thus obtaining the scale and translation of the current frame.

[0080] Scaling value calculation module: This module uses depth alignment technology to address inconsistencies in depth maps across multiple frames during scaling and translation in a video sequence, determining the final scaling value for the current frame. The depth alignment technology employs a triple mechanism of static region constraints, reference frame alignment, and outlier suppression to ensure the spatiotemporal consistency of the depth sequence.

[0081] Depth Alignment Module: Based on the final scaling value of the current frame obtained by the scaling value calculation module and the scale and translation of the current frame obtained by the monocular depth correction module, the depth of the current frame is calculated to achieve alignment with the reference frame.

[0082] In one possible implementation, the monocular depth correction module operates as follows:

[0083] The RGB image captured by the monocular camera is input into the base normal vector model. After generating point cloud normal vectors from the base normal vector model and the depth map output by the depth model, loss is calculated to obtain the scale (s) and translation (t) parameters for reconstructing the depth map geometry. The specific method is as follows:

[0084] First, initialize the Adam optimizer, setting the loss threshold, learning rate, and number of iterations. Calculate the cosine similarity between the point cloud normal vector generated from the current frame depth map output by the deep model and the normal vector generated by the base model. Use (1 - similarity) to form the loss. Backpropagate the current loss. After training, the Adam optimizer outputs the scale of the current frame k-depth map. ) and translation ( ).

[0085] In one possible implementation, the scaling value calculation module operates as follows:

[0086] When a dynamic frame first appears, its depth map is stored in a global cache as a reference base to establish a stable spatiotemporal reference system, ensuring that all subsequent frames are aligned to the same reference base. The specific operations are as follows: Extract the initial frame for mapping from the keyframes, using... This represents the monocular depth map corresponding to the initial frame. Add to the initialized list In the process, the scale of the initial frame depth map is calculated by the trained Adam optimizer. Peaceful relocation The negative translation is amplified by 1.1 times using scale compensation logic to form a reference translation for calculating the optimal scaling. To enhance the alignment of depth map edge regions, the specific formula is as follows:

[0087]

[0088]

[0089] in, This represents the reference depth after scaling and translation.

[0090] Using cumulative mask Indicates reference depth The static regions are used to form a static region depth map of the reference frame. .in This represents the mask of the current frame k, where This represents the mask of the reference frame.

[0091] After the static reference depth is calculated, the scale of the current frame depth map is calculated using the trained Adam optimizer. Peaceful relocation ,pass Calculate the pseudo-translation of the current frame, and then calculate the depth value of the current frame. Finally, the mask accumulated in the current frame. To output the static depth map of the static region in the current frame. .

[0092] Based on the static region depth map of the reference frame And the static region depth map of the current frame A linear regression model is established to solve for the optimal scaling value. The specific formula is as follows:

[0093]

[0094] in Find the optimal scaling value for the current frame k, and then combine the current frame k with the corresponding... The value is stored in a dictionary sn, and then the final scaling value is calculated using the following formula:

[0095]

[0096] in The optimal scaling for the reference frame is achieved, where transient interference is effectively filtered out by a 13.5% mutation threshold.

[0097] In one possible implementation, the depth alignment module operates as follows:

[0098] First, based on the scale of the current frame. Peaceful relocation Obtain the depth data after preliminary transformation The calculation formula is as follows:

[0099]

[0100]

[0101] Check if the current frame k exists in the dictionary sn. If it does, further multiply the initially transformed depth data by the scale of the current frame. Peaceful relocation To achieve alignment with the reference frame, thus ensuring that different video frames are at the same relative scale, the calculation formula is as follows:

[0102] ,

[0103] in It is the final depth map after alignment.

[0104] This application also provides an electronic device, which includes a processor and a memory.

[0105] The memory is used to store computer programs.

[0106] When the processor executes a program stored in the memory, it implements any of the methods described in this application.

[0107] In one possible implementation, the electronic device of this application embodiment further includes a communication interface and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0108] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc.

[0109] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0110] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0111] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0112] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements any of the methods described in this application.

[0113] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the methods described in this application.

[0114] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0115] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0116] The various embodiments in this specification are described in a related manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other.

[0117] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A method for aligning monocular depth maps by combining monocular depth estimation and normals, characterized in that, The specific steps are as follows: First, normal vectors are generated using the normal vector base model to correct the monocular depth, thus obtaining the scale and translation of the current frame; Then, through depth alignment processing technology, the inconsistency problem of depth maps of multiple frames in the video sequence during scale transformation and translation is solved, and the final scaling value of the current frame is determined. Depth alignment processing technology ensures the spatiotemporal consistency of depth sequences through a triple mechanism of static region constraints, reference frame alignment, and outlier suppression; Finally, based on the final scaling value of the current frame, as well as the scale and translation of the current frame, the depth of the current frame is calculated to achieve alignment with the reference frame.

2. The monocular depth map alignment method combining monocular depth estimation and normal mapping according to claim 1, characterized in that, To correct the monocular depth, the specific steps are as follows: The RGB image captured by a monocular camera is input into the basic normal vector model. After generating point cloud normal vectors using the normal vectors generated by the basic normal vector model and the depth map output by the depth model, loss is calculated to obtain the scale s and translation t parameters for restoring the geometry of the depth map. The specific method is as follows: First, initialize the Adam optimizer, setting the loss threshold, learning rate, and number of iterations. Calculate the cosine similarity between the point cloud normal vectors generated from the current frame depth map output by the deep model and the normal vectors generated by the base model. Use [1 - similarity] to form the loss. Backpropagate the current loss. After training, output the scale of the current frame k-depth map through the Adam optimizer. Peaceful relocation .

3. The monocular depth map alignment method combining monocular depth estimation and normal mapping according to claim 2, characterized in that, The depth alignment processing technique is implemented as follows: When a dynamic frame first appears, its depth map is stored in a global cache as a reference base to establish a stable spatiotemporal reference system, ensuring that all subsequent frames are aligned to the same reference base. The specific operations are as follows: Extract the initial frame for mapping from the keyframes, using... This represents the monocular depth map corresponding to the initial frame. Add to the initialized list In the process, the scale of the initial frame depth map is calculated by the trained Adam optimizer. Peaceful relocation The negative translation is amplified by 1.1 times using scale compensation logic to form a reference translation for calculating the optimal scaling. To enhance the alignment of depth map edge regions, the specific formula is as follows: ； ； in, Indicates the reference depth after scaling and translation; Using cumulative mask Indicates reference depth The static regions are used to form a static region depth map of the reference frame. ;in This represents the mask of the current frame k, where Indicates the mask of the reference frame; After the static reference depth is calculated, the scale of the current frame depth map is calculated using the trained Adam optimizer. Peaceful relocation ,pass Calculate the pseudo-translation of the current frame, and then calculate the depth value of the current frame. Finally, the mask accumulated in the current frame. To output the static depth map of the static region in the current frame. ; Based on the static region depth map of the reference frame And the static region depth map of the current frame A linear regression model is established to solve for the optimal scaling value. The specific formula is as follows: ； in Find the optimal scaling value for the current frame k, and then combine the current frame k with the corresponding... The value is stored in a dictionary sn, and then the final scaling value is calculated using the following formula: ； in The optimal scaling is for the reference frame, where transient interference is filtered out by a 13.5% mutation threshold.

4. The monocular depth map alignment method combining monocular depth estimation and normal mapping according to claim 3, characterized in that, Alignment with the reference frame is performed as follows: First, based on the scale of the current frame. Peaceful relocation Obtain the depth data after preliminary transformation The calculation formula is as follows: ； ； Check if the current frame k exists in the dictionary sn. If it does, further multiply the initially transformed depth data by the scale of the current frame. Peaceful relocation To achieve alignment with the reference frame, thus ensuring that different video frames are at the same relative scale, the calculation formula is as follows: , ； in It is the final depth map after alignment.

5. A monocular depth map alignment device combining monocular depth estimation and normal mapping, characterized in that, Includes the following modules: Monocular depth correction module: Generates normal vectors based on the normal vector model to correct the monocular depth, thus obtaining the scale and translation of the current frame; Scaling value calculation module: Through depth alignment processing technology, it solves the inconsistency problem of depth maps of multiple frames in a video sequence during scale transformation and translation, and determines the final scaling value of the current frame; Depth alignment processing technology ensures the spatiotemporal consistency of depth sequences through a triple mechanism of static region constraints, reference frame alignment, and outlier suppression; Depth Alignment Module: Based on the final scaling value of the current frame obtained by the scaling value calculation module and the scale and translation of the current frame obtained by the monocular depth correction module, the depth of the current frame is calculated to achieve alignment with the reference frame.

6. An electronic device, characterized in that, Including processor and memory; The memory is used to store computer programs; When the processor executes the program stored in the memory, it implements the monocular depth map alignment method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the monocular depth map alignment method according to any one of claims 1-4.

8. A computer program product containing instructions, characterized in that, When it is run on a computer, it causes the computer to perform the monocular depth map alignment method according to any one of claims 1-4.

Images