Time-consistent monocular depth estimation method, model and electronic equipment

Abstract

The invention discloses a time-consistent monocular depth estimation method, a model and electronic equipment. Video frames are arranged into multiple rows according to a set mode to form a video frame map; the setting mode comprises that the number of video frames in each row is the same, and adjacent video frames have the same inner time sequence span; the first video frames in adjacent rows have equal outer time sequence spans, and the video frame numbers in the next row and the video frame numbers in the previous row are located in the same column; according to the novel image sequence space-time representation in the mode, only training data and reasoning data need to be subjected to data enhancement in the mode; through fine tuning, the visible field of view of the monocular depth model in the grid video frame becomes large, context information can be considered, and a depth map more conforming to a video scene can be obtained; a target video is input, namely, a depth map with consistent time is output in a rolling mode, and optical flow and camera pose information are not needed.

Description

Technical Field

[0001] The present invention relates to the technical field of monocular depth estimation, and in particular to a time-consistent monocular depth estimation method, model and electronic equipment. Background Art

[0002] During endoscopic examinations or surgeries, dynamic 3D reconstructions can help clinicians locate and identify target areas. Deep learning-based methods, with the help of large-scale labeled and unlabeled datasets, have achieved zero-sample monocular depth estimation in non-in vivo endoscopic scenarios.

[0003] Due to three characteristics of endoscopic scenes: sparse tissue surface texture, dynamic changes in lighting conditions, and complex motion patterns caused by instrument movement and tissue deformation, monocular depth estimation has poor results. For example, it is difficult to achieve temporal consistency, resulting in depth flicker between frames. Summary of the Invention

[0004] The purpose of the present invention is to solve the above problems and provide a time-consistent monocular depth estimation method to solve the temporal inconsistency problem encountered by the zero-sample pre-trained monocular depth estimation model during endoscopic examination or surgery, such as the flickering between depth map frames.

[0005] The present invention provides the following technical solutions: On the one hand, a temporally consistent monocular depth estimation method includes: Obtain and number the video frames of the target video, and arrange the video frames into multiple rows according to a set method to form a video frame map; The setting method includes the first video frames of adjacent rows having the same external timing span, the video frames of the next row having the same number as the video frames of the previous row being located in the same column; the external timing span being the time difference between the first video frames of adjacent rows; The number of video frames in each row is the same, and adjacent video frames have the same internal temporal span; the internal temporal span is the time difference between adjacent video frames; The same row of video frames is filled into an n*n grid to form multiple grids. The video in the grid is input into the preset monocular depth estimation model to obtain a depth map; The final depth map is obtained by averaging the depth maps of the video frames in the same column in the video frame map.

[0006] In a related embodiment, inputting the video in the grid into a preset monocular depth estimation model to obtain a depth map includes: Encoding stage: After preprocessing, local texture features and multi-scale global features are extracted from the video frames and spatially aligned to achieve the same resolution. In the top-down path, high-level semantic features are upsampled step by step and superimposed with the underlying local features to enhance the association between details and context. Meanwhile, in the bottom-up path, the fusion results are convolutionally optimized to form multi-level enhanced features. Multi-level enhanced features are adaptively fused through deformable convolution after deep attention mechanism, and weights are dynamically assigned according to scene content; Decoding stage: Deconvolution is performed on the low-resolution multi-scale global features, and after each level of upsampling, dynamic depth convolution and channel recalibration are performed in sequence to output the depth map.

[0007] In a related embodiment, outputting the depth map includes optimizing the depth discontinuity region by combining the RGB edge detection results; upsampling the optimized depth map to the original input image size by bilinear interpolation to generate a depth map output.

[0008] In a related embodiment, preprocessing the video frame includes scaling the input image to a set size using bicubic interpolation to eliminate illumination differences.

[0009] In a related embodiment, the outer timing span ranges from 1 to 4, corresponding to a 3-0 frame overlap; the inner timing span is less than or equal to 10.

[0010] In a related embodiment, the loss function of the preset monocular depth estimation model is:

[0011] in, is the true depth value, is the predicted depth, is the number of effective pixels.

[0012] In a second aspect, a time-consistent monocular depth estimation model is provided for use in the method, characterized in that it includes Multi-scale feature extractor, used to pre-process video frames, extract local texture features of pre-processed video frames and obtain multi-scale feature pyramids based on global spatial relationships; The depth-specific Transformer module is used to implement the bidirectional feature pyramid BiFPN for cross-scale feature interaction; the deep attention module DAM strengthens the depth clues in the spatial-channel dimension; The decoder gradually restores the resolution of the feature map through progressive upsampling: the low-resolution global features are deconvolved and amplified, and after each level of upsampling, dynamic depth convolution and channel recalibration are performed in sequence to output the depth map.

[0013] In a related embodiment, the depth-specific Transformer module includes a positional encoding: the formula is: ;

[0014] Where r is the distance from the current pixel to the center of the image, R is the maximum possible distance from the center of the image to the four corners, and π / 2 is the phase scaling factor. The purpose is to adjust the input range of the trigonometric function to [0, π / 2]. ⊕ concatenates the outputs of the sin and cos functions into a two-dimensional vector.

[0015] In a related embodiment, the lightweight decoder includes a progressive upsampling structure, a dynamic depth convolution block, a channel recalibration unit, and a multi-scale feature aggregation gate.

[0016] In a third aspect, an electronic device includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor is characterized in that it enables the processor to perform a time-consistent monocular depth estimation method.

[0017] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention arranges video frames into multiple rows according to a set method to form a video frame map; the set method includes that the number of video frames in each row is the same, and adjacent video frames have equal internal temporal spans; the first video frames in adjacent rows have equal external temporal spans, and the video frames in the next row with the same number as the video frames in the previous row are located in the same column; this method has a novel spatiotemporal representation of image sequences, and only requires data enhancement of the training data and inference data using our method; through fine-tuning, the visible field of view of the monocular depth model in the grid video frame becomes larger, and contextual information can be considered to obtain a depth map that is more consistent with the video scene; the input target video outputs a temporally consistent depth map in a rolling manner without the need for optical flow and camera pose information.

[0018] (2) The model in this invention still performs competitively in terms of depth scale consistency even when it has never seen C3VD data, such as data scenes, mirror withdrawal, and camera suspension, and also estimates high-quality depth maps on real colonoscopy sequences. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive work, among which: Figure 1 It is a schematic diagram of the process of the present invention.

[0020] Figure 2 This is a framework diagram of the method of the present invention.

[0021] Figure 3 Comparison chart of depth map sequence slices on C3VD.

[0022] Figure 4 A comparison chart of single-frame depth estimation on C3VD.

[0023] Figure 5 Comparison chart of depth map sequence slices on C3VD. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following Figures 1 to 5 The present invention is further described in detail. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] This paper proposes a time-consistent monocular depth estimation method that uses explicit spatiotemporal representation of images for fine-tuning pre-trained base models and deep reasoning. This method can inherently capture the spatiotemporal relationship between frames without requiring additional sequence, optical flow, or camera pose information; combined with Figure 1 The specific plan is as follows: S100, obtaining and numbering video frames of a target video, and arranging the video frames into multiple rows according to a set method to form a video frame map; The setting method includes that the first video frames of adjacent rows have the same external time sequence span, and the video frames of the next row have the same number as the video frames of the previous row and are located in the same column; The number of video frames in each row is the same, and adjacent video frames have equal internal temporal spans; For example, continuous video frames are arranged according to specific rules. The grid shape is used to capture the spatiotemporal correlation between frames. The specific rules are as follows: Grid Filling Rules

[0026] Fill the four positions of each grid with video frames in chronological order: top left -> top right -> bottom left -> bottom right ->. For example, the first four frames will occupy the four squares of the first grid, frames 5-8 will fill the second grid, and so on.

[0027] Internal timing span control within the grid (Dilation); Dilation=1: The four frames in each grid are consecutive frames in the original video, for example, grid 1 contains frames 1, 2, 3, and 4; Dilation=N: The frame interval in the grid is expanded to N frames, for example, when Dilation=2, grid 1 contains frames 1, 3, 5, and 7; External timing span control between grids (Stride) When Stride=1, the first frames of adjacent grids ("upper left frames") are continuous in the original video (for example, the first frame of grid 1 is frame 1, and the first frame of grid 2 is frame 2). When Stride=N, N frames are skipped between grids (for example, when Stride=2, the first frame of grid 1 is frame 1, and the first frame of grid 2 is frame 3).

[0028] In actual configuration, a hierarchical "outer-first, inner-later" approach is adopted. First, the outer temporal stride (Stride) is determined to construct the basic sampling framework for the video sequence rows. This parameter directly controls the amount of overlap between adjacent video frame rows (Stride values ​​of 1-4 correspond to 3-0 frame overlap). Next, the inner temporal span (Dilation) is adjusted to optimize the temporal correlation of the single-row sequence. By constraining the Dilation threshold to <= 10, the inter-frame span is ensured to be within the range of motion coherence that the model can perceive. Excessively large spans weaken the correlation of geometric context information, affecting model performance.

[0029] S200: Fill the same row of video frames into an n*n grid to form multiple grids, and input the video in the grid into a preset monocular depth estimation model to obtain a depth map; through fine-tuning, the grid video frames increase the model's visible field of view, and can consider contextual information during inference, thereby obtaining a depth map that better matches the video scene; Before inputting video frames into the pre-set monocular depth estimation model, the model is pre-trained. Using the previously described grid-filling rules, the publicly available simulated endoscopy datasets SimCol3D and UCL (abbreviated as S and U) are processed to generate new training datasets (abbreviated as S_d10s1 and U_d10s1), with dilation = 10 and stride = 1. Using S, U, S_d10s1, and U_d10s1 as training data, this base model is trained to obtain a fine-tuned model. The training loss function used is scale-invariant logarithmic loss:

[0030] in, is the true depth value, is the predicted depth, is the number of effective pixels.

[0031] The above is the pre-training of the preset monocular depth estimation model; the following is the specific process of inputting the video in the grid into the preset monocular depth estimation model to obtain the depth map: S210, preprocessing the video frame, extracting local texture features of the preprocessed video frame and obtaining a multi-scale feature pyramid based on global spatial relationships; 1) Input image preprocessing: a) Dynamic resolution adaptation: Use bicubic interpolation to scale the input image to (518, 518); b) Illumination normalization: Eliminate illumination differences using a differentiable histogram equalization (DHE) module; 2) Feature encoding: a) Stages 1-4 (CNN encoding): ResNet-50 extracts local texture features; b) Stages 5-8 (Transformer encoding): Swin-Transformer captures global spatial relationships and generates a multi-scale feature pyramid; S220, Bidirectional Feature Pyramid (BiFPN) for cross-scale feature interaction; Deep Attention Module (DAM) to enhance the depth clues of the spatial-channel dimension; 3) Deep feature fusion: a) Bidirectional Feature Pyramid (BiFPN) for cross-scale feature interaction b) Deep Attention Module (DAM) strengthens the depth clues of spatial-channel dimension In the deep feature fusion process, the high-resolution local texture features extracted by the CNN encoder are first spatially aligned with the multi-scale global feature pyramid generated by the Transformer encoder, and then unified to the same resolution through bilinear interpolation. Then a bidirectional cross-scale fusion strategy is adopted: In the top-down path, high-level semantic features are upsampled step by step and superimposed with the underlying local features to enhance the association between details and context; at the same time, in the bottom-up path, the fusion results are convolutionally optimized to form multi-level enhanced features.

[0032] The deep attention mechanism is then introduced to focus on deep mutation areas such as object edges in the spatial dimension, and dynamically respond to redundant features in the channel dimension.

[0033] Finally, deformable convolution is used to adaptively fuse features at each level and dynamically assign weights based on the scene content to achieve a balance between local texture fineness and global structural consistency, effectively improving the accuracy of depth estimation and edge clarity.

[0034] S230, after progressive upsampling, dynamic depth convolution (DDConv) refines features, and channel recalibration (CRU) enhances effective feature responses; output depth map; 4) Depth map decoding a) Progressive upsampling (4×→2×→2×→2×) b) After each level of upsampling, perform: i. Dynamic deep convolution (DDConv) refines features; ii. Channel recalibration (CRU) enhances effective characteristic response; During the decoding phase, the feature map's resolution is gradually restored through progressive upsampling. First, low-resolution global features are deconvolved and amplified. After each level of upsampling, dynamic depthwise convolution (DDConv) and channel recalibration (CRU) are performed sequentially. DDConv uses deformable convolution kernels to adaptively adjust the local receptive field, refining spatial features and suppressing artifacts in key areas such as object edges. CRU uses a channel-wise attention mechanism to enhance features related to geometric structure and mitigate invalid responses such as illumination interference. It also integrates shallower detail features corresponding to the encoder across layers to supplement texture information. Finally, a 1x1 convolution is performed to compress the image into a single-channel depth map.

[0035] 5) Post-processing optimization: Combine RGB edge detection results to optimize depth discontinuity areas Output generation: Upsample the optimized depth map (518, 518) to the original input image size using bilinear interpolation to generate a 32-bit floating-point depth map (depth value range 0mm-100mm). Each depth map contains four regions, corresponding to four consecutive frames of the video sequence in 3.3.1. S300, averaging the depth maps of the video frames in the same column in the video frame map to obtain a final depth map; obtaining depth maps with different context information by combining video frames in different columns in different time sequences, and after averaging, the relative deviation of the corresponding depth values ​​of adjacent frames in the target video sequence is smaller.

[0036] For example, the depth of a single frame of the original video corresponds to a certain area of ​​​​the 4 grids (such as Figure 2 To obtain the depth map of a frame in the original video, we need to: 1) locate the grid area in the depth map; 2) extract the depth data of the corresponding area; 3) calculate the arithmetic mean of the depth values ​​of the four areas.

[0037] For example, the final depth value of frame 4 = (depth of grid 1 at frame 4 + depth of grid 2 at frame 3 + depth of grid 3 at frame 2 + depth of grid 4 at frame 1) / 4.

[0038] This embodiment also includes a time-consistent monocular depth estimation model, which is as follows: Core architecture

[0039] 1) Model type: Transformer-CNN hybrid model 2) Backbone network: phased hybrid structure a) Stages 1-4: Improved ResNet-50 (Dilated-ResNet): Uses dilated convolution instead of downsampling layers to retain high resolution. Also introduces the ECA-Net channel attention mechanism. b) Stages 5-8: Swin-Transformer v2: The window size is gradually expanded.

[0040] Key components

[0041] Multi-scale feature extractor, used to pre-process video frames, extract local texture features of pre-processed video frames and obtain multi-scale feature pyramids based on global spatial relationships; 1) Multi-scale feature extractor a) Pyramid level: 4 levels (1 / 4, 1 / 8, 1 / 16, 1 / 32 resolution) b) Cross-level connections: Bidirectional Feature Pyramid Network (BiFPN) i. The weight fusion coefficient is dynamically adjusted through learnable parameters.

[0042] Depth-specific Transformer module for implementing bidirectional feature pyramid (BiFPN) for cross-scale feature interaction; Deep Attention Module (DAM) to enhance depth cues in the spatial-channel dimension; 2) Depth-specific Transformer module a) Positional encoding: Improved Radial Positional Encoding, where r is the distance from the pixel to the image center and R is the maximum radius; ; Where r is the distance from the current pixel to the center of the image, R is the maximum possible distance from the center of the image to the four corners, and π / 2 is the phase scaling factor. The purpose is to adjust the input range of the trigonometric function to [0, π / 2]. ⊕ concatenates the outputs of the sin and cos functions into a two-dimensional vector.

[0043] b) Deep Attention Mechanism: Using the Deep Prior Map as the Q Generator A lightweight decoder is used to implement progressive upsampling, dynamic depth convolution (DDConv) feature refinement, and channel recalibration (CRU) to enhance effective feature responses; it outputs a depth map.

[0044] 3) Lightweight decoder a) Progressive upsampling structure (step size) b) Dynamic Depthwise Convolution Block (DDConv) c) Channel Recalibration Unit (CRU) d) Multi-scale feature aggregation gate (MS-FAG) The effect of this embodiment is verified

[0045] The method was tested on the artificially synthesized endoscopy dataset C3VD. Figure 3 (a) shows the cecum_t3_a scene in C3VD, and (b) shows the trans_t4_b scene in C3VD. Compared to the two best-performing models (PPSNet in (a) and MonoLT in (b)), this example achieves better depth-time consistency and is closer to true depth sequence slices. Figure 4 Comparison of single-frame depth estimation between the method of the present invention and PPSNet and MonoLT on C3VD. Figure 5 The figure shows a comparison of single-frame depth estimation between the method of the present invention and PPSNet on real colonoscopy sequences (from the public real endoscopy dataset Colon10K).

[0046] This embodiment has the following advantages and features 1) Novel spatiotemporal representation of image sequences: By simply augmenting the training and inference data using our approach, we can output temporally consistent depth maps in a rolling manner without requiring optical flow and camera pose information; 2) Zero-shot estimation: The fine-tuned model maintains competitive performance in depth scale consistency even in scenarios where it has never seen C3VD data (including data scenarios, defocused scenes, and camera suspension). 3) Clinical practicality: High-quality depth maps were also estimated on real colonoscopy sequences.

[0047] This embodiment also includes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, steps of a time-consistent monocular depth estimation method are implemented.

[0048] The electronic device may be a desktop computer, a laptop, a PDA, a cloud server, or other electronic device. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the figures are merely examples of electronic devices and do not limit the scope of the electronic device. The electronic device may include more, fewer, or different components than shown.

[0049] The processor can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0050] Memory can be an internal storage unit of an electronic device, such as its hard drive or memory. It can also be an external storage device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card. Memory can also include both internal storage units and external storage devices. Memory is used to store computer programs and other programs and data required by the electronic device.

[0051] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a portion of code, and the module, program segment or a portion of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0052] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0053] If the functions are implemented as software modules and sold or used as standalone products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. It should be noted that, in this document, relational terms such as first and second, etc., are used solely to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. Furthermore, the terms "comprise," "include," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or device comprising a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. Without further limitation, the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article, or device comprising the elements.

[0054] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Various modifications and variations are readily apparent to those skilled in the art. Any modifications, equivalent substitutions, improvements, and the like made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention. It should be noted that similar reference numerals and letters denote similar items in the following figures. Therefore, once an item is defined in one figure, it need not be further defined or explained in subsequent figures.

[0055] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A temporally consistent monocular depth estimation method, characterized in that Obtain and number the video frames of the target video, and arrange the video frames into multiple rows according to a set method to form a video frame map; The setting method includes the first video frames of adjacent rows having the same external timing span, the video frames of the next row having the same number as the video frames of the previous row being located in the same column; the external timing span being the time difference between the first video frames of adjacent rows; The number of video frames in each row is the same, and adjacent video frames have the same internal temporal span; the internal temporal span is the time difference between adjacent video frames; The same row of video frames is filled into an n*n grid to form multiple grids. The video in the grid is input into the preset monocular depth estimation model to obtain a depth map; The final depth map is obtained by averaging the depth maps of the video frames in the same column in the video frame map.

2. The method according to claim 1, characterized in that Input the video in the grid into the preset monocular depth estimation model, and the depth map obtained includes: Encoding stage: After preprocessing, local texture features and multi-scale global features are extracted from the video frames and spatially aligned to achieve the same resolution. In the top-down path, high-level semantic features are upsampled step by step and superimposed with the underlying local features to enhance the association between details and context. Meanwhile, in the bottom-up path, the fusion results are convolutionally optimized to form multi-level enhanced features. Multi-level enhanced features are adaptively fused through deformable convolution after deep attention mechanism, and weights are dynamically assigned according to scene content; Decoding stage: Deconvolution is performed on the low-resolution multi-scale global features, and after each level of upsampling, dynamic depth convolution and channel recalibration are performed in sequence to output the depth map.

3. The method according to claim 1, characterized in that The output depth map includes optimizing the depth discontinuity area by combining the RGB edge detection results; upsampling the optimized depth map to the original input image size by bilinear interpolation to generate the depth map output.

4. The method according to claim 2, characterized in that The video frame preprocessing includes scaling the input image to a set size using bicubic interpolation to eliminate lighting differences.

5. The method according to claim 1, wherein The outer timing span range is 1-4, corresponding to a 3-0 frame overlap; the inner timing span is less than or equal to 10.

6. The method according to claim 1, characterized in that The loss function of the preset monocular depth estimation model is: in, is the true depth value, is the predicted depth, is the number of effective pixels.

7. A time-consistent monocular depth estimation model for implementing the method of any one of claims 1 to 5, characterized in that: include Multi-scale feature extractor, used to pre-process video frames, extract local texture features of pre-processed video frames and obtain multi-scale feature pyramids based on global spatial relationships; The depth-specific Transformer module is used to implement the bidirectional feature pyramid BiFPN for cross-scale feature interaction; the deep attention module DAM strengthens the depth clues in the spatial-channel dimension; The decoder gradually restores the resolution of the feature map through progressive upsampling: the low-resolution global features are deconvolved and amplified, and after each level of upsampling, dynamic depth convolution and channel recalibration are performed in sequence to output the depth map.

8. The model according to claim 1, characterized in that The depth-specific Transformer module includes positional encoding: the formula is: ； Where r is the distance from the current pixel to the center of the image, R is the maximum possible distance from the center of the image to the four corners, and π / 2 is the phase scaling factor, which adjusts the input range of the trigonometric function to [0, π / 2]. ⊕ concatenates the outputs of the sin and cos functions into a two-dimensional vector.

9. The model according to claim 1, characterized in that The decoder includes a progressive upsampling structure, dynamic depth convolution blocks, channel recalibration units, and multi-scale feature aggregation gates.

10. An electronic device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, The processor is enabled to execute the time-consistent monocular depth estimation method according to any one of claims 1 to 6.

Images