Endoscope monocular image depth estimation method and system

Abstract

The present application relates to a kind of endoscope monocular image depth estimation method and system, wherein, method includes: obtaining endoscope monocular current frame image;Endoscope monocular current frame image is used to DA-ICGA model to train endoscope monocular, the DA-ICGA model is based on DARES, including PoseNet module, DAM-LoRA module, also creatively introduce double-branch geometric perception module and image-level contrast learning module;Through the depth estimation of the DA-ICGA model trained to the endoscope monocular image to be detected.This application can effectively solve the artifact generated by overexposure area, and then improve monocular depth estimation precision.

Description

Technical Field

[0001] This invention relates to the field of endoscopic monocular image depth estimation technology, and in particular to an endoscopic monocular image depth estimation method and system. Background Technology

[0002] Robot-assisted endoscopic surgery (RAS) significantly improves the precision and flexibility of minimally invasive surgery. Its precise navigation and tissue reconstruction heavily rely on depth perception. Monocular image depth estimation, as a key technology for recovering depth information from a single endoscopic image, plays a crucial role in the surgical process. Traditional monocular image depth estimation methods mainly rely on illumination consistency constraints and geometric consistency constraints. However, in endoscopic scenarios, due to non-Lambertian surfaces and dynamic lighting changes, illumination consistency constraints easily fail in areas lacking rich texture, making it difficult for the model to learn effective depth information. Furthermore, methods relying on geometric constraints also face two challenges in this scenario: first, the lack of fixed scale information in a monocular view makes the depth estimation results prone to scale blurring; second, the presence of overexposed areas in endoscopic images makes feature-matching-based geometric constraints unstable, introducing depth blurring and artifacts. These factors ultimately prevent traditional methods from achieving satisfactory results. Deep learning-based methods have been extensively explored in the field of depth estimation for monocular endoscopic images. One of the most representative methods is self-supervised depth estimation based on optical flow, with AF-SfMLearner being a typical example. This type of method learns optical flow or appearance flow information between adjacent frames, using motion cues to infer the depth structure of the scene. Compared to traditional illumination consistency-based methods, optical flow-based deep learning methods not only focus on pixel motion but also consider brightness variations. However, due to the highly dynamic tissue surfaces and drastic illumination changes in endoscopic scenes, these methods are often susceptible to motion inconsistencies, leading to artifacts in depth estimation. Furthermore, optical flow calculation relies on pixel matching, which is difficult to stably match in overexposed areas, further reducing the accuracy and robustness of depth estimation. In recent years, the Depth Anything Model (DAM), the foundational model for monocular image depth estimation, has been continuously developed, achieving high-precision depth recovery in natural scenes. These methods rely on large-scale datasets for training, possessing advantages such as data-driven representation learning capabilities, strong generalization ability, and no need for manually designed features, thus reducing artifacts caused by reliance on illumination consistency to some extent. Researchers have introduced depth-any-model (DAM) technology into the field of endoscopy, such as Surgical-DINO, successfully expanding the sources and coverage of endoscopic data and providing more diverse scene information for model training. However, DAM is primarily based on data-driven deep learning and lacks geometric consistency constraints, resulting in significant depth estimation errors in endoscopic environments. Particularly in overexposed areas, specular reflections can destroy depth information, making it difficult for the model to accurately reconstruct 3D structures and introducing severe artifacts.In summary, although overexposure remains a common challenge for both types of methods, DAM is more robust than optical flow methods because it is pre-trained on large-scale, diverse deep datasets and learns depth biases in exposed areas more easily.

[0003] DARES, as one of the first studies to adapt the base model DAM V2 (DepthAnything Model V2) to the field of endoscopy through self-supervised learning without requiring a large amount of labeled data, successfully improved the accuracy of depth estimation and achieved state-of-the-art performance. DARES' innovation lies in its use of a pre-trained DAMV2 model and the introduction of a context-aware mechanism to enhance the modeling ability of depth features under complex lighting conditions, reducing depth artifacts caused by local lighting variations. Furthermore, DARES incorporates Low-Rank Adaptation (LoRA) to optimize model parameter allocation, reducing computational complexity and alleviating the extremely time-consuming nature of the base model DAM. It also reduces catastrophic forgetting, improves the stability and generalization ability of self-supervised learning, and alleviates the artifact problem to some extent. However, it still struggles to effectively suppress artifacts in overexposed areas, indicating that even a powerful pre-trained model and optimized self-supervised learning strategy cannot completely solve the artifact problems caused by lighting environments unique to endoscopic images, such as specular reflection, non-Lambertian reflection, and dynamic lighting changes. Analysis of the DARES network structure revealed that although DARES employs a context-aware mechanism, it primarily focuses on local photometric consistency, exhibiting shortcomings in global illumination adaptability. Furthermore, because DARES relies mainly on data-driven depth estimation and fails to explicitly model geometric information, the lack of geometric consistency constraints makes it difficult to accurately infer depth relationships within the exposed area, leading to depth artifacts at tissue boundaries. These issues collectively limit the accuracy and robustness of DARES in endoscopic monocular depth estimation, especially in clinical environments with complex lighting conditions.

[0004] In summary, although there has been a great deal of research on monocular image depth estimation, deep learning-based methods (such as DARES-based methods) still cannot effectively solve the artifact problem caused by overexposed areas, which limits the accuracy and robustness of DARES in endoscopic monocular depth estimation. Summary of the Invention

[0005] Therefore, the technical problem to be solved by the present invention is to overcome the fact that the existing deep learning-based methods still cannot effectively solve the artifacts generated in overexposed areas, which leads to low monocular depth estimation accuracy.

[0006] To address the aforementioned technical problems, this invention provides a method for depth estimation in monocular endoscopic images, comprising:

[0007] Acquire previous and next frame images of an endoscope;

[0008] The DA-ICGA model was trained on the endoscopic monocular system using front and rear frame images from the endoscope. The DA-ICGA model, based on DARES, includes a PoseNet module, a DAM-LoRA module, a two-branch geometric perception module, and an image-level contrastive learning module.

[0009] The DAM-LoRA module is used to process the images of the previous and next frames to generate four first depth maps with different resolutions;

[0010] The PoseNet module is used to process the images of the preceding and following frames, as well as the four first depth maps of different resolutions generated by the DAM-LoRA module, to obtain the synthesized adjacent frame images.

[0011] The first loss function is constructed based on the previous and next frame images of the single endoscope and the synthesized adjacent frame images;

[0012] The image-level contrast learning module is used to generate a second depth map based on images from previous and subsequent frames.

[0013] A second loss function is constructed based on the second depth map and four first depth maps at different resolutions;

[0014] The dual-branch geometry perception module is used to extract geometric priors and texture features from images of consecutive frames and generate a third depth map with geometric priors and texture features.

[0015] A third loss function is constructed based on the third depth map and four first depth maps at different resolutions;

[0016] If the first, second, and third loss functions all reach their respective preset values, the model converges and training ends; otherwise, training continues until the model converges.

[0017] Depth estimation is performed on the monocular image of the endoscope to be inspected using a trained DA-ICGA model.

[0018] In one embodiment of the present invention, the method by which the PoseNet module processes consecutive frame images and four first depth maps of different resolutions generated by the DAM-LoRA module to obtain a synthesized adjacent frame image includes:

[0019] The PoseNet module is used to input the images of the previous and next frames into the ResNet network to obtain camera pose information. The camera pose information is used as a transformation matrix, and the transformation matrix is ​​used to perform an inverse transformation operation on the four first depth maps of different resolutions output by the DAM-LoRA module to generate synthesized adjacent frame images.

[0020] In one embodiment of the present invention, the method by which the DAM-LoRA module processes consecutive frames to generate four first depth maps of different resolutions includes:

[0021] The DAM-LoRA module includes a ViT encoder and a DPT decoder. The ViT encoder is used to extract visual features from consecutive frames, and the DPT decoder is used to perform depth regression on the visual features extracted by the ViT encoder to generate four first depth maps at different resolutions.

[0022] The ViT encoder includes 12 LoRA Transformer modules. Each LoRA Transformer module is followed by an MLP layer and a normalization layer. The parameter allocation of each LoRA Transformer module is dynamically adjusted using a fine-tuning strategy through Vector-LoRA low-rank adaptive vector-LoRA.

[0023] The DPT decoder includes a Neck module and a Head module. The Neck module processes and integrates the features output by the ViT encoder. The Head module is used to perform depth regression on the features processed and integrated by the Neck module to generate four first depth maps with different resolutions.

[0024] In one embodiment of the present invention, the method by which the image-level contrast learning module generates a second depth map based on images from consecutive frames includes:

[0025] The image-level contrast learning module performs a binary Gaussian mask on each pixel in the preceding and following frames to generate image-level contrast samples. The formula for the binary Gaussian mask is as follows:

[0026]

[0027] Among them, M o(i,j) τ is the mask value for each pixel in the source image; M is the preset threshold; o σ is the occlusion mask; (i,j) are the pixels in the source image; (x0,y0) are the mean coordinates of the Gaussian distribution; σ is the standard deviation of the Gaussian distribution.

[0028] The image-level contrast samples are input into the depth estimation network to generate a second depth map.

[0029] In one embodiment of the present invention, the method for the dual-branch geometry perception module to extract geometric priors and texture features of consecutive frames and generate a third depth map with geometric priors and texture features includes:

[0030] The dual-branch geometry sensing module includes a first branch and a second branch;

[0031] The first branch is used to extract the geometric prior of the image, and the second branch is used to extract the texture features of the image through the FPN network;

[0032] The first branch includes a first convolutional layer, two first ResBlock layers, two second ResBlock layers, two third ResBlock layers, a first De-ResBlock layer, a second De-ResBlock layer, a third De-ResBlock layer, and a second convolutional layer connected in sequence, and the texture features of the image are fused with the outputs of the first De-ResBlock layer, the second De-ResBlock layer, and the third De-ResBlock layer, respectively.

[0033] In one embodiment of the present invention, the first ResBlock layer, the second ResBlock layer and the third ResBlock layer each include two 3x3 convolutions, and the outputs of the first 3x3 convolutions and the second 3x3 convolutions are added together, and then connected to a ReLU layer.

[0034] The first De-ResBlock layer, the second De-ResBlock layer, and the third De-ResBlock layer each consist of three 5x5 convolutions connected in sequence.

[0035] In one embodiment of the present invention, the first loss function is constructed based on the preceding and following frame images of a single endoscope and the synthesized adjacent frame images, and the formula is as follows:

[0036] L ms-reproj =α·(1-MS-SSIM(I) target ,I estimate ))+β·|I target -I estimate |

[0037]

[0038] Where MS_SSIM(x,y) is the multi-scale structural similarity index; M (x,y), c j (x,y) and s j (x, y) represent brightness, contrast, and structural similarity, respectively; α M βj and γ j These are the weights used to adjust the relative importance of different components; j is the scale index in the MS_SSIM(x,y) calculation, taking values ​​from 1 to M; M is the final scale in the MS_SSIM(x,y) calculation; α is the first weight parameter; β is the second weight parameter; I target The images are the front and back frames of the original endoscope; I estimate These are the synthesized adjacent frame images.

[0039] In one embodiment of the present invention, the second loss function is constructed based on the second depth map and four first depth maps of different resolutions, and the formula is as follows:

[0040]

[0041] Among them, D R and D IC These represent the four first depth maps output by the DAM-LoRA module and the second depth map output by the image-level contrast learning module, respectively; M c The confidence mask generated by the DAM-LoRA module in depth estimation.

[0042] In one embodiment of the present invention, the third loss function is constructed based on the third depth map and four first depth maps of different resolutions, and the formula is as follows:

[0043]

[0044] Wherein, γ represents the intersection of the depth intervals determined by the first depth map and the third depth map; The filtered depth distribution represents the third depth map generated by the dual-branch geometry sensing module; N GT (z) represents the distribution of the true depth, indicating the four first depth maps output by the DAM-LoRA module; m is the index of the discrete interval after dividing the scene depth space; z is the pixel in the image; M′ represents the division of the depth space of each scene into several discrete intervals.

[0045] To address the aforementioned technical problems, this invention provides an endoscopic monocular image depth estimation system, comprising:

[0046] Acquisition module: used to acquire previous and next frame images of an endoscope;

[0047] Training module: Used to train the DA-ICGA model for endoscope monocular imaging using front and rear frame images of the endoscope. The DA-ICGA model is based on DARES and includes a PoseNet module, a DAM-LoRA module, a two-branch geometric perception module, and an image-level contrastive learning module.

[0048] The DAM-LoRA module is used to process the images of the previous and next frames to generate four depth maps with different resolutions.

[0049] The PoseNet module is used to process the images of the preceding and following frames, as well as the four first depth maps of different resolutions generated by the DAM-LoRA module, to obtain the synthesized adjacent frame images.

[0050] The first loss function is constructed based on the previous and next frame images of the single endoscope and the synthesized adjacent frame images;

[0051] The image-level contrast learning module is used to generate image-level contrast samples based on images of previous and subsequent frames, and uses the image-level contrast samples as a second depth map.

[0052] A second loss function is constructed based on the second depth map and four first depth maps at different resolutions;

[0053] The dual-branch geometry perception module is used to extract geometric priors and texture features from images of consecutive frames and generate a third depth map with geometric priors and texture features.

[0054] A third loss function is constructed based on the third depth map and four first depth maps at different resolutions;

[0055] If the first, second, and third loss functions all reach their respective preset range values, the model converges and training ends; otherwise, training continues until the model converges.

[0056] Depth estimation module: Used to estimate the depth of the monocular image of the endoscope to be inspected using a trained DA-ICGA model.

[0057] The technical solution of the present invention has the following advantages compared with the prior art:

[0058] The endoscopic monocular image depth estimation method constructed in this invention can solve the common artifact problem in overexposed areas, thereby improving the accuracy and robustness of depth estimation;

[0059] This invention introduces a geometric perception module and an image-level contrast learning module into the DA-ICGA model based on DARES, enabling the DA-ICGA model to effectively correct texture loss and blurring caused by overexposure, and enhancing the depth estimation accuracy in high-contrast regions.

[0060] Experimental results verify that the method of the present invention can significantly improve the accuracy of depth estimation, especially in overexposed areas, where the depth estimation model exhibits good robustness and high reconstruction quality. Attached Figure Description

[0061] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0062] Figure 1 This is a flowchart of the method of the present invention;

[0063] Figure 2 This is an architecture diagram of the DA-ICGA model in an embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram comparing conventional samples and image-level comparison samples in an embodiment of the present invention;

[0065] Figure 4 This is a schematic diagram of the dual-branch geometry sensing module in an embodiment of the present invention;

[0066] Figure 5 This is a visual comparison diagram of the depth estimation results of the exposure area between the present invention and existing methods;

[0067] Figure 6 This is a visual comparison diagram of the generalization ability of the present invention and existing methods. Detailed Implementation

[0068] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0069] Example 1

[0070] Reference Figure 1 As shown, this invention relates to a method for depth estimation in monocular endoscopic images, comprising:

[0071] Acquire current and next frame images of an endoscope unit, wherein the current frame image and adjacent frame images are included;

[0072] The DA-ICGA model was trained on the endoscope monocular system using front and rear frame images from the endoscope. The DA-ICGA model, based on DARES, includes a PoseNet module, a DAM-LoRA module, a two-branch geometric perception module, and an image-level contrastive learning module.

[0073] The DAM-LoRA module is used to process the images of the previous and next frames to generate four first depth maps with different resolutions;

[0074] The PoseNet module is used to process the images of the preceding and following frames, as well as the four first depth maps of different resolutions generated by the DAM-LoRA module, to obtain the synthesized adjacent frame images.

[0075] The first loss function is constructed based on the previous and next frame images of the single endoscope and the synthesized adjacent frame images;

[0076] The image-level contrast learning module is used to generate a second depth map based on images from previous and subsequent frames.

[0077] A second loss function is constructed based on the second depth map and four first depth maps at different resolutions;

[0078] The dual-branch geometry perception module is used to extract geometric priors and texture features from images of consecutive frames and generate a third depth map with geometric priors and texture features.

[0079] A third loss function is constructed based on the third depth map and four first depth maps at different resolutions;

[0080] If the first, second, and third loss functions all reach their respective preset values, the model converges and training ends; otherwise, training continues until the model converges.

[0081] Depth estimation is performed on the monocular image of the endoscope to be inspected using a trained DA-ICGA model.

[0082] It should be noted that after the DA-ICGA model training is completed, when performing depth estimation on the actual endoscope monocular image (RGB image) to be detected, the DAM-LoRA module can directly generate a depth map and use it as the final result.

[0083] The following is a detailed description of this embodiment:

[0084] This embodiment addresses the artifact problem in overexposed areas of endoscopic images by proposing a DA-ICGA (Depth Anything with Image-level Contrastive and Geometry-aware) model to combat overexposure artifacts. The DA-ICGA model is based on DARES. DARES' innovation lies in addressing the issues of unreasonable parameter allocation and potential catastrophic forgetting when directly applying the base model DAM to endoscopic scenarios. It employs a novel adaptive technique—Vector Low-Rank Adaptation (Vector-LoRA)—to effectively optimize model parameter allocation. However, while DARES was one of the first methods to introduce DAM as a base model into the field of endoscopy, opening a new direction, it still did not solve the artifact problem in overexposed areas of endoscopic images. Therefore, this embodiment, based on the DARES framework, introduces an image-level contrastive learning module and a two-branch geometry-aware module. The architecture of the DA-ICGA model in this embodiment is as follows: Figure 2As shown, it includes the PoseNet module, the DAM-LoRA module, the dual-branch geometric perception module, and the image-level contrastive learning module. During model training, the images of the previous and next frames are input into the four modules.

[0085] The basic architecture of the DA-ICGA model is mainly composed of the PoseNet module for pose estimation from DARES and the DAM-LoRA module for depth estimation, which is self-supervised training using multi-scale SSIM reprojection loss. For pose estimation, the PoseNet module is responsible for calculating the camera's ego-motion, i.e., the change in camera pose between adjacent frames. This PoseNet module takes the current frame and adjacent frames as input, infers their relative pose, and uses this information for subsequent calculation of the first loss function. For depth estimation, the DAM-LoRA module, as the core component of the DA-ICGA model, is based on a Transformer structure and includes a ViT encoder for extracting image visual features (visual features include edges, textures, color distribution; as well as organ structures, tissue morphology, and spatial relationships between image regions) and a DPT decoder for dense prediction of feature-to-depth map transformation. To optimize model adaptability, this embodiment introduces the Vector-LoRA low-rank adaptive technique in the DAM-LoRA module to dynamically adjust the parameter allocation of the 12 multi-head self-attention blocks (LoRATransformer modules) of the ViT encoder. Unlike traditional LoRA, Vector-LoRA improves the model's adaptability to different feature levels by allocating parameters to different layers with variable rank. In the Transformer structure, each LoRA Transformer module includes Q (query), K (key), V (value), and Softmax normalization, aiming to establish correlations between input features through a self-attention mechanism, thereby enhancing the extraction capability of deep features. Furthermore, the DAM-LoRA module employs a fine-tuning strategy to adapt to endoscopic surgical scenarios, where parameters of certain layers are frozen to avoid updates. This approach effectively preserves general features while reducing computational overhead and mitigating the risk of overfitting. The first loss function (i.e., multi-scale SSIM reprojection loss) plays a core optimization role in the DA-ICGA model. This loss function optimizes depth and pose estimation by comparing the differences between the original and synthesized images. Unlike traditional L1 loss, multi-scale SSIM reprojection loss additionally considers image brightness, contrast, and structural information, better adapting to complex textures and lighting variations in endoscopic images, thus improving the model's robustness in real-world surgical environments.

[0086] The DA-ICGA model also includes an image-level contrastive learning module and a two-branch geometry perception module. Specifically, although the DARES model has made groundbreaking progress in addressing the problem of unreasonable monocular depth estimation parameter allocation in endoscopic scenes, its depth estimation performance remains unsatisfactory when facing artifacts caused by overexposed areas. To address this issue, this embodiment makes specific improvements to the DARES architecture by adding an image-level contrastive learning module and a two-branch geometry perception module. The introduction of the image-level contrastive learning module is mainly to compensate for DARES's insufficient utilization of global contextual information, thereby solving the problem of inaccurate depth estimation in overexposed areas. Previous models mainly relied on the photometric consistency assumption, but the lack of sufficient texture information in overexposed areas makes it difficult for the model to obtain effective supervision signals, thus affecting the accuracy of depth estimation. The image-level contrastive learning module uses Gaussian masks to generate image-level contrast samples. Although Gaussian masks lead to the loss of local texture information, global contextual information still exists, enabling the capture of depth changes in overexposed areas and providing important depth estimation cues for the model. By calculating the depth estimation difference between regular samples and contrast samples, L... ICC The Image-level Contrastive Consistency Loss (i.e., the second loss function) encourages the model to enhance its utilization of global contextual information during deep learning, resulting in more stable depth inference in overexposed regions. The dual-branch geometry-aware module enhances DARES's ability to learn geometric information, reducing artifacts in overexposed areas and improving the structural consistency of depth estimation. This module introduces two branches: one extracts image texture features, and the other extracts the image's geometric prior. The image texture features and the image's geometric prior are then added element-wise to further enrich the representation of geometric information. This module can better capture the details of geometric perception during depth estimation, thus significantly improving the model's performance in complex environments.

[0087] The DA-ICGA model will be described in detail below:

[0088] (I) Basic network structure of the DA-ICGA model (PoseNet module and DAM-LoRA module)

[0089] The DARES architecture, the foundation of the DA-ICGA model, includes a PoseNet module and a DAM-LoRA module. The PoseNet module is primarily used for pose estimation. During training, it works in conjunction with the DAM-LoRA module to provide crucial pose information for depth estimation, assisting in the generation of synthetic images. In the evaluation phase, it, along with the depth estimation results, provides fundamental data for applications such as 3D reconstruction, ensuring the model can accurately perceive the spatial relationships and motion states of the image. The DAM-LoRA module contains a ViT encoder and a DPT decoder. The ViT encoder comprises 12 LoRA Transformer modules, each followed by an MLP layer and a normalization layer. The parameter allocation of each LoRA Transformer module is dynamically adjusted using a fine-tuning strategy via Vector-LoRA low-rank adaptive transformation. The DPT decoder includes a Neck module and a Head module. The Neck module processes and integrates the features output by the ViT encoder. The Head module performs depth regression on the features processed and integrated by the Neck module to generate four first depth maps at different resolutions. The DPT decoder completes the depth regression task through a series of convolutional and upsampling layers in the Neck and Head modules, providing foundational data support for subsequent depth estimation. This embodiment adds Vector-LoRA (Vector Low-Rank Adaptation) layers to the 12 LoRA Transformer modules of the ViT encoder. Based on the hierarchical characteristics of the neural network, different ranks are assigned to different layers. This design enables the network to better adapt to the feature learning needs of different levels during the learning process. Early layers can use more parameters to learn general features, while later layers are allocated appropriate parameters for learning more targeted task details. This effectively improves the model's adaptability to specific tasks and learning efficiency, and avoids the drawbacks of uniform parameter distribution in the traditional LoRA method.

[0090] (1) PoseNet module

[0091] like Figure 2 As shown, the input to the PoseNet module (i.e., the pose estimation network) contains the current frame image Current(I) t ) and adjacent frame images Adjacent (I t+1Specifically, the PoseNet module processes consecutive frame images and four first depth maps of different resolutions generated by the DAM-LoRA module to obtain synthesized adjacent frame images. The specific method includes: the PoseNet module inputs consecutive frame images into a ResNet network to obtain camera pose information (CameraPose); using the camera pose information as a transformation matrix, it performs an inverse warping operation on the four first depth maps output by the DAM-LoRA module to generate the synthesized adjacent frame images I. t ′ +1 (Adjacent Syntheticframe).

[0092] (2) DAM-LoRA module

[0093] like Figure 2 As shown, the input portion of the DAM-LoRA module contains the current frame image Current(I t ) and adjacent frame images Adjacent (I t+1 The two work together to provide the model with image information from adjacent time points, improving the accuracy of depth estimation tasks by leveraging image correlations over time. The DAM-LoRA module contains a ViT encoder and a DPT decoder, where...

[0094] The DPT decoder includes a Neck module and a Head module. Both the Neck and Head modules include several convolutional and upsampling layers. The Neck module processes and integrates the features output by the ViT encoder. The Head module is used to perform depth regression on the features processed and integrated by the Neck module, and calculates and predicts four depth maps with different resolutions, providing depth information of different levels of detail to meet the needs of different application scenarios for depth information detail.

[0095] It should be noted that since the ViT encoder (including 12 LoRA Transformer modules) and DPT decoder (including Neck module and Head module) in the DAM-LoRA module are existing modules in the DARES model, that is, they belong to the prior art, their structural features will not be described in detail in this embodiment.

[0096] First loss function

[0097] In the field of self-supervised monocular depth estimation, most works typically employ a combination of structural similarity index (SSIM) and L1 loss to construct the reprojection error. The first loss function is used to calculate the similarity between images in terms of brightness, contrast, and structure. However, in scenarios such as robot-assisted surgery, there are special cases such as highly complex tissue textures, varying lighting conditions, and motion blur. Therefore, the DARES model proposes a multiscale SSIM (MS_SSIM) reprojection loss combined with a Tikhonov regularization term. The multiscale-SSIM reprojection loss measures the similarity between the original and synthesized images, while the Tikhonov regularization term prevents overfitting. During training, the model adjusts its parameters by minimizing this loss function. The two terms in the first loss function measure the difference between the estimated image and the target image from different perspectives. MS_SSIM performs multiscale processing on the image by iteratively applying filters and downsampling operations, with a downsampling factor of 2 each time. The calculation results at different scales are weighted and combined, as shown in Equation (1).

[0098]

[0099] Where MS_SSIM(x,y) is the multi-scale structural similarity index; M (x,y), c j (x,y) and s j (x, y) represent brightness, contrast, and structural similarity, respectively; α M β j and γ j is the weight used to adjust the relative importance of different components; j is the scale index in the MS_SSIM(x,y) calculation, with a value from 1 to M; M is the final scale in the MS_SSIM(x,y) calculation. Based on this, the multi-scale SSIM reprojection loss (i.e., the first loss function) is as shown in equation (2):

[0100] L ms-reproj =α·(1-MS-SSIM(I) target ,I estimate ))+β·|I target -I estimate | (2)

[0101] Where α is the first weight parameter in the multi-scale SSIM reprojection loss function, and its value is 0.9 after parameter tuning; β is the second weight parameter in the multi-scale SSIM reprojection loss function, and its value is 0.1 after parameter tuning; I target The images are the front and back frames of the original endoscope; I estimate For synthesized adjacent frame images; 1-MS-SSIM(Itarget ,I estimate The term ) prompts the model to improve the structural and content similarity of images, |I target -I estimate The | term further constrains the differences in pixel values, enabling the depth map generated by the model to more accurately reflect the true depth structure of the scene, reduce depth estimation errors, and improve the accuracy and robustness of depth estimation.

[0102] (II) Improved network structure of DA-ICGA model (image-level contrast learning module and dual-branch geometric perception module)

[0103] (1) Image-level contrast learning module

[0104] In monocular depth estimation in endoscopy, overexposed areas often lead to artifacts, primarily because these bright areas lack effective supervision signals, causing the local photometric consistency constraint to fail. Existing research attempts to mitigate this problem through adaptive exposure compensation mechanisms, typically employing dynamic image brightness adjustment to enhance the model's robustness in bright areas. To address this, this embodiment introduces an image-level contrastive learning module based on Gaussian masks. A Gaussian distribution mask is selected according to the characteristics of the exposed areas in the endoscopic image. By applying a local occlusion transformation to the image, the model focuses more on global contextual features when learning depth information, rather than solely relying on local photometric consistency. This module effectively alleviates artifact problems in overexposed areas and improves the accuracy and robustness of depth estimation.

[0105] Image-level contrast samples are obtained by comparing regular sample I i The image (i = 1, ..., N) is generated through a transformation operation. Specifically, an occlusion transformation is performed on the source image by applying a Gaussian distributed binary mask (binary masking is applied to both the preceding and following frames), thereby generating image-level contrast samples. Although these samples are similar in overall structure to the original regular samples, they differ in detail due to the introduction of occlusion. For a source image I of size H × W × C... target In this embodiment, a binary pixel mask M of size H×W is constructed based on a Gaussian distribution. oEach element in the mask is generated based on a probability value from a Gaussian distribution. For example, the mean is set to the image center (in practice, a pixel slightly off-center can be randomly selected from the brightest areas of the image), and the standard deviation is adjusted appropriately based on the scene characteristics (e.g., set to 1) to determine the probability that each pixel is occluded in the mask. During model training, regular samples provide complete information about the endoscope scene, while image-level contrast samples are used to construct masks and process the source images by setting the mean and standard deviation of a Gaussian distribution, thus simulating the failure of local photometric consistency in different regions. Inputting samples processed by the Gaussian distribution mask into the depth estimation network enables the model to learn how to use more contextual information for depth estimation, rather than relying solely on the photometric consistency of local regions, thereby enhancing the model's performance in overexposed areas. For a pixel (i,j) in the image, its value M in the mask... o(i,j) The calculation is shown in equation (3):

[0106]

[0107] Where τ is a threshold used to convert continuous Gaussian distribution values ​​into a binary mask, and M o This is represented as an occlusion mask; (i,j) are pixels in the source image; (x0,y0) are the mean coordinates of the Gaussian distribution, usually taken as the center of the image; σ is the standard deviation of the Gaussian distribution.

[0108] This embodiment uses M o The image to be processed is then subjected to a Gaussian distribution mask, such as... Figure 3 As shown, Figure 3 (a) in the sample is a standard sample. Figure 3 In Figure (b), the image-level contrast sample is generated after Gaussian distribution masking. This embodiment performs simple processing on all source images in this way to generate image-level contrast samples. The generated image-level contrast samples are then input into a depth estimation network (in this embodiment, the depth estimation network is the DAM-LoRA module) to obtain a second depth map.

[0109] Second loss function

[0110] In this embodiment, the images of the previous and next frames are processed by Gaussian distribution masking, and then passed through a depth estimation network to obtain a second depth map. The second depth map is then combined with the four first depth maps output by the DAM-LoRA module to construct a second loss function.

[0111] In monocular image depth estimation tasks, a confidence mask helps the model identify the more reliable pixel depth values. During the depth estimation process, the model generates a probability volume P. vIt measures the probability of each pixel under different depth hypotheses. The probability map P is obtained by summing the probabilities of the four closest depth hypotheses in the depth estimation. m Set a threshold γ and compare the probability map to the threshold. When P m When γ > γ, the pixel at the corresponding position is in the confidence mask M c A confidence mask is marked as 1, indicating that the model has a high confidence level in the depth estimate of that pixel, and vice versa. During model training, especially in contrastive learning, the confidence mask helps the model focus more on reliable regions. When calculating the loss function and performing subsequent processing, only pixels with a confidence value of 1 in the confidence mask are considered. This reduces the impact of inaccurate depth estimates on the overall result and improves the accuracy of depth estimation.

[0112] To address the issue of photometric consistency failure in overexposed areas, and to coordinate consistency between the DA-ICGA infrastructure and the image-level contrast learning module, this embodiment introduces a second loss function, namely the image-level contrast consistency loss L. ICC (Image-level Contrastive Consistency Loss), as shown in Equation (4):

[0113]

[0114] Among them, D R and D IC These represent the four first depth maps output by the DAM-LoRA module and the second depth map output by the image-level contrast learning module, respectively; M c The confidence mask generated by the DAM-LoRA module in depth estimation is used to filter out reliable pixel regions.

[0115] The role of image-level contrast consistency loss is to encourage the model to pay more attention to contextual information, thereby achieving more complete depth estimation in overexposed areas. By minimizing image-level contrast consistency loss, the model can learn to utilize more contextual information, rather than relying solely on the luminance consistency of small areas, thus enhancing the model's robustness and generalization ability in complex and overexposed areas.

[0116] (2) Dual-branch geometry sensing module

[0117] In endoscopic images, complex tissue structures and non-uniform illumination pose challenges to monocular depth estimation, and the lack of geometric information often leads to instability in depth estimation in high-contrast regions. Existing DARES models lack effective utilization of geometric information, especially in overexposed areas, where the model struggles to capture geometric cues and is prone to artifacts. To address this, this embodiment designs a dual-branch geometry perception module to enhance the model's ability to perceive scene geometry. The dual-branch geometry perception module comprises two branches: one branch extracts texture features from the image, and the other branch extracts and fuses prior geometric information. Through the synergistic effect of these two branches, the model can comprehensively utilize texture and geometric information during depth estimation, thereby improving the depth estimation accuracy in overexposed areas. Furthermore, this dual-branch geometry perception module optimizes the fusion of global and local features, enabling the model to better understand tissue structures, reduce texture loss due to overexposure, and thus improve the robustness of depth estimation in high-contrast regions. It should be noted that both the previous and next frame images must be input into the dual-branch geometry sensing module. The input consists of the current frame image (and adjacent frame images) and the corresponding depth image. This depth image is a coarse depth map selected from the four first depth maps output by the DAM-LoRA module, with the same size as the previous frame image (and adjacent frame image). The detailed architecture of the dual-branch geometry sensing module is as follows... Figure 4 As shown.

[0118] Figure 4 The first branch in the algorithm is used to extract the geometric prior of the image, and the second branch is used through the FPN network to extract the texture features of the image and fuse them with the geometric prior.

[0119] Specifically, the method for the dual-branch geometry-aware module to extract geometric priors and texture features from consecutive frames and generate a third depth map with geometric priors and texture features includes:

[0120] The dual-branch geometry sensing module includes a first branch and a second branch;

[0121] The first branch is used to extract the geometric prior of the image, and the second branch is used to extract the texture features of the image through the FPN network;

[0122] The first branch includes a first convolutional layer (a 5x5 convolution, followed by a BN layer and a ReLU layer), two first ResBlock layers, two second ResBlock layers, two third ResBlock layers, a first De-ResBlock layer, a second De-ResBlock layer, a third De-ResBlock layer, and a second convolutional layer (a 3x3 convolution, followed by a BN layer and a ReLU layer). The texture features of the image output by the FPN network are fused with the outputs of the first De-ResBlock layer and the second second ResBlock layer (element-wise addition) and then input into the second De-ResBlock layer. The texture features of the image are fused with the outputs of the second De-ResBlock layer and the second first ResBlock layer (element-wise addition) and then input into the third De-ResBlock layer. The texture features of the image are fused with the outputs of the third De-ResBlock layer and the first convolutional layer (element-wise addition) and then input into the second convolutional layer.

[0123] Preferably, the output of the second ResBlock layer is further processed by the SEnet module (Squeeze-and-Excitation module, an attention mechanism module for convolutional neural networks, mainly used to model the interdependencies between feature channels) and then performed as an inner product with the De-ResBlock layer. The main function of introducing the SEnet attention mechanism in this embodiment is to enhance the network's sensitivity to geometric priors and its representation ability by dynamically adjusting the feature weights.

[0124] The first, second, and third ResBlock layers support geometric feature fusion and downsampling. Each layer includes two 3x3 convolutions (the parameters of the 3x3 convolutions in the three ResBlock layers are different), and the outputs of the first and second 3x3 convolutions are added together before being connected to a ReLU layer. In this embodiment, the convolution parameters (e.g., stride, padding parameters, and activation functions after convolution) in the first, second, and third ResBlock layers can be adjusted as needed.

[0125] The first, second, and third De-ResBlock layers each consist of three sequentially connected 5x5 convolutions (the parameters of the 5x5 convolutions in the three De-ResBlock layers are different). In this embodiment, the parameters of the convolutions in the first, second, and third De-ResBlock layers (e.g., stride, padding parameters, and activation functions after convolution) can be adjusted as needed.

[0126] Third loss function

[0127] This embodiment constructs a third loss function based on a third depth map and four first depth maps of different resolutions generated by the DAM-LoRA module. Specifically, monocular depth estimation requires the model to have a good perception of the overall depth structure of the scene. The depth distribution similarity loss, by calculating the sample distribution similarity between the filtered depth estimate and the true depth, can guide the model to focus on the depth distribution of the entire scene, helping to enhance the model's understanding and estimation ability of depth in monocular situations. Therefore, this embodiment constructs a third loss function, namely the depth distribution similarity loss. dds (DepthDistribution Similarity Loss), calculated as shown in equation (7):

[0128]

[0129] Wherein, γ represents the intersection of the effective pixel set and the depth intervals determined by the image-level contrast learning module (i.e., the intersection of the depth intervals determined by the first depth map and the third depth map), and these depth intervals are used to filter out pixels that are meaningful for calculating the similarity of the depth distribution. The filtered depth distribution represents the third depth map generated by the dual-branch geometry sensing module; N GT (z) represents the true depth distribution, indicating the four first depth maps output by the DAM-LoRA module; m is the index of the discrete depth interval after dividing the scene depth space; z is a pixel in the image; M′ represents the division of the depth space of each scene into 48 discrete depth intervals. It should be noted that a depth interval is defined as the range of depth values ​​distributed in space. In short, depth intervals group objects in an image according to their distance, helping the computer more accurately determine "where is near and where is far," while eliminating interference and making the depth estimation more reliable. This third loss function calculates the sample distribution similarity between the filtered depth estimate and the true depth using the Kullback-Leibler divergence. and N GT The difference between (z) reflects the similarity between the depth distribution estimated by the model and the true depth distribution.

[0130] Experimental Analysis

[0131] This embodiment primarily demonstrates the selection of the dataset, the choice of evaluation criteria, the ablation experiments, and the comparative analysis of the experiments. The training experiments in this embodiment were conducted on a single 4090 server with a 128-core, 256-thread AMD EPYC 9754 CPU with a frequency of 2.25-3.1GHz, 32GB of RAM, and Ubuntu 18.04 running PyTorch 1.10. The inference process was performed on a single 4090 host with a 24-core, 32-thread Intel i9 13900K CPU with a base frequency of 3GHz, 64GB of RAM, and Ubuntu 20.04 running PyTorch 1.10.

[0132] Similar to the DARES model, this embodiment uses DAM V2 as the base model for depth estimation, and the encoder is a ViT encoder based on DINOv2, containing 12 multi-head self-attention blocks (i.e., LoRA Transformer modules). Vector-LoRA is introduced into each attention block of the ViT encoder, with a rank vector r = [14,14,12,12,10,10,8,8,8,8,8,8]. The PoseNet module is used for pose estimation in self-supervised training, taking neighboring frames as input to compute self-motion. The image-level contrastive learning module uses the same network as the regular branch for feature extraction, extracting features at different scales from the input image to provide rich feature information for subsequent volume construction and depth estimation. The dual-branch geometry perception module fuses the low-level features extracted by the first branch with the high-level features extracted by the second branch using the FPN network, further enhancing the expressive power of the features. The entire process was completed within 12 hours. For all sub-networks, the input resolution was 320×256 pixels, and the entire training plan was performed end-to-end.

[0133] This embodiment uses the Adam optimizer, a widely used optimization algorithm in deep learning known for its fast convergence. In this example, β1 = 0.9 and β2 = 0.99. The optimizer smooths the gradient in each iteration by setting parameter β1, making gradient changes more stable. The optimizer adaptively adjusts the learning rate of each parameter by setting parameter β2, helping the model to more effectively adjust the step size in different gradient change regions, thereby improving training speed and convergence. This embodiment also retains the initial learning rate of DARES, set to 0.0001, decaying every 10 steps for a total of 50 epochs.

[0134] Dataset selection

[0135] This experiment primarily uses an open-source endoscopy dataset for model training. This dataset contains real images of different parts of the digestive tract and corresponding high-precision depth data. To ensure experimental reliability, this embodiment divides the SCARED dataset into 15351 frames for training, 1705 frames for validation, and 551 frames for testing. Comparative tests with other monocular depth estimation models were conducted on the SERV-CT dataset. Furthermore, to further verify the model's generalization performance in cross-domain scenes, this embodiment introduces the natural scene depth estimation datasets Make3D and DIODE. These two datasets respectively cover complex outdoor terrain and mixed indoor / outdoor environments, effectively testing the model's performance under complex conditions such as strong light reflection and overexposure. A brief description of each dataset is as follows:

[0136] The SCARED dataset was collected from abdominal dissections of fresh pig carcasses and contains 35 endoscopic videos, providing ground-value data for depth maps, point clouds, and self-motion. In this dataset, the maximum range of the depth maps was scaled to 150 mm, covering the depth range of most of the digestive tract.

[0137] The SERV-CT dataset, extracted from pig torso cadavers, contains 16 pairs of stereo images with ground-value data for depth and parallax. In this dataset, the depth map is capped at 180 mm to ensure coverage of almost the entire effective depth range.

[0138] The Make3D dataset consists of RGB-D images of real-world outdoor scenes, covering various complex environments such as cities and natural landscapes. Its ground truth depth data is acquired using high-precision LiDAR and combined with multi-view geometry optimization to generate dense depth maps, exhibiting high spatial consistency and detail preservation. This dataset contains hundreds of image-depth pairs, covering scenes with different lighting conditions, scale variations, and complex geometries, making it suitable for validating the model's generalization performance in real-world open environments on texture-deficient regions and dynamically exposed areas.

[0139] The DIODE-Dataset dataset is the first monocular depth estimation benchmark dataset to simultaneously encompass diverse indoor and outdoor scenes. Its ground truth depth values ​​are obtained using high-resolution 3D laser scanners, covering complex architectural structures, dynamic lighting, and multi-scale object layouts. The dataset provides rigorously aligned RGB images with accurate depth information, including extreme scene variations from narrow indoor corridors to open outdoor plazas, and covers various weather conditions such as sunny and rainy days, effectively testing the model's adaptability across different environments.

[0140] Evaluation criteria selection

[0141] Similar to the work of DAERS, this embodiment uses standard evaluation metrics: Absolute Relative Error (Abs-Rel), Square Relative Error (Sq-Rel), Root Mean Squared Error (RMSE), Root Mean Squared Error log (RMSE-log), and accuracy (a1, a2, a3). In Table 1, d and d * Let represent the predicted depth value and the corresponding true value, respectively, and D represent a set of predicted depth values.

[0142] Table 1 Evaluation metrics for depth estimation

[0143]

[0144] The Abs-Rel error assesses the depth consistency of overexposed areas in the digestive tract by calculating the ratio of the absolute difference between predicted and true values. Its ability to eliminate scale differences allows it to objectively reflect the model's ability to suppress artifacts in both the near-field mucosal surface and the far-field cavities of the digestive tract. A decrease in the Abs-Rel value in overexposed areas indicates that the model has successfully suppressed specular reflection interference, improving prediction accuracy. Furthermore, Abs-Rel provides a uniform standard, ensuring stable accuracy across objects at different distances. The Sq-Rel error, through its squared amplification effect on large errors, specifically captures abrupt depth changes at the boundaries of overexposed areas. A decrease in the Sq-Rel value in highlight areas directly reflects a reduction in errors caused by artifacts, making it particularly suitable for evaluating depth prediction failures caused by brightness fluctuations in endoscopy. By optimizing the Sq-Rel, this embodiment can more effectively capture and adjust these large errors, ensuring model accuracy in critical areas, thereby optimizing model performance and improving prediction accuracy for larger depth deviations.

[0145] The Root Mean Square Error (RMSE) measures the overall error of a model by calculating the square root of the sum of the squares of the errors between the predicted and true depths. It is particularly sensitive to large errors, comprehensively reflecting the model's accuracy and revealing significant biases. RMSE is suitable for evaluating the overall performance of monocular image depth estimation models, especially in complex scenes. By comprehensively considering the square effect of errors, it accurately assesses the model's performance across different depth ranges. Optimizing RMSE helps improve prediction accuracy, ensures consistent model performance across depth intervals, and reduces the negative impact of large errors on the overall results. The Log-RMSE-log is similar to RMSE, but it takes the logarithm of the predicted and true values ​​before calculating the error. This approach focuses more on proportional errors and is suitable for evaluating the relative error of object distance in depth estimation; the smaller the value, the more accurate the predicted depth proportion.

[0146] Accuracy metrics represent the ratio of predicted depth values ​​to actual depth values ​​within a specific threshold (e.g., 1.25, 1.25). 2 1.25 3 The accuracy of the predicted depth value can be reflected by calculating the accuracy at different thresholds within the range of 100%. The higher the accuracy, the closer the predicted depth value is to the true value, and the better the model performs.

[0147] ablation experiment

[0148] To verify the improvement effect of the image-level contrast learning module (IC) and the bi-branch geometry perception module (GA) on artifact phenomena in endoscopic depth estimation, this embodiment verifies the effectiveness of the proposed method through four progressive ablation experiments ID1-ID4. ID1 represents the baseline model DARES. The addition of the image-level contrast branch in ID2 is used to determine whether image-level contrast can enhance context awareness. The geometry perception module in ID3 is used to prove whether geometric constraints can effectively suppress exposure area distortion. ID4 represents the simultaneous activation of both modules. Based on the baseline model DARES, the image-level contrast module and the geometry perception module are gradually introduced. The experimental results are shown in Table 2. The table shows the evaluation indicators under different module settings, providing quantitative support for experimental analysis. In Table 2, GA represents the bi-branch geometry perception module, and IC represents the image-level contrast learning module.

[0149] Table 2 Ablation Experiment

[0150]

[0151] Experimental results show that, compared to the original DARES model, using the image-level contrastive learning module alone improves the RMSE index through cross-sample contrastive learning, particularly in the depth continuity of the digestive tract structure edges. By maximizing the contrastive loss, it increases the similarity of depth distribution between normal and perturbed samples, thus forcing the network to focus on the overall pattern of the digestive tract structure rather than changes in local pixel brightness, thereby effectively improving the accuracy of depth estimation. On the other hand, introducing the dual-branch geometry perception module significantly reduces the presence of artifacts. This is because the geometric consistency constraint, through explicit alignment of projection errors between consecutive frames, forces the network to learn a depth gradient distribution that conforms to the physical imaging laws. The dual-branch geometry perception module, by introducing geometric cues, effectively suppresses the depth abrupt change problem caused by local texture loss, further improving the model's stability. When both modules are enabled simultaneously, experimental results show that most evaluation metrics are further improved, indicating a synergistic enhancement effect between the dual-branch geometric perception branch and the image-level contrastive learning module. However, this embodiment also notes that the RMSE metric in ID4 is lower than that in ID3 using only the GA module. This is because the shape and size of the Gaussian mask simulating the exposure area may not perfectly match the actual scene. The additional contrastive learning introduces redundant information or additional uncertainty, causing the model to incorrectly learn some depth information, thus slightly increasing the RMSE. Furthermore, adjusting the weights α and β of the multi-scale structural similarity reprojection loss can further optimize the model's performance. The model reaches its optimal balance when α = 0.9 and β = 0.1. At this point, both Abs-Rel and β reach their minimum values, accelerating the training convergence speed. From the visualization results, excessively large β leads to overly smooth details, while excessively small β fails to effectively suppress local artifacts. Therefore, choosing appropriate α and β values ​​is crucial for model performance during weight adjustment.

[0152] Comparative experiment

[0153] The model in this embodiment is compared and evaluated using typical models in the current field of monocular image depth estimation, including several classic monocular image depth estimation models for outdoor scenes such as DeFeat-Net, Fine-tuned DAM V2, and Monodepth2, as well as models that perform well in endoscopic monocular image depth estimation such as AF-SfMLearner, M-DASS, and DARES. DeFeat-Net proposes an innovative self-supervised representation learning model for monocular image depth estimation, utilizing multi-task learning to improve the accuracy of depth estimation and combining unsupervised learning to improve the model's performance in handling common scenes, especially demonstrating significant advantages in cross-dataset generalization. Fine-tuned DAM V2 is an optimized version of the original DAM model, representing the latest progress in monocular image depth estimation. Fine-tuned DAM V2 particularly emphasizes the model's cross-dataset generalization ability, which is crucial for monocular depth estimation. This improvement not only enhances the model's performance but also demonstrates stronger adaptability and stability when handling data from different sources. Monodepth2 has made many influential improvements in the field of self-supervised monocular image depth estimation, especially in its efficient estimation of depth and camera self-motion through photometric consistency loss and smoothing loss. It has become an important benchmark in the field of self-supervised learning, especially when evaluating the optimization of other self-supervised learning methods in depth estimation performance.

[0154] Unlike outdoor scenes, monocular image depth estimation in endoscopy typically faces more challenging environments, such as uneven illumination, motion blur, and limited viewpoint. AF-SfMLearner proposes using appearance flow to assist in depth and self-motion estimation. Appearance flow helps alleviate severe inter-frame brightness fluctuations and extracts more accurate depth, an innovation that enables AF-SfMLearner to achieve state-of-the-art results in endoscopic monocular image depth estimation. M-DASS addresses the challenges of gastrointestinal surface occlusion and semantic understanding difficulties in weakly textured gastrointestinal images under varying illumination by proposing a pseudo-label-based data augmentation generation method and a network feature clustering method, respectively. These methods improve the ability to better estimate depth on the gastrointestinal surface under occlusion and reduce the loss of weak texture information caused by illumination changes. DARES, as one of the first models to introduce the DAM basic model into endoscopic scenes, achieves state-of-the-art results in current endoscopic monocular depth estimation.

[0155] Table 3 shows the evaluation metrics for model experiments on the SCARED dataset.

[0156]

[0157] Table 3 shows the comparative results of each model tested using the SCARED dataset on various evaluation metrics. The results indicate that the model in this embodiment significantly outperforms other models across all evaluation metrics. Abs-Rel can measure the model's error in different depth regions, especially in environments with large brightness fluctuations caused by lighting changes and reflections. Previous models often suffered from unstable depth estimations due to artifacts. The model in this embodiment demonstrates higher stability in dealing with overexposed areas, effectively suppressing artifacts caused by strong reflections and lighting changes, thus improving the accuracy of depth prediction. Sq-Rel is highly sensitive to depth errors, especially under the influence of lighting changes and non-Lambertian reflections, and is easily affected by artifacts, leading to distorted depth predictions. Artifacts not only affect prediction accuracy but may also interfere with the training process. The model in this embodiment performs excellently in removing artifacts and handling high-noise environments, enabling more accurate depth estimation and ensuring the stability and reliability of the prediction results.

[0158] Table 4 presents the experimental results of evaluating each model using the SERV-CT dataset. From the evaluation metrics, the method in this embodiment outperforms the DARES model in several aspects. The Abs-Rel of the model in this embodiment is 0.081, significantly lower than DARES's 0.090, indicating that the model in this embodiment improves the accuracy of depth estimation. The RMSE value also decreased from 10.836 for DARES to 9.949, indicating that the model in this embodiment has better accuracy in global depth prediction than DARES and performs better in maintaining image structural integrity. Overall, the method in this embodiment surpasses the state-of-the-art endoscopic monocular depth estimation model DARES in all performance metrics, demonstrating its effectiveness and robustness in depth estimation tasks.

[0159] Table 4 Comparison of model evaluation metrics on the SERV-CT dataset

[0160]

[0161] This experiment aims to explore and compare commonly used monocular image depth estimation self-supervised models in the field of endoscopy. Therefore, ground truth depth maps are not listed, and the comparative experiment focuses on the cutting-edge DARES model. Figure 5 As shown, the qualitative comparison results clearly demonstrate the performance differences of each model in handling challenges such as overexposed areas. Compared with models such as AF-SfMLearner, M-DASS, and DARES, the DA-ICGA model proposed in this embodiment ( Figure 5The Ours algorithm (in this context, referring to the model's performance) has significant advantages in reducing artifacts, ensuring more accurate depth estimation results, which is crucial for improving spatial awareness and safety during surgery. Furthermore, analysis of quantitative indicators such as mean squared error and absolute relative error (Abs Rel) verifies the superior performance of the proposed model. Experimental results further demonstrate that the model in this embodiment exhibits excellent accuracy and robustness in depth estimation tasks, especially in complex endoscopic imaging environments, consistently outputting high-quality depth results. Through detailed experimental design and analysis, these results not only prove the effectiveness of the new model but also provide a new perspective for further development in this field.

[0162] To further verify the model's generalization ability in cross-domain scenarios, this embodiment directly applies the model trained on the SCARED dataset to the Make3D and DIODE-Dataset datasets in natural scenes for validation without any fine-tuning. Simultaneously, it compares the performance of models with excellent results in the endoscopic monocular depth estimation domain, such as AF-SfMLearner and M-DASS, with the state-of-the-art model DARES. The Make3D dataset covers complex outdoor terrain, while the DIODE-Dataset includes mixed indoor and outdoor environments. The experiment specifically selected images containing complex conditions such as strong light reflection and overexposure to test the model's performance in real-world environments. Experimental results are as follows: Figure 5 As shown, despite the fact that texture-deficient regions and dynamic exposure regions often severely interfere with depth estimation in these challenging scenarios, the model in this embodiment still demonstrates excellent generalization ability and can maintain high spatial consistency and detail preservation.

[0163] Specifically, in Figure 6 In the Make3D corridor scene in the first row, models such as AF-SfMLearner, M-DASS, and DARES exhibit depth jumps or artifacts on shadowed floors and blurred walls due to a lack of geometric constraints. In this embodiment, the DA-ICGA model ( Figure 6 Ours optimizes the global structural fusion of the ground and walls by applying geometric consistency constraints, while suppressing artifacts in exposed areas and significantly improving detail preservation. The model in this embodiment exhibits strong environmental adaptability under complex lighting conditions. Figure 6 Taking the vegetation area in the second row of Make3D as an example, other models suffer from depth planarization due to overfitting of local features. However, the model in this embodiment maintains spatial consistency in fine-grained areas such as leaf gaps through dual-module collaboration, and reduces estimation bias caused by missing local information in overexposed areas of manhole covers. Figure 6In the indoor study scene of the DIODE-Dataset in row 3, the other models suffer from disordered depth estimation of the bookshelf area due to a lack of geometric constraints. The DA-ICGA model effectively restores the geometric coherence of the edge structure while preserving the depth details of the ornaments in the overexposed bookshelf area. Figure 6 In the brightly lit areas of buildings in row 4, the DARES model failed to capture global contextual information, resulting in depth estimation artifacts. In contrast, the model in this embodiment maximizes the depth similarity between regular and contrasting samples, forcing the network to utilize broader contextual information and achieving continuous and complete depth reconstruction at the boundary between the sky and buildings. In summary, the model in this embodiment effectively mitigates the impact of artifacts under complex lighting conditions, providing stable depth estimation results. This further demonstrates the model's robustness in cross-domain applications, its ability to adapt to various environmental changes, and its strong environmental adaptability and practical application potential.

[0164] In summary, this embodiment can solve the artifact problem commonly found in overexposed areas, thereby improving the accuracy and robustness of depth estimation. By introducing a geometric perception module and an image-level contrast learning module, the network can effectively correct texture loss and blurring caused by overexposure, enhancing the depth estimation accuracy in high-contrast areas. Experimental results verify that the method proposed in this embodiment can significantly improve the accuracy of depth estimation, especially in overexposed areas, where the depth estimation model exhibits good robustness and high reconstruction quality.

[0165] Example 2

[0166] This embodiment provides an endoscopic monocular image depth estimation system, including:

[0167] Acquisition module: used to acquire previous and next frame images of an endoscope;

[0168] Training module: Used to train the DA-ICGA model for endoscope monocular imaging using front and rear frame images of the endoscope. The DA-ICGA model is based on DARES and includes a PoseNet module, a DAM-LoRA module, a two-branch geometric perception module, and an image-level contrastive learning module.

[0169] The DAM-LoRA module is used to process the images of the previous and next frames to generate four depth maps with different resolutions.

[0170] The PoseNet module is used to process the images of the preceding and following frames, as well as the four first depth maps of different resolutions generated by the DAM-LoRA module, to obtain the synthesized adjacent frame images.

[0171] The first loss function is constructed based on the previous and next frame images of the single endoscope and the synthesized adjacent frame images;

[0172] The image-level contrast learning module is used to generate image-level contrast samples based on images of previous and subsequent frames, and uses the image-level contrast samples as a second depth map.

[0173] A second loss function is constructed based on the second depth map and four first depth maps at different resolutions;

[0174] The dual-branch geometry perception module is used to extract geometric priors and texture features from images of consecutive frames and generate a third depth map with geometric priors and texture features.

[0175] A third loss function is constructed based on the third depth map and four first depth maps at different resolutions;

[0176] If the first, second, and third loss functions all reach their respective preset values, the model converges and training ends; otherwise, training continues until the model converges.

[0177] Depth estimation module: Used to estimate the depth of the monocular image of the endoscope to be inspected using a trained DA-ICGA model.

[0178] Example 3

[0179] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the endoscopic monocular image depth estimation method described in Embodiment 1.

[0180] Example 4

[0181] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the endoscopic monocular image depth estimation method described in Embodiment 1.

[0182] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0183] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0184] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0185] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0186] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0187] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A method for depth estimation in monocular endoscopic images, characterized in that: include: Acquire previous and next frame images of an endoscope; The DA-ICGA model was trained on the endoscopic monocular system using front and rear frame images from the endoscope. The DA-ICGA model, based on DARES, includes a PoseNet module, a DAM-LoRA module, a two-branch geometric perception module, and an image-level contrastive learning module. The DAM-LoRA module is used to process the images of the previous and next frames to generate four first depth maps with different resolutions; The PoseNet module is used to process the images of the preceding and following frames, as well as the four first depth maps of different resolutions generated by the DAM-LoRA module, to obtain the synthesized adjacent frame images. The first loss function is constructed based on the previous and next frame images of the single endoscope and the synthesized adjacent frame images; The image-level contrast learning module uses the following method to generate a second depth map based on images from consecutive frames: The image-level contrast learning module performs a binary Gaussian mask on each pixel in the preceding and following frames to generate image-level contrast samples. The formula for the binary Gaussian mask is as follows: ； in, This is the mask value for each pixel in the source image; The preset threshold; To mask out the code; The pixels in the source image; The coordinates are the mean coordinates of a Gaussian distribution; The standard deviation of the Gaussian distribution; The image-level contrast samples are input into the depth estimation network to generate a second depth map; A second loss function is constructed based on the second depth map and four first depth maps at different resolutions; The method for the dual-branch geometry perception module to extract geometric priors and texture features from consecutive frames and generate a third depth map with geometric priors and texture features includes: The dual-branch geometry sensing module includes a first branch and a second branch; The first branch is used to extract the geometric prior of the image, and the second branch is used to extract the texture features of the image through the FPN network; The first branch includes a first convolutional layer, two first ResBlock layers, two second ResBlock layers, two third ResBlock layers, a first De-ResBlock layer, a second De-ResBlock layer, a third De-ResBlock layer, and a second convolutional layer connected in sequence. The texture features of the image output by the FPN network are fused with the outputs of the first De-ResBlock layer and the second ResBlock layer before being input into the second De-ResBlock layer; the texture features of the image are fused with the outputs of the second De-ResBlock layer and the second first ResBlock layer before being input into the third De-ResBlock layer; and the texture features of the image are fused with the outputs of the third De-ResBlock layer and the first convolutional layer before being input into the second convolutional layer. A third loss function is constructed based on the third depth map and four first depth maps at different resolutions; If the first, second, and third loss functions all reach their respective preset values, the model converges and training ends; otherwise, training continues until the model converges. Depth estimation is performed on the monocular image of the endoscope to be inspected using a trained DA-ICGA model.

2. The endoscopic monocular image depth estimation method according to claim 1, characterized in that: The method by which the PoseNet module processes consecutive frame images and four first depth maps of different resolutions generated by the DAM-LoRA module to obtain synthesized adjacent frame images includes: The PoseNet module is used to input the images of the previous and next frames into the ResNet network to obtain camera pose information. The camera pose information is used as a transformation matrix, and the transformation matrix is ​​used to perform an inverse transformation operation on the four first depth maps of different resolutions output by the DAM-LoRA module to generate synthesized adjacent frame images.

3. The method for depth estimation of endoscopic monocular images according to claim 1, characterized in that: The method by which the DAM-LoRA module processes consecutive frames to generate four first depth maps at different resolutions includes: The DAM-LoRA module includes a ViT encoder and a DPT decoder. The ViT encoder is used to extract visual features from the images of the preceding and following frames, and the DPT decoder is used to perform depth regression on the visual features of the images of the preceding and following frames extracted by the ViT encoder to generate four first depth maps with different resolutions.

4. The method for depth estimation of endoscopic monocular images according to claim 1, characterized in that: The first ResBlock layer, the second ResBlock layer, and the third ResBlock layer each include two 3x3 convolutions, and the outputs of the first and second 3x3 convolutions are added together, and then connected to a ReLU layer. The first De-ResBlock layer, the second De-ResBlock layer, and the third De-ResBlock layer each consist of three 5x5 convolutions connected in sequence.

5. The method for depth estimation of endoscopic monocular images according to claim 1, characterized in that: The first loss function is constructed based on the preceding and following frames of the single endoscope image and the synthesized adjacent frame images, and the formula is: ； ； in, It is a multi-scale structural similarity index; , and These represent brightness, contrast, and structural similarity, respectively. , and These are weights used to adjust the relative importance of different components; for The scale index used in the calculation takes integer values ​​from 1 to M; for The final scale in the calculation; This is the first weight parameter; This is the second weighting parameter; These are the front and back frame images of the original endoscope. These are the synthesized images of adjacent frames.

6. The method for depth estimation of endoscopic monocular images according to claim 1, characterized in that: The second loss function is constructed based on the second depth map and four first depth maps at different resolutions, and the formula is as follows: ； in, and These represent the four first depth maps output by the DAM-LoRA module and the second depth map output by the image-level contrast learning module, respectively. The confidence mask generated by the DAM-LoRA module in depth estimation.

7. The method for depth estimation of endoscopic monocular images according to claim 1, characterized in that: The third loss function is constructed based on the third depth map and four first depth maps at different resolutions, and the formula is as follows: ； in, This represents the intersection of depth intervals determined by the first depth map and the third depth map; The filtered depth distribution represents the third depth map generated by the dual-branch geometry sensing module; The distribution of the true depth represents the four first depth maps output by the DAM-LoRA module; This is an index for discrete depth intervals after dividing the scene's depth space; For pixels in an image; This means that the depth space of each scene is divided into several discrete depth intervals.

8. An endoscopic monocular image depth estimation system, used to implement the endoscopic monocular image depth estimation method as described in any one of claims 1-7, characterized in that: include: Acquisition module: used to acquire previous and next frame images of an endoscope; Training module: Used to train the DA-ICGA model for endoscope monocular imaging using front and rear frame images of the endoscope. The DA-ICGA model is based on DARES and includes a PoseNet module, a DAM-LoRA module, a two-branch geometric perception module, and an image-level contrastive learning module. The DAM-LoRA module is used to process the images of the previous and next frames to generate four depth maps with different resolutions. The PoseNet module is used to process the images of the preceding and following frames, as well as the four first depth maps of different resolutions generated by the DAM-LoRA module, to obtain the synthesized adjacent frame images. The first loss function is constructed based on the previous and next frame images of the single endoscope and the synthesized adjacent frame images; The image-level contrast learning module is used to generate image-level contrast samples based on images of previous and subsequent frames, and uses the image-level contrast samples as a second depth map. A second loss function is constructed based on the second depth map and four first depth maps at different resolutions; The dual-branch geometry perception module is used to extract geometric priors and texture features from images of consecutive frames and generate a third depth map with geometric priors and texture features. A third loss function is constructed based on the third depth map and four first depth maps at different resolutions; If the first, second, and third loss functions all reach their respective preset values, the model converges and training ends; otherwise, training continues until the model converges. Depth estimation module: Used to estimate the depth of the monocular image of the endoscope to be inspected using a trained DA-ICGA model.

Images