A monocular endoscopic scene scale perception reconstruction method and system

Abstract

The application discloses a monocular endoscope scene scale perception reconstruction method and system, and relates to the technical field of medical image processing and computer vision. A water jet reference line constraint model and a water jet scale mapping relationship model are constructed by pre-calibration with a water jet stagnation point as a physical measurement benchmark, and an image captured by an endoscope camera is subjected to three-dimensional reconstruction. The three-dimensional reconstruction result is combined with the coordinate correction of the water jet stagnation point and the global space consistency positioning to obtain an absolute scale benchmark. The absolute scale benchmark is used to solve a decoupling scale factor to realize three-dimensional scene measurement reconstruction. The method realizes measurement surgery scene reconstruction with certain accuracy and robustness, and can be widely applied to anatomical structure measurement, lesion quantitative evaluation and accurate positioning in surgery robot navigation.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer vision technology, specifically to a method and system for scale perception and reconstruction of a monocular endoscope scene. Background Technology

[0002] Monocular endoscopy 3D reconstruction technology plays a crucial role in minimally invasive surgery, enabling surgeons to more intuitively observe lesions and providing geometric support for surgical planning and robotic automation. However, due to the inherent scale ambiguity in monocular camera imaging, traditional reconstruction algorithms can only recover relatively geometric structures that are "proportionally consistent" but lack absolute physical dimensions. This lack of physical scale severely limits the clinical application value of endoscopic technology: First, in clinical diagnosis, the precise size of lesions (such as polyps and ulcers) is crucial for determining disease staging and selecting treatment options; blind visual assessment often leads to misdiagnosis or biased surgical decisions. Second, for surgical robots, the lack of absolute scale means that it is impossible to establish true distance feedback between the robot's end effector and the tissue, easily causing operational overshoot and serious complications such as collisions or tissue perforation.

[0003] Currently, approaches to solving the problem of real-scale perception in monocular endoscopy mainly fall into three categories. The first category relies on external hardware sensors, such as integrated LiDAR, electromagnetic tracking, structured light, or robotic kinematic feedback. While these solutions can theoretically provide absolute scale, they are limited by the extremely confined end-effector space, making sensor miniaturization difficult. Furthermore, hardware modifications significantly increase equipment costs, disrupt existing clinical surgical procedures, and increase the risk of cross-infection. The second category is based on metric deep learning, using massive datasets of known scales to train deep neural networks, enabling the model to learn to "guess" depth and scale from image texture. Although this method requires no additional hardware, it exhibits vulnerability in complex clinical environments: human anatomy varies greatly, and when the model encounters special cases not covered in the training set or changes in lighting, severe domain shift occurs, leading to uncontrollable distortions in scale prediction. In medical settings, this uncertain "black-box prediction" poses extremely high safety risks. The third category treats the auxiliary water jet as a constant-width ruler, scaling the polyp detection bounding box to millimeters by estimating the pixel width of the jet. However, this method is limited to the measurement of two-dimensional images, fundamentally ignoring the projection distortion caused by depth changes, and fundamentally limiting the propagation of measurement information across multiple frames. In summary, existing technical solutions struggle to achieve a balance between low cost, maintaining surgical procedures, and high reliability and physical determinism.

[0004] Meanwhile, the auxiliary water pumps widely equipped on endoscopes (used to flush blood and mucus from the field of view) create a physically stable stagnation point on the tissue surface during operation. While this phenomenon is extremely common in clinical practice, it has not previously been effectively translated into a physical metric anchor point in three-dimensional space for scale restoration, especially for three-dimensional scene restoration. Therefore, developing a reconstruction method that utilizes existing jet physics for scale perception and possesses cross-view robustness is a critical challenge that urgently needs to be addressed in current clinical applications. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a method and system for scale perception and reconstruction of a monocular endoscope scene. It aims to utilize the auxiliary water column built into the endoscope camera as a physical measurement anchor point. By establishing a precise water jet reference line model and a multi-view spatial consensus aggregation mechanism, it achieves multi-view 3D reconstruction capabilities while improving the accuracy of scene reconstruction. It provides both physically deterministic absolute scale references and effectively resists random noise interference under single-view conditions (such as dynamic blur, specular reflection, lens washing blur, etc.). Thus, without altering the clinical workflow, it achieves a certain degree of accuracy and robustness in metric surgical scene reconstruction, which can be widely applied to anatomical structure measurement, quantitative lesion assessment, and precise positioning in surgical robot navigation.

[0006] The technical solution of this invention is as follows: On the one hand, the present invention provides a method for scene scale perception and reconstruction using a monocular endoscope. Using the water jet retention point as a physical measurement benchmark, a water jet reference line constraint model and a water jet scale mapping relationship model are constructed through pre-calibration. The images captured by the endoscope camera are then reconstructed in three dimensions. The three-dimensional reconstruction results are combined with the coordinate correction of the water jet retention point and global spatial consistency positioning to obtain an absolute scale benchmark. The decoupling scale factor is calculated using the absolute scale benchmark to achieve three-dimensional scene perception and reconstruction.

[0007] Furthermore, the aforementioned method for scale perception reconstruction of a monocular endoscope scene includes the following steps: A water jet scale sensing model and a water jet reference line model are constructed, and water jet parameters in the water jet scale sensing model and the water jet reference line model are obtained by fitting. The water jet scale sensing model is used to describe the relationship between the true physical depth of the water jet retention point and the vertical pixel coordinates of the water jet retention point. The water jet reference line model is used to describe the water jet reference line. An endoscope camera is used to photograph lesions or target anatomical structures to obtain an observation frame sequence, a reference frame, a jet frame, and noisy two-dimensional pixel coordinates of water jet retention points in the jet frame; the observation frame sequence includes several observation frames taken from different angles under waterless jet conditions; the reference frame and the jet frame are images taken at a set observation position under waterless jet conditions and water-jet conditions, respectively. Reconstruct the observation frame sequence and the reference frame to obtain the local point map sequence and camera pose sequence corresponding to the observation frame sequence, as well as the local point map and camera pose corresponding to the reference frame. Linear correction is performed on the noisy two-dimensional pixel coordinates of the water jet retention point based on the water jet parameters in the water jet reference line model to obtain the noiseless pixel coordinates of the water jet retention point in the jet frame. The true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame are calculated by combining the water jet parameters in the water jet scale perception model as the measurement anchor point. The global spatial consensus position of the metric anchor point is calculated based on the noiseless pixel coordinates of the water jet retention point in the jet frame, the local point map sequence corresponding to the observation frame sequence, the camera pose sequence, and the local point map and camera pose corresponding to the reference frame. By using the noiseless pixel coordinates of the water jet retention point in the jet frame and the global spatial consensus position of the metric anchor point, the decoupling scale factor is calculated to realize the 3D scene perception reconstruction and obtain the 3D scene perception model.

[0008] Furthermore, the construction of the water jet scale sensing model and the water jet reference line model, and the obtaining of the water jet parameters in the water jet scale sensing model and the water jet reference line model through fitting, specifically includes the following steps: A1: Perform parameter calibration on the endoscope camera to obtain the intrinsic parameter matrix and distortion coefficient set of the endoscope camera; A2: Design of a nested multi-scale ArUco jet calibration plate; The nested multi-scale ArUco jet calibration plate has at least two nested ArUco control point arrays with different physical sizes printed on its surface. The physical size of the control points in the ArUco control point array decreases in a stepwise manner from the outside to the inside. Each ArUco control point array contains at least four ArUco control points. The nested printing refers to a planar layout in which a smaller physical size ArUco control point array is completely surrounded and nested inside a larger physical size ArUco control point array. A3: Control the endoscope camera to capture images of the nested multi-scale ArUco waterjet calibration plate at at least two different working distances. At each working distance, acquire image pairs, including a baseline image without water jets being ejected onto the nested multi-scale ArUco waterjet calibration plate and an image of the jet with water jets ejected onto the nested multi-scale ArUco waterjet calibration plate. Then, use the intrinsic parameter matrix... The distortion coefficient set is then used for distortion correction, and the extrinsic parameter matrix of the endoscope camera is calculated using the distorted reference image. Simultaneously, the two-dimensional pixel coordinates of the water jet retention point in the jet image are extracted. , Horizontal pixel coordinates Vertical pixel coordinates; A4: Based on the two-dimensional pixel coordinates of the water jet retention point in the jet image. Determine the true physical depth of the water jet's retention point. ; The true physical depth of the water jet's point of repose By solving the transformation equations, we obtain that the transformation equations are geometric projection equations for the transformation from the calibration plate coordinate system to the camera coordinate system: (1); in, The coordinates of the water jet retention point are given in the calibration plate coordinate system. The calibration plate coordinate system has the center point of the nested multi-scale ArUco water jet calibration plate as the origin, the horizontal rightward direction within the plane of the nested multi-scale ArUco water jet calibration plate as the X-axis, the vertical upward direction within the plane of the nested multi-scale ArUco water jet calibration plate as the Y-axis, and the outward direction perpendicular to the plane of the nested multi-scale ArUco water jet calibration plate as the Z-axis. The camera coordinate system has the camera optical center as the origin, the horizontal direction of the image as the X'-axis, the vertical downward direction of the image as the Y'-axis, and the direction of the camera's principal optical axis as the Z'-axis. A5: Establish a water jet scale sensing model and fit the parameters of the water jet scale sensing model. and , that is, the water jet parameters in the water jet scale sensing model; The water jet scale sensing model is as follows: (5); in, Let Z be the coordinate components of the water jet retention point in the camera coordinate system along the Z' axis. and These are the parameters of the water jet scale sensing model to be determined; A6: Establish a water jet reference line model and fit it to obtain the water jet reference line parameters. , and , that is, the water jet parameters in the water jet reference line model; (6); in, This is the reference line for the water jet.

[0009] Furthermore, the step of using an endoscopic camera to photograph the lesion or target anatomical structure to obtain the observation frame sequence, reference frame, jet frame, and noisy two-dimensional pixel coordinates of the water jet retention point in the jet frame specifically includes the following steps: B1: Using an endoscopic camera, multi-angle images of the lesion or target anatomical structure are taken and distortion is removed to obtain images of a length of [length missing]. Observation frame sequence , For the first in the observation frame sequence frame; B2: At the designated observation location, use an endoscopic camera to take fixed-angle images of the lesion or target anatomical structure and remove distortion, obtaining reference frames respectively. and jet frame ; Specifically: First, under normal conditions where the water jet is not activated, the endoscopic camera captures an image of the current field of view without water jet, and uses this as the reference frame. Subsequently, while maintaining the endoscopic camera's position, the auxiliary water pump is activated to spray water jets onto the surface of the lesion or target anatomical structure. Upon contact with the tissue surface, the water jets create a distinct landing point, or retention point. At this moment, the endoscopic camera immediately captures an image containing the water jet retention point, as the jet frame. ; B3: From the jet frame The noisy two-dimensional pixel coordinates of the water jet retention point are extracted as follows: .

[0010] Furthermore, the step of linearly correcting the noisy two-dimensional pixel coordinates of the water jet retention point based on the water jet parameters in the water jet reference line model to obtain the noiseless pixel coordinates of the water jet retention point in the jet frame, and then calculating the true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame using the water jet parameters in the water jet scale perception model as the measurement anchor point, specifically includes the following steps: C1: Orthogonally project the noisy 2D pixel coordinates of the water jet retention point onto the water jet reference line model to obtain the noise-free pixel coordinates of the water jet retention point in the corrected jet frame. ; (7); C2: Noise-free pixel coordinates of the water jet retention point in the corrected jet frame. Combining the water jet scale sensing model and intrinsic parameter matrix The true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame were calculated. ; (8); in, This is a transpose.

[0011] Furthermore, the step of calculating the global spatial consensus position of the metric anchor point based on the noiseless pixel coordinates of the water jet retention point in the jet frame, the local point map sequence corresponding to the observation frame sequence, the camera pose sequence, and the local point map and camera pose corresponding to the reference frame specifically includes the following steps: D1: Local dot plot corresponding to the reference frame In the middle, the noiseless pixel coordinates of the water jet retention point in the jet frame are used. Centered on parameters To find the maximum number of points, search for the nearest neighbors and form a set of points. Then, take the median of this set of points in each dimension to form the spatial median. Through camera pose sequence Transform it to the camera coordinate system of each observation frame to generate the first... The theoretical projection center from a different perspective ; (9); in, For the camera pose in the camera pose sequence for The inverse matrix; D2: Local point plots in each observation frame Sino-Israeli Theoretical Projection Center Centered on parameters To find the upper limit of the number of points, search for the nearest neighbors and construct a point set. Then, take the median of this point set in each dimension to form the spatial hypothetical location of the observation frame. And uniformly project back to the camera coordinate system of the reference frame to form a set of spatial hypotheses. , The assumed spatial position in the camera coordinate system of the reference frame; D3: On the set of spatial hypotheses Using a consistency evaluation method, abnormal spatial assumptions caused by single-view occlusion or water flow washing are removed, and a binary mask is generated for each observation frame view. D4: Using the binarized masks from each observation frame, data aggregation is performed on the spatially assumed locations to obtain the global spatial consensus location of the metric anchor point. ; The data aggregation method can be any statistical method that can calculate a certain trend from a set of data that can be used to assess the overall trend of the data.

[0012] Furthermore, the method for consistency assessment is as follows: First, calculate the spatial hypothesis set. Hypothetical locations in various spaces To the space hypothetical set of the centroid Euclidean distance as spatial deviation Then dynamically establish the interior point threshold. : (10); in, For the set of spatial deviations The third quartile, For the set of spatial deviations Interquartile range; Generate binarized masks from the perspective of each observation frame. : (11); in, This is an indicator function that takes the value 1 if and only if the condition within the parentheses is true, otherwise it takes the value 0.

[0013] Furthermore, calculate the global spatial consensus location. The formula is: (12).

[0014] Furthermore, the decoupling scale factor is calculated using the noiseless pixel coordinates of the water jet retention point in the jet frame and the global spatial consensus position of the metric anchor point to achieve perception reconstruction and obtain a three-dimensional scene perception model, specifically as follows: Calculate the decoupling scaling factor: (13); in, This is the decoupling scaling factor; Based on the binarization mask from the perspective of each observation frame All valid observation frames reconstructed into local point maps are selected, their unions are stacked, and the final stacked results are scaled according to the decoupling scale factor. Scaling is performed to generate a 3D scene perception model. .

[0015] On the other hand, the present invention also provides a monocular endoscope scene scale perception and reconstruction system for implementing a monocular endoscope scene scale perception and reconstruction method, comprising: The water jet parameter calibration module is used to construct a water jet scale sensing model and a water jet reference line model, and obtain the water jet parameters in the water jet scale sensing model and the water jet reference line model through fitting. The acquisition module is used to take pictures of lesions or target anatomical structures using an endoscope camera, and obtain the observation frame sequence, reference frame, jet frame, and noisy two-dimensional pixel coordinates of water jet retention points in the jet frame. The reconstruction module is used to reconstruct the observation frame sequence and the reference frame to obtain the local point map sequence and camera pose sequence corresponding to the observation frame sequence, as well as the local point map and camera pose corresponding to the reference frame. The metric anchor point calculation module is used to linearly correct the noisy two-dimensional pixel coordinates of the water jet retention point based on the water jet parameters in the water jet reference line model, obtain the noiseless pixel coordinates of the water jet retention point in the jet frame, and calculate the true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame as the metric anchor point by combining the water jet parameters in the water jet scale perception model. The global spatial consensus location acquisition module is used to calculate the global spatial consensus location of the metric anchor point based on the noiseless pixel coordinates of the water jet retention point in the jet frame, the local point map sequence corresponding to the observation frame sequence, the camera pose sequence, and the local point map and camera pose corresponding to the reference frame. The perception reconstruction module is used to calculate the decoupling scale factor by using the noiseless pixel coordinates of the water jet retention point in the jet frame and the global spatial consensus position of the metric anchor point, so as to realize perception reconstruction and obtain a three-dimensional scene perception model.

[0016] The beneficial effects of this invention are as follows: Using the water jet stagnation point as the physical measurement benchmark and combining it with the water jet scale perception model to calculate the measurement anchor point, the scene scale perception capability comes from real physical cues, improving the reliability and accuracy of scale estimation. By using a consistency assessment method to eliminate spatially hypothetical locations with anomalies and obtaining the global spatial consensus location of the measurement anchor point through data aggregation, noise interference can be effectively overcome, improving the robustness of reconstructed scale perception. Using only existing equipment in the operating room, without the need to introduce additional hardware sensors or additional integration with the endoscope, the risk of additional cross-infection is fundamentally eliminated, the cost of doctors learning new technologies is reduced, and the clinical universality of monocular endoscopic scene scale perception technology is improved. Attached Figure Description

[0017] Figure 1 This is a flowchart of the monocular endoscope scene scale perception and reconstruction method in an embodiment of the present invention; Figure 2 This is a schematic diagram of a nested multi-scale ArUco jet calibration plate in an embodiment of the present invention; Figure 3 This is a schematic diagram of the jet calibration operation process in an embodiment of the present invention; Figure 4 This is a schematic diagram of the jet calibration data acquisition results in an embodiment of the present invention; Figure 5 This is a schematic diagram of the scene perception data acquisition process in an embodiment of the present invention; Figure 6 This is a schematic diagram of the framework of the monocular endoscope scene scale perception and reconstruction method in an embodiment of the present invention; Figure 7 This is a schematic diagram of the multi-view scale information optimization process in an embodiment of the present invention; Among them, 201-nested multi-scale ArUco jet calibration plate; 301-endoscope camera; 302-water jet; 401-far field reference image; 402-far field jet image; 403-near field reference image; 404-near field jet image; 500-target anatomical structure; 501-observation frame example 1; 502-observation frame example 2; 503-reference frame; 504-jet frame. Detailed Implementation

[0018] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] Example 1: This embodiment provides a method for scale perception reconstruction of a scene using a monocular endoscope, such as... Figure 1 As shown, it includes the following steps: Step 1: Construct a water jet scale sensing model and a water jet reference line model. Obtain the water jet parameters in the water jet scale sensing model and the water jet reference line model through fitting, and realize the water jet parameter calibration. The water jet scale sensing model is used to describe the relationship between the true physical depth of the water jet retention point and the vertical pixel coordinates of the water jet retention point. The water jet reference line model is used to describe the water jet reference line. Specifically, the following steps are included: Step 1.1: Perform parameter calibration on the endoscope camera 301 and obtain the intrinsic parameter matrix of the endoscope camera 301. And the distortion coefficient set; In this embodiment, a camera calibration algorithm is used to calibrate the endoscope camera 301 to obtain the intrinsic parameter matrix of the endoscope camera 301. The camera calibration algorithm can be any camera calibration algorithm commonly used in the industry. Preferably, this embodiment uses the ChArUco camera calibration board, which has significant advantages in endoscopic scenarios with small field of view, easy obstruction, and partial visibility. The obtained calibration results (intrinsic parameter matrix) (and distortion coefficient set) are used to subsequently remove distortion from all acquired camera images; Step 1.2: Design a nested multi-scale ArUco jet calibration plate 201; like Figure 2 As shown, the surface of the nested multi-scale ArUco jet calibration plate 201 adopts a nested multi-scale layout design, on which at least two sets of ArUco control point arrays with different physical sizes are printed in a nested manner. The physical size of the control points in the ArUco control point array decreases in a stepwise manner from the outside to the inside, which is used to achieve sub-pixel-level positioning at different working distances to cover the jet retention point sampling requirements of the endoscope camera 301 in the near and far fields. The nested printing specifically refers to a planar layout where a smaller ArUco control point array is completely surrounded and nested within a larger ArUco control point array. Furthermore, to meet the fundamental mathematical requirements for spatial pose calculation by the endoscope camera 301, each ArUco control point array contains at least four ArUco control points. The core advantage of this nested multi-scale array design is its ability to better adapt to the dynamic changes in field of view and imaging resolution of the water jet within the endoscope camera 301 at different working distances. 1) When the endoscope camera 301 is in the far field emitting a water jet, its field of view is relatively wide. At this time, the small-sized ArUco control point array inside the nested multi-scale ArUco jet calibration plate 201 is prone to becoming blurry or even losing features because it exceeds the physical resolution limit of the endoscope camera 301. Meanwhile, the larger ArUco control point array located on the periphery can still occupy a sufficient pixel area in the image, thereby ensuring that the visual algorithm can stably extract features and perform accurate pose calculation.

[0020] 2) When the endoscope camera 301 is in the near field emitting a water jet, the field of view of the endoscope camera 301 shrinks sharply, and the large-sized control point array on the periphery of the nested multi-scale ArUco jet calibration plate 201 gradually moves away from the camera's field of view boundary. At this time, the small-sized ArUco control point array nested inside gradually becomes clear, allowing the endoscope camera 301 to still be able to capture at least 4 valid ArUco control points and complete the pose calculation.

[0021] As can be seen from the above embodiments, through the nesting and seamless switching of the multi-scale ArUco control point array, the present invention enables the endoscope camera 301 to maintain stable sub-millimeter pose calculations during dynamic movement in both the far field and near field, thereby fully covering and meeting the sampling requirements of the water jet retention point for subsequent parameter calibration.

[0022] Step 1.3: As Figure 3As shown, the endoscope camera 301 is controlled to capture images of the nested multi-scale ArUco waterjet calibration plate 201 at at least two different working distances. Image pairs are acquired at each working distance, including a reference image of the water jet 302 not being projected onto the nested multi-scale ArUco waterjet calibration plate 201 and a jet image of the water jet 302 being projected onto the nested multi-scale ArUco waterjet calibration plate 201. Then, the intrinsic parameter matrix is ​​used... The distortion coefficient set is then used for distortion correction, and the extrinsic parameter matrix of the endoscope camera 301 is calculated using the distorted reference image. Simultaneously, the two-dimensional pixel coordinates of the water jet retention point in the jet image are extracted. , Horizontal pixel coordinates Vertical pixel coordinates; The two-dimensional pixel coordinates of a point in an image refer to the coordinates of the point in the image coordinate system of the image. The image coordinate system is a two-dimensional Cartesian coordinate system with the upper left corner of the image as the origin, the x-axis parallel to the right of the image as the horizontal direction, and the y-axis parallel to the downward direction of the image as the vertical direction.

[0023] The specific process is as follows: In this embodiment, as shown... Figure 4 The image shown illustrates the images within the field of view of the endoscope camera 301 during the specific data acquisition process. At different working distances, image pairs are first acquired, comprising a reference image and a jet image, both with identical viewpoints. For example, at the far-field working distance, the large-scale ArUco control point array is identified using the far-field reference image 401, and the extrinsic parameter matrix of the current endoscope camera 301 is calculated using visual algorithms such as PnP. Next, the two-dimensional pixel coordinates of the water jet retention point are extracted from the corresponding far-field jet image 402. When the endoscope camera 301 advances to the near-field working distance of the nested multi-scale ArUco waterjet calibration plate 201, the outer control points go out of bounds. Then, the extrinsic parameter matrix at this time is adaptively calculated using the internal small-scale ArUco control point array in the near-field reference image 403. Simultaneously, the two-dimensional pixel coordinates of the water jet retention point in the near field are extracted from the near-field jet image 404. ; Step 1.4: Based on the two-dimensional pixel coordinates of the water jet retention point in the jet image. Determine the true physical depth of the water jet's retention point. ; Obtain the two-dimensional pixel coordinates of the water jet retention point in the jet image. Then, its corresponding true physical depth The transformation equations can be obtained by solving them. These transformation equations are geometric projection equations for transforming from the calibration plate coordinate system to the camera coordinate system, and their specific definitions are as follows: (1); in, The coordinates of the water jet retention point in the calibration plate coordinate system are given. The calibration plate coordinate system has its origin at the center point of the nested multi-scale ArUco water jet calibration plate 201, with the X-axis pointing horizontally to the right within the plane of the nested multi-scale ArUco water jet calibration plate 201, the Y-axis pointing vertically upwards within the plane of the nested multi-scale ArUco water jet calibration plate 201, and the Z-axis pointing outwards perpendicular to the plane of the nested multi-scale ArUco water jet calibration plate 201. The camera coordinate system has its origin at the camera optical center, with the horizontal direction of the image as the X' axis, the vertical downward direction of the image as the Y' axis, and the direction of the camera's principal optical axis as the Z' axis. By solving the above equations, the two-dimensional pixel coordinates of the water jet retention point in the jet image can be obtained. Accurate back-projection into three-dimensional space is used to calculate the true physical depth. ; Step 1.5: Establish a water jet scale sensing model and fit the parameters of the water jet scale sensing model. and , that is, the water jet parameters in the water jet scale sensing model; Consider the water jet from the ejection point Shoot, Shooting Point The coordinates in the camera coordinate system are defined as follows: The outgoing direction vector is defined as , Point of firing The components of the X', Y', and Z' axes in the camera coordinate system. The projections of the outgoing direction vector onto the X', Y', and Z' axes; Define the coordinates of the water jet retention point in the camera coordinate system as follows: And satisfy: (2); in, Let X, Y, and Z be the coordinates of the water jet retention point in the camera coordinate system, along the X', Y', and Z' axes. It is an unknown parameter; Then, based on the pinhole imaging model, the two-dimensional pixel coordinates of the water jet retention point in the jet image are determined. Should meet: (3); (4); in, Let be the equivalent focal length of the camera in the horizontal direction. The camera's tilt distortion coefficient is used to characterize the degree of non-orthogonality of the pixel coordinate axes of the physical photosensitive components of an image. The horizontal pixel coordinates of the camera principal point (i.e., the projection of the optical center onto the image plane). Let be the equivalent focal length of the camera in the vertical direction. The vertical pixel coordinates of the camera principal point (i.e., the projection of the optical center onto the image plane); Solve equations (2)-(4) simultaneously to eliminate unknown parameters. After integrating and substituting all parameters, the water jet scale sensing model can be obtained as follows: (5); in, and These are the parameters of the water jet scale sensing model to be determined; In this embodiment, multiple sets of information pairs obtained in the early stage (the actual physical depth of the water jet retention point and the vertical pixel coordinates of the water jet retention point) are substituted into the above formula, and a linear regression algorithm is used for fitting and solving. Finally, the parameters of the water jet scale perception model are determined with high precision. and ; Step 1.6: Establish a water jet reference line model and fit it to obtain the water jet reference line parameters. , and , that is, the water jet parameters in the water jet reference line model; (6); in, Reference line for water jet; This water jet reference line model utilizes the characteristic that water jets exhibit a ray structure within a short distance after ejection, effectively summarizing all possible situations of the water jet stagnation point within the working range. Therefore, this equation will serve as a linear correction reference for the water jet stagnation point in single-view water jet scale perception, for use in subsequent linear correction stages.

[0024] As can be seen from the above embodiments, through the water jet parameter calibration process, a stable mathematical mapping relationship between the two-dimensional image features of the water jet and the three-dimensional spatial depth was successfully established, providing the necessary prior model parameters for the subsequent real-time and accurate scene scale perception of the endoscope on the surface of unknown tissues.

[0025] Step 2: Use the endoscope camera 301 to collect scene-aware data of the lesion or target anatomical structure, and obtain the observation frame sequence, the reference frame 503, the jet frame 504, and the noisy two-dimensional pixel coordinates of the water jet retention point in the jet frame; the observation frame sequence includes several observation frames taken from different angles under waterless jet conditions; the reference frame and the jet frame are one frame image taken at the set observation position under waterless jet conditions and water jet conditions, respectively; like Figure 5 As shown, the specific steps include: Step 2.1: Use the endoscope camera 301 to take multi-angle images of the lesion or target anatomical structure 500 and remove distortion to obtain an image of length 500. Observation frame sequence , For the first in the observation frame sequence Frame; The multi-angle shooting refers to the same subject in the picture but different shooting angles, as shown in Observation Frame Example 1 501 and Observation Frame Example 2 502; Specifically, the operator controls the endoscope camera 301 to perform routine scanning and movement around the lesion or target anatomical structure 500. During this dynamic process, the camera pose of the endoscope camera 301 continuously changes, thereby capturing the surface texture and geometry of the target anatomical structure from multiple angles. At this stage, this series of raw images, excluding the water jet, is simultaneously recorded and collected, forming an observation frame sequence. This multi-view observation frame sequence will provide rich basic visual data for subsequent dense reconstruction of three-dimensional spatial point clouds; Step 2.2: At the set observation position, use the endoscope camera 301 to take fixed-angle pictures of the lesion or target anatomical structure 500 and remove distortion, and obtain reference frames respectively. and jet frame ; Specifically, after completing the multi-angle observation phase, the data acquisition process immediately enters the measurement phase, the purpose of which is to obtain key reference data for introducing real physical scale. In this phase, the operator needs to move the endoscope camera 301 to a suitable observation position for scale measurement, and keep the camera pose as stable as possible. First, under normal conditions without water jet activation, the endoscope camera 301 captures a clear image of the current field of view without water jet, which is used as the reference frame. (As shown in 503). Subsequently, while maintaining the position of the endoscope camera 301, the auxiliary water pump is activated to spray water jets 302 onto the surface of the lesion or target anatomical structure 500. After the water jets contact the tissue surface, they will form a clear jet landing point, i.e., a retention point. (As shown in 504), at this time, the endoscope camera 301 immediately captures an image containing the water jet retention point, as the jet frame. (As shown in 504), this ensures the acquisition of jet frames. Reference frame with waterless jet They have the exact same camera position and viewing angle; Step 2.3: From the jet frame The noisy two-dimensional pixel coordinates of the water jet retention point are extracted as follows: This may contain annotation noise; As can be seen from the above embodiments, the observation stage of the scene-aware data acquisition described in this invention does not introduce any special instruments or requirements, and captures the surface texture and geometric shape of the target anatomical structure from multiple angles, which is basically consistent with the scanning requirements and video retention requirements in routine clinical examinations; although the measurement stage of the scene-aware data acquisition described in this invention requires a reference frame With jet frame Although located at the same viewpoint, the use of water jets in clinical practice only requires the endoscopist to press a pedal, thus allowing for independent execution with the endoscopic handle locked, reducing the learning curve for clinical use. In this procedure, the resulting jet frame... Noisy two-dimensional pixel coordinates of water jet retention points This will be used for subsequent single-view water jet scale sensing, while the reference frame and observation frame sequence This will be used for subsequent multi-view scale information optimization; Step 3: Observe the frame sequence and reference frame These are input together into the 3D reconstruction backend, which processes the observation frame sequence. and reference frame Reconstruct the data and output the observation frame sequence. Corresponding local pointmap sequence Camera pose sequence and reference frame Corresponding local point map and camera pose ; The 3D reconstruction backend includes, but is not limited to, technologies such as COLMAP, 3GDS, NeRF, VGGT, Dust3R, and Pi3; as relevant professionals will readily know, the local point map can also be equivalently replaced by a combination of depth map and camera intrinsic parameters; Preferably, this embodiment uses the Pi3 3D reconstruction algorithm as the 3D reconstruction backend because, compared with traditional 3D reconstruction backends, Pi3 breaks the dependence on fixed reference views and has more accurate and smooth 3D reconstruction performance.

[0026] Step 4: Based on the water jet parameters in the water jet reference line model, linearly correct the noisy 2D pixel coordinates of the water jet retention point to obtain the noiseless pixel coordinates of the water jet retention point in the jet frame. Combined with the water jet parameters in the water jet scale perception model, calculate the true 3D physical coordinates of the water jet retention point in the camera coordinate system of the reference frame as the measurement anchor point. This step provides high-precision, interpretable, and stable physical cues for the scale perception of monocular endoscope scenes. like Figure 6 As shown, this single-view water jet scale perception mainly includes two steps: "linear correction" and "three-dimensional scale perception recovery". Step 4.1: For the noisy two-dimensional pixel coordinates of the water jet retention point Due to interference from factors such as annotation errors in actual clinical operations, these coordinates often contain observation noise. Therefore, the noisy two-dimensional pixel coordinates of the water jet retention point are orthogonally projected onto the water jet reference line model to obtain the noise-free pixel coordinates of the water jet retention point in the corrected jet frame. ; (7); Step 4.2: For the noise-free pixel coordinates of the water jet retention point in the corrected jet frame. Combining the water jet scale sensing model and intrinsic parameter matrix The true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame were accurately calculated. ; (8); in, For transpose; As demonstrated by the above embodiments, the linear correction step in the single-view water jet scale perception of the present invention reduces pixel annotation noise that may be present in the observation stage, thereby improving the accuracy of the model; the three-dimensional scale perception and recovery step establishes the correspondence between two-dimensional pixels and real three-dimensional coordinates, realizing the scale perception capability of the model. Through the above correction and perception steps under single-view conditions, the present invention successfully extracts stable measurement anchor points with high accuracy and physical interpretability from the endoscopic image.

[0027] Step 5: Based on the noiseless pixel coordinates of the water jet retention point in the jet frame, the local point map sequence corresponding to the observation frame sequence, the camera pose sequence, and the local point map and camera pose corresponding to the reference frame, the scale information of the measurement anchor point in the jet frame is propagated to other perspectives, multi-view information is integrated and disturbing perspectives are eliminated, and the global spatial consensus position of the measurement anchor point is calculated. This step effectively integrates observation information from multiple perspectives, effectively overcoming reconstruction errors caused by projection distortion in single-view schemes, resulting in higher reconstruction accuracy; at the same time, it clearly identifies frames with disturbances, improving the algorithm's anti-interference capability.

[0028] Relying solely on scale reconstruction from a single viewpoint can easily introduce local projection distortion. Furthermore, monocular endoscopic observation sequences are highly susceptible to image-level perturbations caused by random noise in a single view, such as motion blur, specular reflection, or water flow blurring. Therefore, this implementation scheme incorporates scale-free 3D structure reconstruction results (a sequence of local point maps corresponding to the observation frame sequence). Camera pose sequence And the local dot plot corresponding to the reference frame. and position (and the noiseless pixel coordinates of the water jet retention point in the jet frame) The scale information of the single-view metric anchor point is propagated to the multi-view sequence, and perturbation views caused by interference are eliminated through consistency evaluation, such as... Figure 7 As shown, the multi-view scale information optimization process specifically includes three stages: "projection", "extraction" and "aggregation".

[0029] Step 5.1: Local dot plot corresponding to the reference frame In the middle, the noiseless pixel coordinates of the water jet retention point in the jet frame are used. Centered on parameters To find the maximum number of points, search for the nearest neighbors and form a set of points. Then, take the median of this set of points in each dimension to form the spatial median. Through camera pose sequence Transform it to the camera coordinate system of each observation frame to generate the first... The theoretical projection center from a different perspective ; (9); in, For the camera pose in the camera pose sequence for The inverse matrix; This step projects the position corresponding to the single-view metric anchor point into the reconstruction space of other frames, thus obtaining the relationship between the single-view metric anchor point given by the reference frame and other observation frames.

[0030] Step 5.2: Local point plots in each observation frame Sino-Israeli Theoretical Projection Center Centered on parameters To find the upper limit of the number of points, search for the nearest neighbors and construct a point set. Then, take the median of this point set in each dimension to form the spatial hypothetical location of the observation frame. And uniformly project back to the camera coordinate system of the reference frame to form a set of spatial hypotheses. , The assumed spatial position in the camera coordinate system of the reference frame; In this embodiment, select This step utilizes the connection established in the previous step to bring the unknown scale information from other observation perspectives back to the reference frame perspective through back projection, thereby intuitively establishing the correspondence between the true scale in the reference frame and the scale-free reconstruction results of each perspective, overcoming the local projection distortion caused by a single perspective.

[0031] Step 5.3: Set of spatial hypotheses Using a consistency evaluation method, abnormal spatial assumptions caused by single-view occlusion or water flow washing are removed, and a binary mask is generated for each observation frame view. The consistency assessment can be any statistical method that evaluates the consistency of a set of data and removes outliers. Preferably, this embodiment uses an outlier detection method based on the interquartile range (IQR) to achieve the consistency assessment. First, calculate the spatial hypothesis set. Hypothetical locations in various spaces To the space hypothetical set of the centroid Euclidean distance as spatial deviation Then dynamically establish the interior point threshold. : (10); in, For the set of spatial deviations The third quartile, For the set of spatial deviations Interquartile range; Finally, the binarized mask for each observation frame is generated according to the following formula. : (11); in, This is an indicator function that takes the value 1 if and only if the condition within the parentheses is true, and 0 otherwise; Step 5.4: Use the binarized masks from each observation frame to aggregate the spatially assumed locations to obtain the global spatial consensus location of the metric anchor point. This step mainly addresses image-level perturbations caused by random noise in a single view; The data aggregation method can be any statistical method that can calculate a certain value from a set of data that can be used to assess the overall trend of the data. Preferably, this embodiment utilizes binarization masks from the perspective of each observation frame. To achieve data aggregation and calculate the global spatial consensus location. The formula is as follows: (12); As can be seen from the above embodiments, the IQR method can achieve data consistency assessment and remove outliers from each observation angle in the monocular endoscope scene based on the viewpoint mask; It covers the main effective scale perception information of the monocular endoscope scene, while filtering out interfering views that exceed the interior point threshold.

[0032] As can be seen from the above embodiments, this implementation scheme can effectively integrate all-round observation information from multiple perspectives, and clearly point out invalid frames with disturbances through a spatial consensus mechanism, thereby greatly improving the anti-interference ability of the reconstruction algorithm and the reconstruction accuracy at the absolute physical scale.

[0033] Step 6: Calculate the decoupling scale factor using the noiseless pixel coordinates of the water jet retention point in the jet frame and the global spatial consensus position of the metric anchor point to achieve perception reconstruction and obtain a 3D scene perception model. ; (13); in, This is the decoupling scaling factor; Finally, based on the binarization mask from the perspective of each observation frame... All valid observation frames reconstructed into local point maps are selected, their unions are stacked, and the final stacked results are scaled according to the decoupling scale factor. By scaling, a final 3D scene perception model with absolute physical scale can be generated. .

[0034] As demonstrated by the above embodiments, the steps efficiently and reliably capture the true scale from the physical cues provided by the water jet's physical anchor point, while simultaneously propagating the scale information to other observation perspectives, mitigating projection distortion that is highly likely to occur in single-view observations. This invention effectively achieves high-precision scale reconstruction in commonly used clinical monocular endoscope systems without requiring additional hardware implantation or interfering with clinical operations, possessing both physical interpretability, anti-interference capabilities, and clinical feasibility.

[0035] Example 2: A monocular endoscope scene scale perception and reconstruction system is used to implement a monocular endoscope scene scale perception and reconstruction method, including: The water jet parameter calibration module is used to construct a water jet scale sensing model and a water jet reference line model, and obtain the water jet parameters in the water jet scale sensing model and the water jet reference line model through fitting. The acquisition module is used to take pictures of lesions or target anatomical structures using an endoscope camera, and obtain the observation frame sequence, reference frame, jet frame, and noisy two-dimensional pixel coordinates of water jet retention points in the jet frame. The reconstruction module is used to reconstruct the observation frame sequence and the reference frame to obtain the local point map sequence and camera pose sequence corresponding to the observation frame sequence, as well as the local point map and camera pose corresponding to the reference frame. The metric anchor point calculation module is used to linearly correct the noisy two-dimensional pixel coordinates of the water jet retention point based on the water jet parameters in the water jet reference line model, obtain the noiseless pixel coordinates of the water jet retention point in the jet frame, and calculate the true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame as the metric anchor point by combining the water jet parameters in the water jet scale perception model. The global spatial consensus location acquisition module is used to calculate the global spatial consensus location of the metric anchor point based on the noiseless pixel coordinates of the water jet retention point in the jet frame, the local point map sequence corresponding to the observation frame sequence, the camera pose sequence, and the local point map and camera pose corresponding to the reference frame. The perception reconstruction module is used to calculate the decoupling scale factor by using the noiseless pixel coordinates of the water jet retention point in the jet frame and the global spatial consensus position of the metric anchor point, so as to realize perception reconstruction and obtain a three-dimensional scene perception model.

[0036] Example 3: This embodiment proposes an electronic device, including: one or more processors, and a memory, wherein the memory is used to store instructions, and when the instructions are executed by the one or more processors, the one or more processors execute the aforementioned monocular endoscope scene scale perception reconstruction method.

[0037] The electronic device may be a mobile phone, computer, or tablet computer, etc., and includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements a monocular endoscope scene scale perception reconstruction method as described in the embodiments. It is understood that the electronic device may also include an input / output (I / O) interface and communication components.

[0038] The processor is used to execute all or part of the steps in the monocular endoscope scene scale perception reconstruction method described in the above embodiments. The memory is used to store various types of data, which may include, for example, instructions for any application or method in the electronic device, as well as application-related data.

[0039] The processor can be implemented as an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a programmable logic device (PLD), a field-programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and is used to execute the monocular endoscope scene scale perception reconstruction method described in the above embodiments.

[0040] Example 4: This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0041] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the monocular endoscope scene scale perception reconstruction method described in the various embodiments of this application.

[0042] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, random access memory (RAM), static random-access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, which, when executed by a processor, can implement the various steps of the monocular endoscope scene scale perception reconstruction method described above.

[0043] Example 5: This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned monocular endoscope scene scale perception and reconstruction method.

[0044] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0045] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0046] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of this disclosure and its equivalents, then the intent of this disclosure also includes these modifications and variations.

Claims

1. A method for scene scale perception and reconstruction using a monocular endoscope, characterized in that, Using the water jet retention point as the physical measurement benchmark, a water jet reference line constraint model and a water jet scale mapping model are constructed through pre-calibration. The images captured by the endoscope camera are then reconstructed in three dimensions. The three-dimensional reconstruction results are combined with the coordinate correction of the water jet retention point and global spatial consistency positioning to obtain an absolute scale benchmark. The decoupling scale factor is calculated using the absolute scale benchmark to achieve three-dimensional scene perception and reconstruction.

2. The method for scene scale perception and reconstruction using a monocular endoscope according to claim 1, characterized in that, Includes the following steps: A water jet scale sensing model and a water jet reference line model are constructed, and the water jet parameters in the water jet scale sensing model and the water jet reference line model are obtained by fitting. The water jet scale sensing model is used to describe the relationship between the true physical depth of the water jet retention point and the vertical pixel coordinates of the water jet retention point. The water jet reference line model is used to describe the water jet reference line; An endoscope camera is used to photograph lesions or target anatomical structures to obtain an observation frame sequence, a reference frame, a jet frame, and noisy two-dimensional pixel coordinates of water jet retention points in the jet frame; the observation frame sequence includes several observation frames taken from different angles under waterless jet conditions; the reference frame and the jet frame are images taken at a set observation position under waterless jet conditions and water-jet conditions, respectively. Reconstruct the observation frame sequence and the reference frame to obtain the local point map sequence and camera pose sequence corresponding to the observation frame sequence, as well as the local point map and camera pose corresponding to the reference frame. Linear correction is performed on the noisy two-dimensional pixel coordinates of the water jet retention point based on the water jet parameters in the water jet reference line model to obtain the noiseless pixel coordinates of the water jet retention point in the jet frame. The true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame are calculated by combining the water jet parameters in the water jet scale perception model as the measurement anchor point. The global spatial consensus position of the metric anchor point is calculated based on the noiseless pixel coordinates of the water jet retention point in the jet frame, the local point map sequence corresponding to the observation frame sequence, the camera pose sequence, and the local point map and camera pose corresponding to the reference frame. By using the noiseless pixel coordinates of the water jet retention point in the jet frame and the global spatial consensus position of the metric anchor point, the decoupling scale factor is calculated to realize the 3D scene perception reconstruction and obtain the 3D scene perception model.

3. The method for scene scale perception and reconstruction using a monocular endoscope according to claim 2, characterized in that, The construction of the water jet scale sensing model and the water jet reference line model, and the obtaining of the water jet parameters in the water jet scale sensing model and the water jet reference line model through fitting, specifically includes the following steps: A1: Perform parameter calibration on the endoscope camera to obtain the intrinsic parameter matrix and distortion coefficient set of the endoscope camera; A2: Design of a nested multi-scale ArUco jet calibration plate; The nested multi-scale ArUco jet calibration plate has at least two nested ArUco control point arrays with different physical sizes printed on its surface. The physical size of the control points in the ArUco control point array decreases in a stepwise manner from the outside to the inside. Each ArUco control point array contains at least four ArUco control points. The nested printing refers to a planar layout in which a smaller physical size ArUco control point array is completely surrounded and nested inside a larger physical size ArUco control point array. A3: Control the endoscope camera to capture images of the nested multi-scale ArUco waterjet calibration plate at at least two different working distances. At each working distance, acquire image pairs, including a baseline image without water jets being ejected onto the nested multi-scale ArUco waterjet calibration plate and an image of the jet with water jets ejected onto the nested multi-scale ArUco waterjet calibration plate. Then, use the intrinsic parameter matrix... The distortion coefficient set is then used for distortion correction, and the extrinsic parameter matrix of the endoscope camera is calculated using the distorted reference image. Simultaneously, the two-dimensional pixel coordinates of the water jet retention point in the jet image are extracted. , Horizontal pixel coordinates Vertical pixel coordinates; A4: Based on the two-dimensional pixel coordinates of the water jet retention point in the jet image. Determine the true physical depth of the water jet's retention point. ; The true physical depth of the water jet's point of repose By solving the transformation equations, we obtain that the transformation equations are geometric projection equations for the transformation from the calibration plate coordinate system to the camera coordinate system: （1）； in, The coordinates of the water jet retention point are given in the calibration plate coordinate system. The calibration plate coordinate system has the center point of the nested multi-scale ArUco water jet calibration plate as the origin, the horizontal rightward direction within the plane of the nested multi-scale ArUco water jet calibration plate as the X-axis, the vertical upward direction within the plane of the nested multi-scale ArUco water jet calibration plate as the Y-axis, and the outward direction perpendicular to the plane of the nested multi-scale ArUco water jet calibration plate as the Z-axis. The camera coordinate system has the camera optical center as the origin, the horizontal direction of the image as the X'-axis, the vertical downward direction of the image as the Y'-axis, and the direction of the camera's principal optical axis as the Z'-axis. A5: Establish a water jet scale sensing model and fit the parameters of the water jet scale sensing model. and , that is, the water jet parameters in the water jet scale sensing model; The water jet scale sensing model is as follows: （5）； in, Let Z be the coordinate components of the water jet retention point in the camera coordinate system along the Z' axis. and These are the parameters of the water jet scale sensing model to be determined; A6: Establish a water jet reference line model and fit it to obtain the water jet reference line parameters. , and , that is, the water jet parameters in the water jet reference line model; （6）； in, This is the reference line for the water jet.

4. The method for scene scale perception and reconstruction using a monocular endoscope according to claim 2, characterized in that, The process of using an endoscopic camera to photograph lesions or target anatomical structures to obtain observation frame sequences, reference frames, jet frames, and noisy two-dimensional pixel coordinates of water jet retention points in the jet frames specifically includes the following steps: B1: Using an endoscopic camera, multi-angle images of the lesion or target anatomical structure are taken and distortion is removed to obtain images of a length of [length missing]. Observation frame sequence , For the first in the observation frame sequence frame; B2: At the designated observation location, use an endoscopic camera to take fixed-angle images of the lesion or target anatomical structure and remove distortion, obtaining reference frames respectively. and jet frame ; Specifically: First, under normal conditions where the water jet is not activated, the endoscopic camera captures an image of the current field of view without water jet, and uses this as the reference frame. Subsequently, while maintaining the endoscopic camera's position, the auxiliary water pump is activated to spray water jets onto the surface of the lesion or target anatomical structure. Upon contact with the tissue surface, the water jets create a distinct landing point, or retention point. At this moment, the endoscopic camera immediately captures an image containing the water jet retention point, as the jet frame. ; B3: From the jet frame The noisy two-dimensional pixel coordinates of the water jet retention point are extracted as follows: .

5. The method for scene scale perception and reconstruction using a monocular endoscope according to claim 2, characterized in that, The process of linearly correcting the noisy two-dimensional pixel coordinates of the water jet retention point based on the water jet parameters in the water jet reference line model to obtain the noiseless pixel coordinates of the water jet retention point in the jet frame, and then calculating the true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame using the water jet parameters in the water jet scale perception model as the measurement anchor point, specifically includes the following steps: C1: Orthogonally project the noisy 2D pixel coordinates of the water jet retention point onto the water jet reference line model to obtain the noise-free pixel coordinates of the water jet retention point in the corrected jet frame. ; （7）； C2: Noise-free pixel coordinates of the water jet retention point in the corrected jet frame. Combining the water jet scale sensing model and intrinsic parameter matrix The true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame were calculated. ; （8）； in, This is a transpose.

6. The method for scene scale perception and reconstruction using a monocular endoscope according to claim 2, characterized in that, The calculation of the global spatial consensus position of the metric anchor point based on the noiseless pixel coordinates of the water jet retention point in the jet frame, the local point map sequence corresponding to the observation frame sequence, the camera pose sequence, and the local point map and camera pose corresponding to the reference frame specifically includes the following steps: D1: Local dot plot corresponding to the reference frame In the middle, the noiseless pixel coordinates of the water jet retention point in the jet frame are used. Centered on parameters To find the maximum number of points, search for the nearest neighbors and form a set of points. Then, take the median of this set of points in each dimension to form the spatial median. Through camera pose sequence Transform it to the camera coordinate system of each observation frame to generate the first... The theoretical projection center from a different perspective ; （9）； in, For the camera pose in the camera pose sequence for The inverse matrix; D2: Local point plots in each observation frame Sino-Israeli Theoretical Projection Center Centered on parameters To find the upper limit of the number of points, search for the nearest neighbors and construct a point set. Then, take the median of this point set in each dimension to form the spatial hypothetical location of the observation frame. And uniformly project back to the camera coordinate system of the reference frame to form a set of spatial hypotheses. , The assumed spatial position in the camera coordinate system of the reference frame; D3: On the set of spatial hypotheses Using a consistency evaluation method, abnormal spatial assumptions caused by single-view occlusion or water flow washing are removed, and a binary mask is generated for each observation frame view. D4: Using the binarized masks from each observation frame, data aggregation is performed on the spatially assumed locations to obtain the global spatial consensus location of the metric anchor point. ; The data aggregation method can be any statistical method that can calculate a certain trend from a set of data that can be used to assess the overall trend of the data.

7. The method for scene scale perception and reconstruction using a monocular endoscope according to claim 6, characterized in that, The method for consistency assessment is as follows: First, calculate the spatial hypothesis set. Hypothetical locations in various spaces To the space hypothetical set of the centroid Euclidean distance as spatial deviation Then dynamically establish the interior point threshold. : （10）； in, For the set of spatial deviations The third quartile, For the set of spatial deviations Interquartile range; Generate binarized masks from the perspective of each observation frame. : （11）； in, This is an indicator function that takes the value 1 if and only if the condition within the parentheses is true, otherwise it takes the value 0.

8. The method for scene scale perception and reconstruction using a monocular endoscope according to claim 6, characterized in that, Calculate the global spatial consensus location The formula is: （12）。 9. The method for scene scale perception and reconstruction using a monocular endoscope according to claim 2, characterized in that, The decoupling scale factor is calculated using the noiseless pixel coordinates of the water jet retention point in the jet frame and the global spatial consensus position of the metric anchor point to achieve perception reconstruction and obtain a 3D scene perception model. Specifically: Calculate the decoupling scaling factor: （13）； in, This is the decoupling scaling factor; Based on the binarization mask from the perspective of each observation frame All valid observation frames reconstructed into local point maps are selected, their unions are stacked, and the final stacked results are scaled according to the decoupling scale factor. Scaling is performed to generate a 3D scene perception model. .

10. A monocular endoscope scene scale perception and reconstruction system, used to implement the monocular endoscope scene scale perception and reconstruction method according to any one of claims 1-8, characterized in that, include: The water jet parameter calibration module is used to construct a water jet scale sensing model and a water jet reference line model, and obtain the water jet parameters in the water jet scale sensing model and the water jet reference line model through fitting. The acquisition module is used to take pictures of lesions or target anatomical structures using an endoscope camera, and obtain the observation frame sequence, reference frame, jet frame, and noisy two-dimensional pixel coordinates of water jet retention points in the jet frame. The reconstruction module is used to reconstruct the observation frame sequence and the reference frame to obtain the local point map sequence and camera pose sequence corresponding to the observation frame sequence, as well as the local point map and camera pose corresponding to the reference frame. The metric anchor point calculation module is used to linearly correct the noisy two-dimensional pixel coordinates of the water jet retention point based on the water jet parameters in the water jet reference line model, obtain the noiseless pixel coordinates of the water jet retention point in the jet frame, and calculate the true three-dimensional physical coordinates of the water jet retention point in the camera coordinate system of the reference frame as the metric anchor point by combining the water jet parameters in the water jet scale perception model. The global spatial consensus location acquisition module is used to calculate the global spatial consensus location of the metric anchor point based on the noiseless pixel coordinates of the water jet retention point in the jet frame, the local point map sequence corresponding to the observation frame sequence, the camera pose sequence, and the local point map and camera pose corresponding to the reference frame. The perception reconstruction module is used to calculate the decoupling scale factor by using the noiseless pixel coordinates of the water jet retention point in the jet frame and the global spatial consensus position of the metric anchor point, so as to realize perception reconstruction and obtain a three-dimensional scene perception model.

Images