An endoscopic polyp information acquisition method based on monocular vision

Abstract

The present application belongs to the field of medical image processing and endoscope detection, and relates to a method for acquiring information of a polyp under an endoscope based on monocular vision, the information of the polyp including the distance between the polyp and the lens of the endoscope, the method comprising: acquiring an in-vivo endoscope image when biopsy forceps are extended from the endoscope channel, the forceps body is straight, the forceps mouth is closed and the forceps mouth is against the polyp; then converting the in-vivo endoscope image into a bird's-eye view B; finally acquiring the pixel length of the forceps body in the bird's-eye view B and bringing it into a mapping model to obtain the physical length of the forceps body in the bird's-eye view B, and taking the sum of the physical length and the physical length of the forceps head as the distance between the polyp and the lens of the endoscope. The present application establishes a high-precision pixel-physical space mapping model, effectively overcomes the measurement error caused by image distortion, variable viewing angle and occlusion interference, and makes the polyp size measurement result have millimeter-level precision and good repeatability.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of medical image processing and endoscopic detection, and relates to an information acquisition method for polyps under an endoscope based on monocular vision. BACKGROUND

[0002] In clinical endoscopy, accurate acquisition of polyp-related information plays a key role in the risk assessment of malignant tumors. However, the measurement methods in current clinical practice have obvious limitations: on the one hand, they rely on the subjective visual estimation of doctors, and the measurement accuracy is significantly affected by the experience of the operator; on the other hand, they use postoperative ex vivo measurement, which cannot realize real-time guidance during surgery and will lead to systematical underestimation of the measurement results due to the shrinkage effect of tissue dehydration.

[0003] To solve the above problems, the prior art uses automatic segmentation technology in the field of computer vision to measure polyp-related information, and has made significant progress. However, when monocular cameras (single 2D images) are used for depth estimation, there is a fundamental "scale ambiguity" problem. Specifically, a very small object in an image may be a large object far away or a small object close by, and only pixel information cannot determine its absolute size. Moreover, when the real-world distance corresponding to a specific point in the image is known, this point can serve as an absolute "scale reference" for the entire depth map, and based on this known point, the absolute depth of other regions in the image can be calculated, rather than the relative distance. Once the accurate object distance (i.e. the distance between the polyp and the endoscope lens) is provided, the "scale ambiguity" problem is eliminated.

[0004] In addition, the object distance information can provide important prior knowledge for models based on deep learning (especially monocular depth estimation). For example, when it is used as an input feature, the object distance information can be input into the neural network as an additional input channel together with the image, guiding the network to learn the mapping from image features to real depth values faster and more accurately; as a supervision signal, the object distance information can be used to construct a more accurate loss function during the training of the model (such as ensuring that the predicted depth at the point with the known distance is highly consistent with the true value, thereby constraining the prediction results of the entire depth map); at the same time, with the object distance information, the model may not need to be trained on a super large-scale dataset covering all possible distances, but can perform well within a specific distance range.

[0005] However, the existing data sets all lack the measurement of the object distance information, which is directly caused by the deficiency of the existing measurement technology. On the one hand, the polyp segmentation data set (such as Kvasir-SEG) lacks synchronous depth information and physical size labeling; on the other hand, the 3D reconstruction data set (such as EndoMapper) lacks fine labeling of the lesion level. After in-depth investigation, many existing technical solutions cannot simultaneously obtain the geometric size and three-dimensional space information of the lesion, and thus it is difficult to construct a high-quality data set that can support accurate quantitative analysis.

[0006] This point is particularly prominent in existing patent technologies. For example, the patent application with the application publication number CN104146711A discloses a lesion size measurement method and system based on an endoscope. Although the method performs visual comparison by projecting an "optical ruler", the measurement principle has inherent defects: it relies heavily on the ideal assumption that "the lesion and the ruler are in the same plane", which is almost impossible to meet in actual clinical practice. Once there is a distance difference, the perspective effect will cause a huge error. Therefore, this method is not only unreliable and difficult to automate, but also cannot provide accurate size labeling with spatial context for the data set.

[0007] For another example, the patent application with the application publication number CN211270764U discloses an endoscopic lesion measuring ruler. The contact type mechanical measuring ruler used by the device has the limitation of the lack of information dimension. The device can only obtain the local two-dimensional size of the lesion, and cannot perceive the spatial distance of the lesion from the lens. Therefore, the measurement result provided by the device is "isolated" and is out of the three-dimensional spatial context, which is difficult to be used for calibrating the scale information in the two-dimensional image and cannot provide any help for constructing a data set containing the depth of the lesion. In addition, the contact operation itself also brings efficiency and safety risk problems.

[0008] In summary, the existing technology lacks an effective method for obtaining the distance between the polyp and the endoscope lens, and the lack of this object distance information seriously restricts the accurate acquisition of polyp-related information under the endoscope. Therefore, it is of great significance to propose a method for obtaining the distance between the polyp and the lens under the endoscope based on monocular vision to solve the above problems. SUMMARY

[0009] The purpose of the present application is to solve the problems existing in the prior art and provide a method for obtaining information of a polyp under an endoscope based on monocular vision.

[0010] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows:

[0011] A method for obtaining information of a polyp under an endoscope based on monocular vision, the information of the polyp including the distance between the polyp and the lens of the endoscope, the obtaining method comprising the following steps:

[0012] (1) Obtain in vivo endoscopic images when the biopsy forceps extend out of the endoscope channel, the forceps body is straight, the forceps jaws are closed and the forceps jaws are against the polyp;

[0013] (2) Convert the in vivo endoscopic images into a bird's-eye view B;

[0014] (3) Obtain the pixel length of the biopsy forceps body in the bird's-eye view B and input it into the mapping model to obtain the physical length of the biopsy forceps body in the bird's-eye view B. Add this length to the physical length of the biopsy forceps head as the distance between the polyp and the endoscope lens. ;

[0015] The mapping model is constructed as follows: After marking the scale along the length direction on the biopsy forceps, an external endoscope image is acquired when the biopsy forceps extend from the endoscope channel and the forceps body is straight. The external endoscope image is then converted into a bird's-eye view A. The pixel length of the biopsy forceps body in bird's-eye view A is then obtained. At the same time, the physical length of the biopsy forceps body in bird's-eye view A is determined according to the scale. Finally, a quadratic function is used to fit the pixel length of the biopsy forceps body in bird's-eye view A with the physical length of the biopsy forceps body in bird's-eye view A, thus obtaining the mapping model.

[0016] As a preferred technical solution:

[0017] The method for acquiring information about polyps under endoscopy based on monocular vision, as described above, involves acquiring an external endoscopic image when the biopsy forceps extend from the endoscope channel and are straight, and then converting the external endoscopic image into a bird's-eye view A. The specific steps are as follows:

[0018] (a) Extend the biopsy forceps out of the endoscope channel and keep the forceps straight, and record this process via video;

[0019] (b) Extract m extracorporeal endoscope images from the video in chronological order from front to back, where m ≥ 8;

[0020] (c) Obtain the homography matrix based on the m-th external endoscopic image;

[0021] (d) Use the homography matrix to convert the first to (m-1) extracorporeal endoscope images into a bird's-eye view A;

[0022] (e) Repeat steps (a) to (d) (n-1) times to obtain a total of n×(m-1) bird's-eye view images A and n homography matrices, where n≥13.

[0023] As described above, the method for acquiring information about polyps under endoscopy based on monocular vision includes step (c) as follows:

[0024] First, extract the pixel coordinates of the four vertices of the clamp from the m-th external endoscope image;

[0025] Then, define a target rectangle in the bird's-eye view coordinate system. Its physical dimensions W and H are the physical dimensions of the part of the clamp body that is photographed. The physical coordinates of the four vertices of the target rectangle are (0,0), (W,0), (W,H), (0,H).

[0026] Next, the physical coordinates of the four vertices of the target rectangle are multiplied by the scale factor to obtain the pixel coordinates of the four vertices of the target rectangle;

[0027] Furthermore, the target rectangle is translated so that it is located in the center of the output bird's-eye view, thus obtaining the updated pixel coordinates of the four vertices of the target rectangle;

[0028] Finally, the pixel coordinates of the four vertices of the clamp are directly linearly transformed with the pixel coordinates of the four vertices of the updated target rectangle using the DLT algorithm to calculate the homography matrix.

[0029] The method for acquiring information about polyps under endoscopy based on monocular vision, as described above, converts the in vivo endoscopic image into a bird's-eye view B using a universally applicable homography matrix. The steps for obtaining the universally applicable homography matrix are as follows:

[0030] (i) Collect n homography matrices, each homography matrix having r elements, where r=9;

[0031] (ii) For each homography matrix, the first (r-1) elements are arranged in order to form an (r-1)-dimensional eigenvector, resulting in a total of n (r-1)-dimensional eigenvectors;

[0032] (iii) Cluster the n (r-1) dimensional feature vectors to find the largest and most concentrated cluster;

[0033] (iv) Determine whether more than 85% of the (r-1) dimensional eigenvectors are located in the largest and most concentrated cluster. If so, proceed to the next step; otherwise, collect n homography matrices again.

[0034] (v) For all (r-1) dimensional eigenvectors in the largest and most concentrated cluster, calculate the median dimension by dimension to obtain an (r-1) dimensional median vector;

[0035] (vi) After the last element of the (r-1)-dimensional median vector, add an element 1 to obtain an r-dimensional eigenvector;

[0036] (vii) Restore the r-dimensional eigenvectors to matrix form to obtain the universally applicable homography matrix.

[0037] The method for acquiring information about polyps under endoscopy based on monocular vision, as described above, further includes the physical size of the polyp, and the method also includes the following steps:

[0038] (4) After outlining the polyp area near the pincers in the bird's-eye view B, obtain the minimum bounding rectangle of the outline and record its pixel size. ;

[0039] (5) Calculate the physical size of the polyp. The calculation formula is as follows:

[0040] ;

[0041] In the formula, The equivalent focal length (horizontal or vertical) of the endoscope lens.

[0042] As described above, a method for obtaining information about polyps under endoscopy based on monocular vision uses a lightweight segmentation model (such as Segment Anything Tiny, i.e., a "segment anything" small model) to outline the polyp region near the clamp in the bird's-eye view B.

[0043] The above describes a method for acquiring information about polyps under endoscopy based on monocular vision. The calculation formula is as follows:

[0044] ;

[0045] In the formula, The pixel size (horizontal or vertical) of the image captured by the endoscope lens. The field of view (horizontal or vertical) is expressed in degrees.

[0046] The above describes a method for acquiring information about polyps under endoscopy based on monocular vision. The calculation formula is as follows:

[0047] ;

[0048] In the formula, The physical size (horizontal or vertical) of the image captured by the endoscope lens. It is the focal length.

[0049] Beneficial effects:

[0050] (1) This invention constructs a standardized endoscopic image dataset that simultaneously covers precise segmentation of polyp contours, physical size, and depth information, effectively compensating for the shortcomings of existing datasets (such as Kvasir-SEG and Endo Mapper) in lesion-level labeling and multimodal information fusion. This dataset provides a reliable mapping relationship from pixel to millimeter scale for deep learning models, significantly enhancing the training reliability and clinical applicability of the models in polyp quantification analysis.

[0051] (2) This invention utilizes the homography matrix and combines quadratic function fitting with biopsy forceps fixation structure features to establish a high-precision pixel-physical space mapping model, which effectively overcomes measurement errors caused by image distortion, changing viewing angles and occlusion interference, and enables polyp size measurement results to have millimeter-level accuracy and good repeatability.

[0052] (3) This invention integrates image acquisition, calibration processing, depth estimation and size conversion into a complete end-to-end measurement process. It relies on a lightweight segmentation model and automated scripts to realize polyp contour extraction and parameter calculation, which greatly reduces manual intervention and improves the efficiency and consistency of the diagnostic process. It is suitable for the high-efficiency operation needs in actual clinical environments.

[0053] (4) The present invention is easy to operate and does not require complex or extra expensive equipment. It can achieve accurate measurement by relying only on conventional endoscopes and biopsy forceps, and has strong hospital applicability and scenario adaptability. In addition, the standardized polyp size data output by the system can provide doctors with objective and quantitative decision-making basis, which helps to improve the standardization and intelligence of endoscopic polyp diagnosis and treatment. Attached Figure Description

[0054] Figure 1 This is a flowchart of the method for acquiring information about polyps under endoscopy based on monocular vision according to the present invention;

[0055] Figure 2 This is the original image of the external endoscope image in Embodiment 1 of the present invention;

[0056] Figure 3 This is a schematic diagram showing the selection of the clamping source points (the four red dots in the figure) for the external endoscope image in Embodiment 1 of the present invention;

[0057] Figure 4 This is a schematic diagram of transforming the original image of the external endoscope into a bird's-eye view A in Embodiment 1 of the present invention;

[0058] Figure 5 This is the original image of the in vivo endoscope image in Embodiment 1 of the present invention;

[0059] Figure 6 This is a schematic diagram of transforming the original image of the in vivo endoscope into a bird's-eye view B in Embodiment 1 of the present invention;

[0060] Figure 7 This is a polyp outline obtained in Embodiment 1 of the present invention;

[0061] Figure 8 This is an image of the polyp in Embodiment 1 of the present invention;

[0062] Figure 9 This is a schematic diagram showing the proportional relationship of similar triangles in the pinhole imaging of the present invention. Detailed Implementation

[0063] The present invention will be further described below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0064] A method for acquiring information about polyps under endoscopy based on monocular vision, such as... Figure 1 As shown, the information about the polyp includes the distance between the polyp and the endoscope lens, and the physical size of the polyp. The steps for obtaining this information are as follows:

[0065] (1) Preparation of experimental equipment and tools;

[0066] Endoscope: Medical endoscope;

[0067] Medical gauze;

[0068] Marker: Oil-based markers resistant to chemical solvents. Before use, a solvent wiping test is performed to verify their corrosion resistance and ensure that their markings will not be easily erased by disinfectant solvents (such as alcohol or iodine).

[0069] Millimeter calipers: with an accuracy of 1 millimeter, clear and unworn graduation lines, and a locking function to fix the measurement position;

[0070] (2) Mark the scale along the length of the biopsy forceps;

[0071] (2.1) Initial alignment: Place the blue plastic clamp body to be marked horizontally on a stable workbench, ensuring that the surface is clean, dust-free, and oil-free to prevent offset or blurring during the marking process. Then, align the zero mark end of the millimeter scale caliper tightly with one end of the clamp body. Use a small clamp or heat-resistant tape to fix the other end of the caliper, ensuring that there is no relative slippage between the caliper and the clamp body. Optimize the alignment process through visual inspection and slight adjustments to ensure the consistency of the reference for subsequent measurements.

[0072] (2.2) Segment Marking: Starting from the beginning of the forceps body, draw a short line on the forceps body every 5 mm along the scale line of the caliper as segment marks. After each mark is completed, use the caliper to check and ensure that the mark position is exactly an integer multiple of 5 mm. If a deviation is found, erase and redraw immediately to maintain high accuracy. Then move 5 mm backward along the caliper and repeat the drawing steps until the entire length of the forceps body is covered. The choice of this marking interval is based on actual clinical needs. The 5 mm segment facilitates subsequent depth fitting and is not too dense to cause marking confusion. When drawing, keep the marker perpendicular to the surface of the forceps body to ensure that the line width is uniform and the straightness is high, and avoid marking deformation caused by tilting. When moving the caliper, operate slowly and check the fit multiple times to reduce cumulative error.

[0073] (2.3) Marking verification: After all markings are completed, a full verification is performed. First, use calipers to slowly slide along the clamp body to check whether the spacing between each marking point is consistent at 5 mm, and record any potential uneven distribution. The full verification process is repeated 2-3 times to enhance reliability and ensure that the scale can provide accurate depth reference in both internal and external environments.

[0074] (3) Construct a mapping model and a universally applicable homography matrix;

[0075] (3.1) After marking the scale along the length of the biopsy forceps, first use medical gauze to gently wipe the surface of the endoscope lens in a circular motion to avoid straight scratches in order to maintain the optical performance of the lens and ensure that all water stains, dirt or fingerprints are removed. Then check the lens visually or with a magnifying glass to confirm its cleanliness.

[0076] (3.2) Insert the prepared biopsy forceps scale into the endoscope channel, extend it at a speed of 1-2 mm per second and keep the forceps straight to avoid image shaking caused by rapid movements. At the same time, keep it in the center of the endoscope field of view. During the acquisition process, ensure that the scale is clearly visible to maintain a clear recording state. Record images of the scale at different extension lengths through video to provide depth reference data for verifying the reliability of in vitro calibration.

[0077] (3.2) Extract m extracorporeal endoscope images from the video in chronological order from front to back, where m ≥ 8;

[0078] (3.3) Obtain the homography matrix;

[0079] First, extract the pixel coordinates of the four vertices (source points) of the clamp from the m-th external endoscope image;

[0080] Then, define a target rectangle in the bird's-eye view coordinate system. Its physical dimensions W and H are the physical dimensions of the part of the clamp body that is photographed. The physical coordinates of the four vertices of the target rectangle are (0,0), (W,0), (W,H), (0,H).

[0081] Next, the physical coordinates of the four vertices of the target rectangle are multiplied by the scale factor to obtain the pixel coordinates of the four vertices of the target rectangle;

[0082] Furthermore, the target rectangle is translated so that it is located in the center of the output bird's-eye view, thus obtaining the updated pixel coordinates of the four vertices of the target rectangle;

[0083] Finally, the pixel coordinates of the four vertices of the clamp body and the pixel coordinates of the four vertices of the updated target rectangle are used to calculate the homography matrix using the Direct Linear Transform (DLT) algorithm.

[0084] The process for determining the scale ratio coefficient is as follows:

[0085] (A) Under the condition of selecting a specific endoscope model and setting a fixed working distance (the fixed working distance can be selected according to the needs of the scenario, the purpose of which is to simulate the object distance range when the doctor performs surgery in the body), a photo of a standard checkerboard calibration board with a precise known size is acquired. Then, the sub-pixel level coordinates of each inner corner point of the checkerboard calibration board are automatically extracted using an image processing algorithm, and these coordinates are used as the pixel coordinates of the feature points.

[0086] (B) Select multiple pairs of feature points, calculate the pixel distance of each pair of points on the image plane, and determine their corresponding physical distance. Then, calculate the preliminary scale coefficient of each pair of feature points according to the formula "scale ratio coefficient (px / mm) = pixel distance (px) / physical distance (mm)". Finally, perform an arithmetic mean operation on all the preliminary scale coefficients, and the result is the scale ratio coefficient.

[0087] (3.4) Use the homography matrix to convert the first to (m-1) images of the external endoscope into a bird's-eye view A;

[0088] (3.5) Repeat steps (3.2) to (3.4) (n-1) times to obtain a total of n×(m-1) bird's-eye view A and n homography matrices, n≥13;

[0089] (3.6) First, obtain the pixel length of the biopsy forceps body in the bird's-eye view A, and simultaneously determine the physical length of the biopsy forceps body in the bird's-eye view A based on the scale. Then, use a quadratic function to fit the pixel length of the biopsy forceps body in the bird's-eye view A with the physical length of the biopsy forceps body in the bird's-eye view A, thus obtaining the mapping model; where the fitted quadratic function is:

[0090] ;

[0091] in, The physical length of the biopsy forceps body. Here, a represents the pixel length of the corresponding biopsy forceps body, and a, b, and c are fitting coefficients determined using fitting techniques such as least squares. , , The value of is such that the fitting function can best fit the data;

[0092] (3.7) Obtain a universally applicable homography matrix;

[0093] (i) Collect the n homography matrices obtained in step (3.5), each homography matrix has r elements, r=9;

[0094] (i) For each homography matrix, the first (r-1) elements are arranged in order to form an (r-1)-dimensional eigenvector, resulting in a total of n (r-1)-dimensional eigenvectors;

[0095] (i) Cluster the n (r-1) dimensional feature vectors to find the largest and most concentrated cluster;

[0096] (iv) Determine whether more than 85% of the (r-1) dimensional eigenvectors are located in the largest and most concentrated cluster. If so, proceed to the next step; otherwise, collect n homography matrices again.

[0097] (v) For all (r-1) dimensional eigenvectors in the largest and most concentrated cluster, calculate the median dimension by dimension to obtain an (r-1) dimensional median vector;

[0098] (vi) After the last element of the (r-1)-dimensional median vector, add an element 1 to obtain an r-dimensional eigenvector;

[0099] (vii) Restore the r-dimensional eigenvectors to matrix form to obtain the universally applicable homography matrix;

[0100] (4) Obtain in vivo endoscopic images when the biopsy forceps extend out of the endoscope channel, the forceps body is straight, the forceps jaws are closed and the forceps jaws are against the polyp;

[0101] (4.1) Cleaning the endoscope lens: On the basis of meeting the hospital's strict hygiene standards, gently wipe the endoscope lens with medical gauze to ensure that there is no residue on the surface. The cleaning process should be carried out in a sterile environment. After wiping, it can be disinfected with ultraviolet light to prevent cross-contamination.

[0102] (4.2) Stabilize the endoscope and accurately locate the polyp: When operating the endoscope, the doctor needs to keep the lens stable, and reduce the image blur caused by shaking by slowly adjusting the angle and position. Then, the lens is gradually moved to the target polyp position to ensure that the polyp is centered and the viewing angle is correct.

[0103] (4.3) Biopsy forceps close to polyp marking: When the closed jaws of the biopsy forceps are horizontally close to the polyp, maintain this action for 2-3 seconds to ensure the image is stable. Then, keep the biopsy forceps straight and slowly extend the forceps to press against the polyp. When the jaws press against the polyp, hold for another 2-3 seconds. During the operation, pay attention to controlling the force to avoid damaging the tissue, and at the same time ensure that the jaws are clearly visible in the image.

[0104] (4.4) Repeated data collection: To reduce the bias of a single data point, repeat steps (4.1) to (4.3) 3-5 times for a single polyp to obtain multiple sets of data. Each repetition follows a standardized process to ensure consistency of actions and records data independently.

[0105] (5) Use a universally applicable homography matrix to convert the in vivo endoscopic images into a bird's-eye view B;

[0106] (6) Obtain the pixel length of the biopsy forceps body in the bird's-eye view B and input it into the mapping model to obtain the physical length of the biopsy forceps body in the bird's-eye view B. Add this length to the physical length of the biopsy forceps head as the distance between the polyp and the endoscope lens. ;

[0107] (7) After outlining the polyp region near the jaw in the bird's-eye view B using a lightweight segmentation model, manual calibration was performed. Finally, the minimum bounding rectangle of the polyp outline was calculated using code, and its pixel size was recorded. The lightweight segmentation model used is Segment Anything Tiny.

[0108] (8) Calculate the physical size of the polyp. The calculation formula is as follows:

[0109] ;

[0110] ;

[0111] (The principle of this formula is as follows) Figure 9 (as shown)

[0112] In the formula, The physical size (horizontal or vertical) of the image captured by the endoscope lens. Focal length The pixel size (horizontal or vertical) of the image captured by the endoscope lens. The field of view (horizontal or vertical) is expressed in degrees. The equivalent focal length (horizontal or vertical) of the endoscope lens.

[0113] The dimensions in this invention refer to the length in the horizontal or vertical direction.

[0114] Example 1

[0115] The above-mentioned method for obtaining information about polyps under endoscopy based on monocular vision was used to obtain polyp information (e.g. Figure 8 The information shown is as follows:

[0116] The endoscope used in step (1) is an Olympus CF-H290I, and the horizontal physical length of its lens image is... It is 70mm, focal length The horizontal pixel length of the endoscope lens image is 14.3mm. The size is 1157px, the length of the forceps head is 11mm, and the physical distance between the two ends of the opening when the biopsy forceps are open is 5mm.

[0117] The extracorporeal endoscope images obtained in step (3.2) are as follows: Figure 2 As shown, m is 8;

[0118] In step (3.3), the selection of the clamp source point for the external endoscope image is as follows: Figure 3 As shown;

[0119] The converted bird's-eye view A in step (3.4) is as follows Figure 4 As shown;

[0120] In step (3.5), n is 13;

[0121] In the mapping model obtained by fitting in step (3.6), a is 0.02308, b is 3.27, and c is -0.001045;

[0122] The process of obtaining the universally applicable homography matrix in step (3.7) is as follows:

[0123] (i) Collect the 13 homography matrices obtained in step (3.5), each homography matrix having 9 elements;

[0124] (i) For each homography matrix, the first 8 elements are arranged in order to form an 8-dimensional eigenvector, resulting in a total of 13 8-dimensional eigenvectors; where the 8-dimensional eigenvector is represented as: v_i = [h11, h12, h13, h21, h22, h23, h31, h32];

[0125] (i.e., cluster the 13 8-dimensional feature vectors to find the largest and most concentrated cluster;

[0126] (iv) Determine whether more than 85% of the 8-dimensional eigenvectors are located in the largest and most concentrated cluster. If so, proceed to the next step; otherwise, recollect the 13 homography matrices.

[0127] (v) For all 8-dimensional feature vectors in the largest and most concentrated clusters, calculate the median dimension by dimension to obtain an 8-dimensional median vector; the specific process of calculating the median dimension by dimension is as follows: first, calculate the median of the first element of all 8-dimensional feature vectors in the largest and most concentrated clusters, and use it as the first element of the 8-dimensional median vector; then calculate the median of the second element of all 8-dimensional feature vectors in the largest and most concentrated clusters, and use it as the second element of the 8-dimensional median vector, and so on, to obtain the values ​​of each element in the 8-dimensional median vector, whose 8-dimensional median vector v_median = [ 11, 12, 13, twenty one, twenty two, twenty three, 31, 32];

[0128] (vi) After the last element of the 8-dimensional median vector, add an element 1 to obtain a 9-dimensional eigenvector, i.e., h_vector = [ 11, 12, 13, twenty one, twenty two, twenty three, 31, 32, 1];

[0129] (vii) Reconstructing the 9-dimensional eigenvectors into a 3×3 matrix form yields the universally applicable homography matrix; where the obtained universally applicable homography matrix... ;

[0130] The in vivo endoscopic images obtained in step (4) are as follows: Figure 5 As shown;

[0131] In step (5), a universally applicable homography matrix is ​​used to convert the in vivo endoscopic images into a format similar to... Figure 6 The bird's-eye view shown in Figure B;

[0132] In step (6), the pixel length of the biopsy forceps body in the bird's-eye view B is 11px, and its physical length is 3.288mm. The distance between the polyp and the endoscope lens is... It is 14.288mm;

[0133] The polyp outline obtained in step (7) is as follows Figure 7 As shown, its horizontal pixel length It is 99.66px;

[0134] The physical length of the polyp in the horizontal direction calculated in step (8) It is 6.02mm.

[0135] To verify the accuracy of the above-mentioned method for acquiring information about polyps under endoscopy based on monocular vision, the polyps were measured using the same type of endoscope and existing techniques. The specific process is as follows:

[0136] First, acquire clinical images of the biopsy forceps containing the polyp and in standard open position. Then, manually annotate the pixel distance between the two ends of the opening of the biopsy forceps when in standard open position. Then, combining the physical distance (5mm) between the two ends of the opening when the biopsy forceps are standardly open, the conversion factor of the current image is calculated. Then, based on this conversion factor, the pixel length in the horizontal direction of the polyp was... Converted to the physical length in the horizontal direction of the polyp The formula is as follows:

[0137] ;

[0138] .

[0139] The physical length of the polyp in the horizontal direction was calculated to be 6.15 mm using the above method. Comparing this with the result measured in Example 1, the error of the result measured in Example 1 is only 0.13 mm. This verifies that the polyp information measurement method based on monocular vision endoscopy of the present invention has higher accuracy and reliability, and can also provide a more accurate quantitative basis for clinical polyp assessment. At the same time, the output of the present invention simultaneously includes complete data containing the two-dimensional size of the polyp and its spatial distance from the endoscope lens. This directly makes up for the core deficiency of existing clinical datasets (such as Kvasir-SEG) which only have segmentation masks and lack depth information. It provides essential basic data with physical scale annotation for training depth estimation algorithms that can understand the geometric attributes of the endoscopic scene (such as monocular depth estimation and size-aware segmentation networks), thus laying a solid data foundation for finally realizing fully automatic and high-precision intraoperative measurement without relying on external equipment.

[0140] The difference between physical dimensions and actual dimensions in this invention is as follows:

[0141] Physical size: In scenarios involving the measurement and description of object dimensions, the emphasis is on indicating that the dimension value is measured in millimeters (mm). The focus is on the measurement standard, not whether the dimension is calculated or derived from actual measurement. The relative concept is pixel size. Physical length is similar to physical size.

[0142] Actual size: This focuses on describing the objective, true size of an object in the real world. It is not limited to a specific unit of measurement. The core is to represent the true size of the object as accurately as possible. It can be obtained through direct measurement or deduced from relevant information. Actual length is similar to actual size.

Claims

1. A method for acquiring information about polyps under endoscopy based on monocular vision, characterized in that, Information about the polyp includes the distance between the polyp and the endoscope lens, and the method for obtaining this information includes the following steps: (1) The specific steps for acquiring an in vivo endoscopic image when the biopsy forceps are extended from the endoscope channel, the forceps body is straight, the forceps jaws are closed and the forceps jaws are against the polyp, and then converting the external endoscopic image into a bird's-eye view A are as follows: (a) Extend the biopsy forceps out of the endoscope channel and keep the forceps straight, and record this process via video; (b) Extract m extracorporeal endoscope images from the video in chronological order from front to back, where m ≥ 8; (c) Obtain the homography matrix based on the m-th external endoscopic image; (d) Use the homography matrix to convert the first to (m-1) extracorporeal endoscope images into a bird's-eye view A; (e) Repeat steps (a) to (d) (n-1) times to obtain a total of n×(m-1) bird's-eye view images A and n homography matrices, where n≥13; (2) The intraoperative endoscopic image is converted into a bird's-eye view B using a universally applicable homography matrix. The steps for obtaining the universally applicable homography matrix are as follows: (i) Collect n homography matrices, each homography matrix having r elements, where r=9; (ii) For each homography matrix, the first (r-1) elements are arranged in order to form an (r-1)-dimensional eigenvector, resulting in a total of n (r-1)-dimensional eigenvectors; (iii) Cluster the n (r-1) dimensional feature vectors to find the largest and most concentrated cluster; (iv) Determine whether more than 85% of the (r-1) dimensional eigenvectors are located in the largest and most concentrated cluster. If so, proceed to the next step; otherwise, collect n homography matrices again. (v) For all (r-1) dimensional eigenvectors in the largest and most concentrated cluster, calculate the median dimension by dimension to obtain an (r-1) dimensional median vector; (vi) After the last element of the (r-1)-dimensional median vector, add an element 1 to obtain an r-dimensional eigenvector; (vii) Restore the r-dimensional eigenvectors to matrix form to obtain the universally applicable homography matrix; (3) Obtain the pixel length of the biopsy forceps body in the bird's-eye view B and input it into the mapping model to obtain the physical length of the biopsy forceps body in the bird's-eye view B. Add this length to the physical length of the biopsy forceps head as the distance between the polyp and the endoscope lens. ; The mapping model is constructed as follows: After marking the scale along the length direction on the biopsy forceps, an external endoscope image is acquired when the biopsy forceps extend from the endoscope channel and the forceps body is straight. The external endoscope image is then converted into a bird's-eye view A. The pixel length of the biopsy forceps body in bird's-eye view A is then obtained. At the same time, the physical length of the biopsy forceps body in bird's-eye view A is determined according to the scale. Finally, a quadratic function is used to fit the pixel length of the biopsy forceps body in bird's-eye view A with the physical length of the biopsy forceps body in bird's-eye view A, thus obtaining the mapping model.

2. The method for acquiring information about polyps under endoscopy based on monocular vision according to claim 1, characterized in that, The process of step (c) is as follows: First, extract the pixel coordinates of the four vertices of the clamp from the m-th external endoscope image; Then, define a target rectangle in the bird's-eye view coordinate system. Its physical dimensions W and H are the physical dimensions of the part of the clamp body that is photographed. The physical coordinates of the four vertices of the target rectangle are (0,0), (W,0), (W,H), (0,H). Next, the physical coordinates of the four vertices of the target rectangle are multiplied by the scale factor to obtain the pixel coordinates of the four vertices of the target rectangle; Furthermore, the target rectangle is translated so that it is located in the center of the output bird's-eye view, thus obtaining the updated pixel coordinates of the four vertices of the target rectangle; Finally, the pixel coordinates of the four vertices of the clamp are directly linearly transformed with the pixel coordinates of the four vertices of the updated target rectangle using the DLT algorithm to calculate the homography matrix.

3. The method for acquiring information about polyps under endoscopy based on monocular vision according to claim 1, characterized in that, Information about the polyp also includes its physical size, and the method for obtaining this information includes the following steps: (4) After outlining the polyp area near the pincers in the bird's-eye view B, obtain the minimum bounding rectangle of the outline and record its pixel size. ; (5) Calculate the physical size of the polyp. The calculation formula is as follows: ； In the formula, This is the equivalent focal length of the endoscope lens.

4. The method for acquiring information about polyps under endoscopy based on monocular vision according to claim 3, characterized in that, A lightweight segmentation model was used to outline the polyp region near the pincers in the bird's-eye view B.

5. The method for acquiring information about polyps under endoscopy based on monocular vision according to claim 3, characterized in that, The calculation formula is as follows: ； In the formula, This refers to the pixel size of the image captured by the endoscope lens. The field of view is the angle of view.

6. The method for acquiring information about polyps under endoscopy based on monocular vision according to claim 5, characterized in that, The calculation formula is as follows: ； In the formula, The physical size of the image captured by the endoscope lens. It is the focal length.

Images