A Method and System for Estimating Polyp Image Size Based on Deep Learning Model

Abstract

This invention discloses a method and system for polyp image size estimation based on a deep learning model. The method includes: acquiring multiple image frames; using the frame with the largest maximum pixel diameter as a representative frame; estimating the depth value of the target polyp using a depth estimation model; calculating a representative depth value using a median-based approach; and calculating the product of the representative pixel diameter, the representative depth value, and a given scaling factor to obtain the estimated size of the target polyp. This invention provides a unified polyp size estimation method that combines a depth estimation model and a segmentation model. By estimating the absolute depth-size scaling factor, it eliminates the need for explicit camera calibration. Simultaneously, by acquiring and analyzing multiple viewpoints and selecting the frame that presents the largest apparent diameter, it measures the polyp cross-section from an approximately vertical viewpoint, effectively solving the influence of angle issues and providing more accurate and reliable polyp size measurements.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for estimating the size of polyp images based on a deep learning model. Background Technology

[0002] The discovery of polyps in endoscopic images has significant medical value, not only helping to analyze the risk of lesions but also providing crucial reference material for doctors to determine subsequent treatment plans.

[0003] After a polyp is detected in an image, accurately estimating its size is a crucial step, as size is an important piece of reference information. For example, smaller polyps often indicate a lower risk of malignancy, while larger polyps often indicate an increased risk of malignancy.

[0004] Of course, polyp size is not the only standard for diagnosing a disease, but only a reference standard. The specific assessment and diagnosis of the type, stage and potential risks of the disease need to be made by a professional doctor. The assessment of polyp size only plays a reference role.

[0005] However, even though it is reference data, accurate assessment of the size of polyp targets in the image is still very important.

[0006] With the development of artificial intelligence and image processing technologies, automated polyp size measurement methods are becoming an effective auxiliary tool to improve the objectivity and consistency of measurements. These technologies combine deep learning algorithms and 3D reconstruction methods, making polyp size estimation more accurate. Existing methods typically combine depth estimation models with 2D images, but often require prior knowledge of the endoscope's intrinsic parameters (especially focal length) to accurately convert from pixels to millimeters. These parameters need to be adjusted according to different cameras, and in some practical applications, they may not be available. Furthermore, to avoid geometric errors, the optical axis of the camera is usually required to be as perpendicular as possible to the largest cross-section of the polyp; otherwise, the diameter measured in the image will not accurately reflect the true size of the polyp. However, in practice, it is difficult to control the optical axis of the endoscope to be completely perpendicular to the largest cross-section of the polyp. This leads to the problem that in some cases where vertical polyp images are not available, it is impossible to assess the size or the size assessment error is large. Summary of the Invention

[0007] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a method and system for estimating the size of polyp images based on a deep learning model.

[0008] To achieve the aforementioned objectives, the technical solution adopted by this invention includes:

[0009] In a first aspect, the present invention provides a method for estimating the size of polyp images based on a deep learning model, comprising:

[0010] Acquire multiple image frames containing the target polyp from multiple different perspectives;

[0011] The image frame with the largest maximum pixel diameter occupied by the target polyp is selected as the representative frame, and the corresponding maximum pixel diameter is used as the representative pixel diameter.

[0012] A depth estimation model is used to estimate the depth value of each pixel occupied by the target polyp. Based on the depth values ​​corresponding to multiple pixels, a representative depth value is calculated using a median-based approach.

[0013] The size estimate of the target polyp is obtained by calculating the product of the representative pixel diameter, the representative depth value, and a given scaling factor, wherein the scaling factor is determined by the camera optical parameters that captured the image frame.

[0014] Secondly, the present invention also provides a polyp image size estimation system based on a deep learning model, for implementing the above-mentioned polyp image size estimation method, comprising:

[0015] The image acquisition module is used to acquire multiple image frames containing the target polyp from multiple different perspectives;

[0016] The representative diameter module is used to select the image frame with the largest maximum pixel diameter occupied by the target polyp as the representative frame, and the corresponding maximum pixel diameter is used as the representative pixel diameter.

[0017] The representative depth module is used to estimate the depth value of each pixel occupied by the target polyp using a depth estimation model, and calculate a representative depth value based on the depth values ​​corresponding to multiple pixels using a median-based method.

[0018] The size estimation module is used to calculate the product of the representative pixel diameter, the representative depth value, and a given scaling factor to obtain the size estimate of the target polyp, wherein the scaling factor is determined by the camera optical parameters that captured the image frame.

[0019] Thirdly, the present invention also provides a readable storage medium storing a computer program, which, when run, performs the steps of the polyp image size estimation method described above.

[0020] Based on the above technical solution, compared with the prior art, the beneficial effects of the present invention include at least the following:

[0021] This invention provides a unified polyp size estimation method that combines depth estimation and segmentation models. By estimating the absolute depth-size scaling factor, it eliminates the need for explicit camera calibration. Simultaneously, by acquiring and analyzing multiple viewpoints and selecting the frame that presents the largest apparent diameter, it measures the polyp cross-section from an approximately perpendicular viewpoint, effectively addressing the impact of angle issues. This invention utilizes state-of-the-art segmentation and depth estimation models, combined with an improved diameter calculation algorithm, to provide more accurate and reliable polyp size measurements. This improvement provides clinicians with more accurate reference information, helping to more rationally formulate subsequent management and treatment strategies.

[0022] The above description is merely an overview of the technical solution of the present invention. In order to enable those skilled in the art to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described below in conjunction with detailed drawings. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the first step of the polyp image size estimation method provided in a typical embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of the second step of the polyp image size estimation method provided in a typical embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of the third step of the polyp image size estimation method provided in a typical embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the fourth step of the polyp image size estimation method provided in a typical embodiment of the present invention. Detailed Implementation

[0027] In view of the shortcomings of the prior art, the inventors of this invention, through long-term research and extensive practice, have proposed the technical solution of this invention. The following will further explain and illustrate this technical solution, its implementation process, and its principles.

[0028] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0029] Moreover, relational terms such as “first” and “second” are used merely to distinguish one component or method step from another that has the same name, and do not necessarily require or imply any such actual relationship or order between these components or method steps.

[0030] This invention provides a method for estimating the size of polyp images based on a deep learning model (hereinafter referred to as the "method"), which includes the following steps:

[0031] Acquire multiple image frames containing the target polyp from multiple different perspectives;

[0032] The image frame with the largest maximum pixel diameter occupied by the target polyp is selected as the representative frame, and the corresponding maximum pixel diameter is used as the representative pixel diameter.

[0033] A depth estimation model is used to estimate the depth value of each pixel occupied by the target polyp. Based on the depth values ​​corresponding to multiple pixels, a representative depth value is calculated using a median-based approach.

[0034] The size estimate of the target polyp is obtained by calculating the product of the representative pixel diameter, the representative depth value, and a given scaling factor, wherein the scaling factor is determined by the camera optical parameters that captured the image frame.

[0035] In some implementations, the process of acquiring multiple image frames specifically includes:

[0036] Provide a video stream and continuously identify the target polyp in the video stream using an image segmentation model;

[0037] When the target polyp is identified, multiple video frames showing the target polyp at different locations in the image are extracted as the image frames.

[0038] Whether during or in images recorded after an endoscopic examination, the above method is a feasible implementation of this invention and is not necessarily limited to simultaneous execution during the endoscopic examination. It should be noted that this invention is a technical solution for information processing and parameter extraction based on images in a computer, based on optical imaging principles and computer principles, not biological or medical principles. It is not a method for diagnosing or treating diseases, and the data obtained are merely assessment values ​​of physical dimensions, not direct diagnostic evidence. For example, doctors cannot easily determine the type, stage, or prognosis of a disease based solely on the estimated dimensions provided by this invention; they must combine other key factors, such as morphology and blood flow, for evaluation. Ultimately, the gold standard remains the pathology report, not the endoscopic image.

[0039] In the aforementioned technical solutions, to achieve polyp size estimation and simultaneously integrate depth estimation and segmentation into a single process, thereby overcoming the limitations of existing methods, this invention proposes a calibration-free and angle-insensitive polyp diameter measurement method. As a typical example, this method can combine advanced target segmentation models (such as YoloSeg, MaskRcnn) with Transformer-based monocular depth estimation models (such as DepthAnythingV2), and ensure reliable depth measurement through multi-stage fine-tuning. By eliminating the need for manual calibration and minimizing inaccuracies caused by camera angle, the polyp size measurement process is significantly simplified and can be easily applied to routine endoscopic images.

[0040] To ensure accuracy in the assessment, it is generally required that the shooting distances (i.e., the physical distance between the camera lens and the polyp) of multiple image frames be roughly consistent to facilitate statistical filtering in each step. Specifically, in some implementation schemes, the shooting distances of the multiple image frames are generally required to fluctuate within ±25%. Of course, in practical applications, this value may not be the limitation and can be adjusted appropriately according to the performance of the endoscope.

[0041] Alternatively, limitations can be imposed by the camera's imaging principle. In some implementations, when selecting image frames, the lens-to-polyp shooting distance is kept consistent across multiple image frames, and the target polyp image is clear in all selected image frames. When the focal length remains constant and the image is generally clear, the shooting distance is usually relatively fixed, thus ensuring that the shooting distance of each image frame does not change significantly. Regarding how to determine image clarity, there are numerous existing technical solutions, which will not be listed here. Generally, clarity is ultimately determined by ensuring that polyp image segmentation can be achieved.

[0042] This invention innovates in four core aspects, further improving the practicality and accuracy of real-time polyp size measurement.

[0043] One of the core innovations lies in the overall process framework. As a typical example, the main steps of the above method include:

[0044] Step 1: After the segmentation model detects a polyp in the image, it maintains a roughly constant distance from the polyp and moderately tilts the endoscope camera in both the vertical and horizontal directions to acquire multiple image frames, thereby covering different viewing angles.

[0045] Step 2: For the selected image frame, use the polyp boundary point set obtained from the segmentation model in Step 1 to calculate the convex hull of the boundary point set. The maximum Euclidean distance between any two pixels on the convex hull in the image coordinate system is the maximum pixel diameter of the polyp.

[0046] Step 3: Perform depth estimation on the entire image using a depth estimation model. Then, based on the polyp mask obtained in Step 1, retain the depth value corresponding to the polyp region, ignore the depth value of the background, and use a median-based outlier filtering method to obtain the representative depth value of the polyp (e.g., in millimeters).

[0047] Step 4: Combining the maximum pixel diameter obtained in Step 2, the representative depth value obtained in Step 3, and the scaling factor designed in this invention, substitute these parameters into the polyp size estimation formula to calculate the millimeter-level size of the polyp in real space (not limited to this, it can also be the size in other units, this invention only uses millimeters as a representative example).

[0048] Regarding the specific process of each of the above steps, in some implementation schemes, the calculation process of the maximum pixel diameter specifically includes:

[0049] Construct the smallest convex polygon that surrounds all pixels of the target polyp;

[0050] Calculate the pixel distance between each vertex and the remaining vertices in the minimum convex polygon;

[0051] The maximum value among all the pixel distances is selected as the maximum pixel diameter.

[0052] In order to improve efficiency and robustness in the process of calculating the maximum diameter and representative depth, embodiments of the present invention have designed and implemented a convex hull-based method to obtain the maximum pixel diameter. This method can more accurately and efficiently complete the polyp size measurement.

[0053] More specifically, in some implementations, the calculation process for the maximum pixel diameter is expressed as follows:

[0054] P={(x1,y1),(x2,y2),…,(x n y n )}

[0055] H(P)={(X1,Y1),(X2,Y2),…,(X m Y m )}

[0056]

[0057] Where P represents the set of boundary pixels of the target polyp, (x., y.) represents the pixel coordinates of the boundary pixels, H(P) represents the minimum convex polygon, (x., y.) represents the coordinates of the vertices, and W... max Let n represent the maximum pixel diameter, n represent the number of boundary pixels, m represent the number of vertices, and i and j represent natural numbers.

[0058] To address the angle-related issues in traditional 2D methods and to select a view that is approximately perpendicular to the largest cross-section of the polyp, a typical implementation of this invention designs and implements a method: by analyzing image frames acquired from different perspectives, the frame that is closest to being perpendicular to the polyp cross-section is automatically selected as the approximately perpendicular frame.

[0059] That is, in some implementations, the process of selecting the representative frame is expressed as:

[0060] F = {F1, F2, ..., F} l}

[0061]

[0062] Where F represents the set of image frames, Fi represents the image frame, and Fi represents the image frame. * W represents the representative frame. k Let l represent the maximum pixel diameter of the k-th image frame, l represent the number of image frames, and k represent a natural number.

[0063] Furthermore, this invention employs a median outlier filtering method to obtain representative depth.

[0064] That is, in some implementations, the calculation process of the representative depth value is expressed as follows:

[0065] M0 = median{d p |p∈Q}

[0066]

[0067] Q′={d p ∈Q||d p -M0|≤3σ0}

[0068] d rep =median{d p |p∈Q}

[0069] Where M0 represents the initial median, d pThis represents the depth estimate of pixel p corresponding to the target polyp, where p represents the pixel, Q represents the set of pixels corresponding to the target polyp, σ0 represents the standard deviation, |Q| represents the number of pixels corresponding to the target polyp, Q′ represents the filtered set of interior points, and d rep This represents the representative depth value.

[0070] To eliminate the need for manual calibration, embodiments of the present invention design and implement a method for converting pixel measurements into true-size scaling factors. This scaling factor can be estimated by training a linear model using known polyp true-size data. Once determined, the scaling factor can be used in the same imaging environment.

[0071] That is, in some implementations, the size estimate is calculated as follows:

[0072] D mm =α×W max ×d rep

[0073] Among them, D mm Let α represent the size estimate, and let α represent the scaling factor. The process of determining the scaling factor includes:

[0074] Extract the corresponding representative pixel diameter and representative depth values ​​from multiple training images, calculate their product as training data, and obtain the ground truth size corresponding to the training images;

[0075] Calculate the initial scaling factor based on the multiple training data and the ground truth size;

[0076] Calculate the size estimate in any training image according to the initial scaling factor, and calculate the residual between the size estimate and the true size.

[0077] Based on the statistics of the residuals corresponding to multiple training images, outliers whose average distance from the residual exceeds 3 times the standard value are removed to obtain retained points;

[0078] Based on the reserved points, the final scaling factor is calculated using the corresponding multiple training data and ground truth size.

[0079] In some implementations, the process of determining the scaling factor is expressed as follows:

[0080]

[0081] r t =f t -α(init)e t

[0082]

[0083] Where α(init) represents the initial scaling factor, e t f represents the training data. t The true value size is represented by t, the sample index is represented by T, and the initial sample set is represented by N. T r represents the total number of samples in set T. t Represents the residual, μ r σ represents the average value of the residuals. r Let S represent the standard deviation of the residuals, and let N represent the set of retained points. S Let α represent the total number of samples in set S. final This represents the final scaling factor.

[0084] A second aspect of this invention provides a polyp image size estimation system based on a deep learning model, used to implement the polyp image size estimation method provided in any of the above embodiments, comprising:

[0085] The image acquisition module is used to acquire multiple image frames containing the target polyp from multiple different perspectives;

[0086] The representative diameter module is used to select the image frame with the largest maximum pixel diameter occupied by the target polyp as the representative frame, and the corresponding maximum pixel diameter is used as the representative pixel diameter.

[0087] The representative depth module is used to estimate the depth value of each pixel occupied by the target polyp using a depth estimation model, and calculate a representative depth value based on the depth values ​​corresponding to multiple pixels using a median-based method.

[0088] The size estimation module is used to calculate the product of the representative pixel diameter, the representative depth value, and a given scaling factor to obtain the size estimate of the target polyp, wherein the scaling factor is determined by the camera optical parameters that captured the image frame.

[0089] This invention also provides a readable storage medium storing a computer program, which, when run, performs the steps of the polyp image size estimation method provided in any of the above embodiments.

[0090] The technical solution of the present invention will be further described in detail below through several embodiments and in conjunction with the accompanying drawings. However, the selected embodiments are only for illustrating the present invention and do not limit the scope of the present invention.

[0091] Example 1

[0092] This embodiment proposes a precise polyp diameter measurement method that requires no calibration and is insensitive to angle. Specifically, this method combines an advanced target segmentation model (such as YoloSeg or MaskRcnn; this embodiment can use multiple target segmentation models and methods, and is not limited to the specific examples) with a Transformer-based monocular depth estimation model (this embodiment uses DepthAnythingV2, but is not limited to this), to obtain relatively accurate depth information for the target location. By eliminating the need for manual calibration and minimizing errors caused by changes in camera angle, this method ultimately simplifies the clinical procedure and can be seamlessly integrated into common endoscopic surgeries.

[0093] The following are the main steps of this embodiment:

[0094] Step 1: As Figure 1 As shown, the system automatically selects the best-aligned frame; once the segmentation model S detects a polyp in the endoscopic video stream, the system acquires multiple frames. Let F = {F1, F2, ..., F...} l They were captured at different tilt angles while maintaining a relatively constant distance. For each frame F, the system measured the pixel diameter of the polyp using a convex hull-based method. The frame corresponding to the maximum pixel diameter is the view most orthogonal to the polyp cross-section, therefore the system selects a representative frame:

[0095]

[0096] That is, the image frame with the largest diameter is selected as the representative frame.

[0097] In the above steps, after inputting image I, a binary mask M∈{0,1} is output. H×W To mark polyp areas. In actual endoscopic examinations, the system acquires a series of frames F = {F1, F2, ..., F...}. l}, where each frame F is captured at a specific tilt angle. By keeping the distance to the polyp approximately constant, the apparent size change caused by the angle change can be decoupled from the size change caused by the distance change. For each frame F, the polyp edge point set P = {(x i y i Then calculate the maximum diameter d of the frame |i = 1, ..., n. i When the camera tilts around its vertical and horizontal axes, the system records the frame with the largest measured diameter, which is the viewing angle closest to being perpendicular to the main cross-section of the polyp.

[0098] This embodiment does not pursue absolute orthogonality, but rather aims to minimize measurement errors caused by angular deviations. Rather than relying on "guessing" a suitable angle during endoscopic operation, the method of this invention can automatically and reliably identify the angle that maximizes the polyp diameter measurement.

[0099] Step 2: As Figure 2 As shown, calculate or retain the maximum pixel diameter of the polyp already calculated on the selected representative frame; calculate the convex hull diameter. Let:

[0100] P={(x i y i |i = 1, ..., n}

[0101] The system calculates the convex hull of the polyp boundary points (pixel coordinates) obtained in step 1 of the segmentation model.

[0102] H(P) = {(X1, Y1), ..., (X...} m Y m )}

[0103] in Let W be the vertex of the convex hull. To obtain the maximum diameter of the polyp in pixel coordinates, calculate the maximum Euclidean distance between the vertices of the convex hull, denoted as W. max , which is the maximum pixel diameter of the polyp.

[0104] To improve the efficiency and robustness of calculating the maximum diameter and representative depth, this embodiment designs and implements a method for extracting the maximum pixel diameter based on the convex hull and calculating the representative depth based on median filtering. In the maximum diameter calculation, the convex hull can first be calculated using the shapely library in Python. The convex hull is the smallest convex polygon containing all points. Calculating the distance only at the vertices of the convex hull is more efficient than calculating it at all boundary points and has better robustness to concave boundaries because the convex hull ignores interior points or depressions that are irrelevant to the maximum span. Subsequently, the maximum Euclidean distance between two points among the convex hull vertices is found.

[0105]

[0106] This value represents the maximum chord length in pixel coordinates, which in turn represents the maximum pixel diameter of the polyp.

[0107] Step 3: As Figure 3 As shown, the representative depth of the polyp is calculated on a selected frame; the depth is predicted and the representative depth of the polyp region is calculated. Let... Let the pixel plane containing the input endoscopic image be... For the depth estimation model, each pixel x∈Ω is mapped to a depth value in millimeters. Let... Given the polyp binary mask obtained in step 1, we have:

[0108] S = {D(x)|x∈M}.

[0109] The representative depth d is obtained by filtering S using a median-based outlier filtering technique. rep , denoted as d rep =median(S filtered ).

[0110] In the representative depth calculation, let M be the binary mask for the polyp obtained in step 1, and define Q = {p|p∈M} as the set of pixels p within the mask region, corresponding to the region where the polyp is located. For each pixel within the region, there is a depth d in millimeters on the depth map. p .

[0111] a. Initial median:

[0112] M0 = median{d p |p∈Q}

[0113] The median is less sensitive to outliers than the mean, therefore M0 serves as a stable initial value for polyp depth.

[0114] b. Outlier filtering: Calculate the variance around M0:

[0115]

[0116] c. Define the set of interior points:

[0117] Q′={d p ∈Q||d p -M0|≤3σ0}

[0118] Any pixel depth that differs from M0 by more than 3σ0 is considered an outlier and is discarded.

[0119] d. Final representative depth:

[0120] d rep =median{d p |p∈Q}

[0121] This final median value excludes extreme depth values, providing a stable and accurate estimate of the distance between the polyp and the endoscope.

[0122] Step 4: As Figure 4 As shown, the results are converted to polyp dimensions in millimeters. Let α be a known scaling factor used to convert pixel measurements to millimeters. The maximum two-dimensional pixel diameter of the polyp is W. max The representative depth of the polyp obtained in the previous step is d. rep(in millimeters). What is the diameter D of the polyp in real space? mm The calculation formula is:

[0123] D mm =α×W max ×d rep

[0124] This method is most accurate when the camera's optical axis is almost perpendicular to the plane of the polyp's cross-section, and the camera is kept at an appropriate distance from the polyp (avoiding getting too close or using zoom operations that change the focal length); if these conditions are violated, the scaling factor will not work accurately.

[0125] To avoid camera calibration, this embodiment designs and implements a method for estimating the scaling factor from pixel to true size. Endoscopic images follow perspective projection; targets of different depths will appear at different scaling ratios in the image. The principle is as follows:

[0126] Let PolypSize(pixels) represent the diameter of the polyp in pixel coordinates, and let Depth(mm) represent the corresponding depth estimate (millimeters). The actual polyp size can be expressed as...

[0127]

[0128] Since directly calibrating the focal length is often inconvenient, it can be defined as follows:

[0129]

[0130] Therefore, the above equation can be rewritten as:

[0131] RealSize(mm)=α×PolypSize(pixels)×PredictedDepth(mm).

[0132] The goal of this step is to estimate the scaling factor α using data that has already been calibrated for polyp size. This scaling value can be reused in subsequent operations without changing the optical properties of the camera.

[0133] Therefore, in the specific implementation, a large number of observation pairs were collected. in:

[0134] yt = the marked actual size of the polyp (mm).

[0135] The main technical approach of introducing a linear model y = α × x is to use least squares estimation:

[0136]

[0137] in It has a closed-form solution.

[0138] However, to improve the stability of the scaling factor, this embodiment also employs a residual statistics-based method to remove outlier samples:

[0139] a. Initial fit: using all sample pairs (e t f t We use this to calculate the initial α(init).

[0140] b. Residual calculation:

[0141] r t =f t -α(init)e t

[0142] c. Standardized residuals: Let μ r With σ r The residuals are {r} i Calculate the mean and standard deviation of}.

[0143]

[0144] And remove any (e t f t ) satisfies |z t |>3 points.

[0145] d. Quadratic fitting: Recalculate α on the remaining set of interior points after filtering to obtain the final estimate of α. final .

[0146] Once α is obtained final This method allows for reuse in subsequent examinations without changing the camera's optical parameters. It eliminates the need for explicit focus calibration. The process is performed offline in a single step, rather than dynamically during the endoscopic procedure. This method is most accurate when the camera's optical axis is nearly perpendicular to the polyp section and maintained at a suitable distance; however, the accuracy of the scaling factor decreases if the lens is too close or if magnification causes a significant change in focus.

[0147] The general operating principle of the above implementation plan is as follows:

[0148] by This represents the set of multiple endoscopic images acquired. Let s be the target segmentation model, which outputs a set of polyp boundary points for the k-th frame. Then calculate the convex hull of these points:

[0149]

[0150] The polygon formed by this subset of boundary points is obtained. The pixel diameter of the k-th frame is defined as:

[0151]

[0152] Where ||·|| represents the Euclidean norm. Then, the frame with the largest diameter is selected.

[0153] Deep models For each pixel in the selected image, predict a reliable depth Z. Apply a mask to polyp regions to filter corresponding depth values, ignore background depth values, and obtain a representative depth d using a median-based outlier filtering algorithm. rep Finally, the scaling factor α and representative depth d were obtained by fitting the polyp size under the same type of lens. rep The pixel measurements are converted to true dimensions to obtain an estimate of the polyp's diameter.

[0154]

[0155] Based on the above embodiments, it is clear that the technical solutions provided by the embodiments of the present invention bring many significant benefits, including:

[0156] Accurate, calibration-free polyp size measurement: Precise polyp size measurement can be achieved without manually acquiring endoscopic camera parameters.

[0157] Angle-insensitive analysis: Automatically identifies the image frame closest to vertical at different angles to ensure the acquisition of the maximum apparent diameter.

[0158] Simplify end-to-end workflow: Integrate advanced target segmentation and depth estimation models to simplify the polyp size measurement process.

[0159] By reliably capturing and calculating polyp size in real time, this invention enables more consistent and accurate risk stratification of suspicious lesions, reduces inconsistencies caused by operator differences, and allows for more informed decisions regarding the need for additional diagnostic or treatment interventions. This invention significantly improves the efficiency and reliability of routine endoscopy.

[0160] It should be understood that the above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for estimating the size of polyp images based on a deep learning model, characterized in that, include: Acquire image frames containing the target polyp from multiple different perspectives; Select an image frame with the largest pixel diameter occupied by the target polyp as a representative frame, and the corresponding largest pixel diameter as the representative pixel diameter; A depth estimation model is used to estimate the depth value of each pixel occupied by the target polyp. Based on the depth values ​​corresponding to multiple pixels, a representative depth value is calculated using a median-based approach. The size estimate of the target polyp is obtained by multiplying the representative pixel diameter, the representative depth value, and a given scaling factor. The size estimate is calculated as follows: D mm =α×W max ×d rep Among them, D mm Let α represent the estimated size, and W represent the scaling factor. max The maximum pixel diameter, d rep This represents the representative depth value; The process of determining the scaling factor includes: Extract the corresponding representative pixel diameter and representative depth values ​​from multiple training images, calculate their product as training data, and obtain the ground truth size corresponding to the training images; Calculate the initial scaling factor based on the multiple training data and the ground truth size; Calculate the size estimate in any training image according to the initial scaling factor, and calculate the residual between the size estimate and the true size. Based on the statistics of the residuals corresponding to multiple training images, outliers whose average distance from the residual exceeds 3 times the standard value are removed to obtain retained points; Based on the reserved points, the final scaling factor is calculated using the corresponding multiple training data and ground value size; The process of determining the scaling factor is expressed as follows: r t =f t -α(init)e t Where α(init) represents the initial scaling factor, e t f represents the training data. t The true value size is represented by t, the sample index is represented by T, and the initial sample set is represented by N. T r represents the total number of samples in set T. t The residual, μ r σ represents the average value of the residuals. r Let S represent the standard deviation of the residuals, and let N represent the set of retained points. S Let α represent the total number of samples in set S. final This represents the final scaling factor.

2. The polyp image size estimation method according to claim 1, characterized in that, The process of acquiring multiple image frames specifically includes: Provide a video stream and continuously identify the target polyp in the video stream using an image segmentation model; When the target polyp is identified, multiple video frames showing the target polyp at different positions and angles in the image are extracted as the image frames.

3. The polyp image size estimation method according to claim 2, characterized in that, The shooting distance of the multiple image frames fluctuates within ±25%. When selecting an image frame, the shooting distance from the lens to the polyp is kept consistent across the multiple image frames, and the image of the target polyp in the selected multiple image frames is clear.

4. The polyp image size estimation method according to claim 1, characterized in that, The calculation process for the maximum pixel diameter specifically includes: Construct the smallest convex polygon that surrounds all pixels of the target polyp; Calculate the pixel distance between each vertex and the remaining vertices in the minimum convex polygon; The maximum value among all the pixel distances is selected as the maximum pixel diameter.

5. The polyp image size estimation method according to claim 4, characterized in that, The calculation process for the maximum pixel diameter is expressed as follows: P={(x1,y1),(x2,y2),…,(x n ,y n )} H(P)={(X1,Y1),(X2,Y2),…,(X m ,Y m )} Where P represents the set of boundary pixels of the target polyp, (x · ,y · ) represents the pixel coordinates of the boundary pixel, H(P) represents the minimum convex polygon, (X) · ,Y · W represents the coordinates of the vertex. max Let n represent the maximum pixel diameter, n represent the number of boundary pixels, m represent the number of vertices, and i and j represent natural numbers.

6. The polyp image size estimation method according to claim 1, characterized in that, The process of selecting the representative frame is expressed as follows: F={F1,F2,…,F l } Where F represents the set of image frames, F · F represents the image frame. * W represents the representative frame. k Let l represent the maximum pixel diameter of the k-th image frame, l represent the number of image frames, and k represent a natural number.

7. The polyp image size estimation method according to claim 1, characterized in that, The calculation process for the representative depth value is expressed as follows: M0=median{d p |p∈Q} d rep =median{d p |p∈Q′} Where M0 represents the initial median, d p This represents the depth estimate of pixel p corresponding to the target polyp, where p represents the pixel, Q represents the set of pixels corresponding to the target polyp, σ0 represents the standard deviation, |Q| represents the number of pixels corresponding to the target polyp, Q′ represents the filtered set of interior points, and d rep This represents the representative depth value.

8. A polyp image size estimation system based on a deep learning model, used to implement the polyp image size estimation method according to any one of claims 1-7, characterized in that, include: The image acquisition module is used to acquire image frames containing the target polyp from multiple different perspectives; The representative diameter module is used to select an image frame with the largest pixel diameter occupied by the target polyp as a representative frame, and the corresponding largest pixel diameter is used as the representative pixel diameter. The representative depth module is used to estimate the depth value of each pixel occupied by the target polyp using a depth estimation model, and calculate a representative depth value based on the depth values ​​corresponding to multiple pixels using a median-based method. The size estimation module is used to calculate the product of the representative pixel diameter, the representative depth value, and a given scaling factor to obtain the size estimate of the target polyp, wherein the scaling factor is determined by the camera optical parameters that captured the image frame.

Images