An amplification endoscope esophageal superficial squamous cell carcinoma grading method and system based on identifying IPCL

Abstract

This invention provides a novel magnifying endoscopic grading method and system for superficial esophageal squamous cell carcinoma based on IPCL recognition, relating to the fields of medical image processing and endoscopic-assisted diagnosis. This invention acquires narrowband or blue light imaging frames, uses a transparent cap for scale calibration, performs vascular enhancement, threshold segmentation, and morphological opening and closing operations on the images, extracts the vascular skeleton and calculates roughness indices, and simultaneously performs connected component analysis and maximum diameter measurement on avascular and stretched regions. Grading is determined by combining preset thresholds across multiple image frames, outputting the grading results and recording key parameters, thus achieving objective, accurate, stable, and repeatable automatic grading of invasion depth.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and endoscopic-assisted diagnosis technology, and in particular to a novel magnifying endoscopic grading method and system for superficial esophageal squamous cell carcinoma based on IPCL identification. Background Technology

[0002] The depth of invasion of superficial esophageal squamous cell carcinoma (SESCC) is closely related to the risk of lymph node metastasis and is a key factor in deciding whether to perform endoscopic submucosal dissection (ESD) or radical surgical resection.

[0003] The three currently mainstream clinical grading standards—Inoue, Arima, and the Japanese Esophageal Society (JES)—have low sensitivity in diagnosing mid-to-deep submucosal lesions (≥SM2) and tend to underestimate tumor stage. This leads to some patients receiving insufficient treatment or requiring secondary surgery, increasing medical risks and costs. The main reason for this problem is that the current standards are all subjective descriptions, and operators may lack experience, make subjective judgments, or provide consistent assessments. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a novel magnifying endoscopy grading method and system for superficial esophageal squamous cell carcinoma based on IPCL recognition. By organically combining scale calibration, vascular enhancement, morphological feature extraction, avascular area quantification, and multi-frame integrated judgment, it achieves objective, accurate, stable, and repeatable automatic grading of the invasion depth of superficial esophageal squamous cell carcinoma.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A novel magnifying endoscopic grading method for superficial squamous cell carcinoma of the esophagus based on IPCL identification includes:

[0007] Acquire image frames of narrowband imaging or blue light imaging under a magnifying endoscope, and establish the conversion relationship between pixels and millimeters using a scale calibration method; the scale calibration method includes either the transparent cap calibration method or the Zhang Zhengyou calibration method;

[0008] The image frame is enhanced with vascular structure enhancement, and threshold segmentation is performed on the enhancement result to obtain a binary mask of the blood vessels;

[0009] Morphological opening and closing operations are performed on the binary mask to remove noise and fill holes, and the vascular skeleton is extracted from the purification result after hole filling.

[0010] A roughness index reflecting the fluctuation of the diameter of the same blood vessel in the length direction is calculated based on the distance sequence to the blood vessel boundary along the blood vessel skeleton, and the blood vessel binary mask is inverted and connected domain analysis is performed to identify the non-blood vessel region and the stretch region surrounded by the rough irregular blood vessels, and the maximum inscribed circle diameter of each region is measured based on the conversion relationship;

[0011] When any of the following conditions is met in a plurality of consecutive valid image frames, the deep infiltration grade is output: the roughness index reaches or exceeds the index preset threshold, or the maximum inscribed circle diameter of the non-blood vessel region or the stretch region reaches or exceeds the length preset threshold; when the conditions are not met, the superficial grade is output;

[0012] The grading result is output, and the maximum roughness index, the maximum region diameter, the number of valid image frames, and the threshold parameters used for judgment are recorded.

[0013] Preferably, the establishment method of the conversion relationship comprises:

[0014] The first frame of image is read, and the pixel diameter of the inner edge of the transparent cap is measured ;

[0015] The conversion relationship is determined according to the formula ; wherein, The conversion relationship is the calculation formula for converting any pixel distance into millimeter value: millimeter value = pixel number × .

[0016] Preferably, the image frame is subjected to blood vessel structure enhancement, and threshold segmentation is performed on the enhancement result to obtain a binary mask of blood vessels, comprising:

[0017] The doctor selects the approximate range of the lesion on the display interface, and the image frame is cropped to obtain a processing region;

[0018] The three-channel color information in the processing region is converted into a single-channel grayscale image;

[0019] The grayscale histogram of the single-channel grayscale image corresponding to the image frame is counted and the cumulative distribution function is calculated;

[0020] A grayscale mapping table is generated based on the cumulative distribution function, and the original grayscale value is converted into a new grayscale value pixel by pixel according to the grayscale mapping table, to obtain an enhanced grayscale image with a histogram tending to be uniformly distributed;

[0021] The enhanced grayscale image is Gaussian smoothed at multiple pixel scales. The second derivative is calculated using the Sobel operator at each scale to form a Hessian matrix. The Frangi tubular structure response is calculated based on the eigenvalues ​​of the Hessian matrix, and the maximum value of the response at all scales is taken to generate a blood vessel probability map. The pixel scales include one pixel, two pixels, and three pixels.

[0022] Based on the blood vessel probability map, automatic global thresholding is used for segmentation, and doctors are allowed to fine-tune it using sliders to obtain a binary mask of blood vessels.

[0023] Preferably, morphological opening and closing operations are performed on the binary mask to denoise and fill holes, and the vascular skeleton is extracted from the purified result after hole filling.

[0024] A morphological opening operation is performed on the binary mask of the blood vessels to remove noise from small connected regions, and the opening operation result is obtained.

[0025] A morphological closing operation is performed on the opening operation result to fill the holes and smooth the boundaries, resulting in the cleaned-up result after hole filling.

[0026] Skeleton extraction is performed on the purification results to generate a vascular skeleton map for radius measurement and morphological feature statistics.

[0027] Preferably, a distance sequence from the vessel boundary is calculated along the vascular skeleton to obtain a roughness characterization index reflecting the degree of fluctuation in the diameter of the same vessel along its length. The binary mask is then inverted and connected component analysis is performed to identify avascular regions and stretched regions surrounded by large, irregular vessels. Based on the conversion relationship, the maximum inscribed circle diameter of each region is measured, including:

[0028] The shortest distance to the edge of the blood vessel is calculated point by point on the vascular skeleton, converted into actual length units according to the conversion relationship, forming a radius sequence, and a diameter sequence is obtained by doubling the radius.

[0029] For the diameter sequence of the same blood vessel, the average diameter and fluctuation / variance coefficient are calculated as roughness characterization indicators, and the curvature and ring integrity are statistically analyzed for auxiliary annotation of B1 and B2.

[0030] The binary mask of the blood vessel is inverted to obtain a non-blood vessel pixel image, and connected component analysis is performed on the non-blood vessel pixel image.

[0031] Identify continuous non-vascular regions, eliminate those that are too small or have shape noise, and retain candidate regions in the vicinity of the lesion center; when a candidate region is surrounded by large, irregular blood vessels with an average diameter of B2, forming a closed or semi-closed area, it is marked as a stretched area; those without obvious enclosure and with large areas of missing blood vessels are marked as avascular areas.

[0032] For each candidate region, calculate either the maximum inscribed circle diameter or the maximum shortest axis length, and use it as the region diameter of the candidate region.

[0033] The diameter of the region is converted to millimeters according to the conversion factor.

[0034] The largest diameter among all candidate regions is denoted as d.

[0035] When d is greater than 3 mm, the corresponding area is marked as a large avascular area or a large stretching area to indicate the danger signal of deep infiltration.

[0036] Preferably, the determination is performed within a series of consecutive valid image frames. When any valid image frame meets any of the following conditions, a deep wetting grade is output: the roughness characterization index reaches or exceeds a preset threshold, or the maximum inscribed circle diameter of the avascular region or stretchable region reaches or exceeds a preset length threshold; if the condition is not met, a superficial grade is output, including:

[0037] Read the vascular roughness index and the maximum diameter of the avascular area or stretched area, and record the conversion factor between pixels and millimeters; the unit of the maximum diameter is millimeters.

[0038] When the vascular roughness index or maximum diameter is missing, the image frame is marked as an undeterminable frame and is not included in the threshold comparison. If necessary, the frame will be prompted to be re-framed.

[0039] In each valid image frame, the blood vessel roughness index is compared with a preset roughness threshold, and the maximum diameter is compared with a preset size threshold.

[0040] In a series of valid image frames, with a number of no less than five frames, if any valid image frame meets the criteria of the vascular roughness index reaching or exceeding the roughness threshold, or the maximum diameter reaching or exceeding the size threshold, it is determined to be a deep infiltration grade; when none of the valid image frames meet either condition, it is determined to be a superficial grade.

[0041] Output the grading results, recording the maximum value of the vessel roughness index, the maximum value of the maximum diameter, the number of effective image frames, and the threshold parameters used;

[0042] The classification results, threshold comparison results, and related metadata are output to the results display module and written to the cache for storage.

[0043] Preferably, the Frangi tubular structure responds The calculation formula is:

[0044] ;

[0045] Among them, in terms of scale Calculating the second derivative on a Gaussian smoothed image The Hessian matrix, taking eigenvalues , and according to Sort; , For tubular similarity index; define noise suppression amount ;in, The adjustment parameter is positive for c; during multi-scale enhancement... Calculate on the sets of one pixel, two pixels, and three pixels respectively. The maximum value of the response at each scale is taken as the vascular probability map.

[0046] A novel magnifying endoscopic grading system for superficial esophageal squamous cell carcinoma based on IPCL recognition includes:

[0047] The image acquisition and scale calibration unit is used to acquire image frames of narrowband imaging or blue light imaging under a magnifying endoscope, and to establish the conversion relationship between pixels and millimeters using a scale calibration method; the scale calibration method includes either the transparent cap calibration method or the Zhang Zhengyou calibration method.

[0048] The blood vessel enhancement and segmentation unit is used to enhance the blood vessel structure of the image frame and perform threshold segmentation on the enhancement result to obtain a binary mask of the blood vessels.

[0049] The morphological purification and skeleton extraction unit is used to perform morphological opening and closing operations on the binary mask to remove noise and fill holes, and to extract the vascular skeleton from the purification result after hole filling.

[0050] The vascular roughness and avascular region analysis unit is used to calculate the distance sequence to the vascular boundary along the vascular skeleton, obtain a roughness characterization index reflecting the degree of fluctuation of the diameter of the same vascular vessel in the length direction, invert the vascular binary mask and perform connected component analysis to identify avascular regions and stretched regions surrounded by large and irregular vascular vessels, and measure the maximum inscribed circle diameter of each region based on the conversion relationship.

[0051] The grading determination unit is used to make a determination within a series of consecutive valid image frames. When any valid image frame meets any of the following conditions, it outputs a deep wetting grade: the roughness characterization index reaches or exceeds a preset threshold, or the maximum inscribed circle diameter of the avascular area or stretchable area reaches or exceeds a preset length threshold; if the condition is not met, it outputs a superficial grade.

[0052] The result output and recording unit is used to output the grading results and record the maximum roughness characterization index, maximum region diameter, number of effective image frames, and threshold parameters used for judgment.

[0053] The present invention discloses the following technical effects:

[0054] This invention establishes a conversion relationship between pixels and millimeters by introducing a transparent cap as a scale reference in the image acquisition process, so that the measurement of blood vessel diameter and avascular area size has a unified physical scale, ensuring the accuracy and comparability of the grading determination.

[0055] This invention employs color space conversion, histogram equalization, and multi-scale Frangi filtering in the blood vessel extraction stage to highlight small blood vessels and suppress noise interference, thereby obtaining a clearer and more stable binary mask and skeleton for blood vessels, providing a reliable data foundation for subsequent feature analysis.

[0056] In the process of quantifying vascular morphology, this invention statistically analyzes the distance sequence from skeleton points to vascular boundaries to form a vascular roughness index. At the same time, by combining the inversion of the binary mask and connected component analysis, it realizes the automatic identification and size quantification of avascular and stretched regions, thereby obtaining multi-dimensional objective indicators for classification.

[0057] This invention introduces a multi-frame integration mechanism in the grading determination process, which performs frame-by-frame threshold comparison and overall fusion on several consecutive valid image frames. It can output deep immersion grading in a timely manner when any abnormal feature is detected, while ensuring that shallow grading is output under the condition of overall consistency, thus taking into account both sensitivity and specificity.

[0058] In addition to providing the grading of infiltration depth in the result output stage, this invention also records information such as the maximum roughness index, the maximum region diameter, the number of effective image frames, and the threshold used, which facilitates the traceability of the grading results and their clinical reference value, and improves the practicality and reliability of intelligent endoscopy-assisted diagnosis. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 A flowchart of the method provided in an embodiment of the present invention;

[0061] Figure 2 This is a schematic diagram of the system structure provided in an embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] The purpose of this invention is to provide a novel endoscopic grading method and system for superficial squamous cell carcinoma of the esophagus based on IPCL identification. By organically combining scale calibration, vascular enhancement, morphological feature extraction, region analysis and multi-frame integration, a endoscopic grading technology for superficial squamous cell carcinoma of the esophagus that can achieve automation, quantification, stabilization and traceability is constructed.

[0064] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0065] Figure 1 The method flowchart provided in the embodiments of the present invention is as follows: Figure 1 As shown, this invention provides a novel magnifying endoscopic grading method for superficial squamous cell carcinoma of the esophagus based on IPCL identification, comprising:

[0066] Step 100: Acquire image frames of narrowband imaging or blue light imaging under a magnifying endoscope, and establish the conversion relationship between pixels and millimeters using a scale calibration method; the scale calibration method includes either the transparent cap calibration method or the Zhang Zhengyou calibration method;

[0067] Step 200: Enhance the blood vessel structure of the image frame and perform threshold segmentation on the enhancement result to obtain a binary mask of the blood vessels;

[0068] Step 300: Perform morphological opening and closing operations on the binary mask to remove noise and fill holes, and extract the vascular skeleton from the purified result after hole filling;

[0069] Step 400: Calculate the distance sequence to the blood vessel boundary along the vascular skeleton to obtain a roughness characterization index that reflects the degree of fluctuation of the diameter of the same blood vessel in the length direction. Invert the binary mask of the blood vessel and perform connected component analysis to identify avascular areas and stretched areas surrounded by large and irregular blood vessels. Measure the maximum inscribed circle diameter of each region based on the conversion relationship.

[0070] Step 500: Make a judgment within a series of consecutive valid image frames. When any valid image frame meets any of the following conditions, output the deep wetting grade: the roughness characterization index reaches or exceeds the preset threshold, or the maximum inscribed circle diameter of the avascular area or stretchable area reaches or exceeds the preset length threshold; if the condition is not met, output the superficial grade.

[0071] Step 600: Output the grading results and record the maximum roughness characterization index, maximum region diameter, number of effective image frames, and threshold parameters used for the determination.

[0072] Specifically, in this embodiment, the magnified NBI image collected by the doctor is first received, and the diameter of the transparent cap is used for pixel-to-millimeter calibration.

[0073] The specific steps are as follows:

[0074] Attach the transparent cap to the lens and focus on the center of the lesion.

[0075] Read the first frame and measure the pixel diameter of the cap edge. The pixel-to-millimeter calibration factor k = 2mm / (The calibration coefficient represents the conversion relationship).

[0076] Continuously write to the image buffer at a rate of 60fps.

[0077] The output of this embodiment is: 1920×1080 RGB frames and calibration coefficient k (µm / pixel).

[0078] Furthermore, the vascular morphology determination in this embodiment is as follows: receiving magnified endoscopic images (NBI / BLI mode) and pixel-to-millimeter calibration coefficient k from the acquisition module, and automatically depicting the outline of the Intrapapillary Capillary Loop (IPCL) through a series of image processing steps based on "enhancing blood vessels - segmentation - skeletonization - measurement", identifying and measuring avascular areas (AVA) and areas surrounded by stretched irregular vessels (SSIV), and using objective quantitative features to assist in the determination of vessels B1 and B2, providing input for subsequent deep infiltration (≥SM2) determination.

[0079] The inputs for this embodiment are: 1) the current magnified endoscope frame; 2) the pixel-to-millimeter conversion factor k (obtained by the transparent cap calibration).

[0080] The output of this embodiment is: a) binary image of blood vessel contour; b) blood vessel skeleton image; c) statistics such as average diameter and diameter fluctuation (roughness) of each blood vessel; d) AVA / SSIV region mask and its maximum diameter d (mm); e) B1 / B2 classification labels based on morphological features (for report display; deep infiltration determination is still based on B3 / AVA threshold).

[0081] Processing flow:

[0082] 1. Select the processing area (optional). The doctor taps on the screen to roughly define the area of ​​the lesion, and the system crops the image area.

[0083] 2. Color space conversion: compress the three-channel color information into a single-channel grayscale image for easier post-processing.

[0084] 3. Histogram equalization: The gray-level histogram of the image is statistically analyzed and the cumulative distribution function (CDF) is calculated. The original gray-level values ​​are mapped to new gray-level values ​​to achieve a uniform histogram distribution, which is used to enhance the contrast between small blood vessels and the background.

[0085] 4. Vascular enhancement filtering, specifically:

[0086] 1) For the input image At different scales Gaussian smoothing is applied to pixels (e.g., 1, 2, 3 pixels) to generate multi-scale images. .

[0087] For each scale The second derivative of the image is calculated using the Sobel operator to obtain the Hessian matrix:

[0088]

[0089] in , , , It is the second-order partial derivative of the image.

[0090] For the Hessian matrix of each pixel, solve for its eigenvalues. For each pixel, first smooth the image using Gaussian smoothing. Taking the second-order partial derivative, we get Hessian matrix ,in The eigenvalue expression of the matrix is Agreement absolute value The absolute value (i.e.) Frangi response functions use these two. Value calculation for "tubular similarity" and .

[0091] Calculate the Frangi response function (Frangi tubular structure response) using eigenvalues:

[0092]

[0093] in: Measure "tubular similarity" (the smaller the value, the more it resembles a blood vessel). Suppress noise, , Hyperparameters (can be selected) , ), for all scales response Take the maximum value to obtain the final enhanced angiography: .

[0094] 5. Threshold segmentation: Separate “vascular” and “non-vascular” pixels based on the vascular probability map, and use automatic global thresholding or doctor adjustment slider to fine-tune and obtain the initial binary vascular mask.

[0095] 6. Mask cleaning and morphological purification: Opening operations are used to remove isolated small points, and closing operations are used to fill in the pores inside the blood vessels and refine the boundaries, reducing noise false positives and ensuring the continuity of the blood vessel area.

[0096] 7. Extract the vascular skeleton: Extract a 1-pixel-wide "skeleton" line along the central axis of the blood vessel, while preserving the topological structure.

[0097] 8. Local radius measurement: Calculate the distance (i.e., radius) from each point on the skeleton to the edge of the blood vessel, and multiply it by the calibration factor k to obtain the true length unit (µm or mm).

[0098] 9. Statistical analysis of vascular morphological characteristics

[0099] For the diameter sequence of the same blood vessel (the diameter sequence is a series of diameters, sampled every 1 pixel in the vascular skeleton, and the distance r from that point to the nearest blood vessel edge is obtained by querying the distance transformation map). i Converted to actual length (multiplied by calibration coefficient), to reduce quantization noise, r i (Using three-point median filtering for smoothing, followed by calculation of mean μ and standard deviation σ) Calculate:

[0100] Average diameter (reflects the overall size of the blood vessel);

[0101] Volatility / Coefficient of Variation (Standard Deviation ÷ Mean);

[0102] Curvature (actual centerline length ÷ distance between two straight lines; >1 indicates curvature);

[0103] Loop integrity (closed loop ratio): In the vascular results after morphological purification and skeleton extraction, the ratio of the number of closed IPCL loops identified based on connected components and loop closure judgment to the total number of identifiable IPCL units is defined as:

[0104] ;

[0105] in This represents the number of closed IPCL rings. The total number of identifiable IPCL elements (obtained by connectivity analysis, after removing elements with excessively small area or shape noise). It is a dimensionless index.

[0106] IPCL ring presence rate: Using the lesion treatment area as the statistical scope, the ratio of the pixel set area labeled as an IPCL ring structure to the pixel set area of ​​the lesion region is defined as follows:

[0107] ;

[0108] in This is a set of ring-shaped pixels obtained from a binary blood vessel mask and loop closure determination. A set of pixels representing the lesion region selected by the doctor or cropped by the system; It is a dimensionless index.

[0109] Judgment Reference:

[0110] B1: Small average diameter (relative to normal background), low variability, preserved ring structure, and low curvature;

[0111] B2: Significantly larger than B1, exhibiting uneven diameter changes or bending; where B1: average diameter μ < 15µm, coefficient of variation R < 0.20, bending degree ≤ 1.15, ring integrity ≥ 70%. B2: 15µm ≤ μ < 30µm, or 0.20 ≤ R < 0.40, or 1.15 < bending degree ≤ 1.40;

[0112] 10. Extraction of non-vascular regions (preparation for AVA / SSIV analysis): Invert the vascular mask to obtain a "non-vascular pixel" image, and perform connected component analysis on it.

[0113] 11. Connected component labeling and region filtering: Identify all continuous non-vascular regions; remove regions that are too small or have noisy shapes; retain candidate regions that match the neighborhood of the lesion center. Distinguishing logic:

[0114] When surrounded by large blood vessels (with an average diameter significantly greater than (≥42μm) B2), forming a closed / semi-closed enclosure, it is designated as SSIV;

[0115] No obvious surrounding area, large area directly lacking blood vessels → denoted as AVA.

[0116] 12. AVA / SSIV size measurement: For each candidate region, calculate the maximum inscribed circle diameter or the maximum and minimum axis length, and take the larger of the two values ​​as the region diameter of the candidate region. Then, convert the region diameter from pixel units to millimeter units according to the conversion relationship between pixels and millimeters.

[0117] Output the maximum value d; if d > 3 mm, record it as AVA / SSIV-large (deep wetting danger signal).

[0118] 13. Module Output Organization

[0119] The following information is packaged and transmitted to the classification determination:

[0120] (Vascular outline) (Skeleton), statistics for each vessel (average diameter, variability, tortuosity, B1 / B2 labels), –AVA / SSIV region mask and maximum diameter d, roughness index for subsequent judgment (if included in system settings).

[0121] Furthermore, the automatic classification of B1 and B2 is only used for reporting layer descriptions; deep wetting (≥SM2) is ultimately determined according to the system master rule: B3 (rough, coarse) or AVA / SSIV-large (>3mm) is obtained.

[0122] Optionally, in this embodiment, based on the vascular roughness R and the maximum diameter d of AVA / SSIV output by the vascular morphology judgment module, and combined with the system preset threshold, the lesion invasion depth is classified in real time; the judgment result is divided into superficial (EP–SM1) and deep invasion (≥SM2), and the result is provided to the result display module and the data storage module.

[0123] Input includes:

[0124] 1. Vascular roughness R (dimensionless).

[0125] 2. Maximum diameter d (mm) of AVA / SSIV;

[0126] 3. Pixel-to-millimeter conversion factor k (used for recording; already in mm for threshold comparison);

[0127] 4. System preset threshold: Vascular roughness threshold AVA / SSIV size threshold (Default is 3.0mm, can be calibrated and adjusted on site).

[0128] The processing flow is as follows:

[0129] 1. Data reception: Read the R and d values ​​output by the blood vessel morphology determination module in the current frame.

[0130] 2. Data validity check: If R or d is missing, it is marked as "undeterminable frame" and the frame will not be included in the threshold comparison; if necessary, prompt to re-capture the scene.

[0131] 3. Single-frame comparison: Compare R with R in each valid frame. d and .

[0132] 4. Multi-frame integration: In a series of consecutive valid frames (≥5), as long as any frame meets the deep immersion judgment condition, ≥SM2 can be output in advance; if all frames do not reach the threshold, EP–SM1 is output.

[0133] 5. Grading determination: Generate the current real-time grading result of the lesion according to the judgment rules (see below), and record the maximum R and maximum d values ​​used in this comparison.

[0134] 6. Data Packaging: Output the classification results, corresponding threshold comparisons, number of valid frames, and other metadata, and pass them to the result display module; at the same time, write them to the cache for storage.

[0135] Variable description:

[0136] R: Vessel roughness; calculated by the vessel morphology judgment module according to R=σ / μ (σ is the standard deviation of the radius along the same vessel, and μ is the average radius).

[0137] d: The maximum inscribed circle diameter (mm) of the AVA or SSIV region, measured by the vessel morphology assessment module. If multiple candidate regions exist in the same frame, the maximum value is taken.

[0138] : Vascular roughness threshold, used to identify highly irregular and rough blood vessels (corresponding to clinical B3 feature); recommended initial value is 0.42, which can be adjusted according to the device or local training data.

[0139] AVA / SSIV size threshold; default 3.0mm, indicating that this size is highly suggestive of deep infiltration in avascular or surrounding areas.

[0140] Valid frames (optional parameter): Frames with clear images, successful blood vessel segmentation, and where R and d can be calculated.

[0141] The judgment rule is as follows:

[0142] In any valid frame, if the vessel roughness R ≥ If the B3 characteristic is present, the lesion is considered to be deeply infiltrated (≥SM2).

[0143] Or when in any valid frame, the maximum diameter d of AVA / SSIV is ≥ It is also judged as deep infiltration (≥SM2).

[0144] If none of the above conditions are met within the specified number of valid frames, it is determined to be shallow (EP–SM1).

[0145] The output is:

[0146] Immersion depth classification results (EP–SM1 or ≥SM2); may include the maximum R, maximum d, effective frame count, and threshold used in this study. , information.

[0147] The workflow of this embodiment is as follows:

[0148] ① The doctor switches to the magnified NBI field of view, and the acquired image is automatically entered into the system.

[0149] ②The endoscopic image acquisition module acquires images of the corresponding area with the assistance of the doctor, delineates capillaries, and completes scale calibration.

[0150] ③ The vascular morphology judgment module is used to identify and classify vascular structures. The IPCL grading judgment module calculates the vascular roughness index based on the identification results and generates a diagnostic report. The IPCL grading judgment module further compares the vascular roughness index and the region diameter with the system preset threshold, and outputs the grading result of the lesion accordingly.

[0151] Specifically, this embodiment demonstrates high diagnostic accuracy, with a sensitivity of 77% and a specificity of 98% for ≥SM2 lesions, significantly superior to the existing JES and Arima standards;

[0152] A control experiment was used as a comparative example to illustrate the significant progress achieved in this embodiment, as shown in Table 1:

[0153] Indicator JES classification 95% CI Arima classification 95% CI New system determination 95% CI Sensitivity (Sensitivity) 0.50 (0.30–0.70) 0.73 (0.52–0.88) 0.77 (0.56–0.91) Specificity (Specificity) 0.98 (0.94–1.00) 0.98 (0.94–1.00) 0.98 (0.93–1.00) Positive predictive value (PPV)* 0.3075 (0.078–1.2815) 0.4594 (0.1115–1.8121) 0.3154 (0.1011–0.9835) Negative predictive value (NPV)* 0.51 (0.35–0.75) 0.27 (0.15–0.52) 0.24 (0.12–0.48)

[0154] The meanings of the indicators are as follows:

[0155] Sensitivity: The proportion of lesions with a true ≥SM2 (pathologically positive) that are classified as "deeply invasive" by this grading system. The higher the sensitivity, the less likely a lesion will be missed.

[0156] Specificity: The proportion of lesions that are not ≥SM2 (pathologically negative) and are correctly classified as "non-deeply invasive" by this grading system. The higher the value, the fewer false alarms.

[0157] Positive predictive value (PPV): The probability that a lesion classified as "deeply invasive" by the grading system actually has a ≥SM2 level. It reflects the reliability of a positive result.

[0158] Negative predictive value (NPV): The probability that a lesion classified as "non-deeply invasive" by the grading system is actually not ≥SM2. It reflects the reliability of a negative result.

[0159] 95% CI (95% confidence interval): Under repeated sampling conditions, the true value of the indicator has a 95% probability of falling within this interval, which is used to reflect the uncertainty of the estimate.

[0160] Corresponding to the above methods, such as Figure 2 As shown. This embodiment also provides a novel magnifying endoscopic grading system for superficial esophageal squamous cell carcinoma based on IPCL recognition, including:

[0161] The image acquisition and scale calibration unit is used to acquire image frames of narrowband imaging or blue light imaging under a magnifying endoscope, and to establish the conversion relationship between pixels and millimeters using a scale calibration method; the scale calibration method includes either the transparent cap calibration method or the Zhang Zhengyou calibration method.

[0162] The blood vessel enhancement and segmentation unit is used to enhance the blood vessel structure of the image frame and perform threshold segmentation on the enhancement result to obtain a binary mask of the blood vessels.

[0163] The morphological purification and skeleton extraction unit is used to perform morphological opening and closing operations on the binary mask to remove noise and fill holes, and to extract the vascular skeleton from the purification result after hole filling.

[0164] The vascular roughness and avascular region analysis unit is used to calculate the distance sequence to the vascular boundary along the vascular skeleton, obtain a roughness characterization index reflecting the degree of fluctuation of the diameter of the same vascular vessel in the length direction, invert the vascular binary mask and perform connected component analysis to identify avascular regions and stretched regions surrounded by large and irregular vascular vessels, and measure the maximum inscribed circle diameter of each region based on the conversion relationship.

[0165] The grading determination unit is used to make a determination within a series of consecutive valid image frames. When any valid image frame meets any of the following conditions, it outputs a deep wetting grade: the roughness characterization index reaches or exceeds a preset threshold, or the maximum inscribed circle diameter of the avascular area or stretchable area reaches or exceeds a preset length threshold; if the condition is not met, it outputs a superficial grade.

[0166] The result output and recording unit is used to output the grading results and record the maximum roughness characterization index, maximum region diameter, number of effective image frames, and threshold parameters used for judgment.

[0167] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0168] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for grading superficial squamous cell carcinoma of the esophagus based on the identification of IPCL (intra-invasive peritoneal cavitary lesions), characterized in that, include: Acquire image frames of narrowband imaging or blue light imaging under magnifying endoscopy, and establish the conversion relationship between pixels and millimeters using a scale calibration method; The scale calibration method includes either the transparent cap calibration method or the Zhang Zhengyou calibration method; The image frame is enhanced with vascular structure enhancement, and threshold segmentation is performed on the enhancement result to obtain a binary mask of the blood vessels; Morphological opening and closing operations are performed on the binary mask to remove noise and fill holes, and the vascular skeleton is extracted from the purification result after hole filling. The distance sequence from the vascular skeleton to the vascular boundary is calculated to obtain a roughness characterization index that reflects the degree of fluctuation of the diameter of the same vascular vessel in the length direction. The binary mask of the vascular vessel is inverted and connected component analysis is performed to identify avascular areas and stretched areas surrounded by large and irregular vascular vessels. The maximum inscribed circle diameter of each region is measured based on the conversion relationship. The determination is performed within a series of consecutive valid image frames. When any valid image frame meets any of the following conditions, the deep wetting grade is output: the roughness characterization index reaches or exceeds the preset threshold, or the maximum inscribed circle diameter of the avascular area or stretchable area reaches or exceeds the preset length threshold; if the condition is not met, the superficial grade is output. Output the grading results and record the maximum roughness characterization index, maximum region diameter, number of effective image frames, and threshold parameters used for the determination. The distance sequence from the vessel boundary to the vascular skeleton is calculated to obtain a roughness index reflecting the degree of fluctuation in the diameter of the same vessel along its length. The binary mask is then inverted and connected component analysis is performed to identify avascular regions and stretched regions surrounded by large, irregular vessels. Based on the conversion relationship, the maximum inscribed circle diameter of each region is measured, including: The shortest distance to the edge of the blood vessel is calculated point by point on the vascular skeleton, converted into actual length units according to the conversion relationship, forming a radius sequence, and a diameter sequence is obtained by doubling the radius. For the diameter sequence of the same blood vessel, the average diameter and fluctuation / variance coefficient are calculated as roughness characterization indicators, and the curvature and ring integrity are statistically analyzed for auxiliary annotation of B1 and B2. The binary mask of the blood vessel is inverted to obtain a non-blood vessel pixel image, and connected component analysis is performed on the non-blood vessel pixel image. Identify continuous non-vascular regions, eliminate those that are too small or have shape noise, and retain candidate regions in the vicinity of the lesion center; when a candidate region is surrounded by large, irregular blood vessels with an average diameter of B2, forming a closed or semi-closed area, it is marked as a stretched area; those without obvious enclosure and with large areas of missing blood vessels are marked as avascular areas. For each candidate region, calculate either the maximum inscribed circle diameter or the maximum shortest axis length, and use it as the region diameter of the candidate region. The diameter of the region is converted to millimeters according to the conversion factor; The largest diameter among all candidate regions is denoted as d. When d is greater than 3 mm, the corresponding area is marked as a large avascular area or a large stretching area to indicate the danger signal of deep infiltration.

2. The method for grading superficial squamous cell carcinoma of the esophagus based on IPCL identification according to claim 1, characterized in that, The steps of the transparent cap calibration method include: Read the first frame image and measure the pixel diameter of the inner edge of the transparent cap. ; According to the formula Determine the conversion relationship; wherein, For the aforementioned conversion relationship, the formula for converting any pixel distance to a millimeter value is: Millimeter value = Number of pixels × .

3. The method for grading superficial esophageal squamous cell carcinoma based on IPCL identification according to claim 1, characterized in that, The image frame is enhanced with vascular structure enhancement, and threshold segmentation is performed on the enhancement result to obtain a binary mask for the blood vessels, including: On the display interface, the doctor selects the approximate area of ​​the lesion, and the image frame is cropped to obtain the processed area. Convert the three-channel color information within the processing area into a single-channel grayscale image; Statistical analysis of gray-level histograms and calculation of cumulative distribution functions are performed on the single-channel gray-level images corresponding to the image frames; A grayscale mapping table is generated based on the cumulative distribution function. The original grayscale values ​​are converted into new grayscale values ​​pixel by pixel according to the grayscale mapping table, resulting in an enhanced grayscale image with a histogram that tends to be uniformly distributed. The enhanced grayscale image is Gaussian smoothed at multiple pixel scales. The second derivative is calculated using the Sobel operator at each scale to form a Hessian matrix. The Frangi tubular structure response is calculated based on the eigenvalues ​​of the Hessian matrix, and the maximum value of the response at all scales is taken to generate a blood vessel probability map. The pixel scales include one pixel, two pixels, and three pixels. Based on the blood vessel probability map, automatic global thresholding is used for segmentation, and doctors are allowed to fine-tune it using sliders to obtain a binary mask of blood vessels.

4. The method for grading superficial esophageal squamous cell carcinoma based on IPCL identification according to claim 1, characterized in that, Morphological opening and closing operations are performed on the binary mask to denoise and fill holes, and the vascular skeleton is extracted from the purified result after hole filling. A morphological opening operation is performed on the binary mask of the blood vessels to remove noise from small connected regions, and the opening operation result is obtained. A morphological closing operation is performed on the opening operation result to fill the holes and smooth the boundaries, resulting in the cleaned-up result after hole filling. Skeleton extraction is performed on the purification results to generate a vascular skeleton map for radius measurement and morphological feature statistics.

5. The method for grading superficial esophageal squamous cell carcinoma based on IPCL identification according to claim 1, characterized in that, The determination is made within a series of valid image frames. When any valid image frame meets any of the following conditions, the deep wetting grade is output: the roughness characterization index reaches or exceeds the preset threshold, or the maximum inscribed circle diameter of the avascular area or stretchable area reaches or exceeds the preset length threshold. If the conditions are not met, output a shallow classification, including: Read the vascular roughness index and the maximum diameter of the avascular area or stretched area, and record the conversion factor between pixels and millimeters; the unit of the maximum diameter is millimeters. When the vascular roughness index or maximum diameter is missing, the image frame is marked as an undecidable frame and is not included in the threshold comparison. In each valid image frame, the blood vessel roughness index is compared with a preset roughness threshold, and the maximum diameter is compared with a preset size threshold. In a series of valid image frames, with a number of no less than five frames, if any valid image frame meets the criteria of the vascular roughness index reaching or exceeding the roughness threshold, or the maximum diameter reaching or exceeding the size threshold, it is determined to be a deep infiltration grade; when none of the valid image frames meet either condition, it is determined to be a superficial grade. Output the grading results, recording the maximum value of the vessel roughness index, the maximum value of the maximum diameter, the number of effective image frames, and the threshold parameters used; The classification results, threshold comparisons, and related metadata are output to the results display module and written to the cache for storage.

6. The method for grading superficial squamous cell carcinoma of the esophagus based on IPCL recognition according to claim 3, characterized in that, The Frangi tubular structure response The calculation formula is: ； Among them, in terms of scale Calculating the second derivative on a Gaussian smoothed image The Hessian matrix, taking eigenvalues , and according to Sort; , For tubular similarity index; define noise suppression amount. ;in, The adjustment parameter is positive for c; during multi-scale enhancement... Calculate on the sets of one pixel, two pixels, and three pixels respectively. The maximum value of the response at each scale is taken as the vascular probability map.

7. A magnifying endoscopic grading system for superficial squamous cell carcinoma of the esophagus based on IPCL recognition, characterized in that, include: The image acquisition and scale calibration unit is used to acquire image frames of narrowband imaging or blue light imaging under a magnifying endoscope, and to establish the conversion relationship between pixels and millimeters using a scale calibration method; the scale calibration method includes either the transparent cap calibration method or the Zhang Zhengyou calibration method. The blood vessel enhancement and segmentation unit is used to enhance the blood vessel structure of the image frame and perform threshold segmentation on the enhancement result to obtain a binary mask of the blood vessels. The morphological purification and skeleton extraction unit is used to perform morphological opening and closing operations on the binary mask to remove noise and fill holes, and to extract the vascular skeleton from the purification result after hole filling. The vascular roughness and avascular region analysis unit is used to calculate the distance sequence to the vascular boundary along the vascular skeleton, obtain a roughness characterization index reflecting the degree of fluctuation of the diameter of the same vascular vessel in the length direction, invert the vascular binary mask and perform connected component analysis to identify avascular regions and stretched regions surrounded by large and irregular vascular vessels, and measure the maximum inscribed circle diameter of each region based on the conversion relationship. The grading determination unit is used to make a determination within a series of consecutive valid image frames. When any valid image frame meets any of the following conditions, it outputs a deep wetting grade: the roughness characterization index reaches or exceeds a preset threshold, or the maximum inscribed circle diameter of the avascular area or stretchable area reaches or exceeds a preset length threshold; if the condition is not met, it outputs a superficial grade. The result output and recording unit is used to output the grading results and record the maximum roughness characterization index, maximum region diameter, number of effective image frames, and threshold parameters used for judgment. The distance sequence from the vessel boundary to the vascular skeleton is calculated to obtain a roughness index reflecting the degree of fluctuation in the diameter of the same vessel along its length. The binary mask is then inverted and connected component analysis is performed to identify avascular regions and stretched regions surrounded by large, irregular vessels. Based on the conversion relationship, the maximum inscribed circle diameter of each region is measured, including: The shortest distance to the edge of the blood vessel is calculated point by point on the vascular skeleton, converted into actual length units according to the conversion relationship, forming a radius sequence, and a diameter sequence is obtained by doubling the radius. For the diameter sequence of the same blood vessel, the average diameter and fluctuation / variance coefficient are calculated as roughness characterization indicators, and the curvature and ring integrity are statistically analyzed for auxiliary annotation of B1 and B2. The binary mask of the blood vessel is inverted to obtain a non-blood vessel pixel image, and connected component analysis is performed on the non-blood vessel pixel image. Identify continuous non-vascular regions, eliminate those that are too small or have shape noise, and retain candidate regions in the vicinity of the lesion center; when a candidate region is surrounded by large, irregular blood vessels with an average diameter of B2, forming a closed or semi-closed area, it is marked as a stretched area; those without obvious enclosure and with large areas of missing blood vessels are marked as avascular areas. For each candidate region, calculate either the maximum inscribed circle diameter or the maximum shortest axis length, and use it as the region diameter of the candidate region. The diameter of the region is converted to millimeters according to the conversion factor; The largest diameter among all candidate regions is denoted as d. When d is greater than 3 mm, the corresponding area is marked as a large avascular area or a large stretching area to indicate the danger signal of deep infiltration.

Images