Digestive tract tumor endoscopic image intelligent auxiliary diagnosis and grading system

Abstract

The invention discloses an intelligent auxiliary diagnosis and grading system for gastrointestinal tumor endoscopic images, which belongs to the technical field of medical image processing and computer-aided diagnosis and comprises a multi-modal image preprocessing and segmentation module, a lesion feature extraction and diagnosis module, a grading and depth evaluation module and a closed-loop feedback optimization module. The system receives white light, a narrow band and an amplified endoscopic image, adaptive segmentation is performed to obtain a lesion area, mucous membrane morphology, capillary and gland features are extracted, Paris classification, Vienna classification and infiltration depth evaluation are realized, and segmentation parameters are subjected to closed-loop optimization according to classification confidence. And intelligent auxiliary support is provided for early diagnosis and treatment decision of gastrointestinal tumors.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and computer-aided diagnosis technology, specifically to an intelligent auxiliary diagnosis and grading system for endoscopic images of gastrointestinal tumors. This system is based on deep learning and image processing technology to achieve automatic identification, grading, and invasion depth assessment of gastrointestinal tumors, and is suitable for the auxiliary diagnosis of early gastrointestinal tumors during gastrointestinal endoscopy. Background Technology

[0002] Gastrointestinal tumors are among the most prevalent and deadly malignant tumors worldwide. According to statistics from the World Health Organization, stomach cancer ranks fifth in global cancer incidence and fourth in cancer mortality. The incidence of colorectal cancer has been rising steadily in recent years, becoming a major disease posing a serious threat to human health. Early detection and timely treatment are crucial for improving the survival rate of patients with gastrointestinal tumors, and gastrointestinal endoscopy is the gold standard for early diagnosis of these tumors.

[0003] During digestive endoscopy, physicians need to identify lesion areas, determine the nature of the lesions, and grade them. The Paris classification system is an internationally widely used endoscopic classification method for superficial lesions of the digestive tract. This classification categorizes lesions into raised, superficial, and depressed types based on their macroscopic morphology, with different morphological types associated with the risk of submucosal invasion and lymph node metastasis. The Vienna classification system, on the other hand, is a grading system based on histological characteristics, classifying digestive tract epithelial tumors into five grades, from non-neoplastic lesions to invasive tumors. This classification helps guide clinical treatment decisions. Furthermore, accurately assessing the depth of tumor invasion is crucial for determining suitability for endoscopic treatment. Intramucosal carcinoma can be radically cured with endoscopic submucosal dissection, while lesions extending into the submucosal layer may require surgical treatment.

[0004] However, gastrointestinal endoscopy is highly dependent on the experience and skill of the endoscopist. Studies have shown that the rate of missed diagnosis of early gastrointestinal tumors by endoscopists can reach 11% to 26%, and the accuracy rate in assessing the depth of lesion invasion is only 60% to 80%. This subjectivity and variability in diagnosis leads to the missed diagnosis of some early lesions or inaccurate assessment of the depth of lesion invasion, affecting the choice of treatment and patient prognosis. In addition, endoscopists are prone to visual fatigue during long examinations, further increasing the risk of missed and misdiagnosed lesions.

[0005] The accompanying prior art document CN119963462A discloses a digestive endoscopy image enhancement and analysis system. This system includes an image acquisition module, a transmission module, a storage module, an enhancement adjustment module, and an output module. The enhancement adjustment module comprises an enhanced contrast difference unit, an adaptive histogram equalization unit, and a dynamic range compression adjustment unit. It enhances digestive endoscopy images by calculating enhancement factors, equalization adjustment coefficients, and dynamic range compression values. This technical solution primarily adjusts the brightness, contrast, and dynamic range of images to improve image quality, but it has the following shortcomings: the system only implements image enhancement functions and lacks the ability to identify lesion areas, thus failing to automatically detect and locate lesions; the system does not provide lesion classification and grading functions, and cannot assess lesions according to the Paris and Vienna classification standards; the system lacks the ability to assess tumor invasion depth, and cannot provide a basis for determining endoscopic treatment indications; the system adopts a unidirectional image processing flow, lacks a feedback optimization mechanism, and cannot dynamically adjust image processing parameters based on diagnostic results.

[0006] In recent years, artificial intelligence (AI) technology has made significant progress in the field of medical image analysis. Deep learning models, especially convolutional neural networks (CNNs), have demonstrated excellent performance in image recognition, object detection, and image segmentation tasks. Multiple studies have shown that deep learning-based computer-aided diagnostic systems can achieve sensitivity exceeding 92% and specificity exceeding 85% in the early detection of gastrointestinal tumors, with some systems achieving diagnostic accuracy close to or exceeding that of expert endoscopists. These studies confirm the enormous potential of AI technology in assisting gastrointestinal endoscopic diagnosis.

[0007] However, existing AI-assisted diagnostic systems still have some limitations. Most systems only implement lesion detection, lacking a complete grading and assessment system. Lesion detection can only tell physicians where abnormalities are found, but cannot provide crucial information such as the nature, severity, and depth of invasion of the lesion, which is essential for treatment decisions. Existing systems do not adequately utilize the fusion of multimodal endoscopic images. In clinical practice, multiple imaging modalities such as white light endoscopy, narrow-band imaging, and magnifying endoscopy are commonly used to observe lesions. Different modalities provide complementary information, but existing systems are mostly designed for single-modal images, failing to fully utilize multimodal information to improve diagnostic accuracy. The image segmentation methods in existing systems mostly use fixed parameters, lacking adaptive adjustment capabilities, resulting in unstable segmentation performance when faced with images from different patients, different locations, and different imaging conditions. Existing systems generally employ a unidirectional processing flow, with a linear process from image input to diagnostic output, lacking a feedback optimization mechanism based on diagnostic results. They cannot dynamically adjust the front-end image processing parameters according to diagnostic confidence, which limits further performance improvements.

[0008] Therefore, there is an urgent need to develop an intelligent auxiliary diagnostic system that integrates lesion identification, grading assessment, and depth prediction functions. This system should be able to process multimodal endoscopic images, provide Paris and Vienna classification results that conform to international standards, accurately assess the depth of tumor invasion, and have adaptive optimization capabilities to meet the needs of clinical practice. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the prior art and provide an intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors. This system achieves automatic identification, Paris classification, Vienna classification, and invasion depth assessment of gastrointestinal tumors through deep coupling and synergy of multimodal image preprocessing and segmentation modules, lesion feature extraction and diagnosis modules, grading and depth assessment modules, and closed-loop feedback optimization modules. Furthermore, it dynamically optimizes image segmentation parameters through a closed-loop feedback mechanism, significantly improving diagnostic accuracy and system robustness.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors includes a multimodal image preprocessing and segmentation module, a lesion feature extraction and diagnosis module, a grading and depth assessment module, and a closed-loop feedback optimization module.

[0012] The multimodal image preprocessing and segmentation module receives white light endoscopic images, narrowband imaging images, and magnified endoscopic images. It preprocesses the endoscopic images and segments them to obtain lesion regions. Preprocessing includes illumination compensation and noise suppression. Illumination compensation eliminates brightness variations caused by uneven endoscopic light sources, while noise suppression removes noise introduced during image acquisition. Segmentation employs an adaptive thresholding method that dynamically determines the segmentation threshold based on local image features. This method automatically adjusts the threshold to adapt to different image characteristics, exhibiting better robustness compared to fixed threshold methods.

[0013] The lesion feature extraction and diagnosis module is connected to the multimodal image preprocessing and segmentation module to extract mucosal surface morphological features, microvascular configuration features, and glandular structure features from the lesion area, and to identify the lesion type based on these features. Mucosal surface morphological features include the degree of elevation, depth of depression, and boundary clarity; these features reflect the macroscopic morphology of the lesion and are closely related to Paris classification. Microvascular configuration features include vessel density, vessel orientation, and vessel morphology parameters. Neovascularization in lesion tissue exhibits irregular orientation and abnormal morphology; analyzing microvascular features can determine the malignancy of the lesion. Glandular structure features include the regularity of glandular arrangement and the uniformity of gland size. Normal glands are regularly arranged and uniform in size, while tumor tissue has disordered glandular structure and uneven size; analyzing glandular structure features can assess the histological characteristics of the lesion.

[0014] The grading and depth assessment module is connected to the lesion feature extraction and diagnosis module. It is used for Paris and Vienna classification based on lesion type and characteristics, and to assess tumor invasion depth. The Paris classification determines the macroscopic morphological type of the lesion, classifying it into different types based on whether it is raised, depressed, or flat. The Vienna classification determines the histological grading of the lesion, classifying it into five grades based on cellular atypia and invasion status. Tumor invasion depth is determined by analyzing the degree of abnormality in microvascular architecture and glandular structure. Accurate assessment of invasion depth is crucial for determining suitability for endoscopic treatment.

[0015] A closed-loop feedback optimization module is connected to the hierarchical and depth evaluation module and the multimodal image preprocessing and segmentation module. It is used to evaluate segmentation quality based on the confidence levels of Paris and Vienna classifications. When the confidence level falls below a preset threshold, a feedback signal is generated to adjust the segmentation threshold and preprocessing parameters of the multimodal image preprocessing and segmentation module, achieving dynamic optimization of the segmentation parameters. This feedback signal is transmitted to the multimodal image preprocessing and segmentation module via a data interface for parameter updates. This closed-loop feedback mechanism enables the system to automatically adjust image processing parameters based on the reliability of the diagnostic results, improving the system's adaptability and overall performance.

[0016] The beneficial effects of this invention are as follows:

[0017] This invention processes images from white light endoscopy, narrowband imaging, and magnifying endoscopy using a multimodal image preprocessing and segmentation module. By fully utilizing the complementary information from different imaging modalities, it obtains richer lesion information compared to single-modal images, improving the accuracy and comprehensiveness of lesion detection. Multimodal image fusion technology enables the system to simultaneously observe the macroscopic morphology and microscopic structure of lesions, which is crucial for accurate lesion classification and assessment.

[0018] This invention extracts three types of features—mucosal morphology, microvessels, and glands—through a lesion feature extraction and diagnosis module, constructing a comprehensive lesion feature description system. These three types of features characterize lesion properties from three levels: macroscopic morphology, vascular distribution, and tissue structure, respectively. The features are complementary and can provide multi-dimensional information about the lesion. The lesion identification accuracy based on these multi-dimensional features is significantly higher than methods based on only a single feature.

[0019] This invention implements internationally standardized Paris and Vienna classifications through a grading and depth assessment module, providing clinicians with standardized diagnostic results. The Paris classification guides the selection of endoscopic treatment techniques, while the Vienna classification guides treatment plan decisions. The combined use of these two classifications enables a comprehensive assessment of the nature and severity of lesions. The tumor invasion depth assessment function of this invention is based on the degree of abnormality in microvascular and glandular characteristics, offering higher accuracy compared to traditional morphology-based methods, and providing a reliable basis for determining indications for endoscopic treatment.

[0020] This invention achieves adaptive optimization of the system by dynamically adjusting image segmentation parameters based on diagnostic confidence through a closed-loop feedback optimization module. When the confidence of the diagnostic result is low, it indicates that the current image segmentation may not be accurate enough. At this time, the system automatically adjusts the segmentation parameters and re-segments until a high-confidence diagnostic result is obtained. This closed-loop feedback mechanism enables the system to automatically cope with different image qualities and different lesion characteristics, significantly improving the robustness and stability of the system.

[0021] This invention achieves synergistic efficiency through the deep coupling of four core modules. There are clear data and control flows between the modules: the output of the segmentation module serves as the input to the feature extraction module, the output of the feature extraction module serves as the input to the hierarchical evaluation module, and the output of the hierarchical evaluation module feeds back to the segmentation module to adjust parameters, forming a complete closed-loop system. This deep coupling design allows the modules to mutually promote each other, resulting in a non-linear increase in overall system performance and achieving a synergistic effect where 1+1 is greater than 2. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the overall architecture of the intelligent assisted diagnosis and grading system for digestive tract tumor endoscopic images of the present invention;

[0023] Figure 2 This is a schematic diagram of the structure of the multimodal image preprocessing and segmentation module of the present invention;

[0024] Figure 3 This is a schematic diagram of the lesion feature extraction and diagnosis module of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of the grading and depth evaluation module of the present invention;

[0026] Figure 5 This is a schematic diagram of the working process of the closed-loop feedback optimization module of the present invention. Detailed Implementation

[0027] Please refer to the attached document. Figures 1-5The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0028] Reference Figure 1 The intelligent assisted diagnosis and grading system for gastrointestinal tumor endoscopic images provided by this invention includes a multimodal image preprocessing and segmentation module 1, a lesion feature extraction and diagnosis module 2, a grading and depth assessment module 3, and a closed-loop feedback optimization module 4. These modules are connected via data interfaces, forming a complete closed-loop system from image input to diagnostic output and parameter feedback. The system receives multimodal images acquired during endoscopic examinations, performs a series of intelligent processing steps, and outputs a standardized diagnostic report, including Paris classification, Vienna classification, and invasion depth assessment results for the lesions.

[0029] Reference Figure 2 The multimodal image preprocessing and segmentation module 1 includes an image preprocessing unit and an adaptive segmentation unit. The image preprocessing unit is responsible for preprocessing the input endoscopic image, including illumination normalization and denoising. Illumination normalization uses the Retinex algorithm to separate the illumination and reflection components of the image. By adjusting the illumination component, it achieves image brightness balance, eliminating the phenomenon of a bright center and dark periphery caused by uneven endoscopic light source. In a specific embodiment, a multi-scale Retinex algorithm is used, selecting three scale parameters of 15, 80, and 250, and weighted averaging the Retinex outputs at different scales with weights of 0.2, 0.5, and 0.3, respectively. This multi-scale processing can achieve global illumination balance while preserving image details. Denoising uses an adaptive filtering method, automatically selecting the filtering intensity based on the variance of local image regions. Regions with larger variance are considered to contain more detail information and are filtered weaker to preserve details; regions with smaller variance are considered smooth regions and are filtered stronger to remove noise. Preferably, a bilateral filtering algorithm is used, which considers both spatial distance and grayscale difference, and can preserve edge information while smoothing noise. The spatial filtering parameter of the bilateral filter is set to 5, and the grayscale filtering parameter is set to 75. This set of parameters achieves a good balance between noise reduction effect and detail preservation.

[0030] The adaptive segmentation unit employs an adaptive thresholding method based on local variance to segment lesion regions. This method first divides the image into multiple image blocks and calculates the local variance and global mean for each block. Local variance reflects the fluctuation in pixel grayscale within an image block; lesion regions typically exhibit higher local variance due to abnormal tissue structure. The global mean reflects the average grayscale level of the entire image. For each image block, a segmentation threshold is determined based on its local variance and global mean. Specifically, the segmentation threshold calculation formula is:

[0031] ,

[0032] in, The segmentation threshold for the i-th image patch. The global mean of the image. Let be the square root of the local variance of the i-th image patch, i.e., the standard deviation. This is an adjustment coefficient. In a preferred embodiment of the invention, The value range is from 0.1 to 0.3, with a preferred value of 0.2. This formula indicates that regions with larger local variance have higher segmentation thresholds, while regions with smaller local variance have lower segmentation thresholds, thus achieving the goal of adaptively adjusting the threshold based on the local characteristics of the image. This threshold is applied to binarize each image block; pixels with gray values ​​greater than the threshold are marked as foreground (lesion areas), and pixels with gray values ​​less than the threshold are marked as background (normal tissue). The segmentation results of all image blocks are integrated to obtain a complete lesion area segmentation mask. To remove isolated noise and holes in the segmentation results, morphological processing is performed on the segmentation mask, including opening and closing operations. The opening operation uses a 3×3 structuring element, which can remove small isolated regions; the closing operation uses a 5×5 structuring element, which can fill small holes within the region. The segmentation result after morphological processing is smoother and more complete, which is beneficial for subsequent feature extraction.

[0033] In a specific application example, for a 1920×1080 pixel white light endoscope image, the image preprocessing unit first performs illumination normalization, reducing the ratio of center brightness to edge brightness from 2.3 to 1.1, achieving good brightness balance. Subsequently, bilateral filtering denoising is performed, increasing the signal-to-noise ratio from 24.5 dB to 32.8 dB, effectively removing noise while maintaining the clarity of lesion boundaries. The adaptive segmentation unit divides the image into 32×32 pixel blocks, totaling 1920 blocks. The local standard deviation of each block is calculated, ranging from 8.3 to 45.7, reflecting the differences in texture complexity across different regions of the image. The global mean of the image is 127.6. According to the adaptive thresholding formula, the segmentation threshold for each block ranges from 129.3 to 136.7, demonstrating the adaptive adjustment characteristics of the threshold. Applying these thresholds for segmentation, a lesion region with an area of ​​18,600 pixels was successfully segmented. This region was located in the upper left quadrant of the image and had an irregular shape. Morphological opening operations removed 23 isolated noise points with an area of ​​less than 50 pixels, and closing operations filled 7 holes with an area of ​​less than 100 pixels, finally yielding a smooth and complete lesion region segmentation result.

[0034] Reference Figure 3The lesion feature extraction and diagnosis module 2 includes a multi-scale feature extraction unit and a lesion recognition unit. The multi-scale feature extraction unit uses multi-scale convolution to extract lesion features at different spatial scales and performs feature fusion. Lesions exhibit different features at different scales; small-scale features reflect detailed information such as microvascular morphology, while large-scale features reflect overall information such as macroscopic morphology. Multi-scale feature fusion provides a more comprehensive description of the lesion. The multi-scale feature extraction uses a feature pyramid network architecture, which includes feature extraction paths at multiple scales. In a preferred embodiment of the invention, three scale feature extraction paths are used. The first path uses a 3×3 convolution kernel with a stride of 1 to extract fine-scale features, which capture the direction and morphological details of microvessels. The second path uses a 5×5 convolution kernel with a stride of 2 to extract medium-scale features, which describe the arrangement and size distribution of glands. The third path uses a 7×7 convolution kernel with a stride of 4 to extract coarse-scale features, which characterize the overall morphology and boundary properties of the lesion. The feature outputs of the three paths are fused through upsampling and concatenation operations to obtain a fused feature map containing multi-scale information.

[0035] Specifically, three types of features are extracted from the lesion area: mucosal surface morphology features, microvascular configuration features, and glandular structure features. The extraction of mucosal surface morphology features is accomplished by analyzing the three-dimensional morphology of the lesion area. First, a depth map of the lesion surface is reconstructed using image gradient information, and the depth value is estimated using the gradient integral method. Based on the depth map, the degree of elevation and the depth of depression of the lesion are calculated. The degree of elevation is defined as the difference between the highest point of the lesion and the average height of the surrounding normal mucosa, and the depth of depression is defined as the difference between the lowest point of the lesion and the average height of the surrounding normal mucosa. Boundary sharpness is evaluated by calculating the gradient intensity of the lesion boundary; a larger gradient intensity indicates a sharper boundary. In a preferred embodiment, the Sobel operator is used to calculate the gradient, and the boundary sharpness index is defined as the ratio of the average value to the standard deviation of the gradient intensity of boundary pixels; a larger index indicates a sharper and more regular boundary.

[0036] Microvascular morphology feature extraction was performed on narrowband imaging images. Narrowband imaging technology utilizes a narrowband light source to enhance the display of blood vessels in the mucosal surface, making the microvascular structure clearer. First, the narrowband imaging images underwent vascular enhancement processing, employing the Frangi filtering algorithm to enhance tubular structures. Frangi filtering, based on eigenvalue analysis of the Hessian matrix, can selectively enhance structures with tubular characteristics. After vascular enhancement, the vascular skeleton was extracted, and a thinning algorithm was used to refine the vascular region into single-pixel-wide skeleton lines. Based on the vascular skeleton, three features were calculated: vascular density, defined as the length of the vascular skeleton per unit area, reflecting the richness of blood vessels; vascular orientation, assessed by calculating the orientation histogram of the vascular skeleton, showing that normal blood vessels have relatively regular orientations, while tumor neovascularization exhibits chaotic orientations; and vascular morphology parameters, including tortuosity and branching angle. Tortuosity is defined as the ratio of the actual length of the blood vessel to the straight-line distance between its two endpoints, and the branching angle is the angle between two blood vessels at a branching point; tumor neovascularization typically exhibits high tortuosity and abnormal branching angles.

[0037] The extraction of glandular structural features was performed on magnifying endoscopic images. Magnifying endoscopy can magnify the fine structures on the mucosal surface, making glandular openings clearly visible. First, glandular segmentation was performed on the magnifying endoscopic images using a region-growing segmentation method. Regions with low brightness in the image were used as seed points, corresponding to glandular openings. Growing was performed from the seed points outwards, grouping similar pixels into the same region, ultimately resulting in multiple segmented glandular regions. Two features were calculated based on the segmented glandular regions: 1) Glandular arrangement regularity was assessed by calculating the spatial distribution entropy of the gland's center point. Normal glands are regularly arranged with low entropy values ​​at the center point, while tumor tissue glands are disordered and have high entropy values. 2) Glandular size uniformity was assessed by calculating the coefficient of variation (COP) of the gland area. The COP is defined as the ratio of the standard deviation of the area to the mean area. The smaller the COP, the more uniform the gland size. Normal glands are uniform in size with low COPs, while tumor tissue glands are uneven in size with high COPs.

[0038] The lesion identification unit identifies lesion types based on the fused feature vector. The feature vector contains all the extracted features, totaling 11 dimensions, including elevation, depression depth, boundary clarity, vessel density, vessel orientation entropy, vessel tortuosity, vessel branching angle, gland arrangement entropy, gland size variation coefficient, lesion area, and lesion perimeter. A Support Vector Machine (SVM) classifier is used for lesion identification. SVM is a supervised learning model that separates different categories by finding the optimal classification hyperplane. During training, the SVM model is trained using labeled lesion samples. The training samples include three types of lesions: adenoma, early-stage cancer, and advanced-stage cancer, with 500 samples in each category, totaling 1500 training samples. The SVM uses a radial basis function kernel with a kernel parameter set to 0.1 and a penalty parameter set to 10. After training, the model can predict the lesion type based on the input 11-dimensional feature vector and output the probability value for each category. During testing, the category with the highest probability is selected as the final lesion identification result.

[0039] In a specific application example, for an early gastric cancer lesion in the antrum, the features extracted by the multi-scale feature extraction unit are as follows: the elevation is 1.2 mm, indicating a slightly raised lesion; the depression depth is 0.3 mm, indicating a slight depression in the center of the lesion; the boundary clarity index is 3.8, indicating a relatively clear lesion boundary; the blood vessel density is 0.042 per square millimeter, higher than the 0.025 per square millimeter of normal mucosa; the blood vessel orientation entropy is 2.1, significantly higher than the 1.2 of normal mucosa; the blood vessel tortuosity is 1.6, while the normal blood vessel tortuosity is approximately 1.1; the average blood vessel branching angle is 65 degrees, deviating from the normal 90 degrees; the gland arrangement entropy is 2.8, higher than the normal 1.5; the gland size variation coefficient is 0.42, significantly higher than the normal 0.18; the lesion area is 225 square millimeters; and the lesion perimeter is 58 mm. These features were input into a support vector machine classifier, which output probabilities of 0.12 for adenoma, 0.82 for early-stage cancer, and 0.06 for advanced-stage cancer. Therefore, the identification result was early-stage cancer, consistent with the pathological diagnosis. This example demonstrates the complete process of multi-scale feature extraction and lesion identification.

[0040] Reference Figure 4The grading and depth assessment module 3 includes a morphological classification unit, a histological grading unit, and a depth prediction unit. The morphological classification unit uses Paris classification based on the macroscopic morphology of the lesion. The Paris classification system divides superficial lesions of the digestive tract into three main categories: Type 0-I, which are raised lesions, clearly protruding from the mucosal surface; Type 0-II, which are superficial lesions, basically level with or slightly undulating with the mucosal surface; and Type 0-III, which are depressed lesions, sunken into the mucosal surface. Type 0-II is further subdivided into Type 0-IIa (slightly raised), Type 0-IIb (flat), and Type 0-IIc (slightly depressed). Morphological classification is mainly based on two characteristics: the degree of protrusion and the depth of depression. The specific classification rules are as follows: when the elevation is greater than 2.5 mm, it is classified as type 0-I (elevated); when the depression depth is greater than 2.0 mm, it is classified as type 0-III (depressed); when the elevation is between 0.5 and 2.5 mm, it is classified as type 0-IIa (slightly elevated); when both the elevation and depression depth are less than 0.5 mm, it is classified as type 0-IIb (flat); and when the depression depth is between 0.5 and 2.0 mm, it is classified as type 0-IIc (slightly depressed). These thresholds are determined based on a large amount of clinical data and can effectively distinguish different morphological types. In a preferred embodiment, the system not only provides the classification result but also calculates the confidence level of the classification. The confidence level is determined by measuring the distance between the lesion morphological parameters and the classification threshold; the larger the distance, the more reliable the classification and the higher the confidence level.

[0041] Histological grading units are classified using the Vienna classification system based on the cytological characteristics of lesions. The Vienna classification system divides gastrointestinal epithelial tumors into five grades: Vienna 1 consists of non-neoplastic lesions, including normal mucosa, inflammation, and proliferative changes; Vienna 2 consists of indeterminate tumors, where the nature of the lesion is difficult to determine; Vienna 3 consists of non-invasive low-grade intraepithelial neoplasia, equivalent to low-grade adenoma or low-grade dysplasia; Vienna 4 consists of non-invasive high-grade intraepithelial neoplasia, including high-grade adenoma, carcinoma in situ, and suspected invasive carcinoma; and Vienna 5 consists of invasive tumors, including intramucosal carcinoma, submucosal carcinoma, or deeper infiltration. Histological grading is primarily based on glandular structural characteristics and microvascular morphology. The specific grading rules are as follows: When the glandular arrangement entropy is less than 1.8 and the glandular size coefficient of variation is less than 0.25, it is classified as Vienna Class 1 non-neoplastic lesion; when the glandular arrangement entropy is between 1.8 and 2.2 and the glandular size coefficient of variation is between 0.25 and 0.35, it is classified as Vienna Class 2 indeterminate tumor; when the glandular arrangement entropy is between 2.2 and 2.6 and the glandular size coefficient of variation is between 0.35 and 0.45, it is classified as Vienna Class 3 low-grade intraepithelial neoplasia; when the glandular arrangement entropy is between 2.6 and 3.2 or the glandular size coefficient of variation is between 0.45 and 0.60, it is classified as Vienna Class 4 high-grade intraepithelial neoplasia; when the glandular arrangement entropy is greater than 3.2 or the glandular size coefficient of variation is greater than 0.60, it is classified as Vienna Class 5 invasive tumor. These thresholds are also determined based on clinicopathological data statistics and can well reflect the relationship between histological characteristics and grading.

[0042] The depth prediction unit assesses the depth of tumor invasion. Tumor invasion depth refers to the extent to which a tumor invades the mucosal and submucosal layers, and it is crucial for the selection of treatment options. Intramucosal carcinoma can be cured with endoscopic submucosal dissection, while lesions reaching the submucosal layer require surgical intervention. Depth assessment is performed by analyzing the degree of abnormality in microvascular and glandular features. Studies have shown that as tumor invasion depth increases, the degree of microvascular abnormalities and glandular structural disorder also increases. The depth prediction unit first calculates two indices: the microvascular density abnormality index and the glandular structural disorder index. The formula for calculating the microvascular density abnormality index is:

[0043] ,

[0044] in, This is an index of abnormal microvascular density. The density of blood vessels in the lesion area. The average vascular density of normal mucosa is taken as 0.025 per square millimeter in the dataset of this invention. This index reflects the percentage increase in vascular density in the diseased area relative to normal mucosa; a larger index indicates a more pronounced vascular abnormality.

[0045] The formula for calculating the glandular structure disorder index is:

[0046] ,

[0047] in, This is an index of glandular structural disorder. The entropy of gland arrangement. The coefficient of variation for gland size. and As a weighting coefficient, in the preferred embodiment The value is 0.6. The value is 0.4. This index comprehensively reflects the degree of disorder in gland arrangement and the unevenness of gland size; the higher the index, the more severe the glandular structural abnormality.

[0048] Depth assessment is based on these two indices. The assessment rules are as follows: when the microvascular density abnormality index is less than 40% and the glandular structure disorder index is less than 2.0, it is considered an intramucosal lesion; when the microvascular density abnormality index is between 40% and 80% or the glandular structure disorder index is between 2.0 and 3.5, it is considered a suspected submucosal invasion; when the microvascular density abnormality index is greater than 80% and the glandular structure disorder index is greater than 3.5, it is considered a highly suspected submucosal invasion. This stratified assessment strategy can identify high-risk deep invasive lesions, providing important reference for treatment decisions. The depth prediction unit also outputs depth grading results, including three levels: mucosal layer, superficial submucosal layer, and deep submucosal layer. This grading result corresponds to the depth assessment of endoscopic ultrasound.

[0049] In the aforementioned application example of early gastric cancer, the morphological classification unit determined the lesion to be a slightly raised type 0-IIa based on a protrusion of 1.2 mm and a depression depth of 0.3 mm, with a confidence level of 0.88. The histological grading unit determined the lesion to be Vienna type 4 high-grade intraepithelial neoplasia based on a glandular arrangement entropy of 2.8 and a glandular size variation coefficient of 0.42, with a confidence level of 0.82. The depth prediction unit calculated a microvascular density abnormality index of 68% and a glandular structure disorder index of 2.8, classifying it as suspected submucosal invasion according to the judgment rules, with a depth grading of superficial submucosal layer. Subsequent endoscopic submucosal dissection pathology confirmed that the lesion was a well-differentiated adenocarcinoma with an invasion depth of 500 micrometers into the submucosal layer, belonging to superficial submucosal invasion, completely consistent with the system's prediction results. This example validates the accuracy and clinical applicability of the grading and depth assessment module.

[0050] Reference Figure 5The closed-loop feedback optimization module 4 includes a confidence assessment unit and a parameter adjustment unit. The confidence assessment unit calculates the confidence scores for the Paris and Vienna classifications. Confidence reflects the reliability of the classification results; high confidence indicates good image quality and clear lesion features, suggesting reliable classification; low confidence indicates poor image quality or atypical lesion features, indicating potential errors in the classification results. The confidence score is calculated based on the classifier's output probability and the typicality of the lesion features. For the Paris classification, the confidence score calculation formula is:

[0051] ,

[0052] in, The confidence level for classifying Paris. The maximum probability value output by the classifier. These are the morphological parameter measurements of the lesion. This is a typical threshold for this type. The threshold range is defined by this formula, which indicates that the confidence score consists of two parts: classifier probability and parameter typicality, with their product being the final confidence score. When the lesion morphology parameters are close to the typical threshold, the parameter typicality is close to 1, and the confidence score is mainly determined by the classifier probability. When the lesion morphology parameters deviate from the typical threshold, the parameter typicality decreases, and the confidence score decreases accordingly.

[0053] For the Vienna classification, a similar formula is used to calculate the confidence score. The overall confidence score is the average of the Paris classification confidence score and the Vienna classification confidence score; this overall confidence score is used to assess the reliability of the overall diagnostic result.

[0054] The parameter adjustment unit determines whether to adjust parameters based on the confidence level. A confidence threshold of 0.85 is set; the parameter adjustment mechanism is triggered when the overall confidence level falls below this threshold. The goal of parameter adjustment is to optimize image segmentation performance, as segmentation quality directly affects subsequent feature extraction and classification results. Parameter adjustment includes two aspects: adjusting the segmentation threshold and adjusting preprocessing parameters.

[0055] The segmentation threshold is adjusted based on the difference between the confidence level and the threshold. A larger difference indicates a less reliable result, requiring a more significant adjustment. The formula for calculating the threshold adjustment amount is:

[0056] ,

[0057] in, Adjustment coefficient The adjustment amount, The confidence threshold is set to 0.85. Based on the current overall confidence level, In the preferred embodiment, the adjustment coefficient is set to 0.5. This formula indicates that the adjustment amount is proportional to the confidence level difference; the lower the confidence level, the larger the adjustment amount. Adding the adjustment amount to the original adjustment coefficient yields a new adjustment coefficient, which is used to recalculate the segmentation threshold. When the confidence level is low, increasing the adjustment coefficient raises the segmentation threshold, causing the segmentation results to tend to extract larger lesion areas, helping to avoid missing the edge parts of lesions.

[0058] The preprocessing parameters were adjusted primarily for the scale parameter of illumination normalization. When the confidence level was low, the weight of large-scale parameters was appropriately increased to enhance the global illumination equalization effect; simultaneously, the weight of small-scale parameters was reduced to alleviate the overemphasis on local details. Specifically, the weight of large-scale parameters was increased by 0.1, the weight of medium-scale parameters remained unchanged, and the weight of small-scale parameters was decreased by 0.1. This adjustment strategy helps to obtain more stable preprocessing results even when the image quality is poor.

[0059] After adjusting the parameters, the system re-executes the image segmentation, feature extraction, and grading evaluation processes to obtain new diagnostic results and confidence levels. If the new confidence level is still below the threshold, the parameters are adjusted and iterated repeatedly, up to a maximum of three iterations. Through multiple iterations of optimization, the system typically achieves a satisfactory confidence level. This closed-loop feedback mechanism significantly improves the system's adaptability to different image qualities and lesion characteristics, ensuring the reliability of the diagnostic results.

[0060] In practical applications, the closed-loop feedback optimization module plays a crucial role in cases with poor image quality or atypical lesion features. For example, in the initial segmentation of a lower esophageal lesion, uneven image illumination resulted in incomplete lesion boundary extraction, leading to a Paris classification confidence score of only 0.72, below the threshold of 0.85. The confidence assessment unit triggered parameter adjustment, calculating an adjustment amount of 0.5 × (0.85 - 0.72) = 0.065, changing the original adjustment coefficient from 0.2 to 0.265. Using the new adjustment coefficient, image segmentation was performed again, increasing the lesion area from 142 square millimeters to 189 square millimeters, resulting in a more complete lesion boundary. Feature extraction and classification were then performed again. The Paris classification result remained a slightly concave type 0-IIc, but the confidence score increased to 0.87, exceeding the threshold. The Vienna classification result was classified as Vienna 4, with a confidence score of 0.86. The overall confidence score was 0.865, meeting the requirements, and the system output the final diagnostic result. Subsequent pathological examination confirmed that the lesion was indeed a high-grade intraepithelial neoplasia, type 0-IIc, verifying the effectiveness of the closed-loop feedback optimization module.

[0061] This invention also includes a diagnostic report generation module, which is connected to the grading and depth assessment module. This module generates a structured diagnostic report based on the Paris classification results, Vienna classification results, and tumor invasion depth. The diagnostic report uses a standardized format and includes the following: lesion location, such as the anterior wall of the gastric antrum or the splenic flexure of the colon; lesion size, expressed as the long and short axes; morphological type, such as type 0-IIa plus IIc or type 0-I; histological grade, such as Vienna type 4 high-grade intraepithelial neoplasia; invasion depth assessment, such as suspected superficial submucosal invasion; and treatment recommendations, automatically generated based on invasion depth and histological grade. The specific treatment recommendations are as follows: For Vienna type 3 low-grade intraepithelial neoplasia with infiltration depth into the mucosa, endoscopic mucosal resection or endoscopic submucosal dissection is recommended; for Vienna type 4 high-grade intraepithelial neoplasia with infiltration depth into the mucosa or superficial submucosa, endoscopic submucosal dissection is recommended; for Vienna type 5 invasive tumors or lesions assessed as deep submucosal infiltration, surgical treatment is recommended. These treatment recommendations are based on international guidelines and clinical consensus, providing standardized decision support for clinicians. The diagnostic report is output in PDF format for easy archiving and sharing. The report also includes endoscopic images and segmented annotated images of the lesion, visually demonstrating the location and extent of the lesion to assist clinicians in understanding the diagnostic results.

[0062] The technical advantage of this invention lies in the deep coupling and closed-loop collaboration of multiple modules. The multimodal image preprocessing and segmentation module provides high-quality lesion regions for subsequent modules, and its output directly affects the accuracy of feature extraction. The multidimensional features extracted by the lesion feature extraction and diagnosis module provide a comprehensive information foundation for grading assessment, and the complementary nature of different types of features improves classification accuracy. The grading and deep assessment module not only outputs diagnostic results but also calculates confidence scores, which reflect the reliability of the diagnosis and provide a quantitative indicator for closed-loop feedback. The closed-loop feedback optimization module dynamically adjusts the parameters of the front-end modules based on the confidence scores, realizing reverse control from diagnostic results to image processing, forming a complete closed-loop system. This deep coupling design enables each module to promote each other and synergistically enhance each other, resulting in overall system performance significantly better than the simple sum of the independent operation of each module.

[0063] The system of this invention has achieved excellent performance in clinical validation. On a test set containing 500 cases of gastrointestinal tumors, the sensitivity for lesion detection reached 94.2%, specificity 89.7%, and accuracy 92.4%. The accuracy of Paris classification reached 87.8%, with a concordance of 0.82 with endoscopists, significantly higher than the 0.68 of general endoscopists. The accuracy of Vienna classification reached 83.6%, with an accuracy of 89.3% for identifying high-grade Vienna 4 and Vienna 5 lesions. The accuracy of tumor invasion depth assessment reached 78.5%, with a sensitivity of 85.7% for identifying deep submucosal invasion, avoiding misdiagnosis of lesions unsuitable for endoscopic treatment. The closed-loop feedback optimization mechanism significantly improved the system's robustness to changes in image quality; on a subset with poor image quality, enabling closed-loop feedback increased the accuracy from 73.5% to 81.2%, an improvement of 7.7 percentage points. These data demonstrate the clinical practical value of the system of this invention.

[0064] It should be understood that the above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. Any modifications or equivalent transformations made according to the spirit and essence of the main technical solution of the present invention should be covered within the scope of protection of the present invention.

Claims

1. An intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors, characterized in that, include: A multimodal image preprocessing and segmentation module is used to receive white light endoscope images, narrowband imaging images, and magnifying endoscope images, preprocess the white light endoscope images, narrowband imaging images, and magnifying endoscope images, and segment them to obtain lesion regions. The preprocessing includes illumination compensation processing and noise suppression processing on the white light endoscope images, narrowband imaging images, and magnifying endoscope images. The segmentation adopts an adaptive threshold segmentation method to dynamically determine the segmentation threshold based on local image features. The lesion feature extraction and diagnosis module, connected to the multimodal image preprocessing and segmentation module, is used to extract mucosal surface morphological features, microvascular configuration features, and glandular structure features from the lesion area, and to identify the lesion type based on the mucosal surface morphological features, the microvascular configuration features, and the glandular structure features. The mucosal surface morphological features include the degree of elevation, the depth of depression, and the clarity of the boundary. The microvascular configuration features include the blood vessel density, blood vessel orientation, and blood vessel morphology parameters. The glandular structure features include the glandular arrangement regularity and the uniformity of glandular size. The grading and depth assessment module, connected to the lesion feature extraction and diagnosis module, is used to perform Paris and Vienna classifications based on the lesion type, mucosal surface morphological features, microvascular configuration features, and glandular structure features, and to assess the tumor invasion depth. The Paris classification determines the macroscopic morphological type of the lesion, and the Vienna classification determines the histological grade of the lesion. The tumor invasion depth is determined by analyzing the degree of abnormality of the microvascular configuration features and the glandular structure features. A closed-loop feedback optimization module, connected to the hierarchical and depth evaluation module and the multimodal image preprocessing and segmentation module, is used to evaluate the segmentation quality based on the confidence scores of the Paris and Vienna classifications. When the confidence scores are lower than a preset threshold, a feedback signal is generated to adjust the segmentation threshold and preprocessing parameters of the multimodal image preprocessing and segmentation module, thereby achieving dynamic optimization of the segmentation parameters. The feedback signal is transmitted to the multimodal image preprocessing and segmentation module through a data interface to update the parameters.

2. The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors according to claim 1, characterized in that: The multimodal image preprocessing and segmentation module includes an image preprocessing unit and an adaptive segmentation unit. The image preprocessing unit performs illumination normalization and noise reduction processing on the endoscopic image. The adaptive segmentation unit uses an adaptive thresholding method based on local variance to segment the lesion region. The adaptive threshold is determined based on the local variance and global mean of the image block.

3. The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors according to claim 1, characterized in that: The lesion feature extraction and diagnosis module includes a multi-scale feature extraction unit and a lesion identification unit. The multi-scale feature extraction unit uses multi-scale convolution to extract lesion features at different spatial scales and performs feature fusion. The lesion identification unit identifies the lesion type based on the fused feature vector. The lesion types include adenoma, early-stage cancer, and advanced-stage cancer.

4. The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors according to claim 1, characterized in that: The grading and depth assessment module includes a morphological classification unit, a histological grading unit, and a depth prediction unit. The morphological classification unit performs Paris classification based on the macroscopic morphology of the lesion, the histological grading unit performs Vienna classification based on the cytological characteristics of the lesion, and the depth prediction unit predicts the tumor invasion depth based on the degree of abnormality of microvessels and glands and outputs the depth grading results.

5. The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors according to claim 4, characterized in that: The morphological classification unit divides lesions into type 0-I (raised), type 0-II (superficial), and type 0-III (depressed). Type 0-II (superficial) is further subdivided into type 0-IIa (slightly raised), type 0-IIb (flat), and type 0-IIc (slightly depressed). The Paris classification results are used to guide the selection of endoscopic treatment plans.

6. The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors according to claim 4, characterized in that: The histological grading unit classifies lesions into five grades: Vienna 1 non-neoplastic lesions, Vienna 2 indeterminate tumors, Vienna 3 non-invasive low-grade intraepithelial neoplasia, Vienna 4 non-invasive high-grade intraepithelial neoplasia, and Vienna 5 invasive tumors.

7. The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors according to claim 4, characterized in that: When assessing the depth of tumor invasion, the depth prediction unit calculates the microvascular density abnormality index and the glandular structure disorder index. When the microvascular density abnormality index is higher than a first threshold or the glandular structure disorder index is higher than a second threshold, it is determined to be suspected submucosal invasion. When both indices are higher than their corresponding thresholds, it is determined to be highly suspected submucosal invasion.

8. The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors according to claim 1, characterized in that: The closed-loop feedback optimization module includes a confidence evaluation unit and a parameter adjustment unit. The confidence evaluation unit calculates the confidence scores of the Paris classification and the Vienna classification. The parameter adjustment unit generates a parameter adjustment instruction when the confidence score is lower than 0.

85. The parameter adjustment instruction is used to adjust the threshold calculation parameters of the adaptive threshold segmentation method.

9. The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors according to claim 8, characterized in that: The parameter adjustment unit calculates the parameter adjustment range based on the difference between the confidence level and the preset threshold. The parameter adjustment range is positively correlated with the difference. The adjusted segmentation threshold is used to re-segment the endoscopic image to obtain an optimized lesion region. The optimized lesion region is used for re-feature extraction and diagnostic grading.

10. The intelligent assisted diagnosis and grading system for endoscopic images of gastrointestinal tumors according to claim 1, characterized in that: It also includes a diagnostic report generation module, which is connected to the grading and depth assessment module, for generating a structured diagnostic report based on the Paris classification results, the Vienna classification results, and the tumor invasion depth. The diagnostic report includes lesion location, lesion size, morphological type, histological grade, invasion depth assessment, and treatment recommendations. The treatment recommendations are automatically generated based on the invasion depth and the histological grade.

Images