Digestive tract tumor lesion image segmentation method and segmentation system based on multimodality

Abstract

The present invention relates to the field of medical image processing technology, and in particular to a multimodal digestive tract tumor lesion image segmentation method and segmentation system. The method comprises the following steps: acquiring a digestive tract tumor lesion image and extracting a digestive tract tumor ultrasound image; assessing the tumor invasion depth based on the digestive tract tumor ultrasound image; performing three-dimensional visualization modeling of the lesion according to the tumor invasion depth to obtain a digestive tract tumor lesion model; extracting a tumor tissue pathology image based on the digestive tract tumor lesion image; performing tumor region segmentation based on the tumor tissue pathology image to obtain digestive tract tumor region data; performing tissue arrangement abnormality detection based on the digestive tract tumor region data to obtain tissue arrangement abnormality data; and calculating a tissue arrangement disorder index based on the tissue arrangement abnormality data and the digestive tract tumor region data. The present invention improves the accuracy of tumor recognition and the accuracy of malignant region segmentation based on medical image processing technology.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a multimodal digestive tract tumor lesion image segmentation method and segmentation system. Background Art

[0002] There is significant heterogeneity between traditional gastrointestinal tumor lesion images, including differences in image resolution, different imaging angles, and inconsistent contrast, which makes inter-modal fusion difficult and makes it difficult to effectively extract complementary feature information. They usually rely on manually designed feature extraction algorithms, which are difficult to adapt to the complex and changeable tumor morphology and tissue structure, especially when the boundaries are blurred, the lesion volume is small, or the density is similar to that of the surrounding tissue, the segmentation accuracy is greatly reduced. Existing systems generally lack sufficient modeling of the temporal, consistency and spatial correspondence of multimodal data, which often leads to distortion of the fused image information and affects the accuracy of subsequent segmentation results. The process processing of data preprocessing, alignment, fusion and segmentation remains in the serialized and low-coupling stage, and cannot achieve end-to-end automated processing of multimodal images. The system has a low level of intelligence and poor generalization and adaptability. Summary of the Invention

[0003] Based on this, it is necessary for the present invention to provide a multimodal digestive tract tumor lesion image segmentation method and segmentation system to solve at least one of the above technical problems.

[0004] To achieve the above objectives, a multimodal digestive tract tumor lesion image segmentation method is provided, comprising the following steps:

[0005] Step S1: Acquire a digestive tract tumor lesion image and extract a digestive tract tumor ultrasound image; assess the tumor invasion depth based on the digestive tract tumor ultrasound image; perform three-dimensional visualization modeling of the lesion based on the tumor invasion depth to obtain a digestive tract tumor lesion model;

[0006] Step S2: extracting a tumor tissue pathology image based on the digestive tract tumor lesion image; performing tumor region segmentation based on the tumor tissue pathology image to obtain digestive tract tumor region data; performing tissue arrangement abnormality detection based on the digestive tract tumor region data to obtain tissue arrangement abnormality data;

[0007] Step S3: calculating a tissue disorder index based on the tissue disorder data and the digestive tract tumor area data; identifying high-risk lesion areas based on the tissue disorder index; and performing lesion edge morphology recognition based on the high-risk lesion areas to obtain lesion edge morphology data.

[0008] Step S4: Evaluate the malignancy of the digestive tract tumor based on the lesion edge morphology data according to the digestive tract tumor lesion model to obtain digestive tract tumor malignancy data; segment the digestive tract tumor lesion image into malignant areas based on the digestive tract tumor malignancy data to generate a digestive tract malignant tumor lesion image.

[0009] The present invention can establish a three-dimensional lesion model that reflects the growth behavior and spatial range of the tumor by acquiring ultrasonic images of digestive tract tumors and evaluating the invasion depth. The model has structural integrity and spatial consistency, laying the foundation for subsequent data fusion and spatial positioning analysis. By introducing tumor tissue pathology images and combining regional segmentation and tissue arrangement abnormality detection methods, high-precision identification of tissue hierarchical structural abnormalities can be achieved, providing key data support for abnormal tumor behavior analysis. Furthermore, edge morphology recognition is performed on the basis of identifying high-risk lesion areas, which can improve the ability to determine the structure of the tumor boundary fuzzy area and effectively solve the problem of misjudgment caused by unclear tumor contours or similar density to surrounding tissues. On this basis, spatial extensibility detection and malignancy assessment are performed through the spatial correspondence between the lesion edge morphology and the lesion model. Not only can quantitative modeling of the spatial distribution law of the lesion structure be achieved, but also highly malignant areas can be identified based on real spatial deformation characteristics, thereby enhancing the ability to reflect the biological behavior of the tumor. Combining these results with malignant region segmentation can significantly improve the accuracy of identifying key lesion areas in multimodal images, enhance the precision and integrity of the final lesion image, mitigate the damage to spatial continuity and modal consistency caused by existing serial processing flows, and significantly enhance the overall processing efficiency and automation of the system. Furthermore, this method integrates multimodal images with multi-source information such as structural data, curvature morphological data, and tissue abnormality indicators throughout the entire process, and performs end-to-end analysis and modeling based on clear spatial correspondences. This can effectively alleviate the training bottleneck caused by small sample labeling issues, achieve accurate modeling of complex gastrointestinal tumor lesions, enhance the system's generalization and adaptability, and has good prospects for clinical promotion and application.

[0010] Preferably, this specification also provides a multimodal digestive tract tumor lesion image segmentation system, which is used to execute the multimodal digestive tract tumor lesion image segmentation method described above. The multimodal digestive tract tumor lesion image segmentation system includes:

[0011] The lesion 3D visualization module is used to obtain images of digestive tract tumor lesions and extract digestive tract tumor ultrasound images; assess the tumor invasion depth based on digestive tract tumor ultrasound images; and perform 3D visualization modeling of the lesion based on the tumor invasion depth to obtain a digestive tract tumor lesion model;

[0012] The tissue arrangement abnormality detection module is used to extract tumor tissue pathology images based on the digestive tract tumor lesion images; perform tumor region segmentation based on the tumor tissue pathology images to obtain digestive tract tumor region data; perform tissue arrangement abnormality detection based on the digestive tract tumor region data to obtain tissue arrangement abnormality data;

[0013] The lesion edge morphology recognition module is used to calculate the tissue disorder index based on the tissue arrangement abnormality data and the digestive tract tumor area data; identify the high-risk lesion area based on the tissue disorder index; and perform lesion edge morphology recognition based on the high-risk lesion area to obtain lesion edge morphology data;

[0014] The malignant region segmentation module is used to evaluate the malignancy of digestive tract tumors based on the lesion edge morphology data according to the digestive tract tumor lesion model to obtain digestive tract tumor malignancy data; based on the digestive tract tumor malignancy data, the malignant region of the digestive tract tumor lesion image is segmented to generate a digestive tract malignant tumor lesion image.

[0015] The multimodal digestive tract tumor lesion image segmentation system of the present invention can implement any multimodal digestive tract tumor lesion image segmentation method of the present invention, and is used to combine the operations and signal transmission media between various modules to complete the multimodal digestive tract tumor lesion image segmentation method. The internal modules of the system cooperate with each other to improve the accuracy of tumor recognition and the accuracy of malignant area segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0017] Figure 1 This is a schematic flow chart of the steps of a multimodal digestive tract tumor lesion image segmentation method according to the present invention;

[0018] Figure 2 Detailed step flow diagram of step S1 in the present invention;

[0019] Figure 3 Detailed step flow diagram of step S4 in the present invention;

[0020] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0021] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative work are within the scope of protection of the present invention.

[0022] In addition, the accompanying drawings are merely schematic illustrations of the present invention and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor and / or microcontroller approaches.

[0023] It should be understood that although the terms "first," "second," and the like may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the exemplary embodiments. The term "and / or" as used herein includes any and all combinations of one or more of the listed associated items.

[0024] To achieve this, please refer to Figures 1 to 3 The present invention provides a multimodal digestive tract tumor lesion image segmentation method, the method comprising the following steps:

[0025] Step S1: Acquire a digestive tract tumor lesion image and extract a digestive tract tumor ultrasound image; assess the tumor invasion depth based on the digestive tract tumor ultrasound image; perform three-dimensional visualization modeling of the lesion based on the tumor invasion depth to obtain a digestive tract tumor lesion model;

[0026] In this example, images of digestive tract tumor lesions were acquired using a gastrointestinal endoscopic ultrasound imaging system commonly used in clinical examinations. The acquisition frequency was set to 7.5 MHz, the imaging depth was set to 0–50 mm, and the image resolution was controlled at 1024 × 768 pixels. After image acquisition, the DICOM format was converted to an 8-bit grayscale PNG image using an image format conversion tool to ensure compatibility with subsequent image processing. The image contained the tumor lesion area, normal tissue areas, and surrounding structures. After image acquisition, an image enhancement algorithm based on image histogram equalization was used to enhance contrast and improve tumor boundary discernibility. Subsequently, the two-dimensional maximum inter-class variance (Otsu) method was used to perform preliminary image segmentation, and suspected lesion areas were extracted as the basis for ultrasound image analysis. To assess the depth of tumor invasion, a multi-region hierarchical edge detection algorithm was used to extract the structural boundaries of the mucosa, submucosa, muscularis, and serosa. Layer identification was performed by combining grayscale distribution curves and corresponding spatial distances. Specifically, a grayscale difference threshold ΔG was set to 45. If the grayscale difference between consecutive regions exceeded this threshold and a pixel density gradient abruptly exceeded 0.75 (units / mm²), it was defined as a tissue layer boundary. After identifying each layer, the depth of tumor invasion was determined based on the maximum depth of the lesion: invasion into the myometrium was classified as stage T2, while penetration through the serosa was classified as stage T4. These analysis results were input into the 3D modeling module, where a 3D structural model of the lesion was constructed using the Marching Cubes algorithm based on point cloud reconstruction. Each layer was sliced (with a 0.5 mm spacing between each layer) and reconstructed using stacking with Z-axis registration. A 3D volume model was generated with a voxel resolution of 0.5 × 0.5 × 0.5 mm³ and exported as an STL file, resulting in a 3D model of the digestive tract tumor lesion.

[0027] Step S2: extracting a tumor tissue pathology image based on the digestive tract tumor lesion image; performing tumor region segmentation based on the tumor tissue pathology image to obtain digestive tract tumor region data; performing tissue arrangement abnormality detection based on the digestive tract tumor region data to obtain tissue arrangement abnormality data;

[0028] In this example, after acquiring an image of a digestive tract tumor lesion, the corresponding histopathology slide image is retrieved from a homologous database based on the image's spatial coordinates and image number information. The image format is high-resolution Whole Slide Image (WSI) with a resolution of 0.25 μm / pixel. Each WSI image is cropped to a 256×256 pixel block using an image block extraction algorithm, and the blank background area is removed. Subsequently, a staining normalization method (Reinhard algorithm) is used to correct for chromatic aberration caused by HE staining to ensure consistent color distribution across different sections. In the preprocessed pathology images, a K-means algorithm based on color histogram clustering is used to segment the cell nucleus region, with the number of cluster categories K set to 3. The nuclear dye color feature channel is extracted using the L and α components of the Lab color space to construct a feature vector. Regions are screened based on contour structure, retaining only regions with an area of 80–800 μm² as candidate tumor cell nuclei. After identifying the cell nuclei, a Voronoi diagram was used to construct an arrangement network between adjacent nuclei. The intercellular connectivity map was then constructed based on the centroid coordinates of each nucleus, resulting in a tissue arrangement topology. This topology was then tested for anomalies. A standard deviation threshold, σ, for the arrangement adjacency angles was set to 25°. If the percentage of connections exceeding this threshold exceeded 30% in the local structure, the region was considered an anomaly. A sliding window (512 × 512 pixels, 128 pixel step size) was used to scan the region, capturing information about the anomaly regions within the entire slice. This information was recorded as tissue arrangement anomaly data, output as a JSON structure with spatial coordinates.

[0029] Step S3: calculating a tissue disorder index based on the tissue disorder data and the digestive tract tumor area data; identifying high-risk lesion areas based on the tissue disorder index; and performing lesion edge morphology recognition based on the high-risk lesion areas to obtain lesion edge morphology data.

[0030] In this embodiment, the disorder index is calculated using tissue arrangement abnormality data and digestive tract tumor area data. The calculation process includes three parameters: the rate of change in the density distribution of cell nuclei (denoted as D), the rate of change in the angle of connection between cell nuclei (denoted as A), and the variance of the Voronoi polygon area (denoted as V). The rate of change in the density of the center of gravity D is calculated by calculating the standard deviation of the number of centers of gravity per unit area in a 16×16 nuclear area (unit: μm). -The connection angle change rate A is calculated by the standard deviation of the angles of all connected edges. The Voronoi polygon area variance V is directly extracted from the constructed graph and its standard deviation is calculated. The permutation disorder index R is calculated using the formula: R = 0.4 × D + 0.3 × A + 0.3 × V and normalized to the interval [0, 1]. Areas with R values greater than 0.65 are defined as high-risk lesion areas and their locations in the original image coordinate system are marked. Lesion edge morphology is identified within these high-risk areas. The Sobel operator is used to extract image edge gradient information with a threshold of 20. Closed regions with edge lengths greater than 10 pixels are selected. The degree of edge direction change is then analyzed using the Histogram of Oriented Gradients (HGs). If the change angle exceeds 60° within a 10-pixel span, the edge is considered irregular. All edge morphology data is encoded and output, including starting point coordinates, edge direction sequence, and edge length.

[0031] Step S4: Evaluate the malignancy of the digestive tract tumor based on the lesion edge morphology data according to the digestive tract tumor lesion model to obtain digestive tract tumor malignancy data; segment the digestive tract tumor lesion image into malignant areas based on the digestive tract tumor malignancy data to generate a digestive tract malignant tumor lesion image.

[0032] In this embodiment, a gastrointestinal tumor lesion model is input into the edge morphology data registration module. The lesion edge morphology data is first projected into three-dimensional space. The ICP (Iterative Closest Point) algorithm based on rigid registration is used to map the two-dimensional edge morphology coordinate system to the three-dimensional model coordinate space. The error threshold is set to 1.5 mm, and the maximum number of iterations is 500. After registration, high-risk morphological areas are marked on the lesion model surface. The density of irregular edge points per unit surface area is calculated, and areas with a density exceeding 0.8 points / mm² are defined as highly abnormal areas. Each high-risk area is assigned a malignancy grade label based on the corresponding tumor invasion level in the model. The grade is categorized as follows: deeper invasion levels and higher edge density result in higher grades, ranging from 0 (low risk) to 3 (high risk). This generates gastrointestinal tumor malignancy data, which is formatted as a three-dimensional structure that records model coordinates, edge density, invasion depth, and malignancy label. Based on the malignancy severity data, the final lesion image is segmented into malignant regions. A region growing algorithm is used, starting from the highly malignant labeled points, with growth similarity criteria set to: pixel grayscale differences less than 25, and edge gradient direction differences less than 15°. Each growth extension is limited to a maximum step size of 20 pixels to avoid overgrowth into non-lesion areas. The resulting region represents the segmented digestive tract malignant tumor lesion image. The output image is a labeled PNG image, with each pixel annotated with a malignancy grade of 0 to 3, and the corresponding color channels labeled blue, yellow, orange, and red, for ease of subsequent analysis.

[0033] Preferably, step S1 is specifically as follows:

[0034] Step S11: Acquire a digestive tract tumor lesion image and extract a digestive tract tumor ultrasound image;

[0035] In this embodiment, an integrated multimodal endoscopic image acquisition device is used to acquire images of digestive tract tumor lesions. This device is equipped with a high-frequency linear array ultrasound probe (set to 12 MHz) and a standard optical camera (1920 × 1080 resolution). Lesion images are acquired from the duodenum to the rectum. A steady, uniform propulsion velocity (1.5 cm / s) is applied during acquisition to prevent image distortion. After acquisition, the images are synchronously calibrated according to their time and spatial coordinates. The images are reordered using the timestamps embedded in the image frames, and invalid frames are removed. Subsequently, a two-dimensional Fourier transform is used to detect noise frequency bands in the entire image set. Wavelet threshold denoising is performed on frames with noise exceeding 25 dB. A screening algorithm based on an energy concentration threshold set at 70% is used to extract frames with high reflectivity signals from the ultrasound image sequence and output them as digestive tract tumor ultrasound images.

[0036] Step S12: identifying the intestinal wall hierarchical structure based on the digestive tract tumor ultrasound image to obtain an intestinal wall hierarchical image;

[0037] In this embodiment, a five-layer differentiation algorithm based on grayscale distribution was used to identify the hierarchical structure of the intestinal wall in ultrasound images of digestive tract tumors. First, the original image was subjected to contrast-limited histogram equalization (CLAHE) to enhance edge contrast. Subsequently, a Gaussian filter (with a kernel standard deviation of 1.2) was applied to the image to smooth the noisy background. A gradient-directed Canny edge detector was used with thresholds set to a low threshold of 50 and a high threshold of 150 to extract the bright and dark boundaries in the image. Next, contour tomography segmentation was used to segment the image into five anatomical layers, from the mucosa to the serosa. Each layer was constrained by brightness difference, boundary regularity, and thickness range, with the thickness ranges set to, in order, mucosa (0.4–1.0 mm), submucosa (1.0–1.5 mm), muscularis (1.5–2.5 mm), serosa (2.5–3.5 mm), and peripheral fat layer (>3.5 mm). Each layer was defined by a specific reflection intensity threshold. The intestinal wall layer image is output through the contour fitting algorithm (based on curvature mean control) for subsequent structure determination.

[0038] It is particularly important that step S12 includes the following steps:

[0039] Step S121: performing grayscale normalization on the digestive tract tumor ultrasound image to obtain a normalized digestive tract tumor ultrasound image;

[0040] In this embodiment, during ultrasound image preprocessing, grayscale normalization is performed to unify the image grayscale value range. The original digestive tract tumor ultrasound image is read into the image processing module, using an 8-bit grayscale image as the input type. The image pixel values range from 0 to 255. The image grayscale histogram is read for the entire image grayscale distribution. After eliminating the influence of noise, the 1% and 99% percentile grayscale values are selected as the upper and lower limits of the grayscale distribution. Specifically, the minimum grayscale threshold is set to the pixel value corresponding to the 1st percentile grayscale value, and the maximum grayscale threshold is set to the pixel value corresponding to the 99th percentile grayscale value. These serve as the normalization upper and lower limits to avoid extreme pixels stretching the image contrast. During the normalization process, a linear mapping is used to proportionally convert all pixel values in the image to integer grayscale values between 0 and 255. Normalization is performed using the image grayscale transformation function in the OpenCV image processing library. After processing, a normalized image with a unified standard grayscale distribution is output and stored in a memory array for subsequent processing steps.

[0041] Step S122: calculating the grayscale average value based on the normalized digestive tract tumor ultrasound image, and drawing a grayscale projection curve according to the grayscale average value;

[0042] In this embodiment, after the grayscale normalized image is generated, the grayscale average value of the image is calculated. This is done by summing the grayscale values of all pixels in each column (i.e., along the image's vertical axis) and dividing the sum by the number of pixel rows. This calculation yields the grayscale average value for each column, forming a set of one-dimensional grayscale projection data. This operation is implemented using the array mean function of the Python NumPy library and is performed column by column over the entire image width. To visualize grayscale fluctuations, the resulting grayscale average array is used as the vertical coordinate and the image column index as the horizontal coordinate. Matplotlib is used to draw a two-dimensional line graph, generating a grayscale projection curve. The horizontal axis of the plotting area is fixed to the image width, for example, 512 pixels, and the vertical axis grayscale range is fixed to 0 to 255. The output image resolution is set to 300 DPI to preserve image detail. The results are used for subsequent analysis of grayscale fluctuation patterns.

[0043] Step S123: Identify the grayscale fluctuation regularity area based on the grayscale projection curve and preliminarily mark it as a suspected intestinal wall area;

[0044] In this embodiment, the grayscale projection curve generated in step S122 is subjected to local fluctuation feature analysis. First, the image sliding window length is set to a 20-pixel step length, and the curve data is subjected to sliding analysis. The grayscale variation amplitude is calculated in each window, that is, the difference between the maximum and minimum values in the window. If the grayscale fluctuation amplitude is greater than the set threshold, the area corresponding to the window is marked as a grayscale significant fluctuation area. The fluctuation threshold is set to 40 (in grayscale value) based on experience. This method is used to slide search and record all fluctuation area index segments that meet the conditions in the entire projection curve to form a candidate set of suspected intestinal wall areas. The candidate sets are then merged. If the interval between two fluctuation segments is less than 10 pixels, they are merged to exclude non-structural intervals caused by slight grayscale fluctuations. Finally, the longitudinal strip areas corresponding to these column indices in the image are extracted and input into the next step as suspected intestinal wall areas. No deep learning model is introduced in this process, and area detection is performed only based on local grayscale statistics and sliding window strategy.

[0045] Step S124: performing texture periodicity detection based on the suspected intestinal wall region to obtain texture periodicity data;

[0046] In this embodiment, after obtaining the suspected intestinal wall area, the internal texture structure of each area is extracted for periodic detection. First, the suspected area is cut into independent sub-images in the image, and the horizontal grayscale vector sequence is extracted row by row in each sub-image, and these sequences are subjected to periodic structure analysis. The structural similarity method is used to detect texture repeatability. The specific steps are: sub-vectors with a length of 20 pixels are cut from left to right in units of every 5 pixels, and the Euclidean distance similarity between the current sub-vector and the subsequent sub-vectors at different positions is calculated. When the similarity is higher than the set threshold (the threshold is set to a normalized similarity of 0.85), the current offset distance is recorded as a periodic reference value. Repeat this operation for the entire area, count the frequency of occurrence of the period length, take the three period lengths with the highest frequency and take the average as the final texture period length output. This period length is used as the basic parameter for the subsequent positioning of the five-layer intestinal wall structure. All similarity calculations are completed based on standard array operation tools without the introduction of convolution or frequency domain methods.

[0047] Step S125: identifying the five-layer intestinal wall structure based on the texture period data;

[0048] In this embodiment, after obtaining the texture period length, layered cutting is performed along the longitudinal axis of the image using the length as a unit. For example, if the texture period length is detected to be 12 pixels, it is cut every 12 pixels downward from the starting position of the suspected intestinal wall area, and a total of five layers are planned. The structure corresponding to each layer is determined by a layer-by-layer grayscale feature determination method. The average grayscale value and grayscale variance in each layer are counted, and the five layers of intestinal wall tissue are matched and sorted according to the common grayscale patterns in the ultrasound image. They are marked as mucosal layer, submucosa, muscular layer, serosal layer and other structures in turn, and the upper and lower boundary ranges of each layer are set and recorded in the structure label map. The layering and labeling process does not rely on a deep learning model, but is completely based on the period length and grayscale difference extracted in the early stage. All image cutting and statistical operations are implemented through the cooperation of OpenCV and NumPy.

[0049] Step S126: dividing the digestive tract tumor ultrasound image into intestinal walls based on the five-layer intestinal wall structure to obtain an intestinal wall layer image.

[0050] In this embodiment, the normalized digestive tract tumor ultrasound image is divided into structural hierarchies according to the five-layer structural boundaries calibrated in step S125. In the original image space, segmentation lines are drawn directly at the upper and lower boundaries of each layer, and different pixel values are used to encode labels for each layer area. For example, the mucosal layer is assigned a value of 50, the submucosal layer is assigned a value of 100, the muscular layer is 150, the serous layer is 200, and so on, to form a grayscale hierarchical image for segmentation expression. The output image size is consistent with the input image, all pixels are classified into a specific structural layer, and the image type is set to an 8-bit single-channel integer grayscale image. After the segmentation operation is completed, the hierarchical image is saved in .tif format to maintain lossless accuracy for multimodal fusion or subsequent lesion identification. The image segmentation operation is processed directly in the array using the Python image drawing module, without involving image resampling or rotation, and maintaining the geometric consistency of the original image.

[0051] Step S13: determining the tumor penetration level based on the intestinal wall layer image to obtain tumor penetration level data;

[0052] In this embodiment, tumor penetration level is determined based on a pixel-level mask overlap strategy. First, a regional mask is constructed using information from the five-layer partitioning of the intestinal wall layer image. Each layer is assigned a unique grayscale encoding value (e.g., 50 for the mucosa, 100 for the submucosa, 150 for the muscularis, 200 for the serosa, and 250 for the peripheral layer). A binary mask of the lesion region is then extracted, and the lesion contour is extracted using the maximum contour method. The lesion contour is compared pixel by pixel with the masks of each intestinal wall layer, and the percentage of pixels overlapping with each layer is calculated. When the number of overlapping pixels in any layer exceeds 40% of the total number of pixels in that layer, the tumor is deemed to have penetrated that layer. The deepest penetration layer is used as the final tumor penetration layer. Tumor penetration level data is output, including labels for the starting layer (e.g., mucosa) and the ending layer (e.g., muscularis), along with the corresponding penetration depth (in millimeters, with an accuracy of 0.1 mm) for subsequent simulation input.

[0053] It is particularly important that step S13 includes the following steps:

[0054] Step S131: Calculating pixel value encoding based on the intestinal wall layer image;

[0055] In this embodiment, after completing the recognition of the five-layer structure of the intestinal wall region in the digestive tract ultrasound image, a hierarchical image of the intestinal wall with the same size as the original image is obtained. Different grayscale values in this image represent different layers of the intestinal wall structure. Specifically, each layer of the structure is assigned a fixed pixel value code to avoid overlap or ambiguity. The specific pixel coding rules are as follows: the mucosal layer corresponds to a grayscale value of 50, the submucosa corresponds to a grayscale value of 100, the muscular layer corresponds to a grayscale value of 150, and the serosa corresponds to a grayscale value of 200. The background or the portion beyond the intestinal wall area is retained as a grayscale value of 0 or 255. In this process, a morphological closing operation based on a watershed transform or Canny edge detection is used to clarify the edges of the intestinal wall, and spatial positioning of the pixel coding is achieved through layer-by-layer mask extraction. Image processing uses a fixed convolution kernel size (e.g., 3×3) combined with structuring elements for boundary enhancement to ensure that the grayscales between layers do not overlap and avoid aliasing areas.

[0056] Step S132: constructing a hierarchical label comparison table according to the pixel value encoding;

[0057] In this embodiment, according to the gray value encoding determined in the previous step, a correspondence is established between the structure of each layer and its corresponding gray value, and a structural hierarchical label comparison table is constructed. The comparison table contains five rows, namely: "mucosal layer: 50", "submucosal layer: 100", "muscular layer: 150", "serosa: 200", and "background: 0 / 255". The table is saved in the form of a dictionary structure, with the key being the pixel gray value and the value being the hierarchical label string. To ensure the stability of the comparison table, a read-only data dictionary is used in the data structure design, and it is prohibited to be modified during subsequent processing. The comparison table will serve as the core reference data for subsequent lesion penetration mapping judgment and needs to be cached in memory for real-time call during regional matching operations.

[0058] Step S133: extracting a lesion area mask based on the intestinal wall layer image;

[0059] In this embodiment, the mask image output after the lesion image segmentation is completed is first obtained. The mask image should be the same size as the original intestinal wall image and use a single-channel binary format. The pixel value corresponding to the tumor area in the image is set to 255, and the non-lesion area is set to 0. In order to enhance the spatial correspondence accuracy between the mask image and the intestinal wall hierarchical image, a bilinear interpolation registration operation is required to ensure that the overlap error of the two images is controlled within 1 pixel. The mask image removes small noise areas through morphological opening operation (using 5×5 structural elements), and then uses connected domain analysis to eliminate pseudo lesions with an area of less than 300 pixels. The pixel coordinates of the mask image are recorded as an array for subsequent pixel matching operations in the intestinal wall hierarchical image. The lesion area may contain multiple unconnected areas, and each area is treated as an independent analysis unit.

[0060] Step S134: mapping the lesion area according to the lesion area mask and the level label comparison table to obtain the intestinal wall level of the lesion area;

[0061] In this embodiment, all non-zero pixels (value 255) in the lesion mask image are used as indices to extract the grayscale values of the corresponding pixels in the intestinal wall hierarchical image. The resulting extraction is a one-dimensional array, storing the set of grayscale values corresponding to the lesion region in the intestinal wall hierarchical image. This set is deduplicated and sorted, and the grayscale values are mapped to intestinal wall structural layer labels using the hierarchical label table constructed in step S132, thereby obtaining the intestinal wall layer covered by the lesion region. If the lesion pixels only correspond to grayscale values of 50 and 100, it indicates that the tumor is confined to the mucosa and submucosa. If grayscale values of 150 or 200 are included, it indicates that the tumor has penetrated into the muscularis or serosa. During this mapping process, undefined grayscale values are prohibited. If abnormal grayscale values (non-0, 50, 100, 150, 200, or 255) are detected, the image ID is recorded and the image is entered into the manual review queue.

[0062] Step S135: performing tumor penetration analysis based on the intestinal wall layer of the lesion area to obtain tumor penetration layer data;

[0063] In this embodiment, based on the intestinal wall layer label set output in step S134, the deepest structural layer is determined to assess the tumor's penetration level. This determination is made according to established rules: if the maximum layer is the "mucosal layer," the tumor is classified as T1a; if the "submucosal layer" is included, it is T1b; if the "muscularis" layer is present, it is T2; if the "serosa" layer is included, it is T3; if the lesion area also corresponds to a background grayscale value of 0 or 255, it indicates that the tumor has penetrated all intestinal wall layers and reached the surrounding tissue, resulting in a T4 level. The determination results are stored in a structured format, including: image ID, maximum penetration layer, penetration layer label, penetration layer number (e.g., 3 for the muscularis), lesion layer area ratio (the percentage of lesion pixels per layer), and total lesion area (in mm²). The output includes three image results: the original image, an overlaid lesion layer, and a layered stained layer.

[0064] Step S14: performing hierarchical anatomical simulation according to the tumor penetration level data to obtain hierarchical anatomical data;

[0065] In this embodiment, after acquiring tumor penetration level data, a baseline intestinal wall structure map is constructed using a standard anatomical template. This template includes a 3D intestinal wall layer reference model constructed from human CT anatomy atlases (with the X-axis representing length, the Y-axis representing thickness, and the Z-axis representing longitudinal curling, with a spatial resolution of 0.5 mm). Based on the known penetration levels, the corresponding lesion anatomical region is reconstructed. A tumor growth volume (modeled as an ellipsoidal mass with a radius along the thickness direction equal to the actual penetration depth + 0.5 mm as a boundary buffer) is inserted into the baseline structure and nested within the original anatomical model. The thickness and boundary structure of the intestinal wall surrounding the lesion are modified. A B-spline surface deformation method is used to simulate the pushing and compression of the tumor on the surrounding tissue, and the adjusted anatomical data structure is output. The structural data is recorded in VTP format (3D structural point cloud), including the spatial topological coordinates, thickness vector, and normal displacement of each layer.

[0066] Step S15: calculating the overlap of lesion hierarchical boundaries based on the hierarchical anatomical data; and evaluating the tumor invasion depth based on the overlap of lesion hierarchical boundaries;

[0067] In this example, the Volume Intersection over Union (V-IoU) method is used to calculate the overlap of lesion layer boundaries. The lesion simulation structure and each intestinal wall layer region are projected into three-dimensional space. Voxel segmentation (voxel side length is set to 0.2 mm) is used to count the number of overlapping voxels and the number of union voxels between the two structures. For layer n, the calculation formula is: Overlap_n = V_intersect_n / V_union_n. The center position of the layer with the largest Overlap_n among all layers is selected as the deepest tumor invasion boundary. The invasion depth is recorded as the distance from the center of this layer to the inner edge of the intestinal lumen. This distance value is calculated (in mm, with an accuracy of 0.1 mm) and combined with the penetration layer information to form a complete invasion depth record structure, including the starting layer, ending layer, maximum boundary overlap value, and invasion depth value. The structure is output in JSON format for subsequent modeling.

[0068] Step S16: performing three-dimensional visualization modeling of the lesion according to the tumor invasion depth to obtain a digestive tract tumor lesion model;

[0069] In this embodiment, 3D visualization modeling utilizes a process based on surface contour reconstruction and volume enhancement overlay. Using tumor invasion depth as the primary axis and intestinal wall spatial coordinate data, the lesion's outer contour curve is first reconstructed. Contour point interpolation (with an interpolation accuracy of 0.25 mm) is used to connect the lesion's edge contour points to construct the tumor's main contour line. Next, cross-sectional layers (set to 1 mm spacing) are inserted at equal intervals within the invasion depth range. Within each layer, region growing segmentation is performed based on the lesion's grayscale distribution within the image. The grayscale gradient at the starting point of growth is set to a connected region greater than 150, and the boundary grayscale gradient is set to >30 as the stopping criterion. The cross-sectional contours are integrated, and a 3D mesh is constructed using equidistant grid interpolation to form a complete volumetric model of the lesion. The final lesion model is output in OBJ 3D format, containing vertex coordinates, boundary surface information, and a tumor invasion depth label, which can be used for subsequent malignancy assessment and image segmentation tasks.

[0070] Preferably, step S14 is specifically as follows:

[0071] Step S141: constructing an intestinal wall hierarchical structure model based on the tumor penetration level data;

[0072] In this embodiment, the process of constructing a hierarchical structural model of the intestinal wall based on acquired tumor penetration data first requires clear demarcation of the five layers of the intestinal wall: the mucosa, muscularis mucosa, submucosa, muscularis, and serosa. The structural model is constructed using a two-dimensional image overlay method combined with thickness vector mapping technology. Contour extraction and morphological reconstruction are performed on the pixel regions of each layer in the hierarchical image of the intestinal wall to generate a polygonal structural representation. Specifically, the Canny operator based on grayscale gradients is used to detect edges in the hierarchical image, identify boundary curves between different layers, and extract pixel regions from each layer. A contour tracing algorithm (such as the Suzuki 85 algorithm) is then used to identify point sets and reconstruct closed regions at the boundaries of each layer. Each closed region is converted to physical coordinates based on its boundary coordinate point set. A structural thickness vector is generated based on the image acquisition resolution (e.g., 0.01 mm per pixel), completing the spatial modeling of the two-dimensional hierarchical structure. The model is stored in a raster format, with a unit grid of 0.01 mm × 0.01 mm.

[0073] Step S142: in the intestinal wall hierarchical structure model, the thickness of the mucosal layer is set to 0.5-1.5 mm, the thickness of the submucosal layer is set to 0.6-2.0 mm, the thickness of the muscular layer is set to 1.0-3.0 mm, and the thickness of the serosal layer is set to 0.3-0.8 mm;

[0074] In this embodiment, the thickness of each layer in the constructed hierarchical intestinal wall model is set according to specific values. The mucosal layer thickness is set to 0.5mm to 1.5mm, with a range of 0.1mm increments; the submucosal layer thickness is set to 0.6mm to 2.0mm, also with 0.1mm increments; the muscularis layer thickness is set to 1.0mm to 3.0mm; and the serosal layer thickness is set to 0.3mm to 0.8mm. The specific thickness values are set based on a standard digestive tract anatomy database (such as Gray's Anatomy). Voxel thickness fitting is performed on each layer using multimodal imaging (ultrasound and MRI). Using a layered projection vector alignment method, the pixel width of each layer segmented on the image is mapped to actual millimeters and set within the specified intervals. The system uses linear interpolation to process regions with fluctuating image thickness, ensuring that the final model layer thickness is continuous and that structural constraints remain unchanged within each region.

[0075] Step S143: setting the penetration depth threshold of the corresponding position of the lesion invasion level in the intestinal wall hierarchical structure model to 0.1-5.0 mm, and setting the width of the tumor boundary fuzzy zone to 0.2-1.0 mm;

[0076] In this example, a penetration depth threshold is set for the lesion invasion layer within a hierarchical intestinal wall model with a set thickness, with a specific range of 0.1mm to 5.0mm. This penetration depth threshold is based on the tumor penetration layer data obtained in the previous step and the maximum depth corresponding to the lesion penetration at different layers in the image. For example, if the lesion penetrates the muscular layer, the upper threshold is set to 3.0mm. Based on this, the width of the tumor boundary fuzzy zone is set to 0.2mm to 1.0mm. This boundary fuzzy zone is generated at the lesion edge using a morphological fuzzy function (e.g., a Gaussian fuzzy radius set to 2-5 pixels of image resolution, i.e., 0.02mm-0.05mm). The fuzzy zone width is determined based on the edge gradient change value. Specifically, the fuzzy boundary zone is defined as an area with a gradient amplitude less than 0.5× the maximum gradient. This parameter setting is used to control the boundary overlap calculation range during subsequent structural overlap analysis.

[0077] Step S144: setting the blood vessel density to 5-50 per square millimeter and the lymphatic vessel density to 3-30 per square millimeter in the intestinal wall hierarchical structure model;

[0078] In this example, microscopic blood vessel and lymphatic vessel distribution parameters were introduced into the hierarchical intestinal wall model, quantitatively setting the blood vessel and lymphatic vessel densities. The blood vessel density was set to 5 to 50 per square millimeter, and the lymphatic vessel density to 3 to 30 per square millimeter. The distribution pattern was generated using a Poisson distribution method and implanted within the grid structure of the intestinal wall model. Within each grid cell (0.01 mm × 0.01 mm), the required number of microvessels per unit area was calculated based on the density value and the total area. A uniform random number generator (e.g., with mean μ = 25 and variance σ² = 10) was used to calculate the actual number of microvessels per layer. The blood vessels and lymphatic vessels were then implanted into each layer as three-dimensional vectors, with a particular emphasis on increasing the density in the submucosal and muscularis layers. The vessel diameters were controlled to range from 10 μm to 80 μm. Spatial paths were connected using the shortest path principle to generate the simulated network.

[0079] Step S145: uploading the intestinal wall hierarchical structure model to the tissue anatomy simulation system, and running the structural anatomy simulation program to output hierarchical structure anatomy data;

[0080] In this example, the constructed hierarchical intestinal wall model, including thickness settings, penetration depth thresholds, boundary fuzzy zone parameters, and vascular and lymphatic vessel distribution parameters, is uploaded to a configured tissue anatomy simulation system. The system configuration must include an image-structure mapping module, a vector network calculation module, and a physical tissue simulation engine. The upload operation is invoked via an API port or local data path, importing the model data into the simulation system in .mesh or .vtk format to ensure topological closure of each layer. The structural anatomy simulation program is then invoked. The program's logic involves: rendering multi-layer nested cross-sectional structures based on the set thickness; calculating structural stress propagation paths using a finite element mesh (with a grid spacing of 0.01 mm); and generating a tissue perfusion hierarchical network diagram based on the vascular distribution model. Finally, the structural anatomy data is output as a set of three-dimensional structural points and a collection of attribute tags (e.g., layer thickness, boundary location, penetration point location, and vascular penetration rate). This data serves as anatomical reference data for subsequent tumor modeling. The output data includes mesh node coordinates, layer IDs, structural thickness values, and related vascular density parameters.

[0081] Preferably, step S2 is specifically as follows:

[0082] Step S21: extracting a tumor tissue pathology image based on the digestive tract tumor lesion image;

[0083] In this embodiment, after obtaining the original tissue slice image within the digestive tract tumor area, the slice image is digitized using a full-cut image scanning device, the scanning resolution is set to 0.25μm / pixel, and a 20x magnification is used to ensure clear cell structure. A tissue pathology image containing a clear tumor area is selected. The image is derived from a tissue slice stained with Hematoxylin and Eosin (HE). The color characteristics include purple-blue nuclear staining areas and pink cytoplasmic areas. The digital slice image file (such as SVS format) is loaded through the OpenSlide image reading interface, and the regional block image containing the tumor area is cropped from it. The cropped area is set to 1024×1024 pixels. The image cropping is based on the coordinates of the lesion area marked by the histologist. The coordinates are annotated and exported through the digital pathology workstation. The coordinate system unit is pixel to ensure that the extracted image accurately covers the tumor tissue. The extracted image will be used as pathology image data for subsequent image processing procedures.

[0084] Step S22: performing color normalization processing based on the tumor tissue pathology image to obtain a tumor tissue pathology image to be processed;

[0085] In this embodiment, the tumor tissue pathology image acquired in step S21 is first standardized using the Reinhard color normalization algorithm to eliminate color differences between batches or scanning devices. A normalized reference image is selected from a color-stable and clearly structured HE-stained pathology image. The mean and standard deviation of the L, a, and b channels of this reference image are calculated: L: (180±10), a: (15±5), and b: (20±5), respectively. The image to be processed is then converted to Lab color space, and the channel distribution of the image to be processed is adjusted based on the channel statistics of the reference image. A linear transformation operation is performed to align the distribution of the target image in Lab space with that of the reference image. After normalization, the image is converted back to RGB color space and saved as a TIFF file to preserve the high-resolution details of the original image. The entire process is implemented using a custom ImageJ macro plug-in, and the processing flow ensures a unified color style in batch image processing.

[0086] Step S23: extracting a nuclear staining channel based on the tumor tissue pathology image to be processed to obtain nuclear staining channel data;

[0087] In this example, the nuclear staining channel is extracted from the normalized tumor histopathology image obtained in step S22 using color deconvolution. Hematoxylin and eosin serve as the basis for channel separation, and the image is subjected to three-channel projection separation using the Ruifrok and Johnston separation matrix (hematoxylin: [0.650, 0.704, 0.286]; eosin: [0.072, 0.990, 0.105]). After extraction, the nuclear staining channel is output as a grayscale image with a pixel value range of 0-255, representing the intensity of nuclear staining, with darker staining resulting in smaller grayscale values. The resulting nuclear channel image maintains the same resolution as the original image (1024×1024 pixels) and is output in PNG format to facilitate subsequent image enhancement and density analysis. This channel extraction is implemented using the color_deconvolution method in the histomicstk library in Python.

[0088] Step S24: performing contrast enhancement based on the nuclear staining channel data, wherein the CLAHE block size is set to 8×8, the contrast limit factor is set to 2.0–4.0, and the gamma adjustment range is set to 0.8–1.2, to obtain enhanced nuclear staining channel data;

[0089] In this example, contrast enhancement was performed on the extracted grayscale image of the nuclear stain channel using the CLAHE (Contrast Limited Adaptive Histogram Equalization) method for local contrast enhancement. The CLAHE tile grid size was set to 8×8, meaning each image was divided into 64 subregions. Histogram equalization was performed independently within each subregion. The contrast limit factor was set between 2.0 and 4.0, with a median value of 3.0 selected as the main experimental parameter. This value limits the maximum height of the histogram of each subregion to prevent image noise diffusion caused by excessive enhancement. The enhanced image was then gamma-corrected, with a gamma adjustment range of 0.8 to 1.2, with a default of 1.0. For bright images, a gamma of 0.8 was used to darken highlights, while for dark images, a gamma of 1.2 was used to brighten dark areas, thereby enhancing contrast at the edges of cell nuclei. This enhancement was implemented using the OpenCV functions cv2.createCLAHE() and cv2.LUT(), outputting an 8-bit grayscale image.

[0090] Step S25: Calculating the cell nuclear density based on the enhanced nuclear staining channel data, wherein the nuclear density calculation window size is set to 100 μm × 100 μm; identifying the cell nuclear dense area based on the cell nuclear density, wherein the dense area determination threshold is set to >500 nuclei / mm²; calculating the edge complexity based on the cell nuclear dense area; performing tumor region segmentation on the tumor tissue pathology image based on the edge complexity to obtain digestive tract tumor region data;

[0091] In this example, the enhanced nuclear stain channel image was used to calculate the cell nucleus density. The image space was first converted to its actual physical size. Based on the image scanning resolution of 0.25 μm / pixel, each 100 μm × 100 μm region was converted into a 400 × 400 pixel calculation window. The image was then traversed using a sliding window method. Within each window, a binarization operation was first performed, and the Otsu adaptive thresholding method was used to distinguish cell nuclei from background areas. The number of connected domains was then counted using the OpenCV cv2.connectedComponentsWithStats() method to obtain the number of cell nuclei. The dense region threshold was set to 500 nuclei per square millimeter, which means that a dense region is marked if there are 5 nuclei per 100 μm × 100 μm window. After all dense regions were marked with a logical mask, their boundaries were contour detected, and the contour boundary length and area were extracted. The edge complexity index (Complexity = Perimeter² / Area) was calculated. Regions with an edge complexity greater than 50 are designated as highly heterogeneous regions. This region set is then used as a segmentation mask to segment the original pathology image into tumor regions. A logical AND operation is then used to combine the original image's position coordinates to generate the segmentation result map. The final output is a digestive tract tumor region data layer in the form of a single-channel PNG file.

[0092] Step S26: performing tissue arrangement abnormality detection based on the digestive tract tumor region data to obtain tissue arrangement abnormality data;

[0093] In this embodiment, tissue arrangement abnormality detection is performed based on the digestive tract tumor area data generated in step S25. First, the centroid of the cell nucleus in each segmented tumor area block is extracted to obtain the center coordinates of all cell nuclei. Then, the nearest neighbor vector analysis is performed to construct the Voronoi diagram structure, and the Delaunay triangulation method is used to construct the inter-nuclear structure diagram. In this diagram, the number of neighbors, the distribution of neighbor angles, and the consistency of direction vectors (the standard deviation of the angles between direction vectors) of each cell nucleus are counted. The area where the standard deviation of the neighborhood angle exceeds 30 degrees is marked as a structural abnormality area; when the standard deviation of the inter-nuclear distance exceeds the set threshold of 20μm, it is identified as a loosely arranged or disordered area; if the aspect ratio distribution of the nuclear shape exceeds 1.8 or is less than 0.6, it is marked as an abnormally stretched or compressed nuclear area. All the above abnormal indicators are rasterized and superimposed to generate a tissue arrangement abnormality mask map, which is output as a binary image (abnormal area is 1, normal area is 0) with the same resolution as the original Figure 1 The final tissue arrangement abnormality detection result layer is used for subsequent analysis.

[0094] Preferably, step S26 is specifically as follows:

[0095] Step S261: extracting cell nucleus coordinates based on digestive tract tumor region data;

[0096] In this embodiment, the image of the located digestive tract tumor region is first processed. The image source is a stained nuclear channel image obtained after high-pass filtering and color separation in the pathological section image. The image is converted to a grayscale image and then binarized using the Otsu automatic thresholding method to separate the foreground (cell nucleus) and background. After processing, the region with a grayscale value of 255 in the image is identified as a candidate cell nucleus region. The contour extraction method in OpenCV is used to extract the contour of each connected region, and the ellipse fitting method (calling the fitEllipse function) is used to determine the geometric center position of each cell nucleus. Only candidate nucleus regions with a contour area between 15 pixels² and 300 pixels² are retained, excluding fragments and aggregation artifacts. The center coordinates of each nucleus are saved in pixel form as (x, y), and finally output as a two-dimensional coordinate array for subsequent use.

[0097] Step S262: constructing a nuclear arrangement direction map according to the cell nucleus coordinates;

[0098] In this embodiment, this step is based on the nuclear center coordinates obtained in step S261. With each nuclear point as the center, the coordinate set of the surrounding nuclear points is retrieved within a radius of 20 pixels in the image. The nuclear group composed of these points is regarded as a local unit, and its main direction is calculated. The direction extraction uses the principal component direction inference technology, that is, by estimating the direction of the distribution trend of the coordinate set, the main axis direction is determined. The direction angle of each local nuclear group obtained is recorded as an angle value and standardized between 0 and 180 degrees. Each direction data is bound to its corresponding nuclear coordinate to form a direction vector layer. The layer size is the same as the original Figure 1 The image is consistent, but each pixel only records whether there is a nucleus and its arrangement direction. The image is visualized using color coding to show different directional trends, which is used for subsequent consistency and entropy calculations.

[0099] Step S263: calculating the arrangement direction consistency coefficient based on the core arrangement direction map;

[0100] In this embodiment, based on the directional map established in the previous step, local area analysis is performed on each core as a unit. For each core, all core direction data within its 20-pixel neighborhood are extracted, and the similarity between these directions is calculated. The specific method is to compare the directional angles in all neighborhoods with the direction of the current core, and to count the concentration of their directional differences. If the directions in the neighborhood are highly concentrated and the changes are small, it is considered that the consistency is high; if the differences are large, the consistency is low. For ease of processing, the results are unified into values between 0 and 1 and expressed in the form of a grayscale image, with each core position corresponding to a consistency value. The entire consistency image is used to judge the degree of structural order in the local area.

[0101] Step S264: evaluating the degree of polarity loss based on the arrangement direction consistency coefficient;

[0102] In this embodiment, the total area of the nuclear region with a consistency value lower than 0.5 is first counted, and the ratio is calculated to the area covered by all cell nuclei to obtain the proportion of the polarity loss area. A fixed threshold rule is used. If the ratio is greater than the set threshold (for example, 30%), it is judged that there is a high degree of polarity loss overall. If local area analysis is the goal, a fixed grid sliding window (such as 200×200 pixels, with a step size of 100 pixels) can be used to block the image and record the polarity state of each area to form a polarity loss heat map.

[0103] Step S265: calculating the arrangement direction distribution entropy based on the kernel arrangement direction map;

[0104] In this example, the orientation angles of all kernels are extracted and divided into 30 angular segments, each 6 degrees. The number of kernel orientations occurring in each angular segment is counted and normalized based on the total number of kernels, forming a directional frequency distribution. Entropy is calculated based on the uniformity of this distribution. If the frequency of occurrence across all angular segments is even, the orientations are considered disordered and have a high entropy value; if the frequency is concentrated in a few directions, the entropy value is low. The final output is a single numerical value representing the complexity of the overall orientation distribution of the image.

[0105] Step S266: evaluating the degree of glandular structure damage based on the arrangement direction distribution entropy;

[0106] In this embodiment, the entire image is divided into blocks using a sliding window (e.g., 100×100 pixels), and the local directional distribution frequency is calculated within each block. Similar to step S265, the directional entropy of each block is calculated. If the entropy value of a particular region exceeds 3.5, it is considered to have a high degree of directional disorder and is considered to indicate glandular structural abnormality. The total area of all such high-entropy regions is calculated and compared with the area of the entire image to obtain area percentage data indicating the degree of glandular structural damage. This data is visualized as a mask image of the structurally damaged regions.

[0107] Step S267: determining tissue arrangement abnormality based on the degree of polarity loss and the degree of glandular structure damage, and obtaining tissue arrangement abnormality data;

[0108] In this embodiment, this step is a joint judgment process of the first two abnormal indicators. When the polarity loss in the image accounts for more than 30% and the glandular structure destruction area accounts for more than 40%, the entire image is judged as abnormal tissue arrangement. If only one of the items exceeds the threshold, it is classified and processed according to the severity. The grid sliding window analysis method can be used to divide the image into multiple local sub-blocks, and the above judgments are performed on them respectively to generate a binary mask. The processing results of each sub-block are used to construct the final tissue arrangement abnormality image, with the abnormal area displayed in white and the normal area in black. The output is a picture with the same size as the original Figure 1 The binary image is named "tissue arrangement anomaly mask map", and the anomaly level of each block is recorded in the corresponding metadata file for subsequent analysis or segmentation algorithm optimization.

[0109] Preferably, the tissue disorder index is calculated in step S3 as follows:

[0110] Calculating gradients based on tissue arrangement abnormality data to obtain gradient data;

[0111] In this embodiment, the tissue arrangement abnormality mask image obtained in the above step S267 is used as the input image. This image is a binary image with the same size as the original digestive tract tissue slice. Figure 1The value of the abnormal area is 255, and the value of the normal area is 0. The Sobel operator is used to calculate the image gradient of this binary image. Specifically, a 3×3 convolution kernel is used in the x and y directions to calculate the gradient. The convolution kernel is defined as follows: the x-direction kernel is [-1, 0, 1], [-2, 0, 2], [-1, 0, 1], and the y-direction kernel is the transposed form. The OpenCV function cv2.Sobel() is used to convolve the abnormal mask image with cv2.CV_64F precision to obtain the gradient maps Gx and Gy. The gradient intensity value G = √(Gx² + Gy²) is then calculated for each pixel position, and the gradient intensity map (i.e., gradient data) is output. This image is a grayscale image, and each pixel represents the intensity of the abnormal change in the neighborhood tissue arrangement at that point. The original image size is retained for the next step of structure tensor calculation.

[0112] Constructing a structure tensor component based on the gradient data; calculating the main direction of the texture based on the structure tensor component;

[0113] In this embodiment, the structure tensor calculation is based on the two gradient maps Gx and Gy obtained above. This step requires calculating three structure tensor components at each pixel: Jxx=Gx², Jyy=Gy², Jxy=Gx×Gy. In order to enhance local smoothness and the integration of neighborhood information, Gaussian filtering is performed on each component map. The Gaussian kernel size is 5×5, and the standard deviation σ is set to 2.0. The OpenCV function cv2.GaussianBlur() is used to perform smoothing operations on Jxx, Jyy, and Jxy respectively to generate smoothed three-channel data of the structure tensor. Finally, at each pixel, a 2×2 symmetric structure tensor matrix is obtained: J=[[Jxx,Jxy],[Jxy,Jyy]], which is used to calculate the main direction information of the texture. Each image is a single-channel grayscale image of the same size as the original image, which is used for main direction angle estimation.

[0114] Calculate the directional mutation intensity based on the main direction of the texture;

[0115] In this embodiment, this step is based on the structure tensor J of each pixel position, and the local texture direction is obtained by eigenvalue decomposition. Without using a complex eigendecomposition algorithm, the main direction angle θ is calculated using an analytical formula, and the calculation formula is: θ=0.5×arctan2(2×Jxy,Jxx-Jyy). The NumPy function np.arctan2() is used to calculate each pixel, and the result is normalized to the range of [0,π] (i.e., [0,180°]) and saved in a texture main direction map in the form of an angle. This image is a floating-point grayscale image, and the pixel value is the main texture direction angle of the tissue structure in the neighborhood of the current pixel, in degrees, with two decimal places. This image provides a direction angle reference for subsequent directional mutation intensity calculations.

[0116] Generate a binary mask of the tumor region based on the digestive tract tumor region data;

[0117] In this embodiment, the main texture direction map obtained in the previous step is used, and a fixed window (size of 11×11 pixels) is used to slide on the image with a step size of 1 pixel. In each window, the main texture direction angles of all pixels in the area are counted, and then the standard deviation of the direction angles in the window is calculated. The standard deviation of the angle value array is calculated using NumPy's np.std() function. The larger the standard deviation value, the more obvious the directional mutation in the area. Using the center position of each window as the index, the standard deviation value is assigned to the pixel position to obtain a directional mutation intensity map. The image size is the same as the original Figure 1 The value of each pixel is the degree of directional disturbance in the neighborhood of that location, in degrees. This image is used to measure the directional disorder inside the tumor core area.

[0118] Based on the binary mask of the tumor region, the tumor core region data of the digestive tract tumor region is identified to obtain the tumor core region data;

[0119] In this embodiment, the acquired multimodal digestive tract images include HE-stained images and immunohistochemically labeled images (such as Ki-67). Based on the digestive tract tumor region image data obtained in the previous step, a color space separation method (using Lab or HED space) is used to extract the tumor region staining, and region identification is performed using a fixed threshold. Taking the HE-stained image as an example, the D channel (i.e., hematoxylin staining) is separated in the HED color space and binarized using the Otsu method or a fixed grayscale threshold (e.g., 128). In the binary image, regions with a pixel value of 255 represent tumor tissue regions. A morphological closing operation (structuring element size of 5×5) is used to fill holes in the mask image, ultimately resulting in a binary mask image containing only the tumor region. This mask image, which is the same size as the original image, is used for further identification of the tumor core region.

[0120] The tissue disorder index of the tumor core area data was calculated according to the directional mutation intensity to obtain the tissue disorder index.

[0121] In this embodiment, the tumor area mask image is used to mask the original HE image, retaining only the tumor area. Subsequently, a kernel density estimation analysis is performed in the area covered by the mask to identify the area with dense cell nuclei. The specific operation is as follows: on the tumor mask image, based on the cell nucleus coordinate information extracted in the aforementioned step S261, the cell nucleus distribution is modeled using the Gaussian kernel density estimation method. A two-dimensional Gaussian kernel function with a bandwidth of 15 pixels is used to perform spatial density estimation on the cell nucleus coordinates. The KernelDensity module of scikit-learn is used to construct a kernel density distribution map. The density map is threshold segmented, and the threshold is set to 65% of the maximum density value of the image, and the dense area is extracted as the tumor core area. The output result is a mask image of the tumor core area, which is used for subsequent calculation of the tissue disorder index.

[0122] Preferably, the identification of high-risk lesion areas in step S3 is specifically as follows:

[0123] Identify areas of high tissue disorder based on the tissue disorder index;

[0124] In this embodiment, the tissue disorder index is calculated based on the intensity of the main texture direction mutation within the core region of the tumor. To identify areas of high tissue disorder, the entire pathology slide image is first divided into non-overlapping image blocks of 64×64 pixels. The tissue disorder index value of each image block is compared and analyzed with the mean value of all image blocks in the entire image. The mean value of the tissue disorder index of the image block is used as the reference baseline, and a standard deviation σ is set. According to the threshold judgment standard: if the tissue disorder index of a certain block is greater than the mean plus 2 times σ (μ + 2σ), then the image block is marked as a high tissue disorder area. The standard deviation σ is obtained by taking the square root of the variance of the tissue disorder index of all image blocks. This method ensures the statistical robustness of the region division. The entire operation is completed by image traversal and block-by-block calculation. The result is marked in the original image as a binary mask.

[0125] The intensity of cell arrangement disorder is calculated based on the area of high tissue disorder;

[0126] In this embodiment, in the area with high disorder of tissue arrangement, the cell nucleus recognition operation is first performed. The threshold segmentation algorithm is combined with the nuclear staining extraction method based on color enhancement, and the H channel in the HSV color space is used to perform Otsu threshold segmentation to extract the cell nucleus outline. For each cell nucleus, its major axis direction is extracted, and the direction is calculated by the main axis direction of the minimum circumscribed ellipse. Then, the angle between the major axes of adjacent cell nuclei is calculated in each area, and the arrangement consistency is expressed by the standard deviation. For all cell nuclei contained in a region, let the angle difference between adjacent nuclei be θ, and calculate the average angle difference θ -The disorder intensity is set to σ_θ and its standard deviation σ_θ. If σ_θ in a region is greater than 30°, it is considered a region with high cell disorder. This threshold is set based on manual observation experience and can be optimized by using the angular difference distribution of multiple samples. All operations are performed on image blocks, ultimately resulting in a two-dimensional matrix containing the cell disorder intensity.

[0127] Fiber bundle structure fracture detection is performed based on the intensity of cell arrangement disorder to obtain fiber bundle structure fracture data;

[0128] In this embodiment, the fiber bundle structure is extracted by analyzing the light and dark strip textures in the tissue image that are linear in direction. The image texture in different directions is enhanced using a Gabor filter. The filter parameters are set to a frequency of f = 0.1, and the direction θ is rotated from 0° to 180° at intervals of 15°. A total of 12 response maps in different directions are extracted. In the obtained filter response map, local directional continuity analysis is performed on areas with high intensity of cell disorder. The judgment logic is: if the same linear structure (based on continuous unidirectional response as the criterion) is interrupted simultaneously in more than three directions and the Gabor response value is less than 0.2, it is determined that there is a fiber bundle structure break in this area. The contour of each broken area is extracted, and its spatial coordinates and area are marked. The output is a binary image of the fiber bundle structure break and vector information of the structural break area for subsequent detection.

[0129] Detection of proliferation activity signals based on fiber bundle structure disruption data;

[0130] In this embodiment, in the detected fracture area of the fiber bundle structure, the proliferation activity signal is judged by the density of the number of cell nuclei in the staining image. In the fracture area, the number of cell nuclei per unit area is calculated. The specific operation is as follows: the fracture area is divided into a fixed-size grid of 10μm×10μm, and the cell nuclei in the grid are counted using a cell nucleus mask. If the density threshold is set to more than 25 cell nuclei per 100μm² area, it is marked as an active proliferation signal. This threshold is based on statistical data from pathology literature, taking into account the actual slice magnification and pixel conversion ratio. All marking information is stored in the output result in the form of coordinates and area masks for subsequent identification of suspicious lesion areas.

[0131] Identify suspicious lesion areas based on active proliferation signals;

[0132] In this embodiment, a spatial overlap analysis is performed on the fiber bundle breakage area and the proliferation activity signal. If a certain area satisfies both the fiber bundle structure breakage and the cell nuclear density exceeds the threshold, it is determined to be a suspicious lesion area. The specific operation process includes: performing a pixel-level logical AND operation on all fiber bundle breakage areas and proliferation signal areas to obtain the overlapping area. Areas with an overlapping area greater than 30% of the original area are retained. This 30% overlap ratio is determined based on the overlap statistics of multiple tumor areas after manual annotation. Finally, the suspicious lesion area is extracted and numbered in the form of a polygonal outline, and a binary mask and vector boundary description are output.

[0133] Detection of cell nuclear area based on suspicious areas of lesions;

[0134] In this embodiment, the area of all segmented cell nuclei in the suspicious lesion area is calculated. First, the image ratio is converted, and the actual area of a single pixel is converted based on the slice magnification (such as 40X) and the image resolution (such as 0.25μm / pixel), that is, 1 pixel corresponds to 0.0625μm². Then, each cell nucleus segmentation mask is traversed, the number of pixels is counted and multiplied by the unit pixel area to obtain the actual area of each cell nucleus. For a certain area, the area of all cell nuclei is counted and their mean μ_A and standard deviation σ_A are recorded to obtain the distribution characteristics of the cell nucleus area in the area, providing basic data support for the identification of high-risk lesion areas.

[0135] Identify high-risk areas of lesions based on nuclear area.

[0136] In this embodiment, the distribution of cell nucleus area in all suspicious lesion areas is analyzed. If the mean cell nucleus area in a certain area is greater than or equal to 60 μm² and the standard deviation σ_A is greater than 20 μm², the area is marked as a high-risk lesion area. The area standard is set according to the average size range of tumor cell nuclei in pathological statistics. The area of each suspicious area is calculated and scored separately. Areas with scores higher than the set threshold are marked with special annotations in the output image. The results are saved in the form of regional boundary polygons and center point coordinates for image segmentation models or doctors to assist in judgment.

[0137] Preferably, the lesion edge morphology recognition in step S3 is specifically as follows:

[0138] Identify the initial edge contour of the lesion based on the high-risk area of the lesion;

[0139] In this embodiment, after obtaining the image of the high-risk area of the lesion, the image of the area is first converted into a two-dimensional grayscale image, and the Sobel operator is used to extract its edge information. A 3×3 Sobel convolution kernel is used to calculate the gradient in the horizontal and vertical directions respectively to obtain the horizontal gradient Gx and the vertical gradient Gy. The formula G= The image gradient magnitude map is calculated. Non-maximum suppression and a double-threshold connection method are then used to extract continuous edges. The low threshold is set at the 20th percentile of the image gradient magnitude distribution, and the high threshold is set at the 80th percentile. The extracted results are binarized, and contours are represented with a width of 1 pixel. Finally, a contour tracing algorithm based on pixel connectivity (such as the 8-neighborhood tracing algorithm based on Freeman chain codes) is used to obtain the complete initial lesion edge contour. All contour points are stored as a sequence of ordered 2D coordinates.

[0140] Perform smoothing according to the initial edge contour of the lesion to obtain the optimized initial edge contour of the lesion;

[0141] In the present embodiment, the coordinate sequence of the initial edge contour of the lesion is used as input and processed by a curve smoothing method based on Gaussian filtering. First, a Gaussian kernel with a window size of 7 is selected, and the standard deviation σ is set to 1.5 to generate a one-dimensional Gaussian weight sequence. For each contour point, three adjacent points are taken before and after it, and the weighted average is used to calculate its new position using Gaussian weights, and the coordinate values of the X-axis and Y-axis are smoothed in sequence. After one round of this operation is performed on the entire contour point sequence, it is repeated three times to improve the smoothing effect. To avoid obvious shrinkage of the edge contour, after each smoothing iteration, a boundary-preserving algorithm is used to detect and retain key turning points. Turning point detection adopts the cosine angle method to calculate the angle formed by three consecutive points. Those with an angle less than 100 degrees are marked as boundary points, and their original positions are retained and do not participate in smoothing. The final output is the optimized initial edge contour of the lesion that has been smoothed for three rounds and retains key boundary points.

[0142] Calculate the edge curvature based on the optimized initial edge contour of the lesion;

[0143] In this embodiment, the curvature of the optimized contour point sequence is calculated, and the three-point approximation method is used to calculate the local curvature. Specifically, for the i-th contour point Pi=(xi,yi) in the sequence, take its adjacent points Pi- and Pi+1, calculate the vectors Vi1=Pi-Pi-1 and Vi2=Pi+1-Pi, and calculate the angle θi=arccos[(Vi1 Vi2) / (||Vi1|| ||Vi2||)]. The curvature κi = θi / d is then calculated, where d is the Euclidean distance between Pi-1 and Pi+1. To improve accuracy, contour points are padded so that the distance between any two adjacent points does not exceed 2 pixels. After the curvature calculation is complete, all curvature values are normalized to the range [0, 1]. This curvature value sequence serves as the basic data input for subsequent high-curvature region identification and concave-convex change feature detection.

[0144] Based on the edge curvature, the high edge curvature area is calibrated, and the concave-convex change characteristics of the high edge curvature area are detected to obtain the concave-convex change data of the lesion edge;

[0145] In this embodiment, the normalized curvature value sequence of the previous stage is used, and the high curvature threshold is set to 0.6. All continuous regions with curvature greater than the threshold are identified, and each continuous region is treated as an independent high curvature region. Concave-convex feature detection is performed on each high curvature region using the contour normal direction variation method. The normal direction of each point is calculated using the three-point method, that is, the tangent direction formed by one point before and after the current point is taken as the normal. The directional change Δθ between adjacent normals is then calculated in sequence within the region. If the Δθ direction change is continuously increasing in the same direction (positive or negative), it is judged to be a convex region; if the Δθ direction first increases and then decreases or first decreases and then increases, it is judged to be a concave region. The concave-convex change information of each high curvature region, including the starting point, end point, concave-convex type, total Δθ change, region length and other data, is recorded as the concave-convex change data of the lesion edge.

[0146] Polyp villus detection is performed based on the concave-convex change data of the lesion edge to obtain polyp villus data;

[0147] In this embodiment, data that meets the structural characteristics of polyp villi is screened from all calibrated concave-convex change regions. A polyp villus is defined as a continuous edge region with frequent curvature fluctuations (Δθ change rate greater than 0.2 radians / pixel), a region length between 10-30 pixels, and at least three alternating concave and convex occurrences. The detection method is to set a sliding window (window size is 30 points, step size is 5 points), and during the sliding process, count the number of concave-convex changes within the window, the maximum Δθ, the minimum Δθ, and the average change amplitude. Window segments that meet the above criteria are marked as polyp villus regions. Each polyp villus data record records the coordinates of the region's start and end points, contour length, number of concave-convex cycles, maximum curvature change, and average curvature fluctuation value, and generates a polygonal contour envelope as input for subsequent edge morphology analysis.

[0148] Lesion edge morphology recognition is performed based on polyp villus data to obtain lesion edge morphology data.

[0149] In this embodiment, edge morphology recognition is performed based on the set morphology classification criteria, combining the number, distribution density, average length, and concave-convex period of polyp villi. The set morphology is divided into three categories: (1) dense villi (the number of polyp villi per 100 pixels of edge length is greater than 5), (2) sparse villi (the number is between 2 and 5), and (3) no obvious villi (the number is less than 2). The polyp villi data of each contour line are counted and classified according to a 100-pixel sliding window, and the morphological labels are assigned values of 1, 2, and 3 respectively. For the case where multiple types appear alternately, the coordinates of the morphological transition points are recorded and the regional changes are marked. Finally, a lesion edge morphology data structure is formed, which contains the morphological classification results of each edge segment, the morphological change location point, the polyp villi density, the average length and distribution trend vector, and other information, providing boundary structure information support for subsequent fine segmentation of lesions or benign and malignant discrimination.

[0150] Preferably, step S4 is specifically as follows:

[0151] Step S41: performing spatial extensibility detection on the lesion edge morphology data according to the digestive tract tumor lesion model to obtain lesion edge spatial extensibility data;

[0152] In this embodiment, the lesion edge morphology data needs to be used as input data for three-dimensional structure reconstruction. This operation adopts the Marching Cubes isosurface extraction algorithm. The specific process is: multiple two-dimensional lesion edge contour image slices are imported in sequence, each image slice corresponds to a slice with a fixed thickness, and the thickness is set to 0.5 mm. The image sequence composed of all slices is stacked in space to form a voxel space structure. The voxel space is input into the Marching Cubes algorithm module, and the gradient value of each voxel unit is judged. The three-dimensional isosurface mesh is constructed according to the grayscale value of the lesion edge morphology data to generate a three-dimensional mesh surface model with a continuous topological structure. The generated three-dimensional surface model is topologically optimized using triangulation to keep the edges tightly connected and continuous. Subsequently, a dense point cloud sampling tool (such as the VoxelGrid filter based on the PCL library) was used to perform spatial point sampling on the 3D edge model. The edge surface was sampled into a set of equally spaced points. Each point recorded its position in 3D space in millimeters. The extracted point coordinates were formatted as (xi,yi,zi), and the inter-sample spacing was kept within 0.1 mm to ensure sufficient spatial resolution for microstructural analysis. Subsequently, the local principal direction curvature of these sampled points was calculated using a structural tensor analysis method. During this calculation, a point set with a 1 mm radius surrounding each sampled point was constructed. A local structure tensor was constructed from these points, and the principal curvature values for each point in the axial (along the main direction of the tissue), radial (perpendicular to the direction of lesion development), and tangential (along the edge contour) directions were obtained by solving the eigenvalues and eigenvectors of the tensor. A 3D curvature analysis tool (such as the curvature estimation module in CGAL) was used to extract the rate of change of curvature in each direction. The spatial extensibility was further quantified by combining it with the 3D edge envelope features. To achieve this, a closed envelope is first generated based on the edge point set. The volume of the interior space of the envelope is calculated using a hexahedral cell approximation algorithm, with an approximation accuracy of 0.1 mm. The edge surface area is then estimated, and the edge point set is constructed into a hole-free triangular mesh using a mesh triangulation method. The total surface area is calculated by accumulating the area of each triangular cell. The spatial extension characteristics of each edge region are analyzed by combining the envelope volume and edge area. The spatial extension calculation results are output as a graph, with each triangular mesh cell corresponding to a spatial extension value. To facilitate subsequent evaluation and processing, these values are visualized as a grayscale image with a grayscale range of 0 to 255, where a grayscale value of 255 represents the most spatially extended location of the edge structure and a grayscale value of 0 represents an area of no extension or extreme contraction. The graph is output as a three-dimensional voxel grayscale image for subsequent malignancy assessment and analysis.The entire process does not involve machine learning models and is entirely based on geometric and spatial analysis algorithms. The parameters and algorithm selections for each processing step are fixed to ensure that the processing process is standardized and repeatable.

[0153] Step S42: evaluating the malignancy of the digestive tract tumor based on the spatial extension data of the lesion margin to obtain malignancy data of the digestive tract tumor;

[0154] In this embodiment, the degree of malignancy of each lesion area is divided according to the spatial extensibility data obtained. First, the grayscale value in the edge spatial extensibility data is graded, and a five-level quantitative grading is adopted: grayscale value 0–50 corresponds to level 1, 51–100 corresponds to level 2, and so on to grayscale value 201–255 corresponds to level 5. The lesion growth characteristics corresponding to each level are corrected by the clinical anatomical statistical data provided by the expert group. In order to avoid the deviation of the evaluation results due to the complexity of the lesion morphology, the edge fractal dimension DfD_fDf is used for verification. The calculation method adopts the box counting method. At a magnification of 10 times, the square grid coverage statistics of the lesion edge contour are performed to obtain the edge fractal dimension value, and it is fitted with the corresponding spatial extensibility level for regression analysis. Finally, the formula M=αEs+βDf was used to quantify the degree of malignancy, where α=0.6 and β=0.4 were weight coefficients, the output range was 0–1, and the intervals [0, 0.3), [0.3, 0.6), [0.6, 0.8), and [0.8, 1.0] were defined as low malignancy, medium-low malignancy, medium-high malignancy, and high malignancy, respectively.

[0155] Step S43: calculating the cell mitosis index based on the digestive tract tumor malignancy data;

[0156] In this example, based on the malignancy grade atlas output from the previous stage, the number of mitotic cells per field of view was calculated by extracting nuclear staining regions within highly malignant areas. This operation used high-resolution HE-stained images (with an image resolution of 0.25 μm / pixel) as input. A color thresholding method based on staining separation was used to isolate the nuclear regions, with the blue-violet staining channel threshold set to [100, 180], retaining only the dense nucleoplasm region. Morphological operations (dilation-erosion) were performed on each field of view (with an area of 0.2 mm²) to exclude nuclear debris and atypical nuclear structures. Subsequently, fixed template matching was used to identify typical mitotic nuclear division features, including bipolar divisions and anaphase separation. The number of mitotic cells counted in each image was divided by the total cell number to obtain the mitotic index, expressed as a percentage, with values ranging from 0–20%.

[0157] Step S44: demarcating a high-malignancy region of the tumor based on the cell mitosis index; and detecting the degree of tissue invasion based on the high-malignancy region of the tumor;

[0158] In this example, based on the distribution of cell mitotic index in different regions, threshold intervals were set: [0–2%], [2–5%], [5–10%], and [10–20%], corresponding to low, relatively low, moderate, and high proliferation states, respectively. Regions with a mitotic index greater than 5% were designated as highly malignant. Region boundaries were segmented using the maximum gradient edge method, and the Sobel operator was used to extract gradient directions within an 8-neighborhood, followed by a region connectivity test. Tissue invasion detection involved extracting the vertical depth between the tumor boundary and anatomical tissues, including the submucosa, muscularis, and serosa. Boundary extraction was performed using a three-channel contrast-enhanced image (the red channel enhances the tumor bulk, the green channel enhances the muscularis, and the blue channel enhances the serosa). Tissue-level boundaries were determined using an adaptive Canny edge detection algorithm. The deepest lesion invasion layer was measured and used to define the invasion depth, corresponding to T1 (mucosa), T2 (muscularis), T3 (serosa), and T4 (beyond the serosa). All measurements were in millimeters, with an error of no more than ±0.5 mm.

[0159] Step S45: Segment the digestive tract tumor lesion image into malignant regions according to the degree of tissue invasion to generate a digestive tract malignant tumor lesion image.

[0160] In this embodiment, after obtaining the tissue invasion level, a multi-layer regional mask merging strategy is used to perform a malignant region segmentation operation on the lesion image. First, a mask layer is constructed based on the invasion depth label map, and different mask grayscale values are assigned to each level of invasion depth area, which are set to 50, 100, 150, and 200 for T1-T4 respectively. The mask layer is then phase-anded with the original image to retain the malignant invasion area corresponding to the high grayscale value area. Image processing uses image mask technology based on OpenCV to perform a mask matching operation on each pixel point. The output image is an 8-bit grayscale image, retaining the malignant area and setting the pixel values of the remaining areas to 0. The final output malignant tumor lesion image is saved in TIFF format, and the image resolution is maintained at the level of the original image (not less than 300dpi). The image file is accompanied by invasion level annotation information for subsequent analysis and archiving comparison.

[0161] Preferably, this specification also provides a multimodal digestive tract tumor lesion image segmentation system, which is used to execute the multimodal digestive tract tumor lesion image segmentation method described above. The multimodal digestive tract tumor lesion image segmentation system includes:

[0162] The lesion 3D visualization module is used to obtain images of digestive tract tumor lesions and extract digestive tract tumor ultrasound images; assess the tumor invasion depth based on digestive tract tumor ultrasound images; and perform 3D visualization modeling of the lesion based on the tumor invasion depth to obtain a digestive tract tumor lesion model;

[0163] The tissue arrangement abnormality detection module is used to extract tumor tissue pathology images based on the digestive tract tumor lesion images; perform tumor region segmentation based on the tumor tissue pathology images to obtain digestive tract tumor region data; perform tissue arrangement abnormality detection based on the digestive tract tumor region data to obtain tissue arrangement abnormality data;

[0164] The lesion edge morphology recognition module is used to calculate the tissue disorder index based on the tissue arrangement abnormality data and the digestive tract tumor area data; identify the high-risk lesion area based on the tissue disorder index; and perform lesion edge morphology recognition based on the high-risk lesion area to obtain lesion edge morphology data;

[0165] The malignant region segmentation module is used to evaluate the malignancy of digestive tract tumors based on the lesion edge morphology data according to the digestive tract tumor lesion model to obtain digestive tract tumor malignancy data; based on the digestive tract tumor malignancy data, the malignant region of the digestive tract tumor lesion image is segmented to generate a digestive tract malignant tumor lesion image.

[0166] The present invention is therefore intended to be illustrative and non-restrictive in all respects, with the scope of the invention being defined by the appended claims rather than the foregoing description, and all changes that come within the meaning and range of equivalents of the application documents are intended to be embraced therein.

[0167] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is to be construed in the widest possible manner consistent with the principles and novel features disclosed herein.

Claims

1. A multimodal digestive tract tumor lesion image segmentation method, characterized in that: The following steps are involved: Step S1: Acquire a digestive tract tumor lesion image and extract a digestive tract tumor ultrasound image; assess the tumor invasion depth based on the digestive tract tumor ultrasound image; perform three-dimensional visualization modeling of the lesion based on the tumor invasion depth to obtain a digestive tract tumor lesion model; Step S2: extracting a tumor tissue pathology image based on the digestive tract tumor lesion image; performing tumor region segmentation based on the tumor tissue pathology image to obtain digestive tract tumor region data; performing tissue arrangement abnormality detection based on the digestive tract tumor region data to obtain tissue arrangement abnormality data; Step S3: calculating a tissue disorder index based on the tissue disorder data and the digestive tract tumor area data; identifying high-risk lesion areas based on the tissue disorder index; and performing lesion edge morphology recognition based on the high-risk lesion areas to obtain lesion edge morphology data. Step S4: Evaluate the malignancy of the digestive tract tumor based on the lesion edge morphology data according to the digestive tract tumor lesion model to obtain digestive tract tumor malignancy data; segment the digestive tract tumor lesion image into malignant areas based on the digestive tract tumor malignancy data to generate a digestive tract malignant tumor lesion image.

2. The multimodal digestive tract tumor lesion image segmentation method according to claim 1, characterized in that: Step S1 is specifically as follows: Step S11: Acquire a digestive tract tumor lesion image and extract a digestive tract tumor ultrasound image; Step S12: identifying the intestinal wall hierarchical structure based on the digestive tract tumor ultrasound image to obtain an intestinal wall hierarchical image; Step S13: determining the tumor penetration level based on the intestinal wall layer image to obtain tumor penetration level data; Step S14: performing hierarchical anatomical simulation according to the tumor penetration level data to obtain hierarchical anatomical data; Step S15: calculating the overlap of lesion hierarchical boundaries based on the hierarchical anatomical data; and evaluating the tumor invasion depth based on the overlap of lesion hierarchical boundaries; Step S16: Perform three-dimensional visualization modeling of the lesion according to the tumor invasion depth to obtain a digestive tract tumor lesion model.

3. The multimodal digestive tract tumor lesion image segmentation method according to claim 2, characterized in that: Step S14 is specifically as follows: Step S141: constructing an intestinal wall hierarchical structure model based on the tumor penetration level data; Step S142: in the intestinal wall hierarchical structure model, the thickness of the mucosal layer is set to 0.5-1.5 mm, the thickness of the submucosal layer is set to 0.6-2.0 mm, the thickness of the muscular layer is set to 1.0-3.0 mm, and the thickness of the serosal layer is set to 0.3-0.8 mm; Step S143: setting the penetration depth threshold of the corresponding position of the lesion invasion level in the intestinal wall hierarchical structure model to 0.1-5.0 mm, and setting the width of the tumor boundary fuzzy zone to 0.2-1.0 mm; Step S144: setting the blood vessel density to 5-50 per square millimeter and the lymphatic vessel density to 3-30 per square millimeter in the intestinal wall hierarchical structure model; Step S145: Upload the intestinal wall hierarchical structure model to the tissue anatomy simulation system, run the structural anatomy simulation program, and output the hierarchical structure anatomy data.

4. The multimodal digestive tract tumor lesion image segmentation method according to claim 1, characterized in that: Step S2 is specifically as follows: Step S21: extracting a tumor tissue pathology image based on the digestive tract tumor lesion image; Step S22: performing color normalization processing based on the tumor tissue pathology image to obtain a tumor tissue pathology image to be processed; Step S23: extracting a nuclear staining channel based on the tumor tissue pathology image to be processed to obtain nuclear staining channel data; Step S24: performing contrast enhancement based on the nuclear staining channel data, wherein the CLAHE block size is set to 8×8, the contrast limit factor is set to 2.0–4.0, and the gamma adjustment range is set to 0.8–1.2, to obtain enhanced nuclear staining channel data; Step S25: Calculating the cell nuclear density based on the enhanced nuclear staining channel data, wherein the nuclear density calculation window size is set to 100 μm × 100 μm; identifying the cell nuclear dense area based on the cell nuclear density, wherein the dense area determination threshold is set to >500 nuclei / mm²; calculating the edge complexity based on the cell nuclear dense area; performing tumor region segmentation on the tumor tissue pathology image based on the edge complexity to obtain digestive tract tumor region data; Step S26: Perform tissue arrangement abnormality detection based on the digestive tract tumor region data to obtain tissue arrangement abnormality data.

5. The multimodal digestive tract tumor lesion image segmentation method according to claim 4, characterized in that: Step S26 is specifically as follows: Step S261: extracting cell nucleus coordinates based on digestive tract tumor region data; Step S262: constructing a nuclear arrangement direction map according to the cell nucleus coordinates; Step S263: calculating the arrangement direction consistency coefficient based on the core arrangement direction map; Step S264: evaluating the degree of polarity loss based on the arrangement direction consistency coefficient; Step S265: calculating the arrangement direction distribution entropy based on the kernel arrangement direction map; Step S266: evaluating the degree of glandular structure damage based on the arrangement direction distribution entropy; Step S267: Determine tissue arrangement abnormality based on the degree of polarity loss and the degree of glandular structure damage to obtain tissue arrangement abnormality data.

6. The multimodal digestive tract tumor lesion image segmentation method according to claim 1, characterized in that: The tissue disorder index is calculated in step S3 as follows: Calculating gradients based on tissue arrangement abnormality data to obtain gradient data; Constructing a structure tensor component based on the gradient data; calculating the main direction of the texture based on the structure tensor component; Calculate the directional mutation intensity based on the main direction of the texture; Generate a binary mask of the tumor region based on the digestive tract tumor region data; Based on the binary mask of the tumor region, the tumor core region data of the digestive tract tumor region is identified to obtain the tumor core region data; The tissue disorder index of the tumor core area data was calculated according to the directional mutation intensity to obtain the tissue disorder index.

7. The multimodal digestive tract tumor lesion image segmentation method according to claim 1, characterized in that: The identification of high-risk lesion areas in step S3 is specifically as follows: Identify areas of high tissue disorder based on the tissue disorder index; The intensity of cell arrangement disorder is calculated based on the area of high tissue disorder; Fiber bundle structure fracture detection is performed based on the intensity of cell arrangement disorder to obtain fiber bundle structure fracture data; Detection of proliferation activity signals based on fiber bundle structure disruption data; Identify suspicious lesion areas based on active proliferation signals; Detection of cell nuclear area based on suspicious areas of lesions; Identify high-risk areas of lesions based on nuclear area.

8. The multimodal digestive tract tumor lesion image segmentation method according to claim 1, characterized in that: The lesion edge morphology recognition in step S3 is specifically as follows: Identify the initial edge contour of the lesion based on the high-risk area of the lesion; Perform smoothing according to the initial edge contour of the lesion to obtain the optimized initial edge contour of the lesion; Calculate the edge curvature based on the optimized initial edge contour of the lesion; Based on the edge curvature, the high edge curvature area is calibrated, and the concave-convex change characteristics of the high edge curvature area are detected to obtain the concave-convex change data of the lesion edge; Polyp villus detection is performed based on the concave-convex change data of the lesion edge to obtain polyp villus data; Lesion edge morphology recognition is performed based on polyp villus data to obtain lesion edge morphology data.

9. The multimodal digestive tract tumor lesion image segmentation method according to claim 1, characterized in that: Step S4 is specifically as follows: Step S41: performing spatial extensibility detection on the lesion edge morphology data according to the digestive tract tumor lesion model to obtain lesion edge spatial extensibility data; Step S42: evaluating the malignancy of the digestive tract tumor based on the spatial extension data of the lesion margin to obtain malignancy data of the digestive tract tumor; Step S43: calculating the cell mitosis index based on the digestive tract tumor malignancy data; Step S44: demarcating a high-malignancy region of the tumor based on the cell mitosis index; and detecting the degree of tissue invasion based on the high-malignancy region of the tumor; Step S45: Segment the digestive tract tumor lesion image into malignant regions according to the degree of tissue invasion to generate a digestive tract malignant tumor lesion image.

10. A multimodal digestive tract tumor lesion image segmentation system, characterized in that: For executing the multimodal digestive tract tumor lesion image segmentation method according to claim 1, the multimodal digestive tract tumor lesion image segmentation system comprises: The lesion 3D visualization module is used to obtain images of digestive tract tumor lesions and extract digestive tract tumor ultrasound images; assess the tumor invasion depth based on digestive tract tumor ultrasound images; and perform 3D visualization modeling of the lesion based on the tumor invasion depth to obtain a digestive tract tumor lesion model; The tissue arrangement abnormality detection module is used to extract tumor tissue pathology images based on the digestive tract tumor lesion images; perform tumor region segmentation based on the tumor tissue pathology images to obtain digestive tract tumor region data; perform tissue arrangement abnormality detection based on the digestive tract tumor region data to obtain tissue arrangement abnormality data; The lesion edge morphology recognition module is used to calculate the tissue disorder index based on the tissue arrangement abnormality data and the digestive tract tumor area data; identify the high-risk lesion area based on the tissue disorder index; and perform lesion edge morphology recognition based on the high-risk lesion area to obtain lesion edge morphology data; The malignant region segmentation module is used to evaluate the malignancy of digestive tract tumors based on the lesion edge morphology data according to the digestive tract tumor lesion model to obtain digestive tract tumor malignancy data; based on the digestive tract tumor malignancy data, the malignant region of the digestive tract tumor lesion image is segmented to generate a digestive tract malignant tumor lesion image.

Images