Gastric mucosa intestinal epithelium metaplasia grading method, device and equipment and storage medium

Abstract

The invention discloses a gastric mucosa intestinal epithelium metaplasia grading method and device, equipment and a storage medium, and the method comprises the steps: obtaining a current full-slice image, and carrying out the preprocessing, and obtaining slice image blocks; the slice image blocks are input to a preset double-branch classification model, a gland classification result is obtained, and the preset double-branch classification model comprises a first branch constructed based on a residual network, a second branch constructed based on a Transform network, a fusion module and a decoder; determining a current intestinal gland ratio based on a gland classification result; and comparing the current intestinal gland ratio with a preset sydnichthys system grading threshold value, and determining a sydnichthys system grading result. According to the method, the local features and the global features are extracted and fused, so that the identification precision of the gland region is improved, and the classification of the intestinal glands and the normal glands is more accurate; and the ratio of the intestine metaplasia glands is combined with the sydney system grading standard, so that objective and quantitative intestine metaplasia grading is realized.

Description

Technical Field

[0001] The present application relates to the technical field of medical image analysis, and in particular to a method, device, equipment and storage medium for grading intestinal metaplasia of gastric mucosa. Background Art

[0002] Intestinal metaplasia (IM), also known as intestinal metaplasia, is an abnormal change in the gastric mucosa characterized by the gradual replacement of gastric epithelial cells with intestinal-like epithelial cells. As a key precancerous lesion, IM plays a crucial role in the development of gastric cancer. Currently, the detection of IM relies primarily on analysis of whole-slice images (WSIs) of gastric mucosal biopsies. Pathologists visually assess the proportion of normal glands and intestinal metaplasia (IM), thereby determining the Sydney grade, which provides a basis for patient diagnosis and clinical trials.

[0003] With the continuous advancement of artificial intelligence (AI) technology, AI models have shown great potential in assisting pathologists in grading IM. However, existing AI models (such as GasMIL) still have many limitations in their application. First, most of these models can only classify IM and fail to effectively establish a mapping relationship between pathological morphology and quantitative criteria. Second, when segmenting glandular structures, existing AI models struggle to simultaneously extract local features and accurately grasp global information. As a result, they cannot provide sufficiently intuitive and interpretable quantitative results of pathological changes, thus limiting their application value and scope in clinical practice. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment and storage medium for grading gastric mucosal intestinal metaplasia, aiming to solve the technical problem of the lack of interpretability of the existing AI model-assisted manual IM grading method.

[0005] To achieve the above objectives, the present application proposes a method for grading gastric mucosal intestinal metaplasia, the method comprising: Acquiring a current full-slice image and preprocessing the current full-slice image to obtain a slice image block; Inputting the slice image block into a preset two-branch classification model to obtain a gland classification result, the preset two-branch classification model includes: a first branch constructed based on a residual network, a second branch constructed based on a Transformer network, a fusion module and a decoder, the first branch is used to extract local features of the slice image block, the second branch is used to extract global features of the slice image block, the fusion module is used to perform feature fusion, and the decoder is used to output the gland category probability according to the fused features; Determining corresponding pixel values ​​of the intestinal metaplasia gland region and the normal gland region in the current full-slice image based on the gland classification result, and determining a current intestinal metaplasia gland ratio value according to the pixel values; The current intestinal metaplasia gland ratio value is compared with a preset Sydney system grading threshold to determine a Sydney system grading result corresponding to the current full-slice image.

[0006] In one embodiment, the step of acquiring a current full-slice image and preprocessing the current full-slice image to obtain a slice image block includes: Acquiring the current full-slice image and performing downsampling processing on the current full-slice image to obtain a low-resolution overview image; segmenting the low-resolution overview image using an automatic threshold segmentation algorithm to obtain an overview image including a tissue segmentation mask; performing channel enhancement on the overview image including the tissue segmentation mask according to the staining condition of the current full-slice image, and performing glandular region marking on the channel-enhanced overview image using a preset morphological algorithm to obtain an overview image including the glandular region segmentation mask; The overview image including the glandular region segmentation mask is aligned with the current full slice image, and the current full slice image is cut based on the glandular region segmentation mask to obtain slice image blocks.

[0007] In one embodiment, the step of using a preset morphological algorithm to mark the glandular region on the channel-enhanced overview image to obtain the overview image including the glandular region segmentation mask includes: Selecting a preset structural element, performing a top-hat transformation operation on the channel-enhanced overview image to obtain a glandular cavity-enhanced overview image; Segmenting the enhanced overview image of the glandular cavity using the automatic threshold segmentation algorithm to determine candidate regions of the glandular cavity; Using the candidate region of the glandular cavity as a marker, a watershed algorithm is used to segment the enhanced overview image of the glandular cavity to obtain an overview image with glandular region markers; The segmentation result of the overview image with glandular region markers is optimized by using a morphological post-processing method to obtain an overview image including a glandular region segmentation mask.

[0008] In one embodiment, the step of inputting the slice image block into a preset dual-branch classification model to obtain a gland classification result includes: Extracting local features from the slice image block through the first branch to obtain local features, wherein the local features include gland edge features, cell texture features, and gland duct structure features; Performing global feature extraction on the slice image block through the second branch to obtain global features, wherein the global features include: horizontal gland distribution features and vertical gland topology features; Determining a current spatial weight through the fusion module, and performing weighted fusion on the local features and the global features based on the current spatial weight to obtain a fused feature; Mapping the fusion features through the decoder to determine the gland category probability corresponding to the slice image block; Masks corresponding to different gland categories are generated in the current full slice image according to the gland category probabilities corresponding to the slice image blocks.

[0009] In one embodiment, the step of determining the current spatial weight and performing weighted fusion on the local features and the global features based on the current spatial weight to obtain the fused features includes: Splicing the local features and the global features to generate a joint feature vector; Performing a nonlinear transformation on the joint feature vector using a unit convolutional layer to obtain a scalar weight value, and normalizing the scalar weight value using a Sigmoid activation function to obtain a current spatial weight, where the current spatial weight includes a local feature weight and a global feature weight; The local features and the global features are weightedly fused according to the local feature weights and the global feature weights to obtain fused features.

[0010] In one embodiment, the step of comparing the current intestinal metaplasia gland ratio value with a preset Sydney system grading threshold to determine the Sydney system grading result corresponding to the current full-slice image includes: Based on the Sydney system grading, several Sydney grades are determined, and the corresponding intestinal metaplasia area percentage thresholds are determined according to the Sydney grades; Comparing the current intestinal metaplasia gland ratio value with each intestinal metaplasia area ratio threshold value to obtain a comparison result; The Sydney system level corresponding to the current full-slice image is determined according to the comparison result.

[0011] In one embodiment, the step of determining the Sydney system level corresponding to the current full-slice image according to the comparison result includes: Determining a threshold difference between the current intestinal metaplasia gland ratio value and each of the intestinal metaplasia area ratio thresholds; When the threshold difference falls within a preset critical value range, determining that the current full-slice image is a critical sample, and triggering a manual review prompt based on the critical sample to enable a professional physician to determine the Sydney system grade corresponding to the current full-slice image; When the threshold difference does not meet the preset critical value range, the Sydney system level corresponding to the current full-slice image is determined according to the comparison result.

[0012] In addition, to achieve the above objectives, the present application also proposes a gastric mucosal intestinal metaplasia grading device, the device comprising: An image preprocessing module is used to obtain a current full-slice image and preprocess the current full-slice image to obtain a slice image block; a model classification module, configured to input the slice image block into a preset two-branch classification model to obtain a gland classification result, wherein the preset two-branch classification model includes: a first branch constructed based on a residual network, a second branch constructed based on a Transformer network, a fusion module, and a decoder, wherein the first branch is configured to perform local feature extraction on the slice image block, the second branch is configured to perform global feature extraction on the slice image block, the fusion module is configured to perform feature fusion, and the decoder is configured to output a gland category probability based on the fused features; a hierarchical mapping module, configured to determine corresponding pixel values ​​of the intestinal metaplasia gland region and the normal gland region in the current full-slice image based on the gland classification result, and determine a current intestinal metaplasia gland ratio value according to the pixel values; The grading mapping module is further configured to compare the current intestinal metaplasia gland ratio value with a preset Sydney system grading threshold to determine a Sydney system grading result corresponding to the current full-slice image.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a gastric mucosal intestinal metaplasia grading device, which includes: a memory, a processor, and a gastric mucosal intestinal metaplasia grading program stored on the memory and runnable on the processor, wherein the gastric mucosal intestinal metaplasia grading program is configured to implement the steps of the gastric mucosal intestinal metaplasia grading method as described above.

[0014] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and stores a gastric mucosal intestinal metaplasia grading program. When the gastric mucosal intestinal metaplasia grading program is executed by a processor, the steps of the gastric mucosal intestinal metaplasia grading method described above are implemented.

[0015] The present application discloses a method for grading intestinal metaplasia of gastric mucosa, comprising: obtaining a current full-slice image, and preprocessing the current full-slice image to obtain a slice image block; inputting the slice image block into a preset two-branch classification model to obtain a gland classification result, the preset two-branch classification model comprising: a first branch constructed based on a residual network, a second branch constructed based on a Transformer network, a fusion module, and a decoder, the first branch being used to perform local feature extraction on the slice image block, the second branch being used to perform global feature extraction on the slice image block, the fusion module being used to perform feature fusion, and the decoder being used to output a gland category probability based on the fused features; determining corresponding pixel values ​​of an intestinal metaplasia gland area and a normal gland area in the current full-slice image based on the gland classification result, and determining a current intestinal metaplasia gland ratio value based on the pixel values; and comparing the current intestinal metaplasia gland ratio value with a preset Sydney system grading threshold to determine a Sydney system grading result corresponding to the current full-slice image.

[0016] Because this application can fuse local features and global features, it improves the recognition accuracy of glandular areas, making the classification of intestinal metaplasia glands and normal glands more accurate. Through the collaboration of the two-branch model, not only the comprehensiveness of feature extraction is improved, but also the adaptability of the model to complex pathological images is enhanced. In addition, this application directly combines the proportion of intestinal metaplasia glands with the Sydney system grading standard to achieve objective and quantitative intestinal metaplasia grading, effectively reducing the subjective differences in manual evaluation, improving the consistency and reliability of diagnosis, and providing a more accurate basis for clinical decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 This is a flow chart of the first embodiment of the gastric mucosal intestinal metaplasia grading method of the present application; Figure 2 A schematic diagram of the model architecture of the dual-branch classification model preset for this application; Figure 3 This is a flow chart of the second embodiment of the gastric mucosal intestinal metaplasia grading method of the present application; Figure 4 This is a flow chart of the third embodiment of the gastric mucosal intestinal metaplasia grading method of the present application; Figure 5 This is a schematic diagram of the module structure of the first embodiment of the gastric mucosal intestinal metaplasia grading device of the present application; Figure 6 This is a schematic diagram of the structure of the gastric mucosal intestinal metaplasia grading device of this application.

[0020] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0022] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The present invention provides a method for grading gastric intestinal metaplasia, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of a method for grading gastric mucosal intestinal metaplasia of the present application. In this embodiment, the method includes steps S10 to S40: Step S10: obtaining a current full-slice image, and preprocessing the current full-slice image to obtain a slice image block.

[0024] It should be noted that the execution subject of the method of this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, mobile phone, system server, etc. It can also be other terminal devices that can access the gastric mucosal intestinal metaplasia grading system, and this embodiment is not limited to this. The following uses the gastric mucosal intestinal metaplasia grading device (hereinafter referred to as the "device") as an example to illustrate various embodiments of this application.

[0025] It should be understood that the current whole slice image is a whole slice image (WSI) obtained from the original gastric mucosal biopsy, and the resolution of the image can be 0.25 m / pixel, meaning a single WSI can reach billions of pixels. Because gastric mucosal WSIs have complex backgrounds (including interference from mucus, red blood cells, necrotic tissue, and other factors) and diverse glandular structures, preprocessing can be performed on the current full-slice image: the glandular region is separated from the complex background to generate standardized input (patches) for model adaptation, i.e., slice image blocks.

[0026] In a specific implementation, the device can obtain the current full-slice image, then perform coarse tissue region segmentation and fine gland region segmentation in sequence to obtain the WSI marked with the gland region, and finally cut the WSI marked with the gland region into several patches with a model-adapted input size (for example, 1024*1024 pixels), thus obtaining the slice image blocks.

[0027] Step S20: Input the slice image block into a preset two-branch classification model to obtain a gland classification result. The preset two-branch classification model includes: a first branch constructed based on a residual network, a second branch constructed based on a Transformer network, a fusion module and a decoder. The first branch is used to extract local features of the slice image block, the second branch is used to extract global features of the slice image block, the fusion module is used to perform feature fusion, and the decoder is used to output the gland category probability based on the fused features.

[0028] It should be noted that here you can refer to Figure 2 The model architecture of the dual-branch classification model preset in this application is specifically described. Figure 2 Schematic diagram of the model architecture of the dual-branch classification model preset for this application.

[0029] Depend on Figure 2 It can be seen that the preset dual-branch classification model can be constructed based on an encoder-decoder architecture, wherein the encoder part can include: a first branch, a second branch, and a fusion module.

[0030] The first branch is the neural network (CNN) branch, which can use the residual network (ResNet) to extract local features (local texture, gland edges); the second branch is the Transformer branch, which can model the long-range spatial dependencies of glands based on axial attention (such as the global distribution pattern of intestinal metaplastic glands in WSI and their adjacent relationship with normal glands).

[0031] The fusion module can dynamically fuse global features and local features using gated weights to obtain fused features.

[0032] The encoder can map the fused features and convert them into pixel-level classification results (normal / intestinal metaplasia / background) to obtain the gland category probability.

[0033] In practice, this pre-set two-branch classification model receives and processes the aforementioned slice image patches. Through a process of "local feature extraction - global feature extraction - feature fusion - classification output," it classifies each pixel within the patch into its gland category (normal gland / intestinal metaplasia gland / background). This yields gland category probabilities for each patch, and then generates masks corresponding to different gland categories within the current full-slice image.

[0034] It should also be noted that the training process of the preset two-branch classification model can be as follows: first, the classification model is initialized based on an encoder-decoder architecture, where the encoder portion can include a first branch constructed based on a residual network and a second branch constructed based on a Transformer network, and the decoder can use a multi-layer decoder. Next, a certain number of WSIs are obtained from an open-source WSI dataset and pixel-wise annotated: normal glands are labeled green, intestinal metaplasia glands are labeled yellow, and the background is left unlabeled, thereby obtaining a training dataset (for example, 320 WSIs containing 5212 intestinal metaplasia glands). Finally, the initial classification model is trained using this training dataset and optimized using the Dice Loss function to address class imbalance. Five-fold cross-validation is used to improve model robustness, ultimately resulting in the preset two-branch classification model.

[0035] Furthermore, in order to specifically illustrate the role of each branch in the preset dual-branch model, step S20 specifically includes: steps S201 to S205: Step S201: extracting local features from the slice image block through the first branch to obtain local features, where the local features include gland edge features, cell texture features, and gland duct structure features.

[0036] It should be noted that the first branch can be built based on a lightweight residual network (ResNet), for example, using ResNet-50 without the fully connected layers as the backbone for feature extraction. The shallow convolutional layers (C2-C3) in this ResNet can extract low-level features such as edges and textures (e.g., gland boundaries and cell textures), while the deep convolutional layers (C4-C5) can extract structural features (e.g., glandular duct branching and cell density distribution).

[0037] In practice, ResNet takes a 1024*1024*3 slice image patch as input, extracts multi-level local features from shallow layers (edges) to deep layers (structure) through shallow and deep convolution layers, and finally outputs a multi-scale local feature map (e.g., 768*768*2048, containing multi-level details from edges to structures). This directly reflects the microscopic morphology of the gland, serving as a key basis for subsequent segmentation boundaries.

[0038] Step S202: performing global feature extraction on the slice image block through the second branch to obtain global features, where the global features include horizontal gland distribution features and vertical gland topology features.

[0039] It should be noted that the second branch can be constructed based on the axial attention Transformer network, which can divide the input slice image block into, for example, 16*16 sub-blocks, and calculate the attention in the row direction and column direction respectively.

[0040] Row attention can capture the global correlation of glandular extension in the horizontal direction (such as the lateral distribution of continuous intestinal metaplasia areas), namely the horizontal glandular distribution characteristics; while column attention can capture the global correlation of glandular topology in the vertical direction (such as the longitudinal aggregation of intestinal metaplasia areas), namely the vertical glandular topology characteristics, and obtain a low-resolution global feature map (for example, 768*768*512).

[0041] Since global features can be used to capture the "aggregation" (such as continuous intestinal metaplasia areas) or "discreteness" (such as isolated intestinal metaplasia foci) of intestinal metaplasia glands, it can avoid the misjudgment of local features due to field of view limitations.

[0042] Step S203: determining the current spatial weight through the fusion module, and performing weighted fusion on the local features and the global features based on the current spatial weight to obtain fused features.

[0043] It should be noted that the spatial weight can be a dynamic weight value assigned to each spatial position (pixel or sub-block), which represents the relative importance of the local feature and the global feature at that position. In order to specifically illustrate how to calculate the spatial weight for feature extraction, step S203 specifically includes: steps S2031 to S2033: Step S2031: concatenate the local features and the global features to generate a joint feature vector.

[0044] In the specific implementation, we can first adjust the local features and global features to the same resolution (for example, 768*768*512), and then splice their information to generate a joint feature vector as follows:

[0045] Among them, (i, j) is the spatial coordinate, f local is a local feature, f global It is a global feature.

[0046] Step S2032: Use a unit convolution layer to perform a nonlinear transformation on the joint feature vector to obtain a scalar weight value, and normalize the scalar weight value through a Sigmoid activation function to obtain a current spatial weight, where the current spatial weight includes a local feature weight and a global feature weight.

[0047] In the specific implementation, a 1*1 convolutional layer (conv) can be used to transform the joint feature vector z i,j Perform nonlinear transformation and output scalar weight value, then normalize the weight value to the [0,1] interval through Sigmoid activation function to obtain the current spatial weight, so as to ensure that the fusion coefficient of local and global features is reasonable.

[0048] Step S2033: performing weighted fusion on the local features and the global features according to the local feature weights and the global feature weights to obtain fused features.

[0049] After determining the current spatial weight, the fusion module can fuse local features and global features with the same resolution through weighted summation and output a fused feature map (768*768*512). The fusion formula is as follows:

[0050] Among them, (i, j) is the spatial coordinate, w i,j is the local feature weight of the position, (1-w i,j ) is the global feature weight of the position, and are the eigenvalues ​​of local features and global features at (i, j) respectively.

[0051] Step S204: mapping the fusion features through the decoder to determine the gland category probability corresponding to the slice image block.

[0052] Step S205: generating masks corresponding to different gland categories in the current full slice image according to the gland category probabilities corresponding to the slice image blocks.

[0053] It should be noted that the decoder can compress the number of channels of the fused features to 3 (corresponding to three categories of probabilities: normal gland / intestinal metaplasia gland / background) through the unit convolution layer, and then normalize it through the Softmax function to obtain the gland category probability of each pixel.

[0054] First, set a classification threshold for each glandular category (e.g., intestinal metaplasia, normal gland, background, etc.). The classification threshold determines the minimum probability for a pixel to belong to a particular category. For example, a classification threshold of 0.5 could indicate that a pixel is considered to belong to a particular category if the probability of that pixel belonging to that category is greater than or equal to 0.5.

[0055] Next, the probability of each pixel in the current slice image block belonging to a different gland category is extracted; according to the classification threshold, each pixel is classified into the corresponding gland category to obtain a preliminary segmentation mask.

[0056] Finally, the preliminary segmentation mask is subjected to morphological post-processing such as closing operation, opening operation and connected domain analysis, and finally masks corresponding to different gland categories are obtained: intestinal metaplasia gland mask and normal gland mask.

[0057] Step S30: determining the corresponding pixel values ​​of the intestinal metaplasia gland area and the normal gland area in the current full-slice image based on the gland classification result, and determining the current intestinal metaplasia gland ratio value according to the pixel values.

[0058] It should be noted that, first, all the slice image blocks of the current full slice image can be traversed, and the number of pixels corresponding to the intestinal gland mask (the pixel area of ​​the intestinal gland region N IM ) and the number of pixels corresponding to the normal gland mask (normal gland area pixel area N NORM ).

[0059] Then, we can calculate the pixel area N of the intestinal gland region. IM Compared with the normal gland area pixel area N NORM The total effective pixels N are calculated by summing TOTAL , thereby eliminating the influence of the background pixel area.

[0060] Finally, the intestinal metaplasia gland ratio (IMRatio) can be calculated using the following formula:

[0061] Step S40: comparing the current intestinal metaplasia gland ratio value with a preset Sydney system grading threshold to determine a Sydney system grading result corresponding to the current full-slice image.

[0062] It should be noted that the Sydney system is an internationally recognized pathological assessment standard for chronic gastritis, which categorizes intestinal metaplasia (IM) into three grades: mild, moderate, and severe. The preset Sydney system grading thresholds are pre-set based on the number of grades described above, thereby mapping the Sydney system's qualitative descriptions to fixed mathematical thresholds.

[0063] In practice, the current intestinal metaplasia glandular ratio is compared with the thresholds for each level to determine the Sydney grade range to which it belongs. This determines the Sydney grading result for the current full-slice image, thus completing a closed-loop quantitative chain of pixel-level segmentation, area ratio, and Sydney grading. Compared to the existing pathologist-based visual grading method, this approach eliminates the subjectivity of Sydney grading results.

[0064] This embodiment can fuse local features and global features to perform glandular category segmentation, thereby improving the recognition accuracy of glandular regions, thereby making the classification of intestinal metaplasia glands and normal glands more accurate. Through the collaboration of the two-branch model, not only the comprehensiveness of feature extraction is improved, but also the adaptability of the model to complex pathological images is enhanced. In addition, this application directly combines the proportion of intestinal metaplasia glands with the Sydney system grading standard to achieve objective and quantitative intestinal metaplasia grading, effectively reducing the subjective differences in manual evaluation, improving the consistency and reliability of diagnosis, and providing a more accurate basis for clinical decision-making.

[0065] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 3 , Figure 3 This is a flow chart of the second embodiment of the gastric mucosal intestinal metaplasia grading method of the present application.

[0066] In this embodiment, in order to specifically illustrate the process of preprocessing the current full slice image to obtain a slice image block, step S10 specifically includes: steps S101 to S104: Step S101: acquiring the current full-slice image, and performing downsampling processing on the current full-slice image to obtain a low-resolution overview image.

[0067] It should be understood that due to the extremely high resolution of WSI, directly processing the entire image would result in excessive computational overhead for the device (for example, a single 100,000×100,000 pixel image contains 10 billion pixels). However, downsampling can quickly capture global structure by reducing resolution (at the expense of some detail) while preserving the contour information of the tissue region (such as the location and shape of the tissue block).

[0068] Therefore, the current full-slice image can first be downsampled according to a fixed ratio (such as 1:4 or 1:8) to generate a low-resolution overview image.

[0069] Step S102: segmenting the low-resolution overview image using an automatic threshold segmentation algorithm to obtain an overview image including a tissue segmentation mask.

[0070] It should be understood that the aforementioned low-resolution overview image can first be grayscaled, and then an automatic threshold segmentation algorithm (such as the Otsu algorithm) can be used to automatically calculate the optimal threshold to segment the image into "tissue area" (high grayscale value) and "background area" (low grayscale value).

[0071] It's important to note that the Otsu algorithm, based on the statistical properties of the image's grayscale histogram, finds a threshold that maximizes the inter-class variance between the segmented foreground (tissue) and background (i.e., the most distinct pixel class). The grayscale values ​​of gastric mucosal tissue (including cells and glands) are significantly higher than those of the background (glass slide and mucus), so the Otsu algorithm can effectively separate the tissue and background regions in the image.

[0072] Next, based on the Otsu threshold segmentation results, the overview image can be binarized (the tissue area is marked as 1 and the background is 0), and morphological operations (such as closing operations) can be used to fill small holes in the tissue area and connect broken small blocks, and finally generate an initial tissue mask (containing only the general outline of the tissue). This can exclude large areas of background such as the edge of the slide and mucus lakes, and limit the subsequent processing range to the tissue area (such as the mucosal tissue of the gastric antrum and gastric body), greatly reducing the amount of calculation.

[0073] Step S103: performing channel enhancement on the overview image including the tissue segmentation mask according to the staining condition of the current full-slice image, and marking the glandular region on the channel-enhanced overview image using a preset morphological algorithm to obtain an overview image including the glandular region segmentation mask.

[0074] It should be understood that since WSI is typically stained with H&E (hematoxylin and eosin), where hematoxylin stains cell nuclei blue and eosin stains cytoplasm / extracellular matrix red, channel enhancement can be performed on the tissue region to obtain a channel-enhanced overview.

[0075] For example, CLAHE (contrast-limited adaptive histogram equalization) can be applied to the hematoxylin channel (blue) to enhance the edge contrast of the cell nucleus; and Gaussian filtering can be performed on the eosin channel (red) to suppress background noise and highlight the weak red signal in the glandular lumen.

[0076] It should be noted that the preset morphological algorithm can be a morphological top-hat algorithm. Using the channel-enhanced overview image as input, the morphological top-hat algorithm first selects a preset structuring element (e.g., a disk-shaped nucleus with a radius of 5 pixels) and performs a top-hat transformation (top-hat transformation = original image - opening operation result) on the channel-enhanced overview image to obtain an enhanced glandular cavity overview image.

[0077] Since the top hat transform can be used to extract bright details (areas brighter than the background) in an image, the glandular lumen (the eosin-stained low-cell density area) appears brighter than the surrounding glandular epithelium after enhancement. Therefore, the top hat transform algorithm can "highlight" the glandular lumen from the background.

[0078] Next, based on the top-hat transformation results, an automatic threshold segmentation algorithm (such as an adaptive threshold segmentation algorithm or the aforementioned Otsu algorithm) can be used to segment the glandular cavity enhanced overview image to determine the candidate area of ​​the glandular cavity; then, the candidate area of ​​the glandular cavity is used as a marker and segmented using a watershed algorithm to segment the adhered glandular area into independent glands, thereby obtaining an overview image with glandular area markers.

[0079] Because gastric mucosal glands often become adhered (multiple glands closely connected) due to inflammation or hyperplasia, traditional threshold segmentation methods cannot separate them. However, the watershed algorithm effectively addresses the segmentation of adhered glands by simulating the process of liquid submersion and utilizing weak boundaries between glands (such as the interstitial region) as natural boundaries. This allows for the generation of finely identified glandular regions (an overview image with glandular region labels).

[0080] Finally, based on the output of the watershed algorithm (an overview image with glandular area markings), the segmentation results can be optimized through morphological post-processing (such as removing small areas with an area of ​​less than 50 pixels and filling tiny holes inside the glands), and finally a gland / background binary mask is generated (glandular area marked as 1 and background as 0), thereby ensuring that the glandular area segmentation mask only contains real glandular structures (normal or intestinal metaplasia).

[0081] Step S104: aligning the overview image including the glandular region segmentation mask with the current full slice image, and segmenting the current full slice image based on the glandular region segmentation mask to obtain slice image blocks.

[0082] It should be understood that since the aforementioned downsampling process and morphological operations may cause a slight shift in the image, the coordinates of the glandular region segmentation mask can be aligned with the coordinates of the current full-slice image; then, with the glandular region as the core, a sliding window is used on the current full-slice image to cut a 1024*1024 pixel patch (including the gland and a small amount of background around it), thereby ensuring that each patch contains sufficient contextual information (such as the boundary between the gland and the stroma).

[0083] The preprocessing process in this embodiment adopts a three-level progressive strategy of "coarse positioning-fine identification-patch cutting": coarse positioning quickly eliminates large areas of background and narrows the processing range; fine identification uses staining enhancement, morphological transformation, and watershed algorithm to accurately separate adherent glands; patch cutting can generate standardized input while retaining the spatial location information of the glands (by pairing the mask with the original image), thus solving the preprocessing difficulties of WSI "high resolution, complex background, and gland adhesion" and providing high-quality input for the high-precision segmentation of subsequent models.

[0084] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar contents as those in the first and second embodiments can be referred to above and will not be described in detail. Figure 4 , Figure 4 This is a flow chart of the third embodiment of the gastric mucosal intestinal metaplasia grading method of the present application.

[0085] In this embodiment, in order to specifically illustrate how to implement the area ratio-Sydney level mapping, step S40 includes: steps S401 to S403: Step S401: determining a number of Sydney grades based on the Sydney system grading, and determining a corresponding intestinal metaplasia area ratio threshold according to each Sydney grade.

[0086] It should be understood that the Sydney grading system can classify intestinal metaplasia (IM) into mild, moderate, and severe. The corresponding IM area percentage intervals for each Sydney grade can then be determined as follows: Level 0 (negative): IMRatio = 0; Grade 1 (mild intestinal metaplasia): 0≤ IMRatio<33.3%; Grade 2 (moderate intestinal metaplasia): 33.3≤IMRatio≤66.7%; Grade 3 (severe intestinal metaplasia): IMRatio>66.7%.

[0087] Among them, 0, 33.3%, and 66.7% are the intestinal metaplasia area percentage thresholds (grading thresholds) corresponding to different Sydney grades.

[0088] Step S402: Compare the current intestinal metaplasia gland ratio value with each intestinal metaplasia area ratio threshold value to obtain a comparison result.

[0089] Step S403: Determine the Sydney system level corresponding to the current full-slice image according to the comparison result.

[0090] It should be understood that by comparing the current intestinal metaplasia gland ratio value with the above-mentioned grading thresholds, it is possible to determine in which Sydney grade interval the current intestinal metaplasia gland ratio value falls, thereby determining the Sydney system grade corresponding to the current full-slice image.

[0091] Furthermore, considering that the current intestinal metaplasia gland ratio may be close to the above-mentioned intestinal metaplasia area ratio threshold, in order to prevent the influence of boundary conditions, step S403 further includes steps S4031 to S4033: Step S4031: determining a threshold difference between the current intestinal metaplasia gland ratio value and each intestinal metaplasia area ratio threshold.

[0092] It should be understood that, first, the grading threshold closest to the current intestinal metaplasia gland ratio value may be determined, and then the difference between the current intestinal metaplasia gland ratio value and the closest grading threshold may be calculated.

[0093] For example, if the current intestinal metaplasia gland ratio is 32.3% or 34.3%, the closest intestinal metaplasia area ratio threshold is 33.3%. The corresponding threshold differences are 1%.

[0094] Step S4032: When the threshold difference meets the preset critical value range, the current full-slice image is determined to be a critical sample, and a manual review prompt is triggered based on the critical sample to enable a professional physician to determine the Sydney system level corresponding to the current full-slice image.

[0095] It should be understood that, from the above example, the current intestinal metaplasia glandular proportion values ​​of 32.3% and 34.3% are close, and since the grading threshold is 33.3%, they will be divided into two different Sydney system grades of mild intestinal metaplasia and moderate intestinal metaplasia, which may lead to misjudgment in clinical applications.

[0096] Therefore, a critical value range can be set in advance, for example, to [-3%, +3%], and the current full-slice image whose corresponding threshold difference is within the critical value range can be regarded as a critical sample and submitted to a professional physician for manual review. Experienced pathologists can then make a final judgment on the critical sample based on the morphological features of the current full-slice image (such as glandular structure and cell atypia), taking into account both efficiency and accuracy, in line with the principle of "rigor first" in pathological diagnosis.

[0097] Step S4033: When the threshold difference does not meet the preset critical value range, the Sydney system level corresponding to the current full-slice image is determined according to the comparison result.

[0098] Accordingly, when the threshold difference is not inconsistent with the preset critical value range, the current full-slice image is not a critical sample, and the Sydney grade interval to which it belongs can be directly determined according to the current intestinal metaplasia gland ratio value, thereby determining the Sydney system grade corresponding to the current full-slice image.

[0099] Furthermore, in another embodiment, to achieve critical sample optimization, the aforementioned preset two-branch classification model can be improved by changing the model task from classification to prediction, and by changing the output layer of the decoder in the preset two-branch classification model to a regression layer, thereby directly predicting the IMRatio value of the input slice image block. Furthermore, when labeling the training data, the glandular type annotation is changed to the actual IMRatio value annotation, and the loss function is changed to the mean squared error (MSE) or smoothing loss function that optimizes the model's ability to fit continuous values. This results in an improved preset two-branch classification model, namely, a prediction model.

[0100] The segmented image blocks are input into the prediction model, and the prediction model can output a prediction result of, for example, IMRatio=32.5%.

[0101] Next, considering that the prediction model's results may still contain uncertainty due to input noise (such as uneven H&E staining) or model generalization error (such as the training set not covering certain pathological subtypes), a Monte Carlo Dropout (MCDropout) approach can be used to perform multiple forward propagations on the prediction model, obtain multiple prediction results, and calculate the mean (as the best estimate of IMRatio) and standard deviation (as a measure of uncertainty) to obtain a confidence interval. A smaller standard deviation indicates a higher confidence level in the prediction model; a larger standard deviation indicates a higher uncertainty.

[0102] For example, after 20 MCDropout inferences on the same input, the prediction results can be [31.8%, 32.5%, 33.2%, ..., 34.1%], with a mean of 33.1% and a standard deviation of 1.2%. The confidence interval is 33.1% 1.2%.

[0103] Since the confidence interval of IMRatio = 33.1% above crosses the grading threshold (33.3%), it is determined to be a critical sample, triggering manual review, thereby compensating for the insufficient predictive reliability of the model in complex or rare cases.

[0104] For example, if the confidence interval is 28.5% to 29.8% (mean 29.1%, standard deviation 0.6%), that is, the confidence interval is completely lower than the classification threshold of 33.3%, then no manual review is required and the corresponding Sydney grade can be directly determined to be 1. For example, if the confidence interval is 34.5% to 36.2% (mean 35.3%, standard deviation 0.8%), that is, the confidence interval is completely higher than the grading threshold of 33.3%, there is no need for manual review, and the corresponding Sydney level can be directly determined to be level 2.

[0105] This predictive model can transform discrete classifications into a refined assessment of "continuous values ​​+ confidence." Regression modeling preserves the specific numerical information of IMRatio, while MCDropout is used to quantify prediction uncertainty. Manual review is only triggered for high-risk, critical samples. This approach balances efficiency and accuracy, aligning with the principle of "rigor first" in pathology diagnosis.

[0106] When determining the Sydney grade based on the current intestinal metaplasia gland ratio and the grading threshold, this embodiment takes into account that critical samples whose current intestinal metaplasia gland ratio is close to the grading threshold may be affected by model input noise and model generalization error. Therefore, manual review is introduced when determining critical samples. Experienced pathologists make the final judgment on critical samples based on the morphological characteristics of WSI (such as glandular structure and cell atypia), significantly improving the accuracy of intestinal metaplasia grading diagnosis and providing more reliable grading results for clinical decision-making.

[0107] This application also proposes a gastric mucosal intestinal metaplasia grading device, referring to Figure 5 , Figure 5 Schematic diagram of the module structure of the gastric mucosal intestinal metaplasia grading device of this application.

[0108] Depend on Figure 5 It can be seen that the gastric mucosal intestinal metaplasia grading device includes: An image preprocessing module 501 is configured to obtain a current full-slice image and preprocess the current full-slice image to obtain a slice image block; A model classification module 502 is configured to input the slice image block into a preset two-branch classification model to obtain a gland classification result. The preset two-branch classification model includes: a first branch constructed based on a residual network, a second branch constructed based on a Transformer network, a fusion module, and a decoder. The first branch is configured to perform local feature extraction on the slice image block, the second branch is configured to perform global feature extraction on the slice image block, the fusion module is configured to perform feature fusion, and the decoder is configured to output a gland category probability based on the fused features. A hierarchical mapping module 503 is configured to determine corresponding pixel values ​​of the intestinal metaplasia gland region and the normal gland region in the current full-slice image based on the gland classification result, and determine a current intestinal metaplasia gland ratio value according to the pixel values; The grading mapping module 503 is further configured to compare the current intestinal metaplasia gland ratio value with a preset Sydney system grading threshold to determine a Sydney system grading result corresponding to the current full-slice image.

[0109] This embodiment can fuse local features and global features to perform glandular category segmentation, thereby improving the recognition accuracy of glandular regions, thereby making the classification of intestinal metaplasia glands and normal glands more accurate. Through the collaboration of the two-branch model, not only the comprehensiveness of feature extraction is improved, but also the adaptability of the model to complex pathological images is enhanced. In addition, this application directly combines the proportion of intestinal metaplasia glands with the Sydney system grading standard to achieve objective and quantitative intestinal metaplasia grading, effectively reducing the subjective differences in manual evaluation, improving the consistency and reliability of diagnosis, and providing a more accurate basis for clinical decision-making.

[0110] The present application also provides a gastric mucosal intestinal metaplasia grading device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the gastric mucosal intestinal metaplasia grading method in the above-mentioned embodiment one.

[0111] Reference below Figure 6 , Figure 6 This is a schematic diagram of the structure of the gastric mucosal intestinal metaplasia grading device of the present application. The gastric mucosal intestinal metaplasia grading device in the embodiments of the present application can include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and fixed terminals such as digital TVs and desktop computers. Figure 6 The gastric mucosal intestinal metaplasia grading device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0112] like Figure 6As shown, the gastric mucosal intestinal metaplasia grading device may include a processing device 1001 (e.g., a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the gastric mucosal intestinal metaplasia grading device. Processing device 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, hard disk, etc.; and communication devices 1009. Communication devices 1009 can allow the gastric mucosal intestinal metaplasia grading device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows a gastric mucosal intestinal metaplasia grading device with various systems, it should be understood that implementation or presence of all the illustrated systems is not required. More or fewer systems may alternatively be implemented or present.

[0113] The gastric mucosal intestinal metaplasia grading device provided in this application utilizes the gastric mucosal intestinal metaplasia grading method described in the aforementioned embodiment, thereby resolving the technical issues surrounding gastric mucosal intestinal metaplasia grading. Compared to existing methods, the gastric mucosal intestinal metaplasia grading device provided in this application has the same beneficial effects as the gastric mucosal intestinal metaplasia grading method described in the aforementioned embodiment. Other technical features of this gastric mucosal intestinal metaplasia grading device are the same as those disclosed in the aforementioned embodiment and are not further elaborated upon here.

[0114] The present application also provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the gastric mucosal intestinal metaplasia grading method in the above-mentioned embodiment.

[0115] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0116] The computer-readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned method for grading gastric mucosal intestinal metaplasia. This computer-readable storage medium can address the technical issues inherent in the gastric mucosal intestinal metaplasia grading method. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the gastric mucosal intestinal metaplasia grading method provided in the aforementioned embodiments, and are not further elaborated here.

[0117] The above descriptions are only some embodiments of the present application and do not limit the scope of the present application. All equivalent structural transformations made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the scope of protection of the present application.

Claims

1. A method for grading gastric mucosal intestinal metaplasia, characterized in that: The method comprises: Acquiring a current full-slice image and preprocessing the current full-slice image to obtain a slice image block; Inputting the slice image block into a preset two-branch classification model to obtain a gland classification result, the preset two-branch classification model includes: a first branch constructed based on a residual network, a second branch constructed based on a Transformer network, a fusion module and a decoder, the first branch is used to extract local features of the slice image block, the second branch is used to extract global features of the slice image block, the fusion module is used to perform feature fusion, and the decoder is used to output the gland category probability according to the fused features; Determining corresponding pixel values ​​of the intestinal metaplasia gland region and the normal gland region in the current full-slice image based on the gland classification result, and determining a current intestinal metaplasia gland ratio value according to the pixel values; The current intestinal metaplasia gland ratio value is compared with a preset Sydney system grading threshold to determine a Sydney system grading result corresponding to the current full-slice image.

2. The method according to claim 1, wherein The step of acquiring the current full-slice image and preprocessing the current full-slice image to obtain a slice image block includes: Acquiring the current full-slice image and performing downsampling processing on the current full-slice image to obtain a low-resolution overview image; segmenting the low-resolution overview image using an automatic threshold segmentation algorithm to obtain an overview image including a tissue segmentation mask; performing channel enhancement on the overview image including the tissue segmentation mask according to the staining condition of the current full-slice image, and performing glandular region marking on the channel-enhanced overview image using a preset morphological algorithm to obtain an overview image including the glandular region segmentation mask; The overview image including the glandular region segmentation mask is aligned with the current full slice image, and the current full slice image is cut based on the glandular region segmentation mask to obtain slice image blocks.

3. The method according to claim 2, wherein The step of using a preset morphological algorithm to mark the glandular region on the channel-enhanced overview image to obtain the overview image containing the glandular region segmentation mask comprises: Selecting a preset structural element, performing a top-hat transformation operation on the channel-enhanced overview image to obtain a glandular cavity-enhanced overview image; Segmenting the enhanced overview image of the glandular cavity using the automatic threshold segmentation algorithm to determine candidate regions of the glandular cavity; Using the candidate region of the glandular cavity as a marker, a watershed algorithm is used to segment the enhanced overview image of the glandular cavity to obtain an overview image with glandular region markers; The segmentation result of the overview image with glandular region markers is optimized by using a morphological post-processing method to obtain an overview image including a glandular region segmentation mask.

4. The method according to claim 1, wherein The step of inputting the slice image block into a preset dual-branch classification model to obtain a gland classification result includes: Extracting local features from the slice image block through the first branch to obtain local features, wherein the local features include gland edge features, cell texture features, and gland duct structure features; Performing global feature extraction on the slice image block through the second branch to obtain global features, wherein the global features include: horizontal gland distribution features and vertical gland topology features; Determining a current spatial weight through the fusion module, and performing weighted fusion on the local features and the global features based on the current spatial weight to obtain a fused feature; Mapping the fusion features through the decoder to determine the gland category probability corresponding to the slice image block; Masks corresponding to different gland categories are generated in the current full slice image according to the gland category probabilities corresponding to the slice image blocks.

5. The method according to claim 4, wherein The step of determining the current spatial weight and performing weighted fusion on the local features and the global features based on the current spatial weight to obtain a fused feature includes: Splicing the local features and the global features to generate a joint feature vector; Performing a nonlinear transformation on the joint feature vector using a unit convolutional layer to obtain a scalar weight value, and normalizing the scalar weight value using a Sigmoid activation function to obtain a current spatial weight, where the current spatial weight includes a local feature weight and a global feature weight; The local features and the global features are weightedly fused according to the local feature weights and the global feature weights to obtain fused features.

6. The method according to claim 1, wherein The step of comparing the current intestinal metaplasia gland ratio value with a preset Sydney system grading threshold to determine the Sydney system grading result corresponding to the current full-slice image includes: Based on the Sydney system grading, several Sydney grades are determined, and the corresponding intestinal metaplasia area percentage thresholds are determined according to the Sydney grades; Comparing the current intestinal metaplasia gland ratio value with each intestinal metaplasia area ratio threshold value to obtain a comparison result; The Sydney system level corresponding to the current full-slice image is determined according to the comparison result.

7. The method according to claim 6, wherein The step of determining the Sydney system level corresponding to the current full-slice image according to the comparison result includes: Determining a threshold difference between the current intestinal metaplasia gland ratio value and each of the intestinal metaplasia area ratio thresholds; When the threshold difference falls within a preset critical value range, determining that the current full-slice image is a critical sample, and triggering a manual review prompt based on the critical sample to enable a professional physician to determine the Sydney system grade corresponding to the current full-slice image; When the threshold difference does not meet the preset critical value range, the Sydney system level corresponding to the current full-slice image is determined according to the comparison result.

8. A gastric mucosal intestinal metaplasia grading device, characterized in that: The device comprises: An image preprocessing module is used to obtain a current full-slice image and preprocess the current full-slice image to obtain a slice image block; a model classification module, configured to input the slice image block into a preset two-branch classification model to obtain a gland classification result, wherein the preset two-branch classification model includes: a first branch constructed based on a residual network, a second branch constructed based on a Transformer network, a fusion module, and a decoder, wherein the first branch is configured to perform local feature extraction on the slice image block, the second branch is configured to perform global feature extraction on the slice image block, the fusion module is configured to perform feature fusion, and the decoder is configured to output a gland category probability based on the fused features; a hierarchical mapping module, configured to determine corresponding pixel values ​​of the intestinal metaplasia gland region and the normal gland region in the current full-slice image based on the gland classification result, and determine a current intestinal metaplasia gland ratio value according to the pixel values; The grading mapping module is further configured to compare the current intestinal metaplasia gland ratio value with a preset Sydney system grading threshold to determine a Sydney system grading result corresponding to the current full-slice image.

9. A gastric mucosal intestinal metaplasia grading device, characterized in that: The gastric mucosal intestinal metaplasia grading device includes: a memory, a processor, and a gastric mucosal intestinal metaplasia grading program stored in the memory and executable on the processor. When the gastric mucosal intestinal metaplasia grading program is executed by the processor, the steps of the gastric mucosal intestinal metaplasia grading method according to any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: The storage medium stores a gastric mucosal intestinal metaplasia grading program, which, when executed by a processor, implements the steps of the gastric mucosal intestinal metaplasia grading method according to any one of claims 1 to 7.

Images