A gastric cancer lesion region segmentation method based on a hierarchical attention mechanism

Abstract

The application discloses a gastric cancer lesion region segmentation method based on a hierarchical attention mechanism. The method first acquires a stomach pathological image; the obtained stomach pathological image is preprocessed to obtain a pathological image block; the pathological image block is input into a hierarchical attention segmentation model encoder for feature extraction to obtain a feature map; the feature map is input into a hierarchical attention segmentation model decoder to obtain a preliminary segmentation result of the pathological image block; and the preliminary segmentation result is post-processed to obtain the gastric cancer lesion region segmentation result. The hierarchical attention segmentation model of the gastric cancer lesion region segmentation method models global semantic information through deep features, models local texture information through shallow features, allocates attention weight coefficients according to feature importance, and effectively fuses multi-layer features of an image. The application can realize full-automatic processing of a stomach pathological image, realize accurate segmentation of a gastric cancer lesion region, and provide more convenient and effective diagnosis guidance for doctors.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer vision, and in particular to a method for segmenting gastric cancer lesion regions based on a hierarchical attention mechanism. Background Technology

[0002] Stomach cancer is one of the most common malignant tumors worldwide. Clinical diagnosis requires the collection of gastric tissue slides, and only after diagnosis by a professional doctor can it be determined whether cancerous lesions have occurred in the stomach.

[0003] Manual segmentation methods are time-consuming, labor-intensive, and require a high level of expertise. Threshold-based segmentation is easy to implement and can effectively segment objects when their grayscale values ​​differ significantly from other features. Region growing and splitting / merging methods are computationally simple but sensitive to noise and difficult to apply to complex images. Edge detection methods rely on the often drastic changes in grayscale values ​​at region edges as a criterion; surface fitting methods treat grayscale as height, fitting a surface to the data within a window and determining edge points based on the surface. These methods require customized design for different scenarios and lack generalization ability. This invention proposes a fully automated gastric cancer lesion region segmentation method based on a hierarchical attention mechanism, which can achieve accurate segmentation of lesion regions, providing doctors with more convenient and effective diagnostic guidance. Summary of the Invention

[0004] In view of this, the purpose of this invention is to propose a gastric cancer lesion region segmentation method based on a hierarchical attention mechanism. The main idea is to use multi-scale feature fusion to extract deep image features to model global semantic information of the image, perform overall image analysis, accurately locate the lesion region, and roughly determine the pixels covered by the lesion region; extract shallow image features to model local texture information, and analyze each pixel covered by the lesion region; and adaptively assign different weight coefficients to deep and shallow features through an attention mechanism, so as to give higher attention to features with higher importance and assign higher weight coefficients, effectively combining multi-layer features to achieve accurate segmentation of the lesion region.

[0005] This invention is implemented using the following scheme:

[0006] A method for segmenting gastric cancer lesion regions based on a hierarchical attention mechanism includes the following steps:

[0007] Step S1: Obtain gastric pathological images;

[0008] Step S2: Preprocess and segment the obtained gastric pathological images to obtain several pathological image blocks;

[0009] Step S3: Using a hierarchical attention segmentation model, perform data augmentation, feature extraction, feature fusion, and preliminary segmentation on the pathological image blocks to obtain the preliminary segmentation results of the gastric cancer lesion region corresponding to each pathological image block; Step S4: Post-process the preliminary segmentation results to obtain the final segmentation results of the gastric cancer lesion region.

[0010] Furthermore, step S2 specifically includes the following steps:

[0011] Step S21: Obtain the contour information of the gastric pathological image;

[0012] Step S22: Locate the gastric biopsy tissue area based on the contour information of the image;

[0013] Step S23: Cropping is performed based on the gastric biopsy tissue area to obtain a rectangular region of interest in the stomach;

[0014] Step S24: Overlap and crop the region of interest in the stomach according to a preset step size to obtain several pathological image blocks of preset size.

[0015] Furthermore, the hierarchical attention segmentation model described in step S3 includes:

[0016] The image preprocessing module is used to perform data augmentation on pathological image blocks.

[0017] The feature encoder module is used to extract local and global features of the enhanced pathological image block to obtain a feature map of the pathological image block;

[0018] The feature decoder module is used to upsample the feature map of the pathological image block to obtain a preliminary segmentation result of the same size as the pathological image block.

[0019] Furthermore, the feature encoder module includes:

[0020] The shallow feature extraction module is used to extract local information of enhanced pathological image patches to obtain shallow feature maps at different scales.

[0021] The deep feature extraction module consists of several cascaded residual attention modules. Each residual attention module is composed of m residual connections of Swin Transformer structures. It is used to extract global information of the enhanced pathological image blocks and obtain deep feature maps at different scales.

[0022] The feature fusion module is used to adaptively assign different weight coefficients to deep and shallow features through an attention mechanism, and to fuse shallow feature maps and deep feature maps of different scales to obtain fused pathological image block feature maps.

[0023] Furthermore, step S4 specifically includes the following steps:

[0024] Step S41: Overlap and stitch together the preliminary segmentation results of all pathological image blocks;

[0025] Step S42: Perform corrosion and expansion treatment to obtain the final segmentation result of the gastric cancer lesion area.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] This invention discloses a method for segmenting gastric cancer lesion regions based on a hierarchical attention mechanism. The method first acquires a gastric pathological image; preprocesses the acquired gastric pathological image to obtain pathological image blocks; inputs the pathological image blocks into a hierarchical attention segmentation model encoder for feature extraction to obtain feature maps; inputs the feature maps into a hierarchical attention segmentation model decoder to obtain preliminary segmentation results for the pathological image blocks; and performs post-processing on the preliminary segmentation results to obtain the segmentation result of the gastric cancer lesion region. The hierarchical attention segmentation model of this gastric cancer lesion region segmentation method models global semantic information through deep features and local texture information through shallow features, assigning attention weight coefficients according to the importance of features, effectively fusing multi-layer features of the image. This invention can achieve fully automatic processing of gastric pathological images and achieve accurate segmentation of gastric cancer lesion regions. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the gastric cancer lesion region segmentation method according to an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of the hierarchical attention segmentation model for gastric cancer lesion region segmentation according to an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram of the encoder structure of the hierarchical attention segmentation model for gastric cancer lesion region segmentation according to an embodiment of the present invention.

[0031] Figure 4 The diagram shows the segmentation process according to an embodiment of the present invention, wherein (a) is a schematic diagram of a pathological image block, (b) is a schematic diagram of the segmentation result, and (c) is a schematic diagram of the segmentation result visualized on the pathological image block. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] See Figure 1 This is a flowchart illustrating an embodiment of a gastric cancer lesion region segmentation method based on a hierarchical attention mechanism provided by the present invention, specifically including the following steps:

[0034] Step S101: Obtain gastric pathological images.

[0035] Among them, gastric pathological images are an important basis for doctors to clinically evaluate gastric cancer. The process involves obtaining a full-scan image of a gastric pathological section containing the lesion area; imaging the section to simulate the effect observed by the human eye; and digitizing the section to obtain the aforementioned gastric pathological image.

[0036] Step S102: Preprocess the obtained gastric pathological images to obtain pathological image blocks.

[0037] Gastric pathological images typically have high resolution and often contain large, irrelevant regions that offer no diagnostic value for cancer. Information from these regions serves as noise in lesion segmentation, easily interfering with the segmentation results and incurring additional computational costs. By eliminating irrelevant regions and segmenting only the regions of interest in the pathological image, interference can be effectively eliminated, improving both segmentation speed and accuracy.

[0038] Since gastric cancer lesions mainly occur in the gastric biopsy tissue area in gastric pathology images, image contour information is obtained from the gastric pathology images, and the gastric biopsy tissue area is located based on the image contour information; the gastric biopsy tissue area is then cropped to obtain a rectangular region of interest in the stomach.

[0039] The images of the regions of interest (ROIs) in the stomach are often very high-resolution, and directly segmenting gastric cancer lesions from these high-resolution images requires significant video memory. By slicing the images of the ROIs and then performing segmentation on the resulting low-resolution images, the video memory requirements of the segmentation method can be reduced, facilitating deployment on a wider range of hardware devices.

[0040] When slicing the image of the region of interest in the stomach, this embodiment uses an overlapping slicing method with a preset step size to cut the region of interest in the stomach into pathological image blocks of a preset size. This can avoid the damage to the cutting edge information and the loss of structural information in the area adjacent to the cutting edge caused by conventional slicing methods.

[0041] Step S103: Process the pathological image block using a hierarchical attention segmentation model to obtain the preliminary segmentation result corresponding to the pathological image block.

[0042] In one embodiment, the hierarchical attention segmentation model structure diagram is as follows: Figure 2 As shown, it includes:

[0043] The image preprocessing module is used to perform data augmentation on pathological image blocks. The data augmentation includes: performing geometric transformations on the image to be processed, such as flipping, rotating, scaling, cropping, and dithering; and performing pixel transformations on the image to be processed, such as adding Gaussian noise, applying Gaussian blur, adjusting white balance, and adjusting contrast.

[0044] The feature encoder module is used to extract features from pathological image blocks. The extracted features include the image's color distribution, contour information, edge gradient information, texture information, color histogram, moment features, entropy, and higher-order semantic information. The feature encoder module enables feature extraction to focus on important information, reduce background interference, and embed the image to be processed into a fixed-dimensional vector, which represents the latent features of the image to be processed.

[0045] The feature decoder module is used to reconstruct images from the features of pathological image blocks. Specifically, the image reconstruction involves using a four-layer deconvolutional network to gradually restore the details and corresponding spatial dimensions of the target based on the multi-scale features extracted by the encoder, and outputting a segmentation result of the same size as the image to be processed while minimizing information loss.

[0046] In this step, the pathological image block is first input into the image preprocessing module for data augmentation; then the augmented pathological image block is input into the encoder module to extract features from the pathological image block, obtain features at different scales, and assign attention weight coefficients according to the importance of the features to obtain the fused feature map of the pathological image block; finally, the fused feature map is decoded using the feature decoder module to output the preliminary segmentation result of the pathological image block.

[0047] In one specific embodiment of the present invention, the feature encoder structure diagram of the hierarchical attention segmentation model is as follows: Figure 3 As shown, it includes:

[0048] The shallow feature extraction module consists of two layers of convolutional neural networks. It is used to extract local information from the data-enhanced pathological image blocks to obtain shallow feature maps at different scales. Due to the small receptive field, the shallow feature maps usually contain some image corner feature information or feature information of a certain part of the image. They have strong discriminative power, but the number of features is relatively large.

[0049] The deep feature extraction module consists of four cascaded residual attention modules. Each residual attention module is composed of six Swin Transformer residual connections. It is used to extract global information from the shallow feature map output by the last layer of the shallow feature extraction module to obtain deep feature maps at different scales. Due to the large receptive field, the deep feature map usually contains the overall semantic feature information of the image, but its discriminative power is weak.

[0050] The feature fusion module is used to fuse shallow feature maps and deep feature maps of different scales in pathological image blocks. Through an attention mechanism, different weight coefficients are adaptively assigned to deep and shallow features, so that features with high importance receive higher attention and are assigned higher weight coefficients, while features with lower importance are appropriately ignored and assigned lower weight coefficients. This effective fusion of shallow and deep features at multiple scales can provide effective information for the accurate localization of gastric cancer lesion areas and for the accurate discrimination of pixels covered by the lesion areas.

[0051] In the feature encoder of the hierarchical attention segmentation model in this embodiment, the data-enhanced pathological image block is first input into the shallow feature extraction module, and then passes through two layers of convolutional neural networks to obtain the first shallow feature map and the second shallow feature map. Specifically, the input of the first layer of convolutional neural network is the data-enhanced pathological image block, and the output is the first shallow feature map; the input of the second layer of convolutional neural network is the first shallow feature map, and the output is the second shallow feature map.

[0052] The second shallow feature map output by the shallow feature extraction module is input into the deep feature extraction module, and then passes through four layers of residual attention modules to obtain the first to fourth deep feature maps. Specifically, the input of the first layer of residual attention module is the second shallow feature map, and the output is the first deep feature map. The input of the next layer of residual attention module is the output of the previous layer of residual attention module, and each layer of residual attention module outputs a deep feature map.

[0053] Finally, the feature fusion module fuses the two shallow feature maps of different scales obtained from the shallow feature extraction module and the four deep feature maps of different scales obtained from the deep feature extraction module to obtain a fused feature map, which is used as the input to the subsequent feature decoder module. Figure 4 The diagram shown is a segmentation process diagram obtained in this embodiment, wherein (a) is a schematic diagram of a certain pathological image block, (b) is a schematic diagram of the segmentation result, and (c) is a schematic diagram of the segmentation result visualized on the pathological image block.

[0054] Step S104: Post-process the preliminary segmentation results to obtain the segmentation results of the gastric cancer lesion region.

[0055] In this embodiment, the preliminary segmentation result obtained is the segmentation result corresponding to the pathological image block obtained by overlapping and tiling. It is necessary to overlap and stitch the segmentation results. In this embodiment, morphological erosion and dilation operation is used to fill in the discontinuous areas of possible local segmentation results, restore the resolution of the original pathological image, and obtain the final segmentation result of the gastric cancer lesion region.

[0056] The above examples are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments and many variations are possible. All variations that can be directly derived or conceived by those skilled in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for segmenting gastric cancer lesion regions based on hierarchical attention mechanism, comprising the following steps: Step S1: Obtain gastric pathological images; Step S2: Preprocess and segment the obtained gastric pathological images to obtain several pathological image blocks; specifically including the following steps: Step S21: Obtain the contour information of the gastric pathological image; Step S22: Locate the gastric biopsy tissue area based on the contour information of the image; Step S23: Cropping is performed based on the gastric biopsy tissue area to obtain a rectangular region of interest in the stomach; Step S24: Overlap and crop the region of interest in the stomach according to a preset step size to obtain several pathological image blocks of preset size; Step S3: Using a hierarchical attention segmentation model, data augmentation, feature extraction, feature fusion, and preliminary segmentation are performed on the pathological image blocks to obtain preliminary segmentation results of the gastric cancer lesion region corresponding to each pathological image block; the hierarchical attention segmentation model includes: An image preprocessing module is used to perform data augmentation on pathological image blocks; The feature encoder module is used to extract local and global features of the enhanced pathological image block to obtain a feature map of the pathological image block. The feature encoder module includes: a shallow feature extraction module, which is used to extract local information of the enhanced pathological image block to obtain shallow feature maps of different scales; and a deep feature extraction module, which is composed of several cascaded residual attention modules, each of which is composed of m residual connections of Swin Transformer structures, used to extract global information of the enhanced pathological image block to obtain deep feature maps of different scales. The feature fusion module is used to adaptively assign different weight coefficients to deep and shallow features through an attention mechanism, and to fuse shallow feature maps and deep feature maps of different scales to obtain fused pathological image block feature maps. The feature decoder module is used to upsample the feature map of the pathological image block to obtain a preliminary segmentation result of the same size as the pathological image block; Step S4: Post-process the preliminary segmentation results to obtain the final segmentation results of the gastric cancer lesion area; specifically including the following steps: Step S41: Overlap and stitch together the preliminary segmentation results of all pathological image blocks; Step S42: Perform corrosion and expansion treatment to obtain the final segmentation result of the gastric cancer lesion area.

Images