Tumor pathological image segmentation method and system based on state space model

Abstract

The invention discloses a tumor pathological image segmentation method and system based on a state space model, the system comprises a state space feature encoder, a multi-scale feature decoder, a side output module and a splicing fusion module, the state space feature encoder comprises four stages of QSSBlock feature encoders, the multi-scale feature decoder comprises a multi-scale feature fusion module and a multi-scale convolution attention module. The multi-scale feature fusion module comprises an upper convolution module and a grouping attention module. According to the method, the global features in the tumor pathological image can be effectively captured, the long-range correlation between the instances can be established, the problem that the instances are mutually independent is solved, segmentation mapping is greatly simplified, and the weak supervision method is more interpretable. By introducing a deep supervision multi-stage side output mode, the hierarchical information of the multi-scale feature map is effectively utilized, high-throughput tumor pathological image segmentation can be quickly and accurately realized, and the method has wide application prospects and important practical values in the field of medical image processing.

Description

Technical Field

[0001] The present invention relates to a tumor pathological image segmentation method and system based on a state space model, belonging to the technical field of image processing. Background Art

[0002] In recent years, tumor pathological sections, as a high-throughput and high-resolution imaging technology, have rapidly emerged in the field of cancer research and have become an indispensable key tool for cancer research, diagnosis, and treatment. Whole Slide Imaging (WSI) technology can convert traditional glass slides into digital images, which contain detailed information of the entire slide and can be infinitely magnified on a computer without loss of resolution, thus allowing pathologists and researchers to conduct more in-depth analysis. Tumor pathological sections play a crucial role in revealing tumor heterogeneity, assessing tumor grade and stage, monitoring treatment response, etc. WSI technology shows great potential in multiple fields, including the study of tumorigenesis mechanisms, personalized medicine, pathology education, and artificial intelligence-assisted diagnosis.

[0003] However, although WSI technology plays an important role in the analysis of tumor pathological sections, the segmentation of histopathological images remains a challenging task. Traditional fully supervised pixel-level segmentation methods require a large number of accurate pixel-level labels for training, and the acquisition of these labels usually consumes a large amount of time and effort of professionals such as pathologists, limiting the popularization of this method in practical applications.

[0004] To alleviate the problem of time-consuming and laborious medical image annotation, weakly supervised segmentation algorithms have emerged. As a subset of weakly supervised learning, multi-instance learning (MIL) has shown good performance in medical image segmentation. The MIL method reduces the dependence on pixel-level labels by using image-level labels for training, thus reducing the annotation cost. However, the MIL method also has certain limitations, such as the lack of inter-instance correlation information in multi-instance learning, which limits the further improvement of segmentation performance.

[0005] To overcome the limitations of the MIL method, researchers have begun to explore other weakly supervised learning methods. Among them, the state space model, as an innovative technology, performs well in capturing global or long-range dependencies and can effectively alleviate the problems brought by the quadratic complexity of network models such as transformers. Therefore, applying the state space model to the pixel-level segmentation of tumor pathological images has become a challenging and forward-looking research direction. By combining the state space model with multi-instance learning, it is expected to achieve efficient and accurate segmentation of tumor pathological images and provide more powerful support for the precision medicine of cancer. Summary of the Invention

[0006] To solve the above problems, the present invention provides a tumor pathological image segmentation method and system based on a state space model. The present invention combines the state space model with multi-instance learning to capture context information from global image dependencies, obtain global or long-range dependencies, and overcome the drawback that instances are independent of each other in multi-instance learning.

[0007] In a first aspect, the present invention provides a tumor pathological image segmentation method, and the method includes:

[0008] Step 1: Obtain tumor pathological image slices and image-level labels;

[0009] Step 2: Process the tumor pathological image using a tumor pathological image segmentation model to complete pixel-level segmentation of the tumor pathological image;

[0010] The tumor pathological image segmentation model includes: a state space feature encoder, a multi-scale feature decoder, a side output module, and a splicing fusion module;

[0011] The state space feature encoder includes QSSBlock feature encoders in four stages, and the QSSBlock feature encoder is used to capture global features in the tumor pathological image slices and establish long-range correlations between instances;

[0012] The multi-scale feature decoder includes a multi-scale feature fusion module and a multi-scale convolutional attention module; the multi-scale feature fusion module includes an up-convolution module and a grouped attention module; the up-convolution module is used to upsample the feature map of the current stage to match the size and resolution of the feature map of the next skip connection; the grouped attention module is used to enhance the low-level semantic features transmitted through the skip connection and suppress irrelevant features using the high-level semantic features obtained by the up-convolution module; the multi-scale convolutional attention module is used to aggregate local information and capture multi-scale context information;

[0013] The side output module is used to generate a segmentation output from the feature maps of four stages of the state space feature encoder;

[0014] The splicing fusion module is used to combine low-level boundary information with high-level semantic information.

[0015] In an implementation manner of the present invention, the training process of the tumor pathological image segmentation model includes:

[0016] Combined with whole-slide tumor pathological image slices, label whether the input image slices contain tumor tissue;

[0017] Construct a training sample set and a test sample set;

[0018] Train a tumor pathological image segmentation model. The training loss function includes: stage loss, fusion loss, and final objective loss. Among them, the stage loss is used to enhance the supervision ability by using pixel-level labels, the fusion loss is used to optimize the high-level semantic information and edge detail information of the extracted features, and the final objective loss is used to guide the network segmentation learning;

[0019] The calculation formula of the stage loss is:

[0020]

[0021] where t represents the number of stages, and I is the indicator function;

[0022] The calculation formula of the fusion loss is:

[0023]

[0024] The calculation formula of the final objective loss is:

[0025]

[0026] In an implementation manner of the present invention, the QSSBlock feature encoder includes a Mamba residual block and a spatial gating block;

[0027] The Mamba residual block sequentially includes: two convolutional layers, a batch normalization layer, a depthwise separable convolutional layer, a batch normalization layer, a convolutional layer, a residual connection, a layer normalization layer, and an SS2D block; The calculation formula is expressed as:

[0028] Y1 = BN(Conv3(Conv1(x1)))

[0029] Y2 = Conv3(BN(DW(Y1))) + Y1

[0030] Y = SS2D(LN(Y2))

[0031] where x1 represents the feature vector input to the QSSBlock feature encoder, Conv1 represents the 1x1 convolutional layer in the feature encoder, Conv3 represents the 3x3 convolutional layer, BN is the batch normalization layer, DW represents the depth convolutional layer, LN represents the layer normalization layer, and SS2D represents the SS2D block;

[0032] The spatial gating block sequentially includes: a layer normalization layer, two depthwise separable convolutional layers, a sigmoid activation function, a matrix multiplication unit, a pointwise convolutional layer, and a residual connection; The calculation formula is expressed as:

[0033] Z2 = PW(σDW(LN(Z1)) * DW(LN(Z1)))

[0034] Z = Z1 + Z2

[0035] Where Z1 represents the feature vector of the input spatial gating block, DW represents the depth convolution layer, LN represents the batch normalization layer, and σ represents matrix multiplication.

[0036] In one embodiment of the present invention, the transposed convolution module includes: a depthwise separable convolution layer, a batch normalization layer, a ReLU activation function, and a convolution layer.

[0037] In one embodiment of the present invention, the grouped attention module includes: two grouped convolution layers, a batch normalization layer, three ReLU activation functions, two residual connections, a convolution layer, a sigmoid activation function, and a matrix multiplication; the calculation formula is expressed as:

[0038] Q(g,y) = R(BN(GC g (g)) + R(BN(GC y (y))

[0039] F(g,y) = Sig(Conv1(R(Q(g,y)))) * y + y

[0040] Where the variables g and y are the feature vectors input to the grouped attention module, GCg and GCy are used to process the input variables g and y, R represents the ReLU activation function, Q represents element-wise addition, Conv1 represents the 1x1 convolution layer, and Sig represents the Sigmoid function.

[0041] In one embodiment of the present invention, the multi-scale convolutional attention module includes: four depthwise separable convolution layers, a residual connection, a matrix multiplication unit, and a convolution layer; the calculation formula is expressed as:

[0042] Y1 = DW3(DW1(x2)) + DW5(DW1(x2)) + DW7(DW1(x2)) + DW1(x2)

[0043] Y = Conv(Y1) * x2

[0044] Where x2 represents the feature vector input to the multi-scale convolutional attention module, and DW represents different depth convolution layers;

[0045] The side output module includes: a convolution layer that produces an output with the number of channels equal to the number of channels in the target dataset; the calculation formula is expressed as:

[0046] S(x) = Sigmoid(Conv1(x3))

[0047] Among them, x3 represents the feature vector of the input-side output module, Conv1 represents a convolutional layer with a 1x1 convolutional kernel, and Sigmoid represents the Sigmoid activation function;

[0048] The splicing and fusion module performs weighted averaging on the side output images;

[0049] After obtaining the side output, each instance in the package is used to predict the classification of the image; represents the probability of the pixel in the nth image, where (i, j) represents the pixel P ij at the position in X n using the softmax function as the activation function, and its calculation formula is:

[0050]

[0051] In an embodiment of the present invention, the step 1 further includes:

[0052] Using Python opencv to slice the obtained whole-slide tumor pathological image to obtain slices of two-dimensional tissue images, labeling the slices containing tumor tissue as 1, and labeling the images without tumor tissue as 0;

[0053] After the sliced image is subjected to feature extraction by the tumor pathological image segmentation model, a probability map is obtained through the side output module, and the multi-scale feature fusion module uses the multi-scale side outputs of each stage to predict the final segmentation map.

[0054] In a second aspect, the present invention provides a tumor pathological image segmentation system, and the system includes:

[0055] A tumor pathological image acquisition module for acquiring labeled tumor pathological images;

[0056] A tumor pathological image segmentation module for using the tumor pathological image segmentation model to process the tumor pathological image to complete pixel-level segmentation of the tumor pathological image;

[0057] The tumor pathological image segmentation model includes: a state space feature encoder, a multi-scale feature decoder, a side output module, and a splicing and fusion module;

[0058] The state space feature encoder includes QSSBlock feature encoders in four stages, and the QSSBlock feature encoder is used to capture the global features in the tumor pathological image and establish long-range correlations between instances to solve the shortcoming that instances are independent of each other;

[0059] The multi-scale feature decoder includes a multi-scale feature fusion module and a multi-scale convolutional attention module; the multi-scale feature fusion module includes an up-convolution module and a grouped attention module; the up-convolution module is used to upsample the feature map of the current stage to match the size and resolution of the feature map of the next skip connection; the grouped attention module is used to enhance the low-level semantic features transmitted through the skip connection and suppress irrelevant features by using the high-level semantic features obtained by the up-convolution module; the multi-scale convolutional attention module is used to aggregate local information and capture multi-scale context information;

[0060] The side output module is used to generate a segmentation output from the feature maps of the four stages of the multi-scale feature decoder, restore the feature map size through an upsampling operation, and use a 1x1 convolutional kernel to achieve channel unification;

[0061] The splicing and fusion module is used to combine low-level boundary information with high-level semantic information and perform weighted averaging on the side output images.

[0062] In a third aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the tumor pathological image segmentation method described above are implemented.

[0063] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the tumor pathological image segmentation method described above are implemented.

[0064] The beneficial effects of the present invention are:

[0065] A tumor pathological image segmentation method and system based on a state space model proposed by the present invention combines the state space model with multi-instance learning to capture context information from the full-image dependence, obtain global or long-range dependencies, and overcome the disadvantage that instances are independent of each other in multi-instance learning. The excellent global or long-range dependence capture ability of the state space model greatly simplifies the segmentation mapping and makes the weakly supervised method more interpretable. The multi-scale feature fusion module, the multi-scale convolutional attention module, and the method of deep-supervised multi-stage side output effectively utilize the hierarchical information of the multi-scale feature maps, solve the problem of low accuracy when the weakly supervised method performs pixel-level segmentation on tumor pathological images, and further improve the robustness and generalization ability of the tumor pathological image segmentation model, and can quickly and accurately achieve high-throughput tumor pathological image segmentation. The present invention not only provides a new technical means for the segmentation of histopathological images, but also lays a solid foundation for the wide application of future medical images. Therefore, the present invention has broad application prospects and important practical value in the field of medical image processing. Description of the Drawings

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings described below are only the embodiments of the present invention. For those of ordinary skill in the art, without creative work, other accompanying drawings can also be obtained according to the provided accompanying drawings.

[0067] Figure 1 It is a flowchart of a tumor pathological image segmentation method based on a state space model provided by the present invention.

[0068] Figure 2 It is a schematic structural diagram of a tumor pathological image segmentation model provided by the present invention.

[0069] Figure 3 It is a schematic structural diagram of a state space feature encoder provided by the present invention.

[0070] Figure 4 It is a schematic structural diagram of a multi-scale feature fusion module provided by the present invention.

[0071] Figure 5 It is a schematic structural diagram of a multi-scale convolutional attention module provided by the present invention.

[0072] Figure 6 It is a schematic structural diagram of a side output module provided by the present invention. Specific Embodiments

[0073] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative work fall within the protection scope of the present invention.

[0074] Embodiment 1:

[0075] This embodiment provides a tumor pathological image segmentation method, and the method includes:

[0076] Step 1: Obtain tumor pathological image slices and image-level labels;

[0077] Step 2: Process the tumor pathological image by using the tumor pathological image segmentation model to complete the pixel-level segmentation of the tumor pathological image;

[0078] The tumor pathological image segmentation model includes: a state space feature encoder, a multi-scale feature decoder, a side output module, and a splicing and fusion module;

[0079] The state space feature encoder includes a QSSBlock feature encoder in four stages, and the QSSBlock feature encoder is used to capture the global features in the tumor pathological image slice and establish long-range correlations between instances to address the drawback that instances are independent of each other;

[0080] The multi-scale feature decoder includes a multi-scale feature fusion module and a multi-scale convolutional attention module; the multi-scale feature fusion module includes an up-convolution module and a grouped attention module; the up-convolution module is used to upsample the feature map of the current stage to match the size and resolution of the feature map of the next skip connection; the grouped attention module is used to enhance the low-level semantic features transmitted through the skip connection and suppress irrelevant features using the high-level semantic features obtained by the up-convolution module; the multi-scale convolutional attention module is used to aggregate local information and capture multi-scale context information;

[0081] The side output module is used to generate a segmentation output from the feature maps of the four stages of the state space feature encoder;

[0082] The splicing fusion module is used to combine low-level boundary information with high-level semantic information.

[0083] Embodiment 2:

[0084] Refer to Figures 1-6 This embodiment provides a method for segmenting tumor pathological images based on a state space model. The method includes:

[0085] S1: Obtain a tumor pathological image dataset; in this dataset, 250 tumor images and 500 non-tumor images are used for training; 80 tumor images and 80 non-tumor images are used for testing; the images are all adjusted to 256×256 pixels; the original images are read using python opencv and converted into three-dimensional pixel matrices; the image data containing tumor tissue is labeled as 1, and the image data without tumor tissue is labeled as 0;

[0086] S2: Use a multi-instance learning weakly supervised tumor pathological image segmentation model based on a state space model to train the labeled image data to obtain a trained tumor pathological image segmentation model;

[0087] Among them, the tumor pathological image segmentation model includes: a state space feature encoder, a multi-scale feature decoder, a side output module, and a splicing fusion module;

[0088] Specifically, referring to Figure 2 the state space feature encoder includes a QSSBlock feature encoder in four stages. Referring to Figure 3, the QSSBlock feature encoder is used to capture the global features in the tumor pathological image and establish long-range correlations between instances to address the shortcoming of instances being independent of each other;

[0089] Specifically, referring to Figure 3 , the QSSBlock feature encoder consists of a Mamba residual block and a spatial gating block.

[0090] The Mamba residual block sequentially includes: two convolutional layers, one batch normalization layer, one depthwise separable convolutional layer, one batch normalization layer, one convolutional layer, one residual connection, one layer normalization layer, one SS2D block; the calculation formula is expressed as:

[0091] Y1 = BN(Conv3(Conv1(x1)))

[0092] Y2 = Conv3(BN(DW(Y1))) + Y1

[0093] Y = SS2D(LN(Y2))

[0094] Where x1 represents the feature vector input to the QSSBlock feature encoder, Conv1 represents the 1x1 convolutional layer in the feature encoder, Conv3 represents the 3x3 convolutional layer, BN is the batch normalization layer, DW represents the depth convolutional layer, LN represents the layer normalization layer, and SS2D represents the SS2D block.

[0095] This Mamba residual block can effectively capture remote instance dependencies and enhance the capture of local features, effectively extract the local spatial information of the input feature map, while reducing the computational cost and the number of parameters.

[0096] The spatial gating block sequentially includes: one layer normalization layer, two depthwise separable convolutional layers, one sigmoid activation function, one matrix multiplication unit, one pointwise convolutional layer, one residual connection; the calculation formula is expressed as:

[0097] Z2 = PW(σDW(LN(Z1)) * DW(LN(Z1)))

[0098] Z = Z1 + Z2

[0099] Where Z1 represents the feature vector input to the spatial gating block, DW represents the depth convolutional layer, LN represents the layer normalization layer, and σ represents matrix multiplication.

[0100] This spatial gating block can capture more global features, while only bringing a slight increase in computational cost, adopts the method of residual splicing to perform more efficient backflow of gradients, and reduces the computational cost by retaining and utilizing the spatial structure information of the image, significantly improving the performance.

[0101] Further, referring to Figure 2 , the multi-scale feature decoder includes a multi-scale feature fusion module and a multi-scale convolutional attention module. The multi-scale feature decoder is used to optimize performance and computational efficiency. By using the multi-scale feature fusion module and the multi-scale convolutional attention module, the feature map is significantly enhanced through multi-scale convolution and grouped convolutional attention, which is very effective in capturing complex spatial relationships while focusing on prominent regions.

[0102] Further, referring to Figure 4 , the multi-scale feature fusion module includes a transposed convolution module and a grouped attention module.

[0103] Further, referring to Figure 4 , the transposed convolution module includes: a depthwise separable convolution layer, a batch normalization layer, a ReLU activation function, and a convolution layer. The transposed convolution module gradually upsamples the feature map at the current stage to match the size and resolution of the feature map of the next skip connection. The transposed convolution module first upgrades the feature map using upsampling with a scale factor of 2. Then, the upgraded feature map is enhanced by applying a 3×3 depth convolution, a batch normalization layer, and a ReLU activation function. Finally, a 1×1 convolution is used to reduce the number of channels to match the next stage.

[0104] Further, referring to Figure 4 , the grouped attention module includes: two grouped convolution layers, a batch normalization layer, three ReLU activation functions, two residual connections, a convolution layer, a sigmoid activation function, and a matrix multiplication; the calculation formula is expressed as:

[0105] Q(g,y) = R(BN(GC g (g)) + R(BN(GC y (y))

[0106] F(g,y) = Sig(Conv1(R(Q(g,y)))) * y + y

[0107] where the variables g and y are the feature vectors input to the grouped attention module, GCg and Gcy are used to process the input variables g and y, R represents the ReLU activation function, Q represents element-wise addition, Conv1 represents a 1x1 convolution layer, and Sig represents the Sigmoid function.

[0108] The grouped attention module progressively combines feature maps with the attention coefficients learned by the network, thereby improving the activation of relevant features and the suppression of irrelevant features. The input variables g and y are processed by applying separate 1×1 grouped convolutions GCg and GCy, respectively. These convolutional features are then normalized using batch normalization and merged by element-wise addition. The resulting feature map is activated by ReLU. After that, 1×1 convolution and batch normalization layers are applied to obtain single-channel feature maps. The resulting single-channel feature map is then passed through the Sigmoid activation function to produce the attention coefficient. The output of this transform scales the input feature x by element-wise multiplication, and finally x is residually connected with this transform, which can reduce the impact of high-level semantic features on low-level semantic features, thereby avoiding the performance degradation of the overall model.

[0109] Further, see Figure 5 ,The multi-scale convolution attention module includes: four depth-separable convolutional layers, a residual connection, a matrix multiplication unit, and a convolutional layer; the calculation formula is expressed as:

[0110] Y1=DW3(DW1(x2))+DW5(DW1(x2))+DW7(DW1(x2))+DW1(x2)

[0111] Y=Conv(Y1)*x2

[0112] Where x2 represents the feature vector input to the multi-scale convolutional attention module, and DW represents different depth convolutional layers.

[0113] The multi-scale convolutional attention module first passes through a 5×5 deep convolution to aggregate local information, then enters a multi-branch deep convolution to capture multi-scale contextual information, and then enters a 1×1 convolution to simulate the relationship between different channels. The output of the 1×1 convolution is used as the attention weight, and the input of the multi-scale convolutional attention module is directly matrix multiplied for weighting.

[0114] Further, see Figure 6 , the side output module includes: a convolutional layer that produces an output with channels equal to the number of channels in the target dataset; the calculation formula is expressed as:

[0115] S(x)=Sigmoid(Conv1(x3))

[0116] Where x3 represents the feature vector of the input-side output module, Conv1 represents the convolution layer with a convolution kernel of 1x1, and Sigmoid represents the Sigmoid activation function.

[0117] First, it is restored to the original size through bilinear upsampling operation. Secondly, the number of channels is restored to be the same as the original image through 1×1 convolution. Finally, it is activated by sigmoid to generate the predicted probability map as the side output.

[0118] Furthermore, after obtaining the side output, the classification of the image can be predicted through each instance in the package. represents the probability of the pixel in the nth image, where (i, j) represents the pixel P ij at the position in X n using the softmax function as the activation function, and its calculation formula is:

[0119]

[0120] where the parameter r controls the sharpness.

[0121] Furthermore, the tumor pathological image segmentation model is trained, and the training loss function includes: stage loss, fusion loss, and final target loss. Among them, the stage loss is used to make full use of the pixel-level labels to enhance the supervision ability, the fusion loss is used to optimize the high-level semantic information and edge detail information of the extracted features, and the final target loss is used to guide the network segmentation learning;

[0122] The calculation formula of the stage loss is:

[0123]

[0124] where t represents the stage number and I is the indicator function; Obtained from formula (1).

[0125] The calculation formula of the fusion loss is:

[0126]

[0127] where I is the indicator function, Obtained from formula (1);

[0128] The calculation formula of the final target loss is:

[0129]

[0130] S3: Use the trained tumor pathological image segmentation model to segment the image data in the test sample set to obtain new segmentation maps. Comparing these predicted segmentation maps with the true pixel-level labels can calculate various metrics, thus completing the image segmentation.

[0131] In summary, the tumor pathological image segmentation method based on the state space model provided by the present invention can effectively capture the global features in the tumor pathological image and establish the long-range correlation between instances, solve the problem that instances are independent of each other, greatly simplify the segmentation mapping, and make the weakly supervised method more interpretable. By introducing the method of deep supervision multi-stage side output, the hierarchical information of the multi-scale feature maps is effectively utilized, and the high-throughput tumor pathological image segmentation can be realized quickly and accurately, which has broad application prospects and important practical value in the field of medical image processing.

[0132] Embodiment 3:

[0133] This embodiment provides a tumor pathological image segmentation system, which includes:

[0134] A tumor pathological image acquisition module for acquiring labeled tumor pathological images;

[0135] A tumor pathological image segmentation module for processing the tumor pathological image using a tumor pathological image segmentation model to complete pixel-level segmentation of the tumor pathological image;

[0136] The tumor pathological image segmentation model includes: a state space feature encoder, a multi-scale feature decoder, a side output module, and a splicing and fusion module;

[0137] The state space feature encoder includes QSSBlock feature encoders in four stages, and the QSSBlock feature encoder is used to capture the global features in the tumor pathological image and establish the long-range correlation between instances to solve the shortcoming that instances are independent of each other;

[0138] The multi-scale feature decoder includes a multi-scale feature fusion module and a multi-scale convolutional attention module; the multi-scale feature fusion module includes an up-convolution module and a grouped attention module; the up-convolution module is used to upsample the feature map of the current stage to match the size and resolution of the feature map of the next skip connection; the grouped attention module is used to use the high-level semantic features obtained by the up-convolution module to enhance the low-level semantic features transmitted through the skip connection and suppress irrelevant features; the multi-scale convolutional attention module is used to aggregate local information and capture multi-scale context information;

[0139] The side output module is used to generate a segmentation output from the feature maps of four stages of the multi-scale feature decoder, restore the feature map size through an upsampling operation, and use a 1x1 convolution kernel to unify the channels;

[0140] The splicing and fusion module is used to combine low-level boundary information with high-level semantic information and perform weighted averaging on the side output images.

[0141] For the specific limitations and beneficial effects of the above tumor pathological image segmentation system, reference can be made to the limitations of a tumor pathological image segmentation method in the foregoing text, which will not be elaborated here. Each of the above modules can be implemented in whole or in part by software, hardware, and their combinations. Each of the above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to each of the above modules.

[0142] Example 4:

[0143] In order to verify and illustrate the technical effects of the present invention, in this example, a traditional tumor pathological image segmentation method is selected for comparison with the segmentation method of the present invention to verify the real effects of the present invention.

[0144] This example will conduct experiments on the collected tumor pathological image dataset. In this dataset, 80 tumor images and 80 non-tumor images are used for testing.

[0145] The tumor pathological image segmentation method of the present invention is compared with the traditional scheme, and the comparison results are shown in Table 1 below:

[0146] Table 1: Comparison results between the present invention and the traditional scheme

[0147] Comparison object Traditional segmentation method Diagnostic system of the present invention Manual segmentation method Accuracy / % 80 94 90 Time / s 0.43 0.25 300-600 Scope of application Specific image Wide Wide

[0148] It can be seen from the above comparison data that the present invention is higher than the traditional scheme and expert manual segmentation in terms of accuracy and efficiency, and is faster in terms of time. The comparison results can reflect that the method of the present invention can quickly, efficiently, accurately, and automatically segment tumors.

[0149] In addition, the present invention also provides a computer device, which may include a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the processor executes the steps of a tumor pathological image segmentation method as described in any of the above embodiments.

[0150] For the working process, working details, and technical effects of the computer device provided in this example, reference can be made to the embodiments of a tumor pathological image segmentation method in the foregoing text, which will not be elaborated here.

[0151] In addition, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a tumor pathological image segmentation method according to any of the above embodiments are implemented. Wherein, the computer-readable storage medium refers to a carrier for storing data, and may include, but is not limited to, floppy disks, optical discs, hard disks, flash memories, USB flash drives, and / or memory sticks, etc. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0152] For the working process, working details, and technical effects of the computer-readable storage medium provided in this embodiment, reference may be made to the embodiments of a tumor pathological image segmentation method in the foregoing text, and details will not be repeated here.

[0153] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application may include non-volatile and / or volatile memories. Non-volatile memories may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0154] Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing embodiments, or equivalently replace some of the technical features. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A tumor pathology image segmentation method, characterized in that: The method comprises: Step 1: Obtain tumor pathology image slices and image-level labels; Step 2: Processing the tumor pathology image using a tumor pathology image segmentation model to complete pixel-level segmentation of the tumor pathology image; The tumor pathology image segmentation model includes: a state space feature encoder, a multi-scale feature decoder, a side output module and a splicing fusion module; The state space feature encoder includes a four-stage QSSBlock feature encoder, and the QSSBlock feature encoder is used to capture the global features in the tumor pathology image slice and establish the long-range correlation between each instance; The multi-scale feature decoder includes a multi-scale feature fusion module and a multi-scale convolution attention module; the multi-scale feature fusion module includes an up-convolution module and a grouping attention module; the up-convolution module is used to up-sample the feature map of the current stage to match the size and resolution of the feature map of the next skip connection; the grouping attention module is used to use the high-level semantic features obtained by the up-convolution module to enhance the low-level semantic features transmitted through the skip connection and suppress irrelevant features; the multi-scale convolution attention module is used to aggregate local information and capture multi-scale contextual information; The side output module is used to generate segmentation outputs from feature maps of four stages of the state-space feature encoder; The splicing and fusion module is used to combine low-level boundary information with high-level semantic information.

2. A tumor pathology image segmentation method according to claim 1, characterized in that: The training process of the tumor pathology image segmentation model includes: Combined with the whole-slice tumor pathology image slices, annotate whether the input image slices contain tumor tissue; Construct training sample set and test sample set; The tumor pathology image segmentation model is trained, and the training loss function includes: stage loss, fusion loss and final target loss, wherein the stage loss is used to enhance the supervision ability by using pixel-level labels, the fusion loss is used to optimize the high-level semantic information and edge detail information of the extracted features, and the final target loss is used to guide network segmentation learning; The calculation formula of the phased loss is: Among them, t represents the number of stages, and I is the indicator function; The calculation formula of the fusion loss is: The calculation formula of the final target loss is:

3. A tumor pathology image segmentation method according to claim 2, characterized in that: The QSSBlock feature encoder includes a Mamba residual block and a spatial gating block structure; The Mamba residual block includes: two convolutional layers, a batch normalization layer, a depth-separable convolutional layer, a batch normalization layer, a convolutional layer, a residual connection, a layer normalization layer, and an SS2D block in sequence; the calculation formula is expressed as: Y1=BN(Conv3(Conv1(x1))) Y2=Conv3(BN(DW(Y1)))+Y1 Y=SS2D(LN(Y2)) Where x1 represents the feature vector of the input QSSBlock feature encoder, Conv1 represents the 1x1 convolution layer in the feature encoder, Conv3 represents the 3x3 convolution layer, BD represents the batch normalization layer, DW represents the depth convolution layer, LN represents the batch normalization layer, and SS2D represents the SS2D block; The spatial gating block includes: a layer normalization layer, two depth-separable convolutional layers, a sigmoid activation function, a matrix multiplication unit, a point-by-point convolutional layer, and a residual connection. The calculation formula is expressed as: Z2=PW(σDW(LN(Z1))*DW(LN(Z1))) Z=Z1+Z2 Where Z1 represents the feature vector of the input spatial gating block, DW represents the deep convolutional layer, LN represents the batch normalization layer, and σ represents matrix multiplication.

4. A tumor pathology image segmentation method according to claim 3, characterized in that: The upper convolution module includes: a depth-separable convolution layer, a batch normalization layer, a ReLU activation function, and a convolution layer.

5. A tumor pathology image segmentation method according to claim 4, characterized in that: The grouped attention module includes: two grouped convolutional layers, a batch normalization layer, three ReLU activation functions, two residual connections, a convolutional layer, a sigmoid activation function, and a matrix multiplication; the calculation formula is expressed as: Q(g,y)=R(BN(GC g (g))+R(BN(GC y (y)) F(g,y)=Sig(Conv1(R(Q(g,y))))*y+y Among them, variables g and y are the feature vectors of the input group attention module, GCg and GCy are used to process the input variables g and y, R represents the ReLU activation function, Q represents element addition, Conv1 represents the 1x1 convolutional layer, and Sig represents the Sigmoid function.

6. A tumor pathology image segmentation method according to claim 5, characterized in that: The multi-scale convolutional attention module includes: four depth-separable convolutional layers, a residual connection, a matrix multiplication unit, and a convolutional layer; the calculation formula is expressed as: Y1=DW3(DW1(x2))+DW5(DW1(x2))+DW7(DW1(x2))+DW1(x2) Y=Conv(Y1)*x2 Where x2 represents the feature vector of the input multi-scale convolutional attention module, and DW represents different depth convolutional layers; The side output module includes: a convolution layer, which generates an output with channels equal to the number of channels in the target data set; the calculation formula is expressed as: S(x)=Sigmoid(Conv1(x3)) Where x3 represents the feature vector of the input-side output module, Conv1 represents the convolution layer with a convolution kernel of 1x1, and Sigmoid represents the Sigmoid activation function; The splicing and fusion module performs weighted averaging on the side output images; After getting the side output, predict the image classification through each instance in the bag; represents the probability of a pixel in the nth image, where (i, j) represents the pixel P ij In X n The position in the , using the softmax function as the activation function, its calculation formula is:

7. A tumor pathology image segmentation method according to claim 6, characterized in that: The step 1 also includes: Use Python opencv to slice the acquired full-slice tumor pathology images to obtain slices of two-dimensional tissue images, labeling the slices containing tumor tissue as 1 and the slices without tumor tissue as 0; After the slice image is subjected to feature extraction by the tumor pathology image segmentation model, a probability map is obtained through a side output module, and the multi-scale feature fusion module uses the multi-scale side outputs of each stage to predict the final segmentation map.

8. A tumor pathology image segmentation system, characterized in that: The system comprises: A tumor pathology image acquisition module is used to acquire annotated tumor pathology images; A tumor pathology image segmentation module, used to process the tumor pathology image using a tumor pathology image segmentation model to complete pixel-level segmentation of the tumor pathology image; The tumor pathology image segmentation model includes: a state space feature encoder, a multi-scale feature decoder, a side output module and a splicing fusion module; The state space feature encoder includes a four-stage QSSBlock feature encoder, which is used to capture the global features in the tumor pathology image and establish long-range correlations between instances to solve the shortcoming of independence between instances; The multi-scale feature decoder includes a multi-scale feature fusion module and a multi-scale convolution attention module; the multi-scale feature fusion module includes an up-convolution module and a grouping attention module; the up-convolution module is used to up-sample the feature map of the current stage to match the size and resolution of the feature map of the next skip connection; the grouping attention module is used to use the high-level semantic features obtained by the up-convolution module to enhance the low-level semantic features transmitted through the skip connection and suppress irrelevant features; the multi-scale convolution attention module is used to aggregate local information and capture multi-scale contextual information; The side output module is used to generate segmentation output from the feature maps of the four stages of the multi-scale feature decoder, restore the feature map size through upsampling operation, and use 1x1 convolution kernel to achieve channel unification; The splicing and fusion module is used to combine low-level boundary information with high-level semantic information and perform weighted averaging on the side output images.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of a tumor pathology image segmentation method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a tumor pathology image segmentation method as described in any one of claims 1 to 7 are implemented.

Images