Gleason grading method and apparatus for prostate cancer pathology images, device and medium

Abstract

Embodiments of the present application provide a Gleason grading method and device for prostate cancer pathological images, equipment and medium. The method comprises: acquiring a prostate cancer pathological image, which is a full-field digitalized section image; performing feature extraction on the prostate cancer image to obtain pathological image features; inputting the pathological image features into a multi-instance learning model based on time-frequency interaction to obtain fusion features, wherein the multi-instance learning model based on time-frequency interaction comprises a feature reconstruction module and a time-frequency feature interaction module, the feature reconstruction module is used to extract high-frequency feature information and low-frequency feature information from the pathological image features, and the time-frequency feature interaction module is used to extract time-domain feature information and frequency-domain feature information from the pathological image features; and inputting the fusion features into a classifier to obtain a Gleason grading result of the prostate cancer pathological image. Based on this, embodiments of the present application can improve the accuracy of Gleason grading of prostate cancer pathological images.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a Gleason grading method, apparatus, device, and medium for prostate cancer pathological images. Background Technology

[0002] Prostate cancer is one of the most common cancers in men, and early diagnosis and accurate grading are crucial for developing effective treatment plans. The Gleason Grading System is the standard scoring tool for prostate cancer, which helps doctors determine the degree of malignancy and prognosis by assessing the glandular differentiation of tumor tissue. Pathological imaging grading, as the foundation and gold standard for tumor diagnosis, is critically important for developing treatment plans and assessing prognosis.

[0003] With the development of digital pathology, the analysis of high-resolution digitized tissue sections has become possible, driving the advancement of computer-aided diagnostic (CAD) technology. However, whole-slide images (WSIs) have high pixel density (typically up to 40,000 × 40,000 pixels), and acquiring labeled data is challenging. Furthermore, significant heterogeneity exists between different grades of tumors, complicating deep learning-based pathological image analysis. This results in insufficient ability to capture key details in pathological images and process complex data, thus affecting the accuracy of Gleason grading. Therefore, improving the accuracy of Gleason grading for prostate cancer pathological images has become an urgent technical problem to be solved. Summary of the Invention

[0004] This invention provides a Gleason grading method, apparatus, device, and medium for prostate cancer pathological images, which can improve the accuracy of Gleason grading for prostate cancer pathological images.

[0005] In a first aspect, embodiments of the present invention provide a Gleason grading method for prostate cancer pathological images, comprising:

[0006] Obtaining pathological images of prostate cancer, wherein the pathological images of prostate cancer are full-field digital slice images;

[0007] Feature extraction was performed on the prostate cancer images to obtain pathological image features;

[0008] The pathological image features are input into a pre-trained multi-instance learning model based on time-frequency interaction to obtain fused features. The multi-instance learning model based on time-frequency interaction includes a feature reconstruction module based on wavelet transform and a time-frequency feature interaction module based on Mamba. The feature reconstruction module based on wavelet transform is used to extract high-frequency feature information and low-frequency feature information from the pathological image features. The time-frequency feature interaction module based on Mamba is used to extract time-domain feature information and frequency-domain feature information from the pathological image features.

[0009] The fused features are input into a classifier to obtain the Gleason classification result of the prostate cancer pathological image.

[0010] In some embodiments, the feature extraction from the prostate cancer image to obtain pathological image features includes:

[0011] The prostate cancer image was processed into non-overlapping slices at a set magnification using a threshold segmentation algorithm to obtain multiple non-overlapping slices of fixed size.

[0012] The non-overlapping slices were used as a feature extractor to extract features to obtain the pathological image features.

[0013] In some embodiments, inputting the pathological image features into a pre-trained multi-instance learning model based on time-frequency interaction to obtain fused features includes:

[0014] The pathological image features are input into the wavelet transform-based feature reconstruction module to obtain the high-frequency feature information and the low-frequency feature information;

[0015] The pathological image features are subjected to standard normalization processing to obtain normalized features;

[0016] The high-frequency feature information, the low-frequency feature information, and the normalized feature are input into the Mamba-based time-frequency feature interaction module to obtain the time-domain feature information and the frequency-domain feature information.

[0017] The fused feature is generated based on the time-domain feature information and the frequency-domain feature information.

[0018] In some embodiments, inputting the pathological image features into the wavelet transform-based feature reconstruction module to obtain the high-frequency feature information and the low-frequency feature information includes:

[0019] The one-dimensional feature tensor of the pathological image features is reshaped into two-dimensional spatial features;

[0020] The feature reconstruction module based on wavelet transform uses discrete wavelet transform to downsample the two-dimensional spatial features to obtain the first high-frequency component and the first low-frequency component in the row direction, and the second high-frequency component and the second low-frequency component in the column direction.

[0021] The first high-frequency component, the first low-frequency component, the second high-frequency component, and the second low-frequency component are connected along the channel dimension to obtain the feature matrix;

[0022] The feature matrix is ​​dynamically reconstructed using a feature modulation matrix to obtain the high-frequency feature information and the low-frequency feature information;

[0023] The inverse discrete wavelet transform is used to flatten the feature tensors of the high-frequency feature information and the low-frequency feature information back into one-dimensional feature vectors.

[0024] In some embodiments, inputting the high-frequency feature information, the low-frequency feature information, and the normalized feature into the Mamba-based time-frequency feature interaction module to obtain the time-domain feature information and the frequency-domain feature information includes:

[0025] The Mamba-based time-frequency feature interaction module performs feature analysis and feature extraction on the high-frequency feature information, the low-frequency feature information, and the normalized features to generate a feature sequence.

[0026] The feature sequence is converted into the time-domain feature information and the frequency-domain feature information.

[0027] In some embodiments, after converting the feature sequence into the time-domain feature information and the frequency-domain feature information, the method further includes:

[0028] The time-domain feature information and the frequency-domain feature information are exchanged through the dual-branch channel of the Mamba-based time-frequency feature interaction module;

[0029] The time-domain feature information and the frequency-domain feature information are fused to obtain the fused feature.

[0030] In some embodiments, the training method for the multi-instance learning model based on time-frequency interaction includes:

[0031] The cross-entropy loss function is determined based on the true labels, the linear projection matrix, and the probability distribution predicted by the model.

[0032] The time-frequency interaction-based multi-instance learning model is trained using the cross-entropy loss function to obtain the trained time-frequency interaction-based multi-instance learning model.

[0033] Secondly, embodiments of the present invention also provide a Gleason grading device for prostate cancer pathological images, the device comprising:

[0034] The acquisition module is used to acquire pathological images of prostate cancer, wherein the pathological images of prostate cancer are full-field digital slice images;

[0035] The extraction module is used to extract features from the prostate cancer image to obtain pathological image features;

[0036] The learning module is used to input the pathological image features into a pre-trained time-frequency interaction-based multi-instance learning model to obtain fused features. The time-frequency interaction-based multi-instance learning model includes a wavelet transform-based feature reconstruction module and a Mamba-based time-frequency feature interaction module. The wavelet transform-based feature reconstruction module is used to extract high-frequency and low-frequency feature information from the pathological image features, and the Mamba-based time-frequency feature interaction module is used to extract time-domain and frequency-domain feature information from the pathological image features.

[0037] The grading module is used to input the fused features into the classifier to obtain the Gleason grading result of the prostate cancer pathological image.

[0038] Thirdly, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the Gleason grading method for prostate cancer pathological images as described in the first aspect.

[0039] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for performing the Gleason grading method for prostate cancer pathological images as described in the first aspect.

[0040] According to embodiments of the present invention, a Gleason grading method, apparatus, device, and medium for prostate cancer pathological images are provided. The Gleason grading method for prostate cancer pathological images includes: acquiring a prostate cancer pathological image, wherein the prostate cancer pathological image is a full-view digital slice image; extracting features from the prostate cancer image to obtain pathological image features; inputting the pathological image features into a pre-trained multi-instance learning model based on time-frequency interaction to obtain fused features. The multi-instance learning model based on time-frequency interaction includes a wavelet transform-based feature reconstruction module and a Mamba-based time-frequency feature interaction module. The wavelet transform-based feature reconstruction module is used to extract high-frequency and low-frequency feature information from the pathological image features, and the Mamba-based time-frequency feature interaction module is used to extract time-domain and frequency-domain feature information from the pathological image features; and inputting the fused features into a classifier to obtain the Gleason grading result of the prostate cancer pathological image. This invention, by introducing a wavelet transform-based feature reconstruction module and a Mamba-based time-frequency feature interaction module, can effectively integrate high- and low-frequency features and their spatial information, and establish effective interaction between time-domain and frequency-domain features, thereby improving the understanding of tumor heterogeneity. Compared to existing technologies, this invention demonstrates superior capabilities in capturing crucial details and processing complex data. Based on this, embodiments of this invention can improve the accuracy of Gleason grading for prostate cancer pathological images. Attached Figure Description

[0041] Figure 1 This is a flowchart of a Gleason grading method for prostate cancer pathological images provided in one embodiment of the present invention;

[0042] Figure 2 This is a sub-flowchart of step S102 provided in one embodiment of the present invention;

[0043] Figure 3 This is a diagram of the WSIs preprocessing process provided in one embodiment of the present invention;

[0044] Figure 4 This is a sub-flowchart of step S103 provided in one embodiment of the present invention;

[0045] Figure 5 This is an overall framework diagram of TFI-MIL provided in one embodiment of the present invention;

[0046] Figure 6 This is a sub-flowchart of step S401 provided in one embodiment of the present invention;

[0047] Figure 7 This is a schematic diagram of a feature reconstruction module based on wavelet transform provided in one embodiment of the present invention;

[0048] Figure 8This is a sub-flowchart of step S403 provided in one embodiment of the present invention;

[0049] Figure 9 This is a flowchart of a training method for a multi-instance learning model based on time-frequency interaction provided in one embodiment of the present invention;

[0050] Figure 10 This is a schematic diagram of the structure of a Gleason grading device for prostate cancer pathological images provided in one embodiment of the present invention;

[0051] Figure 11 This is a schematic diagram of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0053] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the following drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0054] In this embodiment of the invention, the terms "furthermore," "exemplarily," or "optionally" are used to indicate examples, illustrations, or explanations, and should not be construed as being more preferred or advantageous than other embodiments or designs. The use of terms such as "furthermore," "exemplarily," or "optionally" is intended to present the relevant concepts in a specific manner.

[0055] To facilitate a more convenient description of the working principle of the embodiments of the present invention, the following introduction of relevant technical scenarios is given first.

[0056] Prostate cancer is one of the most common cancers in men, and early diagnosis and accurate grading are crucial for developing effective treatment plans. The Gleason Grading System is the standard scoring tool for prostate cancer, which helps doctors determine the degree of malignancy and prognosis by assessing the glandular differentiation of tumor tissue. Pathological imaging grading, as the foundation and gold standard for tumor diagnosis, is critically important for developing treatment plans and assessing prognosis.

[0057] With the development of digital pathology, the analysis of high-resolution digitized tissue sections has become possible, driving the advancement of computer-aided diagnostic (CAD) technology. However, whole-slide images (WSIs) have high pixel density (typically up to 40,000 × 40,000 pixels), and acquiring labeled data is challenging. Furthermore, significant heterogeneity exists between different grades of tumors, complicating deep learning-based pathological image analysis. This results in insufficient ability to capture key details in pathological images and process complex data, thus affecting the accuracy of Gleason grading. Therefore, improving the accuracy of Gleason grading for prostate cancer pathological images has become an urgent technical problem to be solved.

[0058] Based on this, the present invention provides a Gleason grading method, apparatus, device, and medium for prostate cancer pathological images. The Gleason grading method for prostate cancer pathological images includes: acquiring prostate cancer pathological images, wherein the prostate cancer pathological images are full-view digital slice images; extracting features from the prostate cancer images to obtain pathological image features; inputting the pathological image features into a pre-trained multi-instance learning model based on time-frequency interaction to obtain fused features, wherein the multi-instance learning model based on time-frequency interaction includes a wavelet transform-based feature reconstruction module and a Mamba-based time-frequency feature interaction module. The wavelet transform-based feature reconstruction module is used to extract high-frequency and low-frequency feature information from the pathological image features, and the Mamba-based time-frequency feature interaction module is used to extract time-domain and frequency-domain feature information from the pathological image features; and inputting the fused features into a classifier to obtain the Gleason grading result of the prostate cancer pathological image. By introducing a wavelet transform-based feature reconstruction module and a Mamba-based time-frequency feature interaction module, the present invention can effectively integrate high- and low-frequency features and their spatial information, and establish effective interaction between time-domain and frequency-domain features, thereby improving the understanding of tumor heterogeneity. Compared to existing technologies, this invention demonstrates superior capabilities in capturing crucial details and processing complex data. Based on this, embodiments of this invention can improve the accuracy of Gleason grading for prostate cancer pathological images.

[0059] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0060] like Figure 1 As shown, Figure 1 This is a flowchart of a Gleason grading method for prostate cancer pathological images provided in an embodiment of the present invention. The Gleason grading method for prostate cancer pathological images may include, but is not limited to, steps S101 to S104.

[0061] Step S101: Obtain prostate cancer pathological images. The prostate cancer pathological images are full-field digital slice images.

[0062] Step S102: Extract features from prostate cancer images to obtain pathological image features;

[0063] Step S103: Input the pathological image features into a pre-trained multi-instance learning model based on time-frequency interaction to obtain fused features. The multi-instance learning model based on time-frequency interaction includes a feature reconstruction module based on wavelet transform and a time-frequency feature interaction module based on Mamba. The feature reconstruction module based on wavelet transform is used to extract high-frequency feature information and low-frequency feature information from the pathological image features, and the time-frequency feature interaction module based on Mamba is used to extract time-domain feature information and frequency-domain feature information from the pathological image features.

[0064] Step S104: Input the fused features into the classifier to obtain the Gleason classification results of the prostate cancer pathological image.

[0065] It is understood that this invention proposes a time-frequency interaction-based multi-instance learning method for Gleason grading of prostate cancer pathological sections. Specifically, the time-frequency interaction-based multi-instance learning model of this invention includes:

[0066] (1) The Wavelet Transform-based Reconstruction Module (WTRM) is used to learn high-frequency and low-frequency feature information in pathological images. Through frequency domain processing, this module enables the model to more accurately capture the edge, texture and shape information of pathological slice features that are difficult to detect in the time domain, thereby improving the ability to identify key details and establishing spatial relationships between instances within the package.

[0067] (2) The Time-Frequency Feature Interaction Module Based on Mamba (TFIM-BM) aims to enhance the dynamic interaction and information fusion between different features. This module effectively promotes the interaction between features, ensuring that spatial structure and detailed information can be better preserved when processing complex pathological images, thereby improving the accuracy of classification and grading.

[0068] By combining these two modules, the present invention significantly enhances the ability to analyze pathological images, providing more accurate support for the diagnosis of prostate cancer.

[0069] It is understood that this invention constructs a Time-Frequency Interaction Based Multiple Instance Learning (TFI-MIL) method for Gleason grading of prostate cancer pathological sections. First, a wavelet transform-based feature reconstruction module spatially reconstructs the features to learn high- and low-frequency feature information in the pathological image and establish spatial information between instances. Second, a Mamba-based time-frequency feature interaction module is designed to enhance dynamic interaction and information fusion between different features. Even in the absence of fine-grained labels, by utilizing slice-level labels for model training and prediction, it maintains high accuracy when handling complex WSIs. This invention demonstrates good potential for achieving accurate prostate cancer grading, providing important support for clinical pathological analysis. This not only helps improve the accuracy of tumor diagnosis but also provides an important basis for the development of personalized treatment plans.

[0070] In one embodiment, such as Figure 2 As shown, step S102 includes, but is not limited to, the following steps:

[0071] Step S201: The threshold segmentation algorithm is used to process the prostate cancer image into non-overlapping slices at a set magnification to obtain multiple non-overlapping slices of fixed size.

[0072] Step S202: Use ResNet50 pre-trained on ImageNet as a feature extractor to extract features from non-overlapping slices to obtain pathological image features.

[0073] Understandably, preprocessing full-view digital slice images (WSIs) using the Overthreshold Segmentation (OTSU) algorithm at 20x magnification to perform non-overlapping slicing (discarding the background and retaining tissue regions occupying more than 40% of the image area) yields n fixed-size 512×512 pixel non-overlapping image slices. Then, using a ResNet50 pre-trained on ImageNet as a feature extractor, these slices are mapped into n 1024-dimensional low-dimensional feature vectors. The entire preprocessing process is as follows: Figure 3 As shown.

[0074] In one embodiment, such as Figure 4 As shown, step S103 includes, but is not limited to, the following steps:

[0075] Step S401: Input the pathological image features into the wavelet transform-based feature reconstruction module to obtain high-frequency feature information and low-frequency feature information;

[0076] Step S402: Perform standard normalization on the pathological image features to obtain normalized features;

[0077] Step S403: Input the high-frequency feature information, low-frequency feature information, and normalized features into the Mamba-based time-frequency feature interaction module to obtain time-domain feature information and frequency-domain feature information;

[0078] Step S404: Generate fused features based on time-domain feature information and frequency-domain feature information.

[0079] It is understood that the overall framework of TFI-MIL constructed in this invention is as follows: Figure 5 As shown, the specific implementation process of TFI-MIL is as follows: First, the input features are passed to a dual-branch shared deep linear layer module, which aims to initially extract temporal features. Through the weighted combination of multiple neurons, this layer delves deeper into the potential relationships between features, thereby enhancing the representativeness of the features. Next, in branch one, the features are input to the WTRM module for spatial reconstruction. In this module, the original features are first converted into a two-dimensional space for subsequent processing. Then, wavelet transform is applied to decompose and reconstruct the features, effectively separating high-frequency and low-frequency information in the features, thereby better capturing the detailed key information in the data. At the same time, the feature modulation matrix dynamically adjusts the reconstruction parameters to further enhance the overall expressive power of the features. In branch two, the features are normalized. Then, the features from both branches are jointly input into the Mamba-based time-frequency feature interaction module TFIM-BM to achieve interactive learning and effective fusion between time-frequency features to generate the final representation of WSIs. Finally, a linear classification layer processes the final representation of WSIs and outputs the Gleason grading prediction results for prostate cancer WSIs.

[0080] In one embodiment, such as Figure 6 As shown, step S401 includes, but is not limited to, the following steps:

[0081] Step S601: Reconstruct the one-dimensional feature tensor of the pathological image features into two-dimensional spatial features;

[0082] Step S602: The two-dimensional spatial features are downsampled by the feature reconstruction module based on wavelet transform using discrete wavelet transform to obtain the first high-frequency component and the first low-frequency component in the row direction, and the second high-frequency component and the second low-frequency component in the column direction.

[0083] Step S603: Connect the first high-frequency component, the first low-frequency component, the second high-frequency component, and the second low-frequency component along the channel dimension to obtain the feature matrix;

[0084] Step S604: The feature matrix is ​​dynamically reconstructed using the feature modulation matrix to obtain high-frequency feature information and low-frequency feature information;

[0085] Step S605: The inverse discrete wavelet transform is used to flatten the feature tensors of high-frequency and low-frequency feature information back into one-dimensional feature vectors.

[0086] It is understood that the wavelet transform-based feature reconstruction module of the present invention, as well as... Figure 7 As shown, by integrating wavelet transform into the Transformer module, high-resolution image features are downsampled while preserving important high-frequency, low-frequency details and edge information. Furthermore, a learnable feature modulation matrix is ​​used instead of a traditional self-attention layer to more effectively blend labels from different spatial locations. The specific implementation process of feature reconstruction in WTRM is as follows:

[0087] First, for each input image, the one-dimensional feature tokens tensor X∈R C×N×D Reshape it into two-dimensional spatial features Where C represents the number of channels, N represents the number of tokens, and D represents the feature dimension.

[0088] Then, WTRM uses Discrete Wavelet Transform (DWT) to downsample the input feature tensor. To simplify computation, we use the classic Haar wavelet as the wavelet basis in the Discrete Wavelet Transform. After DWT processing, four subbands are obtained:

[0089]

[0090] Specifically, DWT uses a low-pass filter. High-pass filter The input features are decomposed along the row direction to obtain two wavelets: the low-frequency component (X). L ) and high-frequency components (X) H Similarly, by using the same low-pass and high-pass filters to decompose the input features along the column direction, two additional sub-bands can be obtained: the low-frequency component (X). L ) and high-frequency components (X) H Ultimately, four wavelet subbands can be obtained: Among them, X LL This represents low-frequency components, reflecting basic image information at a coarse-grained level. X LH X HL and X HHRepresenting high-frequency components, it preserves fine-grained image texture details. Each sub-band can be viewed as a downsampled result of the input features, covering all the details of the input feature information without losing any detail.

[0091] Next, the four wavelet subbands are connected along the channel dimension to obtain a new feature matrix:

[0092]

[0093] Then, a convolution operation is performed between the feature modulation matrix and the coefficients of the last-level decomposition. The feature modulation matrix is ​​defined as D. M ∈R 4C×h×w This is equivalent to a deep global recurrent convolution of size h / 2 × w / 2. By learning the parameters in the feature modulation matrix, the model can automatically adjust the frequency weights to achieve noise reduction and feature enhancement of the image. Its formulation is inspired by frequency filters in digital image processing and can be viewed as a set of learnable frequency weights applicable to different hidden dimensions. The specific operation is as follows:

[0094]

[0095] Finally, the inverse discrete wavelet transform (IDWT) is used to update and recover the feature tensor, and it is flattened back into a one-dimensional eigenvector:

[0096]

[0097] This method fully leverages the advantages of wavelet transform, enabling the model to utilize information from each frequency component and better capture details and texture information in the input data. Through feature decomposition and reconstruction, the model's ability to process high-resolution features is enhanced, more effectively capturing the spatial structure of the image.

[0098] In one embodiment, such as Figure 8 As shown, step S403 includes, but is not limited to, the following steps:

[0099] Step S801: The high-frequency feature information, low-frequency feature information and normalized features are analyzed and extracted through the Mamba-based time-frequency feature interaction module to generate a feature sequence.

[0100] Step S802: Convert the feature sequence into time-domain feature information and frequency-domain feature information.

[0101] Step S803: Information exchange between time-domain feature information and frequency-domain feature information is performed through the dual-branch channel of the Mamba-based time-frequency feature interaction module;

[0102] Step S804: Information fusion is performed on the time-domain feature information and the frequency-domain feature information to obtain fused features.

[0103] Understandably, in order to more effectively realize the contextual relationships of features and establish interactions between time-frequency features, this invention introduces a time-frequency feature interaction module based on Mamba, such as... Figure 5 As shown. This module aims to enhance the model's ability to understand feature information by processing complex time-frequency features.

[0104] Specifically, this module receives two different types of feature inputs: X t X f By designing specific convolutional and activation layer structures, the module can perform in-depth analysis and extraction of these input features. After processing by these layers, the generated feature sequences will provide richer and more detailed temporal features for the State-Space Model (SSM) block.

[0105] SSM is a mathematical framework that uses a set of linear equations to describe the evolution of a system's state over time. SSM modeling defines how the state changes over time and how the inputs map to the outputs through the following equations:

[0106] h t =Ah t-1 +Bx t

[0107] y t =C T h t

[0108] SSM describes the evolution of the state over time using the state transition matrix A, the input matrix B maps the input x to the state space, and the output matrix C maps the state space to the output space.

[0109] Attention mechanisms enable models to focus on the important parts of sequential data. In the Transformer model, attention is achieved by calculating the relationship between the query (Q), key (K), and value (V). Attention mechanisms are typically defined by the following formula:

[0110]

[0111] Where, d k Indicates the dimension of the key.

[0112] The State-Space Dual Framework combines a state-space model with an attention mechanism to enhance the model's ability to process sequential data. SSD introduces a structured mask matrix L to control state transitions, replacing the traditional softmax operation, thereby achieving a function similar to an attention mechanism and enabling linear time complexity for attention computation. Its formula is expressed as:

[0113] SSD = (LoQK) T )·V

[0114] At the beginning of each module, A, K, Q, and V are mapped in parallel. A data-based location information mask matrix L is used to control the propagation of information over time, thus replacing the traditional heuristic location embedding method.

[0115] To further enhance the model's learning ability and robustness, PhysMamba introduces a dual-channel model. To avoid potential redundancy between independent channels, this invention designs TFIM-BM, such as... Figure 5 As shown, this module promotes information sharing and complementary advantages by sharing feature information between the two branches. One branch provides time-domain feature information, and the other branch provides frequency-domain feature information. This allows the two branches to share feature information, achieving mutual constraint and promotion between the two branches. The attention output of the two channels is:

[0116]

[0117] After passing through the TFIM-BM module, the obtained bi-branch feature X cf X ct Connect along the channel dimension to obtain the fused feature X. fusion .

[0118] X fusion =Concat(X) cf X ct )

[0119] Next, these fused features will be fed into a classifier to achieve Gleason grading of prostate cancer WSI.

[0120] In one embodiment, such as Figure 9 As shown, the training method for a multi-instance learning model based on time-frequency interaction includes, but is not limited to, the following steps:

[0121] Step S901: Determine the cross-entropy loss function, which is determined based on the true labels, the linear projection matrix, and the probability distribution predicted by the model.

[0122] Step S902: Train the time-frequency interaction-based multi-instance learning model based on the cross-entropy loss function to obtain the trained time-frequency interaction-based multi-instance learning model.

[0123] It is understood that this invention uses the cross-entropy loss function to optimize the model. This loss function can be expressed as:

[0124]

[0125] Among them, y i Here, M is the true label, P is the probability distribution predicted by the model, and is the total number of categories. The time-frequency interaction-based multi-instance learning model is trained using the cross-entropy loss function to obtain the trained model.

[0126] Based on this, compared with the prior art, the Gleason grading method for prostate cancer pathological images of the present invention has at least the following advantages:

[0127] Beneficial effects:

[0128] (1) This invention employs a weakly supervised multi-instance learning method to effectively achieve the Gleason classification task for prostate cancer under limited labeled data.

[0129] (2) The present invention constructs a feature reconstruction module based on wavelet transform, which aims to efficiently extract and learn high-frequency and low-frequency feature information in pathological images, establish spatial relationships between instances, and thus enhance the ability to identify key details.

[0130] (3) The present invention designs a time-frequency feature interaction module based on Mamba, which aims to enhance the dynamic interaction and information fusion between different features and improve the overall performance and accuracy of the model.

[0131] It should be noted that existing technologies often neglect the contextual information of images and the dynamic relationships between instances. In contrast, the embodiments of this invention, by introducing a wavelet transform-based feature reconstruction module and a Mamba-based time-frequency feature interaction module, can effectively integrate high- and low-frequency features and their spatial information, and establish effective interaction between time-domain and frequency-domain features, thereby improving the understanding of tumor heterogeneity. Compared with existing technologies, this invention demonstrates stronger capabilities in capturing key details and processing complex data, and has greater clinical application value and promotion potential.

[0132] In addition, such as Figure 10 As shown, one embodiment of the present invention also discloses a Gleason grading device for prostate cancer pathological images, the device comprising:

[0133] The acquisition module 110 is used to acquire pathological images of prostate cancer, which are full-view digital slice images.

[0134] The extraction module 120 is used to extract features from prostate cancer images to obtain pathological image features;

[0135] The learning module 130 is used to input pathological image features into a pre-trained time-frequency interaction-based multi-instance learning model to obtain fused features. The time-frequency interaction-based multi-instance learning model includes a wavelet transform-based feature reconstruction module and a Mamba-based time-frequency feature interaction module. The wavelet transform-based feature reconstruction module is used to extract high-frequency and low-frequency feature information from pathological image features, and the Mamba-based time-frequency feature interaction module is used to extract time-domain and frequency-domain feature information from pathological image features.

[0136] The grading module 140 is used to input the fused features into the classifier to obtain the Gleason grading results of the prostate cancer pathology image.

[0137] The Gleason grading device for prostate cancer pathological images in this embodiment of the invention is used to execute the Gleason grading method for prostate cancer pathological images in the above embodiments. Its specific processing procedure is the same as that of the Gleason grading method for prostate cancer pathological images in the above embodiments, and will not be described in detail here.

[0138] In addition, such as Figure 11 As shown, one embodiment of the present invention also discloses an electronic device, including: at least one processor 210; at least one memory 220 for storing at least one program; when the at least one program is executed by the at least one processor 210, it implements the Gleason grading method for prostate cancer pathological images as in any of the preceding embodiments.

[0139] In addition, one embodiment of the present invention discloses a computer-readable storage medium storing computer-executable instructions for performing the Gleason grading method for prostate cancer pathological images as described in any of the preceding embodiments.

[0140] The system architecture and application scenarios described in the embodiments of this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of system architecture and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0141] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0142] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0143] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process or execution thread, and components may be located on a single computer or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, or a network, such as the Internet interacting with other systems via signals).

Claims

1. A Gleason grading method of a prostate cancer pathology image, comprising: obtaining a prostate cancer pathology image, the prostate cancer pathology image being a whole field digitalized slice image; extracting features of the prostate cancer image to obtain pathology image features; inputting the pathology image features into a pre-trained multi-instance learning model based on time-frequency interaction to obtain fusion features, including: inputting the pathology image features into a feature reconstruction module based on wavelet transform to obtain high-frequency feature information and low-frequency feature information; performing standard normalization processing on the pathology image features to obtain normalized features; inputting the high-frequency feature information, the low-frequency feature information, and the normalized features into a time-frequency feature interaction module based on Mamba to obtain time-domain feature information and frequency-domain feature information; and generating the fusion features according to the time-domain feature information and the frequency-domain feature information, wherein the multi-instance learning model based on time-frequency interaction includes the feature reconstruction module based on wavelet transform and the time-frequency feature interaction module based on Mamba, the feature reconstruction module based on wavelet transform is used to extract the high-frequency feature information and the low-frequency feature information from the pathology image features, and the time-frequency feature interaction module based on Mamba is used to extract the time-domain feature information and the frequency-domain feature information from the pathology image features; inputting the fusion features into a classifier to obtain a Gleason grading result of the prostate cancer pathology image.

2. The method of claim 1, wherein, The extracting features of the prostate cancer image to obtain pathology image features includes: performing non-overlapping slice processing on the prostate cancer image at a set magnification using a threshold segmentation algorithm to obtain a plurality of fixed-size non-overlapping non-overlapping slices; using a ResNet50 pre-trained on ImageNet as a feature extractor to extract features of the non-overlapping slices to obtain the pathology image features.

3. The method of claim 1, wherein, The inputting the pathology image features into the feature reconstruction module based on wavelet transform to obtain the high-frequency feature information and the low-frequency feature information includes: reshaping a one-dimensional feature tensor of the pathology image features into a two-dimensional spatial feature; performing down-sampling on the two-dimensional spatial feature using a discrete wavelet transform through the feature reconstruction module based on wavelet transform to obtain a first high-frequency component and a first low-frequency component in a row direction, and a second high-frequency component and a second low-frequency component in a column direction; connecting the first high-frequency component, the first low-frequency component, the second high-frequency component, and the second low-frequency component along a channel dimension to obtain a feature matrix; reconstructing the feature matrix dynamically using a feature modulation matrix to obtain the high-frequency feature information and the low-frequency feature information; re-flattening feature tensors of the high-frequency feature information and the low-frequency feature information into one-dimensional feature vectors using an inverse discrete wavelet transform.

4. The method of claim 1, wherein, The inputting the high-frequency feature information, the low-frequency feature information, and the normalized features into the time-frequency feature interaction module based on Mamba to obtain the time-domain feature information and the frequency-domain feature information includes: The high-frequency feature information, the low-frequency feature information and the normalized feature are analyzed and extracted by the Mamba-based time-frequency feature interaction module to generate a feature sequence; The feature sequence is converted into the time-domain feature information and the frequency-domain feature information.

5. The method of claim 4, wherein, After the feature sequence is converted into the time-domain feature information and the frequency-domain feature information, the method further comprises: The time-domain feature information and the frequency-domain feature information are interacted by a double-branch channel of the Mamba-based time-frequency feature interaction module; The time-domain feature information and the frequency-domain feature information are fused to obtain the fusion feature.

6. The method of claim 1, wherein, The training method of the time-frequency interaction-based multiple instance learning model comprises: A cross-entropy loss function is determined according to a real label, a linear projection matrix and a probability distribution predicted by a model; The time-frequency interaction-based multiple instance learning model is trained based on the cross-entropy loss function to obtain a trained time-frequency interaction-based multiple instance learning model.

7. A Gleason grading device for prostate cancer pathological images, characterized in that, The device comprises: An acquisition module configured to acquire a prostate cancer pathological image, the prostate cancer pathological image being a full-view digital slice image; An extraction module configured to extract features of the prostate cancer image to obtain pathological image features; A learning module configured to input the pathological image features into a pre-trained time-frequency interaction-based multiple instance learning model to obtain fusion features, comprising: inputting the pathological image features into a wavelet transform-based feature reconstruction module to obtain high-frequency feature information and low-frequency feature information; performing standard normalization on the pathological image features to obtain normalized features; inputting the high-frequency feature information, the low-frequency feature information and the normalized features into a Mamba-based time-frequency feature interaction module to obtain time-domain feature information and frequency-domain feature information; and generating the fusion features according to the time-domain feature information and the frequency-domain feature information, wherein the time-frequency interaction-based multiple instance learning model comprises the wavelet transform-based feature reconstruction module and the Mamba-based time-frequency feature interaction module, the wavelet transform-based feature reconstruction module is configured to extract the high-frequency feature information and the low-frequency feature information from the pathological image features, and the Mamba-based time-frequency feature interaction module is configured to extract the time-domain feature information and the frequency-domain feature information from the pathological image features; A grading module configured to input the fusion features into a classifier to obtain a Gleason grading result of the prostate cancer pathological image.

8. An electronic device comprising: A memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the Gleason grading method of the prostate cancer pathological image according to any one of claims 1 to 6 when executing the computer program. 9.A computer readable storage medium storing computer executable instructions for executing the Gleason grading method of the prostate cancer pathological image according to any one of claims 1 to 6.

Images