A breast cancer molecular typing method and system based on histopathological images

Abstract

The present application belongs to the technical field of histopathological image auxiliary diagnosis, and particularly relates to a breast cancer molecular typing method and system based on histopathological images, wherein a digital pathology image is acquired, feature extraction is performed on the digital pathology image, a feature embedding representation of the digital pathology image is acquired, the feature embedding representation is input into a typing model based on deep learning, and the probability of each molecular subtype is obtained. In the typing model based on deep learning, long-range dependence is captured through an attention mechanism, a local spatial structure is extracted through a convolutional neural network module, and effective fusion of the global and the local is realized through an interaction mechanism. In the present application, dynamic weighting of the attention mechanism is combined with residual learning of the convolutional neural network module, the modeling capability of the model for complex tissue patterns is effectively improved, high efficiency is achieved when high-dimensional spatial heterogeneous data is processed, and thus higher efficiency and accuracy are provided in triple-negative breast cancer molecular subtype classification.

Description

Technical Field

[0001] This invention belongs to the field of histopathological image-assisted diagnosis technology, specifically relating to a molecular subtyping method and system for breast cancer based on histopathological images. Background Technology

[0002] Breast cancer is a malignant tumor with a high incidence rate. Triple-negative breast cancer accounts for 10% to 20% of breast cancer cases. Because its cancer cells lack the expression of estrogen receptor and progesterone receptor, as well as the gene amplification of human epidermal growth factor receptor 2, the application of traditional targeted therapy is limited.

[0003] Triple-negative breast cancer is classified into three subtypes—CS1, CS2, and CS3—based on cell surface protein characteristics. Spatial proteomics can reveal the spatial heterogeneity of cell type composition, intercellular interactions, and tumor microenvironment structure. These differential features can be observed in H&E-stained cell sections. Accurate classification of triple-negative breast cancer subtypes is crucial for improving patient treatment outcomes. Proteomics-based subtyping largely relies on high-throughput proteomics technologies. However, multi-omics analysis typically suffers from time-consuming, costly, and complex operations, limiting its feasibility for achieving accurate diagnosis, improving efficiency, controlling costs, and widespread application in clinical practice. Conventional subtyping results are generally obtained through professional observation and analysis of full-view digital pathology images. However, manual analysis requires highly skilled pathologists for diagnosis and is characterized by low subtyping efficiency and high subjectivity.

[0004] Therefore, existing technologies use large pathological models, such as feature extraction models based on contrastive learning, to achieve quantitative pathological diagnosis and prognostic analysis. However, existing pathological models fail to fully integrate global and local information from full-view digital pathological images, resulting in low accuracy in subtyping triple-negative breast cancer. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the existing pathological models in which the global and local information of the full-view digital pathological images are not fully integrated, resulting in low accuracy of triple-negative breast cancer subtype classification. The present invention provides a method and system for molecular subtyping of breast cancer based on histopathological images.

[0006] A molecular subtyping method for breast cancer based on histopathological images includes the following steps:

[0007] Acquire digital pathological images; extract features from the digital pathological images to obtain feature embedding representations; input the feature embedding representations into a deep learning-based subtyping model; obtain the output of the deep learning-based subtyping model to obtain the probability of each molecular subtype;

[0008] The deep learning-based subtyping model obtains feature embedding representations and outputs the probabilities of each molecular subtype, including the following steps:

[0009] The input feature vector of the deep learning-based subtyping model is constructed by embedding the features of digital pathological images; the input feature vector is then transformed by an initial fully connected layer to obtain the initial features.

[0010] The initial features are input into the first convolutional neural network block, and after two-dimensional reshaping, convolution and flattening, the first convolutional flattened features are formed.

[0011] The initial features are fused with the first convolution flattening features. Attention weights are assigned to each image patch of the digital pathology image through attention. Attention weighted features are calculated based on the attention weights. The attention weighted features are then fused with the initial features and the first convolution flattening features to form deep attention features.

[0012] The deep attention features are fused with the first convolutional flattening features and input into the second convolutional neural network block. After two-dimensional reshaping, convolution and flattening, the second convolutional flattening features are formed.

[0013] Deep attention features are fused with second convolution flattening features to form deep fusion features;

[0014] Global average pooling is used to compress the deep fusion features to form panoramic slice features. The panoramic slice features are then passed through a final fully connected layer to map them to the original prediction scores. The original prediction scores are then normalized to form the fractal probability output.

[0015] Furthermore, before feature extraction from digital pathological images, preprocessing of stained tissue pathological images is also included, comprising the following steps:

[0016] The Otsu thresholding method is used to extract tissue regions from digital pathology images. The tissue regions of the digital pathology images are downsampled at a preset resolution and loaded into memory. The tissue regions are divided into non-overlapping squares of a preset size. The minimum threshold of the bit value in the preset RGB channels is used to filter out squares with less tissue. The pre-processed squares are defined as image blocks.

[0017] Furthermore, feature extraction is performed on digital pathological images, including the following steps:

[0018] The method for obtaining feature embedding representations of digital pathology images includes the following steps:

[0019] A pre-trained visual-linguistic pathology model is loaded, and the output dimension of the attention pooling layer is set to a preset dimension. A feature extractor is provided to generate a feature vector representation of a preset dimension for each image patch. Multi-instance learning is employed, using digital pathology images as packets and image patches as instances. The aggregated feature embedding representation is as follows:

[0020] ;

[0021] Where N is the number of image patches in the digital pathology image. Indicates the feature dimension.

[0022] Furthermore, in the deep learning-based fractal model, the input feature vector is transformed through an initial fully connected layer to obtain initial features, including the following steps:

[0023] A deep learning-based subtyping model is constructed. The input feature vector of the deep learning-based subtyping model is represented by feature embedding of digital pathological images, as follows:

[0024] ;

[0025] in, This indicates the batch size, i.e., the number of digital pathology images. Indicates the number of image patches. Indicates the feature dimension;

[0026] The input feature vector passes through the initial fully connected layer. Transformation yields initial features , represented as:

[0027] ;

[0028] in, , , Indicates the output dimension.

[0029] Furthermore, the initial features are fused with the first convolutional flattening features. Attention weights are assigned to each image patch of the digital pathology image through attention. Attention-weighted features are calculated based on the attention weights. The attention-weighted features are then fused with the initial features and the first convolutional flattening features to form deep attention features. This process includes the following steps:

[0030] The initial features are fused with the first convolution flattened features and input into the gated attention module. By combining the gating signals of the Tanh and Sigmoid activation functions, the input features are dynamically weighted to highlight key image patches related to the target category. The attention-weighted features are fused with the initial features and the first convolution flattened features to form deep attention features.

[0031] The fusion feature of the initial features and the first convolution flattened features Attention weights are assigned to each image patch using gating attention, as follows:

[0032] ;

[0033] in, Indicates the first Attention weights for each image patch, , and These represent the learnable weight parameters, Indicates the first Feature representation of an image patch Indicates the number of pathological image patches. This indicates element-wise multiplication. and It is a non-linear activation function;

[0034] ;

[0035] ;

[0036] in This represents attention-weighted features. This represents the characteristics of deep attention. This indicates the batch size, i.e., the number of digital pathology images. Representing feature dimension, This represents the attention tensor formed by combining the attention weights of multiple image patches.

[0037] Furthermore, the convolutional neural network block includes a first convolutional neural network block and a second convolutional neural network block; features are input into the convolutional neural network block, and after two-dimensional reshaping, convolution, and flattening, convolutional flattened features are formed, including the following method steps:

[0038] Features 2D reshaping converted into reshaping feature map ,in and for The approximate value of the square root;

[0039] The reconstructed feature map is used to form convolutional features through a convolutional neural network, which is represented as follows:

[0040] ;

[0041] Where DWConv represents depthwise convolution, and Conv includes pointwise convolution and standard convolution;

[0042] Convolutional features are flattened to form convolutional flattened features. , This indicates the batch size, i.e., the number of digital pathology images. Indicates the number of image patches. Indicates the feature dimension.

[0043] Furthermore, the deep attention features are fused with the first convolutional flattening features and input into the second convolutional neural network block. After two-dimensional reshaping, convolution, and flattening, the second convolutional flattening features are formed. The deep attention features and the second convolutional flattening features are then fused to form a deep fusion feature. This includes the following steps:

[0044] The deep attention features are reshaped in two dimensions and then fused with the first convolutional flattening features from the output of the first convolutional neural network block. After processing and flattening by the convolutional block, the second convolutional flattening features are formed, represented as follows:

[0045] ;

[0046] in, This represents the two-dimensional feature map corresponding to the first convolution flattened feature;

[0047] Flatten the features of the second convolution and deep feature representation Layer normalization and channel splicing form deep fusion features. , represented as:

[0048] ;

[0049] in, This indicates the batch size, i.e., the number of digital pathology images. Indicates the number of image patches. Indicates the feature dimension.

[0050] Furthermore, the loss function of the deep learning-based fractal model is expressed as:

[0051] ;

[0052] in, Represents the total number of samples. Indicates the first Each sample was predicted to be the true label. The probability, Indicate category Weighting factor This indicates the focus parameter.

[0053] A breast cancer molecular subtyping system based on histopathological images, used to implement the aforementioned breast cancer molecular subtyping method, includes:

[0054] The preprocessing module is used to preprocess stained tissue pathology images to form image blocks before feature extraction from digital pathology images;

[0055] A pre-trained visual-linguistic pathology model is used to extract features from digital pathology images and obtain feature embedding representations of digital pathology images.

[0056] A deep learning-based subtyping model is used to input feature embedding representations into the deep learning-based subtyping model; the output of the deep learning-based subtyping model is obtained to obtain the probability of each molecular subtype.

[0057] Furthermore, the pre-trained visual-linguistic pathology model includes an initial fully connected layer, a gated attention module, a convolutional neural network block, a branching interaction module, and a final fully connected layer;

[0058] The initial fully connected layer is used to transform the input feature vector to obtain initial features;

[0059] The gated attention module is used to dynamically weight input features by combining the gate signals of Tanh and Sigmoid activation functions, highlight key image patches related to the target category, and fuse the attention-weighted features with the initial features and the first convolution flattening features to form deep attention features;

[0060] The convolutional neural network block is used to transform the feature input into convolutional flattened features through two-dimensional reshaping, convolution, and flattening.

[0061] The branch interaction module is used to fuse deep attention features with second convolution flattening features to form deep fusion features;

[0062] The final fully connected layer is used to map panoramic slice features to raw prediction scores, and the raw prediction scores are normalized to form a fractal probability output.

[0063] Beneficial Effects: This invention discloses a molecular subtyping method for breast cancer based on histopathological images. It acquires digital pathological images, extracts features from these images, and obtains feature embedding representations. These feature embedding representations are then input into a deep learning-based subtyping model to obtain the probabilities of each molecular subtype. In the deep learning-based subtyping model, an attention mechanism captures long-range dependencies, a convolutional neural network module extracts local spatial structures, and an interaction mechanism further achieves effective fusion of global and local data. In this invention, the dynamic weighting of the attention mechanism combined with the residual learning of the convolutional neural network module effectively improves the model's ability to model complex tissue patterns and demonstrates high efficiency in processing heterogeneous data in high-dimensional spaces, thus providing higher efficiency and accuracy in the classification of triple-negative breast cancer molecular subtypes. Attached Figure Description

[0064] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0065] Figure 1 This is a schematic diagram of the deep learning-based classification model method of the present invention;

[0066] Figure 2 This is a schematic diagram of the deep learning-based fractal model structure of the present invention;

[0067] Figure 3 This is a schematic diagram of the convolutional neural network module structure of the present invention. Detailed Implementation

[0068] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0069] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0070] Example 1:

[0071] Reference Figure 1 , Figure 2 and Figure 3 As shown, this embodiment provides a method for molecular subtyping of breast cancer based on histopathological images, including the following steps:

[0072] Acquire digital pathological images; extract features from the digital pathological images to obtain feature embedding representations; input the feature embedding representations into a deep learning-based subtyping model; obtain the output of the deep learning-based subtyping model to obtain the probability of each molecular subtype;

[0073] Before feature extraction from digital pathology images, preprocessing of stained tissue pathology images is also included, including:

[0074] The Otsu thresholding method is used to extract tissue regions from digital pathology images (WSI, whole slide image). The tissue regions of the digital pathology images are downsampled at a preset resolution and loaded into memory. The tissue regions are divided into non-overlapping squares of a preset size. The minimum threshold of the bit values ​​in the preset RGB channels is used to filter out squares with less tissue. The preprocessed squares are defined as image patches.

[0075] In this embodiment, the tissue region of the digital pathology image downsampled at 10X resolution is loaded into memory, and the tissue region is divided into image blocks of 256×256 pixels. These blocks are further divided into non-overlapping square blocks with a side length of 360μm and adjusted to 224×224 pixels. By setting a minimum threshold of 20 for the median value of each 8-bit RGB channel, specifically, calculating the median value of the histogram for each RGB channel in the brightness range [100, 200) and setting a threshold of 20, blank image blocks are filtered out, thereby filtering out blocks with no tissue or very little tissue.

[0076] Feature extraction from digital pathology images includes the following steps:

[0077] The method for obtaining feature embedding representations of digital pathology images includes the following steps:

[0078] A pre-trained visual-linguistic pathology model is loaded, and the output dimension of the attention pooling layer is set to a preset dimension. A feature extractor is provided to generate a feature vector representation of a preset dimension for each image patch. Multi-instance learning is employed, using digital pathology images as packets and image patches as instances. The aggregated feature embedding representation is as follows:

[0079] ;

[0080] Where N is the number of image patches in the digital pathology image. This represents the feature dimension. Preferably, in this embodiment, the preset dimension and feature dimension are 512.

[0081] In this embodiment, the pre-trained visual language pathology model is preferably the CONCH model. In other embodiments of this embodiment, the pre-trained visual language pathology model may also be UNI, MUSK or other models.

[0082] Deep learning-based subtyping models acquire feature embedding representations and output the probabilities of each molecular subtype, including:

[0083] Step S1: Construct the input feature vector of the deep learning-based subtyping model by embedding the feature representation of the digital pathology image; the input feature vector is transformed by the initial fully connected layer to obtain the initial features;

[0084] Step S2: Input the initial features into the first convolutional neural network block, and form the first convolutional flattened features through two-dimensional reshaping, convolution, and flattening;

[0085] Step S3: Fuse the initial features with the first convolution flattening features, assign attention weights to each image patch of the digital pathology image through attention, calculate attention-weighted features based on attention weights, and fuse the attention-weighted features with the initial features and the first convolution flattening features to form deep attention features;

[0086] Step S4: Fuse the deep attention features with the first convolutional flattening features, input them into the second convolutional neural network block, and form the second convolutional flattening features through two-dimensional reshaping, convolution, and flattening; fuse the deep attention features with the second convolutional flattening features to form a deep fusion feature;

[0087] Step S5: Perform global average pooling compression on the deep fusion features to form panoramic slice features; pass the panoramic slice features through the final fully connected layer to map the panoramic slice features to the original prediction scores, and normalize the original prediction scores to form the fractal probability output.

[0088] In the deep learning-based subtyping model, step S1 involves transforming the input feature vector through an initial fully connected layer to obtain initial features, including the following steps:

[0089] A deep learning-based subtyping model is constructed. The input feature vector of the deep learning-based subtyping model is represented by feature embedding of digital pathological images, as follows:

[0090] ;

[0091] in, This indicates the batch size, i.e., the number of digital pathology images. Indicates the number of image patches. Indicates the feature dimension;

[0092] The input feature vector passes through the initial fully connected layer. Transformation yields initial features , represented as:

[0093] ;

[0094] in, , , Indicates the output dimension.

[0095] As a preferred embodiment, the output dimension and feature dimension are 512.

[0096] In this embodiment, step S2, fusing the initial features with the first convolutional flattening features, assigning attention weights to each image patch of the digital pathology image through attention, calculating attention-weighted features based on the attention weights, and fusing the attention-weighted features with the initial features and the first convolutional flattening features to form deep attention features, includes the following method steps:

[0097] The initial features are fused with the first convolution flattened features and input into the gated attention module. By combining the gating signals of the Tanh and Sigmoid activation functions, the input features are dynamically weighted to highlight key image patches related to the target category. The attention-weighted features are fused with the initial features and the first convolution flattened features to form deep attention features.

[0098] The fusion feature of the initial features and the first convolution flattened features Attention weights are assigned to each image patch using gating attention, as follows:

[0099] ;

[0100] in, Indicates the first Attention weights for each image patch, , and These represent the learnable weight parameters, Indicates the first Feature representation of an image patch Indicates the number of pathological image patches. This indicates element-wise multiplication. and It is a non-linear activation function;

[0101] ;

[0102] ;

[0103] in This represents attention-weighted features. This represents the characteristics of deep attention. This indicates the batch size, i.e., the number of digital pathology images. Representing feature dimension, This represents the attention tensor formed by the combination of attention weights for all image patches.

[0104] This branch combines the gating signals of the Tanh and Sigmoid activation functions, feeding the input features into two parallel branches, Tanh and Sigmoid, respectively, to capture semantic features and gating information. The two representations are then fused through element-wise multiplication (Hadamard Product). The fused features are then passed through a linear mapping to obtain attention weights, and normalized using a Softmax operation, ultimately achieving weight allocation for all image patches. This structure effectively enhances the expressive power of the attention mechanism, allowing the model to focus more on key regions. By dynamically weighting input features, it highlights key image patches related to the target category, thereby generating deep feature representations. `LayerNorm(·)` represents the layer normalization operation, used to stabilize the training process and improve model convergence speed. This method normalizes the mean and variance of the feature dimensions of each sample, i.e., standardizing the feature vector of each row to reduce scale differences between different features. `LayerNorm` is used for normalization after residual connections, thereby enhancing training stability and reducing the risk of gradient vanishing.

[0105] In this embodiment, step S3, the convolutional neural network block includes a first convolutional neural network block and a second convolutional neural network block; features are input into the convolutional neural network block, and convolution and flattening are performed to form convolutional flattened features, including the following method steps:

[0106] Features 2D reshaping converted into reshaping feature map ,in and for The approximate value of the square root;

[0107] The reconstructed feature map is used to form convolutional features through a convolutional neural network, which is represented as follows:

[0108] ;

[0109] Wherein, DWConv represents depthwise convolution, and in this embodiment the kernel sizes are 7×7 and 5×5 respectively; Conv includes pointwise convolution and standard convolution, and in this embodiment the kernel sizes are 1×1 and 3×3 respectively.

[0110] Convolutional features are flattened to form convolutional flattened features. , This indicates the batch size, i.e., the number of digital pathology images. Indicates the number of image patches. Indicates the feature dimension.

[0111] The first convolutional flattening feature provides local spatial information for subsequent interactions with the gated attention mechanism. This step, through multi-scale convolution and residual connections, significantly improves the model's ability to represent spatially heterogeneous data.

[0112] In this embodiment, step S4 involves fusing the deep attention features with the first convolutional flattening features and inputting the fusion into the second convolutional neural network block. After two-dimensional reshaping, convolution, and flattening, the second convolutional flattening features are formed. The deep attention features and the second convolutional flattening features are then fused to form a deep fusion feature. This includes the following method steps:

[0113] The deep attention features are reshaped in two dimensions and then fused with the first convolutional flattening features from the output of the first convolutional neural network block. After processing and flattening by the convolutional block, the second convolutional flattening features are formed, represented as follows:

[0114] ;

[0115] in, This represents the two-dimensional feature map corresponding to the first convolution flattened feature;

[0116] Flatten the features of the second convolution and deep feature representation Layer normalization and channel splicing form deep fusion features. , represented as:

[0117] ;

[0118] in, This indicates the batch size, i.e., the number of digital pathology images. Indicates the number of image patches. The feature dimension is represented. After obtaining the fused feature representation, it is aggregated into a slice-level representation through global average pooling, and further mapped to logits corresponding to the number of classes through the final fully connected layer fc2. Finally, the logits are normalized by the softmax function, outputting the classification probability corresponding to each slice, which is used for the final class prediction. The entire process realizes the transformation from microscopic feature extraction of multiple image patches to aggregation into macroscopic slice-level diagnostic representations, effectively improving the performance of the model in weakly supervised pathological classification tasks.

[0119] As a preferred embodiment, the loss function of the deep learning-based classification model is represented by the Focal loss objective function, which can solve the class imbalance problem and enhance the learning ability for difficult-to-classify samples.

[0120] ;

[0121] in, Represents the total number of samples. Indicates the first Each sample was predicted to be the true label. The probability, Indicate category Weighting factor This represents the focusing parameter. In this embodiment, the focusing parameter is set to 2, which reduces the loss contribution of easily classified samples and focuses on difficult-to-classify samples.

[0122] In this embodiment, the molecular subtyping of breast cancer includes CS1, CS2, and CS3. This subtyping uses proteomics data to classify triple-negative breast cancer into three subtypes. The CS1 subtype shows upregulation of immune-related signaling pathways, while the CS3 subtype is mainly characterized by inhibition of immune signaling pathways. The CS2 subtype has signaling pathway enrichment but does not show immune characteristics. The specific subtyping method is explained in the section on molecular subtyping gene groups and diagnostic products and applications based on cell membrane targets for solid tumors.

[0123] This embodiment provides a molecular subtyping method for breast cancer based on histopathological images. It acquires digital pathological images, extracts features from these images to obtain feature embedding representations, and inputs these representations into a deep learning-based subtyping model to obtain the probability of each molecular subtype. In the deep learning-based subtyping model, an attention mechanism captures long-range dependencies, a convolutional neural network module extracts local spatial structures, and an interaction mechanism further achieves effective fusion of global and local data. In this invention, the dynamic weighting of the attention mechanism combined with the residual learning of the convolutional neural network module effectively improves the model's ability to model complex tissue patterns and demonstrates high efficiency in processing high-dimensional heterogeneous data, thus providing higher efficiency and accuracy in the classification of triple-negative breast cancer molecular subtypes.

[0124] Example 2:

[0125] This embodiment provides a breast cancer molecular subtyping system based on histopathological images, used to implement the breast cancer molecular subtyping method described in Embodiment 1, including:

[0126] The preprocessing module is used to preprocess stained tissue pathology images to form image blocks before feature extraction from digital pathology images;

[0127] A pre-trained visual-linguistic pathology model is used to extract features from digital pathology images and obtain feature embedding representations of digital pathology images.

[0128] A deep learning-based subtyping model is used to input feature embedding representations into the deep learning-based subtyping model; the output of the deep learning-based subtyping model is obtained to obtain the probability of each molecular subtype.

[0129] The pre-trained visual-linguistic pathology model includes an initial fully connected layer, a gated attention mechanism, a convolutional neural network module, a branching interaction module, and a final fully connected layer.

[0130] The initial fully connected layer is used to transform the input feature vector to obtain initial features;

[0131] The gated attention module is used to dynamically weight input features by combining the gate signals of Tanh and Sigmoid activation functions, highlight key image patches related to the target category, and fuse the attention-weighted features with the initial features and the first convolution flattening features to form deep attention features;

[0132] The convolutional neural network block is used to transform the feature input into convolutional flattened features through two-dimensional reshaping, convolution, and flattening.

[0133] The branch interaction module is used to fuse deep attention features with second convolution flattening features to form deep fusion features;

[0134] The final fully connected layer is used to map panoramic slice features to raw prediction scores, and the raw prediction scores are normalized to form a fractal probability output.

[0135] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0136] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A molecular subtyping method for breast cancer based on histopathological images, characterized in that, The following methods and steps are included: Acquire digital pathology images; extract features from digital pathology images to obtain feature embedding representations of digital pathology images; The feature embedding representation is input into a deep learning-based subtyping model; the output of the deep learning-based subtyping model is obtained to get the probability of each molecular subtype; The deep learning-based subtyping model obtains feature embedding representations and outputs the probabilities of each molecular subtype, including the following steps: The input feature vector of the deep learning-based subtyping model is constructed by embedding the features of digital pathological images. The input feature vector is transformed through an initial fully connected layer to obtain the initial features; The initial features are input into the first convolutional neural network block, and after two-dimensional reshaping, convolution and flattening, the first convolutional flattened features are formed. The initial features are fused with the first convolution flattening features. Attention weights are assigned to each image patch of the digital pathology image through attention. Attention weighted features are calculated based on the attention weights. The attention weighted features are then fused with the initial features and the first convolution flattening features to form deep attention features. The deep attention features are fused with the first convolutional flattening features and input into the second convolutional neural network block. After two-dimensional reshaping, convolution and flattening, the second convolutional flattening features are formed. Deep attention features are fused with second convolution flattening features to form deep fusion features; Global average pooling is used to compress the deep fusion features to form panoramic slice features. The panoramic slice features are then passed through a final fully connected layer to map them into original prediction scores. The original prediction scores are then normalized to form the fractal probability output.

2. The method for molecular subtyping of breast cancer based on histopathological images according to claim 1, characterized in that, Before feature extraction from digital pathological images, preprocessing of stained tissue pathological images is also included, comprising the following steps: The Otsu thresholding method was used to extract tissue regions from digital pathology images; The tissue region of the digital pathology image is downsampled at a preset resolution and loaded into memory. The tissue region is divided into non-overlapping squares of a preset size. The minimum threshold of the midpoint value of the preset RGB channels is used to filter out squares with less tissue. The preprocessed squares are defined as image blocks.

3. The method for molecular subtyping of breast cancer based on histopathological images according to claim 1, characterized in that, Feature extraction from digital pathology images includes the following steps: The method for obtaining feature embedding representations of digital pathology images includes the following steps: Load a pre-trained visual language pathology model, set the output dimension of the attention pooling layer to a preset dimension, and provide a feature extractor to generate a feature vector representation of a preset dimension for each image patch; Using multi-instance learning, digital pathology images are treated as packets, and image patches are treated as instances. The resulting feature embedding is represented as follows: ; Where N is the number of image patches in the digital pathology image. Indicates the feature dimension.

4. The method for molecular subtyping of breast cancer based on histopathological images according to claim 1, characterized in that, In deep learning-based fractal models, the input feature vector is transformed through an initial fully connected layer to obtain initial features, including the following steps: A deep learning-based subtyping model is constructed. The input feature vector of the deep learning-based subtyping model is represented by feature embedding of digital pathological images, as follows: ； in, This indicates the batch size, i.e., the number of digital pathology images. Indicates the number of image patches. Indicates feature dimension; The input feature vector passes through an initial fully connected layer. Transformation yields initial features , represented as: ； in, , , Indicates the output dimension.

5. The method for molecular subtyping of breast cancer based on histopathological images according to claim 1, characterized in that, The initial features are fused with the first convolutional flattening features. Attention weights are assigned to each image patch of the digital pathology image through attention. Attention-weighted features are calculated based on the attention weights. The attention-weighted features are then fused with the initial features and the first convolutional flattening features to form deep attention features. The method includes the following steps: The initial features are fused with the first convolution flattened features and input into the gated attention module. By combining the gating signals of the Tanh and Sigmoid activation functions, the input features are dynamically weighted to highlight key image patches related to the target category. The attention-weighted features are fused with the initial features and the first convolution flattened features to form deep attention features. The fusion feature of the initial features and the first convolution flattened features Attention weights are assigned to each image patch using gating attention, as follows: ； in, Indicates the first Attention weights for each image patch, , and These represent the learnable weight parameters, Indicates the first Feature representation of an image patch Indicates the number of pathological image patches. This indicates element-wise multiplication. and It is a non-linear activation function; ； ； in This represents attention-weighted features. This represents the characteristics of deep attention. This indicates the batch size, i.e., the number of digital pathology images. Representing feature dimension, This represents the attention tensor formed by combining the attention weights of multiple image patches.

6. The method for molecular subtyping of breast cancer based on histopathological images according to claim 1, characterized in that, A convolutional neural network (CNN) block includes a first CNN block and a second CNN block. Features are input into the CNN block, and after two-dimensional reshaping, convolution, and flattening, convolutional flattened features are formed. This process includes the following steps: Features 2D reshaping converted into reshaping feature map ,in and for The approximate value of the square root; The reconstructed feature map is used to form convolutional features through a convolutional neural network, which is represented as follows: ； Where DWConv represents depthwise convolution, and Conv includes pointwise convolution and standard convolution; Convolutional features are flattened to form convolutional flattened features. , This indicates the batch size, i.e., the number of digital pathology images. Indicates the number of image patches. Indicates the feature dimension.

7. The method for molecular subtyping of breast cancer based on histopathological images according to claim 1, characterized in that, The deep attention features are fused with the first convolutional flattening features and input into the second convolutional neural network block. After two-dimensional reshaping, convolution and flattening, the second convolutional flattening features are formed. The deep attention features and the second convolutional flattening features are fused to form the deep fusion features. The following methods and steps are included: The deep attention features are reshaped in two dimensions and then fused with the first convolutional flattening features from the output of the first convolutional neural network block. After processing and flattening by the convolutional block, the second convolutional flattening features are formed, represented as follows: ； in, This represents the two-dimensional feature map corresponding to the first convolution flattened feature; Flatten the features of the second convolution and deep feature representation Layer normalization and channel splicing form deep fusion features. , represented as: ; in, This indicates the batch size, i.e., the number of digital pathology images. Indicates the number of image patches. Indicates the feature dimension.

8. The method for molecular subtyping of breast cancer based on histopathological images according to claim 1, characterized in that, The loss function of the deep learning-based typing model is expressed as: ； in, Represents the total number of samples. Indicates the first Each sample was predicted to be the true label. The probability, Indicates category Weighting factor This indicates the focus parameter.

9. A molecular subtyping system for breast cancer based on histopathological images, characterized in that, A method for implementing the molecular subtyping of breast cancer according to any one of claims 1-8, comprising: The preprocessing module is used to preprocess stained tissue pathology images to form image blocks before feature extraction from digital pathology images; A pre-trained visual-linguistic pathology model is used to extract features from digital pathology images and obtain feature embedding representations of digital pathology images. A deep learning-based subtyping model is used to input feature embedding representations into the deep learning-based subtyping model; the output of the deep learning-based subtyping model is obtained to obtain the probability of each molecular subtype.

10. A molecular subtyping system for breast cancer based on histopathological images according to claim 9, characterized in that, The pre-trained visual-linguistic pathology model includes an initial fully connected layer, a gated attention module, a convolutional neural network block, a branching interaction module, and a final fully connected layer. The initial fully connected layer is used to transform the input feature vector to obtain initial features; The gated attention module is used to dynamically weight input features by combining the gate signals of Tanh and Sigmoid activation functions, highlight key image patches related to the target category, and fuse the attention-weighted features with the initial features and the first convolution flattening features to form deep attention features; The convolutional neural network block is used to transform the feature input into convolutional flattened features through two-dimensional reshaping, convolution, and flattening. The branch interaction module is used to fuse deep attention features with second convolution flattening features to form deep fusion features; The final fully connected layer is used to map panoramic slice features to raw prediction scores, and the raw prediction scores are normalized to form a fractal probability output.

Images