Pathological section-based early-stage breast cancer recurrence and metastasis risk prediction method and system

Abstract

The invention provides a pathological section-based early-stage breast cancer recurrence and metastasis risk prediction method and system, and belongs to the technical field of disease recurrence risk prediction. Comprising the following steps: generating a space attention thermodynamic diagram of a full-scanning digital pathological section level by using a risk prediction model; extracting a high-attention region of interest in the space attention thermodynamic diagram, and constructing a cell-level interaction mode in the tumor ecosystem of the patient; performing phenotypic diversity characterization analysis under different recurrence and metastasis risk groups based on the risk prediction model; and obtaining the risk of recurrence and metastasis of the early breast cancer by combining the obtained space attention thermodynamic diagram, the interaction mode and the characterization analysis result. The method not only overcomes the limitation of the existing multi-gene detection method on time and cost, but also makes up the defect that the existing multi-gene detection method cannot reflect spatial characteristics, and can provide a more scientific and reliable recurrence and metastasis risk assessment and prediction tool for early-stage breast cancer patients.

Description

Technical Field

[0001] The present invention belongs to the technical field of disease recurrence risk prediction, and particularly relates to a method and system for predicting the recurrence and metastasis risk of early breast cancer based on pathological sections. Background Art

[0002] The statements in this part only provide background technical information related to the present invention and do not necessarily constitute prior art.

[0003] Breast cancer is one of the most common malignant tumors in women globally, and its recurrence and metastasis are the main causes of patient death. Accurately assessing the recurrence and metastasis risk of early breast cancer patients is of great significance for formulating personalized treatment plans, optimizing treatment strategies, and improving patient prognosis. However, currently, the prediction and assessment of the recurrence and metastasis risk of early breast cancer in clinical practice mainly rely on multi-gene detection data (such as MammaPrint, Oncotype DX, etc.) for patient treatment plan decision-making. Although multi-gene detection methods can provide detailed information on tumor tissues at the molecular level, they have the following significant technical limitations, for example:

[0004] (1) High time cost: The gene sequencing process is complex and time-consuming, making it difficult to meet the clinical need for rapid assessment.

[0005] (2) High economic cost: The detection cost of gene sequencing technology is expensive, restricting its wide application in resource-limited areas or patients.

[0006] (3) Lack of revealing spatial characteristics: Gene sequencing data mainly focuses on molecular expression characteristics and cannot effectively capture key information such as the spatial arrangement and structural heterogeneity of cells in the tumor microenvironment, while these spatial characteristics are of great significance for predicting the recurrence and metastasis risk. Summary of the Invention

[0007] To overcome the deficiencies of the above-mentioned prior art, the present invention provides a method and system for predicting the recurrence and metastasis risk of early breast cancer based on pathological sections, which not only overcomes the limitations of existing multi-gene detection methods in terms of time and cost but also makes up for their deficiency in not being able to reflect spatial characteristics, and can provide a more scientific and reliable recurrence and metastasis risk assessment and prediction tool for early breast cancer patients.

[0008] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:

[0009] The first aspect of the present invention provides a method for predicting the recurrence and metastasis risk of early breast cancer based on pathological sections.

[0010] The method for predicting the recurrence and metastasis risk of early breast cancer based on pathological sections includes:

[0011] Obtain a whole-slide digital pathology section and construct a risk prediction model based on a weakly supervised learning architecture;

[0012] Use the risk prediction model to generate a spatial attention heatmap at the whole-slide digital pathology section level;

[0013] Extract the high-attention regions of interest in the spatial attention heatmap and construct the interaction patterns at the cellular level in the patient's tumor ecosystem within the high-attention regions of interest;

[0014] Conduct phenotypic diversity characterization analysis based on the established risk prediction model for different recurrence and metastasis risk groups; combine the obtained spatial attention heatmap, interaction patterns, and the results of the characterization analysis to obtain the risk of early breast cancer recurrence and metastasis.

[0015] Furthermore, the risk prediction model includes a whole-slide digital pathology section processing and feature representation module, a feature aggregation module, and an observer based on a regression strategy.

[0016] Furthermore, the whole-slide digital pathology section processing and feature representation module is used to perform non-overlapping sliding window sampling and dimensionality-reduced feature representation on the effective foreground region at a specified magnification of the whole-slide digital pathology section to obtain a set of local image patches and represent different morphological patterns.

[0017] Furthermore, the feature aggregation module adopts a Transformer architecture based on a proxy attention mechanism, that is, the Transformer architecture contains self-optimized proxy tokens and is composed of two stacked Softmax self-attention operations.

[0018] Furthermore, generating a spatial attention heatmap using the risk prediction model includes: First, perform data preprocessing on the whole-slide digital pathology section; Subsequently, use a pathology-based large model to perform feature representation on the local image patches to obtain the attention scores of the local image patches; Finally, arrange the obtained attention scores according to their respective spatial coordinate positions to generate a spatial attention heatmap.

[0019] Furthermore, constructing the interaction patterns at the cellular level in the patient's tumor ecosystem includes: First, extract the high-attention regions of interest in the spatial attention heatmap; Subsequently, use a segmentation model to simultaneously segment and classify the cell nuclei; Select the main components of the segmented and classified cells and construct a topological map between the detected cell nuclei, and use the topological features shown in the topological map to characterize the interaction patterns at the cellular level.

[0020] Further, the phenotypic diversity characterization analysis includes: performing characterization analysis on a set of local image patches and predicting the recurrence and metastasis risk score at the level of the whole-slide digital pathology section; simultaneously, selecting a certain number of local image patches with the highest attention scores in the whole-slide digital pathology section as local regions within the tumor; performing analysis and calculation on the selected local regions and performing a clustering operation; and determining and characterizing the phenotypic diversity under different recurrence and metastasis risk groups based on the clustering results.

[0021] The second aspect of the present invention provides a system for predicting the recurrence and metastasis risk of early breast cancer based on a pathological section.

[0022] The system for predicting the recurrence and metastasis risk of early breast cancer based on a pathological section includes:

[0023] A model construction module, configured to: obtain a whole-slide digital pathology section and construct a risk prediction model based on a weakly supervised learning architecture;

[0024] A spatial attention heatmap generation module, configured to: generate a spatial attention heatmap at the whole-slide digital pathology section level by using the risk prediction model;

[0025] A cell-level interaction module, configured to: extract high-attention regions of interest in the spatial attention heatmap and construct an interaction pattern at the cell level in the patient's tumor ecosystem within the high-attention regions of interest;

[0026] A phenotypic diversity characterization analysis module, configured to: perform phenotypic diversity characterization analysis under different recurrence and metastasis risk groups based on the constructed risk prediction model; and combine the obtained spatial attention heatmap, interaction pattern, and characterization analysis results to obtain the risk of recurrence and metastasis of early breast cancer.

[0027] The third aspect of the present invention provides a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the steps in the method for predicting the recurrence and metastasis risk of early breast cancer based on a pathological section as described in the first aspect of the present invention are implemented.

[0028] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored on the memory and executable on the processor, and when the processor executes the program, the steps in the method for predicting the recurrence and metastasis risk of early breast cancer based on a pathological section as described in the first aspect of the present invention are implemented.

[0029] The above one or more technical solutions have the following beneficial effects:

[0030] (1) The present invention uses a risk prediction model to generate a spatial attention heat map at the level of a whole-scanned digital pathological section. The present invention makes good use of the characteristics of fast acquisition speed and automated processing of digital pathological sections, significantly shortening the evaluation cycle and being able to provide timely prediction results for clinicians.

[0031] (2) Compared with gene sequencing, the method for predicting the risk of early breast cancer recurrence and metastasis based on pathological sections in the present invention has greatly reduced the computational analysis cost, effectively improving the accessibility of the solution for patients.

[0032] (3) The present invention uses a risk prediction model to generate a spatial attention heat map at the level of a whole-scanned digital pathological section to determine the spatial distribution of tumor cells; by extracting the high-attention regions of interest in the spatial attention heat map, an interaction pattern at the cell level in the patient's tumor ecosystem is constructed within the high-attention regions of interest to determine the cell interaction pattern. By quantitatively analyzing features such as the spatial distribution of tumor cells and the cell interaction pattern, the present invention can more comprehensively evaluate the recurrence and metastasis risk of patients and provide more valuable information for clinical decision-making.

[0033] Advantages of additional aspects of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation of the present invention.

[0035] Figure 1 It is a flowchart of the method for predicting the risk of early breast cancer recurrence and metastasis based on pathological sections in Embodiment 1 of the present invention.

[0036] Figure 2 It is an overall architecture diagram of the risk prediction model in Embodiment 1 of the present invention.

[0037] Figure 3 It is a result diagram of visualizing the spatial attention heat map of the digital pathological section of TCGA data in Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0039] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention.

[0040] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0041] Overall idea proposed by the present invention: The present invention provides a method for predicting the recurrence and metastasis risk of early breast cancer based on pathological sections, and the method: ① establishes a clinical cohort, which consists of clinicopathological features, whole-slide digital pathological sections stained with H&E, and multi-gene test results used for recurrence and metastasis risk assessment in clinics; ② based on the whole-slide digital pathological sections of patients, using artificial intelligence and pathological section analysis technology, model and learn the recurrence and metastasis risk groups (such as MammaPrint, Oncotype DX, etc.) corresponding to patients based on multi-gene test results, and finally realize the automatic assessment and prediction of the recurrence and metastasis risk of early breast cancer; ③ use the trained model to mine the cohort data, analyze the spatial localization ability of the model at the whole-slide pathological section level for tumor location, analyze the ability of the model to analyze the cell interaction pattern in the patient's tumor ecosystem, and analyze the morphological phenotype characterization ability of the model for different risk groups, and use these analysis and mining results as the revelation of the spatial pattern characteristics of the patient's tumor by the present invention. In summary, in view of the limitations in time and cost of existing multi-gene detection methods (such as MammaPrint, Oncotype DX, etc.) used in clinics for recurrence and metastasis risk assessment, the present invention proposes an artificial intelligence calculation method based on pathological section analysis, realizing the automatic assessment and prediction of the recurrence and metastasis risk of early breast cancer, not only overcoming the limitations of existing multi-gene detection methods in time and cost, but also making up for the deficiency that they cannot reflect spatial characteristics.

[0042] Embodiment 1

[0043] This embodiment discloses a method for predicting the recurrence and metastasis risk of early breast cancer based on pathological sections.

[0044] As Figure 1 shown, the method for predicting the recurrence and metastasis risk of early breast cancer based on pathological sections includes:

[0045] Obtain whole-slide digital pathological sections and construct a risk prediction model based on a weakly supervised learning architecture;

[0046] Use the risk prediction model to generate a spatial attention heat map at the whole-slide digital pathological section level;

[0047] Extract the high-attention regions of interest in the spatial attention heat map and construct an interaction pattern at the cell level in the patient's tumor ecosystem within the high-attention regions of interest;

[0048] Perform phenotypic diversity characterization analysis based on the established risk prediction model for different recurrence and metastasis risk groups; combine the obtained spatial attention heatmaps, interaction patterns, and characterization analysis results to obtain the risk of early breast cancer recurrence and metastasis.

[0049] Based on the above method, the present invention not only overcomes the limitations of existing multi-gene detection methods in terms of time and cost, but also makes up for their deficiency in not being able to reflect spatial characteristics, and can provide a more scientific and reliable recurrence and metastasis risk assessment and prediction tool for early breast cancer patients. For the convenience of understanding the technical solution of the present invention, the following further explains and illustrates the specific implementation manners in the technical solution of the present invention.

[0050] As Figure 2 shown, the present invention constructs a risk prediction model for predicting the recurrence and metastasis risk of early breast cancer. This framework is a weakly supervised multi-instance learning architecture (i.e., a weakly supervised learning architecture), which includes a whole-slide digital pathology slide processing and feature representation module (corresponding to the whole-slide digital pathology slide processing process and the feature representation part), and realizes the recurrence risk assessment and prediction from unlabeled histopathological whole-slide digital pathology slides.

[0051] The risk prediction model proposed by the present invention is developed and trained based on a complete and sample-matched clinical cohort, which includes the multi-gene detection results of patients for recurrence and metastasis risk assessment in the clinical scenario and the paired whole-slide digital pathology slide data of H&E staining to ensure the slice feature representation and the prediction of recurrence and metastasis risk status.

[0052] On this basis, the present invention uses the trained model to mine the cohort data. Specifically:

[0053] 1) Generate spatial attention heatmaps at the whole-slide digital pathology slide level to provide key morphological patterns and spatial localizations related to the recurrence risk.

[0054] 2) Construct cell-level interaction patterns in the tumor ecosystem to deeply analyze the intercellular spatial relationships in different recurrence and metastasis risk groups.

[0055] 3) Study the tumor morphological patterns by clustering the local image patch embeddings learned from the model to characterize tumor diversity and identify specific and co-localized morphological phenotypes in the recurrence and metastasis risk groups of multi-gene detection methods (such as MammaPrint, Oncotype DX, etc.).

[0056] Regarding the above processes in the present invention, the following further explains and illustrates this embodiment:

[0057] A. Risk prediction model based on a weakly supervised learning architecture.

[0058] The risk prediction model includes a full-scan digital pathology slide processing and feature representation module, a feature aggregation module, and an observer based on a regression strategy. Among them, the full-scan digital pathology slide processing and feature representation module is used to perform non-overlapping sliding window sampling and dimensionality reduction feature representation on the effective foreground region at a specified magnification of the full-scan digital pathology slide, so as to obtain a set of local image patches and characterize different morphological patterns. Specifically:

[0059] Histopathological digital slides usually contain billions of pixels and have high-resolution, multi-scale properties. To improve the efficiency of full-scan digital pathology slide analysis, first, non-overlapping sliding window sampling is performed on the effective foreground region at 20 magnifications of the full-scan digital pathology slide to obtain a set of local image patches with a size of 256×256 pixels.

[0060] Then, based on a pre-trained large model in the pathology field (including but not limited to UNI, Prov-GigaPath, Virchow2, H-Optimus-0, Phikon2), low-dimensional feature representations are generated for the image patches to characterize different morphological patterns; among them, the 256×256-sized image patches are scaled and adjusted to 224×224 pixels and input into the feature encoder of the large model for forward calculation. Considering the spatial relationship between tissue regions, the position coordinate information embedding of each image patch is encoded and merged with the low-dimensional feature representation dimension. As an alternative embodiment, a two-dimensional absolute position encoding method can be used to encode the position coordinate information of the image patches, and the generated position embedding is dimensionally concatenated with the original morphological feature representation.

[0061] Next, the feature aggregation module introduces a Transformer architecture for feature aggregation to achieve the global representation ability of the slide; among them, the feature aggregation module adopts a Transformer architecture based on a proxy attention mechanism, that is, the Transformer architecture contains self-optimized proxy tokens and is stacked by two Softmax self-attention operations. Specifically: Considering that the number of local image patches in a single full-scan digital pathology slide can reach the order of one hundred thousand, this embodiment uses an efficient Transformer architecture based on a proxy attention mechanism to improve the computational efficiency of training and inference while capturing long-range dependencies across local regions. Specifically, this embodiment introduces self-optimized proxy tokens into the model architecture Convert the traditional self-attention mechanism into the proxy attention paradigm. The traditional self-attention mechanism can be expressed as follows:

[0062]

[0063] Among them, is the number of local image patches fed into the Transformer architecture in a single full-scan digital pathology slide, and Sim(·) represents the similarity function; it is usually expressed in the Softmax self-attention mechanism as where represents the query encoding feature represents the key encoding feature represents the value encoding feature represents the i-th dimension of the query encoding feature represents the m-th dimension of the key encoding feature represents the m-th dimension of the value encoding feature, and d is the dimension of the encoding feature

[0064] In the efficient Transformer architecture based on the proxy attention mechanism, based on the self-optimized proxy token of the proxy attention mechanism is expressed as which consists of two stacked Softmax self-attention operations, namely:

[0065]

[0066] where is the proxy value obtained from using the self-attention mechanism This process is described as "one-stage aggregation"; on this basis, the value obtained from using the self-attention mechanism again is the output result of the proxy attention mechanism, and this process is described as "two-stage propagation". In this way, the calculation of is and computationally deconstructed. It should be noted that the number of self-optimized proxy tokens is a very small constant, which theoretically reduces the computational complexity of the attention mechanism to d << n; where

[0067] Finally, a regression-based multi-layer perceptron prediction head is adopted to predict continuous values as the recurrence risk at the patient level, so as to capture a distribution consistent with the genomic recurrence metastasis risk score. The multi-layer perceptron prediction head consists of LayerNorm normalization, a linear transformation layer, and a linear projection layer, which converts the global feature embedding into a single-neuron logit output, and then passes through a Sigmoid activation function to generate a continuous risk value between 0 and 1. The mean squared error loss is used as the overall objective function to train and optimize this model.

[0068] Further, as Figure 2 shown, in the risk prediction model: ① The non-linear transformation layer is used to introduce non-linearity into the model, enabling the model to learn complex feature representations. Its structure usually consists of multiple fully connected layers and non-linear activation functions (such as ReLU, Sigmoid, or Tanh), which are used to perform non-linear transformations on the input features and output the features after non-linear transformation; ② The regression-based multi-layer perceptron predictor aims to map the input features to the target output space through multi-layer non-linear transformations, thereby achieving the regression task. It receives the features processed by the non-linear transformation layer and consists of multiple fully connected layers and non-linear activation functions (such as ReLU), which are used to further extract and combine features. Finally, a fully connected layer is used to directly output continuous values to learn the recurrence metastasis risk value.

[0069] Based on the above design, the present invention realizes an automatic prediction method for the recurrence and metastasis risk of early breast cancer directly based on H&E-stained whole-slide digital pathology sections. This method uses artificial intelligence and pathology section analysis technology to model and learn the recurrence and metastasis risk groups corresponding to patients, including those based on multi-gene detection results such as MammaPrint and Oncotype DX, which can significantly reduce the economic burden and time cost resources for patient recurrence and metastasis risk assessment and improve the efficiency of subsequent clinical treatment.

[0070] B. Generating a spatial attention heatmap at the whole-slide digital pathology section level using the risk prediction model.

[0071] Generating a spatial attention heatmap using the risk prediction model includes: First, performing data preprocessing on the whole-slide digital pathology section; Subsequently, using a pathology-based large model to represent the features of local image patches to obtain the attention scores of the local image patches; Finally, arranging the obtained attention scores according to their respective spatial coordinate positions to generate a spatial attention heatmap. Specifically:

[0072] Full-scan digital pathology slide-level attention heatmap visualization is used to explore the spatial localization of tumors. By obtaining the attention scores of local image patches during the inference stage and mapping them to the corresponding positions on the full-scan digital pathology slide. Specifically, first, data preprocessing is performed on the full-scan digital pathology slide to detect tissue regions, and the identified foreground tissue regions are sampled as a set of local image patches with a size of 256×256 pixels, where the overlap rate is 0.1. Subsequently, a pathology-based large model is used to represent the features of the local image patches, and then the extracted local image patch features are fed into the prediction model obtained from the above-mentioned modeling for inference to generate digital pathology slide-level prediction results and local image patch-level attention scores that contribute to the final risk prediction. The local image patch-level attention scores are calculated based on the proxy attention matrix M AAttn (·), that is:

[0073]

[0074] where σ(·) is the abbreviation of the similarity function Sim(·), represents the transpose of the self-optimized proxy token , represents the transpose of the key encoding feature. The attention scores are normalized to the range between 0 and 1 as attention weights. Finally, the normalized attention scores are arranged according to their corresponding spatial coordinate positions in the full-scan digital pathology slide to achieve the visualization of the attention heatmap. This attention heatmap can also be overlaid on the original H&E-stained full-scan digital pathology slide to potentially provide an attention spatial distribution consistent with the tumor region.

[0075] Based on the above design, the present invention realizes the visualization of the spatial attention heatmap of the full-scan digital pathology slide by using the prediction model trained by modeling, as shown in Figure 3 taking the TCGA data as an example. This shows that the prediction model proposed by the present invention can effectively reveal the tumor spatial characteristics of patients in the recurrence and metastasis risk groups detected by multi-gene tests such as MammaPrint, making up for the deficiency that the existing clinical protocols cannot reflect the tumor spatial characteristics.

[0076] C. Extract the high-attention regions of interest in the spatial attention heatmap and construct the cell-level interaction patterns in the patient's tumor ecosystem within the high-attention regions of interest.

[0077] Constructing the cellular-level interaction pattern in the patient's tumor ecosystem includes: first, extracting the high-attention regions of interest in the spatial attention heat map; then, using the segmentation model to simultaneously segment and classify the cell nuclei; selecting the main components of the segmented and classified cells and constructing a topological map between the detected cell nuclei, and using the topological features shown in the topological map to characterize the cellular-level interaction pattern, specifically:

[0078] In order to further understand the cellular interactions in the tumor ecosystem of different recurrence and metastasis risk groups (such as the results of multi-gene detection assessments such as MammaPrint and Oncotype DX), this embodiment first extracts the high-attention interest regions represented by the spatial attention heat map at the level of the above-mentioned full-scan digital pathology slice. Subsequently, the pre-trained segmentation model is applied to simultaneously segment and classify the cell nuclei. The identified cell nuclei are classified into one of five cell categories, namely: tumor, inflammation, necrosis, connective (stroma) and non-tumor epithelial cells. On this basis, the density of tumor cells (TcD) is defined as the number of tumor cells identified per square millimeter. Among them, the segmentation model refers to a deep learning model for cell nucleus segmentation. As an optional embodiment, the segmentation model uses HoverNet. HoverNet is a model specially designed for cell nucleus segmentation and classification in pathological images, which can simultaneously realize instance segmentation of cell nuclei and classification of cell nucleus types; its core task is to accurately separate the boundaries of each cell nucleus from the pathological image (segmentation) and identify the type of each cell nucleus (classification). HoverNet simultaneously realizes instance segmentation and classification of cell nuclei through a multi-task learning framework. It introduces a horizontal gradient map to solve the problem of adhesion between cell nuclei. The horizontal gradient map helps the model distinguish the boundaries of adjacent cell nuclei by predicting the horizontal and vertical distances from each pixel to the center of the cell nucleus to which it belongs. In the inference stage, the horizontal gradient map can accurately separate the adhered cell nuclei. At the same time as the segmentation, each cell nucleus is classified by the cell nucleus classification map. In the inference stage, the instance segmentation map and the classification map are combined to output the category of each cell nucleus.

[0079] To further analyze the tumor ecosystem, this embodiment selects three main cellular components (tumor, inflammatory and stromal cells) to construct a topological map between detected cell nuclei. This embodiment focuses on the topological features of tumor cells interacting with other cells, especially examining the connectivity graphs of tumor cell-tumor cell, tumor cell-inflammatory cell and tumor cell-stromal cell interactions, and calculates the topological features associated with the edge length of the graph to characterize the spatial intercellular relationships between different cell types in the tumor. This indicator represents the average edge length of each tumor cell interacting with other tumor, matrix and inflammatory cells. Based on this, this embodiment parses the interaction pattern between different cells in the tumor ecosystem.

[0080] Based on the above design, the prediction model trained by the present invention through modeling can analyze the cell interaction patterns within the tumor ecosystem for different recurrence and metastasis risk groups, and characterize the morphological phenotype diversity under different risk groups. It further reflects the morphological relationship of early breast cancer recurrence and metastasis events at the cellular / regional level; establishes the association between the morphological phenotype of digital pathological sections and the prognosis assessment of patients' recurrence and metastasis risks, and realizes multi-omics modeling and mining; provides more possibilities for precision medicine based on multi-omics and multi-modal bioinformatics tumor analysis.

[0081] D. Conduct phenotypic diversity characterization analysis based on the established risk prediction model for different recurrence and metastasis risk groups.

[0082] Conduct phenotypic diversity characterization analysis based on the established risk prediction model for different recurrence and metastasis risk groups, including: performing characterization analysis on the set of local image patches, and predicting the recurrence and metastasis risk scores at the level of the whole-slide digital pathological section; meanwhile, selecting a certain number of local image patches with the highest attention scores in the whole-slide digital pathological section as local regions within the tumor; analyzing and calculating the selected local regions and performing clustering operations; determining and characterizing the phenotypic diversity under different recurrence and metastasis risk groups based on the clustering results. Specifically:

[0083] To study the morphological phenotypes that may be related to different recurrence and metastasis risk groups, in this embodiment, the established prediction model is used to represent the features of a set of local image patches with a size of 256×256 pixels sampled from the whole-slide digital pathological section. These local image patch features are then input into the prediction model for inference and prediction to generate the recurrence and metastasis risk scores at the level of the whole-slide digital pathological section. In addition, this embodiment identifies the top 100 local image patches with the highest attention scores from the digital pathological section, and these patches are considered to represent the local regions within the tumor. Each instance in this set of local image patches is also input into the prediction model for risk score inference at the local image patch level. For each local image patch, the feature vector after feature aggregation through the Transformer architecture is saved as the embedding vector of each local image patch instance.

[0084] The present invention further analyzes the embedded data. All image patch embeddings are merged into a data structure object, which contains the original data matrix of the local image patch embeddings and additional metadata. The metadata consists of the spatial coordinates of the local image patches in the corresponding digital pathological section, the recurrence and metastasis risk group results of the corresponding digital pathological section, the risk scores at the digital pathological section level, and the image patch-level risk scores. Based on this, this embodiment calculates and obtains the nearest neighbor distance matrix and the neighborhood graph of the local image patch embeddings, and further realizes local instance point clustering and community detection. Finally, the results are dimensionally reduced and the image patch embeddings and their sub-clusters in the two-dimensional space are visualized.

[0085] Based on the clustering results of local instance points, subpopulations representing local morphological patterns were identified and further analyzed. First, instance points under different recurrence and metastasis risk groups were extracted from the global subpopulations according to the recurrence and metastasis risk groups of the digital pathology sections corresponding to the local image patches, and only the subpopulations with the number of instance points exceeding 25% of the number of instance points in the corresponding global subpopulations in the risk groups were retained. The coexistence of subpopulations under different recurrence and metastasis risk groups was further determined to identify the intersection overlapping subpopulations, denoted as co-localized subpopulations. The subpopulations unique to their respective risk groups were labeled as group-specific subpopulations. Based on this, the phenotypic diversity under different risk groups was determined and characterized in this embodiment.

[0086] Example Two

[0087] This embodiment discloses a recurrence and metastasis risk prediction system for early breast cancer based on pathological sections.

[0088] A recurrence and metastasis risk prediction system for early breast cancer based on pathological sections includes:

[0089] A model construction module, configured to: obtain a whole-scan digital pathology section and construct a risk prediction model based on a weakly supervised learning architecture;

[0090] A spatial attention heatmap generation module, configured to: generate a spatial attention heatmap at the whole-scan digital pathology section level by using the risk prediction model;

[0091] A cell-level interaction module, configured to: extract high-attention regions of interest in the spatial attention heatmap and construct an interaction pattern at the cell level in the patient's tumor ecosystem within the high-attention regions of interest;

[0092] A phenotypic diversity characterization analysis module, configured to: perform phenotypic diversity characterization analysis under different recurrence and metastasis risk groups based on the established risk prediction model; combine the obtained spatial attention heatmap, interaction pattern, and characterization analysis results to obtain the risk of recurrence and metastasis of early breast cancer.

[0093] Example Three

[0094] The purpose of this embodiment is to provide a computer-readable storage medium.

[0095] A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps in the method for predicting the recurrence and metastasis risk of early breast cancer based on pathological sections as described in Embodiment One of the present disclosure.

[0096] Example Four

[0097] The purpose of this embodiment is to provide an electronic device.

[0098] An electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for predicting the risk of early breast cancer recurrence and metastasis based on pathological sections as described in Embodiment 1 of the present disclosure are implemented.

[0099] In the devices of Embodiments 2, 3, and 4 above, the steps involved correspond to those in Method Embodiment 1. For specific implementation manners, reference may be made to the relevant description part of Embodiment 1. The term "computer-readable storage medium" should be understood to include a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and cause the processor to execute any method in the present invention.

[0100] Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0101] Although the specific implementation manners of the present invention have been described above in conjunction with the accompanying drawings, this is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made without creative efforts by those skilled in the art are still within the protection scope of the present invention.

Claims

1. A method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections, characterized in that: include: Obtain full-scan digital pathology slides and build a risk prediction model based on a weakly supervised learning architecture; Generate spatial attention heatmaps at the full-scan digital pathology slice level using risk prediction models; Extracting a high-attention region of interest in the spatial attention heat map, and constructing a cellular-level interaction pattern in the patient's tumor ecosystem within the high-attention region of interest; Based on the established risk prediction model, phenotypic diversity characterization analysis was performed under different recurrence and metastasis risk groups; the risk of recurrence and metastasis of early breast cancer was obtained by combining the obtained spatial attention heat map, interaction pattern and characterization analysis results.

2. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 1, characterized in that: The risk prediction model includes a full-scan digital pathology slice processing and feature representation module, a feature aggregation module and an observer based on a regression strategy.

3. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 2, characterized in that: The full scan digital pathology slice processing and feature representation module is used to perform non-overlapping sliding window sampling and dimensionality reduction feature representation on the effective foreground area at a specified magnification of the full scan digital pathology slice to obtain a local image block set and characterize different morphological patterns.

4. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 2, characterized in that: The feature aggregation module adopts a Transformer architecture based on a proxy attention mechanism, that is, the Transformer architecture contains a self-optimizing proxy token and is composed of two stacked Softmax self-attention operations.

5. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 1, characterized in that: The spatial attention heat map is generated by using the risk prediction model, including: first, data preprocessing of the full-scan digital pathology slices; then, the local image blocks are feature represented by using the pathology-based large model to obtain the attention scores of the local image blocks; finally, the obtained attention scores are arranged according to their respective spatial coordinate positions to generate a spatial attention heat map.

6. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 1, characterized in that: Constructing the cellular-level interaction pattern in the patient's tumor ecosystem includes: first, extracting the high-attention regions of interest in the spatial attention heat map; then, using a segmentation model to simultaneously segment and classify the cell nuclei; selecting the main components of the segmented and classified cells and constructing a topological map between the detected cell nuclei, and using the topological features shown in the topological map to characterize the cellular-level interaction pattern.

7. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 1, characterized in that: The phenotypic diversity characterization analysis includes: characterizing and analyzing a set of local image blocks, and predicting the recurrence and metastasis risk score at the level of the full-scan digital pathology section; at the same time, selecting a certain number of local image blocks with the highest attention scores in the full-scan digital pathology section as local areas within the tumor; analyzing and calculating the selected local areas and performing clustering operations; and determining and characterizing the phenotypic diversity under different recurrence and metastasis risk groups based on the clustering results.

8. An early breast cancer recurrence and metastasis risk prediction system based on pathological sections, characterized in that: include: The model building module is configured to: acquire full-scan digital pathology slices and build a risk prediction model based on a weakly supervised learning architecture; The spatial attention heat map generation module is configured to: generate a spatial attention heat map at the level of a full scan digital pathology slice using a risk prediction model; The cell-level interaction module is configured to: extract a high-attention region of interest in the spatial attention heat map, and construct a cell-level interaction pattern in the patient's tumor ecosystem within the high-attention region of interest; The phenotypic diversity characterization and analysis module is configured as follows: based on the established risk prediction model, phenotypic diversity characterization and analysis is performed under different recurrence and metastasis risk groups; the risk of recurrence and metastasis of early breast cancer is derived by combining the obtained spatial attention heat map, interaction pattern and characterization analysis results.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections are implemented as described in any one of claims 1 to 7.

Images