Method and system for predicting risk of recurrence and metastasis of early breast cancer based on pathological sections

Abstract

The application provides an early breast cancer recurrence and metastasis risk prediction method and system based on pathological sections, and belongs to the technical field of disease recurrence risk prediction; the method comprises the following steps: generating a full-scan digital pathological section level spatial attention heat map by using a risk prediction model; extracting a high attention region of interest in the spatial attention heat map and constructing a cell level interaction mode in a patient tumor ecosystem; performing phenotype diversity representation analysis under different recurrence and metastasis risk groups based on the risk prediction model; and combining the obtained spatial attention heat map, the interaction mode and the representation analysis result to obtain the recurrence and metastasis risk of early breast cancer. The application not only overcomes the time and cost limitations of the existing multi-gene detection method, but also makes up for the deficiency that the existing multi-gene detection method cannot reflect spatial characteristics, and can provide a more scientific and reliable recurrence and metastasis risk evaluation and prediction tool for early breast cancer patients.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of disease recurrence risk prediction, and particularly relates to a pathological section-based early breast cancer recurrence metastasis risk prediction method and system. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] Breast cancer is one of the most common malignant tumors in women worldwide, and its recurrence and metastasis are the main causes of patient death. Accurate assessment of the recurrence and metastasis risk of early breast cancer patients is of great significance for developing personalized treatment plans, optimizing treatment strategies, and improving patient prognosis. However, current clinical assessment of the recurrence and metastasis risk of early breast cancer mainly relies on multi-gene detection data (such as MammaPrint, Oncotype DX, etc.) for patient treatment plan decision-making. Although multi-gene detection methods can provide fine information of tumor tissue at the molecular level, they have the following significant technical limitations, such as:

[0004] (1) High time cost: The gene sequencing process is complex and time-consuming, making it difficult to meet the demand for rapid assessment in clinical practice.

[0005] (2) High economic cost: The detection cost of gene sequencing technology is expensive, limiting its widespread application in resource-limited areas or among patients.

[0006] (3) Lack of spatial characteristic revelation: Gene sequencing data mainly focuses on molecular expression characteristics and cannot effectively capture key information such as spatial arrangement and structural heterogeneity of cells in the tumor microenvironment, which are important for predicting recurrence and metastasis risk. SUMMARY

[0007] To overcome the shortcomings of the prior art, the present application provides a pathological section-based early breast cancer recurrence metastasis risk prediction method and system, which not only overcomes the limitations of existing multi-gene detection methods in terms of time and cost, but also makes up for the lack of spatial characteristics, providing a more scientific and reliable tool for early breast cancer recurrence and metastasis risk assessment and prediction.

[0008] To achieve the above-mentioned purpose, one or more embodiments of the present application provide the following technical solutions:

[0009] The present application provides a pathological section-based early breast cancer recurrence metastasis risk prediction method.

[0010] The pathological section-based early breast cancer recurrence metastasis risk prediction method comprises:

[0011] Obtaining full-scan digital pathology slides, constructing a risk prediction model based on a weakly supervised learning architecture;

[0012] Generating a spatial attention heat map at the level of full-scan digital pathology slides using the risk prediction model;

[0013] Extracting high-attention regions of interest in the spatial attention heat map, and constructing an interaction pattern at the cell level in the patient's tumor ecosystem within the high-attention regions of interest;

[0014] Performing phenotype diversity characterization analysis under different recurrence and metastasis risk groups based on the constructed risk prediction model; combining the obtained spatial attention heat map, interaction pattern, and characterization analysis results to obtain the risk of recurrence and metastasis of early breast cancer.

[0015] Further, the risk prediction model includes a full-scan digital pathology slide processing and feature representation module, a feature aggregation module, and an observer based on a regression strategy.

[0016] Further, the full-scan digital pathology slide processing and feature representation module is used to perform non-overlapping sliding window sampling and dimensionality reduction feature representation on the effective foreground region at a specified magnification of the full-scan digital pathology slide to obtain a set of local image blocks and represent different morphological patterns.

[0017] Further, the feature aggregation module uses a Transformer architecture based on a proxy attention mechanism, i.e., the Transformer architecture contains self-optimizing proxy tokens and is stacked by two Softmax self-attention operations.

[0018] Further, generating a spatial attention heat map using a risk prediction model includes: first, data preprocessing of full-scan digital pathology slides; then, using a pathology-based large model to represent features of local image blocks to obtain attention scores of local image blocks; finally, arranging the obtained attention scores according to their respective spatial coordinate positions to generate a spatial attention heat map.

[0019] Further, constructing an interaction pattern at the cell level in the patient's tumor ecosystem includes: first, extracting high-attention regions of interest in the spatial attention heat map; then, using a model to simultaneously segment and classify cell nuclei; selecting the segmented and classified cell components and constructing a topological graph between the detected cell nuclei, and using the topological features shown in the topological graph to represent the interaction pattern at the cell level.

[0020] Further, the phenotype diversity characterization analysis comprises: performing characterization analysis on the local image block set, and predicting the recurrence and metastasis risk score at the level of the full-scan digital pathology section; meanwhile, selecting a certain number of local image blocks with the highest attention score in the full-scan digital pathology section as local regions within the tumor; performing analysis and calculation on the selected local regions and executing clustering operation; and determining and characterizing the phenotype diversity under different recurrence and metastasis risk groups based on the clustering results.

[0021] The second aspect of the present application provides a recurrence and metastasis risk prediction system for early breast cancer based on pathology sections.

[0022] The recurrence and metastasis risk prediction system for early breast cancer based on pathology sections comprises:

[0023] The model construction module is configured to: acquire a full-scan digital pathology section, and construct a risk prediction model based on a weakly supervised learning architecture.

[0024] The spatial attention heat map generation module is configured to: generate a spatial attention heat map at the level of the full-scan digital pathology section by using the risk prediction model.

[0025] The cell-level interaction module is configured to: extract a high-attention region of interest in the spatial attention heat map, and construct an interaction mode at the cell level in the tumor ecosystem of the patient in the high-attention region of interest.

[0026] The phenotype diversity characterization analysis module is configured to: perform phenotype diversity characterization analysis under different recurrence and metastasis risk groups based on the constructed risk prediction model; and obtain the risk of recurrence and metastasis of early breast cancer by combining the obtained spatial attention heat map, the interaction mode, and the characterization analysis result.

[0027] The third aspect of the present application provides a computer-readable storage medium having a program stored thereon, wherein the program is executed by a processor to implement the steps in the recurrence and metastasis risk prediction method for early breast cancer based on pathology sections according to the first aspect of the present application.

[0028] The fourth aspect of the present application provides an electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the recurrence and metastasis risk prediction method for early breast cancer based on pathology sections according to the first aspect of the present application.

[0029] The above one or more technical solutions have the following beneficial effects:

[0030] (1) The present application generates a full-scan digital pathology slide level spatial attention heat map using a risk prediction model. The present application makes good use of the characteristics of digital pathology slides, such as fast acquisition and automatic processing, significantly shortens the evaluation period, and can provide timely prediction results for clinicians.

[0031] (2) Compared with gene sequencing, the method for predicting the risk of early breast cancer recurrence and metastasis based on pathological sections in the present application has greatly reduced the cost of calculation and analysis, effectively improving the accessibility of the scheme to patients.

[0032] (3) The present application generates a full-scan digital pathology slide level spatial attention heat map using a risk prediction model to determine the spatial distribution of tumor cells; by extracting the high attention region of interest in the spatial attention heat map, the cell level interaction mode in the patient's tumor ecosystem is constructed in the high attention region of interest to determine the cell interaction mode. The present application can more comprehensively evaluate the patient's risk of recurrence and metastasis by quantitatively analyzing the spatial distribution of tumor cells, cell interaction mode and other characteristics, and provide more valuable information for clinical decision-making.

[0033] The advantages of the additional aspects of the present application will be partially given in the following description, partially will become apparent from the following description, or will be understood by the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0034] The accompanying drawings, which form a part of the present application, are used to provide further understanding of the present application, and the illustrative embodiments of the present application and their description are used to explain the present application, and do not constitute improper limitations on the present application.

[0035] Figure 1 The flowchart of the method for predicting the risk of early breast cancer recurrence and metastasis based on pathological sections in the present application embodiment one.

[0036] Figure 2 The overall architecture diagram of the risk prediction model in the present application embodiment one.

[0037] Figure 3 The result diagram of the spatial attention heat map visualization of the digital pathology section of the TCGA data in the present application embodiment one. DETAILED DESCRIPTION

[0038] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise indicated, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0039] It should be noted that the terms used herein are only for the purpose of describing the specific embodiments, and are not intended to limit the exemplary embodiments according to the present application.

[0040] In the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.

[0041] The overall idea of the present application is: the present application provides an early breast cancer recurrence metastasis risk prediction method based on pathological sections, which comprises the following steps: ① a clinical cohort is established, which is composed of clinical pathological characteristics, H&E stained full scan digital pathological sections and multi-gene detection results for recurrence metastasis risk assessment in clinical; ② based on the full scan digital pathological sections of the patients, artificial intelligence and pathological section analysis technology are used to model and learn the recurrence metastasis risk group (such as MammaPrint, Oncotype DX, etc.) corresponding to the patients based on the multi-gene detection results, and finally realize the automatic evaluation and prediction of the recurrence metastasis risk of early breast cancer; ③ the trained model is used to mine the cohort data, analyze the spatial positioning ability of the model to the tumor position at the full scan pathological section level, analyze the cell interaction mode analysis ability of the model in the tumor ecosystem of the patient, and analyze the morphological phenotype description ability of the model to different risk groups, and the mining results are used as the revelation of the spatial mode characteristics of the tumor of the patient. In summary, the present application proposes an artificial intelligence computing method based on pathological section analysis to overcome the time and cost limitations of the existing multi-gene detection method (such as MammaPrint, Oncotype DX, etc.) for recurrence metastasis risk assessment in clinical, realizes the automatic evaluation and prediction of the recurrence metastasis risk of early breast cancer, not only overcomes the time and cost limitations of the existing multi-gene detection method, but also makes up for the deficiency that it cannot reflect the spatial characteristics.

[0042] Embodiment one

[0043] The present embodiment discloses an early breast cancer recurrence metastasis risk prediction method based on pathological sections.

[0044] As shown in Figure 1 The early breast cancer recurrence metastasis risk prediction method based on pathological sections comprises the following steps:

[0045] Obtaining a full scan digital pathological section, and constructing a risk prediction model based on a weakly supervised learning architecture;

[0046] Generating a spatial attention heat map at the level of the full scan digital pathological section by using the risk prediction model;

[0047] Extracting a high attention region of interest in the spatial attention heat map, and constructing an interaction mode at the cell level in the tumor ecosystem of the patient in the high attention region of interest;

[0048] Based on the built risk prediction model, phenotype diversity characterization analysis is carried out in different recurrence and metastasis risk groups; combined with the obtained spatial attention heat map, interaction mode and characterization analysis result, the risk of recurrence and metastasis of early breast cancer is obtained.

[0049] Based on the above method, the present application not only overcomes the limitations of existing multi-gene detection methods in time and cost, but also makes up for the lack of spatial characteristics, and can provide a more scientific and reliable recurrence and metastasis risk assessment and prediction tool for early breast cancer patients. In order to facilitate the understanding of the technical scheme of the present application, the specific embodiments in the technical scheme of the present application are further explained and described as follows.

[0050] As shown in Figure 2 The present application constructs a risk prediction model for predicting the recurrence and metastasis risk of early breast cancer. The framework is a weakly supervised multi-instance learning architecture (i.e. weakly supervised learning architecture), which includes a full scan digital pathology section processing and feature representation module (corresponding to the full scan digital pathology section processing process and feature representation part), and realizes the recurrence risk assessment and prediction from unannotated histopathology full scan digital pathology section.

[0051] The risk prediction model proposed by the present application is developed and trained based on complete and sample-matched clinical cohort, which contains multi-gene detection results of patients in clinical scenarios for recurrence and metastasis risk assessment and matched H&E stained full scan digital pathology section data, to ensure section feature representation and recurrence and metastasis risk state prediction.

[0052] On this basis, the present application uses the model trained by modeling to mine the cohort data, specifically:

[0053] 1) Generate a spatial attention heat map at the level of full scan digital pathology section to provide key morphological patterns and spatial positioning related to recurrence risk;

[0054] 2) Construct an interaction mode at the cell level in the tumor ecosystem to deeply analyze the spatial relationship between cells in different recurrence and metastasis risk groups;

[0055] 3) Study tumor morphological patterns by clustering local image block embeddings learned from the model to represent tumor diversity and identify specific and co-localized morphological phenotypes in the recurrence and metastasis risk groups of multi-gene detection methods (such as MammaPrint, Oncotype DX, etc.).

[0056] Regarding the above processes in the present application, the present embodiment is further explained and described as follows:

[0057] A, risk prediction model based on weakly supervised learning architecture.

[0058] The risk prediction model comprises a full-scan digital pathology slice processing and feature representation module, a feature aggregation module, and an observer based on a regression strategy. The full-scan digital pathology slice processing and feature representation module is used to perform non-overlapping sliding window sampling and dimension reduction feature representation on the effective foreground area at a specified magnification of the full-scan digital pathology slice, to obtain a set of local image blocks and represent different morphological patterns. Specifically,

[0059] The digital pathology slice usually contains tens of billions of pixels and has high resolution and multi-scale properties. To improve the efficiency of full-scan digital pathology slice analysis, the effective foreground area is first sampled by non-overlapping sliding windows at a magnification of 20 times of the full-scan digital pathology slice to obtain a set of local image blocks with a size of 256x256 pixels.

[0060] Then, a pre-trained basic large model in the pathology field (including but not limited to UNI, Prov-GigaPath, Virchow2, H-Optimus-0, and Phikon2) is used to generate low-dimensional feature representation for the image blocks to represent different morphological patterns. The 256x256 size image block is scaled to 224x224 pixels in size and input to the feature encoder of the basic large model for forward calculation. Given the spatial relationship between tissue regions, the position coordinate information of each image block is encoded and merged with the low-dimensional feature representation dimension. As an optional embodiment, the two-dimensional absolute position encoding method can be used to encode the position coordinate information of the image block, and the generated position embedding is dimensionally spliced with the original morphological feature representation.

[0061] Next, the feature aggregation module introduces a Transformer architecture for feature aggregation to realize the global representation capability of the slice. The feature aggregation module uses a Transformer architecture based on a proxy attention mechanism, that is, the Transformer architecture contains self-optimized proxy tokens, and is stacked by two Softmax self-attention operations. Specifically, considering that the number of local image blocks in a single full-scan digital pathology slice can reach tens of thousands, the embodiment uses an efficient Transformer architecture based on a proxy attention mechanism to improve the computational efficiency of training and inference while capturing long-distance dependencies across local image blocks. Specifically, the embodiment introduces self-optimized proxy tokens in the model architecture to convert the traditional self-attention mechanism into a proxy attention paradigm. The traditional self-attention mechanism can be expressed as follows:

[0062]

[0063] wherein, is the number of local image patches sent into the Transformer architecture in a single whole-slide digital pathology slide, Sim(·) denotes the similarity function; usually denoted as where, denotes the query query encoding feature, denotes the key key encoding feature, denotes the value value encoding feature, denotes the i-th dimension of the query encoding feature, denotes the m-th dimension of the key encoding feature, denotes the m-th dimension of the value encoding feature, d is the dimension of the encoding feature.

[0064] In the efficient Transformer architecture based on the proxy attention mechanism, the self-optimized proxy token is denoted as The proxy attention mechanism of is composed of two Softmax self-attention operations, namely:

[0065]

[0066] where, is the proxy value obtained from using the self-attention mechanism This process is described as "one-stage aggregation"; on this basis, the value obtained from using the self-attention mechanism again is the output result of the proxy attention mechanism, and this process is described as "two-stage propagation". In this way, The calculation of and is deconstructed and represented. It is worth noting that the number of self-optimized proxy tokens is a very small constant, which theoretically reduces the computational complexity of the attention mechanism to d << n; where, denotes the computational complexity, d denotes the dimension of the self-optimized proxy token feature, and n denotes the number of local image patches sent into the Transformer architecture in a single whole-slide digital pathology slide.

[0067] Finally, a regression-based multilayer perceptron prediction head was employed to predict continuous numerical values ​​as patient-level recurrence risk, capturing a distribution consistent with the genomic recurrence and metastasis risk score. The multilayer perceptron prediction head, consisting of LayerNorm normalization, a linear transformation layer, and a linear projection layer, converts the global feature embedding into a single-neuron logit output. This output is then passed through a sigmoid activation function to generate continuous risk values ​​between 0 and 1. The model is trained and optimized using mean squared error loss as the overall objective function.

[0068] Furthermore, such as Figure 2 As shown, in the risk prediction model: ① The nonlinear transformation layer is used to introduce nonlinearity into the model, enabling it to learn complex feature representations. Its structure typically consists of multiple fully connected layers and nonlinear activation functions (such as ReLU, Sigmoid, or Tanh) to perform nonlinear transformations on the input features and output the transformed features; ② The regression-based multilayer perceptron predictor aims to map the input features to the target output space through multiple nonlinear transformations, thereby achieving the regression task. It receives features processed by the nonlinear transformation layer and consists of multiple fully connected layers and nonlinear activation functions (such as ReLU) for further feature extraction and combination. Finally, it uses a fully connected layer to directly output continuous values ​​to learn relapse and metastasis risk values.

[0069] Based on the above design, this invention realizes an automatic prediction method for the risk of recurrence and metastasis in early breast cancer directly based on H&E-stained full-scan digital pathological sections. This method utilizes artificial intelligence and pathological section analysis technology to model and learn the patient's corresponding recurrence and metastasis risk groups, including MammaPrint and Oncotype DX based on multi-gene testing results. This can significantly reduce the economic burden and time cost of assessing the patient's recurrence and metastasis risk, and improve the efficiency of subsequent clinical treatment.

[0070] B. Use a risk prediction model to generate spatial attention heatmaps at the level of full-scan digital pathology slides.

[0071] The spatial attention heatmap is generated using a risk prediction model, including: first, data preprocessing of full-scan digital pathological slides; then, feature representation of local image patches using a large-scale pathological model to obtain attention scores for each patch; finally, arranging the obtained attention scores according to their spatial coordinates to generate a spatial attention heatmap. Specifically:

[0072] The full-scan digital pathology slide-level attention heat map visualization is used to explore the spatial localization of the tumor. The attention scores of the local image patches are obtained in the inference stage and mapped to the corresponding positions of the full-scan digital pathology slide. Specifically, first, the full-scan digital pathology slide is preprocessed to detect the tissue region, and the identified foreground tissue region is sampled as a set of local image patches with a size of 256x256 pixels, where the overlap rate is 0.1. Subsequently, the local image patches are represented by features using the pathology-based large model, and then the extracted local image patch features are input into the prediction model obtained by the above modeling to perform inference to generate the prediction results at the digital pathology slide level and the local image patch-level attention scores that contribute to the final risk prediction. The local image patch-level attention scores are based on the proxy attention matrix M AAttn (·) is calculated, that is,

[0073]

[0074] where σ(·) is the abbreviation of the similarity function Sim(·), denotes the transpose of the self-optimized proxy token , and denotes the transpose of the key-encoding feature. The attention scores are normalized to a range of 0 to 1 as attention weights. Finally, the normalized attention scores are arranged according to their corresponding spatial coordinate positions in the full-scan digital pathology slide to realize the visualization of the attention heat map. The attention heat map can also be overlaid on the original H&E stained full-scan digital pathology slide to potentially provide an attention spatial distribution consistent with the tumor region.

[0075] Based on the above design, the prediction model trained by modeling is used to realize the spatial attention heat map visualization of the full-scan digital pathology slide, as shown in FIG. 8, which is an example of using TCGA data. This shows that the prediction model proposed in the present application can effectively reveal the tumor spatial characteristics of the relapse metastasis risk group patients of MammaPrint and other multi-gene detection, which makes up for the deficiency that the existing clinical scheme cannot reflect the tumor spatial characteristics. Figure 3

[0076] C, extracting a high-attention region of interest in the spatial attention heat map, and constructing a cell-level interaction pattern in the patient tumor ecosystem in the high-attention region of interest.

[0077] ​Constructing the interaction pattern at the cell level in the tumor ecosystem of a patient, including: first, extracting high attention regions of interest in the spatial attention heat map; then, using a segmentation model to simultaneously segment and classify the cell nuclei; selecting the main components of the segmented and classified cells and constructing a topological graph between the detected cell nuclei, using the topological features shown in the topological graph to represent the interaction pattern at the cell level, in particular:

[0078] To further understand the cell interactions in the tumor ecosystem of different recurrence and metastasis risk groups (such as the results of multi-gene detection evaluation such as MammaPrint, Oncotype DX, etc.), the present embodiment first extracts the high attention regions of interest shown in the spatial attention heat map of the full scan digital pathology section level. Then, a pre-trained segmentation model is applied to simultaneously segment and classify the cell nuclei. The identified cell nuclei are classified into one of the five cell categories, i.e., tumor, inflammation, necrosis, connective (stromal), and non-tumor epithelial cells. On this basis, the tumor cell density (TcD) is defined as the number of identified tumor cells per square millimeter. The segmentation model refers to a deep learning model for cell nucleus segmentation. As an optional embodiment, the segmentation model uses HoverNet. HoverNet is a model specifically designed for cell nucleus segmentation and classification in pathology images, which can simultaneously achieve instance segmentation and cell nucleus type classification of cell nuclei; its core task is to accurately separate the boundary of each cell nucleus (segmentation) and identify the type of each cell nucleus (classification) from the pathology image. HoverNet achieves instance segmentation and classification of cell nuclei through a multi-task learning framework. It introduces a horizontal gradient map to solve the problem of cell nucleus adhesion. The horizontal gradient map predicts the horizontal and vertical distance from each pixel point to the center of the cell nucleus it belongs to, helping the model to distinguish the boundaries of adjacent cell nuclei. In the inference stage, the horizontal gradient map can accurately separate the adherent cell nuclei. At the same time of segmentation, each cell nucleus is classified through the cell nucleus classification map. In the inference stage, combining the instance segmentation map and the classification map, the class of each cell nucleus is output.

[0079] To further analyze the tumor ecosystem, the present embodiment selects three main cell components (tumor, inflammation, and stromal cells) to construct a topological graph between the detected cell nuclei. The present embodiment focuses on the topological features of tumor cell interactions with other cells, and specifically examines the connectivity graph of tumor cell-tumor cell, tumor cell-inflammation cell, and tumor cell-stromal cell interactions, and calculates the topological features related to the edge length of the graph to represent the spatial intercellular relationship between different cell types within the tumor. This index represents the average edge length of tumor cells interacting with other tumor, stromal, and inflammation cells for each tumor cell. Based on this, the present embodiment analyzes the interaction pattern between different cells in the tumor ecosystem.

[0080] Based on the above design, the prediction model trained by modeling can analyze the cell interaction mode of different recurrence and metastasis risk groups in the tumor ecosystem, and characterize the morphological phenotype diversity of different risk groups. Further, it reflects the morphological relationship of early breast cancer recurrence and metastasis events at the cell / region level; establishes the correlation between digital pathology section morphological phenotype and patient recurrence and metastasis risk prognosis evaluation, realizes multi-omics modeling mining, and provides more possibilities for precision medicine based on multi-omics multi-modal biological information tumor analysis.

[0081] D, based on the risk prediction model built, phenotype diversity characterization analysis is carried out under different recurrence and metastasis risk groups.

[0082] Based on the risk prediction model built, phenotype diversity characterization analysis is carried out under different recurrence and metastasis risk groups, including: performing characterization analysis on the local image block set, and predicting the recurrence and metastasis risk score at the level of the full-scan digital pathology section; at the same time, a certain number of local image blocks with the highest attention score in the full-scan digital pathology section are selected as local regions within the tumor; the selected local regions are analyzed and calculated and clustering operation is performed; the phenotype diversity under different recurrence and metastasis risk groups is determined and characterized based on the clustering results. Specifically:

[0083] In order to study the morphological phenotype that may be related to different recurrence and metastasis risk groups, the embodiment uses the trained prediction model to perform feature representation on the set of local image blocks with a size of 256x256 pixels sampled from the full-scan digital pathology section. These local image block features are then input into the prediction model for inference and prediction, resulting in a recurrence and metastasis risk score at the level of the full-scan digital pathology section. In addition, the embodiment identifies the top 100 local image blocks with the highest attention score from the digital pathology section, which are considered to represent local regions within the tumor. Each instance in the set of local image blocks is also input into the prediction model for risk score inference at the level of the local image block. For each local image block, the feature vector after feature aggregation through the Transformer architecture is saved as the embedding vector of each local image block instance.

[0084] The present application further analyzes the embedding data. All image block embeddings are merged into a data structure object, which contains the original data matrix of local image block embeddings and additional metadata. The metadata consists of the spatial coordinates of the local image block in the corresponding digital pathology section, the recurrence and metastasis risk group result of the corresponding digital pathology section, the risk score at the level of the digital pathology section, and the image block level risk score. Based on this, the embodiment calculates the nearest neighbor distance matrix and the neighborhood graph of local image block embeddings, further realizing local instance point clustering and community detection. Finally, the results are reduced and visualized in a two-dimensional space of image block embeddings and their sub-clusters.

[0085] Based on the clustering results of local instance points, subgroups representing local morphological patterns were identified, and further analysis was performed on these subgroups. First, according to the recurrence and metastasis risk groups of the digital pathology slides corresponding to the local image blocks, instance points in different recurrence and metastasis risk groups were extracted from the global subgroups, and only subgroups with instance point numbers exceeding 25% of the instance point numbers in the corresponding global subgroup in the risk group were retained. Further determination of the simultaneous presence of subgroups in different recurrence and metastasis risk groups was performed to identify intersection overlapping subgroups, which were represented as co-localization subgroups. Subgroups unique to each risk group were labeled as group-specific subgroups. Based on this, the phenotypic diversity in different risk groups was determined and characterized.

[0086] Embodiment Two

[0087] The embodiment discloses a recurrence and metastasis risk prediction system for early breast cancer based on pathology slides.

[0088] The recurrence and metastasis risk prediction system for early breast cancer based on pathology slides comprises:

[0089] A model construction module is configured to obtain a full-scan digital pathology slide and construct a risk prediction model based on a weakly supervised learning architecture.

[0090] A spatial attention heat map generation module is configured to generate a spatial attention heat map at the level of the full-scan digital pathology slide using the risk prediction model.

[0091] A cell-level interaction module is configured to extract a high-attention region of interest in the spatial attention heat map and construct a cell-level interaction pattern in the patient's tumor ecosystem within the high-attention region of interest.

[0092] A phenotypic diversity characterization analysis module is configured to perform phenotypic diversity characterization analysis under different recurrence and metastasis risk groups based on the constructed risk prediction model. The recurrence and metastasis risk of early breast cancer is obtained by combining the obtained spatial attention heat map, interaction pattern, and characterization analysis result.

[0093] Embodiment Three

[0094] The purpose of the embodiment is to provide a computer-readable storage medium.

[0095] The computer-readable storage medium stores a computer program, which is executed by a processor to implement the steps in the recurrence and metastasis risk prediction method for early breast cancer based on pathology slides according to Embodiment One of the present disclosure.

[0096] Embodiment Four

[0097] The purpose of the embodiment is to provide an electronic device.

[0098] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor, and the processor implements the steps in the early breast cancer recurrence metastasis risk prediction method based on pathological sections according to the embodiment one of the present disclosure when executing the program.

[0099] The steps involved in the devices of the above embodiments two, three and four correspond to the method embodiment one, and the specific implementation can refer to the related description part of the embodiment one. The term "computer readable storage medium" should be understood as including a single medium or multiple media of one or more instruction sets; it should also be understood as including any medium capable of storing, encoding or carrying instruction sets for execution by a processor and causing the processor to perform any method in the present disclosure.

[0100] Those skilled in the art should understand that each module or step of the present disclosure described above can be realized by a general computer device, alternatively, they can be realized by program codes executable by a computing device, so that they can be stored in a storage device for execution by a computing device, or they can be respectively manufactured into each integrated circuit module, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module to realize. The present disclosure is not limited to any specific combination of hardware and software.

[0101] Although the specific embodiments of the present disclosure are described above in combination with the accompanying drawings, it is not a limitation on the protection scope of the present disclosure, and those skilled in the art should understand that various modifications or changes made by those skilled in the art on the basis of the technical solutions of the present disclosure without creative labor are still within the protection scope of the present disclosure.

Claims

1. A method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections, characterized by, The application relates to a risk prediction model based on a weakly supervised learning architecture. The risk prediction model comprises a full-scan digital pathology slice processing and feature representation module, a feature aggregation module and an observer based on a regression strategy; the feature aggregation module adopts a Transformer architecture based on an agent attention mechanism, that is, the Transformer architecture contains self-optimized agent tokens and is stacked by two Softmax self-attention operations. A spatial attention heat map of the full-scan digital pathology slice is generated by using the risk prediction model. A high-attention region of interest in the spatial attention heat map is extracted, and an interaction mode at a cell level in a tumor ecosystem of a patient is constructed in the high-attention region of interest. Based on the constructed risk prediction model, phenotype diversity representation analysis is performed on different recurrence and metastasis risk groups; the spatial attention heat map, the interaction mode and the representation analysis result are combined to obtain the risk of recurrence and metastasis of early breast cancer. The full-scan digital pathology slice processing and feature representation module is used for non-overlapping sliding window sampling and dimension reduction feature representation of an effective foreground region at a specified magnification of a full-scan digital pathology slice, so as to obtain a local image block set and represent different morphological modes.

2. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 1, wherein, The spatial attention heat map is generated by using the risk prediction model, which comprises the following steps: firstly, data preprocessing is performed on the full-scan digital pathology slice; then, a pathology basic model is used to perform feature representation on the local image block to obtain an attention score of the local image block; finally, the obtained attention scores are arranged according to respective spatial coordinate positions to generate the spatial attention heat map.

3. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 1, wherein, The interaction mode at the cell level in the tumor ecosystem of the patient is constructed, which comprises the following steps: firstly, a high-attention region of interest in the spatial attention heat map is extracted; then, a plurality of models are used to simultaneously segment and classify cell nuclei; the segmented and classified cell main components are selected and a topological graph between the detected cell nuclei is constructed, and the topological features in the topological graph are used to represent the interaction mode at the cell level.

4. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 1, wherein, The phenotype diversity representation analysis comprises the following steps: the local image block set is represented and analyzed, and a recurrence and metastasis risk score at a full-scan digital pathology slice level is predicted; meanwhile, a certain number of local image blocks with the highest attention scores in the full-scan digital pathology slice are selected as local regions in a tumor; the selected local regions are analyzed and calculated and a clustering operation is performed; based on the clustering result, the phenotype diversity under different recurrence and metastasis risk groups is determined and represented.

5. The method for predicting the risk of recurrence and metastasis of early breast cancer based on pathological sections according to claim 1, wherein The application relates to a risk prediction model based on a weakly supervised learning architecture.

6. An early stage breast cancer recurrence metastasis risk prediction system based on pathology slides, characterized in that, ​ The model construction module is configured to: acquire a full-scan digital pathology section, and construct a risk prediction model based on a weakly supervised learning architecture; the risk prediction model comprises a full-scan digital pathology section processing and feature representation module, a feature aggregation module, and an observer based on a regression strategy; wherein the feature aggregation module adopts a Transformer architecture based on an agent attention mechanism, that is, the Transformer architecture contains self-optimizing agent tokens, and is stacked by two Softmax self-attention operations; The spatial attention heat map generation module is configured to: generate a spatial attention heat map at the level of the full-scan digital pathology section by using the risk prediction model; The cell-level interaction module is configured to: extract a high-attention region of interest in the spatial attention heat map, and construct an interaction mode at the cell level in the tumor ecosystem of the patient in the high-attention region of interest; The phenotype diversity representation analysis module is configured to: perform phenotype diversity representation analysis under different recurrence and metastasis risk groups based on the constructed risk prediction model; and obtain the risk of recurrence and metastasis of early breast cancer by combining the obtained spatial attention heat map, the interaction mode, and the representation analysis result.

7. A computer-readable storage medium having stored thereon a program, characterized in that, The program is executed by the processor to implement the steps in the early breast cancer recurrence and metastasis risk prediction method based on a pathology section according to any one of claims 1-5.

8. An electronic device comprising a memory, a processor, and a program stored in the memory and capable of running on the processor, characterized by The processor executes the program to implement the steps in the early breast cancer recurrence and metastasis risk prediction method based on a pathology section according to any one of claims 1-5.

Images