Breast cancer detection method and system based on morphological image and spatial transcriptome cross-map collaborative learning

Abstract

The application discloses a breast cancer detection method and system based on morphological images and spatial transcriptome cross-map collaborative learning, and the method comprises the following steps: acquiring initial pathological morphological image data and initial spatial transcriptome data for breast cancer detection; performing cross-modal coordinate registration on the initial pathological morphological image data and the initial spatial transcriptome data to obtain target pathological morphological image data and target spatial transcriptome data under the same physical coordinate system; constructing a pathological morphological image graph and a spatial transcriptome graph based on the target pathological morphological image data and the target spatial transcriptome data; inputting the pathological morphological image graph and the spatial transcriptome graph into a pre-trained target prediction neural network model to output a probability distribution heat map; and detecting a breast cancer lesion area according to the probability distribution heat map to obtain a target lesion area. The application realizes consistent modeling of tissue structure and molecular expression, and reveals the spatial heterogeneity and multi-scale molecular mechanism inside the breast cancer tissue.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and biomedical technology, specifically relating to a breast cancer detection method and system based on cross-graph collaborative learning of morphological images and spatial transcriptomics. Background Technology

[0002] Breast cancer is one of the most common malignant tumors among women worldwide, and its early detection and subtype research are of great significance for biomedical research and clinical decision-making. Traditional breast cancer research mainly relies on the combined use of imaging screening (such as mammography, ultrasound, MRI) and pathological analysis (H&E slides, immunohistochemical detection, FISH, RNA sequencing, etc.). Among them, H&E slides can provide information on cell morphology and histological structure, which is an important basis for pathological diagnosis. However, this method is limited by the following factors: (1) poor staining consistency and imaging noise; (2) differences in doctors' subjective judgment; (3) inability to reveal potential molecular driving mechanisms. On the other hand, although traditional transcriptome sequencing can reveal molecular expression profiles at the cellular level, it loses information on spatial location and is difficult to correspond to tissue morphological characteristics.

[0003] The advent of spatial transcriptomics (ST) technology has enabled researchers to simultaneously obtain gene expression matrices and spatial coordinates from tissue sections, achieving a research model that combines "tissue structure and molecular function." ST technology has shown great potential in cancer microenvironment research, but problems such as inaccurate spatial registration and inaccurate graph structure modeling still exist. Therefore, there is an urgent need for a cross-modal, noise-robust, and interpretable collaborative learning framework to model, register, and jointly optimize H&E images and ST data at the graph structure level, thereby revealing the spatial consistency between breast cancer tissue structure and molecular mechanisms, and improving the accuracy of breast cancer detection. Summary of the Invention

[0004] To address the aforementioned problems in the existing technology, this invention provides a breast cancer detection method and system based on cross-graph collaborative learning of morphological images and spatial transcriptomics.

[0005] The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides a breast cancer detection method based on cross-map collaborative learning of morphological images and spatial transcriptomics, comprising: Acquire initial pathological morphological imaging data and initial spatial transcriptome data for breast cancer detection; Cross-modal coordinate registration was performed on the initial pathological morphology image data and the initial spatial transcriptome data to obtain the target pathological morphology image data and the target spatial transcriptome data in the same physical coordinate system; Based on the target pathological morphology image data and the target spatial transcriptome data, pathological morphology image maps corresponding to the pathological morphology image data and spatial transcriptome maps corresponding to the spatial transcriptome data are constructed respectively. The pathological morphological images and spatial transcriptome maps are input into a pre-trained target prediction neural network model, which outputs a probability distribution heatmap. The target lesion area is obtained by detecting the breast cancer lesion area based on the probability distribution heatmap.

[0006] Secondly, the present invention provides a breast cancer detection system based on cross-map collaborative learning of morphological images and spatial transcriptomics, comprising: The acquisition module is used to acquire initial pathological morphological imaging data and initial spatial transcriptome data for breast cancer detection. The registration module is used to perform cross-modal coordinate registration of initial pathological morphological image data and initial spatial transcriptome data to obtain target pathological morphological image data and target spatial transcriptome data in the same physical coordinate system. The graph construction module is used to construct pathological morphological image maps corresponding to the target pathological morphological image data and spatial transcriptome maps corresponding to the target spatial transcriptome data, respectively, based on the target pathological morphological image data and the target spatial transcriptome data. The model prediction module is used to input pathological morphological images and spatial transcriptome maps into a pre-trained target prediction neural network model and output a probability distribution heatmap. The region detection module is used to detect breast cancer lesion regions based on probability distribution heatmaps to obtain target lesion regions.

[0007] This invention provides a breast cancer detection method and system based on cross-graph collaborative learning of morphological images and spatial transcriptomics. Through a three-layer structure—graph construction layer, cross-graph alignment layer, and fusion learning layer—the morphological information of pathological sections is spatially aligned and feature-level fused with the molecular information of the spatial transcriptome. This achieves consistent modeling of tissue structure and molecular expression, enabling stable and interpretable joint representations to be obtained even under noisy and multimodal real-world conditions, thereby improving the accuracy of breast cancer detection results.

[0008] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0009] Figure 1 This is a schematic flowchart of a breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a breast cancer detection system based on cross-graph collaborative learning of morphological images and spatial transcriptomics, provided in an embodiment of the present invention. Detailed Implementation

[0010] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0011] This invention provides a breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics. See also... Figure 1 The method includes the following steps: S10. Obtain initial pathological morphological imaging data and initial spatial transcriptome data for breast cancer detection.

[0012] For example, the initial pathological morphological image data can be pathological morphological H&E images, and the initial spatial transcriptome data can include coordinate information and gene expression profile data.

[0013] S20. Perform cross-modal coordinate registration on the initial pathological morphology image data and the initial spatial transcriptome data to obtain the target pathological morphology image data and the target spatial transcriptome data in the same physical coordinate system.

[0014] Optionally, step S20 may specifically include: S201. Based on affine transformation, coordinate transformation is performed on the initial spatial transcriptome data to obtain the first spatial transcriptome data that is initially registered with the initial pathological morphological image data.

[0015] For example, due to differences in experimental preparation processes and data acquisition methods, the coordinate systems between pathological morphological images and spatial transcriptomes are not fully compatible. Therefore, before constructing a multimodal graph structure and subsequent information fusion, it is necessary to align the coordinates of the two, i.e., perform cross-modal spatial registration.

[0016] First, before performing spatial registration, a set of reference markers needs to be determined to establish the initial correspondence between the two modalities, forming a set of anchor points. Anchor points are spatial points that can be simultaneously identified in the initial pathological morphological image data and the initial spatial transcriptome data, and whose physical locations correspond to each other. They are used to constrain the parameters of the affine transformations involved in the registration process. Here, the set of anchor points corresponding to the initial pathological morphological image data is denoted as […]. The set of anchor points corresponding to the initial spatial transcriptome data is denoted as Anchor points generally originate from the following sources: tissue boundaries or prominent morphological structures (such as glandular outlines, vascular cavities, and fat cavities), microarray landmarks (present in pathological morphological images and spatial transcriptome arrays), homologous points manually marked by experts, and markers obtained based on automatic feature matching. This embodiment has the following requirements for anchor points: uniform distribution, non-collinearity, and coverage of the entire tissue area as much as possible.

[0017] In the initial registration stage, rotation and translation are needed to complete the initial registration, that is, the spatial coordinates of the two modes need to satisfy the following condition:

[0018] In the formula, Affine transformation is represented by a set of transformation parameters. and The implemented affine transformation process is represented as follows:

[0019] The objective function for optimizing the affine transformation is expressed as:

[0020] Optimization can achieve preliminary registration by rotating, translating, and scaling the spatial transcriptome lattice to a position roughly aligned with the initial pathological morphological image data, thus obtaining the coordinates of the first spatial transcriptome data.

[0021] S202. Based on the free-form mesh model, non-rigid registration is performed on the first spatial transcriptome data to obtain the second spatial transcriptome data mapped to the coordinate system corresponding to the initial pathological morphological image data.

[0022] For example, to further correct local tissue stretching, rotation, and slice deformation, a non-rigid model based on a free-form deformation (FFD) mesh is used for further registration based on the initial registration. This embodiment introduces a free-form deformation (FFD) mesh model to complete the non-rigid registration process.

[0023] In a two-dimensional FFD model, each point in the first-space transcriptome data The deformation is determined by the weighted sum of the surrounding control points:

[0024] In the formula, , The optimal affine transformation parameters obtained in step S201 are... , It is a cubic B-spline basis function. and The values ​​are 0, 1, 2, and 3, which are four odd function indices. This is the control point displacement vector. The basis functions are defined as follows:

[0025] In the formula, Let be any variable representing a basis function.

[0026] The main properties of basis functions are as follows: (1) non-negativity; (2) normalization. (3) Both the first and second derivatives are continuous functions.

[0027] After FFD non-rigid registration, the positional coordinates of the first spatial transcriptome data have been mapped to the coordinate system of the pathological morphological images. The mapping process is represented as follows:

[0028] Here, this embodiment solves the optimization problem of the registration process by maximizing the mutual information between different modalities.

[0029]

[0030] In the formula, This represents the mutual information between pathological morphological imaging modalities and spatial transcriptomic modalities. It is the joint probability distribution of the two modes. These are the marginal probability distributions corresponding to the two modes.

[0031] In this embodiment, mutual information is used as the optimization target for cross-modal registration. This is the initial pathological morphological imaging data. This refers to the coordinate data in the second spatial transcriptome data. The joint probability distribution is estimated based on the joint histogram of both datasets. and marginal distribution The mutual information is calculated according to the method shown in the above formula. During the registration process, the affine transformation parameters and FFD control points are iteratively updated to ensure... This maximizes the alignment between pathological morphological imaging modalities and spatial transcriptomic modalities.

[0032] Therefore, the objective function for non-rigid registration can be expressed as:

[0033] In the formula, The gradient energy represents the deformation field and constrains the local deformation amplitude. These are the weight parameters that control the gradient energy.

[0034] Initial pathological morphological imaging data are defined in a bounded open set. The initial pathological morphological image has a grayscale (or brightness) of 1000 g / m². After the above registration, the second-space transcriptome data are represented as follows:

[0035] in, Represents the coordinates in the second-space transcriptome data. This represents the calculation results of the optimal free-deformation mesh model. It is the gene expression profile in the initial spatial transcriptome data.

[0036] S203. Perform coordinate transformation on the initial pathological morphology image data and the second spatial transcriptome data to obtain the target pathological morphology image data and the target spatial transcriptome data in the same physical coordinate system.

[0037] For example, after registration is completed, in order to ensure the comparability of spatial structures during subsequent mapping, this embodiment further performs a unified transformation on the coordinates of the initial pathological morphological image data and the second spatial transcriptome data, so that they are ultimately in a unified physical space, with micrometers as the actual unit.

[0038] The coordinates of the initial pathological morphological image data and the second spatial transcriptome data are defined on their respective discrete grids (pixel index, spot index), and their units and resolutions are not consistent. To ensure that subsequent registration and mapping are based on a unified physical distance metric, this embodiment first linearly scales and translates the coordinates of the two modalities to the same physical coordinate system in micrometers, based on the spatial resolution parameters (such as μm / pixel, μm / spot spacing) provided by the imaging system and the ST sequencing chip. Specifically, assuming the pixel resolution of the initial pathological morphological image data is... The resolution corresponding to the spot spacing in the second spatial transcriptome data is By multiplying these two resolutions by their corresponding coordinates, a unified coordinate transformation is achieved, resulting in target pathological morphological image data and target spatial transcriptome data in the same physical coordinate system.

[0039] S30. Based on the target pathological morphology image data and the target spatial transcriptome data, construct the pathological morphology image map corresponding to the pathological morphology image data and the spatial transcriptome map corresponding to the spatial transcriptome data, respectively.

[0040] It should be noted that the order in which the pathological morphological image map and the spatial transcriptome map are constructed is not limited in this embodiment.

[0041] Optionally, in step S30, based on the target pathological morphological image data, a pathological morphological image map corresponding to the pathological morphological image data is constructed, which may specifically include: S301. Remove non-tissue regions from the target pathological morphological image data to obtain tissue region image data.

[0042] For example, before constructing the pathological morphology image, the target pathological morphology image data (i.e., the image) is first color-normalized, and a tissue mask is obtained using threshold segmentation or deep neural network segmentation. After removing the background, blank areas on the slide, and obvious voids, the final image data of the tissue area is retained.

[0043] S302. Divide the tissue region image data into grids to obtain multiple grid regions.

[0044] For example, under a unified coordinate system, the tissue region image data is approximately divided into regularized grids to obtain a series of grid regions (patches). Let the grid spacing be... Pixels, for the center position of each grid and Each center point corresponds to a node in the image. It is important to emphasize that... and They are generally not equal. It is determined by the image sampling density, grid division, and scaling strategy.

[0045] S303. Based on preset multi-scale window parameters, feature extraction and cross-scale feature fusion are performed on the grid center of each grid region to obtain the target image features corresponding to each grid region.

[0046] Optionally, step S303 may specifically include: S3031. Based on preset multi-scale window parameters, the grid center of each grid region is offset and sampled multiple times to obtain multiple sampling area data corresponding to each grid region at each scale.

[0047] For example, this embodiment uses a method that can cover 45-110. The patch is used to reflect local cell populations, mitochondrial structures, or tissue microenvironments. To capture pathological structures at different levels, a pre-defined set of multi-scale window parameters is introduced:

[0048] Among them, different Representing cropping windows of different sizes, it enables cross-scale hierarchical sampling and observation from the cell level to the region level.

[0049] To further improve the robustness of pathological image characterization, the center point was analyzed under each preset multi-scale window parameter. conduct Sub-slight offset sampling yields multiple sampling region data corresponding to each grid region at each scale:

[0050] In the formula, Crop means image cropping, which means cropping a rectangular region from each grid I, with its center coordinates... , Indicates the number of times the data was randomly cut. These represent the distances from the center to the boundaries in the X and Y directions, respectively, which can be understood as half the length and half the width of the rectangle. This represents the coordinate offset at the ideal center point. A constraint must be met: the L2 norm of the offset coordinates cannot be greater than a linear ratio of the smaller of the rectangle's length and width. This ratio is represented by a parameter. This indicates that its value ranges from 0 to 0.3.

[0051] S3032. Extract features from multiple sampled regions corresponding to each grid region at each scale to obtain multiple sampled image features corresponding to each grid region at each scale.

[0052] For example, the extraction of sampled image features is achieved by directly using pre-trained deep neural networks (such as ResNet, ViT, etc.):

[0053] In the formula, represent The first scale The sampling image features corresponding to the sampling area data obtained from the second cropping have the following dimensions: , This represents a pre-trained image representation extractor. represent The first scale The sampling area data obtained by random cropping.

[0054] S3033. Calculate the mean of multiple sampled image features corresponding to each grid region at each scale to obtain the average image features corresponding to each grid region at each scale.

[0055] For example, the average image features corresponding to each grid region at each scale are represented as follows:

[0056] S3034. Perform cross-scale fusion on the average image features of each grid region at each scale to obtain the target image features corresponding to each grid region.

[0057] For example, for each grid region, the average image features at all scales are fused across scales to obtain the target image features corresponding to each grid region, represented as follows:

[0058] This yields the target image feature set corresponding to the tissue region image data. .

[0059] S304. Based on the target image features corresponding to each grid region, determine the target adjacency matrix of the pathological morphological image to characterize the pathological morphological image.

[0060] Optionally, the initial adjacency matrix of the pathological morphological image is represented as: ; in, This represents the initial adjacency matrix of the pathological morphological image. and This represents two different data indices within the same modality of data. and To represent different target image features. ; The initial adjacency matrix of the pathological morphology image is sparsified to obtain the target adjacency matrix of the pathological morphology image.

[0061] For example, the granularity of the pathological morphology image modality is patch, while the granularity of the spatial transcriptome modality is cell or cell spot. To reduce the complexity of subsequent calculations, the initial adjacency matrix of the pathological morphology image map is further derived using spatial information. The sparsity process is performed, as shown in the formula:

[0062] In the formula, Representative cells (or cell spot) The neighborhood of ) is constructed based on the nearest neighbor method, and the number of neighborhoods used in this embodiment is 5.

[0063] Optionally, in step S30, based on the target spatial transcriptome data, a spatial transcriptome map corresponding to the spatial transcriptome data is constructed, which may specifically include: S305. Based on the resolution of the target spatial transcriptome data, identify individual cells or cell spots as nodes in the spatial transcriptome map.

[0064] In this system, the nodes of the spatial transcriptome map correspond one-to-one with the coordinates of the target spatial transcriptome data.

[0065] For example, based on the resolution of the target spatial transcriptome data, the node set is defined as a node in a graph, with a cell or cell spot as the node. The set of coordinates of the target spatial transcriptome data One-to-one correspondence.

[0066] S306. Perform linear dimensionality reduction and standardization on the gene expression profile data in the target spatial transcriptome data to obtain gene expression characteristics.

[0067] For example, gene expression profile data in the target spatial transcriptome data. Linear dimensionality reduction and standardization were performed to obtain gene expression characteristics. :

[0068] S307. Based on the coordinate data and gene expression characteristics in the target spatial transcriptome data, determine the target adjacency matrix of the spatial transcriptome map to characterize the spatial transcriptome map.

[0069] Optionally, the initial adjacency matrix of the spatial transcriptome map is represented as:

[0070] in, This represents the initial adjacency matrix of the spatial transcriptome map. and This represents two different data indices within the same modality of data. This represents the coordinate data in the target spatial transcriptome data. , Indicates gene expression characteristics, Weight parameters Used to adjust the importance of coordinate data and gene expression features in the graph construction process; The initial adjacency matrix of the spatial transcriptome map is sparsified to obtain the target adjacency matrix of the spatial transcriptome map.

[0071] For example, the adjacency matrix of the spatial transcriptome map is defined as follows: ,in Representative cell (or cell spot) And cells (or cell spots) The degree of similarity in gene expression.

[0072] To reduce the complexity of subsequent calculations, the initial adjacency matrix of the spatial transcriptome graph is further derived using spatial information. The sparsity process is performed, as shown in the formula:

[0073] in, Representative cells (or cell spot) The neighborhood of a cell is defined as the set of the 15 cells whose spatial location is closest to a given cell.

[0074] S40. Input the pathological morphology image and spatial transcriptome map into the pre-trained target prediction neural network model and output a probability distribution heatmap.

[0075] For example, based on the above mapping process, the core dual-map structure (pathological morphological image map and spatial transcriptome map) and cross-modal registration results of this embodiment are obtained. On this basis, the identification of breast cancer lesion regions is further achieved. This embodiment models lesion identification as a semi-supervised problem and a cross-modal consistency optimization problem based on pathological morphological image map and spatial transcriptome map. It should be emphasized that the method of this embodiment is currently only used for breast cancer detection research and is not directly used as a basis for clinical diagnosis.

[0076] First, within the registered unified physical coordinate system, this embodiment has constructed pathological morphological images and spatial transcriptome maps, respectively. Lesion identification aims to learn the following two mappings:

[0077] In the formula, A mapping function representing the pathological morphological imaging modality. These two mapping functions represent spatial transcriptome modalities and are responsible for converting the node vectors in each graph into two-dimensional real vectors. These represent nodes in the pathological morphological image and the spatial transcriptome map, respectively. The mapping function is implemented through a fully connected neural network, that is, the target prediction neural network model encapsulates two fully connected neural network models.

[0078] Furthermore, the real numbers obtained from the above mapping are mapped to lesion probabilities using softmax mapping:

[0079] in, and These are binary probability vectors representing a node in the pathological morphological imaging modality and the spatial transcriptome modality, respectively. In these binary probability vectors, one value represents the probability that the node is breast cancer, and the other value represents the probability that the node is not breast cancer. The mapping results represent the pathological morphological imaging modalities. This represents the mapping results of spatial transcriptome modes.

[0080] In this embodiment, the supervisory information that can be used during the training of the target prediction neural network model includes: ① obvious lesion areas (such as invasive carcinoma areas, DCIS areas) marked by professional physicians on pathological images, which can be mapped to labels of image map node sets; ② breast cancer marker genes (such as EPCAM, KRTB, ERBB2, etc.) on spatial transcriptomes, as well as tumor enrichment gene patterns that have been discovered in the literature, to determine the node set in spatial transcriptome data.

[0081] In order to propagate local structural information within the graph and ensure consistent lesion prediction results among interconnected nodes, this embodiment uses local smoothing constraints based on the graph Laplacian matrix to achieve internal consistency constraints between the pathological morphological imaging modality and the spatial transcriptome modality.

[0082] Among them, the internal consistency constraints of pathological morphological imaging modalities , is represented as:

[0083] In the formula, The target adjacency matrix represents the pathological morphological image. The Laplacian matrix representing a pathological morphological image. A mapping function representing pathological morphological image modalities.

[0084] Internal consistency constraints of spatial transcriptome modalities , is represented as:

[0085] In the formula, The target adjacency matrix represents the spatial transcriptome map. The Laplacian matrix representing the spatial transcriptome map, A mapping function representing spatial transcriptome modes.

[0086] The Laplace matrix is ​​used to ensure consistency between local neighborhood relationships and prediction results.

[0087] Furthermore, considering cross-graph spatial consistency constraints, the node coordinates of pathological morphological images can be calculated. Coordinates of the spatial transcriptome map Cross-modal distance:

[0088] Then, a cross-modal weight matrix is ​​constructed based on the Gaussian kernel:

[0089] In the formula, This represents the distance between two coordinates across modes. Used to control the range of action corresponding to the space, with a value ranging from 30 to 100. between.

[0090] We also need to consider the potential for supervisory losses. Applying supervisory constraints to labeled nodes is represented as follows:

[0091] in, and These represent the annotation information for two modalities, with CE representing cross-entropy calculation.

[0092] In summary, the optimization objective function used in the pre-trained target prediction neural network model is expressed as:

[0093] In the formula, This represents the objective function to be optimized. This indicates a loss of oversight. This represents the internal consistency constraints of pathological morphological imaging modalities. This represents the internal consistency constraints of spatial transcriptome modalities. Represents the weight parameters. Represents the cross-modal weight matrix. The mapping results represent the pathological morphological imaging modalities. The mapping results representing spatial transcriptome modalities. Data index representing pathological morphological imaging modalities. A data index representing spatial transcriptomic modalities.

[0094] S50. Detect breast cancer lesion areas based on probability distribution heatmaps to obtain target lesion areas.

[0095] Optionally, step S50 may specifically include: determining the region with a probability higher than the preset probability threshold in the probability distribution heatmap as the target lesion region based on a preset probability threshold.

[0096] This embodiment presents a breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics. Through a three-layer structure—graph construction layer, cross-graph alignment layer, and fusion learning layer—it spatially aligns and fuses the morphological information of pathological sections with the molecular information of the spatial transcriptome at the feature level, achieving consistent modeling of tissue structure and molecular expression. This reveals the spatial heterogeneity and multi-scale molecular mechanisms within breast cancer tissue. This method uses a dual-graph structure modeling of pathological morphological images and spatial transcriptome maps, and introduces contrastive optimization and robust regularization strategies into the cross-graph learning framework. This allows for stable and interpretable joint representations even under noisy and multimodal real-world conditions, thereby improving the accuracy of breast cancer detection results.

[0097] In research applications, this method can be used to explore the spatial heterogeneity, subtype differences, and cell-gene correspondence among breast cancer tumor regions, marginal regions, and normal tissues, providing data support for tumor microenvironment research, heterogeneity analysis, and exploration of molecular pathological mechanisms.

[0098] Corresponding to the aforementioned breast cancer detection method based on cross-map collaborative learning of morphological images and spatial transcriptomics, this invention also provides a breast cancer detection system based on cross-map collaborative learning of morphological images and spatial transcriptomics; as follows Figure 2 As shown, the system may include: The acquisition module 201 is used to acquire initial pathological morphological imaging data and initial spatial transcriptome data for breast cancer detection. The registration module 202 is used to perform cross-modal coordinate registration of the initial pathological morphology image data and the initial spatial transcriptome data to obtain the target pathological morphology image data and the target spatial transcriptome data in the same physical coordinate system. The graph construction module 203 is used to construct, based on the target pathological morphology image data and the target spatial transcriptome data, respectively, the pathological morphology image map corresponding to the pathological morphology image data and the spatial transcriptome map corresponding to the spatial transcriptome data. Model prediction module 204 is used to input pathological morphological images and spatial transcriptome maps into a pre-trained target prediction neural network model and output a probability distribution heatmap. The region detection module 205 is used to detect breast cancer lesion regions based on the probability distribution heatmap to obtain the target lesion region.

[0099] For details on the system, please refer to the steps of the breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics provided in the first aspect, which will not be repeated here.

[0100] This invention provides a breast cancer detection system based on cross-graph collaborative learning of morphological images and spatial transcriptomics. Through a three-layer structure—a graph construction layer, a cross-graph alignment layer, and a fusion learning layer—the system spatially aligns and fuses the morphological information of pathological sections with the molecular information of the spatial transcriptome at the feature level. This achieves consistent modeling of tissue structure and molecular expression, thereby revealing the spatial heterogeneity and multi-scale molecular mechanisms within breast cancer tissue. This method uses a dual-graph structure modeling of pathological morphological images and spatial transcriptome maps, and introduces contrastive optimization and robust regularization strategies into the cross-graph learning framework. This allows for stable and interpretable joint representations even under noisy and multimodal real-world conditions, thus improving the accuracy of breast cancer detection.

[0101] It should be noted that, for the system, since it is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0102] It should be noted that the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of systems and methods consistent with some aspects of the invention.

[0103] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0104] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings and the disclosure in carrying out the claimed invention. In the description of the invention, the word "comprising" does not exclude other components or steps, "a" or "an" does not exclude a plurality, and "a plurality" means two or more, unless otherwise explicitly specified. Furthermore, while different embodiments may describe certain measures, this does not mean that these measures cannot be combined to produce good results.

[0105] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A breast cancer detection method based on cross-map collaborative learning of morphological images and spatial transcriptomics, characterized in that, include: Acquire initial pathological morphological imaging data and initial spatial transcriptome data for breast cancer detection; Cross-modal coordinate registration is performed on the initial pathological morphological image data and the initial spatial transcriptome data to obtain target pathological morphological image data and target spatial transcriptome data in the same physical coordinate system; Based on the target pathological morphological image data and the target spatial transcriptome data, respectively construct the pathological morphological image map corresponding to the pathological morphological image data and the spatial transcriptome map corresponding to the spatial transcriptome data; The pathological morphological image and the spatial transcriptome map are input into a pre-trained target prediction neural network model, which outputs a probability distribution heatmap. The optimization objective function used to pre-train the target prediction neural network model is expressed as: In the formula, This represents the objective function to be optimized. This indicates a loss of oversight. This represents the internal consistency constraints of pathological morphological imaging modalities. This represents the internal consistency constraints of spatial transcriptome modalities. Represents the weight parameters. Represents the cross-modal weight matrix. The mapping results represent the pathological morphological imaging modalities. The mapping results representing spatial transcriptome modalities. Data index representing pathological morphological imaging modalities. Data index representing spatial transcriptomic modalities; in: In the formula, The target adjacency matrix represents the pathological morphological image. The Laplacian matrix representing a pathological morphological image. A mapping function representing pathological morphological image modalities; superscript Indicates transpose; In the formula, The target adjacency matrix represents the spatial transcriptome map. The Laplacian matrix representing the spatial transcriptome map, Mapping functions representing spatial transcriptome modes; superscript Indicates transpose; In the formula, This represents the distance between two coordinates across modes. Used to control the scope of action corresponding to the space; The target lesion area is obtained by detecting the breast cancer lesion area based on the probability distribution heatmap.

2. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 1, characterized in that, Cross-modal coordinate registration is performed on the initial pathological morphological image data and the initial spatial transcriptome data to obtain target pathological morphological image data and target spatial transcriptome data in the same coordinate system, including: Based on affine transformation, coordinate transformation is performed on the initial spatial transcriptome data to obtain first spatial transcriptome data that is initially registered with the initial pathological morphological image data. Based on the free-deformation mesh model, the first spatial transcriptome data is non-rigidly registered to obtain the second spatial transcriptome data mapped to the coordinate system corresponding to the initial pathological morphological image data. The initial pathological morphological image data and the second spatial transcriptome data are subjected to a coordinate transformation to obtain target pathological morphological image data and target spatial transcriptome data in the same physical coordinate system.

3. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 1, characterized in that, Based on the target pathological morphological image data, constructing a pathological morphological image map corresponding to the pathological morphological image data includes: Non-tissue regions are removed from the target pathological morphological image data to obtain tissue region image data. The tissue region image data is divided into grids to obtain multiple grid regions; Based on preset multi-scale window parameters, feature extraction and cross-scale feature fusion are performed on the grid center of each grid region to obtain the target image features corresponding to each grid region. Based on the target image features corresponding to each grid region, the target adjacency matrix of the pathological morphological image is determined to characterize the pathological morphological image.

4. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 3, characterized in that, The process involves extracting features and fusing cross-scale features from the grid centers of each grid region based on preset multi-scale window parameters to obtain the target image features corresponding to each grid region, including: Based on preset multi-scale window parameters, the grid center of each grid region is offset and sampled multiple times to obtain multiple sampled region data corresponding to each grid region at each scale. Feature extraction is performed on multiple sampled regions corresponding to each grid region at each scale to obtain multiple sampled image features corresponding to each grid region at each scale. The mean value of multiple sampled image features corresponding to each grid region at each scale is calculated to obtain the average image features corresponding to each grid region at each scale. Cross-scale fusion is performed on the average image features at each scale corresponding to each grid region to obtain the target image features corresponding to each grid region.

5. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 1, characterized in that, Based on the target spatial transcriptome data, a spatial transcriptome map corresponding to the spatial transcriptome data is constructed, including: Based on the resolution of the target spatial transcriptome data, a single cell or cell spot is identified as a node in the spatial transcriptome map, and the nodes of the spatial transcriptome map correspond one-to-one with the coordinates of the target spatial transcriptome data. Linear dimensionality reduction and standardization are performed on the gene expression profile data in the target spatial transcriptome data to obtain gene expression characteristics; Based on the coordinate data in the target spatial transcriptome data and the gene expression characteristics, the target adjacency matrix of the spatial transcriptome map is determined to characterize the spatial transcriptome map.

6. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 3, characterized in that, The initial adjacency matrix of the pathological morphological image is represented as follows: ； in, This represents the initial adjacency matrix of the pathological morphological image. and This represents two different data indices within the same modality of data. and To represent different target image features. ; The initial adjacency matrix of the pathological morphology image is sparsified to obtain the target adjacency matrix of the pathological morphology image.

7. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 5, characterized in that, The initial adjacency matrix of the spatial transcriptome map is represented as follows: ； in, This represents the initial adjacency matrix of the spatial transcriptome map. and This represents two different data indices within the same modality of data. This represents the coordinate data in the target spatial transcriptome data. , Indicates gene expression characteristics, Weight parameters Used to adjust the importance of coordinate data and gene expression features in the graph construction process; The initial adjacency matrix of the spatial transcriptome graph is sparsified to obtain the target adjacency matrix of the spatial transcriptome graph.

8. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 1, characterized in that, The step of detecting breast cancer lesion regions based on the probability distribution heatmap to obtain the target lesion region includes: Based on a preset probability threshold, the regions in the probability distribution heatmap with a probability higher than the preset probability threshold are identified as target lesion regions.

9. A breast cancer detection system based on cross-map collaborative learning of morphological images and spatial transcriptomics, characterized in that, include: The acquisition module is used to acquire initial pathological morphological imaging data and initial spatial transcriptome data for breast cancer detection. The registration module is used to perform cross-modal coordinate registration of the initial pathological morphological image data and the initial spatial transcriptome data to obtain target pathological morphological image data and target spatial transcriptome data in the same physical coordinate system. The graph construction module is used to construct, based on the target pathological morphological image data and the target spatial transcriptome data, respectively, a pathological morphological image map corresponding to the pathological morphological image data and a spatial transcriptome map corresponding to the spatial transcriptome data. The model prediction module is used to input the pathological morphological image and the spatial transcriptome map into a pre-trained target prediction neural network model, and output a probability distribution heatmap; the optimization objective function used to pre-train the target prediction neural network model is expressed as: In the formula, This represents the objective function to be optimized. This indicates a loss of oversight. This represents the internal consistency constraints of pathological morphological imaging modalities. This represents the internal consistency constraints of spatial transcriptome modalities. Represents the weight parameters. Represents the cross-modal weight matrix. The mapping results represent the pathological morphological imaging modalities. The mapping results representing spatial transcriptome modalities. Data index representing pathological morphological imaging modalities. Data index representing spatial transcriptomic modalities; in: In the formula, The target adjacency matrix represents the pathological morphological image. The Laplacian matrix representing a pathological morphological image. A mapping function representing pathological morphological image modalities; superscript Indicates transpose; In the formula, The target adjacency matrix represents the spatial transcriptome map. The Laplacian matrix representing the spatial transcriptome map, Mapping functions representing spatial transcriptome modes; superscript Indicates transpose; In the formula, This represents the distance between two coordinates across modes. Used to control the scope of action corresponding to the space; The region detection module is used to detect breast cancer lesion regions based on the probability distribution heatmap to obtain the target lesion region.

Images