Breast cancer detection method and system based on morphological image and space transcriptome cross-graph collaborative learning

Abstract

The invention discloses a breast cancer detection method and system based on morphological image and space transcriptome cross-graph collaborative learning. The method comprises the following steps: acquiring initial pathological morphological image data and initial space transcriptome data for breast cancer detection; performing cross-modal coordinate registration on the initial pathological morphology image data and the initial space transcriptome data to obtain target pathological morphology image data and target space transcriptome data under the same physical coordinate system; based on the target pathological morphology image data and the target space transcriptome data, respectively constructing a pathological morphology image map and a space transcriptome map; inputting the pathological morphology image map and the spatial transcriptome map into a pre-trained target prediction neural network model, and outputting a probability distribution heat map; and performing breast cancer lesion area detection according to the probability distribution heat map to obtain a target lesion area. According to the invention, consistent modeling of the tissue structure and molecular expression is realized, and spatial heterogeneity and a multi-scale molecular mechanism in breast cancer tissues are disclosed.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence and biomedical technology, and particularly relates to a breast cancer detection method and system based on morphological image and spatial transcriptome cross-graph collaborative learning. BACKGROUND

[0002] Breast cancer is one of the malignant tumors with high incidence in women worldwide, and its early detection and subtype research are of great significance to biomedical research and clinical decision-making. Traditional breast cancer research mainly relies on the comprehensive use of imaging screening (such as mammography, ultrasound, MRI) and pathological analysis (H&E section, immunohistochemical detection, FISH, RNA sequencing, etc.). Among them, H&E section can provide cell morphology and histological structure information, which is an important basis for pathological diagnosis, but this method is subject to the following factors: (1) poor staining consistency and imaging noise; (2) differences in subjective judgment of doctors; (3) unable to reveal the potential molecular driving mechanism. On the other hand, although traditional transcriptome sequencing can reveal the molecular expression profile at the cell level, it loses the spatial location information and is difficult to correspond to the tissue morphology characteristics.

[0003] The emergence of spatial transcriptome (ST) technology enables researchers to obtain gene expression matrix and spatial coordinates on tissue sections simultaneously, realizing the research mode of "tissue structure and molecular function" coexistence. ST technology has shown great potential in cancer microenvironment research, but there are still problems such as inaccurate spatial registration and inaccurate graph structure modeling. Therefore, there is an urgent need for a cross-modal, noise-robust and interpretable collaborative learning framework to model, register and jointly optimize H&E image and ST data at the graph structure level, so as to reveal the spatial consistency of breast cancer tissue structure and molecular mechanism, and improve the accuracy of breast cancer detection. SUMMARY

[0004] In order to solve the above-mentioned problems existing in the prior art, the present application provides a breast cancer detection method and system based on morphological image and spatial transcriptome cross-graph collaborative learning.

[0005] The technical problem to be solved by the present application is solved by the following technical scheme: In a first aspect, the present application provides a breast cancer detection method based on morphological image and spatial transcriptome cross-graph collaborative learning, comprising: obtaining initial pathological morphological image data and initial spatial transcriptome data for breast cancer detection; performing cross-modal coordinate registration on the initial pathological morphological image data and the initial spatial transcriptome data to obtain target pathological morphological image data and target spatial transcriptome data under the same physical coordinate system; construct a pathology morphology image graph corresponding to the pathology morphology image data and a spatial transcriptome graph corresponding to the spatial transcriptome data based on the target pathology morphology image data and the target spatial transcriptome data; input the pathology morphology image graph and the spatial transcriptome graph into a pre-trained target prediction neural network model, and output a probability distribution heat map; detect a breast cancer lesion region according to the probability distribution heat map to obtain a target lesion region.

[0006] In a second aspect, the present application provides a breast cancer detection system based on morphology image and spatial transcriptome cross-graph collaborative learning, comprising: An acquisition module is configured to acquire initial pathology morphology image data and initial spatial transcriptome data for breast cancer detection. A registration module is configured to perform cross-modal coordinate registration on the initial pathology morphology image data and the initial spatial transcriptome data to obtain target pathology morphology image data and target spatial transcriptome data under the same physical coordinate system. A graph construction module is configured to construct a pathology morphology image graph corresponding to the pathology morphology image data and a spatial transcriptome graph corresponding to the spatial transcriptome data based on the target pathology morphology image data and the target spatial transcriptome data. A model prediction module is configured to input the pathology morphology image graph and the spatial transcriptome graph into a pre-trained target prediction neural network model, and output a probability distribution heat map. A region detection module is configured to detect a breast cancer lesion region according to the probability distribution heat map to obtain a target lesion region.

[0007] The breast cancer detection method and system based on morphology image and spatial transcriptome cross-graph collaborative learning provided by the present application can realize spatial level alignment and feature level fusion of morphology information of pathology sections and molecular information of spatial transcriptome through a three-layer structure, i.e., a graph construction layer, a cross-graph alignment layer and a fusion learning layer, realize consistent modeling of tissue structure and molecular expression, obtain stable and interpretable joint representation under actual conditions containing noise and multiple modalities, and thus improve the accuracy of breast cancer detection results.

[0008] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 is a flowchart of a breast cancer detection method based on morphology image and spatial transcriptome cross-graph collaborative learning provided by an embodiment of the present application; Figure 2 is a structural schematic diagram of a breast cancer detection system based on morphology image and spatial transcriptome cross-graph collaborative learning provided by an embodiment of the present application. DETAILED DESCRIPTION

[0010] The application will be described in further detail below with reference to specific embodiments, but the embodiments of the application are not limited thereto.

[0011] The embodiment of the application provides a breast cancer detection method based on morphological image and spatial transcriptome cross-graph collaborative learning. Referring to Figure 1 , the method comprises the following steps: S10, initial pathological morphological image data and initial spatial transcriptome data used for breast cancer detection are acquired.

[0012] Exemplarily, the initial pathological morphological image data can be pathological morphological H&E image, and the initial spatial transcriptome data can comprise coordinate information and gene expression profile data.

[0013] S20, cross-modal coordinate registration is performed on the initial pathological morphological image data and the initial spatial transcriptome data, to obtain target pathological morphological image data and target spatial transcriptome data in the same physical coordinate system.

[0014] Optionally, the step S20 can specifically comprise: S201, based on affine transformation, coordinate conversion is performed on the initial spatial transcriptome data, to obtain first spatial transcriptome data preliminarily registered with the initial pathological morphological image data.

[0015] Exemplarily, because of different experimental preparation processes and data acquisition modes, the coordinate systems between the pathological morphological image and the spatial transcriptome are not completely compatible, and therefore, before the construction of the multi-modal graph structure and the subsequent information fusion, the coordinates of the two need to be aligned, that is, cross-modal spatial registration is performed.

[0016] First, before performing spatial registration, a set of reference marker points need to be determined, which are used to establish an initial correspondence relationship between the two modalities and form a batch of anchor points. The anchor point refers to a spatial point that can be simultaneously recognized in the initial pathological morphological image data and the initial spatial transcriptome data and has a corresponding relationship in physical position, which is used to constrain the parameters of affine transformation involved in the registration process. Here, the anchor point set corresponding to the initial pathological morphological image data is denoted as , and the anchor point set corresponding to the initial spatial transcriptome data is denoted as . The anchor points generally have the following sources: tissue boundaries or significant morphological structures (such as gland outlines, blood vessel cavities, fat cavities, etc.), chip positioning marks (existing in both the pathological morphological image and the spatial transcriptome array), homologous points manually marked by artificial experts, and marker points obtained based on automatic feature matching. The embodiment has the following requirements for the anchor points: uniform distribution, non-collinearity, and covering the entire tissue region as much as possible.

[0017] In the initial registration stage, it is necessary to complete the initial registration by using rotation, translation, i.e. it is necessary to make the spatial coordinates of the two modal satisfy the conditions shown in the following formula:

[0018] In the formula, represents an affine transformation, which is specifically implemented by a set of transformation parameters and The conversion process of the affine transformation is represented as:

[0019] The optimization objective function of the affine transformation is represented as:

[0020] Optimizing it can achieve preliminary registration, i.e. rotating, translating and scaling the spatial transcriptome spatial point array as a whole to a position roughly aligned with the initial pathological morphology image data, to obtain the coordinates of the first spatial transcriptome data.

[0021] S202, based on the free deformation grid model, non-rigid registration is performed on the first spatial transcriptome data to obtain the second spatial transcriptome data mapped to the corresponding coordinate system of the initial pathological morphology image data.

[0022] Exemplarily, to further correct the local tissue stretching, rotation and slice deformation, on the basis of the initial registration, a free deformation grid-based non-rigid model is implemented for further registration. This embodiment introduces a free deformation grid (FFD) model to complete the non-rigid registration process.

[0023] In the two-dimensional FFD model, the deformation of each point in the first spatial transcriptome data is determined by the weighted sum of the surrounding control points:

[0024] In the formula, , is the optimal affine transformation parameter obtained in step S201, , is a cubic B-spline basis function, and the values of 0, 1, 2, 3, i.e. 4 odd function indexes, is a control point displacement vector. The basis function is defined as follows:

[0025] In the formula, Let be any variable representing a basis function.

[0026] The main properties of basis functions are as follows: (1) non-negativity; (2) normalization. (3) Both the first and second derivatives are continuous functions.

[0027] After FFD non-rigid registration, the positional coordinates of the first spatial transcriptome data have been mapped to the coordinate system of the pathological morphological images. The mapping process is represented as follows:

[0028] Here, this embodiment solves the optimization problem of the registration process by maximizing the mutual information between different modalities.

[0029]

[0030] In the formula, This represents the mutual information between pathological morphological imaging modalities and spatial transcriptomic modalities. It is the joint probability distribution of the two modes. These are the marginal probability distributions corresponding to the two modes.

[0031] In this embodiment, mutual information is used as the optimization target for cross-modal registration. This is the initial pathological morphological imaging data. This refers to the coordinate data in the second spatial transcriptome data. The joint probability distribution is estimated based on the joint histogram of both datasets. and marginal distribution The mutual information is calculated according to the method shown in the above formula. During the registration process, the affine transformation parameters and FFD control points are iteratively updated to ensure... This maximizes the alignment between pathological morphological imaging modalities and spatial transcriptomic modalities.

[0032] Therefore, the objective function for non-rigid registration can be expressed as:

[0033] In the formula, The gradient energy represents the deformation field and constrains the local deformation amplitude. These are the weight parameters that control the gradient energy.

[0034] Initial pathological morphological imaging data are defined in a bounded open set. The initial pathological morphological image has a grayscale (or brightness) of 1000 g / m². After the above registration, the second-space transcriptome data are represented as follows:

[0035] wherein, denotes the coordinates in the second spatial transcriptomic data, denotes the optimal free-form deformation mesh model calculation result, is the gene expression profile in the initial spatial transcriptomic data.

[0036] S203, coordinate uniform conversion is performed on the initial pathological morphology image data and the second spatial transcriptomic data to obtain target pathological morphology image data and target spatial transcriptomic data under the same physical coordinate system.

[0037] Illustratively, after registration is completed, in order to ensure that the spatial structure has comparability when subsequent composition is performed, the embodiment further performs uniform conversion on the coordinates of the initial pathological morphology image data and the second spatial transcriptomic data, so that they are finally in a unified physical space, and microns are used as the actual units thereof.

[0038] The coordinates of the initial pathological morphology image data and the second spatial transcriptomic data are respectively defined on discrete grids (pixel index, spot index), and the units and resolutions thereof are not consistent. In order to ensure that subsequent registration and composition are based on uniform physical distance measurement, the embodiment first linearly scales and translates the coordinates of the two modalities to the same physical coordinate system in units of "microns" according to the spatial resolution parameters (such as μm / pixel, μm / spot pitch) provided by the imaging system and the ST sequencing chip. Specifically, assuming that the pixel resolution of the initial pathological morphology image data is , and the resolution corresponding to the spot pitch of the second spatial transcriptomic data is , the two resolutions are multiplied by the corresponding coordinates to realize coordinate uniform conversion, to obtain target pathological morphology image data and target spatial transcriptomic data under the same physical coordinate system.

[0039] S30, based on the target pathological morphology image data and the target spatial transcriptomic data, a pathological morphology image corresponding to the pathological morphology image data and a spatial transcriptomic map corresponding to the spatial transcriptomic data are respectively constructed.

[0040] It should be noted that the order of constructing the pathological morphology image and the spatial transcriptomic map in the embodiment is not limited.

[0041] Optionally, in step S30, based on the target pathological morphology image data, the pathological morphology image corresponding to the pathological morphology image data is constructed, which can specifically include: S301, non-tissue region removal is performed on the target pathological morphology image data to obtain tissue region image data.

[0042] Exemplarily, before constructing the pathological morphology image, the target pathological morphology image data (i.e., the image) is first color standardized, and a tissue mask is obtained by threshold segmentation or deep neural network segmentation After removing the background, blank and obvious hollow areas, the final tissue region image data is retained.

[0043] S302, the tissue region image data is meshed to obtain a plurality of mesh regions.

[0044] Exemplarily, under a unified coordinate system, the tissue region image data is approximately regular meshed to obtain a series of mesh regions (patches). Let the mesh spacing be pixels, the center position of each mesh is and Each center point corresponds to a node in the image. It should be emphasized that and are generally not equal, determined by the image sampling density, meshing, and scale strategy.

[0045] S303, based on the preset multi-scale window parameters, the mesh center of each mesh region is extracted and cross-scale feature fusion is performed to obtain the target image features corresponding to each mesh region.

[0046] Optionally, step S303 can specifically include: S3031, based on the preset multi-scale window parameters, the mesh center of each mesh region is sampled multiple times to obtain a plurality of sampling region data corresponding to each mesh region at each scale.

[0047] Exemplarily, the embodiment uses a patch that can cover 45-110 to reflect local cell groups, linear structures or tissue microenvironments. To capture different levels of pathological structures, a set of preset multi-scale window parameters is introduced:

[0048] Among them, different represent different sizes of the clipping window, realizing cross-scale hierarchical sampling observation from the cell level to the regional level.

[0049] In order to further improve the robustness of pathological image representation, under each preset multi-scale window parameter, the center point is sampled times with slight offset to obtain a plurality of sampling region data corresponding to each mesh region at each scale:

[0050] In the formula, Crop represents image cropping, that is, a rectangular region is cropped from each grid I, and the center coordinates of the rectangular region are , represents the number of random crops, respectively represent the position of the boundary distance center in the X direction and the Y direction, which can be understood as 1 / 2 of the length and width of the rectangle. represents the coordinate offset at the ideal center point. A constraint needs to be met: the two-norm of the offset coordinates cannot be greater than a linear proportion of the smaller value of the length and width of the rectangular region, and the proportion is represented by the parameter , which takes a value in the range of 0 to 0.3.

[0051] S3032, feature extraction is performed on the plurality of sampling region data corresponding to each grid region at each scale to obtain a plurality of sampling image features corresponding to each grid region at each scale.

[0052] Exemplarily, the extraction of the sampling image features is realized by directly using a pre-trained deep neural network (such as Resnet, ViT, etc.):

[0053] In the formula, represents the sampling image feature corresponding to the sampling region data obtained by the i-th random crop at the scale s. , represents a pre-trained image representation extractor, represents the sampling region data obtained by the i-th random crop at the scale s.

[0054] S3033, the mean value of the plurality of sampling image features corresponding to each grid region at each scale is calculated to obtain the average image feature corresponding to each grid region at each scale.

[0055] Exemplarily, the average image feature corresponding to each grid region at each scale is represented as:

[0056] S3034, the average image features of each scale corresponding to each grid region are cross-scale fused to obtain the target image feature corresponding to each grid region.

[0057] Exemplarily, for each grid region, the average image features of all scales are cross-scale fused to obtain the target image feature corresponding to each grid region, which is represented as: ​​​

[0058] a target image feature set corresponding to the tissue region image data is obtained .

[0059] S304, based on the target image features corresponding to each grid region, determine the target adjacency matrix of the pathological morphology image graph to represent the pathological morphology image graph.

[0060] Optionally, the initial adjacency matrix of the pathological morphology image graph is represented as: ; wherein, represents the initial adjacency matrix of the pathological morphology image graph, and represent two different data indices in the same modality data, and represent different target image features, ; The initial adjacency matrix of the pathological morphology image graph is sparsified to obtain the target adjacency matrix of the pathological morphology image graph.

[0061] Exemplarily, the granularity of the pathological morphology image modality is patch, and the granularity of the spatial transcriptome modality is cell or cell spot. In order to reduce the complexity of subsequent calculation, the initial adjacency matrix of the pathological morphology image graph is further sparsified by relying on spatial information, and the specific formula is as follows:

[0062] In the formula, represent the neighborhood of the cell (or cell spot ), and the neighborhood is constructed based on the nearest neighbor method, and the number used in the present embodiment is 5.

[0063] Optionally, in step S30, based on the target spatial transcriptome data, a spatial transcriptome graph corresponding to the spatial transcriptome data is constructed, which can specifically include: S305, according to the resolution of the target spatial transcriptome data, a single cell or cell spot is determined as a node of the spatial transcriptome graph.

[0064] Wherein, the nodes of the spatial transcriptome graph correspond one-to-one to the coordinates of the target spatial transcriptome data.

[0065] Exemplarily, according to the resolution of the target spatial transcriptome data, a cell or a cell spot is taken as a node of the graph, and the node set corresponds one-to-one to the coordinate set of the target spatial transcriptome data. ​

[0066] S306, linear dimension reduction and standardization are performed on the gene expression profile data in the target spatial transcriptome data to obtain gene expression features.

[0067] Exemplarily, linear dimension reduction and standardization are performed on the gene expression profile data in the target spatial transcriptome data to obtain gene expression features

[0068] S307, according to the coordinate data and the gene expression features in the target spatial transcriptome data, a target adjacency matrix of the spatial transcriptome graph is determined to represent the spatial transcriptome graph.

[0069] Optionally, the initial adjacency matrix of the spatial transcriptome graph is represented as:

[0070] wherein, represents the initial adjacency matrix of the spatial transcriptome graph, and represent two different data indexes in the same modality data, represents the coordinate data in the target spatial transcriptome data, , represents the gene expression features, and the weight parameter is used to adjust the importance of the coordinate data and the gene expression features in the graph construction process; The initial adjacency matrix of the spatial transcriptome graph is subjected to sparse processing to obtain the target adjacency matrix of the spatial transcriptome graph.

[0071] Exemplarily, the adjacency matrix of the spatial transcriptome graph is defined as wherein, represents the proximity of a cell (or a cell spot) and a cell (or a cell spot) in gene expression.

[0072] In order to reduce the complexity of subsequent calculation, the initial adjacency matrix of the spatial transcriptome graph is further subjected to sparse processing according to the spatial information, and the specific formula is as follows:

[0073] wherein, represents the neighborhood of a cell (or a cell spot ), and here, a set of 15 cells closest to the spatial position of a cell is regarded as the neighborhood of the cell. ​​​

[0074] S40, inputting the pathological morphology image and the spatial transcriptome image into a pre-trained target prediction neural network model, and outputting a probability distribution heat map.

[0075] Exemplarily, based on the above composition process, the core dual-image structure (pathological morphology image and spatial transcriptome image) and the cross-modal registration result of the embodiment are obtained. On this basis, the recognition of the breast cancer lesion area is further realized. The lesion recognition of the embodiment is modeled as a semi-supervised problem and a cross-modal consistency optimization problem based on the pathological morphology image and the spatial transcriptome image. It should be emphasized that the method of the embodiment is currently only used for breast cancer detection research and is not directly used as a clinical diagnosis basis.

[0076] First, in the unified physical space coordinate system after registration, the pathological morphology image and the spatial transcriptome image have been constructed respectively, and the lesion recognition aims to learn the following two mappings:

[0077] In the formula, represents the mapping function of the pathological morphology image modality, represents the mapping function of the spatial transcriptome modality, and the two mapping functions are responsible for converting the node vector in each image into a two-dimensional real number vector. respectively represent the nodes in the pathological morphology image and the spatial transcriptome image. The mapping function is realized by a fully connected neural network, that is, the target prediction neural network model comprises two fully connected neural network models.

[0078] Further, the real number value obtained by the above mapping is mapped to the lesion probability by a softmax mapping:

[0079] wherein, and respectively represent the binary probability vector of a node in the pathological morphology image modality and the spatial transcriptome modality, wherein one value represents the probability that the node is breast cancer, and the other value represents the probability that the node is not breast cancer. represents the mapping result of the pathological morphology image modality, represents the mapping result of the spatial transcriptome modality.

[0080] The supervision information that can be used in the process of training the target prediction neural network model includes: ①obvious lesion regions (such as invasive cancer regions, DCIS regions) marked by professional doctors on pathological images, which can be mapped as labels of the image graph node set; ②breast cancer marker genes (such as EPCAM, KRTB, ERBB2, etc.) on the spatial transcriptome, and tumor-enriched gene patterns that have been found in the literature, and the node set is determined in the spatial transcriptome data.

[0081] In order to propagate the local structure information in the graph and make the connected nodes have consistent lesion prediction results, the embodiment uses a local smoothing constraint based on a graph Laplacian matrix to realize the internal consistency constraint of the pathological morphology image modality and the spatial transcriptome modality.

[0082] The internal consistency constraint of the pathological morphology image modality is represented as:

[0083] In the formula, A target adjacency matrix of the pathological morphology image graph is represented as A, A Laplacian matrix of the pathological morphology image graph is represented as L, A mapping function of the pathological morphology image modality is represented as f.

[0084] The internal consistency constraint of the spatial transcriptome modality is represented as:

[0085] In the formula, A target adjacency matrix of the spatial transcriptome graph is represented as A, A Laplacian matrix of the spatial transcriptome graph is represented as L, A mapping function of the spatial transcriptome modality is represented as f.

[0086] The Laplacian matrix is used to ensure the consistency of the local field relationship and the prediction result.

[0087] Further, considering the cross-graph spatial consistency constraint, the cross-modality distance of the node coordinates of the pathological morphology image graph and the coordinates of the spatial transcriptome graph can be calculated as:

[0088] Then, a cross-modality weight matrix is constructed based on a Gaussian kernel:

[0089] In the formula, ​​represents the distance between two coordinates across modalities, for controlling the action range of the spatial correspondence, taking values between 30 and 100 .

[0090] The supervised loss also needs to be considered The supervised constraint is performed on the labeled nodes, represented as:

[0091] wherein, and respectively represent the labeled information of the two modalities, and CE represents the cross-entropy operation.

[0092] In summary, the optimization objective function used for pre-training the target prediction neural network model is represented as:

[0093] In the formula, represents the optimization objective function, represents the supervised loss, represents the internal consistency constraint of the pathology morphology image modality, represents the internal consistency constraint of the spatial transcriptome modality, represents the weight parameter, represents the cross-modality weight matrix, represents the mapping result of the pathology morphology image modality, represents the mapping result of the spatial transcriptome modality, represents the data index of the pathology morphology image modality, represents the data index of the spatial transcriptome modality.

[0094] S50, according to the probability distribution heat map, breast cancer lesion region detection is performed to obtain a target lesion region.

[0095] Optionally, step S50 can specifically include: according to a preset probability threshold, determining a region with a probability higher than the preset probability threshold in the probability distribution heat map as the target lesion region.

[0096] This embodiment presents a breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics. Through a three-layer structure—graph construction layer, cross-graph alignment layer, and fusion learning layer—it spatially aligns and fuses the morphological information of pathological sections with the molecular information of the spatial transcriptome at the feature level, achieving consistent modeling of tissue structure and molecular expression. This reveals the spatial heterogeneity and multi-scale molecular mechanisms within breast cancer tissue. This method uses a dual-graph structure modeling of pathological morphological images and spatial transcriptome maps, and introduces contrastive optimization and robust regularization strategies into the cross-graph learning framework. This ensures stable and interpretable joint representations even under noisy and multimodal real-world conditions, thereby improving the accuracy of breast cancer detection results.

[0097] In research applications, this method can be used to explore the spatial heterogeneity, subtype differences, and cell-gene correspondence among breast cancer tumor regions, marginal regions, and normal tissues, providing data support for tumor microenvironment research, heterogeneity analysis, and exploration of molecular pathological mechanisms.

[0098] Corresponding to the aforementioned breast cancer detection method based on cross-map collaborative learning of morphological images and spatial transcriptomics, this invention also provides a breast cancer detection system based on cross-map collaborative learning of morphological images and spatial transcriptomics; as follows Figure 2 As shown, the system may include: The acquisition module 201 is used to acquire initial pathological morphological imaging data and initial spatial transcriptome data for breast cancer detection. The registration module 202 is used to perform cross-modal coordinate registration of the initial pathological morphology image data and the initial spatial transcriptome data to obtain the target pathological morphology image data and the target spatial transcriptome data in the same physical coordinate system. The graph construction module 203 is used to construct, based on the target pathological morphology image data and the target spatial transcriptome data, respectively, the pathological morphology image map corresponding to the pathological morphology image data and the spatial transcriptome map corresponding to the spatial transcriptome data. Model prediction module 204 is used to input pathological morphological images and spatial transcriptome maps into a pre-trained target prediction neural network model and output a probability distribution heatmap. The region detection module 205 is used to detect breast cancer lesion regions based on the probability distribution heatmap to obtain the target lesion region.

[0099] For details on the system, please refer to the steps of the breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics provided in the first aspect, which will not be repeated here.

[0100] The breast cancer detection system based on morphological images and spatial transcriptome cross-graph collaborative learning provided by the application realizes spatial level alignment and feature level fusion of morphological information of pathological sections and molecular information of spatial transcriptome through a three-layer structure, a graph construction layer, a cross-graph alignment layer and a fusion learning layer, realizes consistent modeling of tissue structure and molecular expression, and thus reveals spatial heterogeneity and multi-scale molecular mechanisms inside breast cancer tissue. The method models through a double-graph structure of pathological morphological image graphs and spatial transcriptome graphs, and introduces contrast optimization and robust regularization strategies in the cross-graph learning framework, so that stable and interpretable joint representation can still be obtained under actual conditions containing noise and multiple modalities, thereby improving the accuracy of breast cancer detection.

[0101] It should be noted that the system is basically similar to the method embodiment, so the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0102] It should be noted that the terms "first", "second", etc. are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The implementation described in the following exemplary embodiments does not represent all implementations consistent with the present application. Instead, they are only examples of systems and methods consistent with some aspects of the present application.

[0103] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in the specification.

[0104] Although the present application is described herein in conjunction with various embodiments, other variations of the disclosed embodiments can be understood and implemented by those skilled in the art with reference to the drawings and disclosure. In the description of the present application, the word "comprising" does not exclude other components or steps, "one" or "one" does not exclude multiple cases, and "multiple" means two or more, unless otherwise explicitly specified. In addition, some measures are described in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0105] The above description is further detailed in connection with specific preferred embodiments of the present application, and it is not to be construed that the specific implementation of the present application is limited to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, a number of simple deductions or substitutions can be made, and all of them should be considered as falling within the protection scope of the present application.

Claims

1. A breast cancer detection method based on cross-map collaborative learning of morphological images and spatial transcriptomics, characterized in that, include: Acquire initial pathological morphological imaging data and initial spatial transcriptome data for breast cancer detection; Cross-modal coordinate registration is performed on the initial pathological morphological image data and the initial spatial transcriptome data to obtain target pathological morphological image data and target spatial transcriptome data in the same physical coordinate system; Based on the target pathological morphological image data and the target spatial transcriptome data, respectively construct the pathological morphological image map corresponding to the pathological morphological image data and the spatial transcriptome map corresponding to the spatial transcriptome data; The pathological morphological image and the spatial transcriptome map are input into a pre-trained target prediction neural network model, which outputs a probability distribution heatmap. The target lesion area is obtained by detecting the breast cancer lesion area based on the probability distribution heatmap.

2. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 1, characterized in that, Cross-modal coordinate registration is performed on the initial pathological morphological image data and the initial spatial transcriptome data to obtain target pathological morphological image data and target spatial transcriptome data in the same coordinate system, including: Based on affine transformation, coordinate transformation is performed on the initial spatial transcriptome data to obtain first spatial transcriptome data that is initially registered with the initial pathological morphological image data. Based on the free-deformation mesh model, the first spatial transcriptome data is non-rigidly registered to obtain the second spatial transcriptome data mapped to the coordinate system corresponding to the initial pathological morphological image data. The initial pathological morphological image data and the second spatial transcriptome data are subjected to a coordinate transformation to obtain target pathological morphological image data and target spatial transcriptome data in the same physical coordinate system.

3. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 1, characterized in that, Based on the target pathological morphological image data, constructing a pathological morphological image map corresponding to the pathological morphological image data includes: Non-tissue regions are removed from the target pathological morphological image data to obtain tissue region image data. The tissue region image data is divided into grids to obtain multiple grid regions; Based on preset multi-scale window parameters, feature extraction and cross-scale feature fusion are performed on the grid center of each grid region to obtain the target image features corresponding to each grid region. Based on the target image features corresponding to each grid region, the target adjacency matrix of the pathological morphological image is determined to characterize the pathological morphological image.

4. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 3, characterized in that, The process involves extracting features and fusing cross-scale features from the grid centers of each grid region based on preset multi-scale window parameters to obtain the target image features corresponding to each grid region, including: Based on preset multi-scale window parameters, the grid center of each grid region is offset and sampled multiple times to obtain multiple sampled region data corresponding to each grid region at each scale. Feature extraction is performed on multiple sampled regions corresponding to each grid region at each scale to obtain multiple sampled image features corresponding to each grid region at each scale. The mean value of multiple sampled image features corresponding to each grid region at each scale is calculated to obtain the average image features corresponding to each grid region at each scale. Cross-scale fusion is performed on the average image features at each scale corresponding to each grid region to obtain the target image features corresponding to each grid region.

5. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 1, characterized in that, Based on the target spatial transcriptome data, a spatial transcriptome map corresponding to the spatial transcriptome data is constructed, including: Based on the resolution of the target spatial transcriptome data, a single cell or cell spot is identified as a node in the spatial transcriptome map, and the nodes of the spatial transcriptome map correspond one-to-one with the coordinates of the target spatial transcriptome data. Linear dimensionality reduction and standardization are performed on the gene expression profile data in the target spatial transcriptome data to obtain gene expression characteristics; Based on the coordinate data in the target spatial transcriptome data and the gene expression characteristics, the target adjacency matrix of the spatial transcriptome map is determined to characterize the spatial transcriptome map.

6. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 3, characterized in that, The initial adjacency matrix of the pathological morphological image is represented as follows: ； in, This represents the initial adjacency matrix of the pathological morphological image. and This represents two different data indices within the same modality of data. and To represent different target image features. ; The initial adjacency matrix of the pathological morphology image is sparsified to obtain the target adjacency matrix of the pathological morphology image.

7. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 5, characterized in that, The initial adjacency matrix of the spatial transcriptome map is represented as follows: in, This represents the initial adjacency matrix of the spatial transcriptome map. and This represents two different data indices within the same modality of data. This represents the coordinate data in the target spatial transcriptome data. , Indicates gene expression characteristics, Weight parameters Used to adjust the importance of coordinate data and gene expression features in the graph construction process; The initial adjacency matrix of the spatial transcriptome graph is sparsified to obtain the target adjacency matrix of the spatial transcriptome graph.

8. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to any one of claims 1-7, characterized in that, The optimization objective function used to pre-train the target prediction neural network model is expressed as: In the formula, This represents the objective function to be optimized. This indicates a loss of oversight. This represents the internal consistency constraints of pathological morphological imaging modalities. This represents the internal consistency constraints of spatial transcriptome modalities. Represents the weight parameters. Represents the cross-modal weight matrix. The mapping results represent the pathological morphological imaging modalities. The mapping results representing spatial transcriptome modalities. Data index representing pathological morphological imaging modalities. Data index representing spatial transcriptomic modalities; in: In the formula, The target adjacency matrix represents the pathological morphological image. The Laplacian matrix representing a pathological morphological image. A mapping function representing pathological morphological image modalities; In the formula, The target adjacency matrix represents the spatial transcriptome map. The Laplacian matrix representing the spatial transcriptome map, Mapping functions representing spatial transcriptomic modes; In the formula, This represents the distance between two coordinates across modes. Used to control the scope of action corresponding to the space.

9. The breast cancer detection method based on cross-graph collaborative learning of morphological images and spatial transcriptomics according to claim 1, characterized in that, The step of detecting breast cancer lesion regions based on the probability distribution heatmap to obtain the target lesion region includes: Based on a preset probability threshold, the regions in the probability distribution heatmap with a probability higher than the preset probability threshold are identified as target lesion regions.

10. A breast cancer detection system based on cross-map collaborative learning of morphological images and spatial transcriptomics, characterized in that, include: The acquisition module is used to acquire initial pathological morphological imaging data and initial spatial transcriptome data for breast cancer detection. The registration module is used to perform cross-modal coordinate registration of the initial pathological morphological image data and the initial spatial transcriptome data to obtain target pathological morphological image data and target spatial transcriptome data in the same physical coordinate system. The graph construction module is used to construct, based on the target pathological morphological image data and the target spatial transcriptome data, respectively, a pathological morphological image map corresponding to the pathological morphological image data and a spatial transcriptome map corresponding to the spatial transcriptome data. The model prediction module is used to input the pathological morphological image and the spatial transcriptome map into a pre-trained target prediction neural network model and output a probability distribution heatmap. The region detection module is used to detect breast cancer lesion regions based on the probability distribution heatmap to obtain the target lesion region.

Images