Gene expression prediction model construction method and device, equipment and storage medium

Abstract

The invention relates to the technical field of medical image processing, and discloses a gene expression prediction model construction method and device, equipment and a storage medium, and the method comprises the steps: obtaining a multi-scale multi-mode gene expression data set; performing preprocessing, feature extraction and feature enhancement on the data set; respectively training the multilayer perceptron model according to the training data of the tissue level and the training data of the near-cell level to obtain pre-training models of the tissue level and the near-cell level; inputting training data of the tissue level and the near-cell level at the same time, and finely adjusting the pre-training model of the tissue level and the near-cell level; and adding the domain knowledge constraint into a loss function of the fine tuning model of the tissue level and the near-cell level to carry out model training to obtain a training model of the tissue level and the near-cell level, and generating a gene expression prediction model. According to the method, the problem that data processing of multi-scale difference cannot be compatible in the prior art is solved, and the accuracy and robustness of multi-scale gene expression prediction are improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method, apparatus, device, and storage medium for constructing a gene expression prediction model. Background Technology

[0002] Identifying cell subpopulations associated with key clinical phenotypes (such as tumor metastasis, treatment response, and survival outcome) is crucial for developing targeted cell therapies and discovering novel prognostic biomarkers. Whole-slide images (WSIs) are fundamental to clinical diagnosis and decision-making, offering advantages such as easy accessibility, relatively low cost, and large sample sizes, and naturally containing rich clinical phenotypic information (such as staging, survival outcome, and treatment response). However, the vast majority of WSI samples lack accompanying multi-scale gene expression data, thus limiting the molecular-level identification of phenotype-related cell subpopulations and their characteristic genes.

[0003] In existing technologies, deep learning has been widely applied to WSI-based gene expression prediction. Existing techniques, based on paired WSI and tissue-level expression data, have achieved cross-modal prediction of gene expression from pathological images. However, these methods only predict at the tissue level and fail to address the high heterogeneity within the tumor microenvironment at high resolution, i.e., intratumoral heterogeneity. On the other hand, with the development of spatial transcriptomics technology, methods such as ST-Net, mclSTExp, and iStar have emerged, utilizing spatial location data and high-resolution pathological images provided by spatial transcriptomics data to predict gene expression at the single-cell or near-cell level. However, these methods only focus on spatial transcriptomics data with a limited number of patient samples, failing to integrate and utilize the widely available tissue-level WSI-RNA paired data, and ignoring the intertumoral heterogeneity information contained within.

[0004] Current technologies still lack methods to unify and integrate multimodal data at the tissue and proximal cellular levels, making it impossible to simultaneously utilize the intertumoral and intratumoral heterogeneity reflected by these two types of data. This lack of multi-scale integration capability not only limits the performance of gene expression prediction models but also restricts cross-scale gene expression similarity calculations, thus hindering the systematic identification of cellular subpopulations and their characteristic molecules related to clinical phenotypes at the molecular level. Furthermore, due to significant distributional differences between data at different scales—for example, the presence of numerous zero values ​​in proximal cellular gene expression and the vast difference in sample size between the two scales—even with strategies such as transfer learning, existing prediction models based on a single scale still struggle to achieve effective multi-scale data fusion and joint prediction. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, the present invention aims to provide a method, apparatus, device and storage medium for constructing a gene expression prediction model, which achieves accurate and robust multi-scale gene expression prediction by effectively integrating multi-scale pathological images and gene expression data.

[0006] The first aspect of this invention provides a method for constructing a gene expression prediction model, comprising: A multi-scale, multi-modal gene expression dataset is obtained, including a first image, first data, a second image, and second data. The first image is a pathological image at the tissue level, the first data is gene expression data at the tissue level, the second image is a pathological image at the near-cellular level, and the second data is gene expression data at the near-cellular level. The first image, first data, second image, and second data are preprocessed to obtain the first image patch, the first normalized data, the second image patch, and the second normalized data. The second standardized data is aggregated in the spatial dimension to obtain the third standardized data at the pseudo-organization level; Feature extraction is performed on the first and second image patches respectively to obtain the first feature vector and the second feature vector; The global information of the second feature vector is aggregated, and the global information of the second feature vector and the second feature vector are input into the Vision Mamba model to obtain local features and global features. First training data is generated based on the first feature vector and the first standardized data. The first training data is used to train the preset first multilayer perceptron model to obtain the first pre-trained model. Second training data is generated based on local features, global features, second standardized data, and third standardized data. The second training data is used to train the pre-set second multilayer perceptron model to obtain the second pre-trained model. The first training data and the second training data are jointly input into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model. Domain knowledge constraints are added to the loss functions of the first and second fine-tuned models. The first and second fine-tuned models are then trained to obtain the first and second training models. A gene expression prediction model is then generated based on the first and second training models.

[0007] Optionally, in a first implementation of the first aspect of the present invention, the preprocessing of the first image, first data, second image, and second data to obtain a first image patch, first standardized data, a second image patch, and second standardized data includes: The first data was converted into the number of transcripts per million to obtain the first standardized data; The second data is converted to counts per million to obtain the second standardized data; Cut the first image to obtain the first image patch; Cropping the second image yields the second tile.

[0008] Optionally, in a second implementation of the first aspect of the present invention, the aggregation of global information of the second feature vector, inputting the second feature vector and its global information into the Vision Mamba model, and outputting local and global features, includes: Learnable CLS labels are incorporated into the second feature vector to aggregate global information, resulting in the input sequence; The input sequence is fed into the Vision Mamba model, and the output includes local and global features.

[0009] Optionally, in a third implementation of the first aspect of the present invention, the step of generating first training data based on the first feature vector and the first standardized data, and training a preset first multilayer perceptron model using the first training data to obtain a first pre-trained model, includes: First training data is generated based on the first feature vector and the first standardized data; Input the first feature vector into the first multilayer perceptron model to obtain the first predicted value; Based on the first predicted value and the first standardized data, the first loss function is calculated; The first multilayer perceptron model is trained based on the first loss function and the first training data to obtain the first pre-trained model.

[0010] Optionally, in a fourth implementation of the first aspect of the present invention, the step of generating second training data based on local features, global features, second standardized data, and third standardized data, and training a preset multilayer perceptron model using the second training data to obtain a second pre-trained model, includes: Second training data is generated based on local features, global features, second standardized data, and third standardized data. Input local features into the second multilayer perceptron model to obtain the second predicted value; Input global features into the second multilayer perceptron model to obtain the third predicted value; Based on the second predicted value and the second standardized data, the local loss function is calculated; The global loss function is calculated based on the third predicted value and the third standardized data; The sum of the local loss function and the global loss function is calculated to obtain the second loss function; The second multilayer perceptron model is trained based on the second loss function and the second training data to obtain the second pre-trained model.

[0011] Optionally, in a fifth implementation of the first aspect of the present invention, the step of jointly inputting the first training data and the second training data into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuning model and the second fine-tuning model includes: The second training data is upsampled to obtain the third training data, and the number of samples in the third training data is an integer multiple of the number of samples in the first training data. The first training data and the third training data are jointly input into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model.

[0012] Optionally, in a sixth implementation of the first aspect of the present invention, the step of adding domain knowledge constraints to the loss function of the first fine-tuning model and the loss function of the second fine-tuning model, training the first fine-tuning model and the second fine-tuning model to obtain a first training model and a second training model, and generating a gene expression prediction model based on the first training model and the second training model, includes: Based on the graph Laplacian regularization framework, spatial consistency constraints are obtained; Based on prior knowledge of gene pathways, constraints on pathway co-expression are obtained; Domain knowledge constraints are generated based on spatial consistency constraints and pathway co-expression constraints. Domain knowledge constraints are added to the loss function of the first fine-tuning model and the loss function of the second fine-tuning model in the form of regularization. The first fine-tuning model and the second fine-tuning model are trained to obtain the first training model and the second training model. Gene expression prediction models are generated based on the first and second training models.

[0013] A second aspect of the present invention provides a gene expression prediction model construction apparatus, comprising: The data acquisition module is used to acquire a multi-scale, multi-modal gene expression dataset, including a first image, first data, a second image, and second data. The first image is a pathological image at the tissue level, the first data is gene expression data at the tissue level, the second image is a pathological image at the near-cellular level, and the second data is gene expression data at the near-cellular level. The data preprocessing module is used to preprocess the first image, the first data, the second image, and the second data to obtain the first image patch, the first standardized data, the second image patch, and the second standardized data. The pseudo-organization generation module is used to aggregate the second-standardized data in the spatial dimension to obtain the third-standardized data at the pseudo-organization level. The feature extraction module is used to extract features from the first image patch and the second image patch respectively, to obtain the first feature vector and the second feature vector; The feature enhancement module is used to aggregate the global information of the second feature vector. It inputs the second feature vector and its global information into the Vision Mamba model and outputs local features and global features. The first pre-training module is used to generate first training data based on the first feature vector and the first standardized data, and to train the preset first multilayer perceptron model using the first training data to obtain the first pre-trained model. The second pre-training module is used to generate second training data based on local features, global features, second standardized data and third standardized data, and to train the preset second multilayer perceptron model using the second training data to obtain the second pre-trained model. The joint training module is used to jointly input the first training data and the second training data into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model. The domain knowledge constraint module is used to add domain knowledge constraints to the loss function of the first fine-tuning model and the loss function of the second fine-tuning model, train the first fine-tuning model and the second fine-tuning model to obtain the first training model and the second training model, and generate a gene expression prediction model based on the first training model and the second training model.

[0014] A third aspect of the present invention provides a gene expression prediction model construction device, the gene expression prediction model construction device comprising: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the gene expression prediction model building device to perform the steps of the gene expression prediction model building method described above.

[0015] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the gene expression prediction model construction method described in any of the preceding claims.

[0016] In the technical solution of this invention, a multi-scale, multi-modal gene expression dataset is first acquired, and data preprocessing and feature vector extraction are performed to obtain feature vectors and gene expression standardized data of pathological patches at the tissue level and near-cellular level. Pseudo-tissue level gene expression standardized data is generated, and global information of feature vectors of pathological patches at the near-cellular level is aggregated. The feature vectors and global information are input into the Vision Mamba model to obtain local and global features. Model training includes training pre-trained models at the tissue level and near-cellular level respectively. Then, through joint training and domain knowledge constraint optimization, the processing of multi-scale, multi-modal data is realized, and a unified framework is established to effectively integrate multi-scale pathological images and gene expression data. The prediction results of the constructed model have both high accuracy and biological rationality. This method solves the problem that existing technologies cannot handle data processing with multi-scale differences, and improves the accuracy and robustness of multi-scale gene expression prediction. Attached Figure Description

[0017] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a first flowchart of a gene expression prediction model construction method provided in an embodiment of the present invention; Figure 2 This is a second flowchart of the gene expression prediction model construction method provided in the embodiments of the present invention; Figure 3 This is a third flowchart of the gene expression prediction model construction method provided in the embodiments of the present invention; Figure 4 This is a fourth flowchart of the gene expression prediction model construction method provided in the embodiments of the present invention; Figure 5 This is the fifth flowchart of the gene expression prediction model construction method provided in the embodiments of the present invention; Figure 6 This is the sixth flowchart of the gene expression prediction model construction method provided in the embodiments of the present invention; Figure 7 This is the seventh flowchart of the gene expression prediction model construction method provided in the embodiments of the present invention; Figure 8 This is a schematic diagram of the structure of the gene expression prediction model construction device provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of the gene expression prediction model construction device provided in an embodiment of the present invention. Detailed Implementation

[0018] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the gene expression prediction model construction method in this invention includes: 101. Obtain a multi-scale, multi-modal gene expression dataset, including a first image, first data, a second image, and second data. The first image is a pathological image at the tissue level, the first data is gene expression data at the tissue level, the second image is a pathological image at the near-cellular level, and the second data is gene expression data at the near-cellular level. In this embodiment, obtaining a multi-scale, multi-modal gene expression dataset includes the following steps: Acquire tissue-level paired pathological images and gene expression data; Acquire near-cellular level paired pathological images and gene expression data.

[0020] In this embodiment, tissue-level paired WSI and transcriptome data, as well as near-cellular spatial transcriptome data, were obtained from public databases such as TCGA, HTAN, and CPTAC. This covered three cancer types: invasive breast cancer (BRCA), colon adenocarcinoma (COAD), and kidney clear cell carcinoma (KIRC), with a clear distinction between the model training set and the independent validation set. Specifically, for tissue-level pathological images and gene expression data, paraffin-embedded WSI images and matched RNA-seq data for all three cancer types were obtained from the TCGA database. Model training used only diagnostic sections of tumor tissue, excluding adjacent normal tissue. For near-cellular pathological images and gene expression data, the focus was primarily on the large-volume multimodal data based on Visium 1.0 spatial transcriptome technology from public databases. Near-cellular sequencing point RNA sequencing data and their paired pathological images for all three cancer types were obtained from published literature included in the HTAN database.

[0021] This embodiment obtains a multi-scale, multi-modal gene expression dataset, and pairs pathological images with gene expression data to provide a data foundation for subsequent model training.

[0022] 102. Preprocess the first image, the first data, the second image, and the second data to obtain the first image patch, the first standardized data, the second image patch, and the second standardized data; In this embodiment, a preprocessing procedure is performed on the multi-scale, multi-modal gene expression dataset, and the system achieves standardization and feature unification of multi-scale data. Specifically, the first image patch is a tissue-level pathological image obtained after preprocessing, and the first standardized data is tissue-level gene expression standardized data. The second image patch is a near-cellular level pathological image obtained after preprocessing, and the second standardized data is near-cellular level gene expression standardized data.

[0023] 103. Aggregate the second standardized data in the spatial dimension to obtain the third standardized data at the pseudo-organization level; In this embodiment, the third normalized data is pseudo-tissue-level gene expression normalized data obtained by spatially aggregating near-cellular level gene expression normalized data, which serves as the global supervision target in subsequent model training. Specifically, pseudo-tissue-level gene expression normalized data for the corresponding sample is generated by summing the gene counts of all near-cellular level RNA-seq data in each spatial sample and calculating the CPM, i.e., the third normalized data, which serves as the global supervision target in subsequent multi-scale learning. In this embodiment, by constructing pseudo-tissue-level gene expression normalized data, a data foundation is provided for subsequent model training.

[0024] 104. Extract features from the first and second image patches respectively to obtain the first feature vector and the second feature vector; In this embodiment, the first feature vector is the tissue-level feature vector obtained by feature extraction of pathological patches at the tissue level, and the second feature vector is the near-cellular-level feature vector obtained by feature extraction of pathological patches at the near-cellular level. The pre-trained pathological model CONCH is used to convert the tissue-level and near-cellular-level pathological patches into high-dimensional feature vectors (dimension 512), thus obtaining the tissue-level and near-cellular-level feature vectors, and transforming the image information into a sequential feature representation suitable for deep model processing.

[0025] 105. Aggregate the global information of the second feature vector, input the global information of the second feature vector and the second feature vector into the Vision Mamba model, and output the local features and global features; In this embodiment, by aggregating the global information of the second vector, the global information of the second feature vector and the second feature vector is used as an input sequence and input into the Vision Mamba model to fuse the features of the pathological image, and the output is local features and global features.

[0026] 106. Generate first training data based on the first feature vector and the first standardized data, and train the preset first multilayer perceptron model using the first training data to obtain the first pre-trained model; In this embodiment, the first pre-trained model is an organization-level pre-trained model. The first multilayer perceptron model is trained using organization-level training data to obtain the organization-level pre-trained model, laying the foundation for subsequent joint training.

[0027] 107. Generate second training data based on local features, global features, second standardized data, and third standardized data. Train the pre-set second multilayer perceptron model using the second training data to obtain the second pre-trained model. In this embodiment, the second pre-trained model is a near-cellular level pre-trained model. Using near-cellular level training data, the second multilayer perceptron model is trained to obtain the near-cellular level pre-trained model, laying the foundation for subsequent joint training.

[0028] 108. Input the first training data and the second training data together into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model; In this embodiment, the first fine-tuning model is a tissue-level fine-tuning model obtained through joint training of a tissue-level pre-trained model, and the second fine-tuning model is a near-cellular level fine-tuning model obtained through joint training of a near-cellular level pre-trained model. By inputting tissue-level training data and near-cellular level training data, the tissue-level pre-trained model and the near-cellular level pre-trained model are jointly trained to obtain the tissue-level fine-tuning model and the near-cellular level fine-tuning model, thus fusing multi-scale information.

[0029] 109. Add domain knowledge constraints to the loss function of the first fine-tuning model and the loss function of the second fine-tuning model, train the first fine-tuning model and the second fine-tuning model to obtain the first training model and the second training model, and generate a gene expression prediction model based on the first training model and the second training model.

[0030] In this embodiment, the first training model is a tissue-level training model obtained by incorporating domain knowledge constraints into the fine-tuned tissue-level model, and the second training model is a near-cellular-level training model obtained by incorporating domain knowledge constraints into the fine-tuned near-cellular-level model. By adding domain knowledge constraints to the loss function, the tissue-level and near-cellular-level fine-tuned models are trained to obtain the tissue-level and near-cellular-level training models, which together form the gene expression prediction model. Incorporating domain knowledge constraints avoids the model falling into overfitting due to purely data-driven approaches, ensuring that the prediction results conform to biomedical principles.

[0031] In this embodiment of the invention, a multi-scale, multi-modal gene expression dataset is first acquired, and data preprocessing and feature vector extraction are performed to obtain feature vectors and gene expression normalization data of pathological patches at the tissue and near-cellular levels. The gene expression normalization data at the near-cellular level is aggregated in the spatial dimension to obtain pseudo-tissue level gene expression normalization data. For the feature vectors of pathological patches at the near-cellular level, global information is aggregated. The feature vectors and global information of pathological patches at the near-cellular level are input into the Vision Mamba model to obtain local and global features. Pre-trained models at the near-cellular level and tissue level are trained respectively. Then, through joint training and domain knowledge constraint optimization, a gene expression prediction model is generated, realizing the processing of multi-scale, multi-modal data, establishing a unified framework to effectively integrate multi-scale pathological images and gene expression data, and the prediction results of the constructed model have both high accuracy and biological rationality. This method solves the problem that existing technologies cannot handle data processing with multi-scale differences, and improves the accuracy and robustness of multi-scale gene expression prediction.

[0032] Please see Figure 2 One embodiment of the gene expression prediction model construction method in this invention includes: 201. The first data is converted into the number of transcripts per million to obtain the first standardized data; 202. Convert the second data into counts per million to obtain the second standardized data; In this embodiment, for the first data, i.e., tissue-level gene expression data, the gene counts of the original RNA-seq data are standardized to counts per million transcripts (TPM), which also serve as the global supervision target in multi-scale learning. For the second data, i.e., near-cellular-level gene expression data, such as spatial transcriptome data (e.g., Visium 1.0), the gene counts of the original RNA-seq data are first standardized to counts per million (CPM), which serve as the local supervision target in subsequent multi-scale learning. Given that gene expression values ​​are distributed across orders of magnitude and mean squared error loss is used for training, the values ​​of counts per million transcripts and counts per million are logarithmically transformed to reduce the dominant influence of highly expressed genes on the training process and improve the data training effect. In addition, the prediction target is limited to protein-coding genes to enhance the biological relevance of the model. Biologically consistent standardization is performed on gene expression data from different resolutions to construct training samples suitable for a unified learning framework, which is beneficial for subsequent model construction.

[0033] 203. Cut the first image to obtain the first image patch; 204. Crop the second image to obtain the second tile.

[0034] In this embodiment, the first image is a tissue-level pathological image, and the second image is a near-cellular level pathological image. The first image patch is a tissue-level pathological patch, and the second image patch is a near-cellular level pathological patch. Both the tissue-level and near-cellular level pathological images are downsampled to a magnification of 20 times (0.5 μm / pixel), and appropriate segmentation strategies are applied to each. For the tissue-level pathological image, it is segmented into non-overlapping 256×256 pixel (128 μm×128 μm) tissue-level pathological pixel patches, sorted according to their x and y coordinate sequences, for model training. For the near-cellular level pathological image, a 256×256 pixel region is cropped from the pathological image, centered on the coordinates of the sequencing point, to serve as the near-cellular level pathological pixel patch corresponding to that sequencing point. To reduce background interference, patches with a tissue proportion of less than 15% are filtered out. By converting the pathological image into processable pathological patches and reducing the proportion of irrelevant patches, the quality of the training data is improved.

[0035] Please see Figure 3 One embodiment of the gene expression prediction model construction method in this invention includes: 301. Introduce the learnable CLS label into the second feature vector to aggregate global information and obtain the input sequence; 302. Input the input sequence into Vision Mamba encoding, and output the local features and global features.

[0036] In this embodiment, the second feature vector is a near-cellular level feature vector. x i ,make (in , ), representing a sequence consisting of n pathological tile embeddings. Vision Mamba is used to fuse pathological tile features via a first-order causal recursive equation. This process is achieved by updating the hidden state and generating the output: in, , and They represent the first Input, hidden state, and output of a pathological tile; , , and These are the memory decay coefficient, input injection coefficient, read coefficient, and gating coefficient, respectively. (The text abruptly ends here, likely due to an incomplete sentence or missing information.) A linear transformation yields: In addition, a learnable parameter is added to the end of the input sequence. The entire sequence, after being encoded using Vision Mamba with tags to aggregate global information, is represented as follows: in, For the output local features, This is the global feature output.

[0037] Traditional Transformers suffer from high computational complexity, making it difficult to handle long sequences in high-resolution WSI (Wide Sensor Inference). This embodiment introduces Vision Mamba as its infrastructure, leveraging its first-order causal recursion mechanism to achieve efficient modeling of long sequence dependencies with linear computational complexity. Based on Vision Mamba's cross-scale feature fusion gating mechanism, this mechanism can effectively integrate multi-scale feature information at both the tissue and near-cellular levels.

[0038] Please see Figure 4 One embodiment of the gene expression prediction model construction method in this invention includes: 401. Generate first training data based on the first feature vector and the first standardized data; 402. Input the first feature vector into the first multilayer perceptron model to obtain the first predicted value; 403. Based on the first predicted value and the first standardized data, the first loss function is calculated; 404. Train the first multilayer perceptron model based on the first loss function and the first training data to obtain the first pre-trained model.

[0039] In this embodiment, for constructing a pre-trained model at the tissue level, the first feature vector w is input. i， Gene expression predictions are obtained by mapping using a corresponding multilayer perceptron (MLP). : Calculate the global (organizational level) loss using Mean Squared Error (MSE): Among them, for the organization-level pre-trained model, This represents the first standardized data, namely, tissue-level gene expression standardized data.

[0040] For a tissue-level pre-trained model that uses only tissue-level gene expression normalized data, the optimization objective is the loss function of the tissue-level pre-trained model. .

[0041] The training yields a pre-trained model at the organizational level, laying the foundation for subsequent joint training and improving the model's capabilities.

[0042] Please see Figure 5 One embodiment of the gene expression prediction model construction method in this invention includes: 501. Generate second training data based on local features, global features, second standardized data, and third standardized data; 502. Input local features into the second multilayer perceptron model to obtain the second predicted value; 503. Input global features into the second multilayer perceptron model to obtain the third predicted value; 504. Based on the second predicted value and the second standardized data, the local loss function is calculated; 505. Based on the third predicted value and the third standardized data, the global loss function is calculated; 506. Calculate the sum of the local loss function and the global loss function to obtain the second loss function; 507. Train the second multilayer perceptron model based on the second loss function and the second training data to obtain the second pre-trained model.

[0043] In this embodiment, for the construction of a pre-trained model at the near-cellular level, the local features output by the Vision Mamba model... With global features (All dimensions are 512) and mapped to gene expression prediction values ​​using the corresponding multilayer perceptron (MLP): in, The local features are input to the gene expression prediction values ​​output by the multilayer perceptron model. The global features are input to the gene expression prediction values ​​output by the multilayer perceptron.

[0044] The mean squared error (MSE) is used to calculate local (near-cellular level) loss and global (pseudo-tissue level) loss: Among them, for pre-trained models at the near-cellular level, z i With z global Let represent the second and third normalized data, namely, gene expression normalized data at the near-cellular level and gene expression normalized data at the pseudo-tissue level, respectively. For the pre-trained model trained using the second training data, the optimization objective is the loss function of the near-cellular level pre-trained model. .

[0045] In this embodiment, a multilayer perceptron model is trained using the second training data to obtain a pre-trained model at the near-cellular level. The loss function combines near-cellular and pseudo-tissue level data to improve the model's ability to process multi-scale data, laying the foundation for subsequent joint training and enhancing the model's capabilities.

[0046] Please see Figure 6 One embodiment of the gene expression prediction model construction method in this invention includes: 601. Upsample the second training data to obtain the third training data. The number of samples in the third training data is an integer multiple of the number of samples in the first training data. 602. Input the first training data and the third training data into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model.

[0047] In this embodiment, the fine-tuning stage is based on a tissue-level pre-trained model and a near-cellular-level pre-trained model. For both the first and second pre-trained models, first and second training data are simultaneously input for joint training. To achieve effective joint training, samples from the near-cellular-level training data are upsampled to obtain third training data. The number of samples in the third training data is an integer multiple of the number of samples in the tissue-level training data, ensuring that each training step can process one tissue-level and one near-cellular-level sample simultaneously. During the fine-tuning stage, the loss function of the tissue-level fine-tuning model... Loss function of near-cellular level fine-tuning model The calculation method is as follows: Wherein, the hyperparameter λ is the reciprocal of the upsampling factor of the second training data, used to balance the contribution of data sample sizes at different levels: for the BRCA dataset, Set it to 0.1; for the COAD and KIRC datasets, All values ​​are taken as 0.05. The loss function is calculated using training data at the near-cellular level. The loss function is calculated using training data at the organizational level.

[0048] By jointly inputting the first and second training data and simultaneously performing gene expression prediction, cross-scale feature sharing is achieved, enhancing the model's ability to characterize tumor heterogeneity and improving the accuracy of predicting tissue-level and near-cellular-level gene expression from pathological images.

[0049] Please see Figure 7One embodiment of the gene expression prediction model construction method in this invention includes: 701. Based on the graph Laplacian regularization framework, spatial consistency constraints are obtained; In this embodiment, spatial consistency constraints are based on a graph Laplacian regularization framework, aiming to ensure that the model maintains spatial smoothness of expression values ​​in near-cellular level predictions, avoiding unreasonable and drastic fluctuations in gene expression between adjacent sequencing sites. The specific form of the spatial consistency constraint is as follows: in, This represents the spatial consistency constraint, where E represents the set of all adjacent sequencing point pairs; Points in the adjacency matrix and The elements that connect weights can be defined based on spatial distance or organizational structure; and These represent the gene expression vectors predicted by the model for adjacent sequencing sites; This constraint term is used to balance hyperparameters and adjust their contribution to the total loss. It penalizes the sum of squared differences in expression between adjacent points, thus encouraging a more spatially continuous expression pattern in the model output.

[0050] By incorporating the Graph Laplace regularization framework, the biological rationality and generalization ability of the model's predictions are improved.

[0051] 702. Based on prior knowledge of gene pathways, pathway co-expression constraints are obtained; In this embodiment, pathway co-expression constraints introduce prior knowledge of gene pathways (such as KEGG) to ensure that genes within the same pathway maintain a statistically correlated structure in predicted expression within the model, thereby enhancing the biological rationality of gene module behavior. Pathway co-expression constraints are defined as follows: in, This indicates pathway co-expression constraints. Total number of pathways; Indicates the first The predicted expression matrix of all genes in each pathway on the sample; A function for calculating the covariance matrix; The prior covariance matrix of this pathway can be learned from a public transcriptome database or artificially set based on the pathway structure. β To balance hyperparameters, this constraint ensures that the patterns of coordinated changes among genes conform to the known behavior of functional modules by comparing predicted covariance with biological prior covariance.

[0052] By incorporating prior knowledge of gene pathways, the biological rationality and generalization ability of model predictions can be improved.

[0053] 703. Generate domain knowledge constraints based on spatial consistency constraints and path co-expression constraints; 704. Add domain knowledge constraints in the form of regularization to the loss function of the first fine-tuning model and the loss function of the second fine-tuning model, and train the first fine-tuning model and the second fine-tuning model to obtain the first training model and the second training model. In this embodiment, the loss function integrates multi-scale prediction loss with the aforementioned domain knowledge constraints. The loss function for the tissue-level training model and the loss function for the near-cellular-level training model are extended in form as follows: in, The loss function for the training model at the organization level. The loss function for training models at the near-cellular level. The loss function for fine-tuning the model at the organization level. The loss function for fine-tuning the model at the near-cellular level. Represents spatial consistency constraints. This indicates co-expression constraints of the pathway.

[0054] Based on the loss function with domain knowledge constraints, the first fine-tuning model and the second fine-tuning model are trained to obtain the first training model and the second training model. The first training model is a tissue-level training model, and the second training model is a near-cell-level training model.

[0055] 705. Generate a gene expression prediction model based on the first training model and the second training model.

[0056] In this embodiment, a gene expression prediction model is composed of a first training model and a second training model. Within a unified framework, the gene expression prediction model includes a first training model for processing tissue-level samples and a second training model for processing near-cellular-level samples. The model selects the appropriate training model based on the scale of the input sample to output either tissue-level or near-cellular-level gene expression prediction results. When the input data is tissue-level sample data, the first training model outputs tissue-level gene expression prediction values; when the input data is near-cellular-level sample data, the second training model outputs near-cellular-level gene expression prediction values. Alternatively, the first and second training models can be used simultaneously to output weighted gene expression prediction results. After inputting tissue-level or near-cellular-level sample data into the gene expression prediction model, the first and second training models output gene expression prediction values ​​respectively. These values ​​are then integrated using weighting or other methods. For example, when the input data is tissue-level sample data, the first training model's output result is given a higher weight; when the input data is near-cellular-level sample data, the second training model's output result is given a higher weight. The weighted calculation then outputs the final gene expression prediction value.

[0057] In this embodiment, spatial consistency constraints and pathway co-expression constraints are further introduced to embed biological prior knowledge into the optimization process, thereby improving the biological rationality and generalization ability of the model's prediction results.

[0058] In this embodiment, after obtaining the gene expression prediction model through the gene expression prediction model construction method, the input pathological image data can be processed by the model to obtain gene expression prediction values ​​at the tissue level or near-cellular level. The model can then be used for downstream applications. (1) Identification of cancer recurrence-related genes at the tissue level Based on WSI data from TCGA, tissue-level gene expression prediction values ​​were obtained. A LASSO Cox regression model was then used to construct a gene expression model for predicting cancer recurrence. Genes with non-zero coefficients were selected by using the optimal λ value. A risk score was calculated by linearly combining the expression levels of selected genes with their corresponding coefficients, as shown in the following formula: in Represents gene coefficient, This represents the predicted gene expression value. Patients were divided into low-risk and high-risk groups based on the median risk score, and the log-rank test was used to assess the significance of the difference in relapse-free survival between the two groups. After the model was built, it will be validated on the independent validation set CPTAC to evaluate its generalization ability and reproducibility.

[0059] (2) Annotating cell identity at the near-cellular level For gene expression predictions at sequencing sites, the deconvolutional CARD method was employed. Using cancer-type-specific single-cell RNA sequencing data as a reference, the proportion of each cell type at each sequencing site was estimated, thereby annotating cell identity. After running CARD with default parameters, the correlation between the predicted cell proportions and the actual values ​​was evaluated. Furthermore, the cell type with the highest proportion was used as the predicted label for the sequencing site, and the consistency between the predicted cell type label and the actual label was assessed by calculating the AUC value.

[0060] (3) Phenotype-related cell subsets and their characteristic molecules Based on the predicted gene expression values ​​at the tissue and near-cellular (sequencing site) levels obtained from WSI prediction, the Scissor algorithm was further employed to identify cell subpopulations associated with specific phenotypes from single-cell data based on gene expression similarity. The following two phenotypes were the focus of the study: ① Tumor / Normal Phenotype (Binary Data): In each cancer type, cancerous tissue samples are coded as 1, and adjacent normal samples are coded as 0. Scissor+ cells identified by Scissor represent a subpopulation associated with the malignant phenotype, while Scissor- cells represent a subpopulation associated with the normal phenotype.

[0061] ② Tumor recurrence-related phenotypes (survival data): Among the identified cancer cells, cell subpopulations were further distinguished into those with high recurrence risk (Scissor+, poor prognosis) and low recurrence risk (Scissor-, good prognosis). These subpopulations were identified by cell phenotypes that were significantly associated with survival in the Cox regression model.

[0062] Furthermore, this embodiment uses the analysis results of real gene expression values ​​as a reference to systematically evaluate the accuracy of identifying phenotype-related cell subpopulations based on predicted expression values ​​and the reliability of identifying related characteristic genes from multiple dimensions, such as the consistency of differentially expressed genes in Scissor+ and Scissor- cell subpopulations, the consistency of enriched pathways, and prognostic prediction efficacy.

[0063] The gene expression prediction model construction method in the embodiments of the present invention has been described above. The gene expression prediction model construction apparatus in the embodiments of the present invention will be described below. Please refer to [link / reference]. Figure 8 One embodiment of the gene expression prediction model construction device of the present invention includes: The data acquisition module 801 is used to acquire a multi-scale, multi-modal gene expression dataset, including a first image, first data, a second image, and second data. The first image is a pathological image at the tissue level, the first data is gene expression data at the tissue level, the second image is a pathological image at the near-cellular level, and the second data is gene expression data at the near-cellular level. The data preprocessing module 802 is used to preprocess the first image, the first data, the second image, and the second data to obtain the first image patch, the first standardized data, the second image patch, and the second standardized data. The pseudo-organization generation module 803 is used to aggregate the second standardized data in the spatial dimension to obtain the third standardized data at the pseudo-organization level. The feature extraction module 804 is used to extract features from the first patch and the second patch respectively to obtain a first feature vector and a second feature vector. The feature enhancement module 805 is used to aggregate the global information of the second feature vector, input the global information of the second feature vector and the second feature vector into the Vision Mamba model, and output local features and global features. The first pre-training module 806 is used to generate first training data based on the first feature vector and the first standardized data, and to train the preset first multilayer perceptron model using the first training data to obtain the first pre-trained model. The second pre-training module 807 is used to generate second training data based on local features, global features, second standardized data and third standardized data, and to train the preset second multilayer perceptron model using the second training data to obtain the second pre-trained model. The joint training module 808 is used to jointly input the first training data and the second training data into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model. The domain knowledge constraint module 809 is used to add domain knowledge constraints to the loss function of the first fine-tuning model and the loss function of the second fine-tuning model, train the first fine-tuning model and the second fine-tuning model to obtain the first training model and the second training model, and generate a gene expression prediction model based on the first training model and the second training model.

[0064] In this embodiment, a multi-scale, multi-modal gene expression dataset is acquired by the data acquisition module 801, including first data, a first image, second data, and a second image. The first image is a pathological image at the tissue level, the first data is gene expression data at the tissue level, and the second image is a pathological image at the near-cellular level, while the second data is gene expression data at the near-cellular level. The data preprocessing module 802 preprocesses the first data, the first image, the second data, and the second image to obtain a first patch, first normalized data, a second patch, and second normalized data. The pseudo-tissue generation module 803 aggregates the second normalized data in the spatial dimension to obtain pseudo-tissue level third normalized data. The feature extraction module 804 extracts features from the first patch and the second patch to obtain a first feature vector and a second feature vector. The feature enhancement module 805 aggregates the global information of the second feature vector and inputs the global information of the second feature vector and the second feature vector into Vision. The Mamba model outputs local and global features. A first pre-training module 806 generates first training data based on a first feature vector and first standardized data. This first training data is used to train a pre-defined first multilayer perceptron model, resulting in a first pre-trained model. A second pre-training module 807 generates second training data based on local features, global features, second standardized data, and third standardized data. This second training data is used to train a pre-defined second multilayer perceptron model, resulting in a second pre-trained model. A joint training module 808 inputs the first and second training data into both the first and second pre-trained models for joint training, resulting in a first fine-tuned model and a second fine-tuned model. A domain knowledge constraint module 809 incorporates domain knowledge constraints into the loss functions of both the first and second fine-tuned models, training them to obtain a first training model and a second training model. A gene expression prediction model is then generated based on these two training models.

[0065] This device enables the processing of multi-scale and multi-modal data, establishes a unified framework to effectively integrate multi-scale pathological images and gene expression data, and constructs a gene expression prediction model whose prediction results have both high accuracy and biological rationality. It solves the problem that existing technologies cannot handle multi-scale differences in data processing, and improves the accuracy and robustness of multi-scale gene expression prediction.

[0066] Figure 9This is a schematic diagram of the structure of a gene expression prediction model construction device 900 provided in an embodiment of the present invention. This gene expression prediction model construction device can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 910 (e.g., one or more processors) and a memory 920, and one or more storage media 930 (e.g., one or more mass storage devices) for storing application programs 933 or data 932. The memory 920 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the gene expression prediction model construction device 900. Furthermore, the processor 910 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the gene expression prediction model construction device 900 to implement the steps of the gene expression prediction model construction method provided in the above-described method embodiments.

[0067] The gene expression prediction model construction device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input / output interfaces 960, and / or one or more operating systems 931, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 9 The illustrated structure of the gene expression prediction model building device does not constitute a limitation on the gene expression prediction model building device. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0068] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of a gene expression prediction model construction method.

[0069] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0070] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0071] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for constructing a gene expression prediction model, characterized in that, include: A multi-scale, multi-modal gene expression dataset is obtained, including a first image, first data, a second image, and second data. The first image is a pathological image at the tissue level, the first data is gene expression data at the tissue level, the second image is a pathological image at the near-cellular level, and the second data is gene expression data at the near-cellular level. The first image, first data, second image, and second data are preprocessed to obtain the first image patch, the first normalized data, the second image patch, and the second normalized data. The second standardized data is aggregated in the spatial dimension to obtain the third standardized data at the pseudo-organization level; Feature extraction is performed on the first and second image patches respectively to obtain the first feature vector and the second feature vector; The global information of the second feature vector is aggregated, and the second feature vector and its global information are input into the Vision Mamba model to obtain local features and global features. First training data is generated based on the first feature vector and the first standardized data. The first training data is used to train the preset first multilayer perceptron model to obtain the first pre-trained model. Second training data is generated based on local features, global features, second standardized data, and third standardized data. The second training data is used to train the pre-set second multilayer perceptron model to obtain the second pre-trained model. The first training data and the second training data are jointly input into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model. Domain knowledge constraints are added to the loss functions of the first and second fine-tuned models. The first and second fine-tuned models are then trained to obtain the first and second training models. A gene expression prediction model is then generated based on the first and second training models.

2. The method for constructing a gene expression prediction model according to claim 1, characterized in that: The preprocessing of the first data, the first image, the second data, and the second image to obtain the first image patch, the first normalized data, the second image patch, and the second normalized data includes: The first data was converted into the number of transcripts per million to obtain the first standardized data; The second data is converted to counts per million to obtain the second standardized data; Cut the first image to obtain the first image patch; Cropping the second image yields the second image patch.

3. The method for constructing a gene expression prediction model according to claim 1, characterized in that: The global information of the aggregated second feature vector is input into the Vision Mamba model, and the outputs are local features and global features, including: Learnable CLS labels are incorporated into the second feature vector to aggregate global information, resulting in the input sequence; The input sequence is fed into the Vision Mamba model, and the output includes local and global features.

4. The method for constructing a gene expression prediction model according to claim 1, characterized in that: The step of generating first training data based on a first feature vector and first standardized data, and training a preset first multilayer perceptron model using the first training data to obtain a first pre-trained model includes: First training data is generated based on the first feature vector and the first standardized data; Input the first feature vector into the first multilayer perceptron model to obtain the first predicted value; Based on the first predicted value and the first standardized data, the first loss function is calculated; The first multilayer perceptron model is trained based on the first loss function and the first training data to obtain the first pre-trained model.

5. The method for constructing a gene expression prediction model according to claim 1, characterized in that: The step of generating second training data based on local features, global features, second standardized data, and third standardized data, and training a pre-defined multilayer perceptron model using the second training data to obtain a second pre-trained model, includes: Second training data is generated based on local features, global features, second standardized data, and third standardized data. Input local features into the second multilayer perceptron model to obtain the second predicted value; Input global features into the second multilayer perceptron model to obtain the third predicted value; Based on the second predicted value and the second standardized data, the local loss function is calculated; The global loss function is calculated based on the third predicted value and the third standardized data; The sum of the local loss function and the global loss function is calculated to obtain the second loss function; The second multilayer perceptron model is trained based on the second loss function and the second training data to obtain the second pre-trained model.

6. The method for constructing a gene expression prediction model according to claim 1, characterized in that: The step of jointly inputting the first training data and the second training data into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model includes: The second training data is upsampled to obtain the third training data, and the number of samples in the third training data is an integer multiple of the number of samples in the first training data. The first training data and the third training data are jointly input into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model.

7. The method for constructing a gene expression prediction model according to claim 1, characterized in that: The process of incorporating domain knowledge constraints into the loss functions of the first and second fine-tuned models, training the first and second fine-tuned models to obtain a first training model and a second training model, and generating a gene expression prediction model based on the first and second training models includes: Based on the graph Laplacian regularization framework, spatial consistency constraints are obtained; Based on prior knowledge of gene pathways, constraints on pathway co-expression are obtained; Domain knowledge constraints are generated based on spatial consistency constraints and pathway co-expression constraints. Domain knowledge constraints are added to the loss function of the first fine-tuning model and the loss function of the second fine-tuning model in the form of regularization. The first fine-tuning model and the second fine-tuning model are trained to obtain the first training model and the second training model. Gene expression prediction models are generated based on the first and second training models.

8. A gene expression prediction model construction device, characterized in that, include: The data acquisition module is used to acquire a multi-scale, multi-modal gene expression dataset, including a first image, first data, a second image, and second data. The first image is a pathological image at the tissue level, the first data is gene expression data at the tissue level, the second image is a pathological image at the near-cellular level, and the second data is gene expression data at the near-cellular level. The data preprocessing module is used to preprocess the first image, the first data, the second image, and the second data to obtain the first image patch, the first standardized data, the second image patch, and the second standardized data. The pseudo-organization generation module is used to aggregate the second-standardized data in the spatial dimension to obtain the third-standardized data at the pseudo-organization level. The feature extraction module is used to extract features from the first image patch and the second image patch respectively, to obtain the first feature vector and the second feature vector; The feature enhancement module is used to aggregate the global information of the second feature vector. It inputs the second feature vector and its global information into the Vision Mamba model and outputs local features and global features. The first pre-training module is used to generate first training data based on the first feature vector and the first standardized data, and to train the preset first multilayer perceptron model using the first training data to obtain the first pre-trained model. The second pre-training module is used to generate second training data based on local features, global features, second standardized data and third standardized data, and to train the preset second multilayer perceptron model using the second training data to obtain the second pre-trained model. The joint training module is used to jointly input the first training data and the second training data into the first pre-trained model and the second pre-trained model for joint training to obtain the first fine-tuned model and the second fine-tuned model. The domain knowledge constraint module is used to add domain knowledge constraints to the loss function of the first fine-tuning model and the loss function of the second fine-tuning model, train the first fine-tuning model and the second fine-tuning model to obtain the first training model and the second training model, and generate a gene expression prediction model based on the first training model and the second training model.

9. A gene expression prediction model construction device, characterized in that, The gene expression prediction model construction device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the gene expression prediction model building device to perform the steps of the gene expression prediction model building method as claimed in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the gene expression prediction model construction method as described in any one of claims 1-7.

Images