A deep learning-based method for predicting spatial gene expression from histological images

Abstract

The present invention proposes a method for predicting spatial gene expression from histological images based on deep learning, which relates to the field of spatial gene expression prediction and more efficiently and accurately predicts gene expression at spatial resolution from histological images. First, the number of CNN layers is deepened by residual mapping and an efficient channel attention module is added to each layer of residual blocks, so that the model can extract more comprehensive and detailed image texture features. Then, the visual Transformer and the dynamic graph attention network (Dynamic‑GAT) are used to simultaneously model the global and local relationships between tissue sequencing points, and the correlation between high-resolution histological image phenotypes and spatial gene expression is explored to achieve more accurate spatial gene expression prediction.

Description

Technical Field

[0001] The present invention relates to the field of spatial gene expression prediction, and in particular to a method for predicting spatial gene expression from histological images based on deep learning. Background Art

[0002] Genes are the basic functional and physical units of DNA molecules. Through processes such as transcription and translation, genes control protein synthesis and are the carriers of genetic information that determine an organism's traits. Transcriptome sequencing that captures gene expression plays a key role in monitoring and regulating an organism's growth, development, function, and reproduction. The rapid development of spatial transcriptomics (ST) technology overcomes the limitations of single-cell transcriptomics in terms of loss of spatial context, enabling the correlation of gene expression data with tissue structure and providing insights into the true gene expression signatures of cells within their physiological tissue environment.

[0003] Due to the high cost of spatial transcriptome sequencing, researchers have proposed several deep learning methods to predict spatial gene expression from histological images.

[0004] ST-Net uses convolutional neural networks (CNNs) to extract biological signals such as cell phenotypes from histological images to align the spatial gene expression of cells in tissues; HisToGene captures the global dependencies between image sequencing points by encoding position information into visual image features in the visual Transformer (ViT) model; Hist2ST and THItoGene integrate the advantages of the former, using a CNN + Transformer hybrid architecture to effectively extract image data features and perform global information interaction analysis.

[0005] Although a large number of methods have been used to attempt to predict spatial gene expression from histological images, these methods generally need to be further improved in terms of gene prediction accuracy and model robustness: in traditional molecular biology research, RNA sequencing, such as single-cell transcriptome sequencing (scRNA-seq), is usually required to capture gene expression levels. In recent years, the rapid development of spatial transcriptomics (ST) has made it possible to complete RNA sequencing while retaining the spatial location information of its sequencing points. However, the high experimental consumables and sequencing costs of this type of sequencing technology limit its application in larger-scale studies. In contrast, high-resolution tissue sections generated in clinical routine are cheaper, and the section images contain rich biomedical information, but they have not yet been fully utilized. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for predicting spatial gene expression from histological images based on deep learning, which can more efficiently and accurately predict gene expression at spatial resolution from histological images.

[0007] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions:

[0008] A method for predicting spatial gene expression from histological images based on deep learning, comprising the following steps:

[0009] Establish a histological image dataset, preprocess the histological images, and obtain m image data after preprocessing for each slice , is the image feature of the i-th tissue slice; Image data for each slice The tensor representation of n is the number of sequencing points for each slice. , and Represent the width, height and number of channels of each spot image block respectively, and divide the dataset into training set and validation set;

[0010] A spatial gene expression prediction model was established, comprising three modules: ECA-ResNet, Vision Transformer, and a dynamic graph attention network. This model deepened the CNN layers through residual mapping and incorporated an efficient channel attention module into each residual block. The Vision Transformer and dynamic graph attention network were used to simultaneously model global and local relationships between tissue sequencing points.

[0011] The preprocessed slice image data The spatial gene expression prediction model is input to extract the detailed features of all spot image blocks in the slice image and compress them into a feature sequence of the spot image. The serialized deep image features are fused with the spot spatial position encoding information. The self-attention mechanism of the Vision Transformer is used for global modeling to capture the association relationship between spot cells in different positions, and a spot adjacency relationship graph based on spatial position is constructed. The dynamic graph attention network is used to calculate the attention between spots and dynamically represent the relationship between adjacent spots. The spot gene expression is updated according to the neighbor spot information. Finally, the image features are mapped to the gene number dimension through the fully connected layer to represent the predicted spatial gene expression profile of the tissue slice.

[0012] Furthermore, the preprocessing of histological images includes the following steps:

[0013] The high-resolution image of each tissue slice is cut into slices according to the position of the sequencing space point to characterize the image data of each spot and obtain the pre-processed image data of each slice. ;

[0014] Select some genes in the slice, normalize the retained gene expression data, and calculate the true observation value of each slice , where g is the number of genes screened.

[0015] Furthermore, the true observation value of each slice is calculated as follows:

[0016] The expression count of each gene in each spot was divided by the total count of all genes in that spot, multiplied by 1,000,000, and logarithmically transformed to log(1 + x) to be the true observation value of each slice. The 1000000 here is a common normalization factor. Its main function is to scale the data to a range that is easy to handle and interpret. For RNA-seq (gene expression sequencing) data, it is convenient to compare gene expression between different samples. Even if all data sets are preprocessed uniformly, the normalization factor will not change for different data sets.

[0017] Furthermore, compressing the detailed features of the spot image block into a feature sequence includes the following steps:

[0018] (1) The i-th slice image data Send it to ECA-ResNet and extract preliminary image features through convolutional layer and maximum pooling layer ;

[0019] (2) Further extract deep features through L-layer residual blocks , where Represents the initial image features The number of layers after the residual block:

[0020] (21) Each residual block contains two groups of convolutional layers, each of which includes a convolutional block and a batch normalization layer. A ReLU activation function is attached between the two groups of convolutional layers. It is fed into the first residual block, passes through the first set of convolutional layers, then through the activation and second convolutional layers, and finally through the efficient channel attention layer, through the formula Learn the attention weights of image channel features, where represents the ReLU activation function, Indicates channel The set of k adjacent channels of is a learning parameter shared by all channels, and k represents the number of adjacent channels involved in the calculation of a channel weight;

[0021] k is determined by fast one-dimensional convolution with kernel size k, using the formula Adaptively calculate the coverage of the interaction, where Represents the number of channels, , is the calculation parameter, represents the odd number closest to t;

[0022] (22) Through the residual connection formula Aggregate the output and input of the first layer residual block, where represents the output of the first layer residual block, Represents a linear activation function, and obtains the feature As the input of the next layer of residual block; if the output and input If the number of channels c or stride is inconsistent and cannot be added, then according to the one-dimensional convolution formula Adjust for dimension matching, where represents the batch normalization layer, Indicates the number of input channels, Indicates the number of output channels, Indicates the stride of convolution;

[0023] (23) After each layer of residual blocks performs the above steps (21)-(22), the final formula is Get the captured deep image features ;

[0024] By the formula Will The final output sequence form of the ECA-ResNet module is obtained through the global average pooling layer GAP and the fully connected layer fc .

[0025] Furthermore, the fusion of the serialized depth image features and the spot spatial position encoding information includes the following steps:

[0026] (1) Encode the position coordinates x and y of n sequencing points as and , where d represents the dimension of position encoding, which can be 64, and then the formula Coordinate encoding and output features of ECA-ResNet Splice to Sequencing spots refer to specific locations on tissue samples where gene expression is captured and analyzed in spatial transcriptome sequencing. These spots are usually implemented through probes or barcodes on microarrays. Each spot represents a tissue region, allowing researchers to spatially analyze gene expression patterns.

[0027] (2) The fused features Feed it into the first multi-head attention layer in the Vision Transformer module, through the formula Calculate the attention output of the sequencing points learned by each head. First, calculate the dot product of the query and the key, and scale it by the dimension of the key. The result is converted into a probability distribution through the softmax function to represent the attention weight, and then multiply it to get the attention output. Where Q, K, and V represent query, key, and value respectively. represents the dimension of the key, , is the parameter mapping matrix, represents the output of the i-th attention head;

[0028] (3) Through the formula The output of each attention head is concatenated to get the output features of the first layer Transformer , where The parameter matrix representing the aggregation of attention heads;

[0029] Repeat steps (2) and (3) three times to obtain the global features after the Vision Transformer module captures the correlation between spatial sequencing points. .

[0030] Furthermore, the DropKey technique is introduced when calculating attention to prevent overfitting of feature learning by discarding the key:

[0031] Integrate Dropout and regularization and put them before softmax, through the formula Calculate the attention after the mask discards the Key, where represents element-wise multiplication, and M represents a binary mask matrix. The original Transformer performs a dropout step after the dot-product attention calculation to prevent overfitting. We incorporate this into the attention calculation. By using a key mask, we reduce computational complexity and improve model generalization. This prevents the model from over-relying on specific keys, promoting more robust learning.

[0032] Furthermore, using the dynamic graph attention network to calculate the attention between spots and dynamically represent the relationship between adjacent spots includes the following steps:

[0033] The global features obtained from the Vision Transformer module As the input of the dynamic graph neural network module, the K nearest neighbor algorithm is used to calculate the k nearest neighbors of each spot using the Euclidean formula, and each spot is connected to its k nearest neighbors into a directed edge, which is stored in the edge index E to form an adjacency graph. ,in Indicates the number of spots;

[0034] By formula Dynamically calculate the attention coefficient between each spot i and its neighbor spot j, where is the set of first-order neighbor nodes that have directed edges with spot i, and is the learnable parameter matrix, Represents a splicing operation, 、 Represent the spot features of spots i and j respectively. After calculating the attention scores of spot i and all its neighbors, the formula Normalize the attention coefficient to get , represents the attention value after softmax normalization, and t represents the neighbor node of spot i;

[0035] By formula Aggregate the weighted feature representations of all neighbors of spot i and use the formula The feature representation of the current spot is connected with the aggregated feature representation of its neighboring spots to update the local features, where represents the activation function, and are the parameterized weight and bias matrices, Indicates the current spot characteristics, Represents the aggregated features of the current spot’s neighbor spots, such as spot i’s , the neighbor aggregation feature of spot i is .

[0036] Furthermore, it contains two layers of graph attention layers, and the output of the first layer of graph attention layer Send it to the second layer of graph attention layer, dynamically calculate the attention coefficient between each spot i and its neighbor spot j, then normalize the attention coefficient, aggregate the weighted feature representations of all neighbors of spot i, and connect the feature representation of the current spot with the feature representation of its neighboring spots, update the local features and output When the second-layer graph attention layer calculates the neighbor attention coefficient, the spot feature contains the information of its neighbor spots aggregated by the first layer, so the second-layer dynamic graph attention layer also includes the feature embedding of its second-order neighbors when calculating the attention coefficient and aggregating the first-order neighbor features.

[0037] Furthermore, gene expression prediction includes the following steps:

[0038] The sequence features of the obtained spot image Input to the gene dimension encoding layer, through the first fully connected layer, RELU activation layer, normalization layer, and the second fully connected layer;

[0039] By formula Get the predicted gene expression profile , where n is the total number of sequencing points cut from this tissue section, g is the total number of highly heterogeneously expressed genes screened, Representation layer normalization layer, represents a fully connected layer.

[0040] Furthermore, the inspection step is also included:

[0041] By formula Computational predicted expression profiles and the true observed value The Pearson correlation coefficient between represents the covariance, represents the standard deviation, PCC∈[-1,1] is the correlation of the prediction, and the closer the PCC is to 1, the more accurate the spatial gene expression prediction is.

[0042] The advantages of the present invention are:

[0043] 1. Through the deep channel attention residual neural network, the characteristic information of histological images can be fully and comprehensively mined;

[0044] 2. Incorporating the spatial location of sequencing spots into image features before the Transformer helps the Transformer capture global image dependencies.

[0045] 3. When calculating attention, the DropKey operation is used instead of the DropAttention operation in the original Transformer after the softmax calculation. This overcomes the overfitting problem of specific patterns that occurs in ordinary drop operations and enhances the generalization ability of the ViT model.

[0046] 4. Transformer dropout scheduling results in higher dropout rates for shallow layers and lower dropout rates for deep layers, thus avoiding overfitting on low-level features and retaining sufficient high-level semantics in histological image phenotypes;

[0047] 5. Based on GAT, the parameters are used to calculate neighbor attention. Put it on nonlinear activation function The extra calculation effectively prevents the linearization caused by multi-parameter stacking, making GAT dynamic in capturing neighbor attention;

[0048] 6. Using fully connected layers can transform image features into gene quantity dimensions, aligning the predicted gene expression profiles. Using RELU activation functions and layer normalization operations can enhance the robustness of the model and accurately and efficiently predict spatial gene expression. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of the method for predicting spatial gene expression from histological images based on deep learning in Example 1;

[0050] Figure 2 This is a flow chart of the efficient channel attention residual neural network module in Example 1;

[0051] Figure 3 This is the flow chart of the Vision Transformer module in Example 1;

[0052] Figure 4 This is a flow chart of the dynamic graph attention network module in Example 1;

[0053] Figure 5 This is the gene dimension encoding flow chart in Example 1. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0055] Example 1

[0056] This example presents the performance of a deep learning-based method for predicting spatial gene expression from histological images on four publicly available ST datasets: mouse olfactory bulb (MOB); human cutaneous squamous cell carcinoma (cSCC); human HER2-positive breast cancer (HER2+); and human dorsolateral prefrontal cortex (DLPFC). The MOB was generated by the ST Pipeline, while the others were provided by the 10X Genomics platform.

[0057] S1. The histological image datasets were screened. For the MOB and cSCC datasets, 12 tissue sections from different patients were fully retained. For the HER2+ dataset, 32 tissue sections with more than 180 sequencing points were retained.

[0058] S2. Establish a histological image dataset. For histological images, we crop the high-resolution image of each tissue slice into a 112×112 square domain according to the position of the sequencing space point with a size of 112 pixels for width and height respectively to represent the image data of each spot, and obtain m image data of each slice after preprocessing. , is the image feature of the i-th tissue slice; Image data for each slice The tensor representation of , and They represent the width, height, and number of channels of each spot image block, respectively, and are used to align gene expression profiles for spatial gene prediction.

[0059] For gene expression data of multiple spatial transcriptomics, the top 1000 highly variable genes in tissue sections were selected, and genes with expression greater than 1000 spots in all tissue sections were screened out. The retained gene expression data were normalized. For each spot, its count was divided by the total count of all genes at that point, then multiplied by 1,000,000 and logarithmically transformed to be the true observation value of each section. , where g is the number of genes screened.

[0060] S3. Split the dataset into a training set and a validation set, and perform leave-one-out cross-validation on each ST dataset. Specifically, retain one high-resolution tissue slice histology image for validation, and use the remaining slice images in the dataset as the training set.

[0061] S4. Establish a spatial gene expression prediction model. The spatial gene expression prediction model includes three modules: Efficient Channel Attention Residual Neural Network (ECA-ResNet), Vision Transformer (ViT), and Dynamic Graph Attention Network (Dynamic-GAT). First, The ECA-ResNet module is used to extract the detailed features of all the spot image blocks of the tissue slice and compress them into a sequence form. . Then, the serialized deep image features are fused with the spot spatial position encoding information, and global modeling is performed through the self-attention mechanism of the ViT module to capture the association relationship between spot cells at different positions. Next, a spot adjacency relationship graph based on spatial position is constructed, and the Dynamic-GAT module is used to calculate the attention between spots, dynamically represent the relationship between adjacent spots, and more accurately update the spot gene expression based on the neighboring spot information. Finally, the image features are mapped to the gene quantity dimension through two fully connected layers, representing the predicted spatial gene expression profile of the tissue section. The flowchart is shown as follows Figure 1 shown.

[0062] S5, please refer to Figure 2 ,First, the preprocessed histological image data is input into the ,model, and deep image feature extraction is performed through the ECA-ResNet ,module, and the three-dimensional image features of n spots are compressed into ,sequence encoding.

[0063] S51, pre-processed histological image data The ECA-ResNet module input to the model passes through a convolutional layer (Conv7×7) with a stride of 2 and a convolution kernel size of 7x7 and a maximum pooling layer of 3x3 with a stride of 2. Extract preliminary image features .

[0064] S52, further extract deep features through L-layer residual blocks , where Represents the initial image features Through layers of residual blocks, they address the vanishing and exploding gradient problems in deep networks, helping us extract more detailed and comprehensive image features by training deeper networks. Each residual block consists of two convolutional layers, each consisting of a 3x3 convolutional block (Conv3×3) and a batch normalization layer (BN layer). A ReLU activation function is applied between the two convolutional layers, and finally an Efficient Channel Attention (ECA) block calculates channel weights.

[0065] S53, will Send it to the first layer of residual block, first pass through the first set of convolutional layers Perform a 3x3 convolution operation, then activate and perform the second convolution layer. , and finally through the efficient channel attention layer , through the formula Learn the attention weights of image channel features, where represents the ReLU activation function, Indicates channel The set of k adjacent channels of is a learning parameter shared by all channels, and k represents the number of adjacent channels involved in the calculation of a channel weight. The number of k obviously affects the efficiency and effectiveness of ECA calculation. This strategy can be achieved by fast one-dimensional convolution with kernel size k. Since the ECA module is designed to properly capture local cross-channel interactions, the Adaptively calculate the coverage of the interaction (i.e., the kernel size k of 1D convolution), where Represents the number of channels, , is the calculation parameter, Represents the odd number closest to t. Obviously, by mapping , high-dimensional channels have longer distance interactions, while low-dimensional channels using nonlinear mapping have shorter distance interactions.

[0066] S54, through the residual connection formula Aggregate the output and input of the first layer residual block, where represents the output of the first layer residual block, Represents a linear activation function, and obtains the feature As the input of the residual block of the next layer; if the output is caused by operations such as convolution and input The number of channels c or stride are inconsistent and cannot be added. According to the one-dimensional convolution formula Adjust for dimension matching, where represents the batch normalization layer, Indicates the number of input channels, Indicates the number of output channels, Indicates the stride of the convolution.

[0067] S55. Repeat steps S53-S54 L-1 times, and finally get the formula Get the captured deep image features , L can be adjusted according to the difference of image tasks. Will The final output sequence form of the ECA-ResNet module is obtained through the global average pooling layer GAP and the fully connected layer fc .

[0068] S6, please refer to Figure 3 , the two-dimensional position coordinates x, y of all cropped spots in the tissue slice are encoded separately and fused into the feature sequence of the spot image As the input feature matrix of Vision Transformer (ViT), ViT containing 4 layers of multi-head attention layers is used to capture the global dependencies between spot images.

[0069] S61, encode the position coordinates x and y of n sequencing points as and , where d represents the dimension of position encoding, which can be 64, and then the formula Coordinate encoding and output features of ECA-ResNet Splice to Sequencing points refer to specific locations on tissue samples where gene expression is captured and analyzed in spatial transcriptome sequencing. These points are usually implemented through probes or barcodes on microarrays. Each point represents a tissue area, allowing researchers to spatially analyze gene expression patterns.

[0070] S62, the fused features Feed it into the first multi-head attention layer in the ViT module, through the formula Calculate the attention output of the sequencing points learned by each head. First, calculate the dot product of the query and the key, and scale it by the dimension of the key. The result is converted into a probability distribution through the softmax function to represent the attention weight, and then multiply it to get the attention output. Where Q, K, and V represent query, key, and value respectively. represents the dimension of the key, , is the parameter mapping matrix, represents the output of the i-th attention head.

[0071] S63. Introducing a new dropout technique, DropKey, when calculating attention, which prevents overfitting of feature learning by dropping the key instead of dropping the calculated attention, DropAttention. Integrate Dropout and regularization and put them before softmax, using the formula Calculate the attention after the mask discards the Key, where Represents element-by-element multiplication, and M represents a binary mask matrix. The original transformer has a dropout step after calculating the dot product attention to prevent overfitting. This embodiment incorporates it into the attention calculation, reducing the computational complexity and improving the generalization ability of the model by using the key mask. This can prevent the model from over-relying on certain specific keys and promote more robust learning. Represents element-wise multiplication, and M represents a binary mask matrix. In each training iteration, DropKey randomly masks a certain proportion of the keys in the input key map. This allows the self-attention layer to capture important information globally, overcoming the overfitting problem of specific patterns that occurs in conventional dropout and enhancing the generalization ability of the ViT model. A novel linearly decreasing schedule is also proposed here, gradually reducing the dropout rate along the stack of self-attention layers. This schedule results in higher dropout rates for shallow layers and lower dropout rates for deeper layers, thus avoiding overfitting to low-level features and preserving sufficient high-level semantics in the histological image phenotypes.

[0072] S64, through the formula The output of each attention head is concatenated to get the output features of the first layer Transformer , where Parameter matrix representing the aggregation of attention heads.

[0073] S65, repeat steps S62 and S64 three times to obtain the global features after the association relationship between spatial sequencing points is captured by the ViT module .

[0074] S7, please refer to Figure 4 , the global features obtained from the ViT module As the input of the dynamic graph neural network module (Dynamic-GAT), Dynamic-GAT contains two layers of graph attention. By constructing an adjacency relationship graph, it aggregates the feature information between each sequencing point and its neighboring spots at the local level to enhance the local correlation of the histological image.

[0075] S71. Global features obtained from the ViT module As the input of the dynamic graph neural network module, the K nearest neighbor algorithm is used to calculate the k nearest neighbors of each spot using the Euclidean formula, and each spot is connected to its k nearest neighbors into a directed edge, which is stored in the edge index E to form an adjacency graph. ,in Indicates the number of spots.

[0076] S72, through the formula Dynamically calculate the attention coefficient between each spot i and its neighbor spot j, where is the set of first-order neighbor nodes that have directed edges with spot i, and is the learnable parameter matrix, Represents a splicing operation, 、 Represents the spot characteristics of spots i and j respectively, and the standard GAT score and are used for calculations in sequence, they are likely to collapse into a linear layer, causing attention to become static, so we will Put it on nonlinear activation function After calculating the attention scores of spot i and all its neighbors, the formula Normalize the attention coefficient to get , represents the attention value after softmax normalization, and t represents the neighbor node of spot i.

[0077] S73, through the formula Aggregate the weighted feature representations of all neighbors of spot i and use the formula The feature representation of the current spot is connected with the aggregated feature representation of its neighboring spots to update the local features, where represents the activation function, and are the parameterized weight and bias matrices, Indicates the current spot characteristics, Represents the aggregated features of the current spot’s neighbor spots, such as spot i’s , the neighbor aggregation feature of spot i is .

[0078] S74, the output result of the previous GAT Send it to the second dynamic graph attention layer. Through the formula Dynamically calculate the attention coefficient between each spot i and its neighbor spot j, where is the set of first-order neighbor nodes with directed edges to spot i. After calculating the attention scores of spot i and all its neighbors, the formula Normalize the attention coefficient to get , then, we use the formula Aggregate the weighted feature representations of all neighbors of spot i and use the formula Connect the feature representation of the current spot with the feature representation of its neighboring spots to update the local features. According to the transitivity of the graph neural network, when the second layer of GAT calculates the neighbor attention coefficient, the spot feature The first-layer aggregated information of its neighboring spots is included, so the second-layer dynamic graph attention layer also includes the feature embedding of its second-order neighbors when calculating the attention coefficient and aggregating the first-order neighbor features. The two-layer network covers most of the neighborhood relationships associated with tissues with very few parameters, allowing Dynamic-GAT to better characterize the local neighborhood relationships of histological images and update the embedded expression of spots.

[0079] S8, please refer to Figure 5 , after two layers of dynamic graph attention network, the sequence features of the obtained spot image are Input to the gene dimension encoding layer, through the first layer of fully connected layer, RELU activation layer, normalization layer, the second layer of fully connected layer, through the formula Get the predicted gene expression profile , where n is the total number of sequencing points cut from this tissue section, g is the total number of highly heterogeneously expressed genes screened, Representation layer normalization layer, represents a fully connected layer.

[0080] Example 2

[0081] Based on Example 1, this example further proposes the steps of predicting evaluation indicators:

[0082] By formula Computational predicted expression profiles and the true observed value The Pearson correlation coefficient (PCC) between represents the covariance, represents the standard deviation, PCC∈[-1,1] is the correlation of the prediction, and the closer the PCC is to 1, the more accurate the spatial gene expression prediction is.

[0083] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for predicting spatial gene expression from histological images based on deep learning, characterized in that: Includes steps: Establish a histological image data set, preprocess the histological images, and obtain m image data after preprocessing for each slice , is the image feature of the i-th tissue slice; For each slice image data The tensor representation of n is the number of sequencing points for each slice. , and Represent the width, height and number of channels of each spot image block, and divide the dataset into training set and validation set; Establishing a spatial gene expression prediction model, which includes three modules: ECA-ResNet, VisionTransformer, and dynamic graph attention network. The number of CNN layers is deepened by residual mapping, and an efficient channel attention module is added to each residual block. Vision Transformer and dynamic graph attention network are used to simultaneously model the global and local relationships between tissue sequencing points. The preprocessed slice image data Input the spatial gene expression prediction model, extract the detailed features of all spot image blocks of the slice image and compress them into a feature sequence of the spot image, fuse the serialized deep image features with the spot spatial position encoding information, perform global modeling through the self-attention mechanism of Vision Transformer to capture the association relationship between spot cells in different positions, build a spot adjacency relationship graph based on spatial position, use the dynamic graph attention network to calculate the attention between spots, dynamically represent the relationship between adjacent spots, update the spot gene expression according to the neighbor spot information, and finally map the image features to the gene quantity dimension through the fully connected layer to represent the predicted spatial gene expression profile of the tissue slice; Compressing the detailed features of the spot image block into a feature sequence includes the following steps: (a1) The i-th slice image data Send it to ECA-ResNet, and extract preliminary image features through convolutional layers and maximum pooling layers ; (a2) Further extract deep features through L-layer residual blocks , where Represents the initial image features The number of layers after the residual block: (a21) Each residual block contains two groups of convolutional layers, each of which includes a convolutional block and a batch normalization layer. A ReLU activation function is attached between the two groups of convolutional layers. It is sent to the first residual block, first passes through the first set of convolutional layers, then passes through the activation and second convolutional layers, and finally passes through the efficient channel attention layer. Learn the attention weights of image channel features, where represents the ReLU activation function, Indicates channel The set of k adjacent channels of is a learning parameter shared by all channels, and k represents the number of adjacent channels involved in the calculation of a channel weight; k is determined by fast one-dimensional convolution with kernel size k, using the formula Adaptively calculate the coverage of the interaction, where Represents the number of channels, , is the calculation parameter, represents the odd number closest to t; (a22) Through the residual connection formula Aggregate the output and input of the first layer residual block, where represents the output of the first layer residual block, Represents a linear activation function, and obtains the feature As the input of the next layer of residual blocks; if the output and input If the number of channels c or the stride are inconsistent and cannot be added, then according to the one-dimensional convolution formula Adjust to match the dimensions, where represents the batch normalization layer, Indicates the number of input channels, Indicates the number of output channels, Indicates the stride of the convolution; (a23) After each layer of residual blocks executes the above steps (a21)-(a22), the final formula is Get the captured deep image features ; By formula Will The final output sequence form of the ECA-ResNet module is obtained through the global average pooling layer GAP and the fully connected layer fc ; The fusion of serialized depth image features and spot spatial position encoding information includes the following steps: (b1) Encode the position coordinates x and y of the n sequencing points as and , where d represents the dimension of the position encoding, and then through the formula Coordinate encoding and output features of ECA-ResNet Splice to ; (b2) The fused features Feed it into the first multi-head attention layer in the Vision Transformer module, through the formula Calculate the attention output of the sequencing points learned by each head. First, calculate the dot product of the query and the key, and scale it by the dimension of the key. The result is converted into a probability distribution through the softmax function to represent the attention weight, and then multiply it to get the attention output. In the formula, Q, K, and V represent query, key, and value respectively. represents the dimension of the key, , is the parameter mapping matrix, represents the output of the i-th attention head; (b3) By formula The output of each attention head is concatenated to get the output features of the first layer Transformer , where The parameter matrix representing the aggregation of the mind heads; Repeat steps (b2) and (b3) three times to obtain the global features after the Vision Transformer module captures the association relationship between spatial sequencing points. ; The DropKey technique is introduced when calculating attention to prevent overfitting of feature learning by discarding the key: Integrate Dropout and regularization and put them before softmax, through the formula Calculate the attention after the mask discards the Key, where represents element-by-element multiplication, and M represents a binary mask matrix; Using the dynamic graph attention network to calculate the attention between spots and dynamically represent the relationship between adjacent spots includes the following steps: The global features obtained from the Vision Transformer module As the input of the dynamic graph neural network module, the K nearest neighbor algorithm is used to calculate the k nearest neighbors of each spot through the Euclidean formula, and each spot is connected to its k nearest neighbors into a directed edge, which is stored in the edge index E to form an adjacency graph. ,in Indicates the number of spots; By formula Dynamically calculate the attention coefficient between each spot i and its neighbor spot j, where is the set of first-order neighbor nodes that have directed edges with spot i, and is the learnable parameter matrix, Represents a splicing operation, , Represent the spot features of spots i and j respectively. After calculating the attention scores of spot i and all its neighbors, the formula Normalize the attention coefficient to get , represents the attention value after softmax normalization, and t represents the neighbor node of spot i; By formula Aggregate the weighted feature representation of all neighbors of spot i and use the formula Connect the feature representation of the current spot with the aggregated feature representation of its neighboring spots to update the local features, where represents the activation function, and are the parameterized weight and bias matrices, respectively, Indicates the current spot characteristics, Represents the aggregated features of the current spot's neighbor spots; Contains two layers of graph attention layers, and the output of the first layer of graph attention layer Send it to the second-layer graph attention layer, dynamically calculate the attention coefficient between each spot i and its neighbor spot j, then normalize the attention coefficient, aggregate the weighted feature representations of all neighbors of spot i, and connect the feature representation of the current spot with the feature representation of its neighboring spots, update the local features and output When the second-layer graph attention layer calculates the neighbor attention coefficient, the spot feature contains the information of its neighbor spots aggregated in the first layer, so the second-layer dynamic graph attention layer also includes the feature embedding of its second-order neighbors when calculating the attention coefficient and aggregating the first-order neighbor features; Gene expression prediction includes the following steps: The sequence features of the obtained spot image Input to the gene dimension encoding layer, through the first fully connected layer, RELU activation layer, normalization layer, and the second fully connected layer; By formula Get the predicted gene expression profile , where n is the total number of sequencing points cut from this tissue section, g is the total number of highly heterogeneously expressed genes screened, Representation layer normalization layer, represents a fully connected layer.

2. The method for predicting spatial gene expression from histological images based on deep learning according to claim 1, characterized in that: Preprocessing of histological images includes the following steps: The high-resolution image of each tissue slice is cut into slices according to the position of the sequencing space point to characterize the image data of each spot and obtain the pre-processed image data of each slice. ; Select some genes in the slice, normalize the retained gene expression data, and calculate the true observed value of each slice , where g is the number of genes screened.

3. The method for predicting spatial gene expression from histological images based on deep learning according to claim 2, characterized in that: The calculation method of the true observation value of each slice is: The expression count of each gene in each spot was divided by the total count of all genes in that spot, multiplied by 1,000,000 and logarithmically transformed to log(1 + x) as the true observation value of each slice. .

4. The method for predicting spatial gene expression from histological images based on deep learning according to claim 1, characterized in that: Also includes the inspection steps: By formula Computational prediction of expression profiles The actual observed value The Pearson correlation coefficient between represents the covariance, represents the standard deviation, PCC∈[-1,1] is the correlation of the prediction, and the closer the PCC is to 1, the more accurate the spatial gene expression prediction is.

Images