A method for characterizing learning of a tumor sample missense mutation set

Abstract

This invention discloses a representation learning method for missense mutation sets in tumor samples, comprising the following steps: S1. Acquiring raw mutation data and performing field filtering; S2. Constructing a hybrid gene vocabulary and encoding mutated genes; S3. Encoding the mutation type, pathogenicity score, amino acid change information, and relative position information for each mutation record; S4. Constructing mutation sets according to samples and performing serialization processing; S5. Constructing a comprehensive representation of a single mutation; S6. Using a Transformer encoder to learn the contextual relationships between mutations within a sample; S7. Generating sample-level representations using PMA pooling; S8. Performing self-supervised training using a masked gene prediction task; S9. Enhancing the stability of sample-level representations using contrastive learning; S10. Optimizing the joint loss; S11. Extracting and saving the embeddings of the sample-level mutation set. This invention can reflect the potential associations between missense mutations and sample-level biological characteristics.

Description

Technical Field

[0001] This invention relates to the fields of bioinformatics and deep learning technology, specifically to a representation learning method for a set of missense mutations in tumor samples. Background Technology

[0002] The development and progression of tumors are often accompanied by complex gene mutations, among which missense mutations are the main type. Traditional tumor genomic analysis methods, such as mutation burden calculation and driver gene identification methods, often treat mutations as isolated events or simply perform statistical analysis based on gene frequencies, making it difficult to capture the complex co-occurrence and contextual relationships between different mutated genes in the same patient.

[0003] Currently, although some studies have attempted to use deep learning to process mutation data, such as "Mutation-Attention (MuAt): deep representation learning of somatic mutations for tumour typing and subtyping" published by Prima Sanjaya, Katri Maljanen, Riku Katainen, Sebastian M. Waszak, Genomics England Research Consortium, Lauri A. Aaltonen, Oliver Stegle, Jan O. Korbel, and Esa Pitkänen on July 7, 2023, in Springer Nature, and "Multiple-instance learning of somatic mutations for the classification of tumour type and the prediction of microsatellite status" published by Jordan Anaya, John-William Sidhom, Faisal Mahmood, and Alexander S. Baras on November 2, 2023, in Springer Nature, these methods focus on modeling all somatic mutation types, and the representations learned by the models often mix signals from multiple mutation types. This design has advantages in tasks such as cancer classification or MSI status prediction, but it may also weaken the specific information of functional variations in protein-coding regions, such as missense mutations. Especially in sample-level prediction tasks, a large number of passenger mutations (a core concept in cancer genetics) may introduce noise, making it difficult for the model to focus on key missense mutation combinations that truly affect tumor phenotype, molecular subtyping, or treatment response. Summary of the Invention

[0004] This invention provides a representation learning method for missense mutation sets in tumor samples. By introducing the SetTransformer architecture to model the unordered missense mutation set within the sample, and combining it with a contrastive learning mechanism, it automatically learns a deep representation that can reflect the potential associations between missense mutations and the sample-level biological characteristics.

[0005] The technical solution adopted in this invention is as follows:

[0006] A representation learning method for a set of missense mutations in tumor samples includes the following steps:

[0007] S1. Obtain raw mutation data and perform field filtering: Each mutation record should include at least the sample identifier, mutated gene name, mutation type, pathogenicity score, pathogenicity category, amino acid change information, and the relative position of the mutation in the protein sequence;

[0008] S2. Constructing a hybrid gene vocabulary and encoding mutated genes: First, read the set of cancer-related genes and simultaneously count the frequency of each gene in different tumor samples in the training data. Select several genes with the highest frequency of occurrence as a high-frequency gene set. Then, take the union of the cancer-related gene set and the high-frequency gene set to form the core gene vocabulary, and add a special category to represent unknown or low-frequency genes. <other>For each mutation record, if its gene name belongs to the core gene vocabulary, the gene is mapped to the corresponding gene ID; otherwise, it is mapped to... <other>category;

[0009] This processing step can preserve known cancer-related genes and representative high-frequency genes in the dataset, while avoiding problems such as an excessively large vocabulary, sparse training, and unstable model caused by extremely low-frequency genes.

[0010] S3. Encode the mutation type, pathogenicity score, amino acid alteration information, and relative position information for each mutation record;

[0011] S4. Construct mutation sets by sample and perform serialization: After completing the feature encoding of a single mutation record, the mutation records are grouped according to the sample identifier so that multiple mutation records of the same sample form a mutation set. Before inputting into the model, a maximum mutation sequence length is set for each sample. If the number of mutations in any sample is less than the maximum mutation sequence length, a mask is used for padding and a padding mask is constructed.

[0012] S5. Constructing a comprehensive representation of a single mutation: First, obtain the gene identifier embedding vector, mutation type embedding vector, pathogenicity score projection vector, wild-type amino acid embedding vector, mutant amino acid embedding vector, and relative position projection vector of a single mutation record. Then, concatenate the above vectors along the feature dimension to obtain the concatenated vector. Then, perform dimensionality reduction and nonlinear transformation on the concatenated vector through a fusion network to generate a comprehensive representation of a single mutation.

[0013] S6. Use the Transformer encoder to learn the contextual relationships between mutations within a sample: Input the comprehensive representation sequence of each sample mutation into the Transformer encoder. During the encoding process, the padding mask is passed into the Transformer encoder. Based on the multi-head self-attention mechanism, the Transformer encoder learns the dependencies and cooperative patterns between different mutations within the same sample and outputs the comprehensive representation sequence of the encoded sample mutations.

[0014] S7. Generate sample-level representations using PMA pooling: Input the comprehensive representation sequence of encoded sample mutations into the PMA pooling module to aggregate a variable number of mutation sequences into a fixed-length sample-level representation, thus solving the problem of inconsistent input dimensions caused by the inconsistent number of mutations in tumor samples.

[0015] S8. Self-supervised training is performed using a masked gene prediction task. During training, the cross-entropy loss is calculated only for the masked positions as the masked gene prediction loss.

[0016] S9. Enhance the stability of sample-level representations by using contrastive learning: For the same sample, two enhanced views are constructed. The enhancement methods include randomly discarding some benign mutations and performing masking processing independently in the two enhanced views. The contrastive learning loss is calculated using the NT-Xent contrastive loss function during training.

[0017] Through this contrastive learning task, the model can learn sample-level molecular representations that are robust to benign mutation perturbations, random masks, and changes in intra-sample mutation subsets, avoiding the model's over-reliance on individual low-impact mutations or noisy mutations.

[0018] S10. Perform joint loss optimization, validation, and early stopping: During the training phase, the mask gene prediction loss and contrastive learning loss are weighted and summed to obtain the total training loss. The training data can be divided into training set and validation set according to a preset ratio. Data augmentation and masking are enabled on the training set, while fixed masking is enabled on the validation set but data augmentation is not enabled. After each round of training, the mask gene prediction loss is calculated on the validation set. When the validation loss decreases, the current model parameters are saved as the best model. When the validation loss does not improve for several consecutive rounds, early stopping is triggered.

[0019] S11. Extract and save sample-level mutation set embeddings: After the model training is completed, load the optimal model parameters, and input the sample mutation set into the model without performing masking and data augmentation. After feature encoding, fusion dimensionality reduction, Transformer encoding and PMA pooling, the model outputs the sample-level molecular representation corresponding to each sample.

[0020] This invention constructs a deep learning framework that selects gene identifiers, mutation types, mutation pathogenicity scores, amino acid alteration information, and the relative position of mutations in protein sequences as the main input features for the sample mutation set. The selection of these features is not a simple listing, but rather corresponds to different technical objectives in tumor mutation characterization, used to characterize the molecular features of tumor samples from multiple levels, including "where the mutation occurs," "what mechanism the mutation belongs to," "how much functional impact the mutation causes," "what amino acid changes the mutation leads to," and "the relative region of the mutation within the protein's functional structure."

[0021] First, gene identifier features are selected to characterize the functional gene units affected by mutations. Tumor development and progression are usually related to specific cancer-related genes, driver genes, or frequently mutated genes. Simply using the number of mutations is insufficient to distinguish whether a mutation occurs in a key gene. Therefore, this invention encodes the gene name corresponding to each mutation record as a gene ID and maps it to a continuous vector through a gene embedding layer. This enables the model to learn the co-occurrence relationships, substitution relationships, and potential functional associations between different genes in tumor samples.

[0022] Second, selecting mutation type characteristics is to characterize the mutation mechanism reflected by single base substitution and its context. Different tumor samples may have similar sets of mutated genes, but they may originate from different mutation processes, such as DNA damage repair defects, exogenous exposure, or endogenous mutation processes.

[0023] The preferred mutation type feature of this invention is the SBS96 feature. SBS96 is an abbreviation for "Single Base Substitution, 96-channel," which describes not only the type of base substitution but also contains local sequence context information, providing the model with mechanistic clues at the mutation spectrum level. This invention encodes SBS96 as a mutation type ID and inputs it into the model through an embedding layer. The aim is to enable the model to not only focus on "which genes are mutated" but also learn "what mutation patterns occur," thereby improving its ability to distinguish samples from different molecular mechanisms.

[0024] Third, the selection of mutation pathogenicity score features aims to incorporate information about the intensity of the mutation's functional impact. Whether a gene mutates is a discrete event and cannot reflect the magnitude of its impact on protein function or disease relevance. Different mutations on the same gene can have completely different biological consequences; some are benign or even harmful, while others can significantly affect protein function. This invention uses the pathogenicity score of each mutation as a continuous numerical feature input and transforms it into a learnable vector representation for the model through linear projection. The aim is to enable the model to distinguish between high-impact and low-impact mutations in sample representations, enhancing its focus on potential driver mutations or function-related mutations, thereby reducing information loss caused by simply relying on the presence or absence of mutations. In this invention, the pathogenicity score of each mutation is preferentially scored using the AlphaMissense prediction model.

[0025] Fourth, selecting amino acid alteration information features is to explicitly describe the changes in protein residues before and after mutation. The functional consequences of a mutation depend not only on the gene it originates from but also on which amino acid residues are replaced. For example, different amino acids differ in charge, polarity, hydrophobicity, volume, and structural constraints, and these differences affect protein folding, stability, binding sites, and enzyme activity. This invention prioritizes wild-type and mutant amino acid features, splitting amino acid alteration information into two independent features—wild-type and mutant—and representing them separately through embedding, rather than treating them as a single string feature. The aim of this design is to enable the model to learn the relationship between changes in "residue properties before mutation" and "residue properties after mutation," thereby more accurately characterizing the potential impact of amino acid substitution on protein function.

[0026] Fifth, the relative position information of mutations in the protein sequence is selected, referred to as relative position features, to describe the normalized spatial location of mutations in the protein sequence. Even if two mutations have the same amino acid substitution, they may have different effects when they occur in different regions of the protein. For example, mutations located near functional domains, binding regions, or conserved regions may be more functionally significant than mutations located in non-critical regions. Since different proteins have different lengths, directly using absolute positions is not conducive to cross-gene and cross-protein comparisons. Therefore, this invention uses relative position features and projects them as continuous vectors. The purpose of this processing is to enable the model to obtain the spatial location information of mutations without relying on a fixed protein length, thereby helping to determine whether a mutation may be located in a functionally sensitive region.

[0027] The present invention also has the following preferred designs:

[0028] In step S2 of this invention, when statistically analyzing the frequency of each gene in different tumor samples in the training data, the frequency ranking of each gene in the high-frequency gene list is recorded. This frequency ranking is used to set the mask probability of different genes during mask training in subsequent steps. By employing different masking strategies for genes of different frequencies, the problem of high-frequency genes dominating training and low-frequency genes being under-learned can be alleviated.

[0029] Corresponding to step S8, to improve the model's learning ability for long-tail genes, a hierarchical masking strategy based on gene frequency ranking is adopted: for genes with high frequency, a lower mask probability is set; for genes with medium frequency, a medium mask probability is set; and for genes with low frequency, a higher mask probability is set. This strategy increases the model's learning opportunities for low-frequency genes, alleviating the problem of uneven gene distribution in mutation data. At the model output, a gene prediction header is set. This header outputs the probability distribution of the mutation position belonging to each gene category based on the Transformer-encoded mutation position representation.

[0030] In step S5 of this invention, the fusion network sequentially includes a linear transformation layer, a layer normalization layer, a nonlinear activation layer, and a random deactivation layer. The linear transformation layer reduces the dimensionality of the concatenated vectors, enabling heterogeneous features from different sources to be expressed in a unified latent space, thereby improving the processing efficiency of the subsequent encoder; the layer normalization layer is used to stabilize the numerical distribution of features from different sources; the nonlinear activation layer is used to introduce nonlinear expressive power; and the random deactivation layer is used to reduce the risk of overfitting.

[0031] The present invention has the following beneficial effects:

[0032] 1. This invention selects gene identifiers, mutation types, mutation pathogenicity scores, amino acid alteration information, and the relative position of mutations in protein sequences as the main input features for multi-source fusion of sample mutation sets, avoiding the problem that a single mutation field cannot fully reflect the functional impact of mutations. The selected features cover the gene level, mutation mechanism level, functional impact level, protein residue change level, and protein position level of mutations, respectively. Gene identifier features are used to identify the target of mutations, mutation type features are used to characterize the mutation mechanism, pathogenicity score features are used to measure the intensity of the functional impact of mutations, amino acid alteration information features are used to describe changes in protein residues, and relative position features are used to supplement the positional information of mutations in the protein structural sequence. These features are fused after embedding or linear projection and input into a mutation set model based on a self-attention mechanism. This allows the model to learn the contextual relationships and cooperative patterns between multiple mutations within the same sample, ultimately obtaining a feature vector that better reflects the molecular characteristics of tumor samples at the sample level. This vector can be used for subsequent tumor subtyping, sample clustering, prognostic prediction, treatment response prediction, similar case retrieval, or other downstream analysis tasks.

[0033] 2. This invention combines masked gene prediction in step S8 with comparative learning in step S9 for self-supervised training, enabling the learning of stable sample-level mutation representations without relying on additional artificial labels.

[0034] 3. In the preferred design of the present invention, by adopting different masking strategies for genes of different frequencies, the problem of high-frequency genes dominating training and low-frequency genes being under-learned can be alleviated. Combined with the enhancement method of randomly discarding some benign mutations in step S9, and the weighted summation strategy of masking gene prediction loss and contrastive learning loss in step 10, the robustness of the model to noise mutations, passenger mutations and low-frequency genes is improved.

[0035] 4. This invention learns the contextual relationships between multiple mutations within the same sample through a Transformer encoder, which can capture mutation co-occurrence patterns and potential synergistic effects; it uses PMA pooling to uniformly map samples with different numbers of mutations into fixed-length vectors, thus solving the problem of inconsistent input dimensions caused by the inconsistent number of mutations in tumor samples. Attached Figure Description

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0037] Figure 1 This is an overall flowchart of a representation learning method for a set of missense mutations in tumor samples, which illustrates the overall process from inputting raw mutation data, data preprocessing, constructing the sample mutation set, encoding mutation-level features, learning sample-level representations, to model training and output.

[0038] Figure 2 The diagram shows the multi-source feature encoding and fusion dimensionality reduction structure of a single mutation in the example. The diagram illustrates the process of embedding or linearly projecting gene ID, SBS96 mutation type, pathogenicity score, wild-type amino acid, mutant amino acid and relative position features into a 608-dimensional vector, and then reducing the dimensionality to a 512-dimensional mutation comprehensive representation through a fusion network.

[0039] Figure 3 The diagram shows the structure of the mutation set Transformer sample-level representation generation in the embodiment. The diagram illustrates the process of inputting the mutation comprehensive representation sequence into the Transformer encoder, learning the contextual relationship between mutations through the self-attention mechanism, and then generating a 512-dimensional sample-level representation through PMA pooling.

[0040] Figure 4 The diagram illustrates the joint training of masked gene prediction and contrastive learning in this embodiment. It shows the process of constructing two enhanced views for the same sample, obtaining representations through a shared encoding model, and simultaneously calculating the masked gene prediction loss and contrastive learning loss for joint optimization. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0042] It should be understood that the present invention can be implemented in different forms and should not be construed as being limited to the embodiments set forth herein. Without departing from the inventive concept, those skilled in the art can make appropriate adjustments to the relevant parameters, number of model layers, vector dimensions, number of training rounds, sample selection thresholds, or feature types; such equivalent substitutions or modifications should all fall within the protection scope of the present invention.

[0043] Example:

[0044] A representation learning method for missense mutation sets in tumor samples, such as Figure 1 As shown, this embodiment uses somatic mutation data of tumor samples as input, performs feature encoding, dimensionality reduction fusion, mutation set modeling, and sample-level representation learning on multiple mutations within each sample, ultimately obtaining a low-dimensional continuous vector that can characterize the molecular features of the tumor sample. This vector can be used for subsequent tumor subtyping, sample clustering, prognosis prediction, treatment response prediction, similar case retrieval, or other downstream analysis tasks. This method mainly includes the following steps:

[0045] S1. Obtain the raw mutation data and perform field filtering:

[0046] In this embodiment, the original mutation data can be in MAF format or the processed mutation table data. Each mutation record includes at least the sample identifier, the name of the mutated gene, the mutation type, the pathogenicity score, the pathogenicity category, the amino acid change information, and the relative position of the mutation in the protein sequence.

[0047] Specifically, the sample identifier can be Tumor_Sample_Barcode, used to distinguish different tumor samples; the mutated gene name can be Hugo_Symbol, used to represent the gene in which the mutation occurred; the mutation type can be SBS96 mutation context type, used to represent a single base substitution and its adjacent sequence context; the pathogenicity score selects the AlphaMissense prediction model, using the am_pathogenicity score, used to represent the degree of influence of the mutation on protein function or disease relevance; the pathogenicity category can be the pathogenicity category am_class predicted by AlphaMissense, used to distinguish between benign and non-benign mutations; the amino acid change information can be AAchange, used to represent the change of amino acid residues before and after the mutation; the relative position information can be Relative_Position, used to represent the normalized position of the mutation site in the protein sequence.

[0048] During data preprocessing, mutation records with missing values ​​in the aforementioned essential fields are first removed to ensure that the data entering the model possesses complete multi-source features. Subsequently, sample identifiers are stringified to avoid errors arising during sorting or grouping of different data types. This step improves data consistency and reduces the interference caused by missing fields during model training.

[0049] S2. Construct a hybrid gene vocabulary and encode mutant genes:

[0050] See Figure 1 After completing the field filtering, a hybrid gene thesaurus is constructed. Specifically,

[0051] First, a set of cancer-related genes is read, such as a cancer gene catalog or a list of cancer driver genes. Simultaneously, the frequency of each gene appearing in different tumor samples in the training data is counted, and a number of genes with the highest frequency are selected as a high-frequency gene set. In this embodiment, the top 8000 high-frequency genes can be selected. Then, the union of the cancer-related gene set and the high-frequency gene set is taken to form a core gene vocabulary, and a special category representing unknown or low-frequency genes is added. <other>For each mutation record, if its gene name belongs to the core gene vocabulary, the gene is mapped to the corresponding gene ID; otherwise, it is mapped to... <other>category;

[0052] This processing step can preserve known cancer-related genes and representative high-frequency genes in the dataset, while avoiding problems such as an excessively large vocabulary, sparse training, and unstable model caused by extremely low-frequency genes.

[0053] Furthermore, this embodiment can also record the frequency ranking of each gene in the high-frequency gene list, and this ranking is used to set the mask probability of different genes during subsequent mask training.

[0054] S3. Encode the mutation type, pathogenicity score, amino acid alteration information, and relative position information for each mutation record.

[0055] like Figure 2 As shown, this embodiment constructs a multi-source feature representation for a single mutation record. For a mutation, the input features include gene ID, SBS96 mutation type ID, pathogenicity score, wild-type amino acid ID, mutant amino acid ID, and relative position value.

[0056] In this model, the SBS96 mutation type is mapped to a discrete category ID via a label encoder. This feature is used to represent the type of base substitution that causes the mutation and its contextual sequence pattern, enabling the model to learn the differences between different mutation mechanisms.

[0057] The pathogenicity score, as a continuous numerical feature input, represents the strength of the functional impact of the mutation. Compared to simply using the presence or absence of a mutation, introducing a pathogenicity score allows the model to distinguish between high-impact and low-impact mutations, enhancing the model's ability to focus on potential driver mutations or functionally relevant mutations.

[0058] For amino acid change information, this embodiment pre-constructs an amino acid vocabulary. This vocabulary includes 20 basic amino acids, as well as special symbols such as PAD, UNK, and STOP. For the AAchange field, such as "R>H" or "A>V", this embodiment splits it into wild-type amino acids and mutant amino acids, and encodes them as corresponding amino acid IDs. If the amino acid change format cannot be recognized, it is encoded as UNK. Encoding the amino acids before and after the mutation separately is beneficial for the model to learn the properties of residues before the mutation, the properties of residues after the mutation, and the relationship between the two.

[0059] The relative position feature is a continuous numerical value representing the normalized position of the mutation site within the protein sequence. Using relative position instead of absolute position enhances comparability between samples from different genes and proteins, especially when proteins have varying lengths. This feature helps models determine whether a mutation is likely located near a functional, conserved, or structurally sensitive region of the protein.

[0060] S4. Construct a mutation set based on the samples and perform serialization:

[0061] After completing the feature encoding of a single mutation record, this embodiment groups the mutation records according to the sample identifier Tumor_Sample_Barcode, so that multiple mutation records of the same sample form a mutation set. For each sample, its mutation set includes multiple mutations, and each mutation includes a gene ID, mutation type ID, pathogenicity score, pathogenicity category, wild-type amino acid ID, mutant amino acid ID, and relative position value.

[0062] Before grouping, this embodiment can sort mutation records according to sample identification and pathogenicity score. Specifically, within the same sample, mutations are arranged from high to low pathogenicity score, allowing mutations with higher functional impact to preferentially enter the beginning of the sequence. This process is beneficial for prioritizing the retention of high-impact mutations when the number of mutations in a sample exceeds the maximum length.

[0063] This embodiment also performs quality filtering on the samples. Specifically, only samples with a total number of mutations not less than a preset threshold and a number of different genes involved in the mutations not less than a preset threshold are retained. In one specific implementation, the threshold for the total number of mutations can be set to 10, and the threshold for the number of different genes can be set to 3. In this way, samples with too little mutation information and difficulty in forming a stable sample representation can be removed.

[0064] Before inputting the model, a maximum mutation sequence length is set for each sample. In this embodiment, the maximum sequence length is 256. Since the 0th position of the sequence is used to place a special CLS marker, the actual mutation occupies a maximum of 255 positions. If the number of mutations in a sample exceeds the maximum length, the first 255 mutations are truncated; if the number of mutations in any sample is less than the maximum mutation sequence length, a padding mask is used to fill the gaps, so that the subsequent Transformer encoder ignores the padding positions.

[0065] S5. Construct a comprehensive characterization of a single mutation:

[0066] First, the gene identifier embedding vector, mutation type embedding vector, pathogenicity score projection vector, wild-type amino acid embedding vector, mutant amino acid embedding vector, and relative position projection vector of a single mutation record are obtained. Then, the above vectors are concatenated along the feature dimension to obtain a concatenated vector. Finally, the concatenated vector is subjected to dimensionality reduction and nonlinear transformation through a fusion network to generate a comprehensive representation of a single mutation.

[0067] like Figure 2 As shown, for each mutation record, this embodiment performs embedding or linear projection processing on different features respectively.

[0068] Specifically, the gene ID is input into the gene embedding layer to obtain a 256-dimensional gene embedding vector; the SBS96 mutation type ID is input into the mutation type embedding layer to obtain a 128-dimensional mutation type embedding vector; the pathogenicity score is input into the linear projection layer to obtain a 64-dimensional pathogenicity score vector; the wild-type amino acid ID is input into the amino acid embedding layer to obtain a 64-dimensional wild-type amino acid embedding vector; the mutant amino acid ID is input into the amino acid embedding layer to obtain a 64-dimensional mutant amino acid embedding vector; and the relative position value is input into the linear projection layer to obtain a 32-dimensional relative position vector.

[0069] Subsequently, the six types of vectors are concatenated along their feature dimensions to obtain a 608-dimensional concatenated vector. Where: 608 = 256 + 128 + 64 + 64 + 64 + 32. This 608-dimensional concatenated vector simultaneously contains multi-source information such as the gene containing the mutation, the mutation context type, the pathogenicity of the mutation, the amino acid residues before and after the mutation, and the relative position of the mutated protein.

[0070] After obtaining the concatenated vector, this embodiment uses a fusion network to perform dimensionality reduction and nonlinear transformation on it, generating a comprehensive representation of a single mutation. The fusion network sequentially includes a linear transformation layer, a layer normalization layer, a nonlinear activation layer, and a random deactivation layer. The linear transformation layer maps the 608-dimensional concatenated vector to a 512-dimensional vector; the layer normalization layer is used to stabilize the numerical distribution of features from different sources; the nonlinear (ReLU) activation layer is used to introduce nonlinear expressive power; and the random deactivation layer is used to reduce the risk of overfitting.

[0071] After processing by the fusion network, each mutation record is represented as a 512-dimensional continuous vector. This vector is the comprehensive representation of a single mutation. This comprehensive representation is not an original artificial field, but a latent representation automatically learned by the model based on multi-source features, used to comprehensively reflect the joint information of the mutation in terms of gene background, mutation mechanism, functional impact, amino acid changes, and protein location.

[0072] For a batch of samples, the tensor shape output by the fusion network can be represented as: [batch size, maximum sequence length, 512].

[0073] In this embodiment, the batch size is 256 and the maximum sequence length is 256. Therefore, the shape of the output tensor of the fusion network is [256, 256, 512].

[0074] The second dimension corresponds to the mutation location or special marker location in each sample, and each location corresponds to a 512-dimensional vector.

[0075] S6. Use the Transformer encoder to learn the contextual relationships between mutations within a sample:

[0076] like Figure 3 As shown, in this embodiment, the comprehensive characterization sequence of each sample mutation is input into the Transformer encoder. The Transformer encoder is based on a multi-head self-attention mechanism to learn the dependencies and cooperative patterns between different mutations within the same sample. Compared with simply summing or averaging the mutation features, the self-attention mechanism can dynamically allocate weights according to the contextual relationship between mutations, thereby more accurately representing the overall molecular features of the sample.

[0077] In this embodiment, the Transformer encoder includes 6 encoding layers, each with 8 attention heads. The feedforward network has a dimension of 1024, the activation function is GELU, and dropout regularization is applied to prevent overfitting. The encoder's input and output dimensions are both 512.

[0078] During the encoding process, the padding mask is passed to the Transformer encoder, causing the model to ignore the padding locations (PADs) when calculating attention. This allows the model to handle the problem of inconsistent mutation numbers among different samples and ensures that the padding locations do not interfere with the sample representation.

[0079] The Transformer encoder outputs a comprehensive representation sequence of mutations in the encoded sample. This comprehensive representation sequence is still a mutation-level sequence, with each mutation location corresponding to a 512-dimensional vector that incorporates contextual information. This vector contains not only the multi-source feature information of the mutation itself, but also its association information with other mutations within the same sample.

[0080] S7. Generate sample-level representations using PMA pooling:

[0081] like Figure 3 As shown, the comprehensive representation sequence of the encoded sample mutations is input into the PMA pooling module, which aggregates a variable number of mutation sequences into a fixed-length sample-level representation, thus solving the problem of inconsistent input dimensions caused by the inconsistent number of mutations in tumor samples.

[0082] Specifically, the PMA pooling module sets several learnable seed vectors. In this embodiment, the number of seed vectors is 8. The PMA pooling module uses the seed vectors as query vectors and the Transformer-encoded mutation sequences as keys and values, extracting global information from the mutation set through a multi-head attention mechanism. Subsequently, the output representations corresponding to multiple seed vectors are averaged to obtain a fixed-length sample-level vector. The final sample-level molecular representation is a 512-dimensional continuous vector. This vector can be understood as the mutation set embedding of the tumor sample, comprehensively reflecting the overall pattern of multiple mutations in the sample at the levels of gene, mutation type, pathogenicity, amino acid changes, and protein position.

[0083] S8. Self-supervised training is performed using a masked gene prediction task. During training, the cross-entropy loss is calculated only for the masked positions as the masked gene prediction loss.

[0084] like Figure 4 As shown, this embodiment uses a masked gene prediction task as one of the self-supervised training objectives. Specifically, during the training phase, for partial mutations in each sample, their gene IDs are replaced with special MASK markers, and the model is required to predict the true gene category of the masked gene based on other features of the mutation and the context of other mutations within the sample.

[0085] To improve the model's learning ability for long-tail genes, this embodiment employs a hierarchical masking strategy based on gene frequency ranking. Genes with high frequency are assigned a lower masking probability; genes with medium frequency are assigned a medium masking probability; and genes with low frequency are assigned a higher masking probability. In this embodiment, the masking probability for the top 500 genes is 0.15, the masking probability for genes ranked 501 to 3000 is 0.25, and the masking probability for the remaining genes is 0.40. This strategy increases the model's learning opportunities for low-frequency genes, alleviating the problem of uneven gene distribution in mutation data.

[0086] At the model output, a gene prediction head is set. This prediction head represents the mutation position after Transformer encoding and outputs the probability distribution of that position belonging to each gene category. During training, the cross-entropy loss is calculated only for masked positions; unmasked positions and padded positions are not included in this loss calculation. This loss can be denoted as Loss_mlm.

[0087] S9. Employing contrastive learning to enhance the stability of sample-level representations:

[0088] like Figure 4 As shown, this embodiment further introduces a contrastive learning task. For the same sample, two enhanced views are constructed. The enhancement methods include randomly discarding some benign mutations and performing masking processing independently in the two enhanced views. Specifically, when some mutations are marked as benign, 10% to 30% of the benign mutations can be randomly discarded to simulate the uncertainty of low-impact or passenger mutations in the sample.

[0089] For two augmented views generated from the same sample, they are input into the same encoding model to obtain two sample-level representations. These sample-level representations are then input into a contrastive learning projection head to obtain a low-dimensional contrastive learning projection vector. In this embodiment, the projection vector dimension is 128, and the NT-Xent contrastive loss function is used to calculate the contrastive learning loss during training. This ensures that the two augmented views of the same sample are close to each other in the representation space, making the representations of different samples distinguishable. This loss can be denoted as Loss_cl.

[0090] Through this contrastive learning task, the model can learn sample-level molecular representations that are robust to benign mutation perturbations, random masks, and changes in intra-sample mutation subsets, avoiding the model's over-reliance on individual low-impact mutations or noisy mutations.

[0091] S10. Perform joint loss optimization, validation, and early stopping:

[0092] During the training phase, the mask gene prediction loss and the contrastive learning loss are weighted and summed to obtain the total training loss, Loss:

[0093] Loss=λ_mlm×Loss_mlm+λ_cl×Loss_cl.

[0094] In a specific implementation, λ_mlm is the weight of the masked gene prediction loss, set to 1.0, λ_cl is the weight of the contrastive learning loss, set to 0.1, and the contrastive learning temperature parameter is set to 0.2. The optimizer uses AdamW, with a learning rate and weight decay set to 1e-4, and training is performed using a linear learning rate scheduling strategy. During training, the gradient norm can be pruned to improve training stability.

[0095] Training data can be divided into training and validation sets according to a preset ratio. In this embodiment, the filtered samples are divided into an 85% training set and a 15% validation set. Data augmentation and masking are enabled on the training set, while a fixed mask is enabled on the validation set but data augmentation is not enabled to ensure that the validation loss is comparable.

[0096] After each training round, the masked gene prediction loss is calculated on the validation set. When the validation loss decreases, the current model parameters are saved as the optimal model. If the validation loss does not improve for several consecutive rounds, early stopping is triggered. In this embodiment, the early stopping patience value can be set to 30. By using the validation loss and early stopping mechanism, overfitting can be reduced and the model's generalization ability can be improved.

[0097] S11. Extract and save the sample-level mutation set embeddings:

[0098] After model training is complete, the optimal model parameters are loaded, and the sample mutation set is input into the model without performing masking or data augmentation. In this embodiment, after feature encoding, fusion dimensionality reduction, Transformer encoding, and PMA pooling, the model outputs a 512-dimensional sample-level molecular representation for each sample.

[0099] This sample-level molecular representation can be saved as a matrix file, such as patient_embeddings.npy; simultaneously, a sample identifier file, such as patient_ids.tsv, can be saved to establish the correspondence between sample identifiers and embedding vectors. The saved sample-level embeddings can then be used as input features for subsequent machine learning models, statistical analysis models, or clinical prediction models.

[0100] The above embodiments of the present invention are not intended to limit the scope of protection of the present invention. The implementation of the present invention is not limited thereto. All other modifications, substitutions or alterations made to the above structure of the present invention based on the above content of the present invention, in accordance with ordinary technical knowledge and common practice in the field, without departing from the basic technical idea of ​​the present invention, shall fall within the scope of protection of the present invention.< / other> < / other> < / other> < / other>

Claims

1. A representation learning method for a set of missense mutations in tumor samples, characterized in that, Includes the following steps: S1. Obtain raw mutation data and perform field filtering: Each mutation record should include at least the sample identifier, mutated gene name, mutation type, pathogenicity score, pathogenicity category, amino acid change information, and the relative position of the mutation in the protein sequence; S2. Constructing a hybrid gene vocabulary and encoding mutated genes: First, read the set of cancer-related genes and simultaneously count the frequency of each gene in different tumor samples in the training data. Select several genes with the highest frequency of occurrence as a high-frequency gene set. Then, take the union of the cancer-related gene set and the high-frequency gene set to form the core gene vocabulary, and add a special category to represent unknown or low-frequency genes. <other>For each mutation record, if its gene name belongs to the core gene vocabulary, the gene is mapped to the corresponding gene ID; otherwise, it is mapped to... <other> category;< / other> < / other> S3. Encode the mutation type, pathogenicity score, amino acid alteration information, and relative position information for each mutation record; S4. Construct mutation sets by sample and perform serialization: After completing the feature encoding of a single mutation record, the mutation records are grouped according to the sample identifier so that multiple mutation records of the same sample form a mutation set. Before inputting into the model, a maximum mutation sequence length is set for each sample. If the number of mutations in any sample is less than the maximum mutation sequence length, a mask is used for padding and a padding mask is constructed. S5. Constructing a comprehensive representation of a single mutation: First, obtain the gene identifier embedding vector, mutation type embedding vector, pathogenicity score projection vector, wild-type amino acid embedding vector, mutant amino acid embedding vector, and relative position projection vector of a single mutation record. Then, concatenate the above vectors along the feature dimension to obtain the concatenated vector. Then, perform dimensionality reduction and nonlinear transformation on the concatenated vector through a fusion network to generate a comprehensive representation of a single mutation. S6. Use the Transformer encoder to learn the contextual relationships between mutations within a sample: Input the comprehensive representation sequence of each sample mutation into the Transformer encoder. During the encoding process, the padding mask is passed into the Transformer encoder. Based on the multi-head self-attention mechanism, the Transformer encoder learns the dependencies and cooperative patterns between different mutations within the same sample and outputs the comprehensive representation sequence of the encoded sample mutations. S7. Generate sample-level representations using PMA pooling: Input the comprehensive representation sequence of encoded sample mutations into the PMA pooling module to aggregate a variable number of mutation sequences into a fixed-length sample-level representation; S8. Self-supervised training is performed using a masked gene prediction task. During training, the cross-entropy loss is calculated only for the masked positions as the masked gene prediction loss. S9. Enhance the stability of sample-level representations by using contrastive learning: For the same sample, two enhanced views are constructed. The enhancement methods include randomly discarding some benign mutations and performing masking processing independently in the two enhanced views. The contrastive learning loss is calculated using the NT-Xent contrastive loss function during training. S10. Perform joint loss optimization, validation, and early stopping: During the training phase, the mask gene prediction loss and contrastive learning loss are weighted and summed to obtain the total training loss. The training data is divided into training set and validation set according to a preset ratio. Data augmentation and masking are enabled on the training set, while fixed masking is enabled on the validation set but data augmentation is not enabled. After each round of training, the mask gene prediction loss is calculated on the validation set. When the validation loss decreases, the current model parameters are saved as the best model. When the validation loss does not improve for several consecutive rounds, early stopping is triggered. S11. Extract and save sample-level mutation set embeddings: After the model training is completed, load the optimal model parameters, and input the sample mutation set into the model without performing masking and data augmentation. After feature encoding, fusion dimensionality reduction, Transformer encoding and PMA pooling, the model outputs the sample-level molecular representation corresponding to each sample.

2. The representation learning method for missense mutation sets of tumor samples according to claim 1, characterized in that: The mutation type adopts the SBS96 mutation context type.

3. The representation learning method for missense mutation sets of tumor samples according to claim 1, characterized in that: The pathogenicity score of each mutation was scored using the AlphaMissense prediction model.

4. The representation learning method for missense mutation sets of tumor samples according to claim 1, characterized in that: In step S2, when counting the frequency of each gene in different tumor samples in the training data, the frequency ranking of each gene in the high-frequency gene list is recorded. In step S8, a hierarchical masking strategy based on gene frequency ranking is adopted: for genes with high frequency, a lower masking probability is set; for genes with medium frequency, a medium masking probability is set; and for genes with low frequency, a higher masking probability is set.

5. The representation learning method for missense mutation sets of tumor samples according to claim 1, characterized in that: In step S5, the fusion network sequentially includes a linear transformation layer, a layer normalization layer, a nonlinear activation layer, and a random deactivation layer. The linear transformation layer reduces the dimensionality of the concatenated vectors, enabling heterogeneous features from different sources to be expressed in a unified latent space. The layer normalization layer is used to stabilize the numerical distribution of features from different sources. The nonlinear activation layer is used to introduce nonlinear expressive power. The random deactivation layer is used to reduce the risk of overfitting.

Images