A single cell annotation method of a long-tail data distribution generation gene large model

Abstract

The application discloses a method for generating a single cell annotation method of a long-tail data distribution gene large model. The method comprises the following steps: obtaining a first single cell data set and a second single cell data set; constructing a target loss function, the target function is used to balance the expression difference of different single cell types; based on the first single cell data set, the pre-trained single cell annotation network model is obtained by pre-training the constructed single cell annotation network model; based on the second single cell data set and the target loss function, the target single cell annotation network model is obtained by fine-tuning the pre-trained single cell annotation network model; and the target single cell annotation network model is used to annotate the single cell to be annotated, and the annotated single cell is obtained. The application can effectively solve the long-tail distribution problem of the data set, thereby improving the accuracy and generalization ability of cell annotation.

Description

Technical Field

[0001] This application relates to the field of bioinformatics technology, and in particular to a single-cell annotation method for generative large-scale gene models under long-tailed data distribution. Background Technology

[0002] Current single-cell annotation techniques cannot effectively address the long-tail distribution problem of datasets, resulting in limited accuracy in cell annotation tasks. This prevents models from fully capturing information about low-expression cell types during training, thereby reducing the accuracy and generalization ability of cell annotation. Summary of the Invention

[0003] This application aims to propose a single-cell annotation method for generative large-scale models under long-tailed data distribution, which can effectively solve the problem of long-tailed dataset distribution, thereby improving the accuracy and generalization ability of cell annotation.

[0004] In a first aspect, embodiments of this application provide a single-cell annotation method for generative large-scale models under long-tailed data distributions, the method comprising:

[0005] Obtain the first and second single-cell datasets;

[0006] Construct a target loss function, which is used to balance the expression differences among different single cell types;

[0007] Based on the first single-cell dataset, the constructed single-cell annotation network model is pre-trained to obtain a pre-trained single-cell annotation network model.

[0008] Based on the second single-cell dataset and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain the target single-cell annotation network model.

[0009] The target single-cell annotation network model is used to annotate the single cells to be annotated, resulting in an annotated single cell.

[0010] Compared with the prior art, the first aspect of this application has the following beneficial effects:

[0011] This method constructs a target loss function to balance the expression differences among different single-cell types. Based on a first single-cell dataset, a pre-trained single-cell annotation network model is obtained. Then, based on a second single-cell dataset and the target loss function, the pre-trained model is fine-tuned to obtain a target single-cell annotation network model. Finally, the target model is used to annotate the cells to be annotated, resulting in annotated single cells. Thus, by pre-training the constructed single-cell annotation network model and then fine-tuning it using a target function that balances the expression differences among different single-cell types, the target single-cell annotation network model effectively balances the expression differences among different cell types, improving the identification ability of low-expression cell types. This effectively solves the problem of long-tailed dataset distribution and improves the accuracy and generalization ability of cell annotation.

[0012] In some implementations, the target loss function is constructed as follows:

[0013]

[0014] Where z = z1, z2, ..., z C This represents the data predicted by the target single-cell annotation model. y represents the adjusted predicted probability. i ∈{0,1} represents the true label of the i-th class sample, L SCDE (z) represents the objective loss function, and C represents the total amount of data to be predicted.

[0015] In some implementations, when constructing the target loss function, the method further includes calculating the adjusted prediction probability in the following manner:

[0016]

[0017] in, Let z represent the predicted probability adjusted for the i-th category. i Let represent the i-th category data predicted by the target single-cell annotation model, and A represent the adjustable parameter.

[0018] In some implementations, when constructing the target loss function, the method further includes calculating the adjustable parameter in the following manner:

[0019]

[0020] S ij =M ij ·C ij

[0021] Among them, S ij M represents an adjustable balance factor that adjusts the penalty for a category based on the ratio of the number of instances in categories i and j. ij C represents a mitigation factor that reduces the penalty for the tail category j by adjusting the ratio of the number of instances between categories i and j. ij This indicates that when a misclassified sample of category i is observed, a penalty compensation factor for category j is increased.

[0022] In some implementations, the single-cell annotation network model includes Bayesian downsampling, masking, embedded modules, an encoder, a decoder, and a multilayer perceptron. The step of fine-tuning the pre-trained single-cell annotation network model based on the second single-cell dataset and the target loss function to obtain the target single-cell annotation network model includes:

[0023] The data from the second single-cell dataset is input into the pre-trained single-cell annotation network model;

[0024] The second single-cell dataset is downsampled using the Bayesian downsampling method to obtain downsampled data;

[0025] Perform a masking operation on the downsampled data to obtain the masked data;

[0026] The masked data is input into the embedded module to obtain zero-value mask data and non-zero-value mask data;

[0027] The non-zero value mask data is encoded by the encoder to obtain the encoded data;

[0028] The encoded data is pooled to obtain pooled data;

[0029] The pooled data and the zero-value mask data are input into the multilayer perceptron to obtain the output of the pre-trained single-cell annotation network model.

[0030] Based on the data in the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain the target single-cell annotation network model.

[0031] In some implementations, the step of fine-tuning the pre-trained single-cell annotation network model based on data from the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function to obtain the target single-cell annotation network model includes:

[0032] Based on the data in the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function, the pre-trained single-cell annotation network model is fine-tuned using a gradient accumulation optimization strategy to obtain the target single-cell annotation network model.

[0033] In some implementations, obtaining the first single-cell dataset and the second single-cell dataset includes:

[0034] Obtain raw single-cell sequencing data from the Zheng68k dataset;

[0035] The original single-cell sequencing data were normalized to obtain normalized data.

[0036] Based on the normalized data, the first single-cell dataset and the second single-cell dataset are determined.

[0037] Secondly, embodiments of this application also provide a single-cell annotation system for generative large-scale models under long-tailed data distribution, the system comprising:

[0038] The data acquisition unit is used to acquire the first single-cell dataset and the second single-cell dataset.

[0039] A function construction unit is used to construct a target loss function, which is used to balance the expression differences of different single cell types;

[0040] The model training unit is used to pre-train the constructed single-cell annotation network model based on the first single-cell dataset to obtain the pre-trained single-cell annotation network model.

[0041] The model fine-tuning unit is used to fine-tune the pre-trained single-cell annotation network model based on the second single-cell dataset and the target loss function to obtain the target single-cell annotation network model.

[0042] The cell annotation unit is used to annotate the single cell to be annotated using the target single-cell annotation network model to obtain an annotated single cell.

[0043] Thirdly, embodiments of this application also provide an electronic device, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform a single-cell annotation method for a generative large-scale model under a long-tailed data distribution as described above.

[0044] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to execute a single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution as described above.

[0045] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0046] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0047] Figure 1 This is a flowchart illustrating an embodiment of a single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution provided in this application.

[0048] Figure 2 This is a schematic diagram of the structure of a single-cell annotation network model in the best embodiment of a single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution provided in this application.

[0049] Figure 3 This is a schematic diagram of an embodiment of the single-cell annotation system for a generative large-scale gene model under long-tailed data distribution provided in this application.

[0050] Figure 4 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0051] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0052] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0053] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0054] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0055] Existing related technologies include scGPT and scFoundation. scGPT is a single-cell multi-omics foundational model developed by researchers. This model, through generative pre-training on over 33 million cells, captures the complex relationships between genes and cells, providing a robust foundation for various downstream tasks. The core idea of ​​scGPT is to apply the concept of pre-trained models from natural language processing to single-cell data analysis, learning general representations of cells and genes through large-scale pre-training, thereby supporting various downstream tasks. The main innovations of scGPT include large-scale pre-training, multimodal learning, flexible downstream applications, and an efficient computational architecture.

[0056] scFoundation is a large-scale pre-trained model for single-cell transcriptomics developed by researchers. This model has 100 million parameters, covers approximately 20,000 genes, and has been pre-trained on over 50 million human single-cell transcriptome profiles. scFoundation demonstrates state-of-the-art performance across various single-cell analysis tasks, such as gene expression enhancement and drug response classification. The development of scFoundation reveals the effectiveness and value of large-scale pre-trained models for transcriptomics data and demonstrates its significant application value in advancing biological and medical research.

[0057] However, the aforementioned single-cell annotation techniques cannot effectively address the long-tail distribution problem of datasets, resulting in limited accuracy in cell annotation tasks. Consequently, the model cannot fully capture information about low-expression cell types during training, thus reducing the accuracy and generalization ability of cell annotation.

[0058] To address the aforementioned issues of ineffective handling of long-tailed dataset distribution and reduced accuracy and generalization of cell annotation, this application proposes a single-cell annotation method for generative large-scale models under long-tailed data distribution.

[0059] Reference Figure 1This application provides a single-cell annotation method for generative large-scale gene models under long-tailed data distribution. The method includes the following steps:

[0060] Step S100: Obtain the first single-cell dataset and the second single-cell dataset;

[0061] Step S200: Construct a target loss function, which is used to balance the expression differences among different single cell types;

[0062] Step S300: Based on the first single-cell dataset, pre-train the constructed single-cell annotation network model to obtain the pre-trained single-cell annotation network model.

[0063] Step S400: Based on the second single-cell dataset and the target loss function, fine-tune the pre-trained single-cell annotation network model to obtain the target single-cell annotation network model.

[0064] Step S500: Use the target single-cell annotation network model to annotate the single cells to be annotated, and obtain annotated single cells.

[0065] In this embodiment, a target loss function is constructed to balance the expression differences among different single-cell types. Based on a first single-cell dataset, the constructed single-cell annotation network model is pre-trained to obtain a pre-trained single-cell annotation network model. Based on a second single-cell dataset and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain a target single-cell annotation network model. The target single-cell annotation network model is then used to annotate the single cells to be annotated, resulting in annotated single cells. Thus, by pre-training the constructed single-cell annotation network model and then fine-tuning it using a target function that balances the expression differences among different single-cell types, the target single-cell annotation network model can effectively balance the expression differences among different cell types, improving the recognition ability of low-expression cell types. This effectively solves the problem of long-tailed dataset distribution and improves the accuracy and generalization ability of cell annotation.

[0066] The expression level of the above single-cell type can refer to the expression level of a certain gene in a certain type of single cell, usually expressed as the amount of RNA or protein.

[0067] The constructed single-cell annotation network model described above can be a single-cell annotation network model constructed using the Transformer model.

[0068] The first and second single-cell datasets mentioned above can be the same single-cell dataset or different single-cell datasets; this embodiment does not impose specific limitations.

[0069] In some implementations, the target loss function is constructed as follows:

[0070]

[0071] Where z = z1, z2, ..., z C This represents the data predicted by the target single-cell annotation model. y represents the adjusted predicted probability. i ∈{0,1} represents the true label of the i-th class sample, L SCDE (z) represents the objective loss function, and C represents the total amount of data to be predicted.

[0072] In this embodiment, the constructed target loss function can effectively balance the expression differences of different cell types and significantly improve the ability to identify cell types with low expression levels.

[0073] In some implementations, the method further includes calculating the adjusted prediction probabilities by means of the following when constructing the target loss function:

[0074]

[0075] in, Z represents the predicted probability adjusted for the i-th category. i Let represent the i-th category data predicted by the target single-cell annotation model, and A represent the adjustable parameter.

[0076] In some implementations, the method further includes calculating adjustable parameters when constructing the target loss function in the following manner:

[0077]

[0078] S ij =M ij ·C ij

[0079] Among them, S ij M represents an adjustable balance factor that adjusts the penalty for a category based on the ratio of the number of instances in categories i and j. ij C represents a mitigation factor that reduces the penalty for the tail category j by adjusting the ratio of the number of instances between categories i and j. ij This indicates that when a misclassified sample of category i is observed, a penalty compensation factor for category j is increased.

[0080] In some implementations, the single-cell annotation network model includes Bayesian downsampling, masking, embedded modules, an encoder, a decoder, and a multilayer perceptron. Based on a second single-cell dataset and a target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain the target single-cell annotation network model, including:

[0081] The data from the second single-cell dataset is input into the pre-trained single-cell annotation network model;

[0082] The second single-cell dataset was downsampled using Bayesian downsampling to obtain the downsampled data;

[0083] Perform a masking operation on the downsampled data to obtain the masked data;

[0084] The masked data is input into the embedded module to obtain zero-value mask data and non-zero-value mask data;

[0085] Non-zero value mask data is encoded using an encoder to obtain the encoded data;

[0086] The encoded data is pooled to obtain the pooled data;

[0087] The pooled data and zero-value mask data are input into the multilayer perceptron to obtain the output of the pre-trained single-cell annotation network model.

[0088] Based on the data in the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain the target single-cell annotation network model.

[0089] In this embodiment, the pre-trained single-cell annotation network model is fine-tuned based on the data in the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function. This allows the target single-cell annotation network model to effectively balance the expression differences of different cell types, improve the recognition ability of low-expression cell types, effectively solve the long-tail distribution problem of the dataset, and improve the accuracy and generalization ability of cell annotation.

[0090] In some implementations, the pre-trained single-cell annotation network model is fine-tuned based on data from the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function to obtain the target single-cell annotation network model, including:

[0091] Based on the data in the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function, the pre-trained single-cell annotation network model is fine-tuned using a gradient accumulation optimization strategy to obtain the target single-cell annotation network model.

[0092] In this embodiment, a gradient accumulation optimization strategy is used to fine-tune the pre-trained single-cell annotation network model. The gradient accumulation optimization strategy performs forward and backward propagation of the model in mini-batch manner. During the iteration process, multiple mini-batch gradients are calculated and accumulated. When enough gradients are accumulated, the model optimization steps are executed to update the parameters, which can make the network parameters in the pre-trained single-cell annotation network model more optimized, and enable the target single-cell annotation network model to better identify cell types with low expression levels.

[0093] The basic idea of ​​gradient accumulation is to transform the gradient calculation of a one-time batch parameter update into a step-by-step process. Specifically, this method performs forward and backward propagation of the model in small batches, calculates and accumulates multiple small batch gradients during the iteration process, and executes the optimization steps of the model to update the parameters when enough gradients have been accumulated.

[0094] In some implementations, obtaining the first single-cell dataset and the second single-cell dataset includes:

[0095] Obtain raw single-cell sequencing data from the Zheng68k dataset;

[0096] The raw single-cell sequencing data were normalized to obtain the normalized data.

[0097] Based on the normalized data, the first single-cell dataset and the second single-cell dataset are determined.

[0098] In this embodiment, the raw single-cell sequencing data is normalized so that the raw single-cell sequencing data in the Zheng68k dataset can meet the input requirements of the single-cell annotation network model.

[0099] The Zheng68k dataset mentioned above is a dataset that exists in the prior art, and it will not be described in detail in this embodiment.

[0100] To facilitate understanding by those skilled in the art, a set of preferred embodiments is provided below:

[0101] The existing technology has the following drawbacks:

[0102] 1. Long-tail problem of data distribution: In single-cell annotation tasks, existing technologies often face the problem of long-tail data distribution, that is, the high expression of a few cell types masks other types, resulting in a decrease in the accuracy of cell annotation.

[0103] 2. Lack of targeted loss function: Existing methods lack loss functions specifically designed for RNA single-cell long-tailed distribution datasets, and cannot effectively balance the expression differences of different cell types.

[0104] 3. Limited model generalization ability: Existing single-cell annotation models are usually trained on specific datasets and lack good generalization ability, especially when faced with new and unseen cell types.

[0105] To address the limitation of existing single-cell annotation techniques in effectively handling the long-tail distribution of datasets, which restricts accuracy in cell annotation tasks and prevents models from fully capturing information about low-expression cell types during training, thus reducing the accuracy and generalization ability of cell annotation, this embodiment specifically designs a method to solve the long-tail distribution problem of single-cell data. The technical solution of this embodiment is achieved by proposing a novel loss function, SCDE Loss.

[0106] The technical solution in this embodiment is based on a generative pre-trained Transformer model, hereinafter referred to as the scHigen model, which is a model built based on the Transformer model (i.e., a single-cell annotation network model) to annotate single cells in the Zheng68k dataset. The Zheng68k dataset is a classic peripheral blood mononuclear cell (PBMC) dataset, generated by 10xGenomics' Chromium platform, containing approximately 68,450 cells divided into 11 different cell subtypes. This dataset is characterized by the inclusion of rare cell types and an uneven distribution of cell types, making cell type differentiation challenging. (Refer to...) Figure 2 The single-cell annotation network model includes Bayesian downsampling, masking, embedded modules, encoders, decoders, and multilayer perceptrons, specifically:

[0107] 1. Bayesian downsampling: First, the data is processed through a Bayesian downsampling process to obtain a downsampled data representation.

[0108] 2. Masking operation: Next, a masking operation is performed on the downsampled data to generate masked data.

[0109] 3. Embedded Module: The masked data enters the embedded module, which further processes the data to prepare for subsequent encoding and decoding.

[0110] 4. Zero-value masking: In embedded modules, data is divided into zero-value and non-zero-value parts. The zero-value part is processed through a zero-value masking process. Zero-value masking refers to directly representing missing data points with 0.

[0111] 5. Encoder: Non-zero value data is encoded by the encoder and converted into a more compact representation.

[0112] 6. Pooling: The encoded data undergoes pooling to further reduce data dimensionality and extract important features.

[0113] 7. Decoder: The pooled data enters the decoder, which restores the data to an approximate table of the original data.

[0114] 8. MLP (Multilayer Perceptron): The decoded data is further processed through MLP to improve the model's expressive power.

[0115] 9. Reconstruction Loss: Finally, the output of the single-cell annotation network model is compared with the original data to calculate the reconstruction loss. The SCDE (Spectral Contrastive Divergence Estimation) loss function is used to optimize the model.

[0116] The following are the detailed steps of the implementation principle of the technical solution in this embodiment:

[0117] 1. Pre-training stage: First, the raw single-cell sequencing data in the Zheng68k dataset is normalized to meet the input requirements of the scHigen model (i.e., the single-cell annotation network model). Then, the normalized dataset is used to pre-train the scHigen model.

[0118] 2. Fine-tuning stage: The pre-trained scHigen model is fine-tuned on the Zheng68k dataset to adapt to the characteristics of the specific dataset. A dataset with rare cell types from the Zheng68k dataset can be selected for fine-tuning the pre-trained scHigen model to obtain the target single-cell annotation network model.

[0119] 3. Introducing the SCDE Loss Function: Addressing the long-tailed distribution of the dataset, the SCDE loss function is introduced in the fine-tuning stage of step 2. This function is specifically designed to balance the expression differences between different cell types and improve the identification ability of cell types with low expression levels. Cell type expression level refers to the expression level of a gene in a cell, usually expressed as the amount of RNA or protein. The principle of SCDE is as follows:

[0120] The SCDE loss function aims to reduce the influence of head classes on the gradients of negative samples in tail classes. Specifically, the SCDE loss function adjusts the probability distribution predicted by the model to reduce the influence of head classes (with a large number of samples) on tail classes (with a small number of samples), while increasing the penalty for misclassification. This helps the model better learn the features of tail classes, thereby improving its ability to identify these classes. Its definition is as follows:

[0121]

[0122] Where z = z1, z2, ..., z C This represents the data predicted by the model. y represents the adjusted predicted probability. i ∈{0,1} represents the true label of the i-th class sample, indicating whether the sample belongs to class i. Adjusted predicted probability. The calculation is as follows: S ij It is an adjustable balancing factor that adjusts the penalty for a class based on the ratio of the number of instances between classes i and j. Balance factor S ij The calculation is as follows: S ij =M ij ·C ij , of which M ij It is a mitigation factor that reduces the penalty for the tail category j based on the ratio of the number of instances between categories i and j; C ij These are compensation factors that increase the penalty for class j when a misclassified sample of class i is observed. These factors work together to enable the SCDE loss function to effectively mitigate excessive penalties for tail classes and compensate for the risk of misclassification resulting from reduced penalties.

[0123] The technical solution in this embodiment differs from that in the prior art in the following ways:

[0124] 1. Loss function for long-tailed distribution: The SCDE loss function is the core innovation of this embodiment. It is specifically designed for long-tailed distribution datasets and effectively balances the expression differences of different cell types. Compared with the cross-entropy loss function commonly used in the prior art, SCDE can significantly improve the identification ability of low expression cell types.

[0125] 2. Model Training Strategy Combining Pre-training and Fine-tuning: This embodiment employs a pre-training and fine-tuning strategy. First, pre-training is performed on a large-scale dataset to learn gene and cell representations. Then, fine-tuning is performed on a specific dataset to adapt to its characteristics. Compared to existing methods that directly train the model on a specific dataset, this strategy better utilizes information from large-scale datasets and improves the model's generalization ability.

[0126] 3. Solution to the long-tailed distribution problem of datasets: The technical solution in this embodiment effectively solves the long-tailed distribution problem of datasets and improves the accuracy of cell annotation, which is a key problem that existing technologies have failed to effectively address. By introducing the SCDE loss function and an optimized model training strategy, this embodiment significantly improves the ability to identify low-expression cell types, thereby achieving higher accuracy and generalization ability in cell annotation tasks.

[0127] For better illustration, the following example is provided:

[0128] To verify the effectiveness of this embodiment, the Zheng68k dataset was selected for experimentation. First, the scHigen model was pre-trained and then fine-tuned on the Zheng68k dataset. During the fine-tuning phase, the SCDE loss function was introduced, and a gradient accumulation-based optimization strategy was adopted. The basic idea of ​​gradient accumulation is to transform the gradient calculation for a one-time batch parameter update into a step-by-step process. Specifically, this method performs forward and backward propagation of the model in mini-batches, calculating and accumulating multiple mini-batch gradients during iteration. When a sufficient number of gradients are accumulated, the model's optimization step is executed to update the parameters. Experimental results show that, compared with existing technologies, this embodiment significantly improves both accuracy and generalization ability in cell annotation tasks. Specifically, the model using the SCDE loss function improves the recognition accuracy for low-expression cell types by approximately 15%, and the overall cell annotation accuracy by approximately 10%. These results demonstrate the effectiveness and superiority of this embodiment in processing single-cell multi-omics data.

[0129] Reference Figure 3 This application also provides a single-cell annotation system for generative large-scale gene models under long-tailed data distribution. The system includes a data acquisition unit 100, a function construction unit 200, a model training unit 300, a model fine-tuning unit 400, and a cell annotation unit 500, wherein:

[0130] The data acquisition unit 100 is used to acquire the first single-cell dataset and the second single-cell dataset.

[0131] Function building unit 200 is used to construct the target loss function, which is used to balance the expression differences of different single cell types;

[0132] Model training unit 300 is used to pre-train the constructed single-cell annotation network model based on the first single-cell dataset to obtain the pre-trained single-cell annotation network model.

[0133] Model fine-tuning unit 400 is used to fine-tune the pre-trained single-cell annotation network model based on the second single-cell dataset and the target loss function to obtain the target single-cell annotation network model.

[0134] The cell annotation unit 500 is used to annotate the single cell to be annotated using the target single-cell annotation network model to obtain an annotated single cell.

[0135] It should be noted that since the single-cell annotation system for a generative large-scale gene model under a long-tailed data distribution in this embodiment is based on the same inventive concept as the single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution described above, the corresponding content in the method embodiment is also applicable to this system embodiment, and will not be described in detail here.

[0136] Reference Figure 4 This application also provides an electronic device, which includes:

[0137] At least one memory;

[0138] At least one processor;

[0139] At least one program;

[0140] The program is stored in memory, and the processor executes at least one program to implement the single-cell annotation method for a generative large model under a long-tailed data distribution as described above in this disclosure.

[0141] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0142] The electronic devices according to embodiments of this application will now be described in detail.

[0143] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure.

[0144] The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to execute a single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution according to an embodiment of this disclosure.

[0145] The input / output interface 1800 is used to implement information input and output.

[0146] The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0147] Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900);

[0148] The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0149] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the above-described single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution.

[0150] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0151] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0152] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0155] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0156] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0158] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0160] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. The embodiments of this application have been described in detail above with reference to the accompanying drawings, but this application is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of this application.

Claims

1. A single-cell annotation method for generative large-scale gene models under long-tailed data distribution, characterized in that, The method includes: Obtain the first and second single-cell datasets; A target loss function is constructed to balance the expression differences among different single-cell types. The target loss function is constructed as follows: in, This represents the data predicted by the target single-cell annotation model. This represents the adjusted predicted probability. Indicates the first The true labels of each category of samples Represents the target loss function. This represents the total amount of data to be predicted. Indicates the first The predicted probabilities after category adjustment The target single-cell annotation model predicts the first... Data by category, This indicates an adjustable parameter. Indicates by category and An adjustable balance factor is used to adjust the penalty for a category based on the ratio of the number of instances between them. Indicates by category and The ratio of the number of instances between them decreases for the tail category. The mitigating factor of punishment Indicates by observing categories When misclassifying samples, add category The punitive compensation factor; Based on the first single-cell dataset, the constructed single-cell annotation network model is pre-trained to obtain a pre-trained single-cell annotation network model. Based on the second single-cell dataset and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain the target single-cell annotation network model. The target single-cell annotation network model is used to annotate the single cells to be annotated, resulting in an annotated single cell.

2. The single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution according to claim 1, characterized in that, The single-cell annotation network model includes Bayesian downsampling, masking, embedded modules, an encoder, a decoder, and a multilayer perceptron. Based on the second single-cell dataset and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain the target single-cell annotation network model, including: The data from the second single-cell dataset is input into the pre-trained single-cell annotation network model; The second single-cell dataset is downsampled using the Bayesian downsampling method to obtain downsampled data; Perform a masking operation on the downsampled data to obtain the masked data; The masked data is input into the embedded module to obtain zero-value mask data and non-zero-value mask data; The non-zero value mask data is encoded by the encoder to obtain the encoded data; The encoded data is pooled to obtain pooled data; The pooled data and the zero-value mask data are input into the multilayer perceptron to obtain the output of the pre-trained single-cell annotation network model. Based on the data in the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain the target single-cell annotation network model.

3. The single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution according to claim 2, characterized in that, The step of fine-tuning the pre-trained single-cell annotation network model based on the data in the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function to obtain the target single-cell annotation network model includes: Based on the data in the second single-cell dataset, the output of the pre-trained single-cell annotation network model, and the target loss function, the pre-trained single-cell annotation network model is fine-tuned using a gradient accumulation optimization strategy to obtain the target single-cell annotation network model.

4. The single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution according to claim 1, characterized in that, The acquisition of the first single-cell dataset and the second single-cell dataset includes: Obtain raw single-cell sequencing data from the Zheng68k dataset; The original single-cell sequencing data were normalized to obtain normalized data. Based on the normalized data, the first single-cell dataset and the second single-cell dataset are determined.

5. A single-cell annotation system for a generative large-scale gene model under a long-tailed data distribution, characterized in that, The system includes: The data acquisition unit is used to acquire the first single-cell dataset and the second single-cell dataset. A function construction unit is used to construct a target loss function, which is used to balance the expression differences among different single cell types. The target loss function is constructed as follows: in, This represents the data predicted by the target single-cell annotation model. This represents the adjusted predicted probability. Indicates the first The true labels of each category of samples Represents the target loss function. This represents the total amount of data to be predicted. Indicates the first The predicted probabilities after category adjustment The target single-cell annotation model predicts the first... Data by category, This indicates an adjustable parameter. Indicates by category and An adjustable balance factor is used to adjust the penalty for a category based on the ratio of the number of instances between them. Indicates by category and The ratio of the number of instances between them decreases for the tail category. The mitigating factor of punishment Indicates by observing categories When misclassifying samples, add category The punitive compensation factor; The model training unit is used to pre-train the constructed single-cell annotation network model based on the first single-cell dataset to obtain the pre-trained single-cell annotation network model. The model fine-tuning unit is used to fine-tune the pre-trained single-cell annotation network model based on the second single-cell dataset and the target loss function to obtain the target single-cell annotation network model. The cell annotation unit is used to annotate the single cell to be annotated using the target single-cell annotation network model to obtain an annotated single cell.

6. An electronic device, characterized in that, It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform a single-cell annotation method for a generative large-scale model under a long-tailed data distribution as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform a single-cell annotation method for a generative large-scale gene model under a long-tailed data distribution as described in any one of claims 1 to 4.

Images