Single cell annotation method of generative gene large model under long-tail data distribution

Abstract

The invention discloses a single cell annotation method for a generative gene large model under long-tail data distribution. The method comprises the following steps: acquiring a first single cell data set and a second single cell data set; constructing a target loss function, wherein the target function is used for balancing the expression quantity difference of different single cell types; based on the first single cell data set, pre-training the constructed single cell annotation network model to obtain a pre-trained single cell annotation network model; based on the second single cell data set and the target loss function, performing fine tuning on a pre-trained single cell annotation network model to obtain a target single cell annotation network model; and annotating a single cell to be annotated by adopting the target single cell annotation network model to obtain an annotated single cell. According to the method, the problem of data set long tail distribution can be effectively solved, so that the accuracy and generalization ability of cell annotation are improved.

Description

Technical Field

[0001] The present application relates to the technical field of bioinformatics, and in particular to a single-cell annotation method for a generative gene large model under long-tail data distribution. Background Art

[0002] The current relevant single-cell annotation technology cannot effectively solve the long-tail distribution problem of the data set, resulting in limited accuracy in cell annotation tasks. It makes the model unable to fully capture the information of low-expression cell types during training, thereby reducing the accuracy and generalization ability of cell annotation. Summary of the invention

[0003] This application aims to propose a single-cell annotation method for a generative gene model under long-tail data distribution, which can effectively solve the long-tail distribution problem of the data set, thereby improving the accuracy and generalization ability of cell annotation.

[0004] In a first aspect, an embodiment of the present application provides a single-cell annotation method for a generative gene large model under a long-tail data distribution, the method comprising:

[0005] Acquire a first single-cell dataset and a second single-cell dataset;

[0006] Constructing an objective loss function, wherein the objective function is used to balance the expression differences of different single cell types;

[0007] Based on the first single-cell dataset, pre-training the constructed single-cell annotation network model to obtain a pre-trained single-cell annotation network model;

[0008] Based on the second single-cell dataset and the target loss function, fine-tuning the pre-trained single-cell annotation network model to obtain a target single-cell annotation network model;

[0009] The target single-cell annotation network model is used to annotate the single cells to be annotated, thereby obtaining annotated single cells.

[0010] Compared with the prior art, the first aspect of the present application has the following beneficial effects:

[0011] This method constructs a target loss function, which is used to balance the expression differences of different single cell types; based on the first single cell data set, the constructed single cell annotation network model is pre-trained to obtain a pre-trained single cell annotation network model; based on the second single cell data set and the target loss function, the pre-trained single cell annotation network model is fine-tuned to obtain a target single cell annotation network model; the target single cell annotation network model is used to annotate the single cell to be annotated to obtain an annotated single cell. In this way, by pre-training the constructed single cell annotation network model, and then fine-tuning the pre-trained single cell annotation network model through the target function that can balance the expression differences of different single cell types, the target single cell annotation network model can effectively balance the expression differences of different cell types, improve the recognition ability of low expression cell types, thereby effectively solving the long-tail distribution problem of the data set and improving the accuracy and generalization ability of cell annotation.

[0012] In some embodiments, the objective loss function is constructed as follows:

[0013]

[0014] where z = z1, z2, ..., z C represents the data predicted by the target single-cell annotation model, represents the adjusted predicted probability, y i ∈{0,1} represents the true label of the i-th category sample, L SCDE (z) represents the target loss function, and C represents the total amount of predicted data.

[0015] In some embodiments, when constructing the target loss function, the method further includes calculating the adjusted predicted probability in the following manner:

[0016]

[0017] in, represents the adjusted predicted probability of the i-th category, z i represents the i-th category data predicted by the target single-cell annotation model, and A represents an adjustable parameter.

[0018] In some embodiments, when constructing the objective loss function, the method further includes calculating the adjustable parameter in the following manner:

[0019]

[0020] S ij =M ij ·C ij

[0021] Among them, S ij represents an adjustable balancing factor that adjusts the penalty on a class by adjusting the ratio of the number of instances between classes i and j, M ij represents the mitigation factor by reducing the penalty for tail category j according to the ratio of the number of instances between categories i and j, C ij It indicates that the penalty compensation factor for category j is increased by observing misclassified samples of category i.

[0022] In some embodiments, the single-cell annotation network model includes Bayesian downsampling, masking, embedded modules, encoders, decoders, and multi-layer perceptrons, and the pre-trained single-cell annotation network model is fine-tuned based on the second single-cell dataset and the target loss function to obtain a target single-cell annotation network model, including:

[0023] Inputting the data in the second single-cell dataset into the pre-trained single-cell annotation network model;

[0024] Performing Bayesian downsampling on the second single-cell data set to obtain downsampled data;

[0025] Performing a mask operation on the downsampled data to obtain masked data;

[0026] Inputting the masked data into the embedded module to obtain zero-value masked data and non-zero-value masked data;

[0027] Encoding the non-zero value mask data by the encoder to obtain encoded data;

[0028] Pooling the encoded data to obtain pooled data;

[0029] Inputting the pooled data and the zero-value mask data into the multilayer perceptron to obtain an output result of the pre-trained single-cell annotation network model;

[0030] According to the data in the second single-cell data set, the output result of the pre-trained single-cell annotation network model and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain a target single-cell annotation network model.

[0031] In some embodiments, fine-tuning the pre-trained single-cell annotation network model according to the data in the second single-cell dataset, the output result of the pre-trained single-cell annotation network model, and the target loss function to obtain a target single-cell annotation network model comprises:

[0032] According to the data in the second single-cell dataset, the output results of the pre-trained single-cell annotation network model and the target loss function, a gradient accumulation optimization strategy is used to fine-tune the pre-trained single-cell annotation network model to obtain a target single-cell annotation network model.

[0033] In some embodiments, acquiring the first single cell dataset and the second single cell dataset comprises:

[0034] Obtain the original single-cell sequencing data in the Zheng68k dataset;

[0035] Normalizing the original single-cell sequencing data to obtain normalized data;

[0036] According to the normalized data, a first single-cell data set and a second single-cell data set are determined.

[0037] In a second aspect, the present application embodiment further provides a single-cell annotation system for a generative gene large model under a long-tail data distribution, the system comprising:

[0038] A data acquisition unit, configured to acquire a first single-cell data set and a second single-cell data set;

[0039] A function construction unit, used to construct an objective loss function, wherein the objective function is used to balance the expression differences of different single cell types;

[0040] A model training unit, used for pre-training the constructed single-cell annotation network model based on the first single-cell dataset to obtain a pre-trained single-cell annotation network model;

[0041] A model fine-tuning unit, configured to fine-tune the pre-trained single-cell annotation network model based on the second single-cell dataset and the target loss function to obtain a target single-cell annotation network model;

[0042] The cell annotation unit is used to annotate the single cell to be annotated by using the target single cell annotation network model to obtain the annotated single cell.

[0043] In a third aspect, an embodiment of the present application also provides an electronic device, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the single-cell annotation method for a generative large genetic model under a long-tail data distribution as described above.

[0044] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the single-cell annotation method for a generative gene large model under a long-tail data distribution as described above.

[0045] It can be understood that the beneficial effects of the second to fourth aspects compared with the related art are the same as the beneficial effects of the first aspect compared with the related art. Please refer to the relevant description in the first aspect, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0047] Figure 1 It is a flow chart of an embodiment of a single-cell annotation method of a generative gene large model under a long-tail data distribution provided by the present application;

[0048] Figure 2 It is a schematic diagram of the structure of a single-cell annotation network model in a best embodiment of a single-cell annotation method for a generative gene large model under a long-tail data distribution provided by the present application;

[0049] Figure 3 It is a schematic diagram of the structure of an embodiment of a single-cell annotation system of a generative gene large model under long-tail data distribution provided by the present application;

[0050] Figure 4 It is a schematic diagram of the structure of an embodiment of the electronic device provided by the present application. DETAILED DESCRIPTION

[0051] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0052] In the description of this application, if there is a description of first, second, etc., it is only for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0053] In the description of the present application, it should be understood that the descriptions involving orientation, such as the orientation or positional relationship indicated as up, down, etc., are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0054] In the description of this application, it should be noted that, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meaning of the above terms in this application based on the specific content of the technical solution.

[0055] Existing related technologies include scGPT and scFoundation, among which scGPT is a single-cell multi-omics foundation model developed by researchers. This model captures the complex relationship between genes and cells by generative pre-training on more than 33 million cells, providing a strong foundation for a variety of downstream tasks. The core concept of scGPT is to apply the concept of pre-training models in natural language processing to single-cell data analysis, and learn the general representation of cells and genes through large-scale pre-training, thereby providing support for various downstream tasks. The main innovations of scGPT include large-scale pre-training, multimodal learning, flexible downstream applications and efficient computing architecture.

[0056] scFoundation is a large-scale pre-trained model for single-cell transcriptomics developed by researchers. The model has 100 million parameters, covers about 20,000 genes, and is pre-trained on more than 50 million human single-cell transcriptome profiles. scFoundation demonstrates state-of-the-art performance in a variety of single-cell analysis tasks, such as gene expression enhancement, drug response classification, etc. The development of scFoundation reveals the effectiveness and value of large-scale pre-trained models for transcriptomics data and proves its great application value in promoting biological and medical research.

[0057] However, the above-mentioned single-cell annotation technologies cannot effectively solve the long-tail distribution problem of the data set, resulting in limited accuracy in cell annotation tasks. This makes it impossible for the model to fully capture the information of low-expression cell types during training, thereby reducing the accuracy and generalization ability of cell annotation.

[0058] In order to solve the above-mentioned problems of being unable to effectively solve the long-tail distribution problem of the data set and reducing the accuracy and generalization ability of cell annotation, the present application proposes a single-cell annotation method for a generative gene large model under long-tail data distribution.

[0059] Reference Figure 1The present application embodiment provides a single-cell annotation method for a generative gene model under a long-tail data distribution, the method comprising the following steps:

[0060] Step S100, acquiring a first single-cell data set and a second single-cell data set;

[0061] Step S200, constructing an objective loss function, where the objective function is used to balance the expression differences of different single cell types;

[0062] Step S300: pre-training the constructed single-cell annotation network model based on the first single-cell data set to obtain a pre-trained single-cell annotation network model;

[0063] Step S400: fine-tuning the pre-trained single-cell annotation network model based on the second single-cell data set and the target loss function to obtain a target single-cell annotation network model;

[0064] Step S500: annotate the single cells to be annotated using the target single cell annotation network model to obtain annotated single cells.

[0065] In this embodiment, by constructing a target loss function, the target function is used to balance the expression differences of different single cell types; based on the first single cell data set, the constructed single cell annotation network model is pre-trained to obtain a pre-trained single cell annotation network model; based on the second single cell data set and the target loss function, the pre-trained single cell annotation network model is fine-tuned to obtain a target single cell annotation network model; the target single cell annotation network model is used to annotate the single cell to be annotated to obtain an annotated single cell. In this way, by pre-training the constructed single cell annotation network model, and then fine-tuning the pre-trained single cell annotation network model through the target function that can balance the expression differences of different single cell types, the target single cell annotation network model can effectively balance the expression differences of different cell types, improve the recognition ability of low expression cell types, thereby effectively solving the long-tail distribution problem of the data set and improving the accuracy and generalization ability of cell annotation.

[0066] The expression level of the above-mentioned single cell type may refer to the expression level of a certain gene in a single cell of a certain type, which is usually expressed by the amount of RNA or protein.

[0067] The constructed single-cell annotation network model may be a single-cell annotation network model constructed using a Transformer model.

[0068] The first single-cell dataset and the second single-cell dataset may be the same single-cell dataset or different single-cell datasets, which is not specifically limited in this embodiment.

[0069] In some embodiments, the objective loss function is constructed as follows:

[0070]

[0071] where z = z1, z2, ..., z C represents the data predicted by the target single-cell annotation model, represents the adjusted predicted probability, y i ∈{0,1} represents the true label of the i-th category sample, L SCDE (z) represents the target loss function, and C represents the total amount of predicted data.

[0072] In this embodiment, by constructing a good target loss function, the expression differences of different cell types can be effectively balanced, and the recognition ability of low-expression cell types can be significantly improved.

[0073] In some embodiments, when constructing the target loss function, the method further includes calculating the adjusted predicted probability by:

[0074]

[0075] in, represents the adjusted predicted probability of the i-th category, Z i represents the i-th category data predicted by the target single-cell annotation model, and A represents an adjustable parameter.

[0076] In some embodiments, when constructing the target loss function, the method further includes calculating the adjustable parameters in the following manner:

[0077]

[0078] S ij =M ij ·C ij

[0079] Among them, S ij represents an adjustable balancing factor that adjusts the penalty on a class by adjusting the ratio of the number of instances between classes i and j, M ij represents the mitigation factor by reducing the penalty for tail category j according to the ratio of the number of instances between categories i and j, C ij It indicates that the penalty compensation factor for category j is increased by observing misclassified samples of category i.

[0080] In some embodiments, the single-cell annotation network model includes Bayesian downsampling, masking, embedded modules, encoders, decoders, and multi-layer perceptrons. Based on the second single-cell dataset and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain a target single-cell annotation network model, including:

[0081] Inputting data in the second single-cell dataset into a pre-trained single-cell annotation network model;

[0082] The second single-cell data set is subjected to Bayesian downsampling to obtain downsampled data;

[0083] Perform a mask operation on the downsampled data to obtain masked data;

[0084] Inputting the masked data into the embedded module to obtain zero-value masked data and non-zero-value masked data;

[0085] Encode the non-zero value mask data through an encoder to obtain encoded data;

[0086] Pooling the encoded data to obtain pooled data;

[0087] The pooled data and zero-value mask data are input into a multi-layer perceptron to obtain the output of the pre-trained single-cell annotation network model;

[0088] According to the data in the second single-cell dataset, the output results of the pre-trained single-cell annotation network model and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain the target single-cell annotation network model.

[0089] In this embodiment, the pre-trained single-cell annotation network model is fine-tuned according to the data in the second single-cell dataset, the output results of the pre-trained single-cell annotation network model, and the target loss function, so that the obtained target single-cell annotation network model can effectively balance the expression differences of different cell types and improve the recognition ability of low-expression cell types, thereby effectively solving the long-tail distribution problem of the dataset and improving the accuracy and generalization ability of cell annotation.

[0090] In some embodiments, the pre-trained single-cell annotation network model is fine-tuned according to the data in the second single-cell dataset, the output result of the pre-trained single-cell annotation network model, and the target loss function to obtain the target single-cell annotation network model, including:

[0091] According to the data in the second single-cell dataset, the output results of the pre-trained single-cell annotation network model and the target loss function, the pre-trained single-cell annotation network model is fine-tuned using a gradient accumulation optimization strategy to obtain the target single-cell annotation network model.

[0092] In this embodiment, a gradient accumulation optimization strategy is used to fine-tune the pre-trained single-cell annotation network model. The gradient accumulation optimization strategy performs model forward propagation and back propagation in small batches. Multiple small batch gradients are calculated and accumulated during the iteration process. When enough gradients are accumulated, the optimization step of the model is executed to update the parameters, which can make the network parameters in the pre-trained single-cell annotation network model more optimized, so that the target single-cell annotation network model can better identify cell types with low expression levels.

[0093] The basic idea of ​​the above-mentioned gradient accumulation is to transform the gradient calculation of a one-time batch parameter update into a small step-by-step method. Specifically, this method performs model forward propagation and back propagation in small batches. During the iteration process, multiple small batch gradients are calculated and accumulated. When enough gradients are accumulated, the optimization step of the model is executed to update the parameters.

[0094] In some embodiments, acquiring a first single cell dataset and a second single cell dataset comprises:

[0095] Obtain the original single-cell sequencing data in the Zheng68k dataset;

[0096] Normalizing the original single-cell sequencing data to obtain normalized data;

[0097] According to the normalized data, a first single-cell data set and a second single-cell data set are determined.

[0098] In this embodiment, the original single-cell sequencing data is normalized so that the original single-cell sequencing data in the Zheng68k dataset can meet the input requirements of the single-cell annotation network model.

[0099] The Zheng68k data set mentioned above is a data set existing in the prior art and will not be described in detail in this embodiment.

[0100] To facilitate understanding by those skilled in the art, a set of best embodiments is provided below:

[0101] Because the prior art has the following disadvantages:

[0102] 1. Long-tail problem of data distribution: In single-cell annotation tasks, existing technologies often face the long-tail problem of data distribution, that is, the high expression levels of a few cell types mask other types, resulting in a decrease in the accuracy of cell annotation.

[0103] 2. Lack of targeted loss functions: Existing methods lack loss functions specifically designed for RNA single-cell long-tail distribution datasets and cannot effectively balance the differences in expression levels of different cell types.

[0104] 3. Limited model generalization ability: Existing single-cell annotation models are usually trained for specific datasets and lack good generalization ability, especially when faced with new, unseen cell types.

[0105] To solve the problem that the existing single-cell annotation technology cannot effectively solve the long-tail distribution problem of the data set, resulting in limited accuracy in the cell annotation task, making the model unable to fully capture the information of low-expression cell types during training, thereby reducing the accuracy and generalization ability of cell annotation. This embodiment specifically designs a method to solve the long-tail distribution problem of single-cell data, and the technical solution of this embodiment is achieved by proposing a new loss function SCDE Loss loss function.

[0106] The technical solution of this embodiment is based on a generative pre-trained Transformer model, hereinafter referred to as the scHigen model, that is, a model built based on the Transformer model (i.e., a single-cell annotation network model), to annotate single cells in the Zheng68k dataset. The Zheng68k dataset is a classic peripheral blood mononuclear cell (PBMC) dataset generated by the Chromium platform of 10xGenomics. It contains approximately 68,450 cells, which are divided into 11 different cell subtypes. This dataset is characterized by the inclusion of rare cell types and the uneven distribution of cell types, which makes the distinction between cell types challenging. Figure 2 , the single-cell annotation network model includes Bayesian downsampling, mask, embedded module, encoder, decoder and multi-layer perceptron, specifically:

[0107] 1. Bayesian downsampling: First, the data goes through the Bayesian downsampling process to get the downsampled data representation.

[0108] 2. Mask operation: Next, a mask operation is performed on the downsampled data to generate masked data.

[0109] 3. Embedded module: The masked data enters the embedded module, which further processes the data in preparation for subsequent encoding and decoding.

[0110] 4. Zero value push coding: In the embedded module, the data is divided into two parts: zero value and non-zero value. The zero value part is processed through the zero value masking process. Zero value masking means that the missing data points are directly represented by 0.

[0111] 5. Encoder: Non-zero value data is encoded through the encoder and converted into a more compact representation.

[0112] 6. Pooling: The encoded data undergoes a pooling operation to further reduce the data dimension and extract important features.

[0113] 7. Decoder: The pooled data enters the decoder, which restores the data to an approximate table of the original data.

[0114] 8.MLP(Multi-layer Perceptron): The decoded data is further processed through MLP to improve the expressiveness of the model.

[0115] 9. Reconstruction loss: Finally, the output of the single-cell annotation network model is compared with the original data to calculate the reconstruction loss. The SCDE (Spectral Contrastive Divergence Estimation) loss function is used to optimize the model.

[0116] The following are the detailed steps of the implementation principle of the technical solution of this embodiment:

[0117] 1. Pre-training stage: First, the original single-cell sequencing data in the Zheng68k dataset is normalized to meet the input requirements of the scHigen model (i.e., the single-cell annotation network model). Then, the normalized dataset is used to pre-train the scHigen model.

[0118] 2. Fine-tuning stage: Fine-tune the pre-trained scHigen model on the Zheng68k dataset to adapt to the characteristics of the specific dataset. You can select a dataset of rare cell types on the Zheng68k dataset to fine-tune the pre-trained scHigen model to obtain the target single-cell annotation network model.

[0119] 3. Introducing the SCDE loss function: In view of the long-tail distribution characteristics of the data set, the SCDE loss function is introduced in the fine-tuning stage of step 2. It is specially designed to balance the expression differences of different cell types and improve the recognition ability of low-expression cell types. The expression of a cell type refers to the expression level of a gene in a cell, usually expressed in terms of the amount of RNA or protein. The SCDE principle is as follows:

[0120] The SCDE loss function aims to reduce the impact of the head category on the gradient of negative samples in the tail category. Specifically, the SCDE loss function adjusts the probability distribution predicted by the model to reduce the impact of the head category with a large number of samples on the tail category with a small number of samples, while increasing the penalty for misclassification. This helps the model better learn the characteristics of the tail category, thereby improving the recognition ability of these categories. Its definition is as follows:

[0121]

[0122] where z = z1, z2, ..., z C represents the data predicted by the model, represents the adjusted predicted probability, y i ∈{0,1} represents the true label of the i-th category sample, indicating whether the sample belongs to category i. The adjusted prediction probability The calculation is as follows: S ij is an adjustable balancing factor that adjusts the penalty for a class based on the ratio of the number of instances between classes i and j. The balancing factor S ij The calculation of S is as follows: ij =M ij ·C ij , where M ij is a mitigation factor that reduces the penalty for tail category j according to the ratio of the number of instances between categories i and j; C ij is a compensation factor that increases the penalty for class j when misclassified samples of class i are observed. These factors work together to enable the SCDE loss function to effectively alleviate the excessive penalty for tail classes and compensate for the risk of misclassification caused by reducing the penalty.

[0123] The technical solution of this embodiment is different from that of the prior art in that:

[0124] 1. Loss function for long-tail distribution: The SCDE loss function is the core innovation of this embodiment. It is specially designed for long-tail distribution data sets and effectively balances the expression differences of different cell types. Compared with the cross-entropy loss function commonly used in the prior art, SCDE can significantly improve the recognition ability of low-expression cell types.

[0125] 2. Model training strategy combining pre-training and fine-tuning: This embodiment adopts a strategy of pre-training plus fine-tuning, first pre-training on a large-scale dataset to learn the representation of genes and cells, and then fine-tuning on a specific dataset to adapt to the characteristics of the specific dataset. Compared with the method of directly training the model on a specific dataset in the prior art, this strategy can better utilize the information in the large-scale dataset and improve the generalization ability of the model.

[0126] 3. Solution to the long-tail distribution problem of data sets: The technical solution of this embodiment can effectively solve the long-tail distribution problem of data sets and improve the accuracy of cell annotation, which is a key problem that the existing technology has not effectively solved. By introducing the SCDE loss function and optimized model training strategy, this embodiment significantly improves the ability to identify low-expression cell types, thereby achieving higher accuracy and generalization in cell annotation tasks.

[0127] For better explanation, this embodiment is given as follows:

[0128] In order to verify the effectiveness of this embodiment, this embodiment selected the Zheng68k data set for experiment. First, this embodiment pre-trained the scHigen model and then fine-tuned it on the Zheng68k data set. In the fine-tuning stage, this embodiment introduced the SCDE loss function and adopted an optimization strategy based on gradient accumulation. The basic idea of ​​gradient accumulation is to change the gradient calculation of a one-time batch parameter update to a small step-by-step manner. Specifically, this method performs model forward propagation and back propagation in a small batch manner, calculates multiple small batch gradients and accumulates them during the iteration process, and when enough gradients are accumulated, the optimization step of the model is executed to update the parameters. The experimental results show that compared with the prior art, the accuracy and generalization ability of this embodiment in the cell annotation task are significantly improved. Specifically, the recognition accuracy of the model using the SCDE loss function on low-expression cell types has increased by about 15%, and the accuracy of the overall cell annotation has increased by about 10%. These results demonstrate the effectiveness and superiority of this embodiment in processing single-cell multi-omics data.

[0129] Reference Figure 3 The embodiment of the present application also provides a single-cell annotation system for a generative gene model under a long-tail data distribution, the system comprising a data acquisition unit 100, a function construction unit 200, a model training unit 300, a model fine-tuning unit 400 and a cell annotation unit 500, wherein:

[0130] A data acquisition unit 100 is used to acquire a first single-cell data set and a second single-cell data set;

[0131] A function construction unit 200 is used to construct a target loss function, where the target function is used to balance the expression differences of different single cell types;

[0132] A model training unit 300 is used to pre-train the constructed single-cell annotation network model based on the first single-cell data set to obtain a pre-trained single-cell annotation network model;

[0133] A model fine-tuning unit 400 is used to fine-tune the pre-trained single-cell annotation network model based on the second single-cell data set and the target loss function to obtain a target single-cell annotation network model;

[0134] The cell annotation unit 500 is used to annotate the single cell to be annotated using the target single cell annotation network model to obtain the annotated single cell.

[0135] It should be noted that since the single-cell annotation system of a generative gene large model under a long-tail data distribution in this embodiment and the single-cell annotation method of a generative gene large model under a long-tail data distribution mentioned above are based on the same inventive concept, the corresponding contents in the method embodiment are also applicable to the present system embodiment and will not be described in detail here.

[0136] Reference Figure 4 The embodiment of the present application further provides an electronic device, the electronic device comprising:

[0137] at least one memory;

[0138] at least one processor;

[0139] at least one program;

[0140] The program is stored in the memory, and the processor executes at least one program to implement the single-cell annotation method of the generative gene large model under the long-tail data distribution implemented in the present disclosure.

[0141] The electronic device may be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0142] The electronic device according to the embodiment of the present application is described in detail below.

[0143] The processor 1600 may be implemented by a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present disclosure;

[0144] The memory 1700 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1700 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 1700, and the processor 1600 calls and executes a single-cell annotation method for a generative gene large model under a long-tail data distribution in an embodiment of the present disclosure.

[0145] Input / output interface 1800, used to implement information input and output;

[0146] The communication interface 1900 is used to realize the communication interaction between the device and other devices. The communication can be realized through a wired manner (such as USB, network cable, etc.) or a wireless manner (such as mobile network, WIFI, Bluetooth, etc.);

[0147] Bus 2000 , which transmits information between various components of the device (e.g., processor 1600 , memory 1700 , input / output interface 1800 , and communication interface 1900 );

[0148] The processor 1600 , the memory 1700 , the input / output interface 1800 , and the communication interface 1900 are connected to each other in communication within the device via the bus 2000 .

[0149] The embodiments of the present disclosure also provide a storage medium, which is a computer-readable storage medium, and the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned single-cell annotation method of a generative gene large model under a long-tail data distribution.

[0150] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0151] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0152] Those skilled in the art will appreciate that the technical solutions shown in the figures do not limit the embodiments of the present disclosure and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0153] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0154] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0155] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0156] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0157] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0158] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0159] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0160] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable an electronic device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store programs. The above is a detailed description of the embodiments of the present application in conjunction with the accompanying drawings, but the present application is not limited to the above embodiments. Within the scope of knowledge possessed by ordinary technicians in the relevant technical field, various changes can be made without departing from the purpose of the present application.

Claims

1. A single-cell annotation method for a generative gene model under long-tail data distribution, characterized in that: The method comprises: Acquire a first single-cell dataset and a second single-cell dataset; Constructing an objective loss function, wherein the objective function is used to balance the expression differences of different single cell types; Based on the first single-cell dataset, pre-training the constructed single-cell annotation network model to obtain a pre-trained single-cell annotation network model; Based on the second single-cell dataset and the target loss function, fine-tuning the pre-trained single-cell annotation network model to obtain a target single-cell annotation network model; The target single-cell annotation network model is used to annotate the single cells to be annotated, thereby obtaining annotated single cells.

2. The single-cell annotation method of a generative gene large model under long-tail data distribution according to claim 1, characterized in that: The objective loss function is constructed as follows: where z = z1, z2, ..., z C represents the data predicted by the target single-cell annotation model, represents the adjusted predicted probability, y i ∈{0,1} represents the true label of the i-th category sample, L SCDE (z) represents the target loss function, and C represents the total amount of predicted data.

3. The single-cell annotation method of a generative gene large model under long-tail data distribution according to claim 2, characterized in that: When constructing the target loss function, the method further includes calculating the adjusted prediction probability in the following manner: in, represents the adjusted predicted probability of the i-th category, z i represents the i-th category data predicted by the target single-cell annotation model, and A represents an adjustable parameter.

4. The single-cell annotation method of a generative gene large model under long-tail data distribution according to claim 3 is characterized in that: When constructing the target loss function, the method further includes calculating the adjustable parameters in the following manner: S ij =M ij ·C ij Among them, S ij represents an adjustable balancing factor that adjusts the penalty on a class by adjusting the ratio of the number of instances between classes i and j, M ij represents the mitigation factor by reducing the penalty for tail category j according to the ratio of the number of instances between categories i and j, C ij It indicates that the penalty compensation factor for category j is increased by observing misclassified samples of category i.

5. The single-cell annotation method of a generative gene large model under long-tail data distribution according to claim 1, characterized in that: The single-cell annotation network model includes Bayesian downsampling, masking, embedded modules, encoders, decoders, and multi-layer perceptrons. Based on the second single-cell dataset and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain a target single-cell annotation network model, including: Inputting the data in the second single-cell dataset into the pre-trained single-cell annotation network model; Performing Bayesian downsampling on the second single-cell data set to obtain downsampled data; Performing a mask operation on the downsampled data to obtain masked data; Inputting the masked data into the embedded module to obtain zero-value masked data and non-zero-value masked data; Encoding the non-zero value mask data by the encoder to obtain encoded data; Pooling the encoded data to obtain pooled data; Inputting the pooled data and the zero-value mask data into the multilayer perceptron to obtain an output result of the pre-trained single-cell annotation network model; According to the data in the second single-cell data set, the output result of the pre-trained single-cell annotation network model and the target loss function, the pre-trained single-cell annotation network model is fine-tuned to obtain a target single-cell annotation network model.

6. The single-cell annotation method of a generative gene large model under long-tail data distribution according to claim 5, characterized in that: The method further comprises fine-tuning the pre-trained single-cell annotation network model according to the data in the second single-cell dataset, the output result of the pre-trained single-cell annotation network model, and the target loss function to obtain a target single-cell annotation network model, including: According to the data in the second single-cell dataset, the output results of the pre-trained single-cell annotation network model and the target loss function, a gradient accumulation optimization strategy is used to fine-tune the pre-trained single-cell annotation network model to obtain a target single-cell annotation network model.

7. The single-cell annotation method of a generative gene large model under long-tail data distribution according to claim 1, characterized in that: The acquiring of the first single-cell dataset and the second single-cell dataset comprises: Obtain the original single-cell sequencing data in the Zheng68k dataset; Normalizing the original single-cell sequencing data to obtain normalized data; According to the normalized data, a first single-cell data set and a second single-cell data set are determined.

8. A single-cell annotation system for generative gene models under long-tail data distribution, characterized in that: The system comprises: A data acquisition unit, configured to acquire a first single-cell data set and a second single-cell data set; A function construction unit, used to construct an objective loss function, wherein the objective function is used to balance the expression differences of different single cell types; A model training unit, used for pre-training the constructed single-cell annotation network model based on the first single-cell dataset to obtain a pre-trained single-cell annotation network model; A model fine-tuning unit, configured to fine-tune the pre-trained single-cell annotation network model based on the second single-cell dataset and the target loss function to obtain a target single-cell annotation network model; The cell annotation unit is used to annotate the single cell to be annotated by using the target single cell annotation network model to obtain the annotated single cell.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the single-cell annotation method of a generative gene large model under a long-tail data distribution as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a single-cell annotation method for a generative gene model under a long-tail data distribution as described in any one of claims 1 to 7.

Images