An omics large model-oriented automated parameter efficient fine-tuning method and system

Abstract

The application discloses an automatic parameter efficient fine-tuning method and system for omics large models, which utilizes automatic machine learning algorithms to automatically design parameter efficient fine-tuning strategies according to new omics data analysis tasks, and constructs adaptive omics large models, the method comprising: collecting omics data and preprocessing, taking an omics large model as a backbone network, combining a predefined parameter efficient fine-tuning module to construct a super network; sampling a sub-architecture from the super network for training in each round, and repeating the sampling and training until the architecture converges; in the search stage, a search strategy based on an evolutionary algorithm is adopted to randomly sample a sub-architecture from the trained super network as an initial population, a new population is generated through selection, crossover and mutation operations, and the iteration is repeated until the search process converges, and the parameter efficient fine-tuning configuration of the best performance model is output. Through a small amount of manual operation and professional knowledge, the application can search for a suitable parameter efficient fine-tuning method for an omics large model for a specific omics analysis task.

Description

Technical fields:

[0001] This invention relates to an automated parameter fine-tuning method and system for large-scale omics models, belonging to the field of computer artificial intelligence. Background technology:

[0002] With the rapid development of large language model technology, there is increasing attention being paid to how to better apply large language models to downstream tasks. To enable large language models to better adapt to downstream tasks while reducing the number of training parameters, efficient parameter fine-tuning methods have been proposed. These methods aim to improve the performance of pre-trained models on new tasks by minimizing the number of fine-tuned parameters and computational complexity, thereby reducing the training burden on large pre-trained models. Efficient parameter fine-tuning not only improves model performance but also significantly shortens training time and computational costs, making it a common method for fine-tuning large models on downstream tasks.

[0003] Due to the high dimensionality and sparsity of single-cell data, as well as systematic biases between different experimental platforms, constructing efficient and highly generalizable large-scale models still faces many challenges. Currently, deep learning-based large-scale models have demonstrated powerful feature learning capabilities in single-cell omics, but these models typically rely on a large number of parameters, and the optimal configuration for different tasks remains unclear. Therefore, how to efficiently fine-tune model parameters under different task scenarios while improving computational efficiency has become one of the key issues in current research on large-scale single-cell models.

[0004] For example, the design of many neural architecture search algorithms still requires a lot of manual work and expertise in related fields, which increases human and time costs. In addition, different tasks require different efficient parameter fine-tuning strategies, and choosing the appropriate efficient parameter fine-tuning strategy also requires human and time costs. Summary of the Invention:

[0005] To address the aforementioned problems and challenges in existing technologies, this invention proposes an automated parameter fine-tuning method and system for large-scale omics models, enabling the large-scale omics model and the parameter fine-tuning module to adapt to each other and optimize together, thereby better capturing the complex characteristics of omics data.

[0006] This invention proposes a neural architecture search method for automated parameter fine-tuning of large-scale omics models. The method defines a supernet that includes some common modules for efficient parameter fine-tuning of large language models. By training the supernet, an evolutionary algorithm is used to search for neural architectures within the supernet. After multiple iterations, the optimal parameter fine-tuning configuration is obtained. Thus, while ensuring the model's lightweight nature, the optimal configuration is automatically searched for, improving fine-tuning efficiency, achieving better task generalization ability and data adaptability, and reducing the human cost and technical threshold of using neural network architecture search algorithms.

[0007] An automated and efficient parameter fine-tuning method for large-scale omics models includes the following steps:

[0008] S1: Collect data and preprocess the data;

[0009] S2: Collect existing large-scale omics models containing Transform as a basic model library;

[0010] S3: Define and construct a search space based on modules suitable for efficient fine-tuning of parameters in large omics models, and construct a supernet based on the search space for searching and optimizing the neural architecture of large omics models;

[0011] S4: Sample N parameters from the supernet to efficiently fine-tune the architecture as the initial population, and train the fine-tuned architecture until convergence;

[0012] S5: Sort the architectures in the population according to the model performance and select the top k architectures;

[0013] S6: Select, mutate, and cross over these k architectures to obtain a new population;

[0014] S6-1: Code the architecture;

[0015] S6-2: Randomly select two architectures and cross them to obtain p1;

[0016] S6-3: Randomly select the architecture in the population and mutate it to obtain p2;

[0017] S7: Add the structures p1 and p2 obtained in S6 to the population. If the total population size exceeds the predefined number of population structures P, then...

[0018] Randomly select P architectures from the existing population to form a new population;

[0019] S8: Repeat steps S5 to S7 until the automated machine learning algorithm converges;

[0020] S9: Select the best-performing fine-tuned model from the population's model architecture as the final output.

[0021] Prior to this, the optimization objective for searching neural architectures for efficient parameter fine-tuning of large-scale omics models can be formalized as Equation (1):

[0022]

[0023] Where, m in the formula * S is the efficient fine-tuning configuration of the parameters to be searched; S is the set of all efficient fine-tuning configurations of parameters in the entire search space; D represents the data used to train the model. It is an evaluation function used to assess the performance of the model architecture.

[0024] Preferred, the data used in S1 is either a real-collected dataset or an existing dataset.

[0025] Prior to this, the base model library in S2 can be an open-source single-cell omics large model or a self-trained single-cell omics large model.

[0026] Prioritizes using mathematical formulas in S3 to describe the searchable parameters within the search space, specifically:

[0027] The Adapter module has the following formula:

[0028]

[0029] Where h is the normalized output of the MLP in the Transformer block of the omics big data model, and W down W represents the weights of the downsampling layer. up The weights of the upsampling layer, For non-linear activation layers, ReLU or GELU activation functions are typically used;

[0030] The LoRA module has the following formula:

[0031]

[0032] Where x is the input to the Transformer block of the omics big data model, and W q The weights of the query module in the Transformer block of the large-scale omics model. The weights of the downsampling layer in the query module. W represents the weights of the upsampling layer in the query module. k These are the weights of the key module in the Transformer block of the model. The weights of the downsampling layer in the key module. is the parameter of the upsampling layer of the key module, and s is the scaling factor used to control the contribution of the LoRA module to the original weight matrix.

[0033] The formula for the Prefix module is:

[0034]

[0035] Where x is the input to the Transformer block of the large-scale omics model. Let represent the weights of the key module in the i-th attention head of the Transformer block in the large-scale omics model. W represents the weights of the value module of the i-th attention head in the Transformer block of the omics large model. q The weights of the query module in the Transformer block of the omics big data model. This is the prefix vector used by the i-th attention head in the Transformer block of the omics big data model for the key module. This is the prefix vector used by the i-th attention head in the Transformer block of the omics big data model for the value module.

[0036] Commonly used large model fine-tuning modules can be used in the method proposed in this invention, and appropriate operations can be selected and added to the search space according to the characteristics of different tasks.

[0037] Preferably, the supernet in S3 adopts a Transformer-based Encoder-Decoder structure, and the initial layer of the network receives input data for training omics tasks; the supernet includes a 12-layer structure, each layer includes a Transformer block, and each Transformer block contains a self-attention module and a feedforward network module.

[0038] The self-attention module employs a multi-head attention mechanism, where the outputs of each attention head are concatenated after a linear transformation. The feedforward network module includes two linear transformation layers and an intermediate nonlinear activation function, which is either ReLU or GELU. After the self-attention module and the feedforward network module of each Transformer block, based on the search space described in S3, parameter-efficient fine-tuning modules are inserted according to different insertion layer positions, insertion layer numbers, and module embedding dimensions to construct a supernet structure that covers all combinations of parameter-efficient fine-tuning modules.

[0039] Preferred, the architecture encoding in S6-1 uses a list of length 36, {adapter, lora, prefix}. Here, adapter, lora, and prefix are all lists of length 12. For example, adapter = {d1, d2, d3, ..., d...} 12} indicates that embedding dimensions d1, d2, d3, ..., d are used in layers 1 to 12 of the model, respectively. 12 Similarly, for the Adapter module, lora = {d′1, d′2, d′3, ..., d′} 12} indicates that embedding dimensions d′1, d′2, d′3, ..., d′ are used in layers 1 to 12 of the model, respectively. 12 The LoRA module, This indicates that embedding dimensions are used in layers 1 through 12 of the model. The Prefix module.i =0, d′ i =0 or These indicate that the i-th layer does not require an Adapter, LoRA, or Prefix module, respectively.

[0040] Prior to this, the method for randomly selecting two architectures for cross-referencing in S6-2 is to randomly cross-referencing their architecture codes to obtain a new architecture code. For example, if architectures {adapter_1,lora_1,prefix_1} and {adapter_2,lora_2,prefix_2} are selected, then in the new cross-referencing architecture {adapter,lora,prefix}, the adapter is randomly selected from adapter_1 and adapter_2, similarly, lora is randomly selected from lora_1 and lora_2, and prefix is ​​randomly selected from prefix_1 and prefix_2.

[0041] Prior to this, the method for randomly selecting architectures in the population for mutation in S6-3 is as follows: for the selected architecture {adapter, lora, prefix}, its adapter, lora, and prefix parts will be randomly modified by the insertion depth to obtain a new architecture encoding. For example, if the insertion depth of adapte is randomly changed from l to l1, then the adapter list will change from the previous {d1, d2, ..., d...} l ,0…,0} changed to Similarly, if the insertion level of the lora is randomly changed from l′ to l2, then the lora list will change from the previous {d′1,d′2,…,d′} l′ ,0…,0} changed to The number of insertion layers of the prefix is ​​determined by l p If the prefix is ​​randomly changed to l3, the prefix list will change from the previous one. Change to

[0042] This invention also relates to a neural architecture search system for efficient fine-tuning of parameters in large omics models, comprising:

[0043] The data acquisition and preprocessing module is used to acquire data and preprocess it.

[0044] The supernet module is used to construct a neural network containing all possible parameter-efficient fine-tuning modules based on existing omics big data models and feasible parameter-efficient fine-tuning methods.

[0045] The neural architecture training module is used to train neural architecture models.

[0046] Architecture encoding module, used to encode the neural networks contained in the supernet;

[0047] The neural architecture search module is used to efficiently fine-tune the configuration by obtaining the optimal parameters from the supernet according to the search strategy.

[0048] The model performance evaluation module is used to evaluate the performance of the trained neural architecture model.

[0049] The iterative module repeats steps S5 to S7 until the preset number of iterations is reached;

[0050] The results output module is used to select the model with the best performance as the final output result.

[0051] The present invention also relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the method of the present invention.

[0052] This invention utilizes automated machine learning algorithms to design efficient parameter fine-tuning methods for novel omics data analysis tasks, constructing large-scale omics models. Specifically, it includes: collecting and preprocessing data; constructing a supernet based on the large-scale omics model and predefined efficient parameter fine-tuning modules; in each round, sampling a sub-network from the supernet for training, repeating this process until the automated parameter fine-tuning method converges; using an evolutionary algorithm search strategy to construct an automated parameter fine-tuning method, randomly sampling from the trained supernet as the initial population, obtaining a new population through selection, crossover, and mutation, repeating this process until the algorithm converges, and outputting the optimal parameter fine-tuning configuration for the best-performing model. This invention enables the search for suitable efficient parameter fine-tuning methods for large-scale omics models for specific omics analysis tasks with minimal manual operation and professional knowledge.

[0053] The innovation of this invention is:

[0054] (1) The method of automatically configuring parameters and efficiently fine-tuning the module is combined with the omics big model, so that the omics big model can automatically search for parameters and efficiently fine-tuning modules according to different omics tasks, breaking through the limitation of manual adjustment of traditional omics models under different tasks.

[0055] (2) Constructing a unique hypernet architecture for large-scale omics models. A hypernet is defined that includes modules for efficiently fine-tuning large language models with common parameters. This hypernet can comprehensively consider the combination and interaction of various fine-tuning modules, providing a systematic framework for training large-scale omics models.

[0056] (3) Achieve automated search and collaborative optimization of large-scale omics models. An automated neural architecture search method is adopted to simultaneously search for the optimal parameter fine-tuning configuration during the training process of large-scale omics models. This enables the large-scale omics model and the parameter fine-tuning module to jointly develop towards the optimal performance direction during training, thereby improving the performance and efficiency of model training.

[0057] The present invention has the following advantages:

[0058] (1) Automatic neural architecture search can effectively reduce the time and cost of manual selection of fine-tuning methods, while flexibly adapting to various parameters and efficient fine-tuning methods.

[0059] (2) By constructing a supernet based on a large single-cell omics model, it is possible to handle a variety of single-cell omics tasks;

[0060] (3) By searching simultaneously through the omics big model and the parameter efficient fine-tuning module, the omics big model and the parameter efficient fine-tuning module can adapt to each other and optimize together, thereby achieving better performance in omics tasks.

[0061] (4) The method proposed in this invention can achieve good results on data of different fields and different scales, thereby improving the generalization and robustness of the algorithm. Attached image description:

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is the overall flowchart of the method of the present invention.

[0064] Figure 2 This is a structural diagram of the system of the present invention. Detailed implementation method:

[0065] Exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0066] Example 1

[0067] The gene classification method applying the neural architecture search method for efficient parameter fine-tuning of large omics models provided by this invention includes the following steps:

[0068] S1: Select Dataset

[0069] This embodiment uses four publicly available datasets for gene classification, corresponding to four binary classification tasks:

[0070] Gene dose sensitivity: distinguishing between dose-sensitive genes and dose-insensitive genes.

[0071] Long-range and short-range transcription factors: Transcription factors that differentiate between long-range and short-range regulation. Long-range transcription factors typically regulate gene expression by interacting with enhancer regions far from the promoter, while short-range transcription factors bind directly to the promoter region.

[0072] Gene methylation: Single-cell RNA sequencing of multiple embryonic stem cells is used to distinguish between bivalent genes and unmethylated genes.

[0073] Gene methylation categories: Single-cell RNA sequencing of multiple embryonic stem cells is used to distinguish between bivalent genes and genes with only H3K4 trimethylation.

[0074] S2: Collect existing large-scale omics models containing Transform as a basic model library;

[0075] This embodiment selects Geneformer as the base model library.

[0076] S3: Construct the search space based on the efficient fine-tuning method of feasible parameters of the omics big model, and further construct the supernet;

[0077] The constructed supernet is based on a pre-trained large single-cell omics model, such as Geneformer, and includes two fine-tuning modules, Adapter and LoRA, for fine-tuning the model on downstream tasks. The embedding dimension and the number of inserted layers of these two modules can be used for neural architecture search.

[0078] The Adapter module has the following formula:

[0079]

[0080] Where h is the normalized output of the MLP in the Transformer block of the model, and W down W represents the parameters of the downsampling layer. up These are the parameters of the upsampling layer. For non-linear activation layers, ReLU or GELU activation functions are typically used;

[0081] The LoRA module has the following formula:

[0082]

[0083] Where x is the input in the Transformer block of the model, W q These are the parameters of the query module in the Transformer block of the model. These are the parameters of the downsampling layer in the query module. W represents the parameters of the upsampling layer in the query module. k These are the parameters of the key module in the Transformer block of the model. These are the parameters of the downsampling layer in the key module. Here are the parameters of the upsampling layer of the key module, and s is the weight.

[0084] S4: Sample N parameters from the supernet to efficiently fine-tune the architecture as the initial population, and train the fine-tuned architecture until convergence;

[0085] S5: Sort the architectures in the population according to the model performance and select the top k architectures;

[0086] S6: Select, mutate, and cross over these k architectures to obtain a new population;

[0087] S6-1: Code the architecture;

[0088] Specifically, the architecture encoding uses a list of length 24, {adapter, lora}, where both adapter and lora are lists of length 12. For example, adapter = {d1, d2, d3, ..., d...} 12} indicates that embedding dimensions d1, d2, d3, ..., d are used in layers 1 to 12 of the model, respectively. 12 Similarly, for the Adapter module, lora = {d′1, d′2, d′3, ..., d′} 12} indicates that embedding dimensions d′1, d′2, d′3, ..., d′ are used in layers 1 to 12 of the model, respectively. 12 The LoRA module, if d i =0 or d′ i =0 indicates that the i-th layer does not require an Adapter or LoRA module.

[0089] S6-2: Randomly select two architectures and cross them to obtain p1;

[0090] Specifically, the method for cross-referencing two randomly selected architectures is to randomly cross-reference their architecture codes to obtain a new architecture code. For example, if architectures {adapter_1, lora_1} and {adapter_2, lora_2} are selected, then in the new cross-referenced architecture {adapter, lora}, the adapter is randomly selected from adapter_1 and adapter_2, and similarly, lora is randomly selected from lora_1 and lora_2.

[0091] S6-3: Randomly select the architecture in the population and mutate it to obtain p2;

[0092] Specifically, the method for mutating the architecture in a randomly selected population is as follows: for the selected architecture {adapter, lora}, the insertion depth of its adapter and lora parts will be randomly changed to obtain a new architecture code. For example, if the insertion depth of the adapter is randomly changed from l to l1, then the adapter list will change from the previous {d1, d2, ..., d...} l ,0…,0} changed to Similarly, if the insertion level of the lora is randomly changed from l′ to l2, then the lora list will change from the previous {d′1,d′2,…,d′} l′ ,0…,0} changed to

[0093] S7: Add the architectures p1 and p2 obtained in S6 to the population. If the total number of populations exceeds the predefined number of population architectures P, randomly select P architectures from the population as a new population.

[0094] S8: Repeat steps S5 to S7 until the preset number of iterations is met;

[0095] S9: Select the best-performing fine-tuned model from the population's model architecture as the final output.

[0096] Specifically, performance is the performance of the graph neural network model trained on the training set on the validation set. There are also various performance metrics to choose from, such as accuracy and recall.

[0097] Example 2

[0098] Reference Figure 2 This embodiment relates to a neural architecture search system for efficient fine-tuning of parameters in large omics models, including:

[0099] The data acquisition and preprocessing module is used to acquire data and preprocess it.

[0100] The supernet module is used to construct a neural network containing all possible parameter-efficient fine-tuning modules based on existing large-scale omics models and feasible efficient parameter fine-tuning methods.

[0101] The neural architecture training module is used to train neural architecture models.

[0102] Architecture encoding module, used to encode the neural networks contained in the supernet;

[0103] The neural architecture search module is used to efficiently fine-tune the configuration by obtaining the optimal parameters from the supernet according to the search strategy.

[0104] The model performance evaluation module is used to evaluate the performance of the trained neural architecture model.

[0105] The iterative module repeats steps S5 to S7 until the preset number of iterations is reached;

[0106] The results output module is used to select the model with the best performance as the final output result.

[0107] Example 3

[0108] The present invention also relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the method of Embodiment 1.

[0109] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A gene classification method for searching neural architectures for efficient fine-tuning of parameters in large omics models, characterized in that, Includes the following steps: S1: Collect gene classification data and preprocess the data; The gene classification data includes: datasets distinguishing gene dose sensitivity, datasets distinguishing long and short-range transcription factors, single-cell RNA sequencing datasets distinguishing bivalent and unmethylated genes, and single-cell RNA sequencing datasets distinguishing bivalent and H3K4-trimethylated genes only. S2: Collect existing large-scale single-cell omics models containing Transform as a basic model library; S3: Define and construct a search space based on the parameter efficient fine-tuning module applicable to single-cell omics large models, and construct a supernet based on the search space for searching and optimizing the neural architecture of single-cell omics large models; S4: Sample N parameters from the supernet to efficiently fine-tune the architecture as the initial population, and train the fine-tuned architecture until convergence; S5: Sort the architectures in the population according to the model performance and select the top k architectures; S6: Select, mutate, and cross over these k architectures to obtain a new population; S6-1: Code the architecture; S6-2: Randomly select two architectures and cross them to obtain... ; S6-3: Randomly select the architecture in the population and mutate it to obtain... ; S7: The architecture obtained from S6 If the total number of populations exceeds the predefined number of population architectures P, randomly select P architectures from the population to form a new population. S8: Repeat steps S5 to S7 until the automated machine learning algorithm converges; S9: Select the best-performing fine-tuned model from the population's model architecture as the final output. The parameter fine-tuning module for large-scale single-cell omics models described in step S3 includes an Adapter module, a LoRA module, and a Prefix module. These modules are described using the following formulas: The Adapter is described by the following formula: in, This is the normalized output of the MLP layer in the Transformer block of a large-scale omics model. The weights of the downsampling layer in the Adapter module. The weights of the upsampling layer in the Adapter module. For non-linear activation layers, ReLU or GELU activation functions are typically used; The LoRA description formula is as follows: in, This serves as the input to the Transformer block of the large-scale omics model. The weights of the query module in the Transformer block of the large-scale omics model. The weights of the downsampling layer in the query module. The weights of the upsampling layer in the query module; For the weights of the key module in the Transformer block of the large-scale omics model, The weights of the downsampling layer in the key module. The parameters of the upsampling layer of the key module. This is a scaling factor that controls the contribution of the LoRA module to the original weight matrix; The formula for describing the prefix is ​​as follows: in, This serves as the input to the Transformer block of the large-scale omics model. Let represent the weights of the key module in the i-th attention head of the Transformer block in the large-scale omics model. Let represent the weights of the value module of the i-th attention head in the Transformer block of the omics large-scale model. The weights of the query module in the Transformer block of the omics big data model; This is the prefix vector used by the i-th attention head in the Transformer block of the omics big data model for the key module. For the i-th attention head in the Transformer block of the omics big data model, it is used as the prefix vector for the value module; In step S6-1, the architecture is encoded using a list of length 36, which consists of three parts: adapter, lora, and prefix, each of which is a sublist of length 12. In step S6-2, two architectures are randomly selected for cross-interaction. Specifically, for the three parts of the architecture encoding, adapter, lora, and prefix, one of the corresponding parts of the two architectures is randomly selected as a new encoding item. The selection of all positions is completed in sequence to generate a new architecture encoding. In step S6-3, the architecture in the population is randomly selected for mutation. Specifically, for the adapter, lora, and prefix parts of the selected architecture code, a new insertion layer depth is randomly determined. For each part, based on the new insertion layer, the first depth elements in the original code are retained, and the unused positions are filled with a preset value of 0 to generate a new architecture code.

2. The gene classification method according to claim 1, which is a neural architecture search method for efficient fine-tuning of parameters in large omics models, is characterized in that: In the search space constructed in step S3, the insertion layer position, number of insertion layers, and module embedding dimension, which are suitable for the efficient fine-tuning of parameters of large single-cell omics models, are used as searchable parameters.

3. The gene classification method according to claim 1, which is a neural architecture search method for efficient fine-tuning of parameters in large omics models, is characterized in that: The supernet described in step S3 adopts a Transformer-based Encoder-Decoder structure. The initial layer of the network receives input data for training the omics task. The supernet includes a 12-layer structure, each layer includes a Transformer block, and each Transformer block contains a self-attention module and a feedforward network module. The self-attention module employs a multi-head attention mechanism, where the outputs of each attention head are concatenated after linear transformation. The feedforward network module includes two linear transformation layers and an intermediate nonlinear activation function, which is either ReLU or GELU. After the self-attention module and the feedforward network module of each Transformer block in the supernet, based on the search space described in step S3, parameter-efficient fine-tuning modules are inserted according to different insertion layer positions, insertion layer numbers and module embedding dimensions to construct a supernet structure covering all parameter-efficient fine-tuning module combination methods.

4. The gene classification method according to claim 1, which is a neural architecture search method for efficient fine-tuning of parameters in large omics models, is characterized in that: In step S3, the optimization objective for the neural architecture search of the single-cell omics large-scale model can be formalized as follows: : Among them, in the formula It is the parameter fine-tuning module to be searched; It is a collection of all feasible parameter fine-tuning modules in the entire search space; This represents the data used to train the model; It is an evaluation function used to assess the performance of the model architecture.

5. A neural architecture search system for efficient fine-tuning of parameters in large-scale omics models, used to implement the gene classification method of the neural architecture search method for efficient fine-tuning of parameters in large-scale omics models as described in claim 1, characterized in that, include: The data acquisition and preprocessing module is used to acquire data and preprocess it. The supernet module is used to construct a neural network containing all possible parameter-efficient fine-tuning modules based on existing large-scale omics models and parameter-efficient fine-tuning modules. The neural architecture training module is used to train neural architecture models. Architecture encoding module, used to encode the neural networks contained in the supernet; The neural architecture search module is used to obtain the best-performing fine-tuned omics model from the supernet according to the search strategy. The model performance evaluation module is used to evaluate the performance of the trained neural architecture model. The iteration module repeatedly runs the architecture coding module to the model performance evaluation module until the preset number of iterations is reached. The results output module is used to select the model with the best performance as the final output result.

Images