Machine learning based architecture independent neural network parameter generation method and system

Abstract

The application discloses an architecture-independent neural network parameter generation method and system based on machine learning, and the method generates corresponding base parameters and parameter masks through encoding by a text encoder, the base parameters are trimmed by using the parameter masks, so that artificial intelligence model parameters of different shapes are generated, and the architecture-independent effect is achieved.The method of the application is different from general neural network parameter generation methods, and can quickly generate parameters of neural networks of any architecture.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of machine learning, and particularly relates to a machine learning-based architecture-agnostic neural network parameter generation method and system. BACKGROUND

[0002] In recent years, the progress of large language models (LLMs) such as GPT-4 and LLaMA has greatly impacted people's daily lives, demonstrating their superior performance in automating routine tasks.

[0003] LLMs provide one-stop solutions for diverse user needs. Despite their impressive capabilities, LLMs still struggle to efficiently and conveniently meet the diverse needs of users. First, there is growing interest in deploying LLMs independently to obtain more private, secure, stable, and fast services. However, LLMs require a large amount of resources for training and deployment, which may be unaffordable for ordinary users. For example, pre-training LLaMA-2-70B requires 1.7 million GPU hours (A100-80G) and consumes 2.5 × 10 12 Joules of energy, and the power consumption of its inference phase is also considerable. Moreover, despite the diverse needs of users, LLMs may not always perform optimally in certain specific domains (such as legal, economic, and medical domains). In contrast, custom small models often perform better. However, users may lack sufficient expertise or time and resources to select and fine-tune these models, hindering the use of AI technology by ordinary users.

[0004] However, these pursuits face certain challenges. First, the diversity of user needs requires multi-level model customization. Differences in needs can lead to multi-level changes in models. While minor changes in user needs may only require adjusting the parameters of the target model for better performance, significant changes in tasks (such as from regression to classification, changes in data modalities) may require changes in the output dimension or even the architecture of the model. Second, to alleviate the burden on users, solutions need to have both general capabilities (such as task understanding) and precise customization capabilities, preferably without the need for extensive training requirements. Real-world situations often involve limited or no labeled data, sometimes only simple task descriptions. Without task understanding capabilities, it may not be possible to accurately capture user needs. Moreover, while large models can efficiently understand and handle general tasks without fine-tuning, deployment requires a large amount of resources. In contrast, small models are more efficient and easier to deploy in specific tasks, but require a large amount of time and expertise for fine-tuning to achieve optimal performance. Therefore, how to provide the best model simply and efficiently needs further consideration. SUMMARY

[0005] The present application aims to solve the problems in the prior art: 1) the existing artificial intelligence model optimization method needs to be trained / fine-tuned, that is, it requires additional computing resources and corresponding knowledge and skills, and does not promote the artificial intelligence model to ordinary users to realize the customization of the model; 2) the existing parameter generation method is mainly based on super network, and the super network can only generate fixed artificial intelligence model architecture and cannot respond to the extensive needs of users, and provides an architecture-independent neural network parameter generation method and system based on machine learning. The method can provide customized artificial intelligence model user experience for users on the one hand, greatly reducing the use threshold of artificial intelligence models; on the other hand, unlike the prior art, the method of the present application can quickly generate artificial intelligence models of different architectures.

[0006] In order to achieve the above-mentioned application purposes, the present application specifically adopts the following technical solutions:

[0007] In a first aspect, the present application provides an architecture-independent neural network parameter generation method based on machine learning, comprising the following steps:

[0008] S1. Obtain a machine learning dataset containing multiple tasks, construct a task description for each machine learning dataset, and construct a loss function according to the task type of each machine learning dataset, obtain an artificial intelligence model adapted to each machine learning dataset respectively, obtain the naming and parameter size of each sub-module in each artificial intelligence model, the naming and parameter size of each sub-module are in text format, the naming and parameter size of each sub-module are spliced in text to obtain the description information of each sub-module, after splicing the task description of each machine learning dataset and the description information of each sub-module corresponding to each machine learning dataset, the description information of each machine learning dataset is obtained, each machine learning dataset and the loss function, description information and artificial intelligence model corresponding to each machine learning dataset are combined together to form a four-tuple and serve as a sample of a pre-training database, until all machine learning datasets are combined, a complete pre-training database is obtained; wherein the machine learning dataset is a text dataset, an image dataset or a table dataset, the artificial intelligence model is a text classification model for classifying text datasets, an image classification model for classifying image datasets or a table classification model for classifying table datasets;

[0009] S2. sample a quadruple for pre-training from the pre-training database, obtain a super model comprising a text encoder and a multi-head decoder, pre-train the super model on the quadruple for pre-training to obtain a pre-trained super model; wherein the quadruple for pre-training comprises a sampled machine learning dataset, a loss function corresponding to the sampled machine learning dataset, description information, and an artificial intelligence model;

[0010] S3. obtain description information of a target task and a target artificial intelligence model for executing the target task, input the description information of the target task into the pre-trained super model, take parameters output by the pre-trained super model as parameters of the target artificial intelligence model, add the parameters of the target artificial intelligence model to the target artificial intelligence model, and output the target artificial intelligence model with the parameters to classify text data, image data, or table data.

[0011] On the basis of the above scheme, each step can be implemented in the following preferred specific manner.

[0012] As a preferred embodiment of the first aspect, in step S1, the names of the sub-modules and the parameter sizes of the sub-modules of the artificial intelligence model are obtained by using the native method torch.named_parameters in Pytorch.

[0013] As a preferred embodiment of the first aspect, in step S2, the pre-training operation of the super model is performed in the following specific process:

[0014] S21. In each iteration of the pre-training operation, the sampled description information is input into the text encoder, and the text encoder encodes the description information into a sequence of latent variables;

[0015] S22. The first element in the sequence of latent variables is concatenated before the remaining elements in the sequence of latent variables, and the first element in the sequence of latent variables is removed to obtain a processed sequence of latent variables;

[0016] S23. The i-th latent variable z i ′ in the processed sequence of latent variables is input into the multi-head decoder to obtain the base parameter b i and the parameter mask m i corresponding to the i-th latent variable;

[0017] S24. The base parameter is masked and selected by using the parameter mask to obtain the parameter θ i corresponding to the i-th latent variable:

[0018] θ i = reshape(mask_and_select(mi ,b i ))

[0019] wherein, mask_and_select represents a clipping function, reshape represents a parameter shape adjustment function;

[0020] S25. When each latent variable in the processed latent variable sequence is sequentially subjected to parameter generation, the parameters corresponding to each latent variable are spliced together to obtain a parameter vector and serve as the parameters of the sampled artificial intelligence model;

[0021] S26. The parameter vector θ 1…n is input into the sampled artificial intelligence model T, and the loss L of the sampled artificial intelligence model on the sampled machine learning data set is calculated according to the following formula:

[0022]

[0023] wherein, (x, y) ~ d represents that the input data x and the label y are sampled from the sampled machine learning data set d, represents taking the mathematical expectation: l represents the loss function in the quadruple used for pre-training; T represents the sampled artificial intelligence model;

[0024] S27. The parameters of the super model are updated using the loss, and the pre-training is continuously iterated until the maximum iteration round is reached, and the pre-trained super model is output.

[0025] As a preferred embodiment of the first aspect, in step S3, the target artificial intelligence model with generated parameters is used for inference on the machine learning data set of the target task to obtain a result, or used for training on the machine learning data set of the target task to obtain a target artificial intelligence model with better performance.

[0026] In a second aspect, the present application provides an architecture-independent neural network parameter generation system based on machine learning, comprising:

[0027] The data acquisition module is configured to acquire machine learning data sets containing multiple tasks, construct a task description for each machine learning data set, construct a loss function according to the task type of each machine learning data set, acquire an artificial intelligence model adapted to each machine learning data set, acquire the name and parameter size of each sub-module in each artificial intelligence model, the name and parameter size of each sub-module being in a text format, splice the name and parameter size of each sub-module to obtain description information of each sub-module, splice the task description of each machine learning data set and the description information of the sub-module corresponding to each machine learning data set to obtain description information of each machine learning data set, and combine each machine learning data set, the loss function corresponding to each machine learning data set, the description information and the artificial intelligence model corresponding to each machine learning data set to form a four-tuple as a sample of a pre-training database until all machine learning data sets are combined to obtain a complete pre-training database; wherein the machine learning data set is a text data set, an image data set or a table data set, the artificial intelligence model is a text classification model for classifying the text data set, an image classification model for classifying the image data set or a table classification model for classifying the table data set.

[0028] The model acquisition module is configured to sample a four-tuple for pre-training from the pre-training database, acquire a super model containing a text encoder and a multi-head decoder, pre-train the super model on the four-tuple for pre-training, and obtain a pre-trained super model; wherein the four-tuple for pre-training is composed of the sampled machine learning data set, the loss function corresponding to the sampled machine learning data set, the description information and the artificial intelligence model corresponding to the sampled machine learning data set.

[0029] The output module is configured to acquire description information of a target task and a target artificial intelligence model for executing the target task, input the description information of the target task into the pre-trained super model, use the parameters output by the pre-trained super model as the parameters of the target artificial intelligence model, add the parameters of the target artificial intelligence model to the target artificial intelligence model, and output the target artificial intelligence model with the generated parameters, to classify text data, image data or table data.

[0030] In a third aspect, the present application provides a computer readable storage medium, the storage medium storing a computer program, when the computer program is executed by a processor, the computer program implements the machine learning based architecture independent neural network parameter generation method according to any one of the first aspect.

[0031] In a fourth aspect, the present application provides a computer electronic device, comprising a memory and a processor.

[0032] The memory is configured to store a computer program.

[0033] The processor is configured to implement the machine learning based architecture independent neural network parameter generation method according to any one of the first aspect when executing the computer program.

[0034] In a fifth aspect, the present application provides a computer program product comprising computer programs / instructions, wherein the computer programs / instructions, when executed by a processor, can implement the machine learning based architecture independent neural network parameter generation method according to any one of the first aspect.

[0035] Compared with the prior art, the present application has the following beneficial effects:

[0036] 1) Compared with general machine learning training paradigms (full parameter fine-tuning, LoRA fine-tuning), etc., the method of the present application does not need to train the target artificial intelligence model, but only needs one round of inference process to obtain the target artificial intelligence model parameters, which is more efficient.

[0037] 2) Compared with general supermodel methods (parameter generation paradigm), the method of the present application does not need to adjust the supermodel structure when facing different network architectures (generally, one supermodel is needed to generate one model), and can directly respond to extensive parameter generation requirements (only one supermodel is needed to generate multiple model parameters). BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 A schematic diagram of the steps of the method of the present application is shown;

[0039] Figure 2 A system block diagram of the present application is shown. DETAILED DESCRIPTION

[0040] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. In the following description, a large number of specific details are set forth in order to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without departing from the scope of the present application, therefore the present application is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present application can be combined accordingly without conflict.

[0041] The present application proposes a machine learning-based architecture-independent neural network parameter generation method, which aims to respond to users' diverse and specific needs with the method of the present application, i.e. for users' data, i.e. the artificial intelligence model they want (target artificial intelligence model), the method of the present application needs to generate the parameters of the target artificial intelligence model within a round of inference time, so that the model can achieve sufficient performance effect on the user data. Specifically, the present application improves the general super network method: (1) using pre-training technology, using a large amount of task data to empower the method, so that the present application can meet the user's needs as much as possible. (2) Based on the Lottery Ticket Hypothesis (Lottery Ticket Hypothesis), improve the super model, add a multi-head decoder, generate base parameters and parameter masks at the same time, and use parameter masks to trim the base parameters into the corresponding modules of the target artificial intelligence model, thereby solving the problem of architecture-independent neural network parameter generation.

[0042] As Figure 1 shown, in a preferred implementation of the present application, the above-mentioned machine learning-based architecture-independent neural network parameter generation method comprises the following S1-S3 steps, aiming to: (I) meet the diverse and specific needs of users, quickly realize the adjustment of model parameters to adapt to the data distribution of users; (II) simplify the use of AI models by ordinary users, and the parameter generation method provided must be able to solve the problem of generating artificial intelligence models of multiple architectures at the same time. The specific implementation process is described below.

[0043] S1. Obtain a machine learning dataset d containing multiple tasks, construct a task description x0 for each machine learning dataset, and construct a loss function l according to the task type of each machine learning dataset. Obtain an artificial intelligence model T adapted to each machine learning dataset, obtain the name and parameter size of each sub-module in each artificial intelligence model T, the name and parameter size of each sub-module are in text format, concatenate the name and parameter size of each sub-module to obtain the description information of each sub-module, and concatenate the task description x0 of each machine learning dataset and the description information of each sub-module corresponding to each machine learning dataset to form the description information x of each machine learning dataset 0...n =(x0,x1,...,x n ), where n is the total number of sub-modules of the artificial intelligence model T; concatenate each machine learning dataset d and the loss function l and the description information x 0...nThe artificial intelligence model T is combined with the machine learning dataset to form a quadruple and serves as a sample of the pre-training database, until all machine learning datasets are combined to obtain a complete pre-training database D. The machine learning dataset is a text dataset, an image dataset, or a table dataset, and the artificial intelligence model is a text classification model for classifying the text dataset, an image classification model for classifying the image dataset, or a table classification model for classifying the table dataset.

[0044] It should be noted that in step S1 of the present application, the native method torch.named_parameters in Pytorch is used to obtain the naming of each sub-module of the artificial intelligence model and the parameter size of the sub-module.

[0045] S2. Sampling a quadruple for pre-training from the pre-training database D to obtain a super model containing a text encoder and a multi-head decoder, pre-training the super model on the quadruple for pre-training to obtain a pre-trained super model; wherein the quadruple for pre-training is composed of the sampled machine learning dataset, the loss function corresponding to the sampled machine learning dataset, the description information, and the artificial intelligence model.

[0046] It should be noted that in step S2 of the present application, the specific process of pre-training the super model is as follows:

[0047] S21. In each iteration of the pre-training operation, the sampled description information x 0...n is input into the text encoder, and the text encoder encodes the description information into a sequence of latent variables z 0...n .

[0048] S22. The first element z0 in the sequence of latent variables is sequentially spliced before the remaining elements in the sequence of latent variables, and the first element in the sequence of latent variables is removed to obtain a processed sequence of latent variables z1 ′ …n .

[0049] S23. The i-th latent variable z i ′ in the processed sequence of latent variables is input into the multi-head (parameter) decoder to obtain the i-th latent variable z i ′ corresponding to the base parameter b i and the parameter mask m i .

[0050] It should be noted that the super model in the method of the present application refers to a model used to generate a model, in the present embodiment, the super model is composed of a text encoder and a multi-head decoder, wherein the text encoder adopts a Transformer-like architecture, and the multi-head decoder (hidden variable into parameter) is composed of two separate but structurally identical multi-layer perceptrons. The text encoder is used to encode the description information into a hidden variable sequence, and the multi-head decoder is used to decode the hidden variable into a base parameter and a parameter mask. The core of the present application is the architecture-independent neural network parameter generation based on the lottery ticket hypothesis. The text encoder and the multi-head decoder in the present application have no excessive restrictions on the architecture, and can be replaced by a Transformer-like model, a convolutional neural network or a recurrent neural network model according to the needs of specific applications.

[0051] S24. Utilize the parameter mask m i To the base parameter b i Perform a mask and selection operation to obtain the i-th hidden variable z i ′ The corresponding parameter θ i :

[0052] θ i = reshape(mask_and_select(m i ,b i ))

[0053] Wherein, mask_and_select represents a clipping function, and reshape represents a parameter shape adjustment function.

[0054] In the present embodiment, mask_and_select and reshape are both pytorch native functions, mask_and_select is used to clip and select parameters based on the mask, and reshape only adjusts the shape of the parameters.

[0055] S25. After the parameters of each hidden variable in the processed hidden variable sequence are generated in turn, the parameters of each hidden variable z i ′ are spliced together to obtain a parameter vector θ 1...n =(θ1,…,θ n ), which is used as the parameter of the sampled artificial intelligence model.

[0056] S26. The parameter vector θ 1…n is brought into the sampled artificial intelligence model T, and the loss L of the sampled artificial intelligence model on the sampled machine learning data set is calculated according to the following formula:

[0057]

[0058] Wherein, (x, y) ~ d represents that the input data x and the label y are sampled from the machine learning data set d obtained by sampling, represents taking the mathematical expectation: l represents the loss function in the quadruple used for pre-training; T represents the artificial intelligence model sampled.

[0059] S27. Update the parameters of the super model by using the loss L, and continuously iterate the pre-training until the maximum iteration round is reached, and output the pre-trained super model.

[0060] S3. Obtain the description information of the target task and the target artificial intelligence model used for executing the target task, input the description information of the target task into the pre-trained super model, take the parameters output by the pre-trained super model as the parameters of the target artificial intelligence model, add the parameters of the target artificial intelligence model to the target artificial intelligence model, and output the target artificial intelligence model with generated parameters, which is used for classifying text data, image data or table data.

[0061] It should be noted that in step S3 of the present application, the target artificial intelligence model with generated parameters is used to perform reasoning on the machine learning data set of the target task to obtain a result, or is used to perform training on the machine learning data set of the target task to obtain a target artificial intelligence model with better performance.

[0062] The present application will now be described through a specific example to show the application effect of the machine learning based architecture independent neural network parameter generation method described in S1-S3 of the above embodiment on a specific data set, so as to facilitate understanding of the essence of the present application.

[0063] Embodiment

[0064] The machine learning based architecture independent neural network parameter generation method used in the present embodiment is described above, and will not be repeated here.

[0065] In this embodiment, the application of the method of the present application to text data sets is demonstrated. In this embodiment, three text data sets are used for training and two text data sets are used for testing. The three text data sets are ANLI_R1, ANLI_R2 and ANLI_R3, and the two text data sets are CB and MNLI. ZeroShot and ModelGPT are trained on the three text data sets ANLI_R1, ANLI_R2 and ANLI_R3, and then tested on the two text data sets CB and MNLI. ModelGPT-F is trained on the three text data sets ANLI_R1, ANLI_R2 and ANLI_R3, and then tested on the target data set after an additional round of training. The model generated by all methods is DistilBERT-base, and the accuracy of each method is shown in Table 1.

[0066] Table 1. Accuracy results of different methods

[0067]

[0068] In summary, the method of generating architecture-independent neural network parameters based on machine learning has the following advantages:

[0069] 1) Unlike traditional artificial intelligence model training methods, the method of the present application can generate the required neural network parameters in one inference time without training / tuning.

[0070] 2) Unlike traditional parameter generation methods, the method of the present application can simultaneously solve the parameter generation problem of multiple architectures rather than being limited to the parameter generation problem of one neural network, greatly improving the flexibility of the parameter generation method.

[0071] 3) The method of the present application is based on lottery theory, which uses a super model to generate base parameters and parameter masks at the same time. The base parameters are trimmed by the parameter masks to generate parameters for different architectures of neural networks.

[0072] 4) In order to better respond to the needs of different scenarios, the method of the present application needs to be pre-trained a lot.

[0073] It should be further noted that the machine learning based architecture independent neural network parameter generation method in the above embodiment can be essentially executed by a computer program or module. Therefore, based on the same inventive concept, another preferred embodiment of the present application also provides a machine learning based architecture independent neural network parameter generation system corresponding to the machine learning based architecture independent neural network parameter generation method provided by the above embodiment, as shown in Figure 2 which comprises:

[0074] a data acquisition module, configured to acquire a plurality of machine learning data sets containing tasks, construct a task description for each machine learning data set, construct a loss function according to the task type of each machine learning data set, acquire an artificial intelligence model adapted to each machine learning data set respectively, acquire the name and parameter size of each sub-module in each artificial intelligence model, the name and parameter size of each sub-module being in a text format, splice the name and parameter size of each sub-module to obtain description information of each sub-module, splice the task description of each machine learning data set and the description information of the sub-module corresponding to each machine learning data set respectively to form the description information of each machine learning data set, and combine each machine learning data set, the loss function, the description information and the artificial intelligence model corresponding to each machine learning data set respectively to form a four-tuple as a sample of a pre-training database until all machine learning data sets are combined to obtain a complete pre-training database; wherein the machine learning data set is a text data set, an image data set or a table data set, the artificial intelligence model is a text classification model for classifying the text data set, an image classification model for classifying the image data set or a table classification model for classifying the table data set;

[0075] a model acquisition module, configured to sample a four-tuple for pre-training from the pre-training database, acquire a super model containing a text encoder and a multi-head decoder, and pre-train the super model on the four-tuple for pre-training to obtain a pre-trained super model; wherein the four-tuple for pre-training is composed of the sampled machine learning data set, the loss function, the description information and the artificial intelligence model corresponding to the sampled machine learning data set;

[0076] an output module, configured to acquire the description information of a target task and a target artificial intelligence model for executing the target task, input the description information of the target task into the pre-trained super model, take the parameters output by the pre-trained super model as the parameters of the target artificial intelligence model, add the parameters of the target artificial intelligence model to the target artificial intelligence model, and output the target artificial intelligence model with generated parameters, for classifying text data, image data or table data.

[0077] It can be understood that the machine learning based architecture independent neural network parameter generation method described in the above S1-S3 can be essentially realized by a computer program. Therefore, based on the same inventive concept, another preferred embodiment of the present application also provides a computer program product corresponding to the machine learning based architecture independent neural network parameter generation method provided by the above embodiment, which includes computer programs / instructions that can realize the machine learning based architecture independent neural network parameter generation method as described in the above embodiment when executed by a processor.

[0078] Similarly, based on the same inventive concept, another preferred embodiment of the present application also provides a computer electronic device corresponding to the machine learning based architecture independent neural network parameter generation method provided by the above embodiment, which includes a memory and a processor;

[0079] The memory is configured to store a computer program.

[0080] The processor is configured to realize the machine learning based architecture independent neural network parameter generation method in the above embodiment when executing the computer program.

[0081] In addition, the logical instructions in the above memory can be realized in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the part of the technical solutions that essentially contribute to the prior art or the part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiments of the present application.

[0082] Therefore, based on the same inventive concept, another preferred embodiment of the present application also provides a computer readable storage medium corresponding to the machine learning based architecture independent neural network parameter generation method provided by the above embodiment, which stores a computer program. When the computer program is executed by a processor, the machine learning based architecture independent neural network parameter generation method in the above embodiment can be realized.

[0083] It can be understood that the above storage medium can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk memory. Meanwhile, the storage medium can also be a U disk, a mobile hard disk, a magnetic disk or an optical disk, etc. various media that can store program codes.

[0084] It can be understood that the processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0085] In addition, it should be noted that the skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here. In the embodiments provided in the present application, the division of steps or modules in the system and method is only a logical functional division, and there can be another division mode in actual implementation, for example, a plurality of modules or steps can be combined or integrated together, or a module or step can be split.

[0086] The above-described embodiments are only a preferred scheme of the present application, and are not intended to limit the present application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application. Therefore, any technical scheme obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present application.

Claims

1. A method for generating parameters of an architecture-independent neural network based on machine learning, characterized in that: The following steps are involved: S1. Obtain a machine learning dataset containing multiple tasks, construct a task description for each machine learning dataset, and construct a loss function according to the task type of each machine learning dataset, obtain an artificial intelligence model adapted to each machine learning dataset, obtain the name of each submodule in each artificial intelligence model and the parameter size of each submodule, the name and parameter size of each submodule are in text format, perform text splicing on the name and parameter size of each submodule to obtain the description information of each submodule, splice the task description of each machine learning dataset with the description information of the submodule corresponding to each machine learning dataset to form the description information of each machine learning dataset, combine each machine learning dataset and the loss function, description information and artificial intelligence model corresponding to each machine learning dataset to form a quadruple as a sample of the pre-training database, until all machine learning datasets are combined to obtain a complete pre-training database; wherein the machine learning dataset is a text dataset, an image dataset or a table dataset, and the artificial intelligence model is a text classification model for classifying a text dataset, an image classification model for classifying an image dataset or a table classification model for classifying a table dataset; S2. Sample a pre-training quadruple from the pre-training database, obtain a super-model comprising a text encoder and a multi-head decoder, and perform a pre-training operation on the super-model on the pre-training quadruple to obtain a pre-trained super-model; wherein the pre-training quadruple comprises the sampled machine learning dataset, a loss function corresponding to the sampled machine learning dataset, descriptive information, and an artificial intelligence model; S3. Obtain the descriptive information of the target task and the target artificial intelligence model used to perform the target task, input the descriptive information of the target task into the pre-trained super model, use the parameters output by the pre-trained super model as the parameters of the target artificial intelligence model, add the parameters of the target artificial intelligence model to the target artificial intelligence model, and output the parameters to generate a target artificial intelligence model for classifying text data, image data or tabular data.

2. The method for generating architecture-independent neural network parameters based on machine learning according to claim 1, wherein: In step S1, the native method torch.named_parameters in Pytorch is used to obtain the names of each sub-module of the artificial intelligence model and the parameter size of the sub-module.

3. The method for generating architecture-independent neural network parameters based on machine learning according to claim 1, wherein: In step S2, the specific process of pre-training the super model is as follows: S21. In each iteration of the pre-training operation, the sampled description information is input into the text encoder, and the text encoder encodes the description information into a latent variable sequence; S22. Concatenate the first element in the latent variable sequence one by one before the remaining elements in the latent variable sequence, and remove the first element in the latent variable sequence to obtain a processed latent variable sequence; S23. The i-th latent variable z in the processed latent variable sequence i ′ Input into the multi-head decoder to obtain the basis parameter b corresponding to the i-th hidden variable i and parameter mask m i ; S24. Use parameter mask to perform masking and selection operations on the basis parameters to obtain the parameter θ corresponding to the i-th latent variable i : θ i =reshape(mask_and_select(m i ,b i )) Among them, mask_and_select represents the cropping function, and reshape represents the parameter shape adjustment function; S25. After parameters are generated for each latent variable in the processed latent variable sequence, the parameters corresponding to each latent variable are concatenated to obtain a parameter vector, which is used as the parameter of the sampled artificial intelligence model. S26. Set the parameter vector θ 1…n Bring it into the sampled artificial intelligence model T and calculate the loss L of the sampled artificial intelligence model on the sampled machine learning dataset according to the following formula: Among them, (x,y)~d represents the input data x and label y sampled from the sampled machine learning dataset d, Indicates the mathematical expectation: l represents the loss function in the quadruple used for pre-training; T represents the artificial intelligence model obtained by sampling; S27. Use the loss to update the parameters of the super model, and continuously iterate the pre-training until the maximum number of iterations is reached, and output the pre-trained super model.

4. The method for generating architecture-independent neural network parameters based on machine learning according to claim 1, wherein: In step S3, the target artificial intelligence model with generated parameters is used to perform inference on the machine learning dataset of the target task to obtain results, or is used to perform training on the machine learning dataset of the target task to obtain a target artificial intelligence model with better performance.

5. A machine learning-based architecture-independent neural network parameter generation system, characterized in that: include: A data acquisition module is used to obtain a machine learning dataset containing multiple tasks, construct a task description for each machine learning dataset, and construct a loss function based on the task type of each machine learning dataset, obtain an artificial intelligence model that is adapted to each machine learning dataset, obtain the name of each submodule in each artificial intelligence model and the parameter size of each submodule, the name and parameter size of each submodule are in text format, and perform text splicing on the name and parameter size of each submodule to obtain the description information of each submodule, splicing the task description of each machine learning dataset with the description information of the submodule corresponding to each machine learning dataset to form the description information of each machine learning dataset, and combining each machine learning dataset and the loss function, description information and artificial intelligence model corresponding to each machine learning dataset to form a quadruple as a sample of a pre-training database, until all machine learning datasets are combined to obtain a complete pre-training database; wherein the machine learning dataset is a text dataset, an image dataset or a table dataset, and the artificial intelligence model is a text classification model for classifying a text dataset, an image classification model for classifying an image dataset or a table classification model for classifying a table dataset; a model acquisition module, configured to sample a pre-training quadruple from a pre-training database, obtain a super-model comprising a text encoder and a multi-head decoder, and perform a pre-training operation on the super-model on the pre-training quadruple to obtain a pre-trained super-model; wherein the pre-training quadruple comprises a sampled machine learning dataset, a loss function corresponding to the sampled machine learning dataset, descriptive information, and an artificial intelligence model; The output module is used to obtain the descriptive information of the target task and the target artificial intelligence model used to perform the target task, input the descriptive information of the target task into the pre-trained super model, use the parameters output by the pre-trained super model as the parameters of the target artificial intelligence model, add the parameters of the target artificial intelligence model to the target artificial intelligence model, and output the parameters to generate the target artificial intelligence model for classifying text data, image data or tabular data.

6. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by the processor, the architecture-independent neural network parameter generation method based on machine learning as described in any one of claims 1 to 4 is implemented.

7. A computer electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to implement the architecture-independent neural network parameter generation method based on machine learning as described in any one of items 1 to 4 when executing the computer program.

8. A computer program product comprising a computer program / instructions, characterized in that When this computer program / instruction is executed by a processor, it can implement the architecture-independent neural network parameter generation method based on machine learning as described in any one of items 1 to 4.

Images