Large model management method and device, electronic equipment and storage medium

Abstract

The invention provides a large model management method and device, electronic equipment and a storage medium, and relates to the fields of medicine, law, science and technology, finance and the like, in particular to the technical fields of artificial intelligence, intelligent customer service, text translation, content creation, decision support, text processing and large language models. The specific implementation scheme is as follows: acquiring a first resource quantity actually consumed by a large language model for processing a preset type of target problems within a preset time period; and determining a second resource quantity predicted to be consumed by training the preset small model and a third resource quantity predicted to be consumed by the preset small model for processing the target problem in the preset time period, wherein the third resource quantity is smaller than the first resource quantity. And then determining whether a preset small model training condition is satisfied based on the first resource quantity, the second resource quantity and the third resource quantity. If yes, it is determined that a preset small model is trained, and the trained preset small model is used for processing a preset type of problems. And resources consumed by the service provided based on the large language model are reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to the field of text processing and large language model technology. Background Art

[0002] Large Language Model (LLM) is obtained by training a deep learning model using a large amount of natural language text. It can understand the meaning of the input natural language text and generate natural language text. Large language model can be widely used in artificial intelligence fields such as question answering, dialogue or text classification. Summary of the invention

[0003] The present disclosure provides a large model management method, device, electronic device and storage medium.

[0004] A first aspect of the embodiments of the present disclosure provides a large model management method, including:

[0005] Obtaining a first amount of resources actually consumed by the large language model in processing a preset type of target question within a preset time period;

[0006] Determine a second amount of resources that is expected to be consumed by training a preset small model, and a third amount of resources that is expected to be consumed by the preset small model in processing the target problem within the preset time period, wherein the third amount of resources is less than the first amount of resources;

[0007] Based on the first resource amount, the second resource amount, and the third resource amount, determining whether a preset small model training condition is met;

[0008] If so, determine to train the preset small model, wherein the trained preset small model is used to process the preset type of problem.

[0009] A second aspect of the embodiments of the present disclosure provides a large model management device, including:

[0010] An acquisition module, used to acquire a first amount of resources actually consumed by the large language model in processing a preset type of target question within a preset time period;

[0011] The determination module is further used to determine a second amount of resources that is expected to be consumed by training a preset small model, and a third amount of resources that is expected to be consumed by the preset small model to process the target problem within the preset time period, wherein the third amount of resources is less than the first amount of resources;

[0012] A judgment module, used to determine whether a preset small model training condition is met based on the first resource amount, the second resource amount and the third resource amount;

[0013] The determination module is also used to determine the training of the preset small model if the judgment result of the judgment module is yes, wherein the trained preset small model is used to handle the problem of the preset type.

[0014] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any method according to the first aspect.

[0018] According to a fourth aspect of an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute a method according to any one of the first aspects.

[0019] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method according to any one of the first aspects is implemented.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0022] Figure 1 is a flow chart of a large model management method provided by an embodiment of the present disclosure;

[0023] Figure 2 is a flow chart of a method for training a preset small model provided by an embodiment of the present disclosure;

[0024] Figure 3 is a flowchart of a method for processing a question input by a user provided by an embodiment of the present disclosure;

[0025] Figure 4 is an exemplary schematic diagram of a large model management process provided by an embodiment of the present disclosure;

[0026] Figure 5 It is a structural diagram of a large model management device provided by an embodiment of the present disclosure;

[0027] Figure 6 It is a block diagram of an electronic device used to implement the large model management method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] Although large language models have powerful natural language understanding and generation capabilities, in actual application, large language models have the problem of high resource consumption, which makes the cost of services provided based on large language models high.

[0030] In order to reduce the resources consumed by services provided based on large language models, the present disclosure provides a large model management method, which runs on an electronic device, for example, a server or a desktop computer or other device with data processing capabilities. Figure 1 As shown, the large model management method provided by the embodiment of the present disclosure includes the following steps:

[0031] S101. Obtain a first amount of resources actually consumed by a large language model in processing a preset type of target question within a preset time period.

[0032] Optionally, multiple categories can be divided in advance according to the field to which the question belongs, for example, multiple categories include medicine, law, technology, finance, etc. Alternatively, multiple categories can be divided in advance according to the task type requested by the question, for example, multiple categories include calculation, translation, recommendation, creation, judgment, etc. Alternatively, the above categories can be further divided into finer granularity, which is not specifically described in the embodiments of the present disclosure. The preset category in S101 can be one or more pre-divided categories. When the preset category includes multiple categories, the large model management method provided by the embodiments of the present disclosure can be executed for each category included in the preset category, and the subsequent embodiments are all described by taking the preset category as one category as an example.

[0033] In the disclosed embodiment, a user can input a question in the display interface of the terminal, and when the terminal detects that the user triggers a submission instruction, the terminal sends a question-and-answer request to the electronic device, wherein the question-and-answer request includes the question input by the user. When the preset small model is not deployed, the electronic device processes the question through the large language model, and then returns the processing result of the large language model to the question, that is, the answer to the question, to the terminal, so that the electronic device displays the answer to the user.

[0034] When receiving a management instruction triggered by a user or sent by another device, the electronic device can execute Figure 1 Alternatively, the electronic device may execute the method periodically or periodically Figure 1 Accordingly, the preset time period may be a time period included in the management instruction, or a time period between a historical moment with a preset time from the current moment and the current moment, or a time period of the previous cycle, etc.

[0035] The first resource amount may include multiple computing resources. For example, the first resource amount includes: central processing unit (CPU) resources, graphics processing unit (GPU) resources, memory resources and / or video memory resources.

[0036] S102. Determine a second amount of resources that is expected to be consumed by training a preset small model, and a third amount of resources that is expected to be consumed by the preset small model to process a target problem within a preset time period.

[0037] Since the preset small model has not been trained and deployed at this time, the second resource amount and the third resource amount can be determined manually based on experience, or determined by the electronic device based on historical statistical parameters.

[0038] For example, the electronic device can obtain similar categories of preset categories, determine the similar small models corresponding to the similar categories, use the amount of resources actually consumed in training the similar small models as the second amount of resources, and use the amount of resources actually consumed by the similar small models in processing similar types of problems within a preset time period as the third amount of resources.

[0039] Compared with the large language model, the structure and / or processing logic of the preset small model is simpler. For example, compared with the large language model, the preset small model includes fewer network layers, or the processing logic of the network layer of the preset small model is simpler. Therefore, in the embodiment of the present disclosure, the third resource amount is less than the first resource amount, that is, compared with the large language model, the amount of resources consumed by using the preset small model to process the preset type of problem is less.

[0040] The second resource amount and the third resource amount may include multiple computing resources, and the first resource amount, the second resource amount and the third resource amount include the same computing resources. For example, the first resource amount, the second resource amount and the third resource amount are all GPU resources.

[0041] S103: Based on the first resource amount, the second resource amount and the third resource amount, determine whether the preset small model training condition is met. If yes, execute S104; if no, determine not to train the preset small model temporarily.

[0042] By comparing the first resource amount and the third resource amount, we can get the resources that can be saved by deploying the preset small model, that is, the deployment benefit of the preset small model; and the second resource amount can represent the training cost of the preset small model. Therefore, by using the deployment benefit and training cost of the preset small model, we can analyze whether the gap between the benefit and cost of deploying the preset small model is reasonable from an economic perspective, that is, whether the preset small model training conditions are met.

[0043] S104, determine to train a preset small model, wherein the trained preset small model is used to process a preset type of problem.

[0044] The disclosed embodiment can obtain the first amount of resources actually consumed by the large language model to process a preset type of target problem within a preset time period, the second amount of resources estimated to be consumed by training the preset small model, and the third amount of resources estimated to be consumed by the preset small model to process the target problem within a preset time period, and based on these three amounts of resources, when it is determined that the preset small model training conditions are met, determine to train the preset small model so that the trained preset small model processes the preset type of problem. Since the amount of resources estimated to be consumed by the preset small model to process the preset type of problem is less than the amount of resources actually consumed by the large language model to process the preset type of problem, resource consumption can be reduced by replacing the large language model with the preset small model to process the preset type of problem.

[0045] The following is a detailed description of the large model management method provided by the embodiment of the present disclosure:

[0046] exist Figure 1 Prior to the process shown, the electronic device may also pre-train a large language model.

[0047] Before training the large language model, the electronic device may obtain an initial corpus, wherein the initial corpus includes multiple types of corpora, each of which may be a text in an article, or a piece of news, or a historical conversation in an actual scenario, etc.

[0048] Afterwards, the electronic device can perform unsupervised training on the preset deep learning model based on the initial corpus. For example, each corpus is split into a front half and a back half, the front half is input into the deep learning model, and the back half predicted by the deep learning model is obtained. The loss value is calculated based on the back half predicted by the deep learning model and the back half actually corresponding to the front half. The network parameters of the deep learning model are adjusted using the loss value until the deep learning model converges to obtain a deep learning model after unsupervised training. This method is only an example of unsupervised training provided in an embodiment of the present disclosure, and the unsupervised training method that can be used in an embodiment of the present disclosure in an actual scenario is not limited to this.

[0049] Afterwards, the electronic device can perform supervised training on the deep learning model after unsupervised training based on the fine-tuning corpus. The fine-tuning corpus includes multiple corpus pairs, each of which includes a question and a standard answer to the question. During supervised training, the question included in each corpus pair can be input into the deep learning model after unsupervised training to obtain the predicted answer output by the deep learning model. Then, based on the predicted answer and the standard answer, the loss value is calculated. The network parameters of the deep learning model are adjusted using the loss value until the deep learning model converges, and the current deep learning model is used as a large language model.

[0050] After the large language model is trained, the electronic device can deploy the large language model so that the large language model can provide services to the outside world, that is, use the large language model to generate answers to questions input by users.

[0051] In some embodiments of the present disclosure, before obtaining the first amount of resources actually consumed by the large language model to process a preset type of target question within a preset time period in the above S101, the electronic device may also determine whether an economic analysis condition for the preset type is currently met.

[0052] That is, the electronic device can also determine whether the number of target questions processed by the large language model within a preset time period reaches a preset number. For example, the preset time period is the previous day and the preset number is 5,000.

[0053] Each time the electronic device receives a question, it can determine the category of the question through the classifier, and record the category of each question and the time it takes for the large language model to process each question. Based on this, the electronic device can obtain the number of target questions of a preset category processed by the large language model within a preset time period.

[0054] If the preset number is reached, the step S101 is executed to obtain the first amount of resources actually consumed by the large language model in processing the target question of the preset type within the preset time period.

[0055] If the preset number is not reached, it is determined not to execute S101.

[0056] When the number of target questions processed by the large language model within a preset time period reaches a preset number, it means that users often ask questions of preset types, that is, the processing requirements of the preset types of questions are high. Therefore, S101 is executed in this case to determine whether to train the preset small model, which can reduce the problem of low actual utilization of the trained preset small model, that is, improve the utilization of the preset small model after deployment.

[0057] In the embodiment of the present disclosure, the above-mentioned S101 obtains the first resource amount actually consumed by the large language model in processing the preset type of target question within the preset time period in the following two ways:

[0058] S101: Method 1 for determining the first resource amount, including the following steps:

[0059] Step 1: Obtain the total amount of resources actually consumed by the large language model to process multiple types of questions within a preset time period.

[0060] The electronic device can obtain the total amount of resources actually consumed by all questions processed by the large language model within a preset time period.

[0061] The module that executes the large model management method provided in the embodiment of the present disclosure may be called a management module, and the management module runs in an electronic device. The large language model may also run in the electronic device, or the large language model may run in other devices.

[0062] When the large language model runs in the electronic device, the electronic device can obtain the total amount of resources actually consumed by the large language model in processing multiple types of questions within a preset time period based on its own operation log.

[0063] When the large language model is running in other devices, the electronic device can obtain the operation log of the other device where the large language model is located, and obtain the total amount of resources actually consumed by the large language model in processing multiple types of questions within a preset time period from the operation log.

[0064] Alternatively, the electronic device may obtain from the monitoring module the total amount of resources actually consumed by the large language model in processing multiple types of questions within a preset time period.

[0065] For example, the management module, the large language model, and the monitoring module are respectively in a pod of the container orchestration engine (kubernetes, k8s), each pod includes at least one container, each container is used to run a copy, and each container is in a device. The monitoring module can monitor the real-time consumption of the large language model and the real-time consumption of the deployed preset small model. Based on this, the management module can obtain from the monitoring module the total amount of resources actually consumed by the large language model to process multiple types of problems within a preset time period.

[0066] The total resource amount may include multiple computing resources, and the total resource amount and the first resource amount include the same computing resources. For example, the total resource amount and the first resource amount are both the weighted sum of CPU resources, GPU resources, and memory resources. Since CPU resources have a higher value, the weight of CPU resources can be set higher than the weight of other computing resources.

[0067] Step 2: Determine the proportion of target questions among multiple types of questions.

[0068] Since the electronic device can determine the category of each question it receives, the electronic device can determine the proportion of target questions of a preset category from the multiple categories of questions processed by the large language model within a preset time period.

[0069] Step 3: Determine the first resource amount based on the total resource amount and the ratio.

[0070] The electronic device may use the product of the total resource amount and the ratio as the first resource amount.

[0071] Alternatively, the electronic device may calculate the product of the total resource amount and the ratio, and use the sum of the product and the preset adjustment factor as the first resource amount. The preset adjustment factor may be determined based on a preset category. For example, assuming that the number of words contained in the target question of the preset category is usually small, the preset adjustment factor may be set to a negative number; conversely, assuming that the number of words contained in the target question of the preset category is usually large, the preset adjustment factor may be set to a positive number.

[0072] Since the larger the number of preset target questions is, the more resources the large language model consumes to process the target questions, the resources consumed by the large language model to process the target questions are related to the proportion of the target questions in the various questions processed by the large language model. Therefore, the embodiment of the present disclosure determines the first resource amount based on the total resource amount and the proportion to ensure the accuracy of determining the first resource amount.

[0073] S101: Method 2 for determining the first resource quantity, including the following steps:

[0074] Step 1: Obtain the total amount of resources actually consumed by the large language model to process multiple types of questions within a preset time period.

[0075] The implementation method of step 1 is the same as that of step 1. Please refer to the above description of step 1 and will not be repeated here.

[0076] Step 2: Determine the question length of the target question.

[0077] Before inputting each question into the large language model, the electronic device can perform word segmentation on the question. The smallest unit of the data after word segmentation is a token, and the length of the question is the token length. For example, the question input by the user is "Please help me write an essay." After word segmentation of the question, "Please, help, me, write, an, essay,." is obtained, where "," is used to separate different tokens, and the token length of the question is 7.

[0078] Alternatively, the electronic device may use the number of words or characters included in the question as the question length.

[0079] Step 3: Determine the first resource amount based on the total resource amount and the problem length.

[0080] The electronic device can determine the amount of resources consumed by the large language model to process each target question within a preset time period based on the total amount of resources and the question length of each target question, and then sum the determined amount of resources consumed by each target question as the first amount of resources.

[0081] Alternatively, the electronic device may calculate the average length of each target question processed by the large language model within a preset time period, and then determine the average amount of resources consumed by the large language model to process the target question based on the total amount of resources and the average length. Then, the determined average amount of resources consumed is multiplied by the number of target questions processed by the large language model within the preset time period to obtain the first amount of resources.

[0082] Alternatively, the electronic device may also determine the first resource amount based on other methods, which are not detailed in the embodiments of the present disclosure.

[0083] Since the longer the question is, the greater the amount of computation required by the large language model to process the question, and thus the more resources consumed, the more resources are consumed. Therefore, the embodiment of the present disclosure determines the first amount of resources based on the question length, which can improve the accuracy of determining the first amount of resources.

[0084] In some embodiments of the present disclosure, the above-mentioned S103 electronic device determines whether the preset small model training conditions are met based on the first resource amount, the second resource amount and the third resource amount, which can be implemented as follows: calculating the difference between the first resource amount and the third resource amount, calculating the ratio of the second resource amount to the difference, and then determining whether the ratio is less than or equal to a preset threshold.

[0085] If so, it is determined that the preset small model training conditions are met.

[0086] If not, it is determined that the preset small model training conditions are not met.

[0087] It can be understood that the difference between the first resource amount and the third resource amount represents the estimated amount of resources that can be saved by the preset small model in a preset time period to handle the preset type of problems after the preset small model is deployed. The second resource amount represents the resource cost of training the preset small model. Therefore, the ratio of the second resource amount to the difference represents the time it takes to recover the cost after the small model is deployed. The smaller the ratio, the faster the cost recovery.

[0088] Taking the example that the first, second and third resource amounts are all GPU resources, assuming that the large language model actually consumes 10 billion operations per second (Giga Operations Per Second, G) of GPU resources to process type A problems every day, that is, the first resource amount is 10G. Training the preset small model is expected to consume 100G of GPU resources per day, and the estimated training time is 1 day, that is, the second resource amount is 100G×1=100G. The preset small model is expected to consume 5G of GPU resources to process type A problems every day, that is, the third resource amount is 5G. At this time, 100 / (10-5)=20 days, that is, the cost can be recovered in 20 days.

[0089] The preset threshold value may be determined according to the length of the preset time period, for example, the preset threshold value may be an integer multiple of the length of the preset time period. For example, the length of the preset time period is 1 day, that is, the first resource amount, the second resource amount, and the third resource amount are counted once a day, and the calculated ratio at this time represents the number of days to recover the cost, so the preset threshold value may be set to 10 days.

[0090] Through the above method, the embodiment of the present disclosure can calculate the amount of resources that can be expected to be saved by deploying the preset small model, as well as the training cost of the preset small model, and thereby obtain the time to recover the cost after deploying the preset small model. When the time is short, the preset small model can be deployed, and when the time is long, the preset small model may not be deployed. This realizes the analysis of the gap between the cost and benefit of deploying the preset small model from an economic perspective, and reduces the situation where the cost is far higher than the benefit after deploying the preset small model.

[0091] In some embodiments of the present disclosure, a management module that implements the large model management method provided by the embodiments of the present disclosure and a small model training module that trains a preset small model may run in the same or different devices.

[0092] When the management module and the small model training module run on different devices, the electronic device can also send a training request to the device where the small model training module is located, so that the device trains the preset small model based on the small model training module.

[0093] In the case where both the management module and the small model training module are running on the above electronic device, after determining to train the preset small model in the above S103, the electronic device can also train the preset small model. Figure 2 As shown, the process of training the preset small model includes the following steps:

[0094] S201: Acquire a first corpus, wherein the first corpus includes a plurality of corpora of preset categories.

[0095] The embodiment of the present disclosure may pre-classify each piece of corpus included in the initial corpus used to train the large language model, so the first corpus in S201 may include a plurality of corpora of preset categories in the initial corpus.

[0096] S202: Use the first corpus to train a preset neural network to obtain a preliminary model.

[0097] The preset neural network may be a neural network with a simple network layer structure and / or a simple network layer processing logic. For example, the preset neural network may be: a transformer, a bidirectional encoder representation from transformers (BERT), or a long short-term memory network (LSTM).

[0098] The electronic device can perform unsupervised training on the preset neural network based on the first corpus. The unsupervised training method of the preset neural network can refer to the description of unsupervised training in the above-mentioned method of training a large language model, which will not be repeated here.

[0099] S203. Obtain a second corpus, wherein the second corpus includes a plurality of historical questions of preset categories processed by the large language model and labels of the historical questions, and the labels include: processing results of the large language model on the historical questions and / or features extracted from the historical questions.

[0100] The disclosed embodiment uses the historical questions of the preset types processed by the large language model as the training samples of the preset neural network, and uses the processing results of the historical questions by the large language model and / or the features extracted from the historical questions as the labels of the training samples. Therefore, by training the preliminary model with the second corpus, the preliminary model can learn the processing effect of the large language model on the questions. Therefore, the large language model can be called the teacher model, the preliminary model can be called the student model, and the process of training the small model can also be called distillation.

[0101] The features extracted by the large language model for historical questions are: the output results of a specified intermediate network layer of the large language model, such as the output results of the last layer of the backbone network of the large language model.

[0102] S204: Use the second corpus to train the preliminary model, and use the trained preliminary model as a preset small model.

[0103] The electronic device can input the historical questions included in the second corpus into the preliminary model to obtain the features extracted by the preliminary model for the historical questions and / or the output results. Afterwards, the loss value is calculated based on the processing results of the large language model on the historical questions and the output results of the preliminary model on the historical questions; or, the loss value is calculated based on the features extracted by the large language model on the historical questions and the features extracted by the preliminary model on the historical questions; or, the loss value is calculated based on the processing results of the large language model on the historical questions and the output results of the preliminary model on the historical questions, and based on the features extracted by the large language model on the historical questions and the features extracted by the preliminary model on the historical questions. Then, the network parameters of the preliminary model are adjusted using the loss value until the preliminary model converges, and the trained preliminary model is used as the preset small model.

[0104] Through the above method, the embodiment of the present disclosure can first use the first corpus to pre-train the preset neural network. Since the pre-training can be carried out in an unsupervised training manner, it is not required that each corpus included in the first corpus has a label. Therefore, the acquisition cost of the first corpus is lower, and a large amount of corpus can be obtained, thereby improving the ability of the preliminary model obtained after pre-training to understand the text. Moreover, the embodiment of the present disclosure can also fine-tune the preliminary model after pre-training based on the second corpus, so that the processing results of the preliminary model are closer to the processing results of the large language model. This allows the preset small model after training to have the same processing effect as the large language model and reduce the resources consumed to process the problem.

[0105] When the management module and the small model training module run on different devices, the electronic device can also receive a deployment completion message sent by the device where the small model training module is located. At this time, the electronic device can determine that the preset small model has been trained and deployed.

[0106] When both the management module and the small model training module are running on the above electronic device, the electronic device can deploy the preset small model after the preset small model training is completed. The number of copies of the preset small model can be preset.

[0107] Since the small model can handle the preset types of problems after deployment, Figure 3 As shown, after the electronic device determines to train the preset small model in the above S103, it can also divert the user input questions:

[0108] S301, obtaining a pending question input by a user.

[0109] The specific implementation of S301 can refer to the relevant description in S101, which will not be repeated here.

[0110] S302: Use a classifier to determine the type of the problem to be processed.

[0111] For example, the classifiers that may be used in the embodiments of the present disclosure are: LSTM, Naive Bayes or Support Vector Machine (SVM), etc.

[0112] S303: Determine whether the type of the problem to be processed is a preset type. If yes, execute S304; if no, execute S305.

[0113] It should be noted that when the preset category includes multiple categories, that is, the preset small models corresponding to multiple categories have been trained and deployed, it can be determined whether the category to which the problem to be processed belongs belongs to multiple preset categories, that is, whether it is one of the multiple preset categories. If so, execute S304; if not, execute S305.

[0114] S304: Use the preset small model to process the problem to be processed and obtain the processing result of the problem to be processed.

[0115] When the problem to be processed belongs to a preset type, since the preset small model can handle the preset type of problem, and the resources consumed by the preset small model to process the preset type of problem are less than the resources consumed by the large model to process the problem, the preset small model can be used to process the problem to be processed and obtain the processing result of the problem to be processed.

[0116] When the preset small model is deployed in the electronic device where the above-mentioned management module is located, the electronic device can directly input the problem to be processed into the preset small model to obtain the processing result output by the preset small model.

[0117] When the preset small model is deployed in other devices other than the above electronic devices, the electronic device can send the problem to be processed to the other device, so that the other device inputs the problem to be processed into the preset small model and obtains the processing result output by the preset small model. Then the electronic device can receive the processing result sent by the other device.

[0118] S305: Use the large language model to process the problem to be processed and obtain a processing result for the problem to be processed.

[0119] When the category of the problem to be processed is not a preset category, since the preset small model cannot handle the problem of the preset category, the large language model can be used to process the problem to be processed to obtain the processing result of the problem to be processed.

[0120] When the large language model is deployed in the electronic device where the management module is located, the electronic device can directly input the problem to be processed into the large language model to obtain the processing result output by the large language model.

[0121] When the preset small model is deployed in other devices other than the above electronic devices, the electronic device can send the problem to be processed to the other device, so that the other device inputs the problem to be processed into the large language model and obtains the processing result output by the large language model. The electronic device can then receive the processing result sent by the other device.

[0122] Through the above method, the disclosed embodiment can select a large language model or a preset small model to process questions based on the type of questions input by the user, thereby realizing the diversion of different types of questions. The number of questions processed by the large language model is reduced, and the resources consumed by the large language model in processing questions are saved.

[0123] Since the preset small model can replace the large language model to handle preset types of problems after training and deployment, the pressure on the large language model to handle problems is reduced. Therefore, the electronic device can also reduce the number of copies of the large language model after the above S103.

[0124] The method for reducing the number of copies can be implemented as follows: determining the number of copies to be reduced according to the first resource amount and the third resource amount, and then reducing the number of copies from multiple copies of the large language model by the number of copies to be reduced.

[0125] The electronic device may calculate the difference between the first resource amount and the third resource amount, and obtain the fourth resource amount actually consumed by the multiple copies of the large language model to process multiple types of questions within a preset time period. Then, the ratio of the difference to the fourth resource amount is calculated, and the ratio is rounded as the number of copies reduced. Among them, the rounding up or rounding down method can be adopted.

[0126] The electronic device obtains the amount of resources actually consumed by each copy of the large language model to process all questions within a preset time period, and then takes an average value of the amount of resources consumed by each copy as the fourth amount of resources.

[0127] For example, the first resource volume is 10G, the third resource volume is 2G, the large language model has 10 copies, and the fourth resource volume corresponding to each copy is 3G. That is, two copies of the large language model can be reduced.

[0128] Since the difference between the first resource amount and the third resource amount can represent: the amount of resources that can be saved by deploying the preset small model within a period of time, and the fourth resource amount represents the amount of resources that a single copy of the large language model needs to consume within a period of time, the ratio of the difference to the fourth resource amount can represent the number of large language model copies that can be saved after deploying the preset small model. Therefore, by rounding the ratio as the number of copies reduced, the accuracy of determining the number of copies reduced can be improved, which not only reduces the problem of excessive processing pressure on each copy of the large language model due to too many reduced copies, but also reduces the problem of excessive overall resource consumption of each copy due to too few reduced copies.

[0129] Alternatively, when calculating the number of reduced copies, the electronic device may calculate the difference between the first resource amount and the third resource amount, determine the target range to which the difference belongs from various preset ranges, and obtain the number of reduced copies corresponding to the target range.

[0130] For example, the number of replica reductions corresponding to each preset range is shown in Table 1.

[0131] Table 1

[0132] scope Reducing the number of replicas [0G,1G) 1 [1G,2G) 2 [2G,∞] 3

[0133] Assume that the difference between the first resource amount and the third resource amount is 0.5G. Since 0.5G belongs to [0G, 1G), the number of replica reductions is determined to be 1.

[0134] Alternatively, the electronic device may also determine the number of reduced copies in other ways, which is not specifically limited in the embodiments of the present disclosure.

[0135] Since the preset small model can replace the large language model to handle the preset type of problems, it is equivalent to sharing the pressure of the large language model to handle the problems, so the copies of the large language model can be reduced. In addition, reducing the number of copies of the large language model can reduce the amount of resources allocated to the large language model and improve resource utilization.

[0136] Since the preset small model reduces the pressure of the large language model to process problems after training and deployment, the electronic device can also reduce the resources allocated to each copy of the large language model after the above S103.

[0137] The resource reduction method may be implemented as follows: calculating the difference between the first resource amount and the third resource amount, and then calculating the ratio between the difference and the number of copies of the large language model as the resource reduction amount. Then, reducing resources of the resource reduction amount from the resources allocated to the multiple copies of the large language model.

[0138] For example, the first resource amount is 8G, the third resource amount is 2G, and the large language model has 10 copies. At this time, the difference between the first resource amount and the third resource amount is 8-2=6G, and the resource reduction amount is 6 / 10=0.6G. Afterwards, for each copy of the large language model, 0.6G of resources can be reduced from the resources allocated to the copy.

[0139] Since the preset small model can replace the large language model to handle preset types of problems, it is equivalent to sharing the pressure of the large language model to handle problems. Therefore, the amount of resources for each copy of the large language model can be reduced, reducing the problem that the actual amount of resources used by each copy of the large language model is far less than the allocated amount of resources, thereby improving resource utilization.

[0140] The disclosed embodiment can be applied to the k8s system, in which case the management module, the large language model, the large language model training module, the preset small model and the small model training module can be in different pods, and each pod can share the same resource pool. Therefore, after reducing the copies of the large language model or reducing the resources allocated to each copy of the large language model, that is, after reducing the resources allocated to the pod where the large language model is located, the reduced resources in the resource pool can be used by other pods, for example, can be used by the small model training module, which improves resource utilization and reduces the time cost of training the small model.

[0141] See also Figure 4 The following describes the overall process of the large model management method provided by the embodiment of the present disclosure in combination with actual application scenarios:

[0142] The electronic device pre-uses the automatic distillation deployer to train a large language model based on the initial corpus and the fine-tuning corpus, and deploys the large language model.

[0143] The terminal receives the question input by the user and sends the question to the electronic device.

[0144] The electronic device determines the category to which the question belongs through a classifier, and determines whether the category is a preset category corresponding to at least one small model. If so, the question is input into the preset small model corresponding to the category to which the question belongs, and the processing result output by the small model is obtained, that is, the answer to the question is obtained. If not, the question is input into the large language model, and the processing result output by the large language model is obtained, that is, the answer to the question is obtained. Afterwards, the electronic device returns the answer to the terminal, and the terminal displays the answer to the user.

[0145] Every time the electronic device receives a question, it records the category of the question in the question database and counts the number of questions of each category received. Afterwards, when the electronic device detects that the number of questions processed by the large language model in the time period from a historical moment with a preset time length from the current moment to the current moment reaches a preset number, it determines the first amount of resources actually consumed by the large language model to process the question in the time period, obtains the second amount of resources that are expected to be consumed by the small model corresponding to the training of the question, and the third amount of resources that are expected to be consumed by the small model to process the question in the time period. Then, based on the first amount of resources, the second amount of resources, and the third amount of resources, the electronic device determines whether the preset small model training conditions are met; if so, a first corpus is constructed based on the initial corpus, and a second corpus is constructed based on the question database; the small model is trained based on the first corpus and the second corpus using the automatic distillation deployer, and the small model is deployed; if not, it is determined not to train the small model. Among them, Figure 4 The small models deployed in the example include small model a, small model b, and small model c.

[0146] At present, when staff find that there is a need to deploy a small model in a certain field, they will manually train and deploy the small model corresponding to the field. The embodiment of the present disclosure can automatically analyze the situation of each type of question according to the large language model, obtain the user's response requirements for different types of questions, and automatically train and deploy the preset small model corresponding to the question, reducing dependence on manual labor, thereby quickly responding to market demand. Moreover, since the embodiment of the present disclosure can determine whether to train the preset small model corresponding to a single type of question when the large language model processes a large number of questions within a period of time, that is, to perform an economic analysis on the deployment of the preset small model, it reduces the situation where the actual utilization rate of the preset small model is low after deployment.

[0147] Moreover, the disclosed embodiment performs an economic analysis on the deployment of the preset small model based on the training cost of the preset small model and the estimated resource savings, thereby ensuring the economy of resource investment and shortening the cost recovery time.

[0148] The above-mentioned process of manually training and deploying small models is manual, and the number of small models that can be deployed each time is limited. However, the disclosed embodiment realizes automatic and continuous determination of whether to train the preset small models corresponding to various categories, automatic training of preset small models, and automatic deployment of preset small models, which can continuously improve the efficiency of using large language models.

[0149] Moreover, currently the resources used for model training and the resources used for model deployment cannot be shared, but the embodiments of the present disclosure can be applied to the k8s system, so that the resources used in the training process and the deployment process can be shared, thereby improving resource utilization efficiency.

[0150] It can be seen that the embodiments of the present disclosure have the advantages of high resource utilization and fast demand analysis for various types of problems. Therefore, they can be widely used in various intelligent service scenarios, such as intelligent customer service, text translation, content creation and decision support.

[0151] In the technical solution of this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of the issues involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0152] Based on the same inventive concept, corresponding to the above method embodiment, the embodiment of the present disclosure also provides a large model management device, such as Figure 5 As shown, the device includes: an acquisition module 501, a determination module 502 and a judgment module 503;

[0153] An acquisition module 501 is used to acquire a first amount of resources actually consumed by a large language model in processing a preset type of target question within a preset time period;

[0154] The determination module 502 is further used to determine a second amount of resources that is expected to be consumed by training the preset small model, and a third amount of resources that is expected to be consumed by the preset small model to process the target problem within a preset time period, wherein the third amount of resources is less than the first amount of resources;

[0155] A judgment module 503 is used to determine whether a preset small model training condition is met based on the first resource amount, the second resource amount and the third resource amount;

[0156] The determination module 502 is also used to determine the training of the preset small model if the judgment result of the judgment module 503 is yes, wherein the trained preset small model is used to handle the preset type of problem.

[0157] In some embodiments of the present disclosure, the device further comprises:

[0158] The judgment module 503 is further configured to judge whether the number of target questions processed by the large language model within the preset time period reaches a preset number before obtaining the first amount of resources actually consumed by the large language model to process the target questions of the preset type within the preset time period;

[0159] The calling module is used to call the obtaining module 501 to execute the step of obtaining the first amount of resources actually consumed by the large language model to process a preset type of target question within a preset time period if the judgment result of the judging module 503 is yes.

[0160] In some embodiments of the present disclosure, the acquisition module 501 is specifically used to:

[0161] Obtain the total amount of resources actually consumed by the large language model to process multiple types of questions within a preset time period;

[0162] Determine the proportion of target questions among multiple types of questions;

[0163] Based on the total resource volume and the ratio, a first resource volume is determined.

[0164] In some embodiments of the present disclosure, the acquisition module 501 is specifically used to:

[0165] Obtain the total amount of resources actually consumed by the large language model to process multiple types of questions within a preset time period;

[0166] Determine the question length of the target question;

[0167] Based on the total resource amount and the problem length, a first resource amount is determined.

[0168] In some embodiments of the present disclosure, the determination module 503 is specifically configured to:

[0169] Calculating the difference between the first resource amount and the third resource amount;

[0170] Calculating the ratio of the second resource volume to the difference;

[0171] determining whether the ratio is less than or equal to a preset threshold;

[0172] If so, determine whether the preset small model training conditions are met;

[0173] If not, it is determined that the preset small model training conditions are not met.

[0174] In some embodiments of the present disclosure, the device further includes a training module, the training module being configured to:

[0175] After determining to train the preset small model, obtaining a first corpus, the first corpus including a plurality of corpora of preset types;

[0176] Using the first corpus to train the preset neural network to obtain a preliminary model;

[0177] Acquire a second corpus, the second corpus including a plurality of historical questions of preset categories processed by the large language model and labels of the historical questions, the labels including: processing results of the historical questions by the large language model and / or features extracted from the historical questions;

[0178] Use the second corpus to train the preliminary model and use the trained preliminary model as the preset small model

[0179] In some embodiments of the present disclosure, the device further comprises:

[0180] The acquisition module 501 is also used to acquire the problem to be processed input by the user after determining the preset small model for training;

[0181] A classification module, used to determine the category of the problem to be processed by using a classifier;

[0182] A processing module, used for processing the problem to be processed by using a preset small model if the type of the problem to be processed belongs to a preset type, and obtaining a processing result of the problem to be processed;

[0183] The processing module is also used to use the large language model to process the problem to be processed if the category of the problem to be processed is not a preset category, so as to obtain a processing result of the problem to be processed.

[0184] In some embodiments of the present disclosure, the device further comprises:

[0185] The determination module 502 is further used to determine the number of replica reductions according to the first resource amount and the third resource amount after determining to train the preset small model;

[0186] The reduction module is used to reduce the number of copies from multiple copies of a large language model.

[0187] In some embodiments of the present disclosure, the determining module 502 is specifically configured to:

[0188] Calculating the difference between the first resource amount and the third resource amount;

[0189] Obtaining a fourth amount of resources actually consumed on average by multiple copies of the large language model in processing multiple types of questions within a preset time period;

[0190] The ratio of the difference to the fourth resource amount is calculated, and the ratio is rounded to an integer to obtain the reduced number of replicas.

[0191] In some embodiments of the present disclosure, the device further comprises:

[0192] A calculation module, used for calculating the difference between the first resource amount and the third resource amount after determining the preset small model for training;

[0193] The calculation module is also used to calculate the ratio between the difference and the number of copies of the large language model as the resource reduction;

[0194] The reduction module is used to respectively reduce resources of resource reduction amounts from resources allocated to multiple copies of the large language model.

[0195] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0196] Figure 6A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0197] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0198] Multiple components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0199] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the large model management method. For example, in some embodiments, the large model management method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the large model management method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the large model management method in any other appropriate manner (e.g., by means of firmware).

[0200] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0201] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0202] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0203] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0204] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0205] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0206] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0207] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A large model management method, comprising: Obtaining a first amount of resources actually consumed by the large language model in processing a preset type of target question within a preset time period; Determine a second amount of resources that is expected to be consumed by training a preset small model, and a third amount of resources that is expected to be consumed by the preset small model in processing the target problem within the preset time period, wherein the third amount of resources is less than the first amount of resources; Based on the first resource amount, the second resource amount and the third resource amount, determining whether a preset small model training condition is met; If so, determine to train the preset small model, wherein the trained preset small model is used to process the preset type of problem.

2. The method according to claim 1, before obtaining the first amount of resources actually consumed by the large language model in processing a preset type of target question within a preset time period, further comprising: Determining whether the number of the target questions processed by the large language model within the preset time period reaches a preset number; If so, the step of obtaining the first amount of resources actually consumed by the large language model to process a preset type of target question within a preset time period is executed.

3. The method according to claim 1, wherein: The obtaining of a first amount of resources actually consumed by the large language model in processing a preset type of target question within a preset time period includes: Obtaining the total amount of resources actually consumed by the large language model in processing multiple types of questions within the preset time period; Determining the proportion of the target problems among the multiple types of problems; Based on the total resource amount and the ratio, the first resource amount is determined.

4. The method according to claim 1, wherein: The obtaining of a first amount of resources actually consumed by the large language model in processing a preset type of target question within a preset time period includes: Obtaining the total amount of resources actually consumed by the large language model in processing multiple types of questions within the preset time period; Determining the question length of the target question; The first resource amount is determined based on the total resource amount and the question length.

5. The method according to any one of claims 1 to 4, wherein: The determining whether a preset small model training condition is met based on the first resource amount, the second resource amount, and the third resource amount includes: Calculating a difference between the first resource amount and the third resource amount; Calculating a ratio of the second resource amount to the difference; determining whether the ratio is less than or equal to a preset threshold; If so, determining whether the preset small model training condition is met; If not, it is determined that the preset small model training condition is not met.

6. The method according to any one of claims 1 to 5, after determining to train the preset small model, further comprising: Acquire a first corpus, wherein the first corpus includes a plurality of corpora of the preset type; Using the first corpus to train the preset neural network to obtain a preliminary model; Acquire a second corpus, the second corpus including a plurality of historical questions of the preset type processed by the large language model and labels of the historical questions, the labels including: processing results of the historical questions by the large language model and / or features extracted from the historical questions; The preliminary model is trained using the second corpus, and the trained preliminary model is used as the preset small model.

7. The method according to any one of claims 1 to 6, after determining to train the preset small model, further comprising: Get pending questions input by the user; Using a classifier to determine the type of the problem to be processed; If the type of the problem to be processed is the preset type, the preset small model is used to process the problem to be processed to obtain a processing result of the problem to be processed; If the category of the problem to be processed is not the preset category, the problem to be processed is processed using the large language model to obtain a processing result of the problem to be processed.

8. The method according to any one of claims 1 to 7, after determining to train the preset small model, further comprising: Determine a number of replica reductions according to the first resource amount and the third resource amount; The number of copies of the large language model is reduced by reducing the number of copies from the multiple copies.

9. The method according to claim 8, wherein: The step of determining the number of replica reductions according to the first resource amount and the third resource amount includes: Calculating a difference between the first resource amount and the third resource amount; Obtaining a fourth amount of resources actually consumed on average by the multiple copies of the large language model in processing multiple types of questions within the preset time period; The ratio of the difference to the fourth resource amount is calculated, and the ratio is rounded to an integer to obtain the integer as the number of replica reductions.

10. The method according to any one of claims 1 to 7, after determining to train the preset small model, further comprising: Calculating a difference between the first resource amount and the third resource amount; Calculating a ratio between the difference and the number of copies of the large language model as a resource reduction amount; The resources of the resource reduction amount are respectively reduced from the resources allocated to the multiple copies of the large language model.

11. A large model management device, comprising: An acquisition module, used to acquire a first amount of resources actually consumed by the large language model in processing a preset type of target question within a preset time period; The determination module is further used to determine a second amount of resources that is expected to be consumed by training a preset small model, and a third amount of resources that is expected to be consumed by the preset small model to process the target problem within the preset time period, wherein the third amount of resources is less than the first amount of resources; A judgment module, used to determine whether a preset small model training condition is met based on the first resource amount, the second resource amount and the third resource amount; The determination module is also used to determine the training of the preset small model if the judgment result of the judgment module is yes, wherein the trained preset small model is used to handle the problem of the preset type.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

14. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images