Decision-making method based on collaborative optimization of big and small language models, computer system and computer readable storage medium

Abstract

The invention belongs to the field of artificial intelligence decision-making systems, and particularly relates to a decision-making method based on collaborative optimization of large and small language models, a computer system and a computer readable storage medium. According to the method, for a request input by a user to an overall request processing module in a decision model, whether the request belongs to a simple request or a difficult request is judged through a pre-trained routing model in the overall request processing module, and if the request is the simple request, a small large language model processes the request to obtain a decision result; if the request is a difficult request, processing the request by a relatively large language model to obtain a decision result; the routing model comprises a primary shunt routing model obtained by pre-training the training data comprising the labels corresponding to the simple request and the difficult request. In this way, calling of a large language model can be reduced in the mode that a small language model is called as much as possible through a simple request, so that the system operation cost can be effectively reduced, and the system response speed is increased.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence decision-making systems, and specifically relates to a decision-making method based on collaborative optimization of large and small language models, a computer system, and a computer-readable storage medium. Background Art

[0002] Intelligent decision-making is one of the most important application technologies in the field of artificial intelligence. It not only provides convenience for people's lives, but also greatly reduces the consumption of human resources. With the advent of the era of large models, large language models (LLMs) with powerful understanding and generation capabilities have brought new changes to intelligent decision-making systems.

[0003] LLM stores a large amount of pre-trained knowledge in numerous parameters, and then outputs decisions based on the questions input by the user to complete the question-answering dialogue with the user. However, when users ask questions about certain specific areas, large language models often produce "hallucinations" and give meaningless answers. In order to solve this problem, a retrieval-augmented generation (RAG) technology for language models has been widely used; for example, Figure 1 As shown in the figure, RAG aims to obtain corresponding matching knowledge from the knowledge base according to the user input request, and pass it together with the user request as input to the large model. By adding contextual knowledge about specific requests, LLM will give more specific and reasonable responses.

[0004] However, existing LLM-based intelligent decision-making systems often have the following problems:

[0005] 1) Larger language models (Larger LLM, L-LLM) have too large a parameter scale, making them difficult to deploy locally and slow to infer. Moreover, powerful L-LLMs are usually closed-source models, and their services can only be called through paid interfaces. Specifically, although L-LLM has powerful understanding and generation capabilities and can give reasonable responses to user requests, the huge parameter scale of L-LLM makes it difficult to deploy locally, and calling through interfaces will bring huge additional costs; taking GPT3.5 as an example, its parameter scale is 175B, requiring 320GB of storage space, that is, 5 A100 GPUs to run. The excessive computing power requirements are unacceptable to most product teams. In addition, due to the commercial value of L-LLMs such as GPT3.5 and GLM4, related development companies often adopt a closed-source strategy, which means that the intelligent decision-making system development team can only call L-LLM through paid interfaces to obtain corresponding services; frequent interface calls will bring immeasurable operating costs to related products. Taking GPT3.5 as an example, its interface call fee is 0.0144 yuan per thousand characters. If L-LLM is fine-tuned, the fee will increase exponentially.

[0006] 2) Smaller Large Language Models (S-LLMs), although fast inference speed and open source and free, have limited performance and cannot meet user needs; specifically, S-LLMs, such as open source models such as GLM-4-9B and LLaMA-7B, have relatively small parameter scales, which allows them to be deployed on more civilian terminal devices; taking GLM-4-9B as an example, an RTX 4090Ti with a video memory capacity of 24GB can run or even fine-tune the S-LLM; however, even though S-LLMs have the advantages of easy deployment and open source and free, their poor performance still makes it difficult for S-LLMs to meet the needs of users in decision-making systems. Summary of the invention

[0007] The object of the present invention is to provide a decision-making method, a computer system and a computer-readable storage medium based on the collaborative optimization of large and small language models, so as to solve the problem that the decision-making method using a larger language model in the prior art is costly and time-consuming, while the decision-making method using a smaller language model is less effective.

[0008] In order to achieve the above object, the present invention provides a decision-making method based on the collaborative optimization of large and small language models. For a request of the overall request processing module in the user input decision model, the pre-trained routing model in the overall request processing module is used to determine whether the request is a simple request or a difficult request; if it is a simple request, the smaller large language model processes the request to obtain a decision result; if it is a difficult request, the larger large language model processes the request to obtain a decision result;

[0009] The routing model includes a primary traffic diversion routing model obtained by pre-training using training data including labels corresponding to simple requests and difficult requests.

[0010] Further, the smaller large language model includes a fine-tuned smaller large language model obtained by fine-tuning the learnable smaller large language model using an output obtained by inputting a simple request to a preset smaller large language model, an output obtained by inputting a simple request to a preset larger large language model, and an output obtained by inputting a difficult request to a preset larger large language model, and a smaller large language model that has not been fine-tuned;

[0011] The routing model further includes a secondary diversion routing model obtained by pre-training with data obtained by fine-tuning the learnable smaller large language model, and the secondary diversion routing model is used to further determine whether the difficult request determined by the primary diversion routing model is a simple request or a difficult request;

[0012] If it is a simple request, the smaller large language model processes the request to obtain a decision result; if it is a difficult request, the larger large language model processes the request to obtain a decision result. The method includes: the unfine-tuned smaller large language model processes the simple request determined by the primary diversion routing model to obtain a decision result; the fine-tuned smaller large language model is used to process the simple request determined by the secondary diversion routing model to obtain a decision result; the larger large language model processes the difficult request determined by the secondary diversion routing model to obtain a decision result.

[0013] Furthermore, the method for fine-tuning the learnable smaller large language model to obtain data includes: in the training process of fine-tuning the learnable smaller large language model, inputting the user request into the learnable smaller large language model in each iteration to obtain an output result; if the output result is accurate, labeling the output result with a label of a simple request; if it is inaccurate, labeling the output result with a label of a difficult request; and using the output results and labels of a set number of times as the data obtained for fine-tuning.

[0014] Furthermore, for the request of the fragment request processing module in the user input decision model, the original knowledge matching the request is first found in the knowledge base, and then the redundant information in the original knowledge matching the request is eliminated according to the request through the smaller large language model in the fragment request processing module. The original knowledge matching the request after the redundant information is eliminated is input into the larger large language model for processing to obtain the decision result of the request.

[0015] Furthermore, the LoRA large model fine-tuning algorithm is used to fine-tune the learnable smaller large language model by utilizing the output obtained by inputting a simple request to the preset smaller large language model, the output obtained by inputting a simple request to the preset larger large language model, and the output obtained by inputting a difficult request to the preset larger large language model.

[0016] Further, the un-fine-tuned smaller large language model processes the simple request determined by the primary diversion routing model to obtain a decision result; the fine-tuned smaller large language model is used to process the simple request determined by the secondary diversion routing model to obtain a decision result; the larger large language model processes the difficult request determined by the secondary diversion routing model to obtain a decision result in a manner including: after preliminarily determining whether the request is a simple request or a difficult request through the primary diversion routing model, matching the request with the knowledge in the knowledge base to obtain a pre-set number of documents with the highest prior probability; if the primary diversion routing model If the request is judged to be a simple request by the first diversion routing model, the set number of documents obtained and the request are combined and processed by the smaller large language model that has not been fine-tuned to obtain a decision result; the difficult request judged by the first diversion routing model is further judged to be a simple request or a difficult request by the second diversion routing model; if it is judged to be a simple request, the set number of documents obtained and the request are combined and processed by the smaller large language model after fine-tuning to obtain a decision result; if it is judged to be a difficult request, the set number of documents obtained and the request are combined and processed by the larger large language model to obtain a decision result.

[0017] Furthermore, matching the request with the knowledge in the knowledge base is accomplished using an embedding model.

[0018] Furthermore, the routing model is a BERT network.

[0019] The above-mentioned technical scheme of the present invention provides a new decision-making method based on the collaborative optimization of large and small language models, and its beneficial effects include: when facing the overall request, it can divert simple requests and hand over the simple requests to a smaller large language model for processing, so that these simple requests can be answered relatively accurately through the smaller large language model, and since the call of the smaller large language model is free of charge, it can avoid that all requests are processed by the larger large language model, resulting in high call costs and long time consumption; at the same time, the diverted difficult requests are handed over to the larger large language model for processing, which can make full use of the better processing performance of the larger large language model in the face of more complex problems; in general, it is to reduce the call to the paid L-LLM by calling the smaller large language model as much as possible through simple requests, thereby effectively reducing the system operating cost and improving the system response speed, while ensuring that the processing quality can be guaranteed by calling the L-LLM when encountering complex requests.

[0020] The present invention also provides a computer system, including a processor, wherein the processor stores executable program instructions, and the executable program instructions are used to be executed to implement the above-mentioned decision-making method based on collaborative optimization of large and small language models.

[0021] The technical solution of the computer system of the present invention can achieve the same beneficial effects as the above-mentioned decision-making method based on collaborative optimization of large and small language models.

[0022] The present invention also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed, the decision-making method based on collaborative optimization of large and small language models as described above is implemented.

[0023] The technical solution of the above-mentioned computer-readable storage medium of the present invention can achieve the same beneficial effects as the above-mentioned decision-making method based on collaborative optimization of large and small language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a principle example diagram of the RAG method in the background technology of the present invention;

[0025] Figure 2 This is a diagram showing the principle of fine-tuning a learnable smaller large language model through LoRA in an embodiment of a decision-making method based on collaborative optimization of large and small language models of the present invention;

[0026] Figure 3 It is a principle example diagram of a decision method based on collaborative optimization of large and small language models in an embodiment of the decision method based on collaborative optimization of large and small language models of the present invention;

[0027] Figure 4This is a diagram showing an example of the principle of obtaining a decision result of an input request by a fragment request processing module in an embodiment of a decision method based on collaborative optimization of large and small language models of the present invention;

[0028] Figure 5a This is an example diagram of the relationship between the accuracy of the decision result and the diversion threshold in an embodiment of the decision method based on collaborative optimization of large and small language models of the present invention;

[0029] Figure 5b This is an example diagram of the relationship between the average response time and the diversion threshold in an embodiment of the decision-making method based on collaborative optimization of large and small language models of the present invention;

[0030] Figure 5c This is an example diagram of the relationship between the L-LLM processing request ratio and the diversion threshold in an embodiment of the decision-making method based on collaborative optimization of large and small language models of the present invention;

[0031] Figure 6a It is a schematic diagram of the proportion of L-LLM processing requests of the open source framework Chatchat (L-LLM) based on RAG, RAGFlow (S-LLM) and the intelligent decision-making experiment results of the decision-making method based on the collaborative optimization of large and small language models in the scientific research decision-making scenario in the embodiment of the decision-making method based on the collaborative optimization of large and small language models of the present invention;

[0032] Figure 6b A schematic diagram of the number of LLM input characters of Chatchat (L-LLM), RAGFlow (S-LLM), an open source framework based on RAG, and an intelligent decision-making experiment result of a decision-making method based on collaborative optimization of large and small language models in a scientific research decision-making scenario in an embodiment of the decision-making method based on collaborative optimization of large and small language models of the present invention;

[0033] Figure 6c It is a schematic diagram of the feedback accuracy of the intelligent decision-making experimental results of the RAG-based open source framework Chatchat (L-LLM), RAGFlow (S-LLM) and the decision-making method based on the collaborative optimization of large and small language models in the scientific research decision-making scenario in the embodiment of the decision-making method based on the collaborative optimization of large and small language models of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0035] Implementation of decision-making method based on collaborative optimization of large and small language models

[0036] This embodiment provides a technical solution for a decision-making method based on collaborative optimization of large and small language models. The method completes the diversion prediction of user requests (difficult requests and simple requests) through a routing model, so that L-LLM and S-LLM can process their own appropriate user requests respectively, thereby reducing the decision-making system's dependence on L-LLM, that is, by calling S-LLM as much as possible through simple requests, reducing the call to the paid L-LLM, thereby effectively reducing the system operating cost and improving the system response speed.

[0037] The decision-making method specifically includes the following steps:

[0038] For the request of the overall request processing module in the user input decision model, the pre-trained routing model in the overall request processing module is used to determine whether the input request is a simple request or a difficult request; if it is a simple request, the smaller large language model processes the request to obtain a decision result; if it is a difficult request, the larger large language model processes the request to obtain a decision result;

[0039] The pre-trained routing model in the overall request processing module includes a one-step routing model that is pre-trained using training data containing labels corresponding to simple requests and difficult requests.

[0040] Therefore, when facing the overall request, the decision-making method based on collaborative optimization of large and small language models in this embodiment can divert simple requests and hand them over to the smaller large language model for processing, so that these simple requests can be answered relatively accurately through the smaller large language model. Since the call of the smaller large language model is free of charge, it can avoid the problem that all requests are processed by the larger large language model, resulting in high calling costs and long time consumption; at the same time, the diverted difficult requests are handed over to the larger large language model for processing, which can make full use of the better processing performance of the larger large language model in the face of more complex problems; in general, it is to reduce the call to the paid L-LLM by calling the smaller large language model as much as possible through simple requests, thereby effectively reducing the system operating cost and improving the system response speed, while ensuring that the processing quality can be guaranteed by calling the L-LLM when encountering complex requests.

[0041] On the basis of diverting simple and difficult requests, in this embodiment, the distillation technology is also used to transfer the ability of L-LLM to process difficult requests to S-LLM, so as to further reduce the dependence of the decision method on L-LLM. In this case, the S-LLM processed by the distillation technology can be used to process some difficult requests judged by the one-time diversion routing model; considering that directly using traditional knowledge distillation technology to make S-LLM imitate the output of L-LLM will lead to catastrophic forgetting and make S-LLM lose some general capabilities acquired in the training stage, therefore, in this embodiment, the smaller large language model includes the output obtained by inputting simple requests to the preset smaller large language model, the output obtained by inputting simple requests to the preset larger large language model, and the output obtained by inputting difficult requests to the preset larger large language model, and the fine-tuned smaller large language model obtained by fine-tuning the learnable smaller large language model, and also includes an ordinary S-LLM (i.e., S-LLM that has not been fine-tuned as mentioned above) for processing simple requests judged by the one-time diversion routing model. In this embodiment, the fine-tuned (trained) smaller large language model obtained by fine-tuning the learnable smaller large language model (Learnable S-LLM, LS-LLM) by LoRA (Low-Rank Adaptation of Large Language Models) requires fewer data samples for training than other large model fine-tuning methods, so the cost is lower. The specific expression is as follows:

[0042] L=LoRA LS (A L ,q h )+LoRA LS (A S ,i e )+LoRA LS (A L ,q e )

[0043] Among them, q h Indicates the difficulty of input request, q e Represents a simple request for input; LoRA LS It represents an algorithm for fine-tuning a learnable smaller language model through LoRA. The specific content belongs to the prior art and will not be described here. L Indicates the output obtained by inputting a request to a preset larger language model; A SRepresents the output obtained by inputting a request to a preset smaller large language model; L represents the loss function Loss. This expression actually refers to the goal of achieving the goal of fitting the model parameters to the target data set by reducing L, that is, to achieve the training iteration corresponding to fine-tuning the learnable smaller large language model. The goal of fine-tuning the learnable smaller large language model in each iteration is to reduce the loss function corresponding to the fine-tuning. In other embodiments, the training corresponding to fine-tuning the learned smaller large language model can also be performed by large model fine-tuning methods such as P-tuning. The specific training methods are all prior art and will not be repeated here.

[0044] By using the output obtained by inputting simple requests to the preset smaller large language model, the output obtained by inputting simple requests to the preset larger large language model, and the output obtained by inputting difficult requests to the preset larger large language model, fine-tuning the LS-LLM is equivalent to a multi-source distillation algorithm, which can ensure that the smaller large language model retains the general capabilities acquired during the training process while acquiring new capabilities, so that the LS-LLM can acquire the ability of L-LLM to handle difficult requests, and retain the general capabilities of the smaller large language model, so as not to cause catastrophic forgetting.

[0045] At the same time, on the basis of fine-tuning the LS-LLM, in order to achieve the effect that the fine-tuned LS-LLM undertakes more decision-making work for difficult requests, the pre-trained routing model in the overall request processing module of this embodiment also includes a secondary diversion routing model obtained by pre-training the primary diversion routing model using data obtained by fine-tuning the above-mentioned learnable smaller large language model. The secondary diversion routing model is used to further judge whether the difficult requests determined by the primary diversion routing model are simple requests or difficult requests.

[0046] Reference Figure 2 , the method of fine-tuning the above-mentioned learnable smaller large language model through LoRA to obtain data includes: in the training process of fine-tuning the learnable smaller large language model through LoRA, each iteration inputs the user request into the learnable smaller large language model to obtain the output result; if the output result is accurate, the output result is labeled with a simple request label; if it is inaccurate, the output result is labeled with a difficult request label; the output result and label of the set number of times are used as the data obtained for fine-tuning. Therefore, through the secondary diversion routing model, a part of the requests suitable for fine-tuning LS-LLM processing are diverted from the difficult requests judged by the primary diversion routing model as simple requests, so that more problems can be diverted to LS-LLM on the basis of fine-tuning LS-LLM, thereby further reducing the decision-making system's dependence on L-LLM and reducing the call cost.

[0047] Based on the above-mentioned settings of the primary diversion routing model, the secondary diversion routing model, the L-LLM, the LS-LLM and the S-LLM, in this embodiment, if it is a simple request, the smaller large language model processes the request to obtain a decision result, and if it is a difficult request, the larger large language model processes the request to obtain a decision result. The method includes: the unfine-tuned S-LLM processes the simple request determined by the primary diversion routing model to obtain a decision result; the fine-tuned LS-LLM is used to process the simple request determined by the secondary diversion routing model to obtain a decision result; and the L-LLM processes the difficult request determined by the secondary diversion routing model to obtain a decision result. Specifically, in the process of building the overall request processing module, it is necessary to first train the learnable smaller large language model (LS-LLM) according to the training samples, that is, first perform the above-mentioned process of fine-tuning the LS-LLM, and accumulate data in the training process (that is, fine-tune the learnable smaller large language model to obtain data); after obtaining enough data in the training process, the pre-training of the secondary diversion routing model can be independently realized without affecting the process of fine-tuning the LS-LLM; after completing the fine-tuning of the LS-LLM, it is equivalent to completing the training of the LS-LLM, and the overall request processing can be entered at this time. During the testing or application phase of the request processing module, the pre-trained secondary diversion routing model is added to the overall request processing module to form three processing paths of the overall request processing module: "user request → primary diversion routing model (determined as a difficult request) → secondary diversion routing model (determined as a simple request) → fine-tuned LS-LLM, user request → primary diversion routing model (determined as a difficult request) → secondary diversion routing model (determined as a difficult request) → L-LLM, user request → primary diversion routing model (determined as a simple request) → unfine-tuned S-LLM" to test or apply the overall request processing module.

[0048] In this embodiment, the routing model (including the primary shunt routing model and the secondary shunt routing model) is a BERT network; in other embodiments, other pre-trained binary classification models may also be used. Alternatively, in other embodiments, if the processing volume of L-LLM is not considered to further save costs, and the overall network architecture of the overall request processing module is only simplified, the above-mentioned secondary shunt routing model may not be used, and the judgment results of the simple request and the difficult request determined by the primary shunt routing model may be directly used as the basis for whether the request is processed by a smaller large language model or a smaller large language model to obtain a decision result. In this case, there is no need to use the fine-tuned LS-LLM to process the simple request diverted from the difficult request determined by the primary shunt routing model by the secondary shunt routing model, so only two processing paths of the overall request processing module need to be formed: "user request → primary shunt routing model (determined as a difficult request) → L-LLM, user request → primary shunt routing model (determined as a simple request) → S-LLM that has not been fine-tuned".

[0049] Considering that the existing large language models can often assist in answering questions by matching knowledge with professional knowledge bases, Figure 3 In the two parts ① and ② of the present invention, the unfine-tuned smaller large language model processes the simple request determined by the primary diversion routing model to obtain a decision result; the fine-tuned smaller large language model is used to process the simple request determined by the secondary diversion routing model to obtain a decision result; the larger large language model processes the difficult request determined by the secondary diversion routing model to obtain a decision result in the following manner: after the primary diversion routing model preliminarily determines whether the input request is a simple request or a difficult request, the input request is matched with the knowledge in the knowledge base to obtain a pre-set number of documents with the highest prior probability; if the primary diversion routing model is If the flow routing model determines that it is a simple request, the set number of documents obtained and the request are combined and processed by a smaller large language model that has not been fine-tuned to obtain a decision result; the difficult request determined by the primary diversion routing model is further judged as a simple request or a difficult request by the secondary diversion routing model; if it is determined to be a simple request, the set number of documents obtained and the request are combined and processed by a smaller large language model that has been fine-tuned to obtain a decision result; if it is determined to be a difficult request, the set number of documents obtained and the request are combined and processed by a larger large language model to obtain a decision result. In other embodiments, the input request may not be matched with the knowledge in the knowledge base, and the input request may be directly input into the corresponding L-LLM, LS-LLM or S-LLM according to its classification result (difficult request or simple request) to obtain the output result.

[0050] If the judgment results of simple requests and difficult requests determined by the one-time diversion routing model are directly used as the basis for whether the smaller large language model or the smaller large language model is used to process the request to obtain a decision result, the difficult request and the content of a pre-set number of documents with the highest prior probability obtained by matching the knowledge in the knowledge base are directly input into the L-LLM, and the simple request and the content of a pre-set number of documents with the highest prior probability obtained by matching the knowledge in the knowledge base are input into the S-LLM that has not been fine-tuned, and the decision results are obtained accordingly.

[0051] In addition, in this embodiment, the matching between the request and the knowledge in the knowledge base is completed using the embedding model:

[0052] p(I|q)∝exp(F E (z) T F E (q))

[0053] Among them, z is the knowledge in the knowledge base, F E represents the embedding model; T is the matrix transposition operator; q represents the user request; exp represents the exponential with the natural constant e as the base. According to this formula, the first K documents with the highest prior probability p(I|q) can be obtained. K =[I1,I2,…,I k ]; In other embodiments, other existing matching methods may also be used, which will not be described in detail here.

[0054] Due to factors such as the LLM architecture, the LLM input cannot be too long. However, in the RAG architecture, when a user request involves more knowledge, it is often necessary to match multiple pieces of knowledge in the knowledge base and then use the matched knowledge to input the LLM to get an answer. This situation can easily cause the length of the context input to the LLM to exceed the upper limit of the LLM input. Therefore, referring to Figure 4 In this embodiment, a fragment request processing module is also provided for selection by the user; for the request of the fragment request processing module in the user input decision model, the original knowledge matching the request is first found in the knowledge base, and then the redundant information in the original knowledge matching the request is removed according to the request through the smaller large language model in the fragment request processing module, and the original knowledge matching the request after the redundant information is removed is input into the larger large language model for processing to obtain the decision result of the request. For fragment information request (the request of the fragment request processing module in the user input decision model)q s and an original matching knowledge I C ,The input and output of S-LLM in the fragment request processing module are as follows:

[0055] p S(I F |I C ,q s )=∑ c p S (y c |I C ,q s ,y 1:c-1 ),y c ∈I C

[0056] Among them, p S S-LLM in the fragment request processing module; c is the next predicted character, I F is the matching knowledge after removing redundant information; y 1:c-1 represents all characters from 1 to c-1. From the above formula, we can see that after converting the generation task into a elimination task, the predicted characters all come from the original knowledge I C Therefore, the subsequent character prediction scope of S-LLM will change from the character list of the large model to the original knowledge I C This will greatly reduce the prediction range. It can be seen that the smaller large language model in the fragment request processing module can realize information pruning (IP). IP inputs multiple retrieval knowledge to the S-LLM in the fragment request processing module respectively, and converts the generation task into a deletion task, so that the S-LLM can remove noise information according to the user request when it has global information, thereby reducing the number of L-LLM input characters and enabling L-LLM to better focus on useful content, so that L-LLM can process user fragmented information requests, solving the problem that fragmented information cannot be effectively used.

[0057] This embodiment further verifies the effectiveness of the above-mentioned fragment request processing module by using the Lagrange multiplier method. The specific process is as follows:

[0058] Assume that the number of different characters in the large language model character list is M, and the number of different characters in the original knowledge is N; for the generation task, the entropy H of predicting the next character is G for:

[0059]

[0060] Among them, t c is a character in the character list; c represents the cth character, p S Produce the corresponding character probabilities for the smaller large language model.

[0061] Using the Lagrange multiplier method we get:

[0062]

[0063] For p in the above formula S (t c ) and α, we can get:

[0064]

[0065] Let the above equation be 0, and we can get:

[0066]

[0067] That is H G The maximum value of is logM; when the elimination task is performed, the number of candidate characters changes from M to N; at this time, the entropy H of the next character is predicted IP =logN; Since M>>N and logM>>logN, it can be seen that the entropy of the eliminated task will be much smaller than the entropy of the generated task, that is, H G >>H IP The above results show that the uncertainty of the elimination task is much smaller than that of the generation task, so the lower limit of the elimination task is much higher than that of the generation task. Therefore, when using S-LLM to eliminate redundant information, the effect will be more guaranteed.

[0068] This embodiment first verifies the effectiveness of the overall request processing module of this embodiment in classifying and processing requests on a natural language inference task (NLI), where the goal of the NLI task is to use the model to determine the relationship between "premise" and "assumption", namely "contradiction", "neutrality" and "inclusion".

[0069] The experiment of this embodiment is based on the public dataset Chaos NLI, with a total of 1599 data, of which 1399 are training data and 200 are test data; the L-LLM used in the experiment of this embodiment is GLM-4, and the S-LLM and LS-LLM used are GLM-4-9B-Chat. In addition, the method for fine-tuning LS-LLM is LoRA, and the learning rate is 2e -6 , the number of iterations is 4, the number of gradient accumulation steps is 4, and the temperature coefficient is 0.7.

[0070] Table 1 shows the results of different large language models on the NLI task. It can be seen that although S-LLM has fast inference speed and is free, its accuracy is only 61.00%; in contrast, although L-LLM has improved its response accuracy by 7%, its average response speed is 19.75 times that of S-LLM. The decision-making method based on collaborative optimization of large and small language models (LS-QA for short) in this embodiment fully combines the advantages of the two. By request diversion, S-LLM processes simple requests and L-LLM processes difficult requests, so as to achieve the goal of reducing costs and increasing efficiency. At the same time, the results of LS-QA are better than L-LLM, which shows that even if the overall performance of L-LLM far exceeds that of S-LLM, S-LLM can still be comparable to or even surpass L-LLM in some data distributions. Furthermore, this embodiment also shows the difference in results of using only the one-way split routing model, the one-way split routing model and the two-way split routing model in Table 1. It can be seen that by fine-tuning the LS-LLM, some difficult requests can be effectively transformed into simple requests that can be processed by the S-LLM, thereby further reducing the number of calls to the L-LLM and speeding up the average response time of the decision system. In addition, the fine-tuned S-LLM fits better on Chaos NLI, thereby increasing the possibility of its correct response each time.

[0071] Table 1

[0072] Shunt Threshold ( Specifically, it is a manually set parameter, which appears in the primary diversion routing model and the secondary diversion routing model respectively. In essence, the output of the routing model is a probability, that is, the probability that S-LLM can correctly process the corresponding request. If the routing model determines that the input request value is greater than this value, it will be processed by S-LLM or LS-LLM after fine-tuning, otherwise it will be processed by L-LLM. It determines the flow direction of user requests and is the most important hyperparameter in LS-QA. For simplicity, this embodiment keeps the two diversions. same. Figure 5a-5c The relationship between the decision result accuracy, average response time, and L-LLM processing request ratio and the diversion threshold is shown; Figure 5a (where the horizontal axis is the diversion threshold and the vertical axis is the decision result accuracy) shows that as Increase, the overall system performance steadily improves, however, when , the performance has decreased; this shows that for S-LLM, the confidence level of most requests that can be correctly processed is around 0.9; therefore, when If the size continues to increase, the overall system will lose the ability to correctly handle some of the distributions that S-LLM can handle. Figure 5bThe average response time is shown (where the horizontal axis is the diversion threshold and the vertical axis is the average response time). It can be seen that as As the size of the request increases, more and more requests will be processed by L-LLM, so the average time increases steadily. Figure 5c It shows The relationship with the proportion of L-LLM processing requests (where the horizontal axis is the diversion threshold and the vertical axis is the proportion of L-LLM processing requests). The higher the proportion of L-LLM processing requests, the higher the cost of calling L-LLM. This process is consistent with the average response time. Figure 5a-5c illustrate, The choice of is not the bigger the better, but should be to choose the one that can maximize the effectiveness of S-LLM on a specific distribution.

[0073] In order to verify the effectiveness of LS-QA in real-world scenarios, this embodiment tests and analyzes the proposed framework based on scientific research decision-making scenarios. Specifically, L-LLM is GLM-4, and S-LLM is ChatGLM3-6b; this embodiment uses GPT4-o to generate 1024 questions about specific content of papers based on public papers, and uses GPT-4o to generate 1024 broad questions based on information from different disciplines. In order to train the routing model and test the performance of LS-QA in this embodiment, 1638 samples are used as training sets, 204 samples are used as cross-validation sets, and 206 samples are used as test sets. For ease of testing, in this embodiment, questions about specific paper content are considered simple requests, and broad questions are considered difficult requests. The specific questions generated are shown in Table 2 below:

[0074] Table 2

[0075]

[0076] This embodiment uses the output of the L-LLM with strong general capabilities as the standard answer to the user's request, and uses the L-LLM to judge whether the S-LLM feedback is reasonable. The relevant prompt words are shown in Table 3 below:

[0077] Table 3

[0078]

[0079] Figure 6a-6cThe experimental results of intelligent decision-making based on RAG’s open source framework Chatchat (L-LLM), RAGFlow (S-LLM) and LS-QA in this embodiment in the scenario of scientific research decision-making are shown; it should be noted that in this embodiment, Chatchat and RAGFlow implement corresponding decisions based on L-LLM and S-LLM respectively, but the LLMs used by these two frameworks can be interchangeable; the feedback accuracy of the experimental results of intelligent decision-making based on RAG’s open source framework Chatchat (L-LLM), RAGFlow (S-LLM) and the decision-making method based on the collaborative optimization of large and small language models in the scenario of scientific research decision-making is shown in the figure. Figure 6c As shown (the horizontal axis is accuracy), since this embodiment uses L-LLM to evaluate the feedback effect of the model, the decision accuracy based on L-LLM is 100%, so this embodiment only reflects the "cost reduction" capability of LS-QA. When RAGFlow uses S-LLM to process user requests, the feedback accuracy is 60.19%, which shows the gap between S-LLM and L-LLM in supporting intelligent decision-making systems; Figure 6a and 6b Shows cost-related metrics (i.e. Figure 6a and Figure 6b The horizontal axes are the proportion of requests processed by L-LLM and the number of characters input by LLM, respectively). When RAGFlow uses S-LLM to complete feedback, the proportion of problems processed by L-LLM and the number of characters processed are both 0, which means that RAGFlow will not have any additional expenses. Compared with Chatchat, which only uses L-LLM, LS-QA fully combines the advantages of the two models. It processes simple requests and difficult requests through S-LLM and L-LLM respectively. When the feedback accuracy rate only decreases by 5.34%, the number of characters processed by L-LLM is reduced by 2736 (i.e., a reduction of 37.73%). Assuming that the number of LLM input and output characters is equivalent, LS-QA will eventually save 5472 characters. This shows that in a real scientific research decision-making scenario, the LS-QA of this embodiment can effectively reduce the system operating costs while ensuring the decision-making effect.

[0080] Computer System Embodiment

[0081] This embodiment provides a technical solution for a computer system, which includes a processor storing executable program instructions, and the executable program instructions are used to be executed to implement the decision-making method based on collaborative optimization of large and small language models in the above-mentioned decision-making method embodiment based on collaborative optimization of large and small language models.

[0082] Since the specific working principle and effect of the computer system in this embodiment have been described in detail in the above-mentioned decision-making method embodiment based on collaborative optimization of large and small language models, they will not be repeated here.

[0083] Computer Readable Storage Medium Embodiments

[0084] This embodiment provides a technical solution of a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed, they are used to implement the decision-making method based on collaborative optimization of large and small language models in the above-mentioned decision-making method embodiment based on collaborative optimization of large and small language models.

[0085] Since the specific working principle and effect of the computer-readable storage medium in this embodiment have been described in detail in the above-mentioned decision-making method embodiment based on collaborative optimization of large and small language models, they will not be repeated here.

[0086] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, but do not constitute a limitation to the present invention.

Claims

1. A decision-making method based on collaborative optimization of large and small language models, characterized in that: For a request of the overall request processing module in the user input decision model, the pre-trained routing model in the overall request processing module is used to determine whether the request is a simple request or a difficult request; if it is a simple request, the smaller large language model is used to process the request to obtain a decision result; if it is a difficult request, the larger large language model is used to process the request to obtain a decision result; The routing model includes a primary traffic diversion routing model obtained by pre-training using training data including labels corresponding to simple requests and difficult requests.

2. The decision-making method based on collaborative optimization of large and small language models according to claim 1, characterized in that: The smaller large language model includes a fine-tuned smaller large language model obtained by fine-tuning a learnable smaller large language model using an output obtained by inputting a simple request to a preset smaller large language model, an output obtained by inputting a simple request to a preset larger large language model, and an output obtained by inputting a difficult request to a preset larger large language model, and a smaller large language model that has not been fine-tuned; The routing model further includes a secondary diversion routing model obtained by pre-training with data obtained by fine-tuning the learnable smaller large language model, and the secondary diversion routing model is used to further determine whether the difficult request determined by the primary diversion routing model is a simple request or a difficult request; If it is a simple request, the smaller large language model processes the request to obtain a decision result; if it is a difficult request, the larger large language model processes the request to obtain a decision result. The method includes: the unfine-tuned smaller large language model processes the simple request determined by the primary diversion routing model to obtain a decision result; the fine-tuned smaller large language model is used to process the simple request determined by the secondary diversion routing model to obtain a decision result; the larger large language model processes the difficult request determined by the secondary diversion routing model to obtain a decision result.

3. The decision-making method based on collaborative optimization of large and small language models according to claim 2 is characterized in that: The method for fine-tuning the learnable smaller large language model to obtain data includes: in the training process of fine-tuning the learnable smaller large language model, inputting the user request into the learnable smaller large language model in each iteration to obtain an output result; if the output result is accurate, marking the output result with a label of a simple request; if it is inaccurate, marking the output result with a label of a difficult request; and using the output results and labels of a set number of times as the data obtained for fine-tuning.

4. The decision-making method based on collaborative optimization of large and small language models according to any one of claims 1 to 3, characterized in that: For the request of the fragment request processing module in the user input decision model, the original knowledge matching the request is first found in the knowledge base, and then the redundant information in the original knowledge matching the request is eliminated according to the request through the smaller large language model in the fragment request processing module. The original knowledge matching the request after the redundant information is eliminated is input into the larger large language model for processing to obtain the decision result of the request.

5. The decision-making method based on collaborative optimization of large and small language models according to claim 2 or 3, characterized in that: The LoRA large model fine-tuning algorithm is used to fine-tune the learnable smaller large language model using the output obtained by inputting a simple request to the preset smaller large language model, the output obtained by inputting a simple request to the preset larger large language model, and the output obtained by inputting a difficult request to the preset larger large language model.

6. The decision-making method based on collaborative optimization of large and small language models according to claim 2 or 3, characterized in that: The unadjusted smaller large language model is used to process the simple request determined by the primary shunt routing model to obtain a decision result; the fine-tuned smaller large language model is used to process the simple request determined by the secondary shunt routing model to obtain a decision result; The method of processing the difficult request determined by the secondary diversion routing model by the larger language model to obtain a decision result includes: after preliminarily determining whether the request is a simple request or a difficult request through the primary diversion routing model, matching the request with the knowledge in the knowledge base to obtain a pre-set number of documents with the highest prior probability; if the primary diversion routing model determines that it is a simple request, the obtained set number of documents and the request are combined and processed by a smaller large language model that has not been fine-tuned to obtain a decision result; the difficult request determined by the primary diversion routing model is further determined by the secondary diversion routing model as a simple request or a difficult request; if it is determined that it is a simple request, the obtained set number of documents and the request are combined and processed by the fine-tuned smaller large language model to obtain a decision result; if it is determined that it is a difficult request, the obtained set number of documents and the request are combined and processed by the larger large language model to obtain a decision result.

7. The decision-making method based on collaborative optimization of large and small language models according to claim 6, characterized in that: Matching the request with the knowledge in the knowledge base is accomplished using an embedding model.

8. The decision-making method based on collaborative optimization of large and small language models according to any one of claims 1 to 3, characterized in that: The routing model is a BERT network.

9. A computer system comprising a processor, characterized in that: The processor stores executable program instructions, and the executable program instructions are used to be executed to implement the decision-making method based on collaborative optimization of large and small language models according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The storage medium stores computer executable instructions, wherein the computer executable instructions are used to implement the decision-making method based on collaborative optimization of large and small language models as described in any one of claims 1 to 8 when executed.

Images