Large language model decision optimization method and device and electronic equipment

Abstract

The invention provides a big language model decision optimization method and device and electronic equipment, and the method comprises the steps: obtaining an input text, generating each sequence unit through a big language model based on the input text, calculating the correlation degree of each sequence unit and the input text, and obtaining a big language model decision optimization result; the target sequence units with the correlation degree larger than the preset correlation degree threshold value are taken as the basis for generating the reply text, and a large amount of interference information irrelevant to the input text can be screened out. And performing content verification on the generated reply text by calling a preset cross-domain knowledge base, adjusting model parameters of the large language model according to a verification result until the generated reply text conforms to knowledge and facts specified by the preset cross-domain knowledge base, and finally outputting the reply text conforming to the specification. The influence of information overload on the precision of the output reply text is effectively avoided, the knowledge and facts in the professional knowledge base are adopted to verify the output reply text, the output reply text can more comply with objective facts, and the precision is higher.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular, to a method, device, and electronic device for optimizing the decision-making of large language models. Background Art

[0002] LLMs (Large Language Models) are natural language processing models based on deep learning technology. By connecting through large-scale text data, they can generate natural and fluent language texts and are commonly used in many natural language processing tasks such as text generation, translation, question answering, sentiment analysis, and generating complex decisions. Among them, decision-making refers to when facing multiple choices or solutions, in order to achieve a certain goal, by analyzing, evaluating, and comparing various information, and then selecting an action plan or making a judgment.

[0003] However, although LLMs perform excellently in many tasks, they still face some challenges. Exemplarily, when current LLMs face complex tasks, they may be affected by information overload, resulting in a decrease in the accuracy of the generated content. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a method, device, and electronic device for optimizing the decision-making of large language models to improve the stability and accuracy of large model reasoning.

[0005] In a first aspect, the embodiments of the present application provide a method for optimizing the decision-making of large language models. The method includes:

[0006] Obtain the input text, and input the input text into the large language model to determine the degree of association between each sequence unit generated by the large language model based on the input text and the input text;

[0007] Generate a response text according to the target sequence units whose degree of association is greater than a preset degree-of-association threshold;

[0008] Call a preset cross-domain knowledge base to verify the response text, and adjust the model parameters of the large language model according to the verification result until the response text conforms to the knowledge and facts stipulated by the preset cross-domain knowledge base;

[0009] Output the response text that conforms to the knowledge and facts stipulated by the preset cross-domain knowledge base.

[0010] In combination with the first aspect, in a second possible embodiment, each attention layer in the large language model is constructed based on the self-attention mechanism. The determining the degree of association between each sequence unit generated by the large language model based on the input text and the input text includes:

[0011] Perform a weight analysis on each of the attention layers to determine an attention weight matrix, where each element in the attention weight matrix is an attention weight value between the sequence unit and the input information that generates the sequence unit, and the attention weight value has a positive correlation with the degree of association;

[0012] The method further includes:

[0013] Determine the sequence units with the attention weight value greater than a preset attention weight threshold as the target sequence units.

[0014] Combined with the first aspect, in the third possible embodiment, the determining the degree of association between each sequence unit generated by the large language model based on the input text and the input text includes:

[0015] Calculate the information gain of each sequence unit through a preset Shannon information gain algorithm; where the information gain has a positive correlation with the degree of association;

[0016] The method further includes:

[0017] Determine the sequence units with the information gain greater than a preset information gain threshold as the target sequence units.

[0018] Combined with the first aspect, in the fourth possible embodiment, the obtaining the input text and inputting the input text into the large language model includes:

[0019] Obtain the original input text, call an external tool, and extract the key information included in the original input text through the external tool, where the key information includes: keywords, entities, timestamps, factual statements;

[0020] Input the preprocessed key information after formatting the key information and the original input text into the large language model, and the large language model generates each sequence unit.

[0021] Combined with the fourth possible embodiment of the first aspect, in the fifth possible embodiment, the calling a preset cross-domain knowledge base to verify the reply text and adjusting the model parameters of the large language model according to the verification result includes:

[0022] Obtain verification information associated with the key information from the preset cross-domain knowledge base through the external tool;

[0023] Perform a factual verification on the reply text according to the factual statement in the key information and the verification information;

[0024] If the verification is successful, the response text is output. If the verification fails, negative sample data is generated, and the model parameters of the large language model are adjusted using the negative sample data until the proportion of the generated negative sample data is less than a preset proportion threshold.

[0025] Combined with the first aspect, in a sixth possible embodiment, generating a response text according to the target sequence unit whose association degree is greater than a preset association degree threshold includes:

[0026] Determine a prompt template according to the input text, and generate the response text based on the prompt template; wherein, the prompt template contains several prompts;

[0027] If the input text corresponds to a multi-step reasoning task, each of the prompts is used to: instruct the large language model to generate the response text according to the input format, output format, and the step order indicated by each of the prompts for each step specified by each of the prompts;

[0028] If the input text corresponds to a fact verification and reasoning task, each of the prompts is used to: instruct the large language model to generate the response text according to the factual basis corresponding to each of the prompts.

[0029] Combined with the first aspect, in a seventh possible embodiment, invoking a preset cross-domain knowledge base to verify the response text includes:

[0030] The large language model evaluates the response text in three dimensions of correctness, interpretability, and practicality based on the knowledge and facts provided by the preset cross-domain knowledge base, and outputs the evaluation scores corresponding to each dimension; wherein, the correctness means that the content of the output response text conforms to the provisions of the knowledge and facts, the interpretability means that the output response text has a traceable basis, and the practicality means that the output response text meets the requirements of the actual application scenario;

[0031] If one of the evaluation scores corresponding to each dimension is less than a set evaluation score threshold, it is determined that the verification of the response text fails.

[0032] In a second aspect, an embodiment of the present application provides a large language model decision optimization device, wherein the device includes:

[0033] A text input module, configured to obtain an input text, input the input text into a large language model, and determine the association degree between each sequence unit generated by the large language model based on the input text and the input text;

[0034] A text generation module, configured to generate a response text according to a target sequence unit whose association degree is greater than a preset association degree threshold;

[0035] A verification module, configured to call a preset cross-domain knowledge base to verify the response text, and adjust the model parameters of the large language model according to the verification result until the response text conforms to the knowledge and facts specified in the preset cross-domain knowledge base;

[0036] A text output module, configured to output a response text that conforms to the knowledge and facts specified in the preset cross-domain knowledge base.

[0037] Combined with the second aspect, in a second possible embodiment, each attention layer constructed based on the self-attention mechanism is included in the large language model, and the text input module is specifically configured to:

[0038] Perform weight analysis on each of the attention layers to determine an attention weight matrix, where each element in the attention weight matrix is an attention weight value between the sequence unit and the input information that generates the sequence unit, and the attention weight value has a positive correlation with the degree of association;

[0039] The text generation module is further configured to determine the target sequence unit as the sequence unit whose attention weight value is greater than a preset attention weight threshold.

[0040] Combined with the second aspect, in a third possible embodiment, the text input module is specifically configured to:

[0041] Calculate the information gain of each of the sequence units through a preset Shannon information gain algorithm; where the information gain has a positive correlation with the degree of association;

[0042] The text generation module is further configured to determine the target sequence unit as the sequence unit whose information gain is greater than a preset information gain threshold.

[0043] Combined with the second aspect, in a fourth possible embodiment, the text input module is specifically configured to:

[0044] Obtain the original input text, call an external tool, and extract the key information included in the original input text through the external tool, where the key information includes: keywords, entities, timestamps, factual statements;

[0045] Input the preprocessed key information after formatting the key information and the original input text into the large language model, and generate each of the sequence units by the large language model.

[0046] Combined with the fourth possible embodiment of the second aspect, in a fifth possible embodiment, the verification module is specifically applied to:

[0047] Obtain verification information associated with the key information from the preset cross - domain knowledge base through the external tool;

[0048] Perform factual verification on the response text according to the factual statements in the key information and the verification information;

[0049] If the verification is successful, output the response text. If the verification fails, generate negative sample data, and use the negative sample data to adjust the model parameters of the large - language model until the proportion of the generated negative sample data is less than the preset proportion threshold.

[0050] Combined with the second aspect, in the sixth possible embodiment, the text generation module is specifically configured to:

[0051] Determine a prompt template according to the input text, and generate the response text based on the prompt template; wherein, the prompt template contains several prompts;

[0052] If the input text corresponds to a multi - step reasoning task, each of the prompts is used to: instruct the large - language model to generate the response text according to the input format, output format of each step specified by each of the prompts, and the step order indicated by each of the prompts;

[0053] If the input text corresponds to a factual verification reasoning task, each of the prompts is used to: instruct the large - language model to generate the response text according to the factual basis corresponding to each of the prompts.

[0054] Combined with the second aspect, in the seventh possible embodiment, the verification module is further configured to:

[0055] Evaluate the response text by the large - language model in three dimensions of correctness, interpretability, and practicality based on the knowledge and facts provided by the preset cross - domain knowledge base, and output the evaluation scores corresponding to each dimension; wherein, the correctness means that the content of the output response text conforms to the provisions of the knowledge and facts, the interpretability means that the output response text has a traceable basis, and the practicality means that the output response text meets the requirements of the actual application scenario;

[0056] If one of the evaluation scores corresponding to each dimension is less than the set evaluation score threshold, determine that the verification of the response text fails.

[0057] In a third aspect, an embodiment of the present application provides an electronic device, where the electronic device includes: a processor; and a memory storing a program; wherein, the program includes instructions that, when executed by the processor, cause the processor to execute the large - language model decision optimization method described in the first aspect.

[0058] Fourthly, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the large language model decision optimization method described in the first aspect.

[0059] Advantages of the present application:

[0060] The present application provides a large language model decision optimization method, apparatus and electronic device. The method obtains an input text, inputs the input text into a large language model, and the large language model generates corresponding sequence units based on the input text, calculates the degree of association between each sequence unit and the input text, and selects the target sequence unit with the degree of association greater than a preset association degree threshold as the basis for generating a response text, which can filter out a large amount of interference information irrelevant to the input text, making the generated response text more conform to the text content corresponding to the input text. In addition, the present application calls a preset cross-domain knowledge base to verify the content of the generated response text. During the process, the model parameters of the large language model are adjusted according to the verification results until the generated response text conforms to the knowledge and facts stipulated by the preset cross-domain knowledge base, and finally outputs the response text that conforms to the knowledge and facts stipulated by the preset cross-domain knowledge base.

[0061] In this way, by selecting the embodiment of the present application, through determining the degree of association between the sequence units generated by the large model and the input text and performing relevance analysis on the input text, the interference of interference information on the output response text can be reduced, effectively avoiding the influence of information overload on the accuracy of the output response text, better improving the output accuracy of the large language model, and verifying the output response text with the knowledge and facts in the professional knowledge base, which can make the output response text more compliant with objective facts and have higher accuracy. Description of the Drawings

[0062] In the following description of the exemplary embodiments in conjunction with the drawings, more details, features and advantages of the present application are disclosed. In the drawings:

[0063] Figure 1 Shows a schematic flowchart of a large language model decision optimization method provided by an embodiment of the present application;

[0064] Figure 2 Shows another schematic flowchart of a large language model decision optimization method provided by an embodiment of the present application;

[0065] Figure 3 Shows a schematic logical architecture diagram of a large language model decision optimization apparatus provided by an embodiment of the present application;

[0066] Figure 4 Shows a structural block diagram of an exemplary electronic device capable of implementing the embodiments of the present application. Detailed implementation manners

[0067] Embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not used to limit the protection scope of the present application.

[0068] It should be understood that the various steps described in the method embodiments of the present application can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this regard.

[0069] The term "including" and its variations used herein are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first" and "second" mentioned in the present application are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions executed by these devices, modules or units or their interdependent relationships.

[0070] It should be noted that the modifications of "one" and "plural" mentioned in the present application are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0071] As described in the background art, in the application process of existing technologies in LLMs, especially when dealing with complex tasks, there are still limitations in aspects such as processing long texts, task reasoning, and model interpretability. Specifically, it is reflected in the following aspects:

[0072] (1) Insufficient decision-making accuracy. Although large language models perform well in many common natural language processing tasks, in some complex or information-intensive tasks, the decision-making accuracy of the models is often insufficient. Especially when faced with long texts or multi-step reasoning tasks, LLMs are often interfered by irrelevant information and ignore key content, resulting in incorrect judgments. Exemplarily, the following two examples can be combined for understanding. Example 1: In the fact-checking task, LLMs may mistake irrelevant background information for relevant information, leading to wrong conclusions. Example 2: In the medical field, LLMs may give wrong advice because they cannot accurately parse key information in drug molecules or medical literature. Therefore, it is difficult for existing technologies to ensure the decision-making accuracy of LLMs in all tasks, especially in complex tasks involving details and professional fields.

[0073] (2) Insufficient interpretability. As a black-box model, the internal decision-making mechanism of large language models lacks transparency, and the reasoning process of the models is difficult for users to understand. In the process of engineering practice, although existing interpretability techniques such as LIME (Local Interpretable Model-Agnostic Explanations) and SHAP (Shapley Additive Explanations) can be used to help understand the basis for the model to generate outputs, these methods mainly focus on local interpretability and are difficult to provide a comprehensive explanation for the entire decision-making process. Especially for generative tasks, interpretation tools such as LIME and SHAP cannot effectively reveal the specific decision-making basis of the model in each step of the reasoning process. Because users' distrust of the model decision-making process will limit the use of the model, this lack of transparency problem restricts the application of LLMs in some high-risk fields (such as medical, legal, etc.).

[0074] (3) Poor adaptability to complex tasks. Current model training and fine-tuning methods pay more attention to single-task optimization and lack adaptability to complex, cross-domain tasks. Especially when LLMs face joint tasks involving multiple domains and multiple tasks, problems such as overfitting during training or insufficient generalization ability are likely to occur. Exemplarily, in scenarios that require handling multiple tasks, existing fine-tuning methods often lack targeted optimization strategies, resulting in the unbalanced development of the model's reasoning ability and decision-making performance in all tasks, and the model may perform well in some tasks but show obvious deviations in other tasks.

[0075] Therefore, how to further improve the inference stability and accuracy of large language models based on existing training methods and fine-tuning methods remains a technical problem to be solved urgently. In view of this, the present application provides a large language model optimization method, device and electronic device. Among them, in the first aspect, the present application provides a large language model decision optimization method, which is applied to any electronic device with large language model decision optimization function, including but not limited to personal mobile terminals, computers or servers, etc. As Figure 1 shown, the method includes the following steps:

[0076] S11. Obtain the input text, input the input text into the large language model, and determine the degree of association between each sequence unit generated by the large language model based on the input text and the input text;

[0077] S12. Generate a response text according to the target sequence unit whose degree of association is greater than the preset degree of association threshold;

[0078] S13. Invoke a preset cross-domain knowledge base to verify the response text, and adjust the model parameters of the large language model according to the verification result until the response text conforms to the knowledge and facts specified in the preset cross-domain knowledge base;

[0079] S14. Output the response text that conforms to the knowledge and facts specified in the preset cross-domain knowledge base.

[0080] In the embodiment of the present application, by obtaining the input text and inputting the input text into the large language model, the large language model generates corresponding sequence units based on the input text, calculates the degree of association between each sequence unit and the input text, and takes the target sequence unit whose degree of association is greater than the preset degree of association threshold as the basis for generating the response text, a large amount of interference information irrelevant to the input text can be filtered out, so that the generated response text is more in line with the text content corresponding to the input text. In addition, in the present application, by invoking a preset cross-domain knowledge base to verify the content of the generated response text, during the process, the model parameters of the large language model are adjusted according to the verification result until the generated response text conforms to the knowledge and facts specified in the preset cross-domain knowledge base, and finally the response text that conforms to the knowledge and facts specified in the preset cross-domain knowledge base is output.

[0081] Thus, by selecting the embodiments of the present application and analyzing the relevance of the large model generated sequence units to the input text, the relevance analysis of the input text can be carried out, which can reduce the interference of interference information to the output response text, effectively avoid the influence of information overload on the accuracy of the output response text, better improve the output accuracy of the large language model, and use the knowledge and facts in the professional knowledge base to verify the output response text, so that the output response text can comply with objective facts more and has higher accuracy.

[0082] The following will combine specific examples to elaborate on the above steps S11 to S14 in detail:

[0083] The large language model decision optimization method provided by the present application is mainly applied to the training stage of the large language model. The core purpose is to improve the accuracy of the response text output by the large language model when dealing with complex reasoning tasks and multi-step reasoning tasks, as well as to improve the stability and interpretability of the output results of the large language model. Based on this, when performing step S11, the input text is obtained. The input text can be the training sample data for training the large language model, and the input text can be the problem text simulated when the user uses the large language model. Correspondingly, the text output by the large language model after organizing, summarizing, and analyzing the problem text can be called the response text.

[0084] As an implementation manner, the input text is input into the large language model, and the natural language processing network built into the large language model extracts key information from the input text. The key information extraction process can be: the large language model preprocesses the input text, splits the large text into several words, and preprocesses the redundant white spaces, special characters, inconsistent case, etc. in the text, and converts the words included in the input text into a unified format. Then, the large language model determines the corresponding potential output words for each word according to the pre-constructed vocabulary table. The subsequent response text will be generated based on the determined potential output words. In this article, the words used to construct and generate the response text are determined as sequence units. This sequence unit is also often referred to as a token in the industry, and it can be understood that one token corresponds to one word.

[0085] Since large language models may be designed with different areas of expertise at the beginning, for example, some large language models are developed to be good at handling human-machine dialogue tasks, some large language models are developed to be good at summarizing professional knowledge documents, and some large language models are developed to be good at generating work-related documents. Thus, in actual applications, under different application scenario requirements, the text processing capabilities of large language models are different. To reduce the problem of poor accuracy of the response text output by the large language model due to the lack of key information extraction ability that may exist in a single large language model, as an implementation manner, when the embodiment of the present application executes step S11, it can be implemented through the following steps:

[0086] Obtain the original input text, call an external tool, and extract the key information contained in the original input text through the external tool, where the key information includes: keywords, entities, timestamps, factual statements;

[0087] Input the preprocessed key information after formatting the key information and the original input text into the large language model, and generate each sequence unit by the large language model.

[0088] Among them, the original input text can be understood as the initially obtained training sample data, which is the text data originally used to improve the accuracy of the response text output by the large language model. In the embodiment of the present application, the external tool can be understood as a language processing tool independent of the large language model, which can be another large language model or a device or functional module with language processing capabilities. Among them, data interaction between the external tool and the deployment device of the method provided in the present application is performed through a preset data communication interface. The deployment device can call the external tool to run by calling the preset data communication interface, and the external tool performs a text processing on the input original input text to extract the key information contained in the input text.

[0089] In the embodiment of the present application, the key information contained in the input text mainly includes: keywords, entities, timestamps, factual statements, etc. Specifically, keywords can also be understood as core entities, including keyword vocabularies such as people, places, organizations, events, and professional concepts. Entities specifically refer to objects with independent existence meanings in the input text, which can be specific things, people, animals, places, buildings, products. Entities can also refer to abstract concepts, such as organizations, institutions, theories, policies, etc. A timestamp refers to a specific occurrence time or the time mentioned in the text. A factual statement refers to words, short sentences in the text that can be verified as true or consistent with objective reality.

[0090] Further, formatting the key information into preprocessed key information may involve classifying the key information according to a preset sorting format. For example, each word belonging to an entity is counted in the data set of [entity], and each data set is the preprocessed key information. Then, the preprocessed key information and the original input text are input into the large language model together, and the large language model further generates each sequence unit associated with the key information in the input text.

[0091] Selecting the embodiment of the present application can extract the key information contained in the input text by introducing an external tool. In particular, the external tool can extract factual statements from the input text, compare them with the given verification questions, and generate structured input information to be provided to the large language model. In this way, it is possible to assist the large language model in extracting information from the input text, improve the information extraction accuracy of the large language model, and further help improve the accuracy of the subsequent reply text output by the large language model.

[0092] In the embodiment of the present application, when performing step S11, determining the degree of association between each sequence unit generated by the large language model based on the input text can be understood as using this degree of association to screen each potential vocabulary initially determined by the large language model for outputting the reply text. So that when performing step S12, the target sequence units with the degree of association greater than the preset association threshold are selected to generate the reply text, that is, the vocabulary with a higher degree of association with the information in the input text among the potential output vocabularies is selected to generate the reply text, which can effectively reduce the influence of redundant information in the miscellaneous data on the output accuracy of the reply text.

[0093] Among them, as an implementation method, if the large language model contains each attention layer constructed based on the self-attention mechanism, when performing step S11 to determine the degree of association between each sequence unit and the input text, the degree of association between each sequence unit and the input text can be determined through the following steps:

[0094] Perform a weight analysis on each of the attention layers to determine an attention weight matrix, where each element in the attention weight matrix is the attention weight value between the sequence unit and the input information that generates the sequence unit, and the attention weight value has a positive correlation with the degree of association.

[0095] Based on this, when performing step S12, the sequence units with the attention weight value greater than the preset attention weight threshold can be used as the target sequence units for generating the reply text.

[0096] Specifically, if the network architecture of the large language model is the Transformer architecture, the architecture will default to include multiple attention layers constructed by the multi-head self-attention mechanism. Among them, when the large language model generates sequence units, the encoder performs data transformation based on the input text sequence, and the decoder generates sequence units according to the information transformed by the encoder. Among them, there is a self-attention layer in the encoder, and this self-attention layer can be used to process the internal relationships existing in the input sequence. Among them, the attention weights can be obtained by a specific method. Based on this, weight analysis is performed on each attention layer to determine the attention weight matrix. The weight values of each attention head can be obtained by a specific method. As an example, assuming that the network architecture of the large language model is the Transformer architecture, the weight values of each attention head in the encoder and decoder of the large language model can be obtained by the attention weight acquisition method in the PyTorch library. Then, the attention weight matrix is generated based on the weight values of each attention head, and each element in the matrix corresponds to the weight value of an attention head.

[0097] In the embodiment of the present application, the output word is determined based on each input word in the input sequence, and is a word associated with the input word. Under the attention mechanism, the higher the attention weight, it indicates that when outputting a certain word, the model has a greater attention weight value when outputting this word, and usually indicates that the generated sequence unit has a higher degree of association with the input sequence for generating this sequence unit. Using this attention weight value can be characterized as the degree of association between the sequence unit and the input information for generating this sequence unit, that is, there is a positive correlation between the attention weight value and the degree of association. The higher this degree of association, the higher the reliability of the sequence unit with a higher degree of association, the lower the probability of belonging to redundant information, the higher the probability that the response text generated based on this sequence unit fits the input text, and the higher the accuracy of the obtained response text.

[0098] As another implementation manner, when performing step S11, the degree of association between each sequence unit and the input text can also be determined in the following manner:

[0099] By using a preset Shannon information gain algorithm, calculate the information gain of each of the sequence units; wherein, the information gain is positively correlated with the degree of association;

[0100] When performing step S12, the sequence units with information gain greater than the preset information gain threshold can be determined as the target sequence units.

[0101] Among them, the preset Shannon information gain algorithm can be the Shannon information gain algorithm pre-set in the large language model. Among them, the Shannon information entropy describes the uncertainty of a random variable, and the Shannon information gain can describe the reduction of the information entropy of a random variable under a certain known condition. Among them, when generating the sequence unit token, assuming that the vocabulary corresponding to the input text task is V, the text sequence T={t1, t2, t3···tn} can be generated, where ti is the i-th sequence unit. At this time, based on the corpus or prior knowledge, the prior probability of each word in the vocabulary can be calculated, and further based on the Shannon information entropy calculation function, the information entropy of each sequence unit can be calculated. Since the distribution probability of each word will change accordingly when each sequence unit is generated, at this time, the information gain of each sequence unit can be calculated according to the information entropy of the sequence units at different times.

[0102] In the embodiment of the present application, the information gain can clearly describe the contribution degree of each generated sequence unit to the sequence unit. Among them, the greater the information gain, the higher the degree of association between the sequence unit and the input information, the more the uncertainty of the generated response text is reduced, and the higher the contribution to the generated response text. In this way, the sequence unit with an information gain greater than the preset information gain threshold is determined as the target sequence unit, so that the model gradually focuses on the input information most relevant to the task, thereby improving the accuracy of decision-making.

[0103] On this basis, in some possible embodiments, when generating the response text according to the target sequence unit in step S12, it can be implemented in the following manner:

[0104] Determine the prompt word template according to the input text, and generate the response text based on the prompt word template; where, several prompt words are included in the prompt word template;

[0105] If the input text corresponds to a multi-step reasoning task, each of the prompt words is used to: instruct the large language model to generate the response text according to the input format, output format of each step specified by each of the prompt words, and the step order indicated by each of the prompt words;

[0106] If the input text corresponds to a fact verification reasoning task, each of the prompt words is used to: instruct the large language model to generate the response text according to the fact basis corresponding to each of the prompt words.

[0107] Among them, text can be input for data extraction to determine the key information related to the reasoning process and fact-verified reasoning in the input text, such as key information like changing time nodes, changing geographical locations, or the order of execution steps. Each piece of key information related to the reasoning process and fact verification is a prompt word, and the prompt words can vary according to the type of reasoning task corresponding to the input text. Specifically, if the input text corresponds to a multi-step reasoning task, such as when inputting the calculation method of Shannon information gain, the prompt words can be the concept of Shannon information gain, the calculation formula of Shannon information gain, the derivation process of Shannon information gain, the application scenarios of Shannon information gain, etc. Among them, the derivation process can be further broken down into: the first step, the second step, the third step, the fourth step... Or, if the input text corresponds to a fact-verified reasoning task, the fact-verified reasoning task requires the large language model to determine whether the facts described in the input text are true and accurate based on a large amount of data analysis. At this time, the prompt words can include various professional terms mentioned in the fact-verified reasoning task, and then by querying the data explanations corresponding to each professional term, the authenticity and accuracy of the input fact-verified reasoning task can be verified.

[0108] Then, based on each piece of key information, the corresponding prompt word template is determined. This prompt word template is similar to a vocabulary outline or an output text directory. Based on this prompt word template, the generated target sequence units can be grouped into words, formed into sentences, and logically sorted to output reply texts of different types such as short and long sentences, text paragraphs, etc.

[0109] When selecting the embodiments of the present application, the large language model can automatically check the validity of the reply text output during the reasoning process, ensure that the generated content is triggered by factual basis, and ensure that the generated content is consistent with the knowledge of the real world. In addition, by optimizing the structure and content of the prompt words, the large language model can be more precisely focused on the task objective and improve the generation effect of the reply text.

[0110] On this basis, when executing step S13, the reply text can be verified through the following steps:

[0111] The large language model evaluates the reply text in three dimensions of correctness, interpretability, and practicality based on the knowledge and facts provided by the preset cross-domain knowledge base, and outputs the evaluation scores corresponding to each dimension; where the correctness means that the content of the output reply text conforms to the regulations of the knowledge and facts, the interpretability means that the output reply text has a traceable basis, and the practicality means that the output reply text meets the requirements of the actual application scenario;

[0112] If one of the evaluation scores corresponding to each dimension is less than the set evaluation score threshold, it is determined that the verification of the reply text fails.

[0113] As an implementation manner, step S13 is executed to verify the reply text. Specifically, step S13 can also be implemented through the following steps:

[0114] Obtain verification information associated with the key information from the preset cross-domain knowledge base through the external tool;

[0115] Perform factual verification on the reply text according to the factual statements in the key information and the verification information;

[0116] If the verification is successful, output the reply text. If the verification fails, generate negative sample data, and use the negative sample data to adjust the model parameters of the large language model until the proportion of the generated negative sample data is less than the preset proportion threshold.

[0117] In the embodiment of the present application, the preset cross-domain knowledge base is similar to the external tool, and there is a preset data communication interface with the large language model. The preset cross-domain knowledge base can be called through the set data communication interface. The preset cross-domain knowledge base can be any type of professional knowledge database, such as a medical knowledge database, a chemical knowledge database, a financial knowledge database, etc. The specific type is not strictly limited in this application. When executing step S13, after calling the preset cross-domain knowledge base through this data communication interface, according to the key information extracted by the external tool, it can be matched with the knowledge in the preset cross-domain knowledge base to obtain the knowledge or factual discussion associated with the key information extracted by the external tool, and this associated knowledge or factual discussion can be used as the verification information for subsequent reply text verification.

[0118] Furthermore, according to the factual statements in the key information and the verification information obtained from the preset cross-domain knowledge base, factual verification can be performed on the initially generated reply text to determine whether the reply text follows the basic laws or facts of the corresponding field.

[0119] Exemplarily, taking the input text as a medical diagnosis text as an example, the disease type, disease development degree, patient allergen information, etc. involved in the medical diagnosis text can be extracted through an external tool, and a preset medical field knowledge database can be called to generate a corresponding sequence unit of drug recommendation. Further, with the help of various facts, knowledge, historical prescriptions, etc. associated with the disease type, disease development degree, and patient allergen information in the preset medical field knowledge database, the generated sequence unit of drug recommendation is verified to determine whether the information such as the molecular structure and pharmacological action of the drug corresponding to the sequence unit of drug recommendation can be used for this patient, so as to realize the verification of the generated drug recommendation text.

[0120] If the verification is successful, it means that the response text generated by the large language model can accurately give a response text that matches the relevant field according to the relevant field, and the accuracy is relatively high or very high. If the verification fails, it means that the accuracy of the response text generated by the large language model is still relatively low. At this time, it is necessary to optimize the large language model to improve the accuracy of the response text generation of the large language model. Specifically, as an implementation method, the large language model can be regarded as a child with some understanding ability, and verification questions can be input to the large language model, such as: According to the relevant knowledge in the given cross-domain knowledge base, do you think the result you currently output is accurate?

[0121] Then, the large language model comprehensively evaluates the response text it generates according to the input verification question, calculates the degree of association between the generated response text and the verification information in the cross-domain knowledge base. When the degree of association is relatively high and lower than a set verification association threshold, it can be determined that the response text verification is successful, and the response text can be determined as positive sample data. If the degree of association is relatively low and higher than the verification association threshold, it can be determined that the response text verification fails, and the response text can be determined as negative sample data.

[0122] Regard the negative sample data as the negative feedback of the large language model, and adjust the model parameters of the large language model, such as the weight value of the attention head, according to the negative sample data until the proportion of the generated negative sample data is less than the preset proportion threshold. Among them, various thresholds such as the preset proportion threshold, the preset association degree threshold, the preset information gain threshold, the preset attention weight threshold, and the preset evaluation score threshold can be flexibly set according to actual experience or the actual application scenario of the large model, and are not strictly limited here.

[0123] In this way, by selecting the embodiment of the present application, verifying the response text output by the large language model by using the professional knowledge and facts in the preset cross-domain knowledge base, and feeding the verification result back to the large language model, it can be ensured that the response text given by the trained large language model conforms to the domain knowledge and facts, and the accuracy is higher.

[0124] To facilitate the understanding of the method provided by the embodiment of the present application, it can be understood in combination with the system framework diagram as shown in Figure 2 In order to optimize the output decision of the large language model, the large language model is specifically optimized from three aspects in the embodiment of the present application:

[0125] Relevance analysis decision optimization, external tool-assisted optimization, and decision fine-tuning based on prompt reconstruction. Specifically, the relevance analysis decision optimization can be understood as performing step S11, screening the generated tokens, determining the target tokens with a relevance degree greater than the preset relevance degree threshold to the input information, or calculating the information gain of each token, and using this information gain for relevance quantitative analysis to further reduce the interference of redundant information on the output response text, optimize the output response text, so as to achieve optimized feedback and relevance guidance.

[0126] External tool-assisted optimization can be understood as, after preprocessing information such as extraction and formatting with the help of external tools, integrating the external tools with a cross-domain knowledge base to achieve real-time verification of the output response text through the professional knowledge in the cross-domain knowledge base, and adjusting the response text according to the verification results.

[0127] Finally, the decision fine-tuning based on prompt reconstruction is to set different prompts according to different task types, construct a prompt template, and then use a multi-dimensional task evaluation mechanism constructed from three dimensions: correctness, interpretability, and practicality, and then continuously iteratively fine-tune the model parameters of the model according to the evaluation results to achieve the effect of optimizing the response text finally output by the large language model.

[0128] In a second aspect, the present application provides a large language model decision optimization device, wherein, as Figure 3 shown, the device 30 includes:

[0129] A text input module 301, configured to obtain an input text, input the input text into the large language model, and determine the relevance degree of each sequence unit generated by the large language model based on the input text to the input text;

[0130] A text generation module 302, configured to generate a response text according to the target sequence unit with a relevance degree greater than the preset relevance degree threshold;

[0131] A verification module 303, configured to call a preset cross-domain knowledge base, verify the response text, and adjust the model parameters of the large language model according to the verification results until the response text conforms to the knowledge and facts stipulated by the preset cross-domain knowledge base;

[0132] A text output module 304, configured to output a response text that conforms to the knowledge and facts stipulated by the preset cross-domain knowledge base.

[0133] Combined with the second aspect, in a second possible embodiment, each attention layer constructed based on the self-attention mechanism is included in the large language model, and the text input module is specifically configured to:

[0134] Perform a weight analysis on each of the attention layers to determine an attention weight matrix, where each element in the attention weight matrix is an attention weight value between the sequence unit and the input information that generates the sequence unit, and the attention weight value has a positive correlation with the degree of association;

[0135] The text generation module is further configured to determine, as the target sequence unit, a sequence unit whose attention weight value is greater than a preset attention weight threshold.

[0136] Combined with the second aspect, in a third possible embodiment, the text input module is specifically configured to:

[0137] Calculate the information gain of each of the sequence units through a preset Shannon information gain algorithm; where the information gain has a positive correlation with the degree of association;

[0138] The text generation module is further configured to determine, as the target sequence unit, a sequence unit whose information gain is greater than a preset information gain threshold.

[0139] Combined with the second aspect, in a fourth possible embodiment, the text input module is specifically configured to:

[0140] Obtain the original input text, call an external tool, and extract the key information included in the original input text through the external tool, where the key information includes: keywords, entities, timestamps, factual statements;

[0141] Input the preprocessed key information after formatting the key information and the original input text into the large language model, and generate each of the sequence units by the large language model.

[0142] Combined with the fourth possible embodiment of the second aspect, in a fifth possible embodiment, the verification module is specifically applied to:

[0143] Obtain verification information associated with the key information from the preset cross-domain knowledge base through the external tool;

[0144] Perform factual verification on the reply text according to the factual statement in the key information and the verification information;

[0145] If the verification is successful, output the reply text; if the verification fails, generate negative sample data, and adjust the model parameters of the large language model using the negative sample data until the proportion of the generated negative sample data is less than a preset proportion threshold.

[0146] Combined with the second aspect, in a sixth possible embodiment, the text generation module is specifically configured to:

[0147] Determine a prompt template based on the input text, and generate the response text based on the prompt template; wherein, the prompt template contains several prompts.

[0148] If the input text corresponds to a multi-step reasoning task, each of the prompts is used to: instruct the large language model to generate the response text according to the input format, output format, and the step order indicated by each of the prompts for each step.

[0149] If the input text corresponds to a fact verification and reasoning task, each of the prompts is used to: instruct the large language model to generate the response text according to the factual basis corresponding to each of the prompts.

[0150] Combined with the second aspect, in the seventh possible embodiment, the verification module is further configured to:

[0151] The large language model evaluates the response text in three dimensions of correctness, interpretability, and practicality based on the knowledge and facts provided by the preset cross-domain knowledge base, and outputs the evaluation scores corresponding to each dimension; wherein, the correctness means that the content of the output response text conforms to the provisions of the knowledge and facts, the interpretability means that the output response text has a traceable basis, and the practicality means that the output response text meets the requirements of the actual application scenario.

[0152] If one of the evaluation scores corresponding to each dimension is less than the set evaluation score threshold, it is determined that the verification of the response text fails.

[0153] Among them, in this application, the collection, storage, use, processing, transmission, provision, and disclosure of user personal information and other processing all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0154] The names of the messages or information exchanged between multiple devices in the embodiments of this application are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0155] In a third aspect, an exemplary embodiment of this application further provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program that can be executed by the at least one processor, and when the computer program is executed by the at least one processor, it is used to cause the electronic device to execute the method according to the embodiments of this application.

[0156] An exemplary embodiment of this application further provides a non-transitory computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor of a computer, it is used to cause the computer to execute the method according to the embodiments of this application.

[0157] An exemplary embodiment of the present application further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute the method according to the embodiments of the present application.

[0158] Referring Figure 4 , the structural block diagram of the electronic device 400 that can be used as the server or client of the present application will now be described. It is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer devices, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0159] As Figure 4 shown, the electronic device 400 includes a computing unit 401, which can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM 402) or the computer program loaded from the storage unit 408 into the random access memory (RAM 403). In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. The input / output interface (I / O interface 405) is also connected to the bus 404.

[0160] A plurality of components in the electronic device 400 are connected to the I / O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 can be any type of device that can input information into the electronic device 400. The input unit 406 can receive input digital or character information, and generate key signal inputs related to the user settings and / or function controls of the electronic device. The output unit 407 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 408 can include but is not limited to a magnetic disk, an optical disk. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a BluetoothTM device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0161] The computing unit 401 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 executes the various methods and processes described above. For example, in some embodiments, the aforementioned large language model decision optimization method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 400 via the ROM 402 and / or the communication unit 409. In some embodiments, the computing unit 401 can be configured to execute the aforementioned large language model decision optimization method in any other suitable manner (e.g., by means of firmware).

[0162] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0163] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0164] As used in this application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) that provides machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal that provides machine instructions and / or data to a programmable processor.

[0165] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0166] The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described here), or a computing system that includes any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0167] A computer system can include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship to each other.

Claims

1. A large language model decision optimization method, characterized in that: The method comprises: Acquire an input text, and input the input text into a large language model, and determine a correlation degree between each sequence unit generated by the large language model based on the input text and the input text; Generate a reply text according to the target sequence unit whose correlation degree is greater than a preset correlation degree threshold; Calling a preset cross-domain knowledge base to verify the reply text, and adjusting the model parameters of the large language model according to the verification result until the reply text conforms to the knowledge and facts specified in the preset cross-domain knowledge base; Output a response text that complies with the knowledge and facts specified in the preset cross-domain knowledge base.

2. The method according to claim 1, characterized in that The large language model includes attention layers constructed based on a self-attention mechanism, and determining the degree of association between each sequence unit generated by the large language model based on the input text and the input text includes: Performing weight analysis on each of the attention layers to determine an attention weight matrix, wherein each element in the attention weight matrix is ​​an attention weight value between the sequence unit and the input information for generating the sequence unit, and the attention weight value is positively correlated with the degree of association; The method further comprises: The sequence unit whose attention weight value is greater than a preset attention weight threshold is determined as the target sequence unit.

3. The method according to claim 1, characterized in that The determining the degree of association between each sequence unit generated by the large language model based on the input text and the input text includes: The information gain of each sequence unit is calculated by a preset Shannon information gain algorithm; wherein the information gain is positively correlated with the degree of association; The method further comprises: The sequence unit whose information gain is greater than a preset information gain threshold is determined as the target sequence unit.

4. The method according to claim 1, characterized in that The step of obtaining input text and inputting the input text into the large language model includes: Obtaining original input text, calling an external tool, and extracting key information contained in the original input text through the external tool, wherein the key information includes: keywords, entities, timestamps, and factual statements; The pre-processed key information after formatting the key information and the original input text are input into the large language model, and the large language model generates each of the sequence units.

5. The method according to claim 4, characterized in that The calling of the preset cross-domain knowledge base, verifying the reply text, and adjusting the model parameters of the large language model according to the verification result, includes: Acquiring verification information associated with the key information from the preset cross-domain knowledge base through the external tool; Performing factual verification on the reply text based on the factual statements in the key information and the verification information; If the verification is successful, the reply text is output; if the verification fails, negative sample data is generated and the model parameters of the large language model are adjusted using the negative sample data until the proportion of the generated negative sample data is less than a preset proportion threshold.

6. The method according to claim 1, characterized in that The generating of the reply text according to the target sequence unit whose correlation degree is greater than a preset correlation degree threshold comprises: Determine a prompt word template according to the input text, and generate the reply text based on the prompt word template; wherein the prompt word template includes a plurality of prompt words; If the input text corresponds to a multi-step reasoning task, each of the prompt words is used to: instruct the large language model to generate the reply text according to the input format and output format of each step specified by each of the prompt words, and the order of the steps indicated by each of the prompt words; If the input text corresponds to a fact verification reasoning task, each prompt word is used to instruct the large language model to generate the reply text according to the factual basis corresponding to each prompt word.

7. The method according to claim 1, characterized in that The calling of the preset cross-domain knowledge base to verify the reply text includes: The large language model evaluates the reply text in three dimensions, namely, correctness, interpretability, and practicality, based on the knowledge and facts provided by the preset cross-domain knowledge base, and outputs the evaluation score corresponding to each dimension; wherein the correctness means that the content of the output reply text complies with the provisions of the knowledge and facts, the interpretability means that the output reply text has a traceability basis, and the practicality means that the output reply text complies with the requirements of the actual application scenario; If one of the evaluation scores corresponding to each dimension is less than the set evaluation score threshold, it is determined that the reply text verification has failed.

8. A large language model decision optimization device, characterized in that: The device comprises: A text input module, used for acquiring input text, inputting the input text into a large language model, and determining the degree of association between each sequence unit generated by the large language model based on the input text and the input text; A text generation module, used for generating a reply text according to the target sequence units whose correlation degree is greater than a preset correlation degree threshold; A verification module, used to call a preset cross-domain knowledge base, verify the reply text, and adjust the model parameters of the large language model according to the verification result until the reply text conforms to the knowledge and facts specified in the preset cross-domain knowledge base; The text output module is used to output the reply text that conforms to the knowledge and facts specified in the preset cross-domain knowledge base.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing a program; wherein the program comprises instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

Images