Method, apparatus and related device for generating input sequence of large language model

Abstract

The application provides a method, device and related equipment for generating an input sequence of a large language model. The method comprises: obtaining question information of a user, and then obtaining K pieces of auxiliary information related to the question information; when the total length of the question information and the K pieces of auxiliary information is greater than an input sequence threshold of the large language model, determining the relevance of the K pieces of auxiliary information to the question information, determining M pieces of strong auxiliary information with the top M ranks of the relevance in the K pieces of auxiliary information according to the relevance, and determining N pieces of weak auxiliary information other than the M pieces of strong auxiliary information; compressing the N pieces of weak auxiliary information; when a first input sequence formed by the compressed N pieces of weak auxiliary information, the M pieces of strong auxiliary information and the question information is less than or equal to the input sequence threshold, inputting the first input sequence into the large language model for reasoning. The input sequence formed by the application can meet the input sequence threshold of the large language model, and can also improve the reasoning result quality of the large language model as much as possible.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, and related equipment for generating input sequences for large language models. Background Technology

[0002] Large Language Models (LLMs) are deep learning models trained on large amounts of text data, capable of understanding the meaning of natural language and generating natural language text. LLMs can be used to handle various natural language tasks, such as text classification, question answering, and chatbots, and are an important pathway to artificial intelligence.

[0003] The basic principle of current large language models is probability and statistics. When a user asks a question, the large language model generates a response based on the knowledge it has learned from the training dataset. However, in question-answering scenarios lacking relevant knowledge, the large language model may generate inaccurate or misleading answers. To enhance the generative capabilities of large language models, we can search for auxiliary information that helps the model understand and answer user questions. Then, we can concatenate the auxiliary information and the user question into an input sequence and input it into the large language model, enabling the model to reason and generate a more accurate response.

[0004] Because the above approach introduces additional auxiliary information based on the user's question, it increases the length of the input sequence. However, large language models have limitations on the length of the input sequence; too much auxiliary information will cause the input sequence to exceed the model's input sequence threshold, resulting in the model failing to function properly. Summary of the Invention

[0005] This application provides a method, apparatus, and related equipment for generating input sequences for a large language model. The resulting input sequences can satisfy the input sequence threshold of the large language model and improve the quality of the inference results of the large language model as much as possible.

[0006] In a first aspect, this application provides a method for generating an input sequence for a large language model. The method includes: acquiring a user's question information, and then acquiring K pieces of auxiliary information related to the question information; when the total length of the question information and the K pieces of auxiliary information is greater than the input sequence threshold of the large language model, determining the relevance between the K pieces of auxiliary information and the question information, and determining the M strong auxiliary information pieces with the highest relevance among the K pieces of auxiliary information, and the N weak auxiliary information pieces other than the M strong auxiliary information pieces, based on the relevance; compressing the N weak auxiliary information pieces; and when the first input sequence formed by the compressed N weak auxiliary information pieces, the M strong auxiliary information pieces, and the question information is less than or equal to the input sequence threshold, inputting the first input sequence into the large language model for inference.

[0007] It should be understood that obtaining auxiliary information related to the user's question is to form a semantically rich input sequence in the subsequent process. Compared to directly inputting the user's question (which has less semantic information) into the large language model for inference, the large language model can output higher-quality (more accurate and richer) inference results based on the above input sequence. Here, a total of K auxiliary information items are obtained. If the total length of these K auxiliary information items and the question information is less than or equal to the input sequence threshold of the large language model, then these K auxiliary information items and the question information can be directly input into the large language model for inference, thereby obtaining high-quality inference results. However, when the total length of these K auxiliary information items and the question information exceeds the input sequence threshold of the large language model, these K auxiliary information items and the question information cannot be directly input into the large language model as an input sequence and need to be appropriately processed.

[0008] The above solution involves first dividing the K pieces of auxiliary information into M strong auxiliary information and N weak auxiliary information based on their relevance to the question information. The M strong auxiliary information pieces are ranked by their relevance in the top M positions of the K auxiliary information pieces. The remaining N auxiliary information pieces, excluding the M strong auxiliary information pieces, are designated as N weak auxiliary information pieces. It should be understood that strong auxiliary information has a higher relevance to the question information than weak auxiliary information, indicating that strong auxiliary information is more conducive to generating high-quality inference results for the large language model. Strong auxiliary information contains more useful information (information that helps the large language model generate high-quality inference results). Therefore, we choose not to compress the strong auxiliary information, but only to appropriately compress the weak auxiliary information to maximize the generation capability of the large language model. When the first input sequence formed by the compressed N weak auxiliary information pieces, M strong auxiliary information pieces, and the question information is less than or equal to the input sequence threshold, the first input sequence can be input into the large language model for inference, resulting in high-quality inference results.

[0009] Based on the first aspect, in a possible implementation, the relevance of the K auxiliary information items is compared with the relevance threshold, and the M auxiliary information items with a relevance greater than the relevance threshold among the K auxiliary information items are taken as M strong auxiliary information items.

[0010] The above scheme divides K pieces of auxiliary information into strong and weak auxiliary information based on a relevance threshold. Auxiliary information with a relevance greater than the relevance threshold to the user's question is considered strong auxiliary information, while auxiliary information with a relevance less than or equal to the relevance threshold is considered weak auxiliary information. The relevance threshold can be set by the user or optimized through an algorithm; this application does not impose specific limitations on this.

[0011] Based on the first aspect, in a possible implementation, the K pieces of auxiliary information include contextual information and retrieval information of the question information in multi-turn question answering. The contextual information includes historical question information and / or historical reasoning results prior to the generation of the question information, and the retrieval information is the information retrieved from the knowledge base based on the question information. In this case, the most relevant retrieval information to the question information can be determined, and the M pieces of contextual information preceding the most relevant retrieval information can be considered as M strong auxiliary information.

[0012] In the above scheme, the K pieces of auxiliary information include contextual information of the question and retrieval information. Here, the most relevant retrieval information is used as the benchmark to divide strong and weak auxiliary information. The M pieces of contextual information whose relevance to the question information ranks before the most relevant retrieval information are considered as the M strong auxiliary information, and the remaining auxiliary information is considered as weak auxiliary information. This is equivalent to defaulting all retrieval information to weak auxiliary information, and using the most relevant retrieval information to determine the strong auxiliary information from the contextual information. This scheme does not require setting a relevance threshold (an inappropriate relevance threshold may lead to improper division of strong and weak auxiliary information, thus affecting the quality of the reasoning results of the large language model, and a fixed relevance threshold is difficult to adapt to different scenarios), and can more flexibly divide strong and weak auxiliary information.

[0013] Based on the first aspect, in a possible implementation, the value of M is set by the user. In this case, based on the relevance of the K auxiliary information and the value of M, the M auxiliary information with the highest relevance among the K auxiliary information can be determined as the M strong auxiliary information.

[0014] In the above scheme, the user can actively set the value of M, which is the number of strong auxiliary information. This means that the top M most relevant auxiliary information items out of the K auxiliary information items are considered strong auxiliary information. It should be understood that the larger the value of M, the more strong auxiliary information there is, meaning more uncompressed auxiliary information among the K auxiliary information items. This helps improve the quality of the inference results output by the large language model. However, M should not be set too large, lest the resulting input sequence exceed the input sequence threshold of the large language model.

[0015] Based on the first aspect, in a possible implementation, a first difference is determined between the total length of the question information and K auxiliary information and the input sequence threshold; the first length of the weak auxiliary information to be compressed is predicted based on the first difference and the historical compression rate; if the total length of N weak auxiliary information is greater than or equal to the first length, P weak auxiliary information with a total length greater than or equal to the first length are determined from the N weak auxiliary information, the P weak auxiliary information are compressed, and the compressed P weak auxiliary information and the information other than the P weak auxiliary information in the N weak auxiliary information are used as the compressed N weak auxiliary information.

[0016] The above scheme first determines how much the total length of the question information and K auxiliary information exceeds the input sequence threshold, i.e., the first difference. Then, based on the historical compression rate and the first difference, it estimates how much weak auxiliary information needs to be compressed to ensure that the subsequent input sequence meets the input sequence threshold, thus obtaining the estimated first length. The M strong auxiliary information items are not allowed to be compressed, while the N weak auxiliary information items are allowed to be compressed. If the total length of the N weak auxiliary information items is greater than or equal to the first length, it means that compressing the N weak auxiliary information items will likely meet the input sequence threshold requirement. Then, P weak auxiliary information items with a total length greater than or equal to the first length are determined from the N weak auxiliary information items. These P weak auxiliary information items are then compressed, and the compressed P weak auxiliary information items and the information from the N weak auxiliary information items excluding the P weak auxiliary information items are used as the compressed N weak auxiliary information items. Optionally, any or specific P items with a total length greater than or equal to the first length can be selected from the N weak auxiliary information items as the P weak auxiliary information items. The P weak auxiliary information items can be some or all of the information from the N weak auxiliary information items.

[0017] If the total length of the N weak auxiliary information is less than the first length mentioned above, it means that compressing the N weak auxiliary information is unlikely to meet the input sequence threshold requirement. It is necessary to remove some weak auxiliary information from the N weak auxiliary information. The removed weak auxiliary information will no longer be used to form the corresponding input sequence of the question information for this round of dialogue.

[0018] Based on the first aspect, in possible implementations, the relevance between the aforementioned P weak auxiliary information pieces and the question information is less than the relevance between the information other than the P weak auxiliary information pieces among the N weak auxiliary information pieces and the question information.

[0019] The above scheme prioritizes selecting weak auxiliary information with low relevance to the question information from N weak auxiliary information for compression. In other words, it selects the weak auxiliary information with the lowest relevance from the N weak auxiliary information for compression, while the remaining weak auxiliary information with higher relevance is not compressed and will be directly used to form the input sequence. This helps to improve the quality of the inference results output by the large language model.

[0020] Based on the first aspect, in a possible implementation, when the first input sequence formed by the compressed N weak auxiliary information, M strong auxiliary information, and question information is greater than the input sequence threshold, some weak auxiliary information is removed from the N weak auxiliary information to obtain Q weak auxiliary information; the Q weak auxiliary information is compressed; when the second input sequence formed by the compressed Q weak auxiliary information, M strong auxiliary information, and question information is less than or equal to the input sequence threshold, the second input sequence is input into the large language model for inference.

[0021] In the above scheme, when the first input sequence formed by compressing N weak auxiliary information cannot meet the input sequence threshold of the large language model, it is necessary to remove some weak auxiliary information to generate an input sequence that meets the input sequence threshold of the large language model. Here, we choose to remove some weak auxiliary information from the N weak auxiliary information (any weak auxiliary information can be removed, or weak auxiliary information with low relevance can be removed first; this application does not limit the number of weak auxiliary information removed each time), while retaining the M strong auxiliary information (without compression, directly used to form the input sequence). Then, the remaining weak auxiliary information is compressed, and a second input sequence is formed based on the compressed remaining weak auxiliary information, the M strong auxiliary information, and the question information. If the second input sequence meets (is less than or equal to) the input sequence threshold, it is input into the large language model for inference, and the large language model outputs the response information corresponding to the question information. If the second input sequence also does not meet (is greater than) the input sequence threshold, more weak auxiliary information is removed according to the same principle, and a corresponding input sequence is generated and compared with the input sequence threshold. This process continues until the generated input sequence meets the input sequence threshold, at which point the final generated input sequence is input into the large language model for inference.

[0022] Secondly, this application also provides an apparatus for generating input sequences of a large language model, including functional modules for performing any of the possible implementations in the first aspect.

[0023] Thirdly, this application also provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory. The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs operational steps as described in any possible embodiment of the first aspect.

[0024] Fourthly, this application also provides a chip system including a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, and the processor being used to perform operational steps as described in any possible embodiment of the first aspect.

[0025] Fifthly, this application also provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform operational steps as described in any possible embodiment of the first aspect.

[0026] Sixthly, this application also provides a computer program product containing instructions. When the aforementioned instructions are executed by a cluster of computing devices, the cluster of computing devices causes the cluster of computing devices to perform the operational steps of the method as described in any possible embodiment of the first aspect.

[0027] Based on the implementation schemes provided in the above aspects, this application can be further combined to provide more implementation schemes. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments are briefly introduced below.

[0029] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0030] Figure 2 This is a flowchart illustrating a method for generating input sequences for a large language model, as provided in an embodiment of this application.

[0031] Figure 3 This is a schematic diagram illustrating a relevance ranking provided in an embodiment of this application;

[0032] Figure 4 This is a schematic diagram of a process for selecting auxiliary information provided in an embodiment of this application;

[0033] Figure 5 This is a schematic diagram of the structure of an apparatus for generating input sequences of a large language model according to an embodiment of this application;

[0034] Figure 6 This is a schematic diagram of the structure of a computing device provided in this application;

[0035] Figure 7 This is a schematic diagram of the structure of a computing device cluster provided in this application;

[0036] Figure 8 This is a schematic diagram showing the connection between two computing devices provided in this application via a network. Detailed Implementation

[0037] To facilitate understanding of the technical solution of this application, an application scenario involved in this application will be introduced below.

[0038] Please see Figure 1 , Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application, including a user and a dialogue system. The user and the dialogue system can conduct one or more rounds of dialogue (a question and answer between the user and the dialogue system constitutes a round of dialogue).

[0039] Specifically, users can input questions into the dialogue system to describe their current problem or need (i.e., the current round of dialogue). This application does not specifically limit the method by which users input questions. For example, users can directly input natural language text as questions; users can input speech, and the dialogue system can generate natural language text based on speech recognition technology as questions; users can input images, and the dialogue system can extract natural language text based on image recognition technology as questions. It should be understood that any text input by a user, directly or indirectly, can be considered a question. However, from the user's perspective, the text input into the dialogue system is not limited to interrogative sentences; it can also be declarative sentences (such as commands, statements, emotional expressions, suggestions, and other types of natural language text).

[0040] When the system receives a user's question, it can search the knowledge base based on that question to obtain one or more search results. The knowledge base can be a corpus, database, webpage, table, or other data source; this application does not impose specific limitations on this. The knowledge base can be updated and maintained as needed, and this application also does not impose specific limitations on this. Figure 1 The location and number of knowledge bases mentioned are merely examples and do not constitute a limitation. In actual application scenarios, there may be one or more knowledge bases, and the knowledge bases may be located inside or outside the dialogue system. This application does not make any specific limitations.

[0041] It should be understood that the aforementioned knowledge base differs from the training dataset of the large language model. Retrieving data from the knowledge base can provide the large language model with new knowledge and information, helping it to better understand or respond to questions and enhancing its generative capabilities (i.e., improving the accuracy and quality of the generated responses). This application does not specify the retrieval method for the knowledge base. For example, a search can be conducted based on keywords in the question information to obtain matching keyword information; alternatively, the question information can be converted into a vector form, and the distance between the vector of the question information and the vectors of each piece of information in the knowledge base (such as Euclidean distance, cosine distance, or other distance metrics) can be calculated. Information with a distance less than or equal to a distance threshold can be used as retrieval information, or the i-th information with the smallest distance can be used as retrieval information, where i is a positive integer. This application does not specify i or the distance threshold.

[0042] The dialogue system can also obtain one or more dialogue contexts (stored within the dialogue system or elsewhere) corresponding to the question information. These dialogue contexts provide the communication history and context preceding the current round of dialogue, helping the large language model better understand or respond to the question information. Each dialogue context corresponds to a previous round of dialogue between the user and the dialogue system before the current round of dialogue (i.e., before the user asked the question in this round). Each dialogue context includes the user's historical question information and / or the large language model's inference results for that historical question information (i.e., historical response information). Different dialogue contexts correspond to different historical dialogues. It should be understood that there may be one or more previous rounds of historical dialogue before the current round of dialogue, or there may be no previous historical dialogue; that is, the current round of dialogue is the user's first round of dialogue with the dialogue system, in which case the dialogue context is not obtained.

[0043] Upon obtaining the user's current question information, along with the corresponding retrieval information and dialogue context, the dialogue system can construct an input sequence for a large language model (described in detail later) based on this information. Then, an input sequence that meets the input sequence threshold of the large language model is input into the model for inference. The large language model infers and outputs the response information corresponding to the question information based on the input sequence, and the dialogue system then displays the response information to the user. This application does not specifically limit the type of large language model.

[0044] Users can continue to communicate with the dialogue system in the same way as described above, which will not be elaborated on here.

[0045] based on Figure 1 The following describes the method for generating input sequences for large language models provided in this application.

[0046] Please see Figure 2 , Figure 2 This is a flowchart illustrating a method for generating input sequences for a large language model, provided in an embodiment of this application, including steps S201 to S205.

[0047] S201. Obtain user's question information.

[0048] Specifically, such as Figure 1 As shown, users can input questions into the dialogue system (i.e., the user's question in this round of dialogue) to describe their current problem or need (query). The method for users to input questions is explained above and will not be repeated here.

[0049] S202. Obtain K pieces of auxiliary information related to the question information.

[0050] The aforementioned auxiliary information related to the question information refers to information that is semantically relevant to the question information. Compared to directly inputting the question information as an input sequence into a large language model for reasoning, constructing the input sequence of the large language model based on the question information and auxiliary information here can enrich the semantic information represented by the input sequence, help the large language model better understand the question information, and thus enhance the generative ability of the large model, that is, improve the accuracy and quality of the reasoning results of the large language model (such as richer content).

[0051] The amount of auxiliary information related to the question information obtained here is K, where K is a positive integer greater than 1. K can be set by the user or be a default value, and this application does not make specific restrictions on it.

[0052] Optionally, X pieces of information can be obtained based on the question information, where X is greater than or equal to K mentioned above. Any one of these X pieces of information may or may not be relevant to the question information, thus requiring judgment and filtering. Then, the relevance of each of the X pieces of information to the user's question is obtained, and based on these relevance scores, K pieces of information relevant to the question information are selected from the X pieces of information as the aforementioned K auxiliary information. It should be understood that information in the X pieces of information that is irrelevant to the question information (such as a relevance score of 0, or other values ​​representing irrelevance) does not have the effect of enhancing the large language model's generation capabilities, and therefore is excluded from the K auxiliary information.

[0053] This application does not specify how to determine the relevance of each of the X pieces of information to the question. For example, a semantic relevance model can be used to determine the relevance of each piece of information to the question. A semantic relevance model is a computational model used to measure the semantic relevance between texts. It converts text into vector representations, which are typically embeddings that capture the semantic features of the text. Cosine similarity, Euclidean distance, or other distance metrics are then used to calculate the relevance between the vector representations of different texts. This application does not limit the specific type of semantic relevance model. For example, a language relevance model could be a bge-reranker-large model, a Cohere Rerank model, or other models capable of ranking relevance.

[0054] This application does not specifically limit the type of the aforementioned X pieces of information. For example, the aforementioned X pieces of information may include one or more of at least one dialogue context and at least one piece of retrieval information. Each dialogue context includes historical question information and / or the historical inference results of the large language model based on that historical question information. Each piece of retrieval information is retrieved from a knowledge base based on the question information, and the knowledge base is different from the training dataset of the large language model. Regarding the methods for obtaining dialogue context and retrieval information, please refer to the previous section... Figure 1 The relevant information will not be elaborated here.

[0055] The number of search results retrieved from the knowledge base can be set by the user or be a default value; this application does not impose specific limitations on this. For example, assuming the default number of search results is 8, the dialogue system will retrieve 8 search results from the knowledge base based on the question information. As another example, assuming the user sets the number of search results to 5, the dialogue system will only retrieve 5 search results from the knowledge base based on the question information.

[0056] This application does not specify the number of dialogue contexts to be obtained.

[0057] For example, assuming there have been 10 previous rounds of dialogue, we can obtain the historical questions and / or corresponding answers from these 10 rounds of dialogue. The historical questions and / or corresponding answers from a single historical dialogue constitute a dialogue context, thus allowing us to obtain 10 dialogue contexts corresponding to the question information.

[0058] For example, assuming there have been 15 rounds of historical dialogues prior to the current round, we can obtain historical question information and / or corresponding response information from the j-th round of historical dialogues closest to the current round. Here, j is a positive integer, which can be set by the user or be a default value; alternatively, j is determined by a dialogue context threshold, which can be set by the user or be a default value (e.g., set to 4,000 characters). This application does not specifically limit this threshold. The dialogue context threshold is used to limit the number j of dialogue context entries obtained in terms of information length, specifically satisfying the following relationship: the total length of information from the j-th round of historical dialogues closest to the current round is less than or equal to the dialogue context threshold, and the total length of information from the (j+1)-th round of historical dialogues closest to the current round is greater than the dialogue context threshold. Here, the information length of each round of historical dialogue refers to the total length of the historical question information in that historical dialogue and the historical inference results of the large language model for that historical question information.

[0059] S203. When the total length of the question information and the K auxiliary information is greater than the input sequence threshold of the large language model, determine the relevance between the K auxiliary information and the question information. Based on the relevance, determine the M strong auxiliary information with the highest relevance among the K auxiliary information, and the N weak auxiliary information other than the M strong auxiliary information.

[0060] It should be understood that, due to limited computing resources, memory resources, and / or the design of large language models, large language models have length limitations on input sequences; the length of the input sequence cannot exceed the input sequence threshold of the large language model. For example, the length of the input sequence of a large language model may be limited to a few thousand tokens.

[0061] Here, a total of K auxiliary information pieces related to the question information are obtained. If the total length of these K auxiliary information pieces and the question information is less than or equal to the input sequence threshold of the large language model, then these K auxiliary information pieces and the question information can be directly input into the large language model for inference, thereby obtaining high-quality inference results. However, when the total length of these K auxiliary information pieces and the question information exceeds the input sequence threshold of the large language model, these K auxiliary information pieces and the question information cannot be directly input into the large language model and need to be processed appropriately.

[0062] The specific processing method is as follows: First, based on the relevance of the K auxiliary information pieces to the question information, the K auxiliary information pieces are divided into M strong auxiliary information pieces and N weak auxiliary information pieces. The relevance of these M strong auxiliary information pieces is ranked in the top M positions of the K auxiliary information pieces. The N weak auxiliary information pieces, excluding the M strong auxiliary information pieces, are considered as weak auxiliary information. It should be understood that the strong auxiliary information has a higher relevance to the question information than the weak auxiliary information. This indicates that strong auxiliary information is more conducive to enabling the large language model to generate high-quality reasoning results (response information to the question information). Strong auxiliary information contains more useful information (information that helps the large language model generate high-quality reasoning results). Therefore, by dividing the strong and weak auxiliary information, we choose not to compress the strong auxiliary information, but only to appropriately compress the weak auxiliary information, in order to maximize the generation capability of the large language model.

[0063] Optionally, a semantic relevance model can be used to determine the relevance between each piece of auxiliary information and the question information. The linguistic relevance model can be referred to in the previous section. Therefore, this application can first use a semantic relevance model to evaluate the relevance between the question information and each piece of auxiliary information (measuring the relevance between each piece of auxiliary information and the question information at the linguistic level), and then sort the K pieces of auxiliary information according to the calculated relevance in ascending or descending order, thus obtaining the relevance ranking of the K pieces of auxiliary information. Furthermore, based on the relevance ranking, M strong auxiliary information pieces and N weak auxiliary information pieces can be determined from the K pieces of auxiliary information.

[0064] Optionally, a relevance threshold can be used to distinguish M strong auxiliary information and N weak auxiliary information from the K auxiliary information. The specific principle is as follows: The relevance of the K auxiliary information (referring to the relevance of each auxiliary information to the question information) is compared with the relevance threshold. The M auxiliary information with a relevance greater than the relevance threshold is regarded as M strong auxiliary information. It is not recommended to compress the strong auxiliary information, and it will be used directly to form the input sequence. All auxiliary information other than these M strong auxiliary information in the K auxiliary information is regarded as weak auxiliary information, thus determining N weak auxiliary information. Weak auxiliary information can be compressed so that the input sequence generated later can meet the input sequence threshold of the large language model.

[0065] The aforementioned relevance threshold can be set by the user or optimized and adjusted according to the algorithm; this application does not impose any specific limitations on this.

[0066] For example, users can set a relevance threshold according to their needs to distinguish between strong and weak auxiliary information among K pieces of auxiliary information. Users can also adjust the relevance threshold appropriately by referring to the accuracy and quality of the inference results of the large language model; this application does not impose specific limitations on this.

[0067] Optionally, the aforementioned K pieces of auxiliary information include contextual information and retrieval information of the question information in multi-turn question answering. Contextual information includes historical question information and / or historical reasoning results prior to the generation of the question information, and retrieval information is information retrieved from the knowledge base based on the question information. In this case, the most relevant retrieval information to the question information can be determined, and the M pieces of contextual information preceding the most relevant retrieval information can be used as M strong auxiliary information.

[0068] In the above scheme, the K pieces of auxiliary information include contextual information of the question and retrieval information. Here, the most relevant retrieval information is used as the benchmark to divide strong and weak auxiliary information. The M pieces of contextual information whose relevance to the question information ranks before the most relevant retrieval information are considered as the M strong auxiliary information, and the remaining auxiliary information is considered as weak auxiliary information. This is equivalent to defaulting all retrieval information to weak auxiliary information, and using the most relevant retrieval information to determine the strong auxiliary information from the contextual information. This scheme does not require setting a relevance threshold (an inappropriate relevance threshold may lead to improper division of strong and weak auxiliary information, thus affecting the quality of the reasoning results of the large language model, and a fixed relevance threshold is difficult to adapt to different scenarios), and can more flexibly divide strong and weak auxiliary information.

[0069] For example, suppose Figure 1The dialogue system retrieved four pieces of information from the knowledge base based on the user's question, denoted as retrieval information 1 to 4, and also obtained five dialogue contexts prior to the question's generation, denoted as dialogue context 1 to 5, for a total of 4 + 5 = 9 pieces of information (X = 9 at this point). Then, a semantic relevance model can be used to evaluate the relevance of these 9 pieces of information to the question, and the 9 pieces of information are sorted from highest to lowest relevance to obtain... Figure 3 The relevance ranking shown is as follows: Figure 3 The higher the information is in the question, the stronger its relevance to the question; the lower the information is in the question, the weaker its relevance to the question.

[0070] like Figure 3 As shown, dialogue context 4 out of the 9 pieces of information is irrelevant to the question information (the relevance value is 0 or other values ​​representing no relevance; this application does not limit this). The remaining information is relevant to the question information. Therefore, there are a total of 8 pieces of auxiliary information related to the question information (K=8 at this time). Among the 4 pieces of search information, the one with the strongest relevance to the question information is search information 2. Search information 2 can be used as the benchmark. The context information ranked before the benchmark is considered as strong auxiliary information, that is, dialogue context 1, dialogue context 5, and dialogue context 3 ranked before search information 2 are all considered as strong auxiliary information (M=3 at this time). Furthermore, the benchmark and the auxiliary information ranked after the benchmark are considered as weak auxiliary information, that is, search information 2 and search information 1, dialogue context 2, search information 3, and search information 4 ranked after search information 2 are all weak auxiliary information (N=5 at this time).

[0071] Optionally, the value of M is set by the user. In this case, based on the relevance of the K auxiliary information and the value of M, the M auxiliary information with the highest relevance among the K auxiliary information can be determined as the M strong auxiliary information.

[0072] In the above scheme, the user can actively set the value of M, which is the number of strong auxiliary information. This means that the top M most relevant auxiliary information items out of the K auxiliary information items are all considered strong auxiliary information. It should be understood that the larger the value of M, the more strong auxiliary information there is, i.e., the more uncompressed auxiliary information, which helps improve the quality of the inference results output by the large language model. However, M should not be set too large, lest the resulting input sequence exceed the input sequence threshold of the large language model.

[0073] For example, suppose 10 pieces of information are obtained based on the question (X=10). Then, a semantic relevance model is used to calculate the relevance of these 10 pieces of information to the user's question. Based on the relevance of these 10 pieces of information to the question, it is determined that two of these pieces of information are irrelevant to the question (e.g., the relevance value is 0), and the remaining 8 pieces of information are relevant to the question (the higher the relevance value, the stronger the relevance). Therefore, there are a total of 8 pieces of auxiliary information relevant to the question (K=8). Assuming the user sets M=3, based on the relevance of these 8 pieces of auxiliary information to the question, the top 3 pieces of auxiliary information are identified as strong auxiliary information, and the bottom 5 pieces of auxiliary information are identified as weak auxiliary information.

[0074] Optionally, in addition to setting the number of strong auxiliary information items M, users can also set the number of weak auxiliary information items N. Then, based on the value of N set by the user and the relevance of the K auxiliary information items, the N weak auxiliary information items with the lowest relevance ranking N are selected as N weak auxiliary information items, and the remaining auxiliary information items are selected as strong auxiliary information items.

[0075] Optionally, when the X pieces of auxiliary information include at least one retrieval message and multiple dialogue contexts, the retrieval message can be defaulted to weak auxiliary information. There is no need to use a semantic relevance model to calculate the relevance between the retrieval message and the question message. In this case, it is only necessary to use the semantic relevance model to obtain the relevance between the above multiple dialogue contexts and the question message. Then, based on the relevance and relevance threshold corresponding to these multiple dialogue contexts, or based on the relevance corresponding to these multiple dialogue contexts and the number of strong auxiliary information (or weak auxiliary information) set by the user, strong auxiliary information and weak auxiliary information are identified in these multiple dialogue contexts.

[0076] For example, suppose Figure 1 The dialogue system retrieved four pieces of information from the knowledge base based on the question information, denoted as retrieval information 1 to 4, and these four pieces of information were considered weak auxiliary information by default. The dialogue system also obtained five dialogue contexts prior to the generation of the current question information, denoted as dialogue context 1 to 5, for a total of 4 + 5 = 9 pieces of information. Then, a semantic relevance model can be used to calculate the relevance between the above five dialogue contexts and the question information. It is assumed that the relevance between the above five dialogue contexts and the question information indicates that all five dialogue contexts are relevant to the question information, i.e., there are no dialogue contexts unrelated to the question information.

[0077] Assuming the user sets N=6, meaning 6 weak auxiliary information needs to be identified, since the search information 1 to 4 above have already been identified as weak auxiliary information, it is also necessary to identify 2 weak auxiliary information from 5 dialogue contexts. The dialogue contexts that rank last two in relevance to the question information are identified as weak auxiliary information, and the remaining 3 dialogue contexts are identified as strong auxiliary information.

[0078] S204. Compress N weak auxiliary information items.

[0079] As described in the preceding steps, all K auxiliary information pieces are related to the question information. Among them, M strong auxiliary information pieces are strongly correlated with the question information, while N weak auxiliary information pieces are weakly correlated. Compared to weak auxiliary information, strong auxiliary information enriches the semantic information represented by the input sequence and is more effective in enhancing the generation capability of the large model. Therefore, the aforementioned M strong auxiliary information pieces are not suitable for compression to avoid losing too much semantic information related to the question information. These strong auxiliary information pieces will be directly used to generate the input sequence of the large language model, ensuring that the input sequence contains relatively rich semantic information and maximizing the generation capability of the large language model. The aforementioned N weak auxiliary information pieces are allowed to be compressed because they are weakly correlated with the question information. Therefore, even if these N weak auxiliary information pieces are compressed, too much semantic information related to the question information will not be lost, and this compression has a relatively small impact on the quality of the inference results of the large language model.

[0080] Regarding the compression method, this application does not specify any particular limitation. For example, weak auxiliary information can be summarized and compressed. Summarization compression refers to summarizing and outlining the semantics of the original text to shorten its length while retaining the key information. Document summarization models can be used to achieve this. A document summarization model is a model that generates a concise document summary / overview through extraction and abstraction techniques. It is often used to extract key information from long texts and refine the content of the original text. Regarding the type of document summarization model, this application does not specify any particular limitation. For example, extractive summarization models are one type of document summarization model that generates a summary by selecting important sentences from the original text; abstractactive summarization models are another type of document summarization model that summarizes the document by generating new sentences, producing a concise document overview. Therefore, this application can input auxiliary information into a document summarization model, which summarizes and compresses the auxiliary information, and then outputs a summary of the auxiliary information. This summary is the corresponding compressed information of the auxiliary information. The length of the summary is shorter than the length of the auxiliary information, but it still retains the key semantic information of the auxiliary information. For example, weak auxiliary information can be converted into an encoding vector using a compression model, where the length of the encoding vector is shorter than the length of the weak auxiliary information. Alternatively, assuming the weak auxiliary information is a piece of natural language text, this natural language text can be converted to another feature domain for compression encoding, and then the compressed encoding result can be converted back into text. This converted text can then be used as the compressed result of the weak auxiliary information.

[0081] Optionally, in order to maximize the generation capability of the large language model and meet the input sequence threshold of the large language model, P auxiliary information can be determined from the above N weak auxiliary information, and then the P auxiliary information can be compressed to obtain the first compressed information. Then, the first compressed information, the question information, and the remaining auxiliary information (the auxiliary information that has not been compressed among the K auxiliary information) are used to construct the input sequence.

[0082] Where P is less than or equal to N. The P weak auxiliary information can be any P items from the N weak auxiliary information; or, the relevance of the above P weak auxiliary information to the question information is less than the relevance of the information other than the P weak auxiliary information among the N weak auxiliary information to the question information. In other words, P weak auxiliary information with the lowest relevance to the question information are selected from the N weak auxiliary information. Here, we prioritize selecting weak auxiliary information with lower relevance to the question information for compression, which can enhance the generation capability of the large language model as much as possible.

[0083] Optionally, P weak auxiliary information can be determined from N weak auxiliary information in the following way: First, determine the first difference between the total length of the question information and K auxiliary information and the input sequence threshold. Then, predict the first length of the weak auxiliary information to be compressed based on the first difference and the historical compression rate. If the total length of N weak auxiliary information is greater than or equal to the first length, then determine P weak auxiliary information with a total length greater than or equal to the first length from the N weak auxiliary information, compress the P weak auxiliary information, and use the compressed P weak auxiliary information and the information other than the P weak auxiliary information in the N weak auxiliary information as the compressed N weak auxiliary information. Then, together with the question information and M strong auxiliary information, they are used to form the input sequence of the large language model.

[0084] The first length mentioned above can be calculated using the following formula (1):

[0085]

[0086] Where Ct is the first length, representing the length of the weak auxiliary information that needs to be compressed for prediction. query It is the length of the question message, L k,info L is the total length of the K auxiliary information items. max α is the input sequence threshold of the large language model, and α is the historical compression rate. α can be calculated using the following formula (2):

[0087]

[0088] Among them, L in L is the text length of the input document summary model. outThis is the text length output by the document compression model. Optionally, the length of weak auxiliary information (one or more pieces) related to historical questions in the historical dialogue (generated before the questions in the current round of the dialogue) can be used as L. in The length of the compressed information obtained by compressing the weak auxiliary information before the document summary model is taken as L. out Substituting into formula (2), the compression ratio of the historical dialogue can be calculated, and this compression ratio can be used as the historical compression ratio α. Alternatively, the compression ratio of multiple historical dialogues can be calculated in the same way as above, and then the average of the compression ratios of multiple historical dialogues can be used as the historical compression ratio α.

[0089] After calculating the first length using formulas (1) and (2) above, if the total length of the N weak auxiliary information is greater than or equal to the first length, it indicates that compressing the N weak auxiliary information can likely meet the input sequence threshold requirement of the large language model, without needing to reduce the number of weak auxiliary information used to generate the input sequence of the large language model. Then, P weak auxiliary information with a total length not less than the first length can be determined from the N weak auxiliary information, and these P weak auxiliary information can be compressed. It should be noted that the prediction here is based on the historical compression rate α. The predicted compressed information length after compressing P weak auxiliary information is Ct*(1-α). However, the compression rate of the document summary model is uncontrollable and not fixed. The actual compression rate of document summary compression for P weak auxiliary information may be equal to or not equal to α. This means that the actual compressed information length after compressing P weak auxiliary information may be equal to or not equal to Ct*(1-α). Therefore, it is necessary to verify whether the total length of the input sequence formed by the above compressed information, question information and other auxiliary information (including M strong auxiliary information and weak auxiliary information that was not compressed in N weak auxiliary information) exceeds the input sequence threshold. The above input sequence cannot be directly input into the large language model.

[0090] Optionally, if the total length of the N weak auxiliary information pieces is less than the first length, it indicates that compressing the N weak auxiliary information pieces is unlikely to meet the input sequence threshold. It is necessary to remove some of the weak auxiliary information pieces from the N pieces, and the removed weak auxiliary information will no longer be used to form the corresponding input sequence for the question information. This can be done by removing some weak auxiliary information pieces from the N pieces to obtain Y weak auxiliary information pieces. Then, the second difference between the total length of the user question, the Y weak auxiliary information pieces, and the M auxiliary information pieces and the input sequence threshold is calculated. Based on this second difference and the historical compression rate, the second length of the weak auxiliary information pieces that need to be compressed is predicted.

[0091] If the second length mentioned above is less than or equal to the total length of the Y weak auxiliary information pieces, it means that compressing the Y weak auxiliary information pieces can most likely meet the input sequence threshold requirement of the large language model. After compressing the Y weak auxiliary information pieces, if the total length of the input sequence formed by the Y weak auxiliary information pieces, M strong auxiliary information pieces, and the question information is less than or equal to the input sequence threshold, then this input sequence is input into the large language model for inference. If the total length of the input sequence is greater than the input sequence threshold, then it is necessary to further reduce the number of weak auxiliary information pieces used to form the input sequence.

[0092] If the second length mentioned above is greater than the total length of Y weak auxiliary information pieces, it indicates that compressing Y weak auxiliary information pieces is unlikely to meet the input sequence threshold of the large language model. It is necessary to further remove some weak auxiliary information, and then construct the input sequence based on the question information, the remaining weak auxiliary information, and M strong auxiliary information pieces. Only when the length of this input sequence is less than or equal to the input sequence threshold can the input sequence be input into the large language model for inference; otherwise, the number of weak auxiliary information pieces needs to be further reduced.

[0093] S205. When the first input sequence formed by the compressed N weak auxiliary information, M strong auxiliary information and question information is less than or equal to the input sequence threshold, the first input sequence is input into the large language model for inference.

[0094] This application does not impose specific limitations on how to form the first input sequence based on the compressed N weak auxiliary information, M strong auxiliary information, and question information. For example, the compressed N weak auxiliary information, M strong auxiliary information, and question information can be connected (splittered / fused) together to obtain the first input sequence. This application does not impose specific limitations on the connection method and order.

[0095] Optionally, when the first input sequence formed by the compressed N weak auxiliary information, M strong auxiliary information, and question information is greater than the input sequence threshold, some weak auxiliary information is removed from the N weak auxiliary information to obtain Q weak auxiliary information; the Q weak auxiliary information is compressed; when the second input sequence formed by the compressed Q weak auxiliary information, M strong auxiliary information, and question information is less than or equal to the input sequence threshold, the second input sequence is input into the large language model for inference.

[0096] In the above scheme, even if all N weak auxiliary information are compressed, the resulting first input sequence still cannot meet the input sequence threshold of the large language model. Therefore, it is necessary to reduce the number of auxiliary information used to construct the input sequence of the large language model. To ensure that the input sequence has relatively rich semantic information, M strong auxiliary information are still retained. All strong auxiliary information is used to construct the input sequence of the large language model. Here, only a portion of the weak auxiliary information is removed from the N weak auxiliary information. The removed weak auxiliary information will not be used to construct the input sequence of this round of dialogue. The remaining Q weak auxiliary information will be used to construct the input sequence.

[0097] This application does not specifically limit the method for determining the aforementioned Q weak auxiliary information items. For example, the number of weak auxiliary information items removed each time can be set to 2. In this case, only two weak auxiliary information items are removed from the N weak auxiliary information items each time, and the remaining weak auxiliary information items after removing these two are the aforementioned Q weak auxiliary information items. In addition to removing 2 weak auxiliary information items each time, the number of weak auxiliary information items removed each time can also be other values, and this application does not specifically limit this.

[0098] Optionally, the aforementioned Q weak auxiliary information items are randomly determined from N weak auxiliary information items, and can be any Q items from the N weak auxiliary information items. Alternatively, the relevance of the Q weak auxiliary information items to the question information is greater than the relevance of the other auxiliary information items in the N weak auxiliary information items (excluding these Q weak auxiliary information items) to the question information. In other words, based on the relevance of these N weak auxiliary information items, the top Q weak auxiliary information items with the highest relevance to the question information can be retained, and the remaining weak auxiliary information items are removed. That is, weak auxiliary information items with lower relevance are removed first, and weak auxiliary information items with higher relevance are retained as much as possible, so that the semantic information contained in the subsequently generated input sequence can be richer, enabling the large language model to input higher quality inference results.

[0099] If the total length of the second input sequence formed by the question information, M strong auxiliary information items, and Q compressed weak auxiliary information items is greater than the input sequence threshold, then, as described above, some weak auxiliary information can be further removed, and the remaining weak auxiliary information can be compressed. Determine whether the total length of the input sequence formed by the remaining compressed weak auxiliary information, the question information, and the M strong auxiliary information items is less than or equal to the input sequence threshold: if yes, then input the above input sequence into the large language model for inference; if no, then continue to reduce the number of weak auxiliary information items, continue compression, and compare the length of the resulting input sequence with the input sequence threshold. This will not be elaborated further here.

[0100] Next, let's combine... Figure 4 The process of selecting auxiliary information for generating the input sequence is illustrated with an example.

[0101] Please see Figure 4 , Figure 4 This is a flowchart illustrating the selection of auxiliary information provided in an embodiment of this application, including steps 1 to 15.

[0102] Step 1. Obtain the user's question information and obtain K pieces of auxiliary information related to the question information.

[0103] For details on how to obtain the above-mentioned question information and K pieces of auxiliary information, please refer to the descriptions in steps S201 and S202, which will not be repeated here.

[0104] Step 2. Determine if the total length of the question information and the K auxiliary information items does not exceed Lmax. If yes, proceed to step 15; otherwise, proceed to step 3.

[0105] The Lmax mentioned above is the input sequence threshold for a large language model.

[0106] Specifically, if the total length of the question information and the K auxiliary information does not exceed (is less than or equal to) Lmax, the question information and the K auxiliary information can be used directly to form the input sequence of the large language model. This input sequence can meet the input sequence threshold of the large language model, and then the input sequence is input into the large language model for reasoning to obtain the reasoning result.

[0107] If the total length of the question information and the K auxiliary information exceeds Lmax, then it is necessary to divide the K auxiliary information into strong auxiliary information and weak auxiliary information (i.e., step 3). Among them, it is not recommended to compress the strong auxiliary information, as it will be used directly to form the input sequence, while the weak auxiliary information is allowed to be compressed in order to form an input sequence that meets the input sequence threshold of the large language model.

[0108] Step 3. Determine M strong auxiliary information items and N weak auxiliary information items from the K auxiliary information items based on their relevance.

[0109] The method for determining the M strong auxiliary information items and N weak auxiliary information items can be found in the description of step S203, which will not be repeated here.

[0110] Step 4. Predict the length Ct of the weak auxiliary information that needs to be compressed.

[0111] As can be seen from the relevant introductions of formulas (1) and (2) in step S203, the length of weak auxiliary information Ct to be compressed (as described in the first length above) can be calculated based on the historical compression rate α, the length of the question information, the total length of K auxiliary information, and the input sequence threshold Lmax of the large language model. This will not be elaborated here.

[0112] Step 5. Determine whether Ct does not exceed the total length of the remaining weak auxiliary information. If yes, proceed to step 6; otherwise, proceed to step 12.

[0113] Specifically, among the aforementioned K auxiliary information items, there are N weak auxiliary information items. Before removing any weak auxiliary information items from these N weak auxiliary information items, these N weak auxiliary information items constitute the remaining weak auxiliary information. After removing some weak auxiliary information items from these N weak auxiliary information items, the remaining weak auxiliary information items are the ones that were not removed. If the aforementioned Ct does not exceed (is less than or equal to) the total length of the remaining weak auxiliary information, it means that compressing the remaining weak auxiliary information is likely sufficient to meet the input sequence threshold, and there is no need to reduce the number of weak auxiliary information items used to generate the large language model. Therefore, proceed to step 6.

[0114] If Ct exceeds (is greater than) the total length of the remaining weak auxiliary information, it means that compressing the remaining weak auxiliary information is unlikely to meet the input sequence threshold, and it is necessary to further reduce the number of weak auxiliary information. Therefore, the process jumps to step 12.

[0115] Step 6. Select weak auxiliary information with a length greater than or equal to Ct from the remaining weak auxiliary information and compress it to obtain the compressed information Ct*.

[0116] The weak auxiliary information with a length greater than or equal to Ct can be randomly selected from the remaining weak auxiliary information, or selected in order of increasing relevance (i.e., prioritizing weak auxiliary information with lower relevance). Then, the selected weak auxiliary information is compressed to obtain the compressed information Ct*.

[0117] Step 7. Determine if the total length of Ct*, the question information, the M strong auxiliary information, and the remaining uncompressed weak auxiliary information does not exceed Lmax. If yes, proceed to step 15; otherwise, proceed to step 8.

[0118] Wherein, Ct* is the result after a portion of the auxiliary information has been compressed, and the remaining uncompressed auxiliary information refers to the uncompressed weak auxiliary information in the remaining weak auxiliary information (i.e. the weak auxiliary information that has not been removed).

[0119] Specifically, the process ends if the total length of Ct*, the question information, the M strong auxiliary information items, and the remaining uncompressed weak auxiliary information does not exceed Lmax. Figure 4 The auxiliary information selection process then proceeds to form an input sequence based on the question information, Ct*, M strong auxiliary information items, and the remaining uncompressed weak auxiliary information items. This input sequence must satisfy the input sequence threshold Lmax of the large language model. If the total length of Ct*, the question information, the M strong auxiliary information items, and the remaining uncompressed weak auxiliary information items exceeds Lmax, the process jumps to step 8 to reduce the number of weak auxiliary information items.

[0120] Step 8. Reduce the amount of weak auxiliary information.

[0121] Specifically, reducing the amount of weak auxiliary information means removing some of the remaining weak auxiliary information. This removed weak auxiliary information will no longer be used to generate the input sequence for this round of dialogue. Optionally, some weak auxiliary information can be randomly selected from the remaining information, or it can be selected and removed in ascending order of relevance to the question information, i.e., prioritizing the removal of weak auxiliary information with lower relevance to the user's question.

[0122] Step 9. Determine if the number of remaining weak auxiliary information is greater than 0. If yes, proceed to step 10; otherwise, proceed to step 14.

[0123] Specifically, after reducing the number of weak auxiliary information in step 8, if the number of remaining weak auxiliary information is greater than 0, then proceed to step 10. If the number of remaining weak auxiliary information becomes 0, it means that weak auxiliary information cannot be introduced to construct the input sequence, and only strong auxiliary information and user questions can be used to form the input sequence of the large language model, so proceed to step 14.

[0124] Step 10. Compress the remaining weak auxiliary information to obtain the compressed information Cr#.

[0125] Step 11. Determine if the total length of Cr#, the question message, and the M strong auxiliary messages does not exceed Lmax. If yes, proceed to step 15; otherwise, proceed to step 8.

[0126] Specifically, if the total length of Cr#, the question information, and the M strong auxiliary information does not exceed Lmax, the process can end. Then, the input sequence for the large language model is formed based on the question information, the M strong auxiliary information, and Cr#. This input sequence satisfies the input sequence threshold Lmax, and is then input into the large language model for inference. If the total length of Cr#, the question information, and the M strong auxiliary information exceeds Lmax, the process jumps to step 8 to further reduce the number of weak auxiliary information.

[0127] Step 12. Reduce the amount of weak auxiliary information.

[0128] Regarding the method for reducing the amount of weak auxiliary information in step 12, please refer to the introduction in step 8 above, which will not be repeated here.

[0129] Step 13. Determine if the number of remaining weak auxiliary information is greater than 0. If yes, proceed to step 4; otherwise, proceed to step 14.

[0130] Specifically, if the number of remaining weak auxiliary information is greater than 0, you can jump to step 4 to re-predict the length Ct of the weak auxiliary information that needs to be compressed. At this time, you can refer to the following formula (3) for prediction.

[0131]

[0132] Among them, L query It is the length of the question message, L k,remain It is the total length of the auxiliary information in the K auxiliary information pieces, excluding the weak auxiliary information that was removed. In other words, L k,remain L is the total length of M strong auxiliary information items and the remaining weak auxiliary information items out of K auxiliary information items. max α is the input sequence threshold of the large language model, and α is the historical compression rate, which can be calculated by referring to formula (2) in step 203, and will not be elaborated here.

[0133] If the number of remaining weak auxiliary information is 0, proceed to step 14.

[0134] Step 14. Construct a structure that satisfies L based on some or all of the strong auxiliary information and the question information. max The process ends when the input sequence is received.

[0135] It should be understood that if the total length of the M strong auxiliary information and the question information is less than or equal to L... max Then, the input sequence can be directly formed based on the M strong auxiliary information and the question information, and this input sequence can satisfy L max The input sequence is then fed into a large language model for inference. If the total length of the M strong auxiliary information pieces and the question information is greater than L... max Then, select a portion of the strong auxiliary information from the M strong auxiliary information, and form a solution that satisfies L based on this portion of strong auxiliary information and the user's question. max The input sequence is then fed into a large language model for inference. Regarding the method of selecting some strong auxiliary information, it can be random selection or priority selection of strong auxiliary information with high relevance to the question information; this application does not specifically limit this. Alternatively, some strong auxiliary information can be compressed (this application does not limit the compression method), and then a model that satisfies L can be generated based on the compressed strong auxiliary information, the uncompressed strong auxiliary information, and the question information. max The input sequence is then fed into a large language model for inference.

[0136] Step 15. End the process.

[0137] In summary, the method for generating input sequences for a large language model provided in this application enriches the semantic information of the input sequence by acquiring auxiliary information related to the user's question and then generating the input sequence based on the question and auxiliary information. This helps the large language model better understand the question and enhances its generation capabilities, enabling it to generate more accurate and informative reasoning results (i.e., responses to the question in the current dialogue). Assuming a total of K pieces of auxiliary information related to the question are acquired, if the total length of the K pieces of auxiliary information and the question does not exceed the input sequence threshold of the large language model, an input sequence can be formed based on the K pieces of auxiliary information and the question, and then input into the large language model for reasoning to obtain high-quality reasoning results. However, when the total length of the K pieces of auxiliary information and the question exceeds the input sequence threshold, the K pieces of auxiliary information and the question cannot be directly input into the large language model as an input sequence and require appropriate processing.

[0138] The processing method in this application is as follows: First, based on the relevance between the K auxiliary information pieces and the question information, the K auxiliary information pieces are divided into M strong auxiliary information pieces and N weak auxiliary information pieces. It should be understood that the strong auxiliary information has a higher relevance to the question information than the weak auxiliary information, indicating that the strong auxiliary information is more conducive to the large language model generating high-quality inference results. Strong auxiliary information contains more useful information (information that helps the large language model generate high-quality inference results). Therefore, this application chooses not to compress the strong auxiliary information, but only to appropriately compress the weak auxiliary information to maximize the generation capability of the large language model. When the first input sequence formed by the compressed N weak auxiliary information pieces, M strong auxiliary information pieces, and the question information is less than or equal to the input sequence threshold, the first input sequence can be input into the large language model for inference, resulting in a high-quality inference result.

[0139] When the first input sequence formed by compressing N weak auxiliary information pieces fails to meet the input sequence threshold of the large language model, some weak auxiliary information can be appropriately removed to generate an input sequence that meets the threshold. To maximize the quality of the inference results of the large language model, weak auxiliary information with low relevance to the question information can be removed first. For details, please refer to the previous introduction, which will not be repeated here.

[0140] Please see Figure 5 This application also provides an apparatus 500 for generating input sequences of a large language model, including an acquisition module 501, a determination module 502, a compression module 503, and a transmission module 504.

[0141] The acquisition module 501 is used to: acquire user question information.

[0142] The acquisition module 501 is also used to: acquire K pieces of auxiliary information related to the question information.

[0143] The determination module 502 is used to: determine the relevance between the K auxiliary information and the question information when the total length of the question information and the K auxiliary information is greater than the input sequence threshold of the large language model; and determine the M strong auxiliary information with the highest relevance among the K auxiliary information and the N weak auxiliary information other than the M strong auxiliary information based on the relevance.

[0144] Compression module 503 is used to compress N weak auxiliary information items.

[0145] Transmission module 504: When the first input sequence formed by the compressed N weak auxiliary information, M strong auxiliary information and question information is less than or equal to the input sequence threshold, the first input sequence is input into the large language model for inference.

[0146] Optionally, the determining module 502 is specifically used to: compare the relevance of the K auxiliary information with the relevance threshold, and take the M auxiliary information with a relevance greater than the relevance threshold as the M strong auxiliary information.

[0147] Optionally, the K auxiliary information items include contextual information and retrieval information of the question information in multi-round question answering. The contextual information includes historical question information and / or historical reasoning results before the question information was generated, and the retrieval information is the information retrieved from the knowledge base based on the question information. The determining module 502 is specifically used to: determine the retrieval information with the strongest relevance to the question information among the retrieval information, and select the M contextual information items with the highest relevance to the question information as the M strong auxiliary information items.

[0148] Optionally, the value of M is set by the user, and the determining module 502 is specifically used to: determine the M auxiliary information items with the highest relevance among the K auxiliary information items as M strong auxiliary information items based on the relevance of the K auxiliary information items and the value of M.

[0149] Optionally, the compression module 503 is specifically used to: determine the first difference between the total length of the question information and K auxiliary information and the input sequence threshold; predict the first length of the weak auxiliary information to be compressed based on the first difference and the historical compression rate; if the total length of N weak auxiliary information is greater than or equal to the first length, then determine P weak auxiliary information with a total length greater than or equal to the first length from the N weak auxiliary information, compress the P weak auxiliary information, and use the compressed P weak auxiliary information and the information other than the P weak auxiliary information from the N weak auxiliary information as the compressed N weak auxiliary information.

[0150] Optionally, the relevance between the above P weak auxiliary information pieces and the question information is less than the relevance between the information other than the P weak auxiliary information pieces among the N weak auxiliary information pieces and the question information.

[0151] Optionally, the compression module 503 is further configured to: when the first input sequence formed by the compressed N weak auxiliary information, M strong auxiliary information, and question information is greater than the input sequence threshold, remove some weak auxiliary information from the N weak auxiliary information to obtain Q weak auxiliary information, and compress the Q weak auxiliary information. The transmission module 504 is further configured to: when the second input sequence formed by the compressed Q weak auxiliary information, M strong auxiliary information, and question information is less than or equal to the input sequence threshold, input the second input sequence into the large language model for inference.

[0152] It should be noted that the acquisition module 501, determination module 502, compression module 503, and transmission module 504 described above can be implemented in software, hardware, or a combination of both. For example, the implementation of the determination module 502 will be described below. Similarly, the implementation of the other modules can be referenced to the implementation of the determination module 502.

[0153] As an example of a software functional unit, module 502 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, module 502 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0154] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0155] As an example of a hardware functional unit, module 502 may include at least one computing device, such as a server. Alternatively, module 502 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0156] The multiple computing devices included in module 502 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in module 502 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in module 502 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0157] In other embodiments, the acquisition module 501, determination module 502, compression module 503, and transmission module 504 described above can all be used to perform... Figure 2 Any step in the process, the steps that each of these modules is responsible for implementing can be specified as needed, and implemented through them respectively. Figure 2 The different steps in the process enable the device 500 to perform all its functions in generating the input sequence of a large language model.

[0158] It should also be noted that the device 500 for generating the input sequence of the large language model can correspond to Figure 1 The dialog system in the middle can be used to perform Figure 2 For the specific steps and methods outlined in the document, please refer to [link / reference]. Figure 1 and Figure 2 The relevant descriptions are not repeated here. Figure 5 The apparatus 500 for generating input sequences of a large language model is merely an example of the above-described division of functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the apparatus can be divided into other different functional modules to complete all or part of the functions described above.

[0159] Please see Figure 6This application also provides a computing device 600, including a bus 602, a processor 604, a memory 606, and a communication interface 608. The processor 604, the memory 606, and the communication interface 608 communicate with each other via the bus 602. The computing device 600 can be a server, laptop, tablet, desktop computer, edge device, smartphone, smart screen, etc., and this application does not specifically limit it, nor does it limit the number of processors and memories in the computing device 600.

[0160] Bus 602 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 6 The bus 602 may be represented by a single line, but this does not mean that there is only one bus or one type of bus. The bus 602 may include a path for transmitting information between various components of the computing device 600 (e.g., memory 606, processor 604, communication interface 608).

[0161] Processor 604 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0162] Memory 606 may include volatile memory, such as random access memory (RAM). Processor 604 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0163] The memory 606 stores executable program code. The processor 604 executes the executable program code to implement... Figure 5 The functions of each module in the apparatus 500 for generating the input sequence of a large language model are implemented to achieve the purpose of this application. Figure 2 The operational steps in the method.

[0164] The communication interface 608 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 600 and other devices or communication networks.

[0165] As one possible implementation, the computing device 600 may also include a chip system, which includes a processor and a power supply circuit. The power supply circuit provides power to the processor, and the processor performs execution... Figure 2 The operational steps in the method are omitted here for brevity. The processor can be implemented using a CPU, or it can be implemented using computing devices or AI chips such as GPUs, DPUs, NPUs, XPUs, SoCs, offloading cards, and accelerator cards.

[0166] As one possible implementation, the computing device 600 may include multiple types of processors 604, meaning the computing device 600 is a heterogeneous device. For example, the computing device 600 may include a CPU and a GPU, and execution can be performed by at least one of the processors 604. Figure 2 The operational steps in the method are omitted here for brevity.

[0167] like Figure 7 As shown, this application also provides a computing device cluster, which includes at least one computing device 600. The memory 606 of one or more computing devices 600 in the computing device cluster may store the same memory for implementing… Figure 2 The instructions for the method.

[0168] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store information for implementing the aforementioned... Figure 2 The method's partial instructions. In other words, a combination of one or more computing devices 600 can jointly execute instructions for implementing... Figure 2 The instructions for the method.

[0169] It should be noted that the memory 606 in different computing devices 600 within the computing device cluster can store different instructions, each for execution. Figure 5 This refers to a portion of the functions of the device 500 that generates the input sequence for a large language model. Specifically, the instructions stored in the memory 606 of different computing devices 600 can implement the functions of one or more modules among the acquisition module 501, determination module 502, compression module 503, and transmission module 504.

[0170] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 8 One possible implementation is shown. For example... Figure 8 As shown, the two computing devices 600A and 600B are connected via a network. Specifically, they are connected to the network through the communication interfaces in each computing device. In this possible implementation, the memory 606 in computing device 600A stores instructions for executing the functions of the acquisition module 501 and the determination module 502. Meanwhile, the memory 606 in computing device 600B stores instructions for executing the functions of the compression module 503 and the transmission module 504.

[0171] It should be understood that Figure 8 The functions of computing device 600A shown can also be performed by multiple computing devices 600. Similarly, the functions of computing device 600B can also be performed by multiple computing devices 600.

[0172] This application also provides another type of computing device cluster. The interconnection relationships between the computing devices in this computing device cluster can be similarly referenced. Figure 8 The connection method of the computing device cluster. The difference is that the memory 606 in one or more computing devices 600 within this computing device cluster can store the same information used to implement the aforementioned... Figure 2 The instructions for the method.

[0173] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store information for implementing... Figure 2 The method's partial instructions. In other words, a combination of one or more computing devices 600 can jointly execute instructions for implementing... Figure 2 The instructions for the method.

[0174] This application also provides a chip system, including a processor and a power supply circuit, wherein the power supply circuit supplies power to the processor, and the processor is used to execute... Figure 2 The operational steps of the method.

[0175] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium capable of being stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that, when executed on a cluster of computing devices (including at least one computing device), cause the cluster of computing devices to perform... Figure 2 The operational steps in the method.

[0176] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product runs on a cluster of computing devices (including at least one computing device), it causes the cluster of computing devices to execute... Figure 2 The operational steps in the method.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A method for generating input sequences for a large language model, characterized in that, The method includes: Obtain user question information; Obtain K pieces of auxiliary information related to the question information; When the total length of the question information and the K auxiliary information is greater than the input sequence threshold of the large language model, the relevance between the K auxiliary information and the question information is determined. Based on the relevance, the top M strong auxiliary information in terms of relevance among the K auxiliary information are determined, along with the N weak auxiliary information excluding the M strong auxiliary information. The N weak auxiliary information items are compressed; When the first input sequence formed by the compressed N weak auxiliary information, the M strong auxiliary information, and the question information is less than or equal to the input sequence threshold, the first input sequence is input into the large language model for reasoning.

2. The method according to claim 1, characterized in that, The step of determining the top M strongly related auxiliary information items from the K auxiliary information items based on the relevance includes: The relevance of the K auxiliary information items is compared with the relevance threshold, and the M auxiliary information items with a relevance greater than the relevance threshold are taken as the M strong auxiliary information items.

3. The method according to claim 1, characterized in that, The K auxiliary information items include context information and retrieval information of the question information in the multi-round question-and-answer process. The context information includes historical question information and / or historical reasoning results before the question information is generated. The retrieval information is information retrieved from the knowledge base based on the question information. The step of determining the top M strongly related auxiliary information items from the K auxiliary information items based on the relevance includes: Identify the search information that is most relevant to the question information from the search information; The M contextual information items that rank higher in relevance to the query information than the most relevant search information are designated as the M strong auxiliary information items.

4. The method according to claim 1, characterized in that, The value of M is set by the user, and the determination of the top M strongly auxiliary information items in terms of relevance among the K auxiliary information items based on the relevance includes: Based on the relevance of the K auxiliary information items and the value of M, the M auxiliary information items with the highest relevance among the K auxiliary information items are determined as the M strong auxiliary information items.

5. The method according to any one of claims 1 to 4, characterized in that, The compression of the N weak auxiliary information items includes: Determine the first difference between the total length of the question information and the K auxiliary information items and the input sequence threshold; The first length of the weak auxiliary information that needs to be compressed is predicted based on the first difference and the historical compression rate. If the total length of the N weak auxiliary information is greater than or equal to the first length, then P weak auxiliary information with a total length greater than or equal to the first length are determined from the N weak auxiliary information, the P weak auxiliary information are compressed, and the compressed P weak auxiliary information and the information in the N weak auxiliary information excluding the P weak auxiliary information are taken as the compressed N weak auxiliary information.

6. The method according to claim 5, characterized in that, The relevance of the P weak auxiliary information pieces to the question information is less than the relevance of the other N weak auxiliary information pieces to the question information.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: When the first input sequence formed by the compressed N weak auxiliary information, the M strong auxiliary information, and the question information is greater than the input sequence threshold, some weak auxiliary information is removed from the N weak auxiliary information to obtain Q weak auxiliary information. The Q weak auxiliary information items are compressed; When the second input sequence formed by the compressed Q weak auxiliary information, the M strong auxiliary information, and the question information is less than or equal to the input sequence threshold, the second input sequence is input into the large language model for inference.

8. An apparatus for generating input sequences for a large language model, characterized in that, include: The acquisition module is used to acquire user question information; The acquisition module is also used to acquire K pieces of auxiliary information related to the question information; The determination module is used to determine the relevance between the K auxiliary information and the question information when the total length of the question information and the K auxiliary information is greater than the input sequence threshold of the large language model, and to determine the M strong auxiliary information with the highest relevance among the K auxiliary information and the N weak auxiliary information other than the M strong auxiliary information based on the relevance. A compression module is used to compress the N weak auxiliary information items; The transmission module is used to input the first input sequence into the large language model for inference when the first input sequence formed by the compressed N weak auxiliary information, the M strong auxiliary information and the question information is less than or equal to the input sequence threshold.

9. The apparatus according to claim 8, characterized in that, The determining module is specifically used for: The relevance of the K auxiliary information items is compared with the relevance threshold, and the M auxiliary information items with a relevance greater than the relevance threshold are taken as the M strong auxiliary information items.

10. The apparatus according to claim 8, characterized in that, The K auxiliary information items include context information and retrieval information of the question information in the multi-round question-and-answer process. The context information includes historical question information and / or historical reasoning results before the question information is generated. The retrieval information is information retrieved from the knowledge base based on the question information. The determining module is specifically used to: determine the search information with the strongest relevance to the question information among the search information, and take the M context information items with the highest relevance to the question information as the M strong auxiliary information items.

11. The apparatus according to claim 8, characterized in that, The value of M is set by the user, and the determining module is specifically used for: Based on the relevance of the K auxiliary information items and the value of M, the M auxiliary information items with the highest relevance among the K auxiliary information items are determined as the M strong auxiliary information items.

12. The apparatus according to any one of claims 8 to 11, characterized in that, The compression module is specifically used for: Determine the first difference between the total length of the question information and the K auxiliary information items and the input sequence threshold; The first length of the weak auxiliary information that needs to be compressed is predicted based on the first difference and the historical compression rate. If the total length of the N weak auxiliary information is greater than or equal to the first length, then P weak auxiliary information with a total length greater than or equal to the first length are determined from the N weak auxiliary information, the P weak auxiliary information are compressed, and the compressed P weak auxiliary information and the information in the N weak auxiliary information excluding the P weak auxiliary information are taken as the compressed N weak auxiliary information.

13. The apparatus according to claim 12, characterized in that, The relevance of the P weak auxiliary information pieces to the question information is less than the relevance of the other N weak auxiliary information pieces to the question information.

14. The apparatus according to any one of claims 8 to 13, characterized in that, The compression module is further configured to: when the first input sequence formed by the compressed N weak auxiliary information, the M strong auxiliary information, and the question information is greater than the input sequence threshold, remove some weak auxiliary information from the N weak auxiliary information to obtain Q weak auxiliary information, and compress the Q weak auxiliary information; The transmission module is further configured to: input the second input sequence into the large language model for inference when the second input sequence formed by the compressed Q weak auxiliary information, the M strong auxiliary information and the question information is less than or equal to the input sequence threshold.

15. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1 to 7.

16. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster performs the method as described in any one of claims 1 to 7.

17. A computer-readable storage medium, characterized in that, It includes computer program instructions that, when run by a computing device cluster, cause the computing device cluster to perform the method as described in any one of claims 1 to 7.

Images