Data processing based on large language model

Abstract

A data processing method based on a large language model, and a device, a medium and a program product. The large language model can perform processing on the basis of data acquired from a storage space, wherein the storage space comprises a plurality of data queues, and data stored in different data queues is obtained on the basis of historical data of different data volumes acquired by the large language model. The method comprises: acquiring data from a plurality of data queues; inputting the data acquired from the plurality of data queues as part of a prompt into a large language model for processing; and acquiring a processing result of the large language model.

Description

Data processing based on large language model TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, and in particular, to data processing based on a large language model. BACKGROUND

[0002] Large language model external memory refers to a mechanism or system attached to the model, enabling the large language model to access, store and retrieve data beyond its internal parameter information. This memory attachment enhances the memory capacity of the large language model, allowing it to utilize additional information such as historical conversations, specific facts or updated knowledge when processing requests, without the need for the large language model itself to remember all details or undergo costly retraining. This approach helps to address the knowledge limitations and context understanding problems of large language models, making them perform more intelligently and adaptively. In related technologies, the form of external memory is relatively fixed, and the use effect is poor. SUMMARY

[0003] In a first aspect, the embodiments of the present disclosure provide a data processing method based on a large language model, the large language model being capable of processing based on data obtained from a storage space, the storage space including a plurality of data queues, the data stored in different data queues being obtained based on historical data of different data amounts obtained by the large language model; the method comprising: obtaining data in the plurality of data queues; inputting the data obtained from the plurality of data queues as part of a prompt word to the large language model for processing; obtaining a processing result of the large language model.

[0004] In a second aspect, the embodiments of the present disclosure provide a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the method of any of the embodiments of the present disclosure.

[0005] In a third aspect, the embodiments of the present disclosure provide a computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the method of any of the embodiments of the present disclosure when executing the program.

[0006] In a fourth aspect, the embodiments of the present disclosure provide a computer program product comprising a computer program, the computer program being executed by a processor to implement the method of any of the embodiments of the present disclosure.

[0007] In the embodiments of the present disclosure, the large language model is externally connected with multiple data queues, and the large language model can simultaneously obtain data from the multiple data queues. Since the data stored in different data queues is obtained based on different data volume historical data obtained by the large language model, and the different data volume historical data can reflect different memory capabilities of the large language model, the data in different data queues can be used to enhance different memory capabilities of the large language model respectively, thereby improving the use effect of the memory external connection.

[0008] It should be understood that the general description above and the following detailed description are merely exemplary and explanatory, rather than limiting the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0009] The accompanying drawings, which are incorporated into the specification and constitute a part of the present disclosure, illustrate embodiments consistent with the present disclosure, and together with the specification, serve to explain the technical solutions of the present disclosure.

[0010] FIG. 1 is a schematic diagram of an application scenario of an embodiment of the present disclosure.

[0011] FIG. 2 is a flowchart of a data processing method based on a large language model according to an embodiment of the present disclosure.

[0012] FIG. 3 is a schematic diagram of a multi-turn dialogue of a large language model according to an embodiment of the present disclosure.

[0013] FIG. 4A, FIG. 4B and FIG. 4C are schematic diagrams of data volumes obtained from respective data queues according to embodiments of the present disclosure.

[0014] FIG. 5 is a schematic diagram of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0016] The terms used in the present disclosure are merely for the purpose of describing particular embodiments and are not intended to limit the present disclosure. The singular forms "a", "an" and "the" used in the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein means and includes any or all possible combinations of one or more associated listed items. In addition, the term "at least one" herein means any combination of any one or more of the plurality.

[0017] It should be understood that, although the terms first, second, third, etc. can be employed in this disclosure to describe various information, these information should not be limited to these terms. These terms are only used to differentiate one piece of information from another piece of information. For example, without departing from the scope of this disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0018] In order to better understand the technical solutions in the embodiments of the present disclosure by those skilled in the art, and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more apparent and easy to understand, the technical solutions in the embodiments of the present disclosure are further described in detail below with reference to the drawings.

[0019] A large language model (LLM) is an artificial intelligence technology that can understand and generate natural language close to human level by learning a large amount of text data. This model uses deep learning methods to train on a large amount of corpus in order to perform various language tasks such as answering questions, writing articles, translating languages and conducting conversations. In order to enhance the memory capacity of the large language model, a large language model external memory can be used, that is, a storage space is configured for the large language model, so that the large language model can access, store and retrieve data beyond its internal parameter information. Figure 1 shows a schematic diagram of a specific scenario, which includes a large language model 10, a storage space 20 and an interaction unit 30. Among them, the interaction unit 30 is used for the user to interact with the large language model 10, for example, the user can input prompt information to the large language model 10 through the interaction unit 30 to prompt the large language model 10 to perform a specified task. The large language model 10 can access the storage space 20 and perform the task based on the data obtained from the storage space 20 to obtain a task execution result. The data stored in the storage space 20 can be obtained based on the historical data processed by the large language model 10. After the large language model 10 obtains the task execution result, it can send the task execution result to the interaction unit 30 for display, for example, the interaction unit 30 can include a display interface for visual display of the task execution result; or the interaction unit 30 includes an audio output module for audio playing of the task execution result. In addition, the large language model 10 can also input the user's input data (such as the aforementioned prompt information), its own output data (such as the aforementioned task execution result) and / or other information into the storage space 20 for storage, so as to be used by the large language model 10 in subsequent task execution.

[0020] For the convenience of understanding and description, the scheme of the embodiments of the present disclosure is exemplified below in combination with the application scenario shown in FIG. 1. It can be understood that the above-mentioned application scenario is only an exemplary description, and is not intended to limit the present disclosure.

[0021] In the related art, the form of memory externalization is relatively fixed. For example, in the related art, a large language model can only obtain data from a single data queue, and the data queue only stores data related to long-term tasks. Therefore, the use effect of the memory externalization function in the related art is poor.

[0022] Based on this, the embodiments of the present disclosure externally connect multiple data queues to the large language model 10, and the large language model 10 can simultaneously obtain data from multiple data queues. Since the data stored in different data queues is obtained based on different data volume history data obtained by the large language model 10, and the different data volume history data can reflect different memory capabilities of the large language model 10, the data in different data queues can be used to enhance different memory capabilities of the large language model 10, thereby improving the use effect of memory externalization.

[0023] Referring to FIG. 2, the embodiments of the present disclosure provide a data processing method based on a large language model 10. The large language model 10 can process based on data obtained from a storage space 20. The storage space 20 includes multiple data queues. The data stored in different data queues is obtained based on different data volume history data obtained by the large language model 10. The method comprises the following steps: step S12: obtaining data in multiple data queues; step S14: inputting the data obtained from the multiple data queues as part of the prompt word to the large language model 10 for processing; and step S16: obtaining the processing result of the large language model 10.

[0024] In step S12, a plurality of data queues can be pre-established in the storage space 20, and the data stored in the plurality of data queues are respectively based on historical data of different data amounts obtained by the large language model 10. The historical data includes input data and / or output data of the large language model 10. For example, the historical data (such as input data and output data) obtained by the large language model 10 can be processed and stored in the plurality of data queues. The processing can be to extract key information from the input data and output data of the large language model 10. The key information stored in the plurality of data queues has different information densities, i.e., the accuracy of the key information stored in the plurality of data queues is different. Since the above processing process is a summarization process of data, a part of non-key information is lost in the summarization process, so as to retain important information under the premise of reducing data amount, remove redundant, irrelevant or noise information, make the final output more refined and value concentrated, and thus improve the information quality. Therefore, the information density of the key information can represent the information loss rate of the key information relative to the original data (i.e., the historical data obtained by the large language model 10). The lower the information loss rate, the higher the information density, and the higher the information accuracy. In addition, the information density can also be represented by the compression rate of the information. The lower the compression rate, the higher the information density, and the higher the information accuracy. In addition, the information density can also be represented by the redundancy of the information. The lower the redundancy, the higher the information density, and the higher the information accuracy.

[0025] The key information can include but is not limited to at least one of the following: background information, process information and detailed information extracted by the large language model 10 from the historical data. The background information is used to describe the background corresponding to the historical data, the process information is used to describe the specific process adopted by the large language model 10 when performing a task, and the detailed information is used to describe the task details of the large language model 10 when performing a task.

[0026] In some embodiments, the plurality of data queues correspond one-to-one to a plurality of preset data amounts.

[0027] For example, the plurality of data queues include a first data queue and a second data queue. The data stored in the first data queue is based on historical data of a first data amount obtained by the large language model 10, and the data stored in the second data queue is based on historical data of a second data amount obtained by the large language model 10, and the first data amount is greater than the second data amount.

[0028] For example, the plurality of data queues include a first data queue, a second data queue, and a third data queue. The data stored in the first data queue is based on historical data of a first data amount obtained by the large language model 10, the data stored in the second data queue is based on historical data of a second data amount obtained by the large language model 10, and the data stored in the third data queue is based on historical data of a third data amount obtained by the large language model 10. The first data amount, the second data amount, and the third data amount decrease in turn.

[0029] In other examples, the total number of the plurality of data queues can also be greater than 3, and the disclosure will not be repeated here.

[0030] In some embodiments, for any target data queue in the plurality of data queues, the target data amount corresponding to the target data queue can be determined based on the number of historical conversations by the large language model 10. Where one round of conversation is defined as the large language model 10 obtaining input data and obtaining output data based on the input data. As shown in FIG. 3, in the argument conversation, the user inputs a question to the large language model 10, and the large language model 10 gives an answer to the question. In the example shown in FIG. 3, the total number of conversations is 3 rounds. It can be understood that there can be more rounds of conversation in actual application.

[0031] In the example where the target data amount corresponding to the target data queue is determined based on the number of historical conversations by the large language model 10, the target data amount corresponding to the target data queue can be the total data amount of the input data and the output data obtained in the process of N rounds of historical conversations by the large language model 10. Where N is a positive integer set in advance. Taking the total number of the plurality of data queues as an example, the total data amount of the input data and the output data included in 32 rounds of conversations can be determined as the first data amount corresponding to the first data queue, the total data amount of the input data and the output data included in 8 rounds of conversations can be determined as the second data amount corresponding to the second data queue, and the total data amount of the input data and the output data included in 4 rounds of conversations can be determined as the third data amount corresponding to the third data queue. Of course, the values here are only exemplary and not intended to limit the disclosure.

[0032] In other embodiments, for any target data queue in the plurality of data queues, the target data amount corresponding to the target data queue can be determined based on the total number of word pieces in the input data and the output data obtained in the historical conversations by the large language model 10.

[0033] In the example in which the target data quantity corresponding to the target data queue is determined based on the total number of tokens in the input data and the output data obtained by the large language model 10 in the historical conversation, the target data quantity corresponding to the target data queue can be a preset number of tokens obtained by the large language model 10, and the preset number of tokens refers to that the total number of tokens in the input data and the output data obtained by the large language model 10 reaches a preset number. The token refers to the smallest unit of text segmentation, which is usually a word, a subword, a character, or a combination thereof. Tokenization is a preprocessing step that converts raw text into a format that can be processed by the large language model 10, and each token represents an element in the text for input and output of the large language model 10. By converting the text into a token sequence, the large language model 10 can understand and generate new text content.

[0034] In some embodiments, the maximum input data quantity (denoted as maxContext) of the large language model 10 can be obtained, and the preset data quantity can be determined based on the product of the maximum input data quantity maxContext and a preset value, and the preset value is a constant greater than 0 and less than or equal to 1. Taking the total number of the plurality of data queues as 3 as an example, for the first data queue, the preset value can be 1, so that the first data quantity corresponding to the first data queue can be determined as the maximum input data quantity maxContext of the large language model 10; for the second data queue, the preset value can be 1 / 4, so that the second data quantity corresponding to the second data queue can be determined as 1 / 4 of the maximum input data quantity maxContext of the large language model 10; for the third data queue, the preset value can be 1 / 16, so that the third data quantity corresponding to the first data queue can be determined as 1 / 16 of the maximum input data quantity maxContext of the large language model 10. Of course, the values herein are only exemplary and are not intended to limit the disclosure.

[0035] In other examples, the target data quantity corresponding to the target data queue can also be determined based on the time length of data obtained by the large language model 10. Taking the total number of the plurality of data queues as 3 as an example, the first data quantity can be the data quantity obtained by the large language model 10 in 3 days, the second data quantity can be the data quantity obtained by the large language model 10 in 1 day, and the third data quantity can be the data quantity obtained by the large language model 10 in 1 hour.

[0036] In addition to the above-mentioned various ways of determining the target data quantity, other ways of determining the target data quantity can also be used, which will not be described here.

[0037] In different application scenarios, different ways can be selected to determine the target data amount. For example, in the case where the maximum input length of the large language model 10 is greater than the preset data amount threshold, the target data amount corresponding to the target data queue can be determined based on the number of rounds of historical conversations performed by the large language model 10. In the case where the maximum input length of the large language model 10 is less than or equal to the preset data amount threshold, the target data amount corresponding to the target data queue can be determined based on the total number of tokens in the input data and the output data obtained when the large language model 10 performs historical conversations.

[0038] Since the length of the input data obtained by the large language model 10 is not fixed, for the large language model 10 with a short maximum input length (i.e., weak input capability), determining the target data amount based on the total number of tokens in the input data and the output data of the large language model 10 can reduce the case where the data amount of the input data exceeds the input capability of the large language model 10. For the large language model 10 with a long input length (i.e., strong input capability), the target data amount can be determined based on the number of conversation rounds, and even if the number of conversation rounds is large, the data amount of the input data is not likely to exceed the input capability of the large language model 10. Obtaining the data stored in the data queue based on multiple rounds of conversations facilitates the large language model 10 to summarize the content of multiple rounds of conversations.

[0039] In the example where the plurality of data queues correspond one-to-one to the plurality of preset data amounts, the historical data of the large language model 10 can be obtained. If the data amount of the currently obtained historical data reaches the target data amount corresponding to any one of the target data queues in the plurality of data queues, the data block corresponding to the target data queue can be generated based on the historical data that has been obtained and has not been written into the target data queue, and the data block corresponding to the target data queue can be written into the target data queue.

[0040] For example, in a case where the target data amount is determined based on the number of dialogue turns, assuming that the multiple data queues include a first data queue, a second data queue, and a third data queue, and the corresponding target data amounts are respectively a data amount included in 32 turns of dialogues, a data amount included in 8 turns of dialogues, and a data amount included in 4 turns of dialogues, the most recent 32 turns of data that have been acquired and not written into the first data queue can be written into the first data queue when the current acquired data amount of the historical data reaches the data amount included in 32 turns of dialogues. Similarly, the most recent 8 turns of data that have been acquired and not written into the second data queue can be written into the second data queue when the current acquired data amount of the historical data reaches the data amount included in 8 turns of dialogues; and the most recent 4 turns of data that have been acquired and not written into the third data queue can be written into the third data queue when the current acquired data amount of the historical data reaches the data amount included in 4 turns of dialogues. Then, the above process can be repeated, so as to write data into the first data queue, the second data queue, and the third data queue respectively in a manner that the writing operation is performed once to the first data queue every 32 turns of data acquired, the writing operation is performed once to the second data queue every 8 turns of data acquired, and the writing operation is performed once to the third data queue every 4 turns of data acquired.

[0041] For another example, in a case where the target data amount is determined based on the total number of word pieces in the input data and the output data acquired when the large language model 10 processes the historical dialogues, assuming that the multiple data queues include a first data queue, a second data queue, and a third data queue, and the corresponding target data amounts are respectively maxContext, 1 / 4*maxContext, and 1 / 16*maxContext, the data with a length of maxContext that have been acquired and not written into the first data queue can be written into the first data queue when the current acquired data amount of the historical data reaches maxContext. Similarly, the data with a length of 1 / 4*maxContext that have been acquired and not written into the second data queue can be written into the second data queue when the current acquired data amount of the historical data reaches 1 / 4*maxContext; and the data with a length of 1 / 16*maxContext that have been acquired and not written into the third data queue can be written into the third data queue when the current acquired data amount of the historical data reaches 1 / 16*maxContext.

[0042] Since the data stored in different data queues is based on different amounts of historical data obtained by the large language model 10, the roles and focuses of the data in different data queues are also different. If the data in a data queue (for example, the first data queue in the foregoing embodiment) is based on historical data with a large amount of data, the data in this data queue can reflect the long-term memory of the large language model 10 and can be used as background information for the large language model 10 to perform the current task. If the data in a data queue (for example, the second data queue in the foregoing embodiment) is based on historical data with a small amount of data, the data in this data queue can reflect the medium-term memory of the large language model 10 and can be used as process information for the current task performed by the large language model 10. If the data in a data queue (for example, the third data queue in the foregoing embodiment) is based on historical data with a smaller amount of data, the data in this data queue can reflect the short-term memory of the large language model 10 and can be used as task detail information for the current task performed by the large language model 10.

[0043] When obtaining the data in the plurality of data queues, the data in the plurality of data queues can be obtained according to the data amount proportions respectively corresponding to the plurality of data queues. Specifically, the maximum input length of the large language model 10 can be determined, and the amount of data obtained from each data queue can be determined based on the maximum input length and the preset amount proportion. For example, assuming that the maximum input length of the large language model 10 is M, the plurality of data queues include a first data queue, a second data queue, and a third data queue, and the data amount proportions of the three data queues are 1:1:1, then the amount of data obtained from the first data queue, the second data queue, and the third data queue is M / 3. For ease of understanding and description, assuming that the data amount of each data block in the same data queue is the same (which can also be different in actual application), if the data amount of a data block in the first data queue, the second data queue, and the third data queue is x, y, and z (any two of x, y, and z can be the same or different), then the number of data blocks obtained from the first data queue is M / 3x, the number of data blocks obtained from the second data queue is M / 3y, and the number of data blocks obtained from the third data queue is M / 3z.

[0044] The data amount proportion can be related to the task performed by the large language model 10. The focus and needs of different types of tasks can be different, and therefore, determining the data amount proportion for obtaining data from each data queue according to the task performed by the large language model 10 can make the data obtaining manner match the focus and needs of the task currently performed by the large language model 10, thereby improving the task execution effect. Still taking the example in which the plurality of data queues include the first data queue, the second data queue, and the third data queue, and combining FIG. 4A, FIG. 4B, and FIG. 4C for example illustration. In FIG. 4A, FIG. 4B, and FIG. 4C, each square represents a data block, and the gray square represents a data block expected to be taken from the corresponding data queue.

[0045] As shown in FIG. 4A, when the task currently performed by the large language model 10 pays more attention to the background information of the task, more data blocks can be obtained from the first data queue. For example, when the data migration is assisted to be completed by the large language model 10, since the data migration process can last for a long time and go through many steps, at this time, the large language model 10 can need to determine whether the data migration of the previous stage is successful according to the migration results of a long period of time, so as to determine how to proceed with the data migration of the current stage. At this time, the large language model 10 can pay more attention to the background information of the task, and therefore, more data blocks can be obtained from the first data queue.

[0046] As shown in FIG. 4B, when the task currently performed by the large language model 10 pays more attention to the flow of the task process and focuses on which step of the task execution, more data can be obtained from the second data queue. For example, when the transaction processing is performed, the processing flow can be stuck at a certain step, and the large language model 10 can analyze the transaction processing flow and give suggestions for improving the transaction processing flow. At this time, the large language model pays more attention to the flow of the task process, and therefore, more data blocks can be obtained from the second data queue.

[0047] As shown in FIG. 4C, when the task currently performed by the large language model 10 pays more attention to the specific needs, detailed information, and results of the task, more data blocks can be obtained from the third data queue. For example, during the running of an application, if an exception occurs, the application will record an exception log. The large language model 10 can process the exception log, so as to troubleshoot the cause of the application running exception. At this time, the task performed by the large language model 10 tends to focus on the specific details in the running process of the application, and therefore, more data blocks can be obtained from the third data queue.

[0048] In some embodiments, the current input data can be input by a user to the large language model 10, and by performing inference on the current input data by the large language model 10, the semantics of the current input data can be extracted to fully understand the user's intention represented by the current input data, so as to determine the influence degree of the data in each data queue on the task currently performed by the large language model 10. For example, when the user's intention is to determine the task flow, the data in the second data queue has a higher influence degree on the task currently performed by the large language model 10; when the user's intention is to determine the specific details of the task, the data in the first data queue has a higher influence degree on the task currently performed by the large language model 10.

[0049] In some embodiments, the maximum input length of the large language model 10 can also be obtained, and based on the maximum input length and the data amount proportions respectively corresponding to the plurality of data queues, the initial data amount of the data taken from the plurality of data queues can be obtained. For any one data queue, based on the proportion factor corresponding to the data queue and the initial data amount of the data taken from the data queue, the target data amount of the data taken from the data queue can be determined, and based on the target data amount, the data in the data queue can be obtained. Wherein, the proportion factors respectively corresponding to the plurality of data are all constants greater than 0 and less than or equal to 1, and the proportion factor corresponding to at least one data queue of the plurality of data queues is less than 1.

[0050] Taking the first data queue in the foregoing embodiments as an example, it is still assumed that the data amount proportions of the first data queue, the second data queue and the third data queue are 1:1:1, and the maximum input length is M, then the initial data amount of the data taken from the first data queue is M / 3. It is assumed that the proportion factor corresponding to the first data queue is a, 0

[0051] Since the more data is obtained from the multiple data queues, the more resources are occupied by the large language model 10 when processing, and thus the higher the data processing cost is, in order to reduce the cost, the above-mentioned scaling factor can be set to reduce the amount of data obtained from at least one of the multiple data queues.

[0052] The scaling factors corresponding to the respective data queues can be the same or different. In some embodiments, the scaling factors can be determined according to the task performed by the large language model 10. For example, if the task performed by the large language model 10 focuses more on the background information of the task, the scaling factor corresponding to the third data queue can be set to a larger value, and the scaling factors corresponding to the first data queue and the second data queue can be set to smaller values; if the task performed by the large language model 10 focuses more on the specific details of the task, the scaling factor corresponding to the first data queue can be set to a larger value, and the scaling factors corresponding to the second data queue and the third data queue can be set to smaller values; if the task performed by the large language model 10 focuses more on the flow of the task process, the scaling factor corresponding to the second data queue can be set to a larger value, and the scaling factors corresponding to the first data queue and the third data queue can be set to smaller values.

[0053] In some embodiments, since the large language model 10 has a maximum input length limit, the sum of the amount of data obtained from the multiple data queues and the amount of data of the prompt information input into the large language model 10 should be less than or equal to the maximum input length of the large language model 10. For example, the length of the prompt information is K, and the maximum input length is maxContext, then the amount of data obtained from the multiple data queues should be less than or equal to maxContext-K.

[0054] In steps S14 and S16, after obtaining data from the multiple data queues, the data obtained from the multiple data queues can be spliced with the input data input by the user into the large language model 10 to obtain the total input data of the large language model 10, and then the total input data is processed by the large language model 10 to obtain the processing result.

[0055] The overall flow of the present disclosure will be illustrated below in combination with a specific embodiment. First, the total context limit (i.e. the maximum input length) of the large language model 10 is denoted as maxContext. The specific steps of the present disclosure are as follows.

[0056] Step 1: Establish three memory queues (i.e., the data queues in the aforementioned embodiment), namely, a "long-term memory queue" (i.e., the first data queue in the aforementioned embodiment), a "medium-term memory queue" (i.e., the second data queue in the aforementioned embodiment), and a "short-term memory queue" (i.e., the third data queue in the aforementioned embodiment). The "short-term memory queue" focuses on recording the specific requirements, details, and results of task execution, and the accuracy of the recorded data is relatively high. The "medium-term memory queue" focuses on recording the flow of task processes and pays attention to which specific step the task execution has reached, as well as the derived sub-tasks and difficulties, and the accuracy of the recorded data is medium. The "long-term memory queue" focuses on task background information, such as the original requirements of the task, the changes in the overall goal of the task, the overall completion degree, and the progress of the implementation of the stage milestones, and the accuracy of the recorded data is relatively low.

[0057] Step 2: The conversation interaction history between the user and the large language model 10 is divided into blocks according to certain rules and recorded into the memory queue. The specific division method is more suitable for the business scenario. Generally speaking, in the scenario where maxContext is large (greater than the preset data volume threshold), it is suitable to divide by the number of conversation rounds, and in the scenario where maxContext is small, it is suitable to divide by the number of token usage (i.e., the total number of word pieces in the input data and output data obtained by the large language model 10).

[0058] a. Divide by the number of conversation rounds: Step (i): divide every 2 rounds of conversation, use the large language model 10 to summarize, pay more attention to task execution details, form a data block, and record it to the "short-term memory queue"; Step (ii): divide every 8 rounds of conversation, use the large language model 10 to summarize, pay more attention to task process, i.e., goal conversion, form a data block, and record it to the "medium-term memory queue"; Step (iii): divide every 32 rounds of conversation, use the large language model 10 to summarize, pay more attention to task background information, form a data block, and record it to the "long-term memory queue".

[0059] b. Divide by the number of token usage: Step (i): divide the conversation of 1 / 16*max_context token quantity, use the large language model 10 to summarize, pay more attention to task execution details, form a data block, and record it to the "short-term memory queue"; Step (ii): divide the conversation of 1 / 4 max_context token quantity, use the large language model 10 to summarize, pay more attention to task process, i.e., goal conversion, form a data block, and record it to the "medium-term memory queue"; Step (iii): divide the conversation of max_context token quantity, use the large language model 10 to summarize, pay more attention to task background information, form a data block, and record it to the "long-term memory queue".

[0060] Step 2: At the beginning of a new round of dialogue, record this round of input, and the input length is recorded as inputLength. Use the large language model 10 to perform an additional round of reasoning on the input data. This round of reasoning mainly analyzes the intention and tendency of the input data, and outputs the dependence degree of "short-term task details", "medium-term task process" and "long-term task background" respectively (i.e. the data proportion in the aforementioned embodiment), recorded as depL, depM and depS respectively.

[0061] Step 3: According to depL, depM and depS, select the most recent content with a token length not exceeding maxContext-inputLength from the three memory queues, and combine it as a new context. The next round of dialogue input is input into the large language model 10 to perform specific task processing and execution. The selection of data blocks from the queue can also be flexible and adaptive. For example, if there are cost considerations, a certain proportion of data blocks can be reduced. If the business focuses more on process continuity, the proportion of depM can be appropriately increased.

[0062] Step 4: As the number of dialogue rounds increases, when the dialogue that has not been summarized by blocks meets the conditions in step 3 again, dialogue summarization is performed, and the data blocks are placed in different memory queues according to their span.

[0063] The scheme of the embodiments of the present disclosure has the following advantages.

[0064] (1) The use of long / medium / short three specifications of memory queues makes up for the lack of details caused by only a single "long-term memory", and can be applied to different scenarios that focus on accuracy and task continuity. In the same task, the memory weight (i.e. data proportion) of short-term / medium-term / long-term can be flexibly switched according to the intention of each input.

[0065] (2) According to the depS, depM and depL parameters generated by each round of dialogue, the memory that is more in line with the needs is dynamically adjusted, rather than simply adding external memory to the context. The memory blocks are flexibly assembled according to the needs of each round of dialogue, and can be adjusted according to business needs and cost needs.

[0066] (3) The data blocks in the memory queue are sequentially taken, rather than searched, avoiding the search accuracy problem and memory incoherence problem in the RAG process.

[0067] The embodiments of the present disclosure also provide a computer device, which at least includes a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any of the preceding embodiments when executing the program.

[0068] FIG. 5 shows a more specific schematic diagram of a hardware structure of a computing device according to an embodiment of the present disclosure. The device can include a processor 202, a memory 204, an input / output interface 206, a communication interface 208, and a bus 210. The processor 202, the memory 204, the input / output interface 206, and the communication interface 208 are connected to each other through the bus 210.

[0069] The processor 202 can be implemented in the form of a general-purpose central processing unit, a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present disclosure. The processor 202 can also include a graphics card, which can be an Nvidia titan X graphics card or a 1080Ti graphics card, etc.

[0070] The memory 204 can be implemented in the form of a read only memory (ROM), a random access memory (RAM), a static storage device, a dynamic storage device, etc. The memory 204 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present disclosure are implemented by software or firmware, the related program codes are stored in the memory 204 and are called and executed by the processor 202.

[0071] The input / output interface 206 is configured to connect input / output modules to realize information input and output. The input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. The input devices can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output devices can include a display, a speaker, a vibrator, an indicator light, etc.

[0072] The communication interface 208 is configured to connect a communication module (not shown in the figure) to realize the communication interaction between the device and other devices. The communication module can realize communication through a wired manner (such as a USB, a network cable, etc.) or a wireless manner (such as a mobile network, WIFI, Bluetooth, etc.).

[0073] The bus 210 includes a path for transmitting information between various components (such as the processor 202, the memory 204, the input / output interface 206, and the communication interface 208) of the device.

[0074] It should be noted that although the above device only shows the processor 202, the memory 204, the input / output interface 206, the communication interface 208 and the bus 210, in the specific implementation process, the device can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device can also only contain the components necessary to implement the embodiments of the present disclosure, and does not necessarily contain all the components shown in the figure.

[0075] The embodiment of the present disclosure provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method described in any embodiment of the present disclosure.

[0076] The embodiment of the present disclosure also provides a computer readable storage medium, which stores a computer program, which, when executed by a processor, implements the method described in any of the preceding embodiments.

[0077] The computer readable medium includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition in this paper, computer readable medium does not include transitory computer readable medium, such as modulated data signals and carriers.

[0078] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of the present disclosure can be implemented by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of the present disclosure.

[0079] The systems, apparatuses, modules or units in the above embodiments can be implemented by computer devices or entities, or by products with certain functions. A typical implementation device is a computer, which can be specifically a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0080] The various embodiments in the present disclosure are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the device embodiments are described more simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the part of the method embodiments. The device embodiments described above are merely illustrative, and the modules described as separate components can or can not be physically separated, and the functions of each module can be implemented in one or more software and / or hardware when implementing the embodiments of the present disclosure. Part or all of the modules can be selected to achieve the purpose of the embodiments of the present disclosure according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0081] The above is only a specific implementation of the embodiments of the present disclosure. It should be noted that those skilled in the art can make several improvements and refinements without departing from the principles of the embodiments of the present disclosure, and these improvements and refinements should also be considered within the protection scope of the embodiments of the present disclosure.

Claims

1. A data processing method based on a large language model, the large language model being capable of processing based on data obtained from a storage space, the storage space including a plurality of data queues, data stored in different data queues being obtained based on historical data of different data amounts obtained by the large language model; the method comprising: obtaining data in the plurality of data queues; inputting the data obtained from the plurality of data queues as part of a prompt to the large language model for processing; obtaining a processing result of the large language model.

2. The method of claim 1, the plurality of data queues corresponding to a plurality of preset data amounts one-to-one; the method further comprising: obtaining historical data of the large language model; if a data amount of the historical data currently obtained reaches a target data amount corresponding to any one target data queue in the plurality of data queues, generating a data block corresponding to the target data queue based on historical data that has been currently obtained and not written into the target data queue, and writing the data block corresponding to the target data queue into the target data queue.

3. The method of claim 2, the target data amount corresponding to the target data queue being determined based on a number of rounds of historical conversations performed by the large language model; or the target data amount corresponding to the target data queue being determined based on a total number of word pieces in input data and output data obtained when the large language model performs historical conversations.

4. The method of claim 3, if a maximum input length of the large language model is greater than a preset data amount threshold, the target data amount corresponding to the target data queue being determined based on the number of rounds of historical conversations performed by the large language model; if the maximum input length of the large language model is less than or equal to the preset data amount threshold, the target data amount corresponding to the target data queue being determined based on the total number of word pieces in input data and output data obtained when the large language model performs historical conversations.

5. The method of claim 1, the plurality of data queues including a first data queue, a second data queue, and a third data queue; data stored in the first data queue being obtained based on historical data of a first data amount obtained by the large language model, data stored in the second data queue being obtained based on historical data of a second data amount obtained by the large language model, data stored in the third data queue being obtained based on historical data of a third data amount obtained by the large language model, and the first data amount, the second data amount, and the third data amount decreasing in turn.

6. The method of claim 5, the first data queue being used to store background information of a current task performed by the large language model, the second data queue being used to store process information of the current task performed by the large language model, and the third data queue being used to store task detail information of the current task performed by the large language model.

7. The method of claim 1, wherein the plurality of data queues are configured to store key information extracted from input data and output data of the large language model, and information density of the key information in the plurality of data queues is different.

8. The method of claim 1, wherein the obtaining data in the plurality of data queues comprises: obtaining data in the plurality of data queues according to respective data amount proportions of the plurality of data queues.

9. The method of claim 8, further comprising: performing an additional inference on current input data by the large language model to obtain an inference result, wherein the inference result is configured to represent an influence degree of data in the plurality of data queues on a current task performed by the large language model, and the current input data is input data when the large language model performs a current dialogue; and determining the respective data amount proportions of the plurality of data queues according to the inference result.

10. The method of claim 8, wherein the obtaining data in the plurality of data queues according to the respective data amount proportions of the plurality of data queues comprises: obtaining a maximum input length of the large language model; obtaining initial data amounts of data taken from the plurality of data queues based on the maximum input length and the respective data amount proportions of the plurality of data queues; and for any one data queue, determining a target data amount of data taken from the data queue based on a proportional factor corresponding to the data queue and the initial data amount of data taken from the data queue, and obtaining data in the data queue based on the target data amount.

11. The method of claim 1, wherein a sum of data amounts of data taken from the plurality of data queues and prompt information input to the large language model is less than or equal to the maximum input length of the large language model.

12. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any one of claims 1 to 11.

13. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the method of any one of claims 1 to 11 when executing the program.

14. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 11. ​ ​ ​ ​ ​ ​ ​ ​

Images