Text-based question answer generation method and device, equipment and storage medium

Abstract

The invention provides a text-based question answer generation method and device, equipment and a storage medium, and relates to the technical field of data processing, in particular to the technical fields of artificial intelligence, deep learning, large model and retrieval enhancement generation (RAG) and the like. According to the specific implementation scheme, a first cache token sequence of first sub-content in a target text is obtained from a cache database, and the target text is a text determined based on a to-be-answered question; obtaining a first target token sequence of the target text based on the first cache token sequence; obtaining a second target token sequence of the question to be answered; and performing model reasoning at least based on the first target token sequence and the second target token sequence to obtain a target answer for the to-be-answered question.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to technical fields such as artificial intelligence, deep learning, large models, and retrieval-augmented generation (RAG). Background Art

[0002] In the long document reading comprehension scenario, the Large Language Model (LLM) usually takes a long time to output the first token, which affects the user experience. Summary of the invention

[0003] The present invention provides a method, device, equipment and storage medium for generating text-based question answers.

[0004] According to one aspect of the present disclosure, a method for generating a text-based question answer is provided, comprising:

[0005] Acquire, from a cache database, a first cache token sequence of a first sub-content in a target text, wherein the target text is a text determined based on a question to be answered;

[0006] Based on the first cached token sequence, a first target token sequence of the target text is obtained; and a second target token sequence of the question to be answered is obtained;

[0007] Model reasoning is performed based on at least the first target token sequence and the second target token sequence to obtain a target answer to the question to be answered.

[0008] According to another aspect of the present disclosure, there is provided a text-based question answer generation device, comprising:

[0009] An acquisition unit, configured to acquire, from a cache database, a first cache token sequence of a first sub-content in a target text, wherein the target text is a text determined based on a question to be answered;

[0010] A preprocessing unit, configured to obtain a first target token sequence of a target text based on the first cache token sequence; and obtain a second target token sequence of a question to be answered;

[0011] A model reasoning unit is used to perform model reasoning based on at least the first target token sequence and the second target token sequence to obtain a target answer to the question to be answered.

[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0018] In this way, the disclosed solution can first obtain the first cache token sequence of the first sub-content in the target text that has been pre-calculated from the cache database in the pre-filling stage, and then quickly obtain the complete token sequence of the target text and the complete token sequence of the question to be answered based on the first cache token sequence. Here, since in the pre-filling stage, a partial token sequence can be first obtained from the cache database, and then the complete token sequence required for the subsequent reasoning stage can be calculated, the disclosed solution effectively reduces the amount of token calculation in the pre-filling stage, shortens the time required to obtain the complete token sequence, and thereby reduces the time taken for the model to output the first token. In other words, it effectively solves the problem of long time taken for the first token and poor user experience in the long document reading comprehension scenario, thereby effectively improving the user experience.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0021] Figure 1 This is a schematic diagram of a method for generating text-based question answers according to an embodiment of the present application. Figure 1 ;

[0022] Figure 2 This is a schematic diagram of a method for generating text-based question answers according to an embodiment of the present application. Figure 2 ;

[0023] Figure 3This is a scenario diagram of a specific example of a method for generating a text-based question answer according to an embodiment of the present application. Figure 1 ;

[0024] FIG4( a ) is a schematic diagram of the structure of a target model according to an embodiment of the present application;

[0025] FIG4( b ) is a schematic diagram illustrating a token sequence stored in a cache database according to an embodiment of the present application;

[0026] Figure 5 is a flowchart of a method for generating a text-based question answer in a specific example according to an embodiment of the present application;

[0027] Figure 6 This is a schematic diagram of a method for generating text-based question answers according to an embodiment of the present application. Figure 3 ;

[0028] FIG. 7( a ) is a schematic diagram of a scenario in a specific example of a method for generating a text-based question answer according to an embodiment of the present application. Figure 2 ;

[0029] FIG. 7( b ) is a schematic diagram of a scenario in a specific example of a method for generating a text-based question answer according to an embodiment of the present application. Figure 3 ;

[0030] Figure 8 is a fourth schematic diagram of a method for generating text-based question answers according to an embodiment of the present application;

[0031] Fig. 9 is a structural diagram of a text-based question answer generation device according to an embodiment of the present application;

[0032] Fig.10 It is a block diagram of an electronic device used to implement the text-based question answer generation method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0034] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0035] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0036] In existing long document reading and comprehension scenarios, the first token takes a long time and the user experience is poor. To solve the above problems, the existing solutions usually adopt the following solutions. However, the following solutions usually have corresponding problems:

[0037] First, shorten the length of the prompt information. Although this solution can reduce the time spent on the first token, the model effect will decrease.

[0038] Second, perform token pruning, model compression, or optimize the attention mechanism. However, although the above solutions can reduce the amount of model calculation, they will cause a serious decline in model performance.

[0039] Therefore, there is an urgent need for a solution that can shorten the time required for the first token while effectively ensuring the model effect.

[0040] Based on this, the disclosed solution provides a text-based question answer generation method, which can obtain a pre-calculated token sequence of partial text content related to the question to be answered from a cache database in the pre-filling stage, and then quickly obtain the complete token sequence required for model reasoning. In this way, the time required to obtain the complete token sequence in the pre-filling stage is shortened, the reasoning efficiency of the model is improved, and the time taken for the model to output the first token is reduced. In other words, it effectively solves the problems of long time taken for the first token and poor user experience in the long document reading comprehension scenario, thereby effectively improving the user experience.

[0041] Specifically, Figure 1 This is a schematic flow chart of a method for generating text-based question answers according to an embodiment of the present application. Figure 1 The method may be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0042] Further, the method includes at least part of the following contents. Figure 1 As shown, including:

[0043] Step S101: acquiring a first cache token sequence of a first sub-content in a target text from a cache database.

[0044] Here, the target text is the text determined based on the question to be answered.

[0045] For instance, in one example, the target text may be text retrieved from a document database based on the question to be answered and required to answer the question to be answered; or, in another example, the target text may be text uploaded by the target object, and accordingly, the question to be answered may be a corresponding question posed in response to the target text uploaded by the target object.

[0046] Step S102: Based on the first cache token sequence, a first target token sequence of the target text is obtained.

[0047] Here, compared with the first cache token sequence (which may be a token sequence of partial content in the target text, in other words, may be an incomplete token sequence of the target text), the first target token sequence is a complete token sequence of the target text, in other words, the first target token sequence is a token sequence containing the complete content of the target text.

[0048] Step S103: Obtain a second target token sequence for the question to be answered.

[0049] Here, the second target token sequence is a token sequence that includes the complete content of the question to be answered.

[0050] Here, step S102 and step S103 may be executed simultaneously or sequentially, and the disclosed solution does not limit the execution order of the two.

[0051] Step S104: Perform model reasoning based on at least the first target token sequence and the second target token sequence to obtain a target answer to the question to be answered.

[0052] That is to say, in the disclosed solution, after obtaining the question to be answered and determining the target text required to answer the question to be answered, first obtain all cache token sequences for the target text from the cache database. For example, the cache database currently only stores the first cache token sequence for the first sub-content in the target text, and has not yet stored the token sequences of the remaining content. At this time, the first cache token sequence for the first sub-content in the target text can be obtained from the cache database; further, in the pre-filling stage, based on the first cache token sequence obtained from the cache database, obtain the first target token sequence for the target text (that is, the complete token sequence for the target text), and obtain the second target token sequence for the question to be answered (that is, the complete token sequence for the question to be answered); finally, in the model reasoning stage, use the first target token sequence and the second target token sequence to perform model reasoning, so as to obtain the target answer to the question to be answered, and the target answer is the answer to the question to be answered based on the target text.

[0053] In this way, the disclosed solution can first obtain the first cache token sequence of the first sub-content in the target text that has been pre-calculated from the cache database in the pre-filling stage, and then quickly obtain the complete token sequence of the target text (that is, the first target token sequence) based on the first cache token sequence, and obtain the complete token sequence of the question to be answered (that is, the second target token sequence). Here, since in the pre-filling stage, a partial token sequence can be first obtained from the cache database, and then the complete token sequence required for the subsequent reasoning stage can be calculated, the disclosed solution effectively reduces the amount of token calculation in the pre-filling stage, shortens the time required to obtain the complete token sequence, and thereby reduces the time taken for the model to output the first token. In other words, it effectively solves the problem of long time taken for the first token and poor user experience in the long document reading comprehension scenario, thereby effectively improving the user experience.

[0054] In addition, the disclosed solution does not change the model architecture and computing logic, but only shortens the processing time of the pre-filling stage, which has little impact on the reasoning effect and the additional cost is also low.

[0055] It should be noted that the large model used in the disclosed solution can specifically be a large language model, a retrieval enhanced generation RAG model, etc., and the disclosed solution does not impose any restrictions on this.

[0056] Figure 2 This is a schematic flow chart of a method for generating text-based question answers according to an embodiment of the present application. Figure 2 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figure 1The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0057] Further, the method includes at least part of the following contents. Figure 2 As shown, including:

[0058] Step S201: acquiring a first cache token sequence of a first sub-content in a target text from a cache database.

[0059] Here, the target text is the text determined based on the question to be answered.

[0060] It should be noted that the relevant exemplary descriptions of the target text can be found in the above content and will not be repeated here.

[0061] Step S202: Based on the first cache token sequence, a first target token sequence of the target text is obtained.

[0062] Step S203: Obtain a second target token sequence for the question to be answered.

[0063] Here, step S202 and step S203 may be executed simultaneously or sequentially, and the disclosed solution does not limit the execution order of the two.

[0064] Step S204: Determine the target prompt template.

[0065] Here, the target prompt template at least includes: a question to be answered and a target text determined based on the question to be answered.

[0066] It should be noted that the execution order of step S204, step S203 and step S202 can be interchanged, for example, step S204 is executed first, and then steps S202 and S203 are executed. The present disclosure does not limit this.

[0067] Step S205: Based at least on the first target token sequence and the second target token sequence, obtain a total target token sequence corresponding to the target prompt template.

[0068] It should be noted that the relevant exemplary descriptions of the first target token sequence and the second target token sequence can be found in the above content and will not be elaborated here.

[0069] Step S206: Perform model reasoning based on the total target token sequence corresponding to the target prompt template to obtain a target answer to the question to be answered.

[0070] That is to say, in this example, after obtaining a complete token sequence for the target text (i.e., the first target token sequence) and a complete token sequence for the question to be answered (i.e., the second target token sequence), the total target token sequence can be constructed based on the target prompt template, and then the total target token sequence can be filled into the large model, so as to infer the target answer to the question to be answered based on the total target token sequence and the reasoning ability of the large model. In this way, on the one hand, the prompt function of the target prompt template can be used to guide the large model to think and reason, so that the large model can understand the relationship between the question to be answered and the target text more quickly during reasoning, and obtain satisfactory reasoning results, thereby improving the user experience; on the other hand, it also effectively reduces the amount of token calculation in the pre-filling stage, shortens the time required to obtain a complete token sequence, and thereby reduces the time it takes for the model to output the first token, further improving the user experience.

[0071] Here, in one example, the target prompt template may include three parts, namely: prefix, search result and suffix. Here, the prefix may mainly include the preset content required to guide the large model to perform reasoning; the search result may mainly include the target text for answering the question to be answered; and the suffix may mainly include the question to be answered entered by the target object.

[0072] For example, if Figure 3 As shown, the target prompt template may be specifically a RAG prompt template (RAG Prompt), wherein the RAG Prompt may include a fixed prefix, a search result, and a variable suffix; further, in this example, the fixed prefix may specifically include a task description, such as, "If you are a professional robot, please refer to the search results to answer the question"; the search results may specifically include the complete text content of the target text; the variable suffix may be specifically a question request (Query) or task instruction input by the target object, such as the question to be answered.

[0073] Further, when receiving the unanswered question uploaded by the target object, according to the content of the task description in RAG Prompt, the complete token sequence of the task description is obtained from the cache database. Here, since the task description is a fixed content, its complete token sequence can be pre-stored in the cache database; and, according to the target text in RAGPrompt, the entire token sequence for the target text currently stored in the cache database is obtained from the cache database. Here, it should be noted that for the long document reading comprehension scenario, the target text may be the document currently uploaded by the target object. At this time, in order to be able to quickly answer multiple questions input by the target object based on the target text later, the cache database can pre-store the token sequence of the target text. However, it is understandable that when receiving the question to be answered, the cache database may not store the token sequence of the complete content of the target text. For example, only the cache token sequence for the first sub-content of the target text is currently stored. At this time, only the cache token sequence for the first sub-content of the target text can be obtained from the cache database. In addition, it is understandable that since the question to be answered is constantly changing, the cache database does not need to pre-store the token sequence for the question to be answered.

[0074] Furthermore, in the pre-filling stage, the first target token sequence of the complete content of the target text can be calculated based on the first cache token sequence of the first sub-content of the target text obtained from the cache database, and the second target token sequence can be calculated based on the question to be answered, so as to obtain the total target token sequence corresponding to the RAG Prompt; in this way, in the reasoning stage, model reasoning can be performed based on the total target token sequence corresponding to the RAG Prompt to obtain the target answer to the question to be answered.

[0075] Here, it should be noted that the cache token sequence for the target text cached in the cache database can be obtained based on the target model. For example, the token sequence of the target text is calculated using the target model, and the calculated token sequence is stored in the cache database.

[0076] Further, in a specific example, the first cache token sequence stored in the cache database may be obtained in any of the following ways:

[0077] Mode 1: The token sequence output by the first-layer network module in the target model after extracting features from the first sub-content; that is, in Mode 1, the first cache token sequence is the sequence output by the first-layer network module in the target model.

[0078] Mode 2: The token sequence output by the intermediate layer network module in the target model after extracting features from the first sub-content; that is, in Mode 2, the first cache token sequence is the sequence output by the intermediate layer network module in the target model.

[0079] Mode 3: The token sequence output by the last network module in the target model after extracting features from the first sub-content. That is, in Mode 3, the first cache token sequence is the sequence output by the last network module in the target model.

[0080] Here, the target model in this example includes multiple network modules with a series relationship, that is, in this target model, the output of the network module in the previous layer is used as the input of the network module in the next layer, the output content of the network module in the last layer is the final output result of the target model, and the input content of the network module in the first layer is the relevant text in the target text.

[0081] Based on this, it can be seen that among the above three methods, only the sequence obtained by method 3 is the final output sequence of the target model, while the sequences output by methods 1 and 2 are intermediate sequences.

[0082] For example, in one example, as shown in FIG4(a), it is assumed that the target model includes N network modules, and N is an integer greater than or equal to 2; further, the N network modules are connected in series, that is, the output of the previous network module is used as the input of the next network module; and the input of the target model (that is, the first sub-content of the target text) is directly used as the input of the first network module (that is, network module 1), and the output of the last network module is the final output result of the target model.

[0083] Furthermore, each network module in the target model is used to extract feature data of the input content and output a token sequence.

[0084] For example, taking a target model including three network modules as an example, as shown in FIG4(b), in the process of extracting features of the first sub-content in the target text using each network module in the target model, when the cache timing of the token sequence is reached, if the first sub-content has only completed feature extraction by network module 1, the output result of network module 1 (for example, can be recorded as token sequence 1) can be used as the first cache token sequence and stored in the cache database.

[0085] Alternatively, if the first sub-content has completed feature extraction by the first two network modules (i.e., network module 1 and network module 2) and has not yet completed feature extraction by network module 3 on the output result of network module 2, then the output result of network module 2 (for example, which can be recorded as token sequence 2) is used as the first cache token sequence and stored in the cache database.

[0086] Alternatively, if all network modules in the target model have completed feature extraction of the first sub-content, the output result of network module 3 (for example, may be recorded as token sequence 3) is used as the first cache token sequence and stored in the cache database.

[0087] Here, it should be noted that the cache timing of the token sequence in this example may specifically be the time when the target object initiates the question to be answered, or it may also be a pre-set cache time, and the present disclosure does not impose specific restrictions on this.

[0088] In this way, when the caching timing is reached, the disclosed solution can obtain the processing result of the first sub-content at the network module level (that is, the first cached token sequence) according to the processing status of each network module in the target model, and store it in the cache database. In this way, the pre-caching of the token sequence is effectively realized. In the pre-filling stage, there is no need to calculate the complete token sequence of the target text from scratch, which effectively reduces the amount of token calculation in the pre-filling stage and shortens the time required to obtain the complete token sequence, thereby reducing the time taken for the model to output the first token and improving the user experience.

[0089] For example, if Figure 5As shown, after obtaining the target text uploaded by the target object to the platform, the target model described above can be used to pre-calculate the key-value cache (KV Cache) of the target text; at this time, if the cache timing of the key-value cache is reached (such as receiving a question to be answered initiated by the target object), the token sequence output by the network module in the target model is cached in a cache database (such as a key-value cache database (KV Cache Database)) according to the processing of the target text by each network module (block) in the target model. In other words, the token sequence stored in the KV Cache Database is the output at the network module level. For example, for the first sub-content, when the cache timing is reached, if only the processing of the upper-layer network module is completed, but the processing of the lower-layer network module is not completed, the output result of the upper-layer network module is stored in the KV Cache Database as the cache token sequence of the first sub-content. Alternatively, when the cache opportunity is reached, if the processing of the next layer of network modules has been completed, there is no need to store the output results of the previous layer of network modules. The output results of the next layer of network modules only need to be stored in the KV Cache Database as the cache token sequence of the first sub-content, so as to ensure that the cache token sequence stored in the KV Cache Database is the sequence closest to the output of the last layer of network modules, thus providing strong support for the subsequent rapid calculation of the first target token sequence. Otherwise, when the cache opportunity has not been reached, the KV Cache pre-calculation of the target text continues until the cache opportunity of the key value cache is reached or the KV Cache pre-calculation of all the content in the target text is completed.

[0090] Furthermore, if Figure 5As shown, after obtaining the target object to initiate the question to be answered for the target text, first, load the cache token sequence matching the target text from the KV Cache Database, for example, load the first cache token sequence of the first sub-content in the target text; secondly, use the target model, according to the first cache token sequence of the first sub-content in the target text, continue to complete the KV Cache pre-calculation of all the contents in the target text, so as to obtain the first target token sequence for the target text, and obtain the second target token sequence for the question to be answered, so as to complete the pre-filling and obtain the total target token sequence corresponding to the target prompt template; finally, perform model reasoning according to the total target token sequence corresponding to the target prompt template to obtain the target answer for the question to be answered. In this way, since the disclosed scheme can retrieve the pre-calculated token sequence from the cache database in real time, and pre-fill the complete token sequence on this basis, so that the pre-filling stage does not need to calculate the complete token sequence of the target text from scratch, effectively reducing the token calculation amount in the pre-filling stage, shortening the time required to obtain the complete token sequence, thereby reducing the time taken for the model to output the first token and improving the user experience.

[0091] Figure 6 This is a schematic flow chart of a method for generating text-based question answers according to an embodiment of the present application. Figure 3 The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figures 1 to 5 The relevant contents of the method shown can also be applied to this example, and the relevant contents will not be described in detail in this example.

[0092] Further, the method includes at least part of the following contents. Figure 6 As shown, including:

[0093] Step S601: acquiring a first cache token sequence of a first sub-content in a target text from a cache database.

[0094] Here, the target text is the text determined based on the question to be answered.

[0095] It should be noted that the relevant exemplary descriptions of the target text can be found in the above content and will not be repeated here.

[0096] Step S602: based on the first cache token sequence, obtain an initial token sequence of the first sub-content.

[0097] Here, the initial token sequence of the first sub-content is the token sequence for the first sub-content output by the last layer network module in the target model.

[0098] Step S603: obtaining an initial token sequence of the remaining sub-contents in the target text except the first sub-content.

[0099] Here, the initial token sequence of the remaining sub-content is also the token sequence for the remaining sub-content output by the last layer network module in the target model.

[0100] Step S604: Based on the initial token sequence of the first sub-content and the initial token sequences of the remaining sub-contents, a first target token sequence of the target text is obtained.

[0101] It should be noted that in order to better capture the contextual relationship of the text, the cached token sequence of the first sub-content needs to be processed to obtain the token sequence for the first sub-content output by the last layer network module in the target model (that is, the initial token sequence of the first sub-content).

[0102] At the same time, for the remaining sub-contents in the target text except the first sub-content (that is, other sub-contents that have not been loaded to obtain the token sequence), the target model is called to extract features of the remaining sub-contents in the target text. At this time, the output result of the last layer of network modules in the target model is the initial token sequence of the remaining sub-content. In this way, it is convenient to use each initial token sequence to obtain the complete token sequence of the target text, thereby reducing the calculation amount and processing time of the pre-filling stage.

[0103] Step S605: Obtain a second target token sequence for the question to be answered.

[0104] Here, the steps of obtaining the second target token sequence of the question to be answered and obtaining the first target token sequence of the target text (for example, step S602-step S604) can be performed simultaneously or interchangeably, and the disclosed solution does not impose any specific restrictions on this.

[0105] It should be noted that the relevant exemplary descriptions of the first target token sequence and the second target token sequence can be found in the above content and will not be elaborated here.

[0106] Step S606: Perform model reasoning based on at least the first target token sequence and the second target token sequence to obtain a target answer to the question to be answered.

[0107] In this way, the disclosed solution can quickly obtain the token sequence output by the last layer network module in the target model based on the token sequence obtained from the cache database, and then quickly obtain the token sequence for the complete content of the target text. In this way, the calculation amount of the token sequence of the target text in the pre-filling stage is effectively reduced, the time required for pre-filling is shortened, and the time taken for the model to output the first token is reduced, thereby effectively improving the user experience.

[0108] Further, in a specific example, the above-mentioned step of obtaining the initial token sequence of the first sub-content based on the first cache token sequence (for example, the above-mentioned step S602) may specifically include:

[0109] Step S602-1: Determine whether the first cache token sequence is the token sequence output by the last layer network module in the target model. If so, proceed to step S602-4; otherwise, proceed to step S602-2.

[0110] Step S602-2: when the first cache token sequence is a token sequence output by a non-last layer network module in the target model, determine the network module that outputs the first cache token sequence.

[0111] Step S602-3: Process the first cache token sequence using the next layer network module of the network module that outputs the first cache token sequence in the target model to obtain the initial token sequence for the first sub-content output by the last layer network module in the target model.

[0112] Step S602-4: when the first cached token sequence is the token sequence output by the last layer network module in the target model, the first cached token sequence is used as the initial token sequence of the first sub-content.

[0113] For example, continuing with Figure 4(b), if the first cached token sequence of the first sub-content in the target text is a token sequence not output by network module 3, and it is determined that the network module that outputs the first cached token sequence is network module 2, then the first cached token sequence can be further processed using the next layer of network module (i.e., network module 3) of network module 2 in the target model to obtain the token sequence output by the last layer of network module (i.e., network module 3) in the target model. The token sequence output by the last layer of network module (i.e., network module 3) is the initial token sequence for the first sub-content.

[0114] Alternatively, if the first cached token sequence of the first sub-content in the target text is the token sequence output by the last layer of network module (ie, network module 3), the first cached token sequence can be directly used as the initial token sequence for the first sub-content.

[0115] In this way, the disclosed solution can quickly obtain the token sequence output by the last layer network module in the target model according to the first cached token sequence of the cached first sub-content. This provides strong support for quickly obtaining the token sequence for the complete content of the target text, thereby effectively reducing the amount of calculation of the token sequence of the target text in the pre-filling stage, shortening the time required for pre-filling, and reducing the time taken for the model to output the first token, thereby effectively improving the user experience.

[0116] For example, as shown in Figure 7(a), the target text is divided into three parts (respectively recorded as sub-content 1, sub-content 2 and sub-content 3) and input into the target model for KV Cache pre-calculation; further, when the question to be answered initiated by the target object is received, the cache token sequence 1 of sub-content 1 is loaded from the cache database, and the cache token sequence 1 is the token sequence output by the last layer of network module in the target model, and the token sequence of sub-content 2 and the token sequence of sub-content 3 are not loaded; at this time, the cache token sequence 1 of sub-content 1 can be directly used as the initial token sequence 1 of sub-content 1. Further, in the pre-filling stage, the target model is called to extract features of the remaining sub-contents of the target text (i.e., sub-content 2 and sub-content 3), and the token sequence for the remaining sub-contents output by the last layer of network module in the target model is obtained, for example, the initial token sequence 2 for sub-content 2 and the initial token sequence 3 for sub-content 3 are obtained. In this way, the first target token sequence of the target text is obtained according to each initial token sequence, so as to obtain the total target token sequence for model reasoning according to the first target token sequence and the second target token sequence calculated based on the question to be answered.

[0117] Alternatively, in another example, as shown in FIG7(b), when a question to be answered initiated by the target object is received, a cached token sequence 1 of sub-content 1 is loaded from the cache database, and the cached token sequence 1 is the token sequence output by the last layer of network module in the target model, and a cached token sequence 2 of sub-content 2 is loaded, and the cached token sequence 2 is the token sequence output by a non-last layer of network module in the target model, and the token sequence of sub-content 3 is not loaded; at this time, the cached token sequence 1 of sub-content 1 is used as the initial token sequence 1 of sub-content 1; at the same time, the cached token sequence 2 is processed by using the next layer of network module of the network module that outputs the cached token sequence 2 in the target model until the token sequence for sub-content 2 output by the last layer of network module in the target model (that is, the initial token sequence 2) is obtained, and the target model is called to perform feature extraction on sub-content 3 of the target text until the token sequence for sub-content 3 output by the last layer of network module in the target model (that is, the initial token sequence 3) is obtained. In this way, the first target token sequence of the target text is obtained according to each initial token sequence, so as to obtain the total target token sequence for model reasoning according to the first target token sequence and the second target token sequence calculated based on the question to be answered.

[0118] Figure 8 FIG4 is a schematic flow chart of a method for generating text-based question answers according to an embodiment of the present application. The method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It can be understood that the above Figure 1 The related contents of the method shown in FIG. 7 can also be applied to this example, and the related contents will not be described in detail in this example.

[0119] Further, the method includes at least part of the following contents. Figure 8 As shown, including:

[0120] Step S801: acquiring a first cache token sequence of a first sub-content in a target text from a cache database.

[0121] Here, the target text is the text determined based on the question to be answered.

[0122] It should be noted that the relevant exemplary descriptions of the target text can be found in the above content and will not be repeated here.

[0123] Step S802: Based on the first cache token sequence, a first target token sequence of the target text is obtained.

[0124] Here, the relevant exemplary description of the first target token sequence can be found in the above content and will not be repeated here.

[0125] Step S803: Use the multiple network modules included in the target model to perform feature extraction on the question to be answered, so as to obtain a token sequence for the question to be answered output by the last layer of network modules.

[0126] Step S804: using the token sequence for the question to be answered output by the last layer of network module as the second target token sequence for the question to be answered.

[0127] Here, step S802 and obtaining the second target token sequence of the question to be answered (ie, step S803 and step S804) can be executed simultaneously or successively, and the disclosed solution does not limit the execution order of the two.

[0128] Step S805: Perform model reasoning based on at least the first target token sequence and the second target token sequence to obtain a target answer to the question to be answered.

[0129] In this way, the disclosed solution provides a detailed solution for quickly obtaining the second target token sequence of the question to be answered. The solution is simple and efficient, and can provide strong support for rapid model reasoning. In addition, since the disclosed solution can first obtain a partial token sequence from the cache database in the pre-filling stage, and then calculate the complete token sequence required for the subsequent reasoning stage, the token calculation amount in the pre-filling stage is effectively reduced, and the time required to obtain the complete token sequence is shortened, thereby reducing the time taken by the model to output the first token. In other words, it effectively solves the problem of long time spent on the first token and poor user experience in the long document reading and comprehension scenario, thereby effectively improving the user experience.

[0130] The disclosed solution also provides a text-based question answer generation device, such as Fig. 9 As shown, including:

[0131] An acquisition unit 901 is used to acquire a first cache token sequence of a first sub-content in a target text from a cache database, wherein the target text is a text determined based on a question to be answered;

[0132] A preprocessing unit 902 is used to obtain a first target token sequence of a target text based on the first cache token sequence; and obtain a second target token sequence of a question to be answered;

[0133] The model reasoning unit 903 is used to perform model reasoning based on at least the first target token sequence and the second target token sequence to obtain a target answer to the question to be answered.

[0134] In a specific example of the disclosed solution, the first cache token sequence is one of the following:

[0135] The token sequence output by the first-layer network module in the target model after extracting features from the first sub-content;

[0136] A token sequence outputted by the middle layer network module in the target model after feature extraction of the first sub-content;

[0137] The token sequence output by the last network module in the target model after extracting features from the first sub-content;

[0138] The target model includes a plurality of network modules in a serial relationship, and each network module in the target model is used to extract feature data of input content and output a token sequence.

[0139] In a specific example of the disclosed solution, the preprocessing unit is specifically used to:

[0140] Based on the first cached token sequence, an initial token sequence of the first sub-content is obtained; wherein the initial token sequence of the first sub-content is a token sequence for the first sub-content output by the last layer network module in the target model;

[0141] Obtaining an initial token sequence of the remaining sub-contents except the first sub-content in the target text, wherein the initial token sequence of the remaining sub-contents is a token sequence for the remaining sub-contents output by the last layer network module in the target model;

[0142] Based on the initial token sequence of the first sub-content and the initial token sequences of the remaining sub-contents, a first target token sequence of the target text is obtained.

[0143] In a specific example of the disclosed solution, the preprocessing unit is specifically used to:

[0144] In the case where the first cache token sequence is output by a non-last layer network module in the target model, determining a network module that outputs the first cache token sequence;

[0145] The first cache token sequence is processed by using a network module of a next layer of the network module that outputs the first cache token sequence in the target model to obtain an initial token sequence for the first sub-content output by the last layer of the network module in the target model.

[0146] In a specific example of the disclosed solution, the preprocessing unit is specifically used to:

[0147] In the case where the first cached token sequence is output by the last layer network module in the target model, the first cached token sequence is used as the initial token sequence of the first sub-content.

[0148] In a specific example of the disclosed solution, the preprocessing unit is specifically used to:

[0149] Using multiple network modules included in the target model to extract features of the questions to be answered, so as to obtain a token sequence for the questions to be answered output by the last layer of network modules;

[0150] The token sequence for the question to be answered output by the last layer of network module is used as the second target token sequence for the question to be answered.

[0151] In a specific example of the disclosed solution, the preprocessing unit is further used to determine a target prompt template; wherein the target prompt template at least includes: a question to be answered and a target text determined based on the question to be answered; and a total target token sequence corresponding to the target prompt template is obtained based at least on the first target token sequence and the second target token sequence;

[0152] The model inference unit is used to perform model inference based on the total target token sequence corresponding to the target prompt template to obtain a target answer to the question to be answered.

[0153] For the description of the specific functions and examples of each unit of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0154] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0155] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0156] Fig.10 A schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0157] like Fig.10As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0158] A number of components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0159] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as a text-based question answer generation method. For example, in some embodiments, the text-based question answer generation method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the text-based question answer generation method described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the text-based question answer generation method in any other appropriate manner (eg, by means of firmware).

[0160] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0162] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0163] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0164] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0165] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0166] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0167] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A text-based question answer generation method, comprising: Acquire, from a cache database, a first cache token sequence of a first sub-content in a target text, wherein the target text is a text determined based on a question to be answered; Based on the first cached token sequence, a first target token sequence of the target text is obtained; and a second target token sequence of the question to be answered is obtained; Model reasoning is performed based on at least the first target token sequence and the second target token sequence to obtain a target answer to the question to be answered.

2. The method according to claim 1, wherein: The first cache token sequence is one of the following: The token sequence output by the first-layer network module in the target model after extracting features from the first sub-content; A token sequence outputted by the middle layer network module in the target model after feature extraction of the first sub-content; The token sequence output by the last network module in the target model after extracting features from the first sub-content; The target model includes a plurality of network modules in a serial relationship, and each network module in the target model is used to extract feature data of input content and output a token sequence.

3. The method according to claim 2, wherein: The step of obtaining a first target token sequence of the target text based on the first cache token sequence includes: Based on the first cached token sequence, an initial token sequence of the first sub-content is obtained; wherein the initial token sequence of the first sub-content is a token sequence for the first sub-content output by the last layer network module in the target model; Obtaining an initial token sequence of the remaining sub-contents except the first sub-content in the target text, wherein the initial token sequence of the remaining sub-contents is a token sequence for the remaining sub-contents output by the last layer network module in the target model; Based on the initial token sequence of the first sub-content and the initial token sequences of the remaining sub-contents, a first target token sequence of the target text is obtained.

4. The method according to claim 3, wherein: The obtaining, based on the first cache token sequence, an initial token sequence of the first sub-content includes: In the case where the first cache token sequence is output by a non-last layer network module in the target model, determining a network module that outputs the first cache token sequence; The first cache token sequence is processed by using a network module of a next layer of the network module that outputs the first cache token sequence in the target model to obtain an initial token sequence for the first sub-content output by the last layer of the network module in the target model.

5. The method according to claim 3, wherein: The obtaining, based on the first cache token sequence, an initial token sequence of the first sub-content includes: In the case where the first cached token sequence is output by the last layer network module in the target model, the first cached token sequence is used as the initial token sequence of the first sub-content.

6. The method according to any one of claims 2 to 5, wherein: The step of obtaining a second target token sequence for the question to be answered comprises: Using multiple network modules included in the target model to extract features of the questions to be answered, so as to obtain a token sequence for the questions to be answered output by the last layer of network modules; The token sequence for the question to be answered output by the last layer of network module is used as the second target token sequence for the question to be answered.

7. The method according to any one of claims 2 to 5, further comprising: Determine a target prompt template; wherein the target prompt template at least includes: a question to be answered and a target text determined based on the question to be answered; The performing of model reasoning based on at least the first target token sequence and the second target token sequence to obtain a target answer to the question to be answered includes: Based at least on the first target token sequence and the second target token sequence, obtaining a total target token sequence corresponding to the target prompt template; Model inference is performed based on the total target token sequence corresponding to the target prompt template to obtain the target answer to the question to be answered.

8. A text-based question answer generation device, comprising: An acquisition unit, configured to acquire, from a cache database, a first cache token sequence of a first sub-content in a target text, wherein the target text is a text determined based on a question to be answered; A preprocessing unit, configured to obtain a first target token sequence of a target text based on the first cache token sequence; and obtain a second target token sequence of a question to be answered; A model reasoning unit is used to perform model reasoning based on at least the first target token sequence and the second target token sequence to obtain a target answer to the question to be answered.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

11. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images