Text-based question-answer generation method, apparatus, device, and storage medium

Abstract

This disclosure provides a method, apparatus, device, and storage medium for generating text-based question answers, relating to the field of data processing technology, particularly to the fields of artificial intelligence, deep learning, large models, and retrieval-enhanced generation (RAG). The specific implementation scheme is as follows: A first cached token sequence of a first sub-content from a cached database is obtained, wherein the target text is text determined based on the question to be answered; a first target token sequence of the target text is obtained based on the first cached token sequence; and a second target token sequence of the question to be answered is obtained; model inference is performed at least based on the first and second target token sequences to obtain the target answer for the question to be answered.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to the fields of artificial intelligence, deep learning, large models, and retrieval-augmented generation (RAG). Background Technology

[0002] In long document reading comprehension scenarios, the Large Language Model (LLM) typically takes a long time to output the first token, which affects the user experience. Summary of the Invention

[0003] This disclosure provides a text-based question-answer generation method, apparatus, device, and storage medium.

[0004] According to one aspect of this disclosure, a text-based question-answer generation method is provided, comprising:

[0005] Retrieve the first cache token sequence of the first sub-content in the target text from the cache database, where the target text is the text determined based on the question to be answered;

[0006] Based on the first cached token sequence, a first target token sequence for the target text is obtained; and a second target token sequence for the question to be answered is obtained.

[0007] Model reasoning is performed based at least on the first target token sequence and the second target token sequence to obtain the target answer for the question to be answered.

[0008] According to another aspect of this disclosure, a text-based question-answer generation apparatus is provided, comprising:

[0009] The acquisition unit is used to retrieve the first cache token sequence of the first sub-content in the target text from the cache database, wherein the target text is the text determined based on the question to be answered;

[0010] The preprocessing unit is configured to obtain a first target token sequence of the target text based on a first cached token sequence; and to obtain a second target token sequence of the question to be answered.

[0011] The model reasoning unit is used to perform model reasoning based at least on the first target token sequence and the second target token sequence to obtain the target answer for the question to be answered.

[0012] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0013] At least one processor; and

[0014] The memory is communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform any of the methods described in the present disclosure.

[0016] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the methods according to embodiments of this disclosure.

[0017] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the methods according to embodiments of this disclosure.

[0018] In this way, the proposed solution can first obtain the first cached token sequence of the first sub-content in the target text from the cache database during the pre-filling stage, and then quickly obtain the complete token sequence of the target text and the complete token sequence of the question to be answered based on the first cached token sequence. Here, since a partial token sequence can be obtained from the cache database in the pre-filling stage, and then the complete token sequence required for the subsequent inference stage can be calculated, the proposed solution effectively reduces the amount of token calculation in the pre-filling stage, shortens the time required to obtain the complete token sequence, and thus reduces the time spent by the model to output the first token. In other words, it effectively solves the problem of long first token consumption and poor user experience in long document reading comprehension scenarios, thereby effectively improving the user experience.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0021] Figure 1 This is a schematic diagram of a text-based question-answer generation method according to an embodiment of this application. Figure 1 ;

[0022] Figure 2 This is a schematic diagram of a text-based question-answer generation method according to an embodiment of this application. Figure 2 ;

[0023] Figure 3This is a scenario illustration of a text-based question-answer generation method according to an embodiment of this application in a specific example. Figure 1 ;

[0024] Figure 4(a) is a schematic diagram of the structure of a target model according to an embodiment of this application;

[0025] Figure 4(b) is a schematic illustration of a token sequence stored in a cache database according to an embodiment of the present application;

[0026] Figure 5 This is a flowchart illustrating a text-based question-answer generation method according to an embodiment of this application in a specific example;

[0027] Figure 6 This is a schematic diagram of a text-based question-answer generation method according to an embodiment of this application. Figure 3 ;

[0028] Figure 7(a) is a scene illustration of a text-based question-answer generation method according to an embodiment of this application in a specific example. Figure 2 ;

[0029] Figure 7(b) is a scene illustration of a text-based question-answer generation method according to an embodiment of this application in a specific example. Figure 3 ;

[0030] Figure 8 This is a schematic diagram four of a text-based question-answer generation method according to an embodiment of this application;

[0031] Figure 9 This is a schematic diagram of the structure of a text-based question-answer generation device according to an embodiment of this application;

[0032] Figure 10 This is a block diagram of an electronic device used to implement the text-based question-answer generation method of the embodiments of this disclosure. Detailed Implementation

[0033] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0034] In this document, the term "and / or" merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The term "at least one" in this document indicates any combination of at least two of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this document refer to and distinguish between multiple similar technical terms, not to restrict the order or to limit there to only two. For example, "first feature" and "second feature" refer to two categories / two features; the first feature can be one or more, and the second feature can also be one or more.

[0035] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can still be practiced even without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0036] In existing long document reading comprehension scenarios, the first token takes a long time to process, resulting in a poor user experience. To address these issues, current solutions typically employ the following approach; however, this approach usually suffers from its own problems:

[0037] First, shorten the length of the prompt message. Although this solution can reduce the time spent on the first token, the model's performance will decrease.

[0038] Secondly, token pruning, model compression, or optimization of attention mechanisms can be implemented. However, while these solutions can reduce the computational load of the model, they can also lead to a significant drop in model performance.

[0039] Therefore, there is an urgent need for a solution that can shorten the time required for the first token while effectively ensuring the model's performance.

[0040] Based on this, the present invention provides a text-based question-answer generation method that can obtain a pre-calculated token sequence of partial text content related to the question to be answered from the cache database during the pre-filling stage, thereby quickly obtaining the complete token sequence required for model inference. This shortens the time required to obtain the complete token sequence during the pre-filling stage, improves the model's inference efficiency, and reduces the time it takes for the model to output the first token. In other words, it effectively solves the problem of long first token consumption and poor user experience in long document reading comprehension scenarios, thereby effectively improving the user experience.

[0041] Specifically, Figure 1 This is an illustrative flow diagram of a text-based question-answer generation method according to an embodiment of this application. Figure 1 This method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices.

[0042] Furthermore, the method includes at least a portion of the following: For example... Figure 1 As shown, it includes:

[0043] Step S101: Obtain the first cache token sequence of the first sub-content in the target text from the cache database.

[0044] Here, the target text is the text determined based on the question to be answered.

[0045] For example, in one example, the target text could be the text retrieved from a document database based on the question to be answered, which is needed to answer the question; or, in another example, the target text could be the text uploaded by the target object, and the question to be answered could be the corresponding question posed to the target text uploaded by the target object.

[0046] Step S102: Based on the first cached token sequence, obtain the first target token sequence of the target text.

[0047] Here, compared to the first cached token sequence (which may be a token sequence containing part of the target text, in other words, it may be an incomplete token sequence of the target text), the first target token sequence is a complete token sequence of the target text; in other words, the first target token sequence is a token sequence containing the complete content of the target text.

[0048] Step S103: Obtain the second target token sequence for the question to be answered.

[0049] Here, the second target token sequence is a token sequence containing the complete content of the question to be answered.

[0050] Here, steps S102 and S103 can be executed simultaneously or sequentially. This disclosure does not restrict the execution order of the two.

[0051] Step S104: Perform model reasoning based at least on the first target token sequence and the second target token sequence to obtain the target answer for the question to be answered.

[0052] In other words, in this disclosed solution, after obtaining the question to be answered and determining the target text required to answer the question, the system first retrieves all cached token sequences for the target text from the cache database. For example, if the cache database currently only stores the first cached token sequence for the first sub-content of the target text and has not yet stored token sequences for the remaining content, then the system can retrieve the first cached token sequence for the first sub-content of the target text from the cache database. Further, in the pre-filling stage, based on the first cached token sequence retrieved from the cache database, the system obtains the first target token sequence for the target text (i.e., the complete token sequence for the target text) and the second target token sequence for the question to be answered (i.e., the complete token sequence for the question to be answered). Finally, in the model reasoning stage, the system uses the first target token sequence and the second target token sequence to perform model reasoning, thereby obtaining the target answer for the question to be answered. The target answer is the answer to the question to be answered based on the target text.

[0053] In this way, the proposed solution can first obtain the first cached token sequence of the first sub-content in the target text from the cache database during the pre-filling stage, and then quickly obtain the complete token sequence of the target text (i.e., the first target token sequence) and the complete token sequence of the question to be answered (i.e., the second target token sequence) based on the first cached token sequence. Here, since a partial token sequence can be obtained from the cache database in the pre-filling stage, and then the complete token sequence required for the subsequent inference stage can be calculated, the proposed solution effectively reduces the amount of token calculation in the pre-filling stage, shortens the time required to obtain the complete token sequence, and thus reduces the time spent by the model to output the first token. In other words, it effectively solves the problem of long first token consumption and poor user experience in long document reading comprehension scenarios, thereby effectively improving the user experience.

[0054] In addition, the disclosed solution does not change the model architecture and computational logic, but only shortens the processing time of the pre-filling stage, which has little impact on the inference effect and the additional cost is also low.

[0055] It should be noted that the large model used in this disclosure can be a large language model, a retrieval enhancement generation RAG model, etc., and this disclosure does not limit it.

[0056] Figure 2 This is an illustrative flow diagram of a text-based question-answer generation method according to an embodiment of this application. Figure 2 This method can be optionally applied to electronic devices, such as personal computers, servers, and server clusters. It is understood that the above... Figure 1The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0057] Furthermore, the method includes at least a portion of the following: For example... Figure 2 As shown, it includes:

[0058] Step S201: Obtain the first cache token sequence of the first sub-content in the target text from the cache database.

[0059] Here, the target text is the text determined based on the question to be answered.

[0060] It should be noted that relevant exemplary descriptions of the target text can be found above, and will not be repeated here.

[0061] Step S202: Based on the first cached token sequence, obtain the first target token sequence of the target text.

[0062] Step S203: Obtain the second target token sequence for the question to be answered.

[0063] Here, steps S202 and S203 can be executed simultaneously or sequentially. This disclosed solution does not restrict the execution order of the two.

[0064] Step S204: Determine the target prompt template.

[0065] Here, the target prompt template should at least include: the question to be answered and the target text determined based on the question to be answered.

[0066] It should be noted that the execution order of steps S204, S203, and S202 can be interchanged. For example, step S204 can be executed first, followed by steps S202 and S203. This disclosure does not limit this.

[0067] Step S205: Obtain the total target token sequence corresponding to the target prompt template based at least on the first target token sequence and the second target token sequence.

[0068] It should be noted that the relevant exemplary descriptions of the first target token sequence and the second target token sequence can be found above, and will not be repeated here.

[0069] Step S206: Perform model reasoning based on the total target token sequence corresponding to the target prompt template to obtain the target answer for the question to be answered.

[0070] In other words, in this example, after obtaining the complete token sequence for the target text (i.e., the first target token sequence) and the complete token sequence for the question to be answered (i.e., the second target token sequence), a total target token sequence can be constructed based on the target hint template. This total target token sequence is then populated into the large model. Based on this total target token sequence and utilizing the reasoning capabilities of the large model, the target answer for the question to be answered can be inferred. In this way, on the one hand, the hint function of the target hint template can be used to guide the large model in thinking and reasoning, enabling the large model to more quickly understand the relationship between the question to be answered and the target text, and obtain satisfactory reasoning results, thereby improving the user experience. On the other hand, it also effectively reduces the token computation in the pre-filling stage, shortens the time required to obtain the complete token sequence, and thus reduces the time required for the model to output the first token, further improving the user experience.

[0071] In one example, the target prompt template may include three parts: a prefix, search results, and a suffix. The prefix may primarily contain preset content needed to guide the large model in inference; the search results may primarily contain the target text used to answer the question; and the suffix may primarily contain the question entered by the target user.

[0072] For example, such as Figure 3 As shown, the target prompt template can be specifically a RAG Prompt, which can include a fixed prefix, search results, and a variable suffix. Further, in this example, the fixed prefix can specifically include a task description, such as, "If you are a professional robot, please refer to the search results to answer the question." The search results can specifically include the complete text content of the target text. The variable suffix can specifically be the question request (Query) or task instruction entered by the target object, such as the question to be answered.

[0073] Furthermore, upon receiving the question to be answered uploaded by the target, the system retrieves the complete token sequence of the task description from the cache database based on the content of the task description in the RAG Prompt. Since the task description is fixed, its complete token sequence can be pre-stored in the cache database. Additionally, based on the target text in the RAG Prompt, the system retrieves all token sequences currently stored in the cache database for that target text. It should be noted that in long document reading comprehension scenarios, the target text might be the document currently uploaded by the target. In this case, to quickly answer multiple questions entered by the target based on that target text, the cache database can pre-store the token sequence for that target text. However, it is understandable that when the question to be answered is received, the cache database may not yet have the complete token sequence of the target text stored. For example, it might only have the cache token sequence for the first sub-content of the target text stored. In this case, only the cache token sequence for the first sub-content of the target text can be retrieved from the cache database. Furthermore, it is understandable that since the question to be answered is constantly changing, the cache database does not need to pre-store the token sequence for that specific question.

[0074] Furthermore, in the pre-filling stage, the first target token sequence of the complete content of the target text can be calculated based on the first cache token sequence of the first sub-content of the target text obtained from the cache database, and the second target token sequence can be calculated based on the question to be answered, thereby obtaining the total target token sequence corresponding to the RAG Prompt; thus, in the inference stage, model inference can be performed based on the total target token sequence corresponding to the RAG Prompt to obtain the target answer for the question to be answered.

[0075] It should be noted that the cached token sequence for the target text cached in the cache database can be obtained based on the target model. For example, the token sequence for the target text can be calculated using the target model and then stored in the cache database.

[0076] Furthermore, in a specific example, the first cache token sequence stored in the cache database can be obtained in any of the following ways:

[0077] Method 1: The token sequence output by the first layer network module in the target model after extracting features from the first sub-content; that is, in Method 1, the first cached token sequence is the sequence output by the first layer network module in the target model.

[0078] Method 2: The token sequence output by the intermediate layer network module in the target model after extracting features from the first sub-content; that is, in Method 2, the first cached token sequence is the sequence output by the intermediate layer network module in the target model.

[0079] Method 3: The token sequence output by the last layer of the target model after extracting features from the first sub-content. In other words, in Method 3, the first cached token sequence is the sequence output by the last layer of the target model.

[0080] Here, the target model in this example contains multiple network modules that are connected in series. That is, in this target model, the output of the previous layer network module serves as the input of the next layer network module, the output of the last layer network module is the final output of the target model, and the input of the first layer network module is the relevant text in the target text.

[0081] Based on this, it can be seen that among the three methods mentioned above, only the sequence obtained by method 3 is the final output sequence of the target model, while the sequences output by methods 1 and 2 are intermediate sequences.

[0082] For example, as shown in Figure 4(a), assume that the target model contains N network modules, where N is an integer greater than or equal to 2; furthermore, the N network modules are concatenated, that is, the output of the previous network module is used as the input of the next network module; and the input of the target model (that is, the first sub-content of the target text) is directly used as the input of the first network module (that is, network module 1), and the output of the last network module is the final output of the target model.

[0083] Furthermore, each network module in the target model is used to extract feature data of the input content and output a token sequence.

[0084] For example, taking a target model containing three network modules as an example, as shown in Figure 4(b), when using each network module in the target model to extract features from the first sub-content of the target text, if the first sub-content has only completed feature extraction by network module 1 when the caching time of the token sequence is reached, the output result of network module 1 (for example, it can be recorded as token sequence 1) can be used as the first cached token sequence and stored in the cache database.

[0085] Alternatively, if the first sub-content has completed feature extraction by the first two network modules (i.e., network module 1 and network module 2) but has not yet completed feature extraction by network module 3 for the output result of network module 2, then the output result of network module 2 (for example, it can be denoted as token sequence 2) is used as the first cached token sequence and stored in the cache database.

[0086] Alternatively, if feature extraction of the first sub-content has been completed for all network modules in the target model, the output of network module 3 (for example, it can be denoted as token sequence 3) is used as the first cached token sequence and stored in the cache database.

[0087] It should be noted that the timing of caching the token sequence in this example can be the moment when the target object initiates a question to be answered, or it can be a pre-set caching time. This disclosed solution does not impose any specific restrictions on this.

[0088] In this way, when the caching time is reached, the proposed solution can obtain the processing result of the first sub-content at the network module level (i.e., the first cached token sequence) based on the processing status of each network module in the target model, and store it in the cache database. This effectively realizes the pre-caching of the token sequence. In the pre-filling stage, it is not necessary to calculate the complete token sequence of the target text from scratch, which effectively reduces the token calculation amount in the pre-filling stage, shortens the time required to obtain the complete token sequence, and thus reduces the time taken for the model to output the first token, thereby improving the user experience.

[0089] For example, such as Figure 5As shown, after obtaining the target text uploaded to the platform by the target object, the target model described above can be used to pre-compute a key-value cache (KV Cache) for the target text. At this point, if the key-value cache timing is reached (e.g., receiving a question from the target object), the token sequence output by each network module (block) in the target model is cached in the cache database (e.g., a key-value cache database (KV Cache Database)) based on the processing status of the target text by each network module (block) in the target model. In other words, the token sequence stored in the KV Cache Database is the output at the network module level. For example, for the first sub-content, if only the processing of the upper-level network module is completed but the processing of the lower-level network module is not completed when the caching timing is reached, the output result of the upper-level network module is stored in the KV Cache Database as the cache token sequence for the first sub-content. Alternatively, if the processing of the next-layer network module has been completed when the caching time is reached, there is no need to store the output of the previous-layer network module. Instead, the output of the next-layer network module is stored in the KV Cache Database as the cache token sequence for the first sub-content. This ensures that the cache token sequence stored in the KV Cache Database is closest to the sequence output by the last-layer network module, providing strong support for the subsequent rapid calculation of the first target token sequence. Otherwise, if the caching time is not reached, KV Cache pre-computation continues on the target text until the key-value caching time is reached or the KV Cache pre-computation of all content in the target text is completed.

[0090] Furthermore, such as Figure 5As shown, after obtaining the question to be answered from the target text initiated by the target object, firstly, the cached token sequence matching the target text is loaded from the KV Cache Database, for example, the first cached token sequence of the first sub-content in the target text is loaded; secondly, using the target model, based on the first cached token sequence of the first sub-content in the target text, the KV Cache pre-computation of all content in the target text is completed to obtain the first target token sequence for the target text and the second target token sequence for the question to be answered, thus completing the pre-filling and obtaining the total target token sequence corresponding to the target hint template; finally, model inference is performed based on the total target token sequence corresponding to the target hint template to obtain the target answer for the question to be answered. Thus, because this disclosed solution can retrieve the pre-computed token sequence from the cache database in real time and pre-fill the complete token sequence based on it, the pre-filling stage does not need to calculate the complete token sequence of the target text from scratch, effectively reducing the token calculation amount in the pre-filling stage, shortening the time required to obtain the complete token sequence, thereby reducing the time required for the model to output the first token and improving the user experience.

[0091] Figure 6 This is an illustrative flow diagram of a text-based question-answer generation method according to an embodiment of this application. Figure 3 This method can be optionally applied to electronic devices, such as personal computers, servers, and server clusters. It is understood that the above... Figures 1 to 5 The methods shown can also be applied to this example, and the related content will not be elaborated further in this example.

[0092] Furthermore, the method includes at least a portion of the following: For example... Figure 6 As shown, it includes:

[0093] Step S601: Obtain the first cache token sequence of the first sub-content in the target text from the cache database.

[0094] Here, the target text is the text determined based on the question to be answered.

[0095] It should be noted that relevant exemplary descriptions of the target text can be found above, and will not be repeated here.

[0096] Step S602: Based on the first cached token sequence, obtain the initial token sequence of the first sub-content.

[0097] Here, the initial token sequence for the first sub-content is the token sequence for the first sub-content output by the last layer network module in the target model.

[0098] Step S603: Obtain the initial token sequence of the remaining sub-contents in the target text, excluding the first sub-content.

[0099] Here, the initial token sequence for the remaining sub-content is also the token sequence for the remaining sub-content output by the last layer network module in the target model.

[0100] Step S604: Based on the initial token sequence of the first sub-content and the initial token sequences of the remaining sub-contents, obtain the first target token sequence of the target text.

[0101] It should be noted that, in order to better capture the contextual relationships of the text, the cached token sequence of the first sub-content needs to be processed to obtain the token sequence for the first sub-content output by the last layer network module in the target model (that is, the initial token sequence of the first sub-content).

[0102] Meanwhile, for the remaining sub-contents in the target text other than the first sub-content (i.e., other sub-contents for which the token sequence has not been loaded), the target model is invoked to extract features from the remaining sub-contents. At this point, the output of the last layer of the target model is the initial token sequence for that remaining sub-content. This facilitates the use of each initial token sequence to obtain the complete token sequence of the target text, thereby reducing the computational load and processing time in the pre-filling stage.

[0103] Step S605: Obtain the second target token sequence for the question to be answered.

[0104] Here, the steps of obtaining the second target token sequence of the question to be answered and obtaining the first target token sequence of the target text (e.g., steps S602-S604) can be executed simultaneously or interchanged. This disclosure does not impose any specific restrictions on this.

[0105] It should be noted that the relevant exemplary descriptions of the first target token sequence and the second target token sequence can be found above, and will not be repeated here.

[0106] Step S606: Perform model reasoning based at least on the first target token sequence and the second target token sequence to obtain the target answer for the question to be answered.

[0107] In this way, the disclosed solution can quickly obtain the token sequence output by the last layer network module in the target model based on the token sequence obtained from the cache database, and then quickly obtain the token sequence for the complete content of the target text. This effectively reduces the amount of computation of the token sequence of the target text in the pre-filling stage, shortens the time required for pre-filling, and reduces the time spent by the model to output the first token, thereby effectively improving the user experience.

[0108] Furthermore, in a specific example, the above-described method of obtaining the initial token sequence of the first sub-content based on the first cached token sequence (e.g., step S602 described above) can specifically include:

[0109] Step S602-1: Determine whether the first cached token sequence is the token sequence output by the last layer network module in the target model. If yes, proceed to step S602-4; otherwise, proceed to step S602-2.

[0110] Step S602-2: If the first cached token sequence is the token sequence output by a non-last layer network module in the target model, determine the network module that outputs the first cached token sequence.

[0111] Step S602-3: Using the next layer network module of the network module that outputs the first cached token sequence in the target model, process the first cached token sequence to obtain the initial token sequence for the first sub-content output by the last layer network module in the target model.

[0112] Step S602-4: If the first cached token sequence is the token sequence output by the last layer network module in the target model, the first cached token sequence is used as the initial token sequence of the first sub-content.

[0113] For example, continuing with Figure 4(b), if the first cached token sequence of the first sub-content in the target text is not the token sequence output by network module 3, and the network module that outputs the first cached token sequence is determined to be network module 2, then the next layer network module (i.e., network module 3) in the target model can be used to further process the first cached token sequence to obtain the token sequence output by the last layer network module (i.e., network module 3) in the target model. The token sequence output by the last layer network module (i.e., network module 3) is the initial token sequence for the first sub-content.

[0114] Alternatively, if the first cached token sequence of the first sub-content in the target text is the token sequence output by the last layer network module (i.e., network module 3), then the first cached token sequence can be directly used as the initial token sequence for the first sub-content.

[0115] In this way, the disclosed solution can quickly obtain the token sequence output by the last layer network module in the target model based on the first cached token sequence of the first cached sub-content. This provides strong support for quickly obtaining the token sequence of the complete content of the target text, thereby effectively reducing the computational amount of the token sequence of the target text in the pre-filling stage, shortening the time required for pre-filling, reducing the time spent by the model to output the first token, and thus effectively improving the user experience.

[0116] For example, as shown in Figure 7(a), the target text is divided into three parts (denoted as sub-content 1, sub-content 2, and sub-content 3, respectively) and input into the target model for KV Cache pre-computation. Further, when a question to be answered is received from the target object, the cache token sequence 1 of sub-content 1 is loaded from the cache database. This cache token sequence 1 is the token sequence output by the last layer network module in the target model, and the token sequences of sub-content 2 and sub-content 3 are not loaded. At this time, the cache token sequence 1 of sub-content 1 can be directly used as the initial token sequence 1 of sub-content 1. Further, in the pre-filling stage, by calling the target model, the remaining sub-contents of the target text (i.e., sub-content 2 and sub-content 3) are feature extracted to obtain the token sequences for the remaining sub-contents output by the last layer network module in the target model. For example, the initial token sequence 2 for sub-content 2 and the initial token sequence 3 for sub-content 3 are obtained. Thus, based on each initial token sequence, the first target token sequence of the target text is obtained, which facilitates the subsequent calculation of the second target token sequence based on the first target token sequence and the question to be answered, and then the total target token sequence used for model inference is obtained.

[0117] Alternatively, in another example, as shown in Figure 7(b), upon receiving an unanswered question initiated by the target object, the cached token sequence 1 of sub-content 1 is loaded from the cache database. This cached token sequence 1 is the token sequence output by the last layer network module in the target model. The cached token sequence 2 of sub-content 2 is also loaded. This cached token sequence 2 is the token sequence output by a non-last layer network module in the target model and is not loaded into the token sequence of sub-content 3. At this time, the cached token sequence 1 of sub-content 1 is used as the initial token sequence 1 of sub-content 1. Simultaneously, the next layer network module of the network module that outputs the cached token sequence 2 in the target model is used to process the cached token sequence 2 until the token sequence for sub-content 2 (i.e., the initial token sequence 2) output by the last layer network module in the target model is obtained. The target model is then called to extract features from the sub-content 3 of the target text until the token sequence for sub-content 3 (i.e., the initial token sequence 3) output by the last layer network module in the target model is obtained. Thus, based on each initial token sequence, the first target token sequence of the target text is obtained, which facilitates the subsequent calculation of the second target token sequence based on the first target token sequence and the question to be answered, and then the total target token sequence used for model inference is obtained.

[0118] Figure 8 This is a schematic flowchart four illustrating a text-based question-and-answer generation method according to an embodiment of this application. This method can be optionally applied to electronic devices, such as personal computers, servers, server clusters, and other electronic devices. It is understood that the above... Figure 1 The relevant content of the method shown in Figure 7 can also be applied to this example, and the related content will not be described again in this example.

[0119] Furthermore, the method includes at least a portion of the following: For example... Figure 8 As shown, it includes:

[0120] Step S801: Obtain the first cache token sequence of the first sub-content in the target text from the cache database.

[0121] Here, the target text is the text determined based on the question to be answered.

[0122] It should be noted that relevant exemplary descriptions of the target text can be found above, and will not be repeated here.

[0123] Step S802: Based on the first cached token sequence, obtain the first target token sequence of the target text.

[0124] For relevant exemplary descriptions of the first target token sequence, please refer to the above content, which will not be repeated here.

[0125] Step S803: Use the multiple network modules contained in the target model to extract features from the question to be answered, so as to obtain the token sequence for the question to be answered output by the last layer network module.

[0126] Step S804: Take the token sequence output by the last layer network module for the question to be answered as the second target token sequence for the question to be answered.

[0127] Here, step S802 and obtaining the second target token sequence of the question to be answered (i.e., steps S803 and S804) can be executed simultaneously or sequentially. This disclosed scheme does not restrict the execution order of the two.

[0128] Step S805: Perform model reasoning based at least on the first target token sequence and the second target token sequence to obtain the target answer for the question to be answered.

[0129] Thus, this disclosed solution provides a refined method for quickly obtaining the second target token sequence for the question to be answered. This solution is simple and efficient, providing strong support for rapid model inference. Furthermore, because this disclosed solution can retrieve a partial token sequence from the cache database during the pre-filling stage and then calculate the complete token sequence required for subsequent inference stages, it effectively reduces the amount of token computation in the pre-filling stage, shortens the time required to obtain the complete token sequence, and consequently reduces the time required for the model to output the first token. In other words, it effectively solves the problem of long first token processing time and poor user experience in long document reading comprehension scenarios, thereby effectively improving the user experience.

[0130] This disclosure also provides a text-based question-answer generation device, such as... Figure 9 As shown, it includes:

[0131] The acquisition unit 901 is used to acquire a first cache token sequence of the first sub-content in the target text from the cache database, wherein the target text is the text determined based on the question to be answered;

[0132] The preprocessing unit 902 is configured to obtain a first target token sequence of the target text based on the first cached token sequence; and to obtain a second target token sequence of the question to be answered.

[0133] The model reasoning unit 903 is used to perform model reasoning based at least on the first target token sequence and the second target token sequence to obtain the target answer for the question to be answered.

[0134] In a specific example of the scheme disclosed herein, the first cache token sequence is one of the following:

[0135] The token sequence output by the first-layer network module in the target model after extracting features from the first sub-content;

[0136] The token sequence output by the intermediate layer network module in the target model after extracting features from the first sub-content;

[0137] The token sequence output by the last layer of the network module in the target model after extracting features from the first sub-content;

[0138] The target model contains multiple network modules that are connected in series. Each network module in the target model is used to extract feature data of the input content and output a token sequence.

[0139] In a specific example of the disclosed solution, the preprocessing unit is specifically used for:

[0140] Based on the first cached token sequence, the initial token sequence of the first sub-content is obtained; wherein, the initial token sequence of the first sub-content is the token sequence output by the last layer network module in the target model for the first sub-content;

[0141] Obtain the initial token sequence of the remaining sub-contents in the target text excluding the first sub-content, where the initial token sequence of the remaining sub-contents is the token sequence for the remaining sub-contents output by the last layer network module in the target model;

[0142] Based on the initial token sequence of the first sub-content and the initial token sequences of the remaining sub-contents, the first target token sequence of the target text is obtained.

[0143] In a specific example of the disclosed solution, the preprocessing unit is specifically used for:

[0144] If the first cached token sequence is output by a non-last layer network module in the target model, the network module that outputs the first cached token sequence is determined.

[0145] The first cached token sequence is processed by the next layer network module of the network module that outputs the first cached token sequence in the target model, so as to obtain the initial token sequence for the first sub-content output by the last layer network module in the target model.

[0146] In a specific example of the disclosed solution, the preprocessing unit is specifically used for:

[0147] If the first cached token sequence is the output of the last layer network module in the target model, the first cached token sequence is used as the initial token sequence of the first sub-content.

[0148] In a specific example of the disclosed solution, the preprocessing unit is specifically used for:

[0149] The target model uses multiple network modules to extract features from the question to be answered, so as to obtain the token sequence for the question to be answered output by the last network module.

[0150] The token sequence output by the last layer network module for the question to be answered is used as the second target token sequence for the question to be answered.

[0151] In a specific example of the scheme disclosed herein, the preprocessing unit is further configured to determine a target prompt template; wherein the target prompt template includes at least: a question to be answered and target text determined based on the question to be answered; and at least based on a first target token sequence and a second target token sequence, a total target token sequence corresponding to the target prompt template is obtained;

[0152] The model reasoning unit is used to perform model reasoning based on the total target token sequence corresponding to the target prompt template, so as to obtain the target answer for the question to be answered.

[0153] For a description of the specific functions and examples of each unit of the apparatus in this disclosure embodiment, please refer to the relevant descriptions of the corresponding steps in the above method embodiments, which will not be repeated here.

[0154] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0155] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0156] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0157] like Figure 10As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1002 or a computer program loaded from storage unit 1008 into random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0158] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0159] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as a text-based question-and-answer generation method. For example, in some embodiments, the text-based question-and-answer generation method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the text-based question-and-answer generation method described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform a text-based question-answer generation method by any other suitable means (e.g., by means of firmware).

[0160] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0162] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0163] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0164] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0165] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0166] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0167] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A text-based question-answer generation method, comprising: The first cache token sequence of the first sub-content in the target text is obtained from the cache database. The target text is the text determined based on the question to be answered. The first cache token sequence is obtained by using the network module in the target model to extract features of the first sub-content after obtaining the target text, and caching the token sequence output by the network module in the target model into the cache database when the caching time of the token sequence is reached. Based on the first cached token sequence and using the target model, the target text is further processed to obtain the first target token sequence of the target text; and to obtain the second target token sequence of the question to be answered. Model reasoning is performed based at least on the first target token sequence and the second target token sequence to obtain the target answer for the question to be answered.

2. The method according to claim 1, wherein, The first cached token sequence is one of the following: The token sequence output by the first-layer network module in the target model after extracting features from the first sub-content; The token sequence output by the intermediate layer network module in the target model after extracting features from the first sub-content; The token sequence output by the last layer of the network module in the target model after extracting features from the first sub-content; The target model contains multiple network modules that are connected in series. Each network module in the target model is used to extract feature data of the input content and output a token sequence.

3. The method according to claim 2, wherein, The step of further processing the target text based on the first cached token sequence and utilizing the target model to obtain the first target token sequence of the target text includes: Based on the first cached token sequence, the initial token sequence of the first sub-content is obtained; wherein, the initial token sequence of the first sub-content is the token sequence output by the last layer network module in the target model for the first sub-content; Obtain the initial token sequence of the remaining sub-contents in the target text excluding the first sub-content, where the initial token sequence of the remaining sub-contents is the token sequence for the remaining sub-contents output by the last layer network module in the target model; Based on the initial token sequence of the first sub-content and the initial token sequences of the remaining sub-contents, the first target token sequence of the target text is obtained.

4. The method according to claim 3, wherein, The process of obtaining the initial token sequence for the first sub-content based on the first cached token sequence includes: If the first cached token sequence is output by a non-last layer network module in the target model, the network module that outputs the first cached token sequence is determined. The first cached token sequence is processed by the next layer network module of the network module that outputs the first cached token sequence in the target model, so as to obtain the initial token sequence for the first sub-content output by the last layer network module in the target model.

5. The method according to claim 3, wherein, The process of obtaining the initial token sequence for the first sub-content based on the first cached token sequence includes: If the first cached token sequence is the output of the last layer network module in the target model, the first cached token sequence is used as the initial token sequence of the first sub-content.

6. The method according to any one of claims 2-5, wherein, The second target token sequence for obtaining the question to be answered includes: The target model uses multiple network modules to extract features from the question to be answered, so as to obtain the token sequence for the question to be answered output by the last network module. The token sequence output by the last layer network module for the question to be answered is used as the second target token sequence for the question to be answered.

7. The method according to any one of claims 2-5, further comprising: Determine the target prompt template; wherein the target prompt template shall at least include: the question to be answered and the target text determined based on the question to be answered; Wherein, the step of performing model reasoning based at least on the first target token sequence and the second target token sequence to obtain the target answer for the question to be answered includes: Based at least on the first target token sequence and the second target token sequence, the total target token sequence corresponding to the target prompt template is obtained; The model reasoning is performed based on the total target token sequence corresponding to the target prompt template to obtain the target answer for the question to be answered.

8. A text-based question-answer generation device, comprising: The acquisition unit is used to acquire a first cached token sequence of the first sub-content in the target text from the cache database. The target text is the text determined based on the question to be answered. The first cached token sequence is obtained after the target text is acquired, the network module in the target model is used to extract features of the first sub-content, and the token sequence output by the network module in the target model is cached into the cache database when the caching time of the token sequence is reached. A preprocessing unit is configured to further process the target text based on the first cached token sequence and using the target model to obtain a first target token sequence of the target text; and to obtain a second target token sequence of the question to be answered. The model reasoning unit is used to perform model reasoning based at least on the first target token sequence and the second target token sequence to obtain the target answer for the question to be answered.

9. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images