Method, device, equipment and storage medium for processing text reasoning tasks

Abstract

The present disclosure provides a method, device, equipment and storage medium for processing text reasoning tasks, and relates to the field of computer technology, and in particular to the fields of artificial intelligence, deep learning, neural network models, and large language model technologies. The specific implementation scheme is: using a preset model to execute multiple first text reasoning tasks in parallel; when the completion status of multiple first text reasoning tasks meets preset conditions, determine a second text reasoning task from the text reasoning tasks to be executed according to the completion status; determine multiple third text reasoning tasks according to the second text reasoning tasks and the unfinished first text reasoning tasks; and use the preset model to execute multiple third text reasoning tasks in parallel. The present disclosure can improve the throughput of the model in executing text reasoning tasks and improve processing efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the fields of artificial intelligence, deep learning, neural network models, and large language model technologies. Background Art

[0002] When training and reasoning a neural network model, the input information of a group of tasks is usually grouped into a batch and processed in parallel to improve computing efficiency. Since the generative large language model is iteratively generated based on probability, the lengths of the output results of multiple tasks in the same batch are usually not the same, and there may even be a large gap. Summary of the invention

[0003] The present disclosure provides a method, apparatus, device and storage medium for processing text reasoning tasks.

[0004] According to one aspect of the present disclosure, a method for processing a text reasoning task is provided, comprising:

[0005] Utilize a preset model to perform multiple first text reasoning tasks in parallel;

[0006] When the completion status of the plurality of first text reasoning tasks meets a preset condition, determining a second text reasoning task from the text reasoning tasks to be performed according to the completion status;

[0007] Determining a plurality of third text reasoning tasks according to the second text reasoning task and the unfinished first text reasoning task; and

[0008] Using the preset model, multiple third-party text reasoning tasks are performed in parallel.

[0009] According to another aspect of the present disclosure, a device for processing a text reasoning task is provided, comprising:

[0010] A first execution module, used to execute multiple first text reasoning tasks in parallel using a preset model;

[0011] A new task determination module, used to determine a second text reasoning task from the text reasoning tasks to be executed according to the completion status when the completion status of the plurality of first text reasoning tasks meets a preset condition;

[0012] A task scheduling module, configured to determine a plurality of third text reasoning tasks according to the second text reasoning task and the unfinished first text reasoning task; and

[0013] The second execution module is used to use a preset model to execute multiple third text reasoning tasks in parallel.

[0014] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0015] at least one processor; and

[0016] a memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any method in the embodiments of the present disclosure.

[0018] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any method according to the embodiments of the present disclosure.

[0019] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements any method according to the embodiments of the present disclosure when executed by a processor.

[0020] The present disclosure can improve the throughput of the model in executing text reasoning tasks and improve processing efficiency.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0023] Figure 1 is a flowchart of a method for processing a text reasoning task according to an embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of a preset model operation flow according to an embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram of a text reasoning task case according to an embodiment of the present disclosure;

[0026] Figure 4 is a schematic diagram of a timeline of task processing according to an embodiment of the present disclosure;

[0027] Figure 5 is a structural diagram of a device for processing a text reasoning task according to an embodiment of the present disclosure;

[0028] Figure 6 The block diagram is a block diagram of an electronic device used to implement the method for processing a text reasoning task according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0030] With the development of natural language generation technology, large language models (LLM) have gradually become popular. In large language models and common natural language generation models, autoregressive decoding is an indispensable part. As described above in the background technology, in the related technology, for a batch of tasks, it is necessary to start the generation of the next batch after all the tasks of the batch are uniformly generated. Specifically, the lengths of the output results of a batch of tasks are usually different, and the task with a shorter output needs to wait for the task with a longer output until the task with the longest output is completed before starting to process the next batch of tasks.

[0031] In order to at least partially solve one or more of the above problems and other potential problems, the embodiments of the present disclosure propose a method for processing text reasoning tasks. Using the technical solution of the embodiments of the present disclosure, new text reasoning tasks can be dynamically inserted to replace completed tasks based on the completion status of several text reasoning tasks in a batch. Avoid waiting for long outputs for short outputs in the generation (inference) task, use resources more efficiently, and improve the throughput of task processing.

[0032] Figure 1 FIG. 1 is a flow chart of a method for processing a text reasoning task provided by an embodiment of the present disclosure. Figure 1 As shown, the method comprises at least the following steps:

[0033] S101. Execute multiple first text reasoning tasks in parallel using a preset model.

[0034] The preset model can be a trained generative model, for example, a natural language generation model. In specific examples, it can be an ERNIE Bot model, a Generative Pretrained Transformer (GPT) model, a Chat-based Generative Pretrained Transformer (ChatGPT) model, a Dialogue Generative Pretrained Transformer (DialoGPT) model, or a Bidirectional and Auto-Regressive Transformers (BART) model.

[0035] Common text reasoning tasks or text generation tasks in the field of natural language may include chat dialogues, question-answering, machine translation, article writing and other generation tasks. The first text reasoning task can be determined based on the demand text obtained from the user demand information. The user demand information can be obtained in the form of text, in the form of voice, or in a combination of multiple data forms. For example, the user demand information may be a voice, and the content of the voice is: "What's the weather like today?" Based on the user demand information, the corresponding demand text can be obtained through voice recognition technology. For another example, the user inputs a text through text and asks: "Please translate this text", then the demand text corresponding to the user demand information includes the original text to be translated and "Please translate this text".

[0036] According to the user demand information, a specific demand text is obtained. The demand text reflects the user's actual request (query). The demand text is input into the trained model for reasoning operation to generate output text. When the first text reasoning task is executed, the model obtains the task result through at least one reasoning operation, and performs one reasoning operation in each cycle. Each cycle outputs a vocabulary unit (token). After multiple rounds of cycles, the model outputs multiple vocabulary units, and the output text is composed of multiple vocabulary units. The output text can be a sentence, a paragraph, or an article.

[0037] For another example, the user's query is to translate a text into a specified language. The text to be translated is input into the model for loop operation. The model generates a token each time it operates in a loop. That is, in each loop operation, the model generates a translation word corresponding to a word in the text to be translated. After n loop operations, the text is translated and the corresponding translated text is obtained.

[0038] The first text reasoning task can be a text generation task in text form, and the corresponding vocabulary units in the output sequence include characters, phrases, punctuation marks, etc. It can also be a text generation task in non-text form, for example, a music score generation task, and the corresponding vocabulary units in the output sequence include notes, key signatures, beats, bar lines, etc.

[0039] Multiple first-text inference tasks form a batch. In natural language generation tasks, a batch usually refers to the number of tasks processed in parallel at a time, also known as batch size. Batch size is a very important hyperparameter in deep learning models and usually needs to be adjusted according to specific tasks and computing resources.

[0040] The input of a natural language generation model is usually one or more text sequences. For example, in a machine translation task, the input can be a text sequence in the source language, and the output can be a text sequence in the target language. In a dialogue generation task, the input can be a text sequence of the dialogue history, and the output can be a text sequence of the reply.

[0041] In natural language generation tasks, each text reasoning task corresponds to an input sequence, which usually refers to the input information of the text to be generated, including but not limited to context, task description, conditions, etc. The input information can be in the form of text, image, voice and other forms, and different tasks require different input information. For input information in text form, it is usually a sequence consisting of several vocabulary units, so it is called an input sequence. For example, for machine translation tasks, the input information is usually the source language sentence, and for dialogue generation tasks, the input information is usually the dialogue history. In the input sequence, each vocabulary unit (element) usually represents a word or a subword, and these elements form a token sequence.

[0042] S102: When the completion status of the plurality of first text reasoning tasks meets a preset condition, determine a second text reasoning task from the text reasoning tasks to be executed according to the completion status.

[0043] The completion status can be understood as whether each of the multiple first text reasoning tasks has been completed. Usually, the input sequence of each first text reasoning task has a corresponding serial number in the batch. Since the serial numbers are generally continuous, they can also be understood as the position of the input sequence in the batch. The completion status may include which positions of the input sequence have been completed. In one example, the batch size is 10, and there are currently 10 first text reasoning tasks being executed, and the corresponding input sequence numbers are 0 to 9. After multiple rounds of cyclic calculations of the model, the completion status at a certain moment may be: text reasoning tasks numbered 1, 2, 5, 6, 8, and 9 have been completed. Tasks numbered 0, 3, 4, and 7 are not yet completed.

[0044] The preset condition is used to determine whether the currently executed first text reasoning task needs to be replaced based on the completion status. Replacement refers to inserting a new text reasoning task into the position of the completed first text reasoning task. Task replacement requires stopping the model's decoder Decoder, returning to the encoding stage, using the encoder Encoder to encode the input sequence of the new text reasoning task, and returning to the decoding stage after encoding is completed. It can be seen that the insertion of the new task will affect the decoding calculation of the unfinished first text reasoning task. Therefore, replacing the task when the completion status meets the preset conditions can reduce the impact on the unfinished first text reasoning task.

[0045] The text reasoning task to be executed can be understood as a new text reasoning task that has not been executed yet, which can be obtained by requests made by different users, or by a new request made by the same user when the previous text reasoning task has not been completed.

[0046] The second text reasoning task can be understood as one or more text reasoning tasks to be executed selected from the text reasoning tasks to be executed, and the specific number can be determined according to the completion status.

[0047] S103: Determine a plurality of third text reasoning tasks according to the second text reasoning task and the unfinished first text reasoning task.

[0048] The selected new text reasoning task and the previously unfinished first text reasoning task are reorganized into a batch.

[0049] S104: Utilize the preset model to execute multiple third text reasoning tasks in parallel.

[0050] After reorganizing a batch of text reasoning tasks, continue to use the model to execute the batch of tasks. It should be noted that in the process of the preset model executing multiple third text reasoning tasks, the third text reasoning tasks can be regarded as the first text reasoning tasks, and the above steps S101 to S104 are re-executed, so that when the completion of multiple third text reasoning tasks meets the preset conditions, the completed third text reasoning tasks are replaced again and new text reasoning tasks are inserted.

[0051] According to the scheme of the embodiment of the present disclosure, the completed text reasoning task can be replaced with a new text reasoning task according to the completion status of the text reasoning task, so as to improve the throughput of the model in executing the text reasoning task and improve the processing efficiency. At the same time, the task replacement needs to meet the preset conditions to reduce the impact on the unfinished text reasoning tasks.

[0052] In a possible implementation, step S102 determines a second text reasoning task from the text reasoning tasks to be performed according to the completion status when the completion status of the plurality of first text reasoning tasks meets a preset condition, further comprising the steps of:

[0053] S1021: When an output sequence corresponding to any first text reasoning task among the plurality of first text reasoning tasks includes an end tag, determining that the completion status of the first text reasoning task is completed, wherein the output sequence includes a plurality of vocabulary units generated by a preset model according to the first text reasoning task.

[0054] The output sequence is usually composed of multiple vocabulary units, tokens, which can be words, phrases, punctuation marks, etc. When the model is input, the text is usually split into several tokens, and each token is assigned a unique ID as the input of the model. In the natural language generation model, the Encoder encodes the input token sequence into a set of vector representations, and the Decoder generates the next token based on the output of the Encoder and the previously generated tokens. The Decoder does not know whether the generated text is complete. The usual practice is to preset an end marker stop_flag in the model. For example, the end marker of a sentence can be a period or a question mark. The model will continue to generate tokens when generating text until the end marker is generated. This method is called "autoregressive decoding" because the model needs to refer to the previously generated token when generating each token. When the model generates an end marker, it is considered that the current text has been generated and the text generation process ends. The condition for the Decoder to generate an end marker is usually a predefined special symbol. For example, in machine translation tasks, " <eos>"Indicates the end of the sentence, the Decoder is generating" <eos>" will stop generating the next word, thus judging that the output text has been generated.

[0055] S1022: When the number of completed first text reasoning tasks meets a target threshold value of a preset condition, stop executing unfinished first text reasoning tasks.

[0056] The target threshold of the preset condition can be used to control the timing of stopping the execution of the unfinished first text reasoning task to avoid stopping too frequently. Specifically, it can be avoided that a new task is inserted every time a text reasoning task is completed, because the insertion of the new task will cause the unfinished first text reasoning task to stop executing.

[0057] It should be noted that the completed first text reasoning task does not automatically stop after the end tag is generated, because the completed first text reasoning task is still running in the model after generating the end tag. The decoder will perform a pad operation on short output sequences to ensure that the length of the output sequence is the same as the longest output sequence in the batch until the batch is fully generated. Therefore, even if the generation of short sequences has been completed, the decoder still needs to perform a pad operation on them. However, the computational resources required for the sequence that has been completed are smaller than those for the output sequence that has not been completed, that is, the computational resources required for the pad operation are smaller, and the impact on performance is also smaller. However, since the completed first text reasoning task is still running, when the execution of the unfinished first text reasoning task is stopped, the completed first text reasoning task is also stopped at the same time.

[0058] It should also be noted that the embodiment of the present disclosure needs to replace the completed first text reasoning task with the text reasoning task to be executed. The absence of a text reasoning task to be executed is a special case and is not within the scope of discussion of the embodiment of the present disclosure. In order to avoid exceptions caused by special cases, it is also possible to determine whether there is a text reasoning task to be executed at the same time or before determining whether the number of completed first text reasoning tasks meets the target threshold of the preset condition. If there is no text reasoning task to be executed, there is no need to stop executing the current first text reasoning task.

[0059] S1023. Determine a second text reasoning task from the text reasoning tasks to be performed according to the completion status.

[0060] In one example, according to the completion status, 24 of the 30 first text reasoning tasks being executed have been completed, and then 24 of them can be obtained from the text reasoning tasks to be executed to obtain 24 second text reasoning tasks.

[0061] In the disclosed embodiment, whether the number of completed first text reasoning tasks reaches a target threshold is used as a preset condition that needs to be met for inserting a new task.

[0062] According to the solution of the embodiment of the present disclosure, when a completed task is replaced with a new task, the impact on the unfinished text reasoning tasks is taken into consideration.

[0063] In a possible implementation, step S1023 determines a second text reasoning task from the text reasoning tasks to be performed according to the completion status, and further includes the steps of:

[0064] S1023a. Determine the number of unfinished first text reasoning tasks and the maximum value of the task sequence numbers of the unfinished first text reasoning tasks according to the completion status.

[0065] The task sequence number of the first text reasoning task refers to the sorting number of the first text reasoning task in its batch. The maximum value of the task sequence number refers to the maximum value of the task sequence numbers of one or more unfinished first text reasoning tasks. The task sequence number can also be understood as the position of the task in the batch of tasks. The task sequence number is usually reflected in the subscript of the input sequence corresponding to the text reasoning task. The task sequence number can also be called the position subscript. The maximum value of the task sequence number can also be called the maximum unfinished position or the maximum subscript of the unfinished position. Still taking the previous example, the tasks corresponding to task numbers 0, 3, 4, and 7 are still unfinished, then the maximum value (maximum subscript) of the task sequence number is 7.

[0066] S1023b, determining the number of text reasoning tasks to be performed.

[0067] S1023c: Determine a second text reasoning task from the text reasoning tasks to be executed according to the number of unfinished first text reasoning tasks, the maximum value of the task sequence number, and the number of text reasoning tasks to be executed.

[0068] In the above example, the first text reasoning tasks corresponding to task numbers 1, 2, 5, 6, 8, and 9 have been completed, that is, 6 first text reasoning tasks have been completed, the number of unfinished tasks is 4, and the maximum value of the task number is 7. If the number of text reasoning tasks to be executed is sufficient, 6 can be obtained from the text reasoning tasks to be executed to obtain 6 second text reasoning tasks to replace all completed first text reasoning tasks. If the number of text reasoning tasks to be executed is not enough to replace the corresponding tasks of all the ended positions in the batch, in order to ensure that the model resumes normal operation, it is necessary to ensure that there are a certain number of new tasks based on the maximum unfinished position to replace the ended position before the maximum unfinished position. The specific implementation method will be described below.

[0069] According to the solution of the embodiment of the present disclosure, it can be ensured that an appropriate number of second text reasoning tasks are selected to replace the completed first text reasoning tasks.

[0070] In a possible implementation, step S1023c determines the second text reasoning task from the text reasoning tasks to be executed according to the number of unfinished first text reasoning tasks, the maximum value in the task sequence number, and the number of text reasoning tasks to be executed, further comprising the steps of:

[0071] When the sum of the number of text reasoning tasks to be executed and the number of unfinished first text reasoning tasks is less than the maximum value of the task sequence number, the second text reasoning task is determined based on all the text reasoning tasks to be executed and the empty tasks, wherein the number of empty tasks is the difference between the maximum value in the task sequence number and the number of text reasoning tasks to be executed.

[0072] In the disclosed embodiment, the number of ended positions that must be filled with tasks can be determined based on the maximum value of the task sequence number and the number of unfinished first text reasoning tasks. If the text reasoning tasks to be executed are insufficient, empty tasks are used to fill them. Continuing with the above example, the maximum unfinished position is 7, and the number of unfinished first text reasoning tasks is 4 (the task sequences are 0, 3, 4, and 7, respectively), which means that there are 4 ended positions that must be filled with tasks before the maximum unfinished position (the task sequences are 1, 2, 5, and 6, respectively). If the number of text reasoning tasks to be executed is less than 4, empty tasks are used to fill them. For example, if there are only 2 text reasoning tasks to be executed, the text reasoning tasks to be executed can be inserted into positions 1 and 2, and positions 5 and 6 can be filled with empty tasks. Of course, other combinations are also possible.

[0073] According to the scheme of the embodiment of the present disclosure, tasks are completed by maintaining the maximum unfinished position of the previous round. When the text reasoning tasks to be executed are not sufficient to replace all the completed first text reasoning tasks, partial replacement can also be performed by using the empty task completion method.

[0074] In a possible implementation, step S1023c determines the second text reasoning task from the text reasoning tasks to be executed according to the number of unfinished first text reasoning tasks, the maximum value in the task sequence number, and the number of text reasoning tasks to be executed, further comprising the steps of:

[0075] When the sum of the number of text reasoning tasks to be executed and the number of unfinished first text reasoning tasks is greater than or equal to the maximum value of the task sequence number, the number of insertable tasks is determined based on the number of unfinished first text reasoning tasks and the maximum number of tasks to be processed in parallel.

[0076] According to the number of insertable tasks, at least a portion of the text reasoning tasks to be executed are selected from the text reasoning tasks to be executed to determine a second text reasoning task.

[0077] In the disclosed embodiment, if the sum of the number of text reasoning tasks to be executed and the number of unfinished first text reasoning tasks is greater than or equal to the maximum value of the task sequence number, there is no need to use empty tasks to complete the tasks. The text reasoning tasks to be executed can be used to replace the first text reasoning tasks corresponding to the completed positions before the maximum unfinished position. At this time, there are still two situations, one is: the text reasoning tasks to be executed are sufficient to replace the first text reasoning tasks corresponding to all the ended positions, and the other is: the number of text reasoning tasks to be executed is not enough to fill the ended positions after the maximum unfinished position. For the first case, it is necessary to select a part of the text reasoning tasks to be executed to obtain the second text reasoning task. The specific number depends on the batch size. For the first case, all the text reasoning tasks to be executed are used as the second text reasoning tasks, and the insufficient part can be left blank or the completed first text reasoning tasks corresponding to the position can continue to be retained.

[0078] According to the solution of the embodiment of the present disclosure, it can be ensured that task replacement can be implemented regardless of whether the task load is under high pressure or low pressure.

[0079] In a possible implementation, the method for processing a text reasoning task in an embodiment of the present disclosure further includes the steps of:

[0080] In response to the preset model stopping the execution of the unfinished first text reasoning task, the target threshold is updated according to the maximum number of tasks processed in parallel, the adjustment parameters and the number of completed first text reasoning tasks to obtain the preset conditions that need to be met when the preset model stops executing the unfinished first text reasoning task next time.

[0081] In the disclosed embodiment, whenever the completion status of the first text reasoning task meets the preset conditions and the preset model stops executing the text reasoning task, the next target threshold is calculated based on the maximum number of tasks processed in parallel, the adjustment parameters, and the number of completed first text reasoning tasks at the time of this round of stopping, thereby obtaining the preset conditions that need to be met for stopping the execution of the text reasoning task next time.

[0082] The maximum number of tasks processed in parallel is recorded as batch_size, the number of completed first text inference tasks is recorded as stop_nums, the maximum value of stop_nums is batch_size-1, and when stop_nums is equal to batch_size, it means that all tasks in this batch are completed. The process of calculating the stop threshold stop_threshold from stop_nums and batch_size can be expressed as:

[0083]

[0084] Among them, 1 / 2 in the formula is an adjustment parameter, and the denominator 2 can also be set to other values ​​greater than 1, preferably between 1 and 10. When the number of completed tasks in the batch reaches stop_threshold, the generation process is stopped and new tasks are inserted. For example, when a batch task with 10 tasks processed in parallel just starts to execute, stop_nums is 0. According to the calculation result of the formula, it is 5.5, and the calculation result can be rounded down, so stop_threshold is 5, which means that when the number of completed first text reasoning tasks is 5, the generation process can be stopped and new tasks can be inserted. The next target threshold calculated at this time is 8. Therefore, when the number of completed tasks of the third text reasoning task reaches 8, the generation process is stopped again and new tasks are inserted.

[0085] In the high pressure steady state (all tasks in the batch are not completed and are still being generated), the stop_threshold value is In low-pressure scenarios (most tasks have been generated), stop_threshold will gradually approach batch_size. As can be seen, each time the process stops, the next target threshold calculated will be larger than the target threshold of the current round. stop_threshold will gradually approach batch_size, that is, when stop_threshold is equal to batch_size, it means that no new tasks will be inserted, and all text inference tasks will be completed, and the batch of tasks will be completely completed.

[0086] It should be noted that the adjustment parameter can be a fixed value, or it can be gradually reduced as each round of model stops. For example, it can be reduced by 1 from 10 each round until the value is 1. This makes the dynamic adjustment range of the target threshold larger.

[0087] According to the solution of the embodiment of the present disclosure, by dynamically adjusting the target threshold, different target thresholds are matched under different task load conditions, thereby avoiding idling of hardware resources, greatly improving throughput, and also preventing a batch from being unable to complete.

[0088] In a possible implementation, the target threshold is determined according to at least one of the following methods:

[0089] Determined by preset value.

[0090] Determined by the maximum number of tasks processed in parallel and adjustment parameters.

[0091] It is determined based on the maximum number of tasks processed in parallel, adjustment parameters, and the number of completed first text reasoning tasks when the preset model last stopped executing the unfinished first text reasoning tasks.

[0092] In the embodiment of the present disclosure, the preset value may be any value within the batch size, and may be used when a batch is just being executed. The adjustment parameter may be 1 / 2, 1 / 3, 1 / 4, 1 / 5, etc. as described above.

[0093] According to the solution of the embodiment of the present disclosure, a variety of target threshold design methods are provided, which can be flexibly used according to different types of text reasoning tasks.

[0094] In a possible implementation, the method for processing a text reasoning task in an embodiment of the present disclosure further includes the steps of:

[0095] An output text of the first text reasoning task is determined according to a plurality of vocabulary units in an output sequence corresponding to the first text reasoning task.

[0096] In the disclosed embodiment, for the first text reasoning task, multiple vocabulary units in the corresponding output sequence are used as output text.

[0097] In one example, if Figure 3 As shown in the figure, there are two text reasoning tasks, namely Task 1. "Help me write a year-end summary of the BIM center of the construction unit" and Task 2. "How to make more money". The model infers predictor to generate output text. The text generated for Task 2 may be: "The application of BIM technology in the implementation unit [ <n>]Abstract: With the rapid development of my country's economy, construction... ". At this time, the text generated for Task 2 may be: "1. Find a good project, 2. Promote it well, 3. Persist in execution, 4. Learn to review [ <n>][gEND]". Task 2 is completed before Task 1, and an end marker is generated. Then, a new query can be inserted, for example: "Why are mosquitoes not killed by raindrops?" After updating the input sequence through the above batch scheduling method, reasoning is restarted.

[0098] The text generated for Task 2 might be: "The construction industry has also developed rapidly. BI##M technology, as a new technical means, has been widely used in the construction industry." The text generated for Task 2 might be: "The body of a mosquito is very small. When raindrops hit its body, a concave surface is formed. The body of a mosquito is streamlined, so it will not be killed. [gEND]" At this time, Task 2 is generated and a new task can be inserted again.

[0099] It should be noted that after generating one or a preset number of vocabulary units or a whole sentence, the output text can be combined and output. Therefore, during the execution of the third text reasoning task, the output text can also be determined and output according to the multiple vocabulary units in the corresponding output sequence. It can also be output after the end tag is generated. It should be noted that the token after the end tag needs to be ignored.

[0100] According to the solution of the embodiment of the present disclosure, during the generation process of the first text reasoning task, the output text is obtained according to the output sequence.

[0101] In a possible implementation, step S104 uses a preset model to perform multiple third text reasoning tasks in parallel, including the following steps:

[0102] Using the encoder of the preset model, a context vector corresponding to the second text reasoning task is determined according to the input information of the second text reasoning task.

[0103] According to the preset mapping relationship, the context vector and output sequence corresponding to the unfinished first text reasoning task are read from the cache space.

[0104] Utilizing the decoder of the preset model, the next vocabulary unit of the output sequence corresponding to multiple third text reasoning tasks is generated based on the context vector corresponding to the second text reasoning task, the context vector corresponding to the unfinished first text reasoning task, and the output sequence, so as to execute the multiple third text reasoning tasks in parallel.

[0105] In the embodiments of the present disclosure, the natural language generation model generally adopts a neural network structure called a transformer. The runtime includes an encoding phase and a decoding phase. Figure 2 As shown in the figure, the encoding stage mainly uses the encoder to transform the input sequence through a series of hierarchical transformations and then generate a context to obtain a special representation called a "context vector". This context vector can be regarded as a summary and abstraction of the entire input sequence.

[0106] In the decoding phase, the context vector is provided as input to the decoder. The decoder decodes the context vector and the previously generated partial output results, and gradually generates the next token in the sentence. In this process, the model also maintains a cache space Cache to store the previously generated intermediate results. The specific content of the cache varies from model to model, but usually includes the following information:

[0107] Context vector: The context vector is generated in the Encoder stage, so it is also passed to the Decoder stage and is part of the Cache.

[0108] Already generated words: When the Decoder generates output, it generates each part of the sentence word by word, so the already generated words are necessary information and will also be stored in the Cache.

[0109] Encoder output: The decoding stage writes some information to the cache at each step. This information is usually extracted from the output of the Encoder to assist the Decoder in generating the next vocabulary unit.

[0110] Decoder status: The status information of the decoder will also be stored in the cache. This status information usually includes some information related to the current generation position, such as the position of the decoder, the current calculation status, etc.

[0111] The role of the cache is to store the previous intermediate results to assist the decoder in generating the next vocabulary unit. By using the context vector generated by the encoder and the partial output results that have been generated previously, the decoder stage can maintain the context coherence and semantic consistency in the process of generating sentences.

[0112] When the batch task is executed, first apply for cache space according to the batch size and the maximum length of the input and output sequences, and establish a mapping relationship between each text inference task and multiple cache areas in the cache to obtain a preset mapping relationship. All subsequent batches of cache management are performed in this space. When part of the text inference task has been completed (partial_stop), the corresponding position of the task in the cache space can be determined according to the position information of the batch task, which is convenient for task replacement.

[0113] According to the solution of the embodiment of the present disclosure, the cache of each batch is managed in a unified manner, which can reduce the delay increase caused by the state recovery of the unfinished text reasoning task.

[0114] In a possible implementation, the method further includes:

[0115] According to a preset mapping relationship, output sequences corresponding to the plurality of fourth text reasoning tasks are cached in a cache space used to cache context vectors corresponding to the plurality of fourth text reasoning tasks.

[0116] The preset mapping relationship includes a correspondence between task serial numbers of a plurality of fourth text reasoning tasks and storage location information in the cache space. The fourth text reasoning task includes a plurality of first text reasoning tasks and a third text reasoning task.

[0117] In the embodiment of the present disclosure, when applying for Cache, an additional generated character cache area will be applied for caching the output sequence. In the decoding stage, the vocabulary units generated by each decoding will be input into the cache area in real time. In the related art, the output sequence and the input sequence are cached in different cache space locations, and the output sequence of the unfinished first text reasoning task needs to be moved when the model stops. However, the solution of the embodiment of the present disclosure does not need to move all the generated characters after the stop condition is triggered. When continuing to generate, the unfinished position can be efficiently restored to the last decoding state.

[0118] According to the solution of the embodiment of the present disclosure, the context vector and the output sequence are cached in the same cache space, the cache space is managed uniformly, and the first text inference task at an unterminated position can be efficiently restored to the last decoding state.

[0119] In a possible implementation, the method further includes:

[0120] According to the preset mapping relationship and the task sequence number of the second text reasoning task, target position information of the completed first text reasoning task corresponding to the second text reasoning task in the cache space is determined.

[0121] According to the target position information, a context vector corresponding to the second text reasoning task is stored in the cache space.

[0122] In the disclosed embodiment, when performing task replacement, a task number is assigned to the second text reasoning task according to the correspondence between the replaced first text reasoning task and the second text reasoning task that replaces the first text reasoning task, and replacement is performed in the cache space according to the task number. For example, the completed first text reasoning task A has a task number of 4, and the second text reasoning task A that replaces the first text reasoning task A will inherit its task number 4. According to the task number and the preset mapping relationship, the cache area used by the first text reasoning task A is determined in the cache space. The data required to be cached by the second text reasoning task A is stored in the cache area used by the first text reasoning task A.

[0123] According to the solution of the embodiment of the present disclosure, the problem of inconsistency in caches of different batches can be avoided.

[0124] In a possible implementation, the method further includes:

[0125] When an output sequence corresponding to any text reasoning task among the multiple first text reasoning tasks includes an end tag, the decoding length of the first text reasoning task is set to 0.

[0126] In the disclosed embodiment, when tasks are stopped one by one, the decoding length associated with the task will be set to 0, that is, the generation will be completely terminated, and the calculation amount at that position will be completely eliminated during the reasoning process.

[0127] According to the solution of the embodiment of the present disclosure, the computational complexity of the completed first text reasoning task is completely eliminated, further improving the reasoning performance.

[0128] In one example, if Figure 4 As shown in the figure, there are 4 first text reasoning tasks in a batch, namely batch[0], batch[1], batch[2], and batch[3]. After batch scheduling, cache space is applied, and the cache and tasks are mapped according to the storage location information to obtain a preset mapping relationship. Then the input sequence seq[0], seq[1], seq[2], and seq[3] corresponding to each first text reasoning task is input to the Encoder, and the Encoder calculates and obtains the corresponding context vector. The context vector corresponding to the first text reasoning task is cached in the cache. The Decoder reads the context vector corresponding to the first text reasoning task and starts decoding. Each round of calculation generates the next token of each first text reasoning task, which is recorded as token[0], token[1], etc. These tokens are also cached in the above cache according to the preset mapping relationship. After multiple rounds of cycles, the tokens generated by the first text reasoning tasks at positions 0, 1, and 2 are end markers. At this time, the completion status meets the preset conditions, all first text reasoning tasks are stopped, and batch scheduling is performed again. Insert three text reasoning tasks to be executed at positions 0, 1, and 2 to form a new batch of text reasoning tasks, among which the first text reasoning task at position 3 is retained. The model needs to encode the second text reasoning task, and the first text reasoning task at position 3 is still in the stopped decoding state. After the second text reasoning task is encoded, it returns to the decoding stage. The third text reasoning task at positions 0, 1, and 2 begins decoding, and the third text reasoning task at position 3 reads its context vector and the generated token[3][0], token[3][1], and token[3][2] from the cache, and continues to generate token[3][3]. After multiple rounds of cyclic calculations, the third text reasoning tasks at positions 1, 2, and 3 are generated one by one. At this time, batch scheduling can be performed again to insert new tasks for positions 1, 2, and 3.

[0129] According to the solution of the embodiment of the present disclosure, through cache external management and task completion, adaptive termination and dynamic insertion module, the waiting of short output for long output in the generation task is avoided, resources are used more efficiently, and the throughput of the whole system is improved.

[0130] Figure 5 Schematic diagram of the structure of a device for processing text reasoning tasks provided by an embodiment of the present disclosure. Figure 5 As shown, the device comprises:

[0131] The first execution module 501 is used to execute multiple first text reasoning tasks in parallel using a preset model.

[0132] The new task determination module 502 is used to determine a second text reasoning task from the text reasoning tasks to be executed according to the completion status when the completion status of the plurality of first text reasoning tasks meets a preset condition.

[0133] The task scheduling module 503 is used to determine a plurality of third text reasoning tasks according to the second text reasoning tasks and the unfinished first text reasoning tasks.

[0134] The second execution module 504 is used to execute multiple third text reasoning tasks in parallel using a preset model.

[0135] In a possible implementation, the new task determination module 502 includes:

[0136] The completion detection submodule is used to determine that the completion of the first text reasoning task is completed when the output sequence corresponding to any first text reasoning task among the multiple first text reasoning tasks contains an end tag. The output sequence includes a plurality of vocabulary units generated by a preset model according to the first text reasoning task.

[0137] The stop detection submodule is used to stop executing the unfinished first text reasoning tasks when the number of completed first text reasoning tasks meets a target threshold of a preset condition.

[0138] The determination submodule is used to determine a second text reasoning task from the text reasoning tasks to be performed according to the completion status.

[0139] In a possible implementation, the submodule is determined to be used for:

[0140] According to the completion status, the number of unfinished first text reasoning tasks and the maximum value of the task sequence numbers of the unfinished first text reasoning tasks are determined.

[0141] Determines the number of text reasoning tasks to perform.

[0142] According to the number of unfinished first text reasoning tasks, the maximum value of the task sequence number, and the number of text reasoning tasks to be executed, a second text reasoning task is determined from the text reasoning tasks to be executed.

[0143] In a possible implementation, the determination submodule is further used for:

[0144] When the sum of the number of text reasoning tasks to be executed and the number of unfinished first text reasoning tasks is less than the maximum value of the task sequence number, the second text reasoning task is determined based on all the text reasoning tasks to be executed and the empty tasks, where the number of empty tasks is the difference between the maximum value in the task sequence number and the number of text reasoning tasks to be executed.

[0145] In a possible implementation, the determination submodule is further used for:

[0146] When the sum of the number of text reasoning tasks to be executed and the number of unfinished first text reasoning tasks is greater than or equal to the maximum value of the task sequence number, the number of insertable tasks is determined based on the number of unfinished first text reasoning tasks and the maximum number of tasks to be processed in parallel.

[0147] According to the number of insertable tasks, at least a portion of the text reasoning tasks to be executed are selected from the text reasoning tasks to be executed to determine a second text reasoning task.

[0148] In a possible implementation, the device further includes:

[0149] A threshold update module is used to update the target threshold in response to the preset model stopping the execution of the unfinished first text reasoning task, according to the maximum number of tasks processed in parallel, the adjustment parameters and the number of completed first text reasoning tasks, so as to obtain the preset conditions that need to be met when the preset model stops executing the unfinished first text reasoning task next time.

[0150] In a possible implementation, the target threshold is determined according to at least one of the following methods:

[0151] Determined by preset value.

[0152] Determined by the maximum number of tasks processed in parallel and adjustment parameters.

[0153] It is determined based on the maximum number of tasks processed in parallel, adjustment parameters, and the number of completed first text reasoning tasks when the preset model last stopped executing the unfinished first text reasoning tasks.

[0154] In a possible implementation, the device further includes:

[0155] The output module is used to determine the output text of the first text reasoning task according to a plurality of vocabulary units in the output sequence corresponding to the first text reasoning task.

[0156] In a possible implementation, the second execution module 504 includes:

[0157] The encoding submodule is used to determine the context vector corresponding to the second text reasoning task according to the input information of the second text reasoning task by using the encoder of the preset model.

[0158] The reading submodule is used to read the context vector and output sequence corresponding to the unfinished first text reasoning task from the cache space according to a preset mapping relationship.

[0159] A decoding submodule is used to use a decoder of a preset model to generate the next vocabulary unit of the output sequence corresponding to multiple third text reasoning tasks based on the context vector corresponding to the second text reasoning task, the context vector corresponding to the unfinished first text reasoning task, and the output sequence, so as to execute the multiple third text reasoning tasks in parallel.

[0160] In a possible implementation, the device further includes:

[0161] A cache module is used to cache output sequences corresponding to multiple fourth text reasoning tasks in a cache space used to cache context vectors corresponding to multiple fourth text reasoning tasks according to a preset mapping relationship.

[0162] The preset mapping relationship includes a correspondence between task serial numbers of a plurality of fourth text reasoning tasks and storage location information in the cache space. The fourth text reasoning task includes a plurality of first text reasoning tasks and a third text reasoning task.

[0163] In a possible implementation, the device further includes:

[0164] The cache replacement submodule is used to determine the target position information of the completed first text reasoning task corresponding to the second text reasoning task in the cache space according to the preset mapping relationship and the task sequence number of the second text reasoning task.

[0165] According to the target position information, a context vector corresponding to the second text reasoning task is stored in the cache space.

[0166] In a possible implementation, the device further includes:

[0167] The computation amount elimination module is used to set the decoding length of the first text reasoning task to 0 when the output sequence corresponding to any text reasoning task among the multiple first text reasoning tasks includes an end tag.

[0168] For the description of specific functions and examples of each module and submodule of the device in the embodiment of the present disclosure, reference can be made to the relevant description of the corresponding steps in the above method embodiment, which will not be repeated here.

[0169] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0170] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0171] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0172] like Figure 6 As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0173] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0174] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as the method of text reasoning task processing. For example, in some embodiments, the method of text reasoning task processing may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method of text reasoning task processing described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the method of text reasoning task processing in any other appropriate manner (e.g., by means of firmware).

[0175] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0176] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0177] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0178] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0179] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0180] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0181] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0182] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the principles of the present disclosure shall be included in the protection scope of the present disclosure.< / n> < / n> < / eos> < / eos>

Claims

1. A method for processing a text reasoning task, comprising: Utilize a preset model to perform multiple first text reasoning tasks in parallel; When the completion status of the plurality of first text reasoning tasks meets a preset condition, determining a second text reasoning task from the text reasoning tasks to be performed according to the completion status; Determining a plurality of third text reasoning tasks according to the second text reasoning task and the unfinished first text reasoning task; as well as Using the preset model, executing the plurality of third text reasoning tasks in parallel; Wherein, determining a second text reasoning task from the text reasoning tasks to be performed according to the completion status includes: According to the completion status, determining the number of the unfinished first text reasoning tasks and the maximum value of the task sequence numbers of the unfinished first text reasoning tasks; determining the number of text reasoning tasks to be performed; According to the number of the unfinished first text reasoning tasks, the maximum value of the task sequence number, and the number of the text reasoning tasks to be executed, a second text reasoning task is determined from the text reasoning tasks to be executed.

2. The method according to claim 1, wherein: When the completion status of the plurality of first text reasoning tasks meets a preset condition, determining a second text reasoning task from the text reasoning tasks to be performed according to the completion status includes: When an output sequence corresponding to any first text reasoning task among the plurality of first text reasoning tasks includes an end tag, determining that the completion status of the first text reasoning task is completed; wherein the output sequence includes a plurality of vocabulary units generated by the preset model according to the first text reasoning task; When the number of completed first text reasoning tasks meets a target threshold value of a preset condition, stopping execution of unfinished first text reasoning tasks; According to the completion status, a second text reasoning task is determined from the text reasoning tasks to be performed.

3. The method according to claim 1, wherein: Determining a second text reasoning task from the text reasoning tasks to be executed according to the number of the unfinished first text reasoning tasks, the maximum value in the task sequence number, and the number of the text reasoning tasks to be executed, comprising: When the sum of the number of the text reasoning tasks to be executed and the number of the unfinished first text reasoning tasks is less than the maximum value of the task sequence number, the second text reasoning task is determined based on all the text reasoning tasks to be executed and the empty tasks, wherein the number of the empty tasks is the difference between the maximum value in the task sequence number and the number of the text reasoning tasks to be executed.

4. The method according to claim 1, wherein: Determining a second text reasoning task from the text reasoning tasks to be executed according to the number of the unfinished first text reasoning tasks, the maximum value in the task sequence number, and the number of the text reasoning tasks to be executed, comprising: In the case where the sum of the number of the to-be-executed text reasoning tasks and the number of the unfinished first text reasoning tasks is greater than or equal to the maximum value of the task sequence number, determining the number of insertable tasks according to the number of the unfinished first text reasoning tasks and the maximum number of tasks to be processed in parallel; According to the number of insertable tasks, at least a portion of the text reasoning tasks to be executed are selected from the text reasoning tasks to be executed to determine a second text reasoning task.

5. The method according to claim 2, further comprising: In response to the preset model stopping the execution of the unfinished first text reasoning task, the target threshold is updated according to the maximum number of tasks processed in parallel, the adjustment parameters and the number of completed first text reasoning tasks to obtain the preset conditions that need to be met when the preset model stops executing the unfinished first text reasoning task next time.

6. The method according to claim 2 or 5, wherein: The target threshold is determined according to at least one of the following methods: Determined according to preset values; Determined by the maximum number of tasks to be processed in parallel and the adjustment parameters; The method is determined based on the maximum number of tasks to be processed in parallel, the adjustment parameter, and the number of completed first text reasoning tasks when the preset model last stopped executing an unfinished first text reasoning task.

7. The method according to claim 1, further comprising: An output text of the first text reasoning task is determined according to a plurality of vocabulary units in an output sequence corresponding to the first text reasoning task.

8. The method according to claim 1, wherein: Using the preset model, executing the plurality of third text reasoning tasks in parallel includes: Determining, using the encoder of the preset model, a context vector corresponding to the second text reasoning task according to the input information of the second text reasoning task; According to a preset mapping relationship, read the context vector and output sequence corresponding to the unfinished first text reasoning task from the cache space; Utilizing the decoder of the preset model, the next vocabulary unit of the output sequence corresponding to the multiple third text reasoning tasks is generated based on the context vector corresponding to the second text reasoning task, the context vector corresponding to the unfinished first text reasoning task, and the output sequence, so as to execute the multiple third text reasoning tasks in parallel.

9. The method according to claim 8, further comprising: According to the preset mapping relationship, the output sequences corresponding to the plurality of fourth text reasoning tasks are cached in a cache space used for caching context vectors corresponding to the plurality of fourth text reasoning tasks; Among them, the preset mapping relationship includes the corresponding relationship between the task serial numbers of multiple fourth text reasoning tasks and the storage location information in the cache space; the fourth text reasoning task includes the multiple first text reasoning tasks and the third text reasoning task.

10. The method according to claim 9, further comprising: Determine, according to the preset mapping relationship and the task sequence number of the second text reasoning task, target location information of the completed first text reasoning task corresponding to the second text reasoning task in the cache space; According to the target position information, a context vector corresponding to the second text reasoning task is stored in the cache space.

11. The method according to claim 2, further comprising: When an output sequence corresponding to any text reasoning task among the multiple first text reasoning tasks includes an end tag, the decoding length of the first text reasoning task is set to 0.

12. A device for processing a text reasoning task, comprising: A first execution module, used to execute multiple first text reasoning tasks in parallel using a preset model; A new task determination module, configured to determine a second text reasoning task from the text reasoning tasks to be executed according to the completion status of the plurality of first text reasoning tasks when the completion status of the plurality of first text reasoning tasks meets a preset condition; A task scheduling module, used for determining a plurality of third text reasoning tasks according to the second text reasoning tasks and the unfinished first text reasoning tasks; as well as A second execution module, configured to execute the plurality of third text reasoning tasks in parallel using the preset model; Wherein, the new task determination module is used for: According to the completion status, determining the number of the unfinished first text reasoning tasks and the maximum value of the task sequence numbers of the unfinished first text reasoning tasks; determining the number of text reasoning tasks to be performed; According to the number of the unfinished first text reasoning tasks, the maximum value of the task sequence number, and the number of the text reasoning tasks to be executed, a second text reasoning task is determined from the text reasoning tasks to be executed.

13. The device according to claim 12, wherein: The new task determination module comprises: a completion detection submodule, configured to determine that the completion status of any first text reasoning task among the plurality of first text reasoning tasks is completed when the output sequence corresponding to the first text reasoning task includes an end tag; wherein the output sequence includes a plurality of vocabulary units generated by the preset model according to the first text reasoning task; A stop detection submodule, configured to stop executing the unfinished first text reasoning tasks when the number of completed first text reasoning tasks meets a target threshold value of a preset condition; The determination submodule is used to determine a second text reasoning task from the text reasoning tasks to be performed according to the completion status.

14. The device according to claim 13, wherein: The determining submodule is also used for: When the sum of the number of the text reasoning tasks to be executed and the number of the unfinished first text reasoning tasks is less than the maximum value of the task sequence number, the second text reasoning task is determined based on all the text reasoning tasks to be executed and the empty tasks, wherein the number of the empty tasks is the difference between the maximum value in the task sequence number and the number of the text reasoning tasks to be executed.

15. The device according to claim 13, wherein: The determining submodule is also used for: In the case where the sum of the number of the to-be-executed text reasoning tasks and the number of the unfinished first text reasoning tasks is greater than or equal to the maximum value of the task sequence number, determining the number of insertable tasks according to the number of the unfinished first text reasoning tasks and the maximum number of tasks to be processed in parallel; According to the number of insertable tasks, at least a portion of the text reasoning tasks to be executed are selected from the text reasoning tasks to be executed to determine a second text reasoning task.

16. The apparatus according to claim 13, further comprising: A threshold updating module is used to update the target threshold in response to the preset model stopping the execution of the unfinished first text reasoning task, according to the maximum number of tasks processed in parallel, the adjustment parameters and the number of the completed first text reasoning tasks, so as to obtain the preset conditions that need to be met when the preset model stops executing the unfinished first text reasoning task next time.

17. The device according to claim 13 or 16, wherein: The target threshold is determined according to at least one of the following methods: Determined according to preset values; Determined by the maximum number of tasks to be processed in parallel and the adjustment parameters; The method is determined based on the maximum number of tasks to be processed in parallel, the adjustment parameter, and the number of completed first text reasoning tasks when the preset model last stopped executing an unfinished first text reasoning task.

18. The apparatus according to claim 12, further comprising: An output module is used to determine the output text of the first text reasoning task according to a plurality of vocabulary units in an output sequence corresponding to the first text reasoning task.

19. The device according to claim 12, wherein: The second execution module includes: An encoding submodule, used to determine a context vector corresponding to the second text reasoning task according to input information of the second text reasoning task using an encoder of the preset model; A reading submodule, configured to read the context vector and output sequence corresponding to the unfinished first text reasoning task from the cache space according to a preset mapping relationship; A decoding submodule is used to use the decoder of the preset model to generate the next vocabulary unit of the output sequence corresponding to the multiple third text reasoning tasks according to the context vector corresponding to the second text reasoning task, the context vector corresponding to the unfinished first text reasoning task and the output sequence, so as to execute the multiple third text reasoning tasks in parallel.

20. The apparatus according to claim 19, further comprising: A cache module, used for caching the output sequences corresponding to the plurality of fourth text reasoning tasks in a cache space used for caching the context vectors corresponding to the plurality of fourth text reasoning tasks according to the preset mapping relationship; Among them, the preset mapping relationship includes the corresponding relationship between the task serial numbers of multiple fourth text reasoning tasks and the storage location information in the cache space; the fourth text reasoning task includes the multiple first text reasoning tasks and the third text reasoning task.

21. The apparatus according to claim 20, further comprising: A cache replacement submodule, used to determine target position information of the completed first text reasoning task corresponding to the second text reasoning task in the cache space according to the preset mapping relationship and the task sequence number of the second text reasoning task; According to the target position information, a context vector corresponding to the second text reasoning task is stored in the cache space.

22. The apparatus according to claim 13, further comprising: A computational complexity elimination module is used to set the decoding length of any text reasoning task among the multiple first text reasoning tasks to 0 when the output sequence corresponding to the first text reasoning task contains an end tag.

23. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-11.

25. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 11.

Images