Data processing method, server and client device

Abstract

The invention discloses a data processing method, a server and a client device, the server obtains a reference text sequence and a candidate text sequence to be verified, the candidate text sequence is a text sequence reasoned by a client based on the reference text sequence by using a first model, and the candidate text sequence comprises at least one candidate lexical element; the reference text sequence comprises prompt words used for indicating reasoning tasks and lexical elements verified by the server; a second model is called to process the reference text sequence and the candidate text sequence, comprehensive probability distribution output by the second model is obtained, the comprehensive probability distribution comprises probability distribution of lexical element positions of all candidate lexical elements in the candidate text sequence in a word list, and the parameter magnitude of the second model is larger than that of the first model; based on the probability distribution corresponding to each candidate lexical element in the candidate text sequence, determining a corrected candidate text sequence; and returning the corrected candidate text sequence to the client as a verified reference text sequence.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, server, and client device. Background Technology

[0002] With the continuous development of artificial intelligence technology, the application of model-based task reasoning is becoming increasingly widespread. For example, models can be used to generate prompts based on input to answer questions or summarize content.

[0003] Given the complexity of task inference, in many cases, it is necessary to utilize large-scale models deployed on the server side to complete the task inference. However, due to the large scale of the models deployed on the server side, each call to the model to perform inference operations consumes a lot of resources, and completing task inference requires multiple loop calls to the intelligent model, resulting in excessive resource consumption for completing task inference. Summary of the Invention

[0004] In a first aspect, this application provides a data processing method applied to a server for task inference, comprising:

[0005] Obtain a reference text sequence and a candidate text sequence to be verified. The candidate text sequence is a text sequence inferred by the client based on the reference text sequence using a first model. The candidate text sequence includes at least one candidate word. The reference text sequence includes prompt words for indicating the inference task and words verified by the server.

[0006] The second model is invoked to process the reference text sequence and the candidate text sequence to obtain the comprehensive probability distribution output by the second model. The comprehensive probability distribution includes the probability distribution of the position of each candidate word in the candidate text sequence in the vocabulary. The parameter magnitude of the second model is greater than that of the first model.

[0007] Based on the probability distribution corresponding to each candidate word in the candidate text sequence, the corrected candidate text sequence is determined;

[0008] The corrected candidate text sequence is returned to the client as a verified reference text sequence.

[0009] In one possible implementation of the first aspect, determining the modified candidate text sequence based on the probability distribution corresponding to each candidate word in the candidate text sequence includes:

[0010] Based on the probability distribution of each candidate word in the candidate text sequence, each candidate word in the candidate text sequence is corrected in sequence according to the order of the candidate words in the candidate text sequence to obtain the corrected candidate text sequence.

[0011] In yet another possible implementation of the first aspect, obtaining the reference text sequence and the candidate text sequence to be verified includes:

[0012] Obtain the target text sequence and verification indication information sent by the client, wherein the verification indication information is used to indicate the at least one candidate word character to be verified in the target text sequence;

[0013] Based on the target text sequence and the verification indication information, a reference text sequence and a candidate text sequence to be verified are determined.

[0014] In another possible implementation of the first aspect, determining the reference text sequence and the candidate text sequence to be verified based on the target text sequence and the verification indication information includes:

[0015] In response to the verification indication information indicating that each word in the target text sequence is a candidate word, the target text sequence is determined as a candidate text sequence to be verified, and the reference text sequence determined by the server and corresponding to the reasoning task is obtained.

[0016] In response to the verification indication information indicating that some words in the target text sequence are candidate words, a reference text sequence and a candidate text sequence to be verified are determined from the target text sequence based on the verification indication information.

[0017] In another possible implementation of the first aspect, the verification indication information includes at least one of the number of verified lexical units and the number of lexical units to be verified;

[0018] The verification indication information indicates that each word in the target text sequence is a candidate word, including at least one of the following:

[0019] The number of verified lexical units is zero;

[0020] The number of lexical units to be verified is the total number of lexical units in the target text sequence;

[0021] The verification indication information indicates that some words in the target text sequence are candidate words, including at least one of the following:

[0022] The number of verified lexical units is not zero;

[0023] The number of lexical units to be verified is less than the total number of lexical units in the target text sequence.

[0024] A second aspect of this application provides a data processing method applied to a client for task reasoning, comprising:

[0025] In response to the detection of a task request, the prompt words in the task request are identified as the reference text sequence corresponding to the reasoning task, and the prompt words are used to indicate the reasoning task;

[0026] Based on the reference text sequence, the first model is invoked to perform task inference to obtain the inferred candidate text sequence, wherein the candidate text sequence includes at least one candidate word;

[0027] At least the candidate text sequence should be sent to the server;

[0028] Obtain the corrected candidate text sequence returned by the server. The corrected candidate text sequence is determined by the server based on the candidate text sequence using a second model. The parameter magnitude of the second model is greater than that of the first model.

[0029] If, based on the corrected candidate text sequence, it is determined that the reasoning task has not yet been completed, the reference text sequence and the corrected candidate text sequence are used as the updated reference text sequence, and the operation of calling the first model to perform the task reasoning is returned.

[0030] If the reasoning required to complete the reasoning task is determined based on the modified candidate text sequence, the reasoning result corresponding to the reasoning task is determined based on the reference text sequence and the modified candidate text sequence.

[0031] In one possible implementation of the second aspect, sending at least the candidate text sequence to the server includes:

[0032] The reference text sequence and the candidate text sequence are combined into a target text sequence;

[0033] The target text sequence and verification indication information are sent to the server, wherein the verification indication information is used to indicate at least one candidate word to be verified in the candidate text sequence.

[0034] In another possible implementation of the second aspect, combining the reference text sequence and the candidate text sequence into a target text sequence includes:

[0035] In response to the fact that the reference text sequence only includes the prompt word, the reference text sequence and the candidate text sequence are combined into a target text sequence;

[0036] In response to the reference text sequence including prompt words and a modified historical candidate text sequence, the candidate text sequence is determined as the target text sequence.

[0037] A third aspect of this application provides a server, including: a processor, a memory, and a transmission module;

[0038] The memory is used to store the computer programs required for the processor to perform operations;

[0039] The processor is configured to execute a computer program stored in the memory to perform the following operations:

[0040] Obtain a reference text sequence and a candidate text sequence to be verified. The candidate text sequence is a text sequence inferred by the client based on the reference text sequence using a first model. The candidate text sequence includes at least one candidate word. The reference text sequence includes prompt words for indicating the inference task and words verified by the server.

[0041] The second model is invoked to process the reference text sequence and the candidate text sequence to obtain the comprehensive probability distribution output by the second model. The comprehensive probability distribution includes the probability distribution of the position of each candidate word in the candidate text sequence in the vocabulary. The parameter magnitude of the second model is greater than that of the first model.

[0042] Based on the probability distribution corresponding to each candidate word in the candidate text sequence, the corrected candidate text sequence is determined;

[0043] The sending module is used to return the corrected candidate text sequence to the client as a verified reference text sequence.

[0044] A fourth aspect of this application provides a client device, including: a processor, a transmitting module, and a receiving module;

[0045] The processor is configured to, in response to detecting a task request, determine the prompt word in the task request as a reference text sequence corresponding to the inference task, the prompt word being used to indicate the inference task; and, based on the reference text sequence, invoke a first model to perform task inference to obtain a candidate text sequence, the candidate text sequence including at least one candidate word.

[0046] The sending module is used to send at least the candidate text sequence to the server;

[0047] The receiving module is used to obtain the corrected candidate text sequence returned by the server. The corrected candidate text sequence is determined by the server based on the candidate text sequence using a second model. The parameter magnitude of the second model is greater than that of the first model.

[0048] The processor is further configured to: if, based on the corrected candidate text sequence, determine that the reasoning task has not yet been completed, use the reference text sequence and the corrected candidate text sequence as the updated reference text sequence, and return to the operation of calling the first model to perform task reasoning; if, based on the corrected candidate text sequence, determine that the reasoning task has been completed, determine the reasoning result corresponding to the reasoning task based on the reference text sequence and the corrected candidate text sequence. Attached Figure Description

[0049] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0050] Figure 1 A schematic diagram of the data processing method provided in this application on the client side;

[0051] Figure 2 A schematic diagram of the data processing method provided in this application on the server side;

[0052] Figure 3 A comparative example diagram is shown, illustrating the difference between current solutions for handling reasoning tasks and the solution proposed in this application.

[0053] Figure 4 This invention illustrates a flowchart of the data processing method provided in this application.

[0054] Figure 5 A comparative example diagram is shown, illustrating existing solutions for handling reasoning tasks and the solution of this application.

[0055] Figure 6 This paper illustrates an example of an interaction between a client and a server in performing a reasoning task in this application.

[0056] Figure 7 A schematic diagram of the server architecture provided in this application;

[0057] Figure 8 This is a schematic diagram of the architecture of a client device provided in this application. Detailed Implementation

[0058] The solution proposed in this application is applicable to scenarios where a server is used to perform task reasoning on a reasoning task obtained from a client. The reasoning task in this application is mainly for processing text content, such as refining, expanding, and answering questions on the text information obtained from the client using a model deployed on the server side, without any specific limitations.

[0059] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is only for explaining specific embodiments and is not intended to limit the application. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0060] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0061] To facilitate understanding of the scheme in this application, we will first introduce it from the perspectives of the client and server sides of task inference.

[0062] First, the data processing method of this application will be introduced from the client side.

[0063] like Figure 1 This illustration shows a flowchart of a data processing method provided in this application. The method of this embodiment can be applied to a task reasoning client, and the method of this embodiment may include:

[0064] S101, in response to detecting a task request, the prompt words in the task request are identified as the reference text sequence corresponding to the reasoning task.

[0065] The prompt word is used to instruct the reasoning task. For example, if the prompt word is "Please name the highest mountain in my country", then the prompt word is used to instruct the client to deduce the highest mountain in my country.

[0066] In this application, the reference text sequence is a text sequence used for performing task reasoning, and the reference text sequence includes at least one token. A token (also called a word) is the smallest semantic unit of the text sequence; for example, a token can be a word, a Chinese character, a punctuation mark, a character sequence, or a phrase.

[0067] It is understandable that when a client receives a task request, it will perform task inference based on the prompt words. Therefore, when a client receives a task request, the prompt words can be used as a reference text sequence.

[0068] S102, based on the reference text sequence, call the first model to perform task inference and obtain the inferred candidate text sequence.

[0069] The first model is the client-side deployed model. Considering the limited computing resources of the electronic devices on the client side, the client-side deployed first model is relatively smaller in scale compared to the server-side deployed second model. Therefore, the number of parameters in the first model is smaller than that in the server-side deployed second model.

[0070] In one possible implementation, the second model is a model determined based on the first model deployed on the server side, and whose model structure similarity to the second model meets the requirements. For example, the second model can be a model whose compositional structure is more similar to the first model than a set threshold. For instance, the first model and the second model can be two models with the same structure but different parameter scales developed by the same model developer.

[0071] The candidate text sequence includes at least one candidate word.

[0072] This candidate text sequence is inferred by the client based on the reference text sequence. Because the client's first model is relatively small, the inference efficiency and time consumption of the client-inferred candidate text sequence are relatively low. However, the accuracy of the candidate text sequence inferred by the client is also relatively low. Therefore, the candidate words in the candidate text sequence may contain errors; hence, each candidate word in the candidate text sequence is a word to be verified. Accordingly, the client will treat this candidate text sequence as a text sequence that needs to be verified by the server, so that the server can subsequently verify the candidate words in this candidate text sequence.

[0073] In this application, because the accuracy of the candidate text sequence obtained by the client through pre-inference based on the reference text sequence is low, the client needs to send the pre-inferred candidate text sequence to the server, whereby the server verifies the candidate text sequence. Then, the server returns the verified candidate text sequence to the client, and the words in the verified candidate text sequence are those verified by the server. Based on this, the client uses the prompt word and the verified words as the reference text sequence needed for inferring the candidate text sequence, and continues to perform pre-inference. This process is repeated iteratively until the inference task is completed.

[0074] Therefore, in this application, the reference text sequence upon which the client bases the reasoning candidate text each time is a text sequence that does not require server verification, i.e., a text sequence that has been confirmed as correct. Based on this, the reference text sequence may include prompt words for the reasoning task, and may also include tokens that have already been verified by the server. In particular, the verified tokens in the reference text sequence can be empty. For example, when step S102 is executed for the first time, the reference text sequence only includes prompt words; while if step S102 is not executed for the first time, the reference text sequence may include prompt words and at least one token that has been verified by the server.

[0075] Understandably, based on the reference text sequence, the client can only generate one candidate word each time it calls the first model. Therefore, the client needs to schedule the first model to perform task inference at least once based on the candidate text sequence in order to obtain a candidate text sequence that includes at least one candidate word.

[0076] In one possible implementation, to improve inference efficiency, the client-side can set the target sequence length for each inferred candidate text sequence. This avoids generating excessively long candidate text sequences, which may contain many erroneous candidate words, leading to the client inferring numerous meaningless candidate words and resulting in frequent calls to the first model and excessive resource waste. Therefore, the number of candidate words in the candidate text sequence inferred by the client should not exceed the target sequence length.

[0077] For example, assuming the target sequence length is 6, the client needs to call the first model 6 times to generate 6 candidate words sequentially, resulting in a candidate text sequence containing 6 candidate words. Of course, if the client confirms that the inference termination condition has been met during the inference process, the number of candidate words in the inferred candidate text sequence may be less than 6.

[0078] S103, at least send the candidate text sequence to the server.

[0079] For example, a client sends an inference request to the server for the inference task, and the inference request includes at least the candidate text sequence. Since the client may generate different candidate text sequences by iteratively executing the operations of this embodiment multiple times, the candidate text sequences included in different inference requests sent by the client to the server for this inference task will be different. To distinguish the inference task corresponding to the inference request, so that the server can identify the inference task corresponding to the candidate text sequence in the inference request, the inference request may also carry a unique identifier for the inference task.

[0080] Understandably, since the candidate text sequence generates a text sequence to be verified for the client, the client needs to send the candidate text sequence to the server so that the server can verify each candidate word in the candidate text sequence.

[0081] In this application, after the client generates a candidate text sequence each time, it may only send the candidate text sequence to the server. The server may cache the verified text sequence so that the currently obtained candidate text sequence can be verified in subsequent iterations by combining the cached verified text sequence.

[0082] Specifically, if the client has not yet sent any hints for the inference task to the server before sending the candidate text sequence, the client can still send the hints and the currently generated candidate text sequence to the server. The client can indicate the hints using identifiers or separators, etc., without restriction. Of course, if the client has already sent the hints to the server before sending the candidate text sequence, then the client only needs to send the currently inferred candidate text sequence to the server.

[0083] S104, obtain the corrected candidate text sequence returned by the server.

[0084] The corrected candidate text sequence is determined by the server using a second model based on the candidate text sequence. The process by which the server verifies the candidate text sequence to obtain the corrected candidate text sequence will be described later in the server-side embodiments and will not be repeated here.

[0085] Understandably, since the number of parameters in the second model is larger than that in the first model, the accuracy of each word in the corrected candidate text sequence is relatively higher after the candidate text sequence is verified by the server. Therefore, each word in the corrected candidate text sequence is a word that has been verified by the server and determined to be an acceptable correct word.

[0086] S105, if it is determined that the reasoning task has not yet been completed based on the corrected candidate text sequence, the reference text sequence and the corrected candidate text sequence are used as the updated reference text sequence, and the process returns to step S102 to call the first model to perform the task reasoning.

[0087] Based on the corrected candidate text sequence, the client can determine whether the reasoning termination condition of the reasoning task is met. If so, the reasoning task is completed; otherwise, the client still needs to continue to determine the candidate text sequence after the corrected candidate text sequence through pre-reasoning.

[0088] Understandably, the client needs to continue performing subsequent reasoning based on the prompt words and the words that have been inferred and confirmed as correct. Therefore, since the reference text sequence includes prompt words (which do not require verification, and each word in the prompt words is considered correct by default) and verified words, while the words in the corrected candidate text sequence are words verified by the server, each word in the corrected candidate text sequence can also be added to the reference text sequence to obtain an updated reference text sequence, and then the process returns to step S102.

[0089] In this application, the specific implementation of the client's determination of whether the reasoning task has been completed (i.e. whether the reasoning termination condition is met) based on the modified candidate text sequence can have multiple implementation possibilities, without any specific restrictions.

[0090] For example, in one possible implementation, if the modified candidate text sequence includes an end marker to indicate the end of reasoning, then the reasoning task is determined to be completed; conversely, if the modified candidate text sequence does not include the end marker, then the reasoning task is confirmed to be incomplete. The end marker can be a pre-specified special word, such as: <eos>.

[0091] The end marker can be a word or phrase contained in the candidate text sequence inferred by the client and verified as correct by the server. For example, during the client's inference of the candidate text sequence, the next word or phrase is determined. <eos>If the inference of the candidate text sequence ends, the candidate text sequence is sent to the server. The server then verifies the sequence from the first word to the last word. <eos>If all are correct, the corrected candidate text sequence will be returned. This corrected candidate text sequence will include... <eos>.

[0092] This end marker can also be added to the corrected candidate text sequence during the server-side verification process of the candidate text sequence generated by the client. For example, if the server determines that a certain candidate word in the candidate text sequence should be... <eos>Modify the candidate terminology to <eos>This yields the corrected candidate text sequence. Correspondingly, the client will also receive the corrected candidate text sequence including... <eos>The client can also determine the reasoning that completes the reasoning task.

[0093] In another possible implementation, the inference request received by the client may also carry the maximum length of the result, which indicates the upper limit of the length of the inference result. In this case, the client can determine the total number of words in the current reference text sequence and the corrected candidate text sequence. If the total number reaches the maximum number of results, then the inference task is completed.

[0094] In practical applications, the two implementation methods described above can be used in combination without any restrictions. Of course, other possible implementations that determine the termination condition of the inference task based on the text sequence inferred by the model are also applicable to this application without any limitations.

[0095] S106, if the reasoning to complete the reasoning task is determined based on the corrected candidate text sequence, the reasoning result corresponding to the reasoning task is determined based on the reference text sequence and the corrected candidate text sequence.

[0096] For example, the corrected candidate text sequence can be concatenated after the reference text sequence to obtain the inference result.

[0097] As can be seen from the above, after the client receives the inference task request, it performs pre-inference based on the prompt words and sends each inferred candidate text sequence to the server for verification. Since the number of parameters in the client's first model is smaller than that in the server's second model, the resources required for the client to infer the candidate text sequence using the first model are relatively less. After the client sends the inferred candidate text sequence to the server, the server only needs to verify the candidate text sequence without inferring word by word. This naturally reduces the number of times the server calls the second model, thus reducing the resources required for the server to run the second model to perform the inference task, thereby reducing the resources required to process the inference task.

[0098] Furthermore, since the number of parameters in the first model on the client side is relatively small, the time required for the client side to run the first model to infer each candidate word in the candidate text sequence is relatively small. As those skilled in the art know, the server side only needs to call the second model once each time it verifies the candidate text sequence, without having to call the second model repeatedly to complete the verification of the candidate text sequence. Therefore, in the case where the solution of this application adopts client pre-inference and the server only verifies the text sequence pre-inferred by the client, the time required to perform the inference task can be reduced, which naturally improves the efficiency of the inference task.

[0099] It is understandable that after the client infers a candidate text sequence each time, it may only send the candidate text sequence to the server, as detailed in the previous introduction.

[0100] Considering that the amount of data in a text sequence is relatively small, the network resources required for the client to transmit text data to the server are relatively small, and the time consumption is also relatively small. Based on this, in order to reduce the complexity of the interaction, in one possible implementation, this application can also combine the reference text sequence and the candidate text sequence into a target text sequence, and send the target text sequence and the verification indication information to the server.

[0101] The verification indication information is used to indicate at least one candidate word to be verified in the candidate text sequence. Based on the verification indication information, the server can determine each candidate word in the target text sequence, and naturally, it can also determine the reference text sequence in the target text sequence other than the candidate text sequence.

[0102] In this possible approach, the client can combine the current reference text sequence and the currently inferred candidate text sequence each time, and send the combined target text sequence to the server. Therefore, the server can directly verify the candidate text sequence based on the target text sequence sent by the client without caching the historical reference text sequence.

[0103] Furthermore, as mentioned above, in order to further reduce the amount of data transmission, this application may also combine the reference text sequence and the candidate text sequence into the target text sequence only when the reference text sequence is a prompt word.

[0104] Specifically, in response to the reference text sequence containing only prompt words, the reference text sequence and candidate text sequence are combined into a target text sequence, and the target text sequence and verification instruction information are sent to the server.

[0105] Accordingly, in response to the reference text sequence including prompt words and a revised historical candidate text sequence, the candidate text sequence currently inferred by the client is determined as the target text sequence. For example, the client sends the target text sequence and verification indication information to the server, where the verification indication information indicates that each word in the target text sequence is a candidate word to be verified. Alternatively, the client can send only the target text sequence to the server; if the client does not send verification indication information to the server, the server will default to determining the target text sequence as the candidate text sequence.

[0106] The data processing method of this application is described below from the server side.

[0107] like Figure 2 This illustration shows a flowchart of a data processing method provided in this application. The method of this embodiment is applied to a task inference server and may include:

[0108] S201, obtain the reference text sequence and the candidate text sequence to be verified.

[0109] The reference lexicon may include at least one lexicon.

[0110] In this application, the reference text sequence is the text sequence referenced by the inference candidate text sequence, and the reference text sequence is a text sequence that does not require server-side verification. The candidate text sequence is a text sequence inferred by the client based on the reference text sequence using a first model. The candidate text sequence includes at least one candidate word, and each candidate word in the candidate text sequence is a word to be verified.

[0111] Based on the task reasoning process, it is known that in the task reasoning process, it is necessary to combine the prompt words and call the model to infer the subsequent word units in sequence. Based on this, the reference text sequence can include: prompt words used to indicate the reasoning task and word units verified by the server.

[0112] Understandably, for this inference task, if this step is being executed by the server for the first time, the server will obtain a reference text sequence that only includes the prompt words, while the tokens verified by the server will be empty. Accordingly, the reference text sequence can be sent from the client to the server. For this inference task, if this step is not being executed by the server for the first time, the reference text sequence can be a historical text sequence cached on the server side.

[0113] S202, call the second model to process the reference text sequence and candidate text sequence to obtain the comprehensive probability distribution output by the second model.

[0114] The comprehensive probability distribution includes the probability distribution of the position of each candidate word in the candidate text sequence within the vocabulary. This probability distribution includes the probability that the position of a candidate word corresponds to a word in the vocabulary.

[0115] Understandably, when performing task inference based on prompts for inference tasks, it is necessary to call the second model to infer the probability distribution of the next word and determine the next word based on the probability distribution of the next word. This requires calling the second model multiple times to perform task inference, resulting in a large number of calls to the second model on the server side, which consumes a lot of resources. Moreover, the time required for each call to the second model to perform task inference is also relatively long, resulting in a relatively long total time required to process the inference task.

[0116] In this application, after the client pre-infers the candidate text sequence based on the prompt words and the verified lexical units (i.e., the reference text sequence), the server only needs to call the second model once to obtain the probability distribution corresponding to each candidate lexical unit in the candidate text sequence. This provides a basis for the server to verify whether each candidate lexical unit in the candidate text sequence is correct and to process each candidate lexical unit. Moreover, since there is no need to call the second model multiple times, it can also reduce resource consumption and task inference time.

[0117] Of course, this comprehensive probability distribution may also include: the probability distribution of the position of each word in the reference text sequence within the vocabulary.

[0118] For example, after inputting the reference text sequence and the candidate text sequence into the second model, the second model can output a matrix with the number of rows matching the total number of words in the reference and candidate text sequences. Based on the order of the words in the reference and candidate text sequences, each row of this matrix represents the probability distribution corresponding to each word in the reference and candidate text sequences.

[0119] As mentioned above, in this application, the number of parameters in the second model is greater than that in the first model.

[0120] S203, Based on the probability distribution of each candidate word in the candidate text sequence, determine the corrected candidate text sequence.

[0121] Understandably, the comprehensive probability distribution inferred by the server using the second model is more accurate. Based on this, the server can determine the actual word corresponding to the word position of each candidate word in the vocabulary based on the probability distribution of each candidate word in the comprehensive probability distribution, thereby determining the corrected candidate text sequence.

[0122] In this revised candidate text sequence, each word is a word that has been verified by the server, meaning that the server has confirmed it to be correct.

[0123] Understandably, if all candidate words in the candidate text sequence are correct words, then the corrected candidate text sequence can be the same as the original candidate text sequence. However, if there are incorrect candidate words in the candidate text sequence, then the corrected candidate text sequence and the original candidate text sequence may contain different words and have different numbers of words.

[0124] S204, the corrected candidate text sequence is returned to the client as a verified reference text sequence.

[0125] It is understandable that, since each word in the corrected candidate text sequence is a correct word that has been verified by the server, the corrected candidate text sequence is actually the newly added reference text sequence.

[0126] As described above regarding the client side, after the server returns the corrected candidate text sequence to the client, if the client determines that the reasoning task has not yet been completed based on the corrected candidate text sequence, the client will use the reference text sequence and the corrected candidate text sequence as the updated reference text sequence and continue to perform pre-inference. This process will be repeated until the reasoning task is completed.

[0127] Based on this, it can be seen that for this inference task, each time the client infers the candidate text sequence corresponding to the inference task, the client will send the candidate text sequence to the server at least once. The server can then execute the relevant operations of steps S201 to S204 above. Therefore, steps S201 to S204 above in this embodiment are an iterative processing process of the server for the candidate text sequence inferred by the client during the processing of the inference task. If different candidate text sequences are inferred multiple times during the processing of the inference task, then steps S201 to S204 above can be repeatedly executed multiple times.

[0128] For example, for this inference task, each time the client infers a candidate text sequence, the client sends an inference request to the server for that inference task. This inference request includes at least the candidate text sequence currently inferred by the client. Accordingly, in response to the inference request for the inference task, the server obtains a reference text sequence and a candidate text sequence to be verified, and performs the relevant operations in steps S202 to S204 above.

[0129] Specifically, after confirming the completion of the inference task, the client can send an inference completion indication to the server, indicating the end of the inference process. Upon receiving this indication, the server determines the inference task is complete and no longer needs to wait for the client to send an inference request. Furthermore, in response to the inference completion indication, the server can clear relevant data cached during the second model's processing of candidate text sequences.

[0130] As can be seen from the above, in this application, during the inference task processing, the client can use the first model to pre-infer candidate text sequences for the inference task, while the server only needs to call the second model once to obtain the probability distribution corresponding to each candidate word in the candidate text sequence. Based on the probability distribution corresponding to each candidate word in the candidate text sequence, the corrected candidate text sequence is determined. Therefore, the server does not need to call the second model multiple times to generate each word in the candidate text sequence, reducing the number of times the server calls the second model. Since the parameter magnitude of the first model on the client side is smaller than that of the second model on the server side, that is, the model size of the first model on the client side is relatively small, while the model size of the second model on the server side is relatively large, the resources required for pre-inferring candidate text sequences using the first model are relatively less, while the resources required for running the second model are more. On this basis, reducing the number of times the second model is called during the inference task processing can reduce the resources and time required for the server to run the second model, naturally reducing the resources required to execute the inference task.

[0131] Moreover, since the second model is relatively large, the time required for the server to call and run the second model to perform inference tasks is relatively long each time. Therefore, in the process of processing inference tasks, by reducing the number of times the second model is called and run to perform inference, the time spent running the second model can be reduced, thereby reducing the time required to process inference tasks and naturally improving the efficiency of performing inference tasks.

[0132] In this application, the specific implementation of the server determining the modified candidate text sequence can be varied and is not limited thereto.

[0133] In one possible scenario, the server can, based on the probability distribution corresponding to each candidate word in the candidate text sequence, sequentially modify each candidate word in the candidate text sequence according to the order of their appearance, to obtain a modified candidate text sequence.

[0134] The candidate text sequence corrects each candidate word. This can be done by sequentially identifying and replacing the correct word corresponding to each candidate word in the candidate text sequence; or by identifying the first erroneous candidate word and replacing it with the correct word, and then determining the replaced candidate word and all candidate words preceding it as the corrected candidate text sequence. There are no specific restrictions on the method.

[0135] To address this possibility, we will illustrate one implementation method below. For example... Figure 3 This illustrates another implementation flow diagram of the data processing method on the server side. The method in this embodiment may include:

[0136] S301, obtain the reference text sequence and the candidate text sequence to be verified.

[0137] The candidate text sequence is a text sequence inferred by the client based on the reference text sequence using the first model. The candidate text sequence includes at least one candidate word.

[0138] The reference text sequence includes: cue words indicating the reasoning task and server-verified lexical units. The server-verified lexical units in the reference text sequence can be empty.

[0139] S302, call the second model to process the reference text sequence and the candidate text sequence, and obtain the comprehensive probability distribution output by the second model.

[0140] The comprehensive probability distribution includes the probability distribution of the position of each candidate word in the candidate text sequence within the vocabulary.

[0141] The second model has a larger number of parameters than the first model.

[0142] The above steps can be found in the relevant descriptions of the previous embodiments, and will not be repeated here.

[0143] S303, determine the target candidate word to be verified in the candidate text sequence according to the order of each candidate word in the candidate text sequence.

[0144] In this embodiment, the server determines one candidate word as the target candidate word to be verified each time according to the order of the candidate words in the candidate text sequence, and then executes subsequent steps S304 to S305. This process is repeated until there are no candidate words to be verified in the candidate text sequence, or an erroneous candidate word is detected in the candidate text sequence.

[0145] S304. Based on the probability distribution of the position of the target candidate word in the vocabulary, determine whether the target candidate word is an acceptable word. If yes, proceed to step S305; otherwise, proceed to step S306.

[0146] In this context, if the target candidate word is an acceptable word, it means that the target candidate word is the word that is correctly inferred at its position.

[0147] There are several possible ways to determine whether a target candidate word is an acceptable candidate word. The specific method depends on the server-side strategy for selecting words based on the probability distribution corresponding to a certain word position, and there are no restrictions on this.

[0148] For example, based on the probability distribution corresponding to the target candidate word, if the probability of the target candidate word in the word list is the highest, then the target candidate word is determined to be the correct word in its position, that is, the target candidate word is an acceptable word.

[0149] For example, based on the probability distribution corresponding to the target candidate word, a set number of candidate words with the highest probabilities can be determined from the vocabulary. Based on this, if the target candidate word belongs to the set number of candidate words, it is determined to be an acceptable word; or, according to a set selection strategy (such as random selection or other selection strategies), a target candidate word is selected from the set number of candidate words. If the target candidate word is the target candidate word, it is determined to be an acceptable word.

[0150] S305, determine whether there are still unverified candidate words after the target candidate word in the candidate text sequence. If yes, return to step S303 to redetermine the next target candidate word to be verified. If no, determine the candidate text sequence as the corrected candidate text sequence and proceed to step S307.

[0151] If there are no candidate words that have not yet been verified in the candidate text sequence (i.e., there are no candidate words to be verified), it means that each candidate word in the candidate text sequence has been verified as a correct word. Therefore, the candidate text sequence can be identified as the corrected candidate text sequence.

[0152] Understandably, the server, based on the comprehensive probability distribution inferred from the second model, can also determine the next word after the last candidate word in the candidate text sequence.

[0153] For example, assuming the reference text sequence and the candidate text sequence together contain 10 tokens, and assuming the vocabulary contains 1000 tokens, then the combined probability distribution predicted by the server based on the second model can be 10. The matrix represents a set of 1000 words, where each row shows the probability distribution of a single word. The first row actually represents the probability distribution of the word following the first word (i.e., the second word). Based on this probability distribution, the specific word at the second word position can be predicted. Similarly, the second row represents the probability distribution of the third word.

[0154] Similarly, the probability distribution in the last row of this matrix represents the probability distribution of each word in the vocabulary, corresponding to the position of the next word after the 10th word (i.e., the 11th word). Therefore, based on the probability distribution in the last row of this matrix, the next word after the candidate text sequence can be determined from the vocabulary.

[0155] Therefore, based on the comprehensive probability distribution predicted by the second model, the server can determine the next word after the last candidate word in the candidate text sequence, and combine the candidate text sequence and the next word after the candidate text sequence into a corrected candidate text sequence.

[0156] S306. Based on the probability distribution of the lexical position of the target candidate lexical in the vocabulary, determine the correct lexical corresponding to the lexical position of the target candidate lexical, replace the target candidate lexical in the candidate text sequence with the correct lexical, and delete the candidate lexical after the target candidate lexical in the candidate text sequence to obtain the modified candidate text sequence.

[0157] It is understandable that if the target candidate word to be verified in the candidate text sequence is an incorrect word, since all candidate words after the target candidate word are predicted based on the target candidate word, if the target candidate word is an incorrect word, then the candidate words after the target candidate word should also be considered incorrect words.

[0158] Therefore, if the target candidate word is identified as an unacceptable word, then after determining the correct word at the word position of the candidate word from the word list based on the probability distribution corresponding to the target candidate word, there is no need to verify subsequent candidate words.

[0159] For example, suppose the candidate text sequence is "This winter isn't too cold," and the sequence includes three candidate words: "this year," "winter," and "not too cold." If the server determines "this year" as an acceptable word based on its probability distribution, then "winter" will be selected as the target candidate word to be verified. However, if the server determines "winter" as an unacceptable word based on its probability distribution, then it will determine the correct word at the corresponding position in the vocabulary based on the probability distribution of "winter."

[0160] Assuming the correct word for "winter" should be "summer," the server can replace "winter" with "summer" in the candidate text sequence. The server will no longer validate "not too cold." After deleting the word "not too cold" which originally followed "winter" from the candidate text sequence, the server obtains the corrected candidate text sequence "this summer," and returns it to the client. The client can then use "this summer," along with previously validated words and prompts, to infer the candidate text sequence following "this summer."

[0161] S307, the corrected candidate text sequence is returned to the client as a verified reference text sequence.

[0162] In this embodiment and the previous server-side embodiments, the server can either return only the corrected candidate text sequence to the client, or it can concatenate the reference text sequence and the corrected candidate text sequence to obtain an updated reference text sequence, and return the updated reference text sequence to the client as a verified reference text sequence. In this case, the client can directly call the first model to perform task inference based on the verified reference text sequence.

[0163] Furthermore, in order to enable the client to accurately determine the tokens in the reference text sequence returned by the server that belong to the server's current verification, the server can also return the corrected candidate text sequence (or the updated reference text sequence) and the latest number of verified tokens to the client. The latest number of verified tokens (e.g., can be represented by the field num_tokens_generated) is used to indicate the number of tokens verified by the server in this instance.

[0164] This step can be found in the previous introduction, and will not be repeated here.

[0165] In any of the above embodiments of this application, since there are multiple possible implementations for the client to send the candidate text sequence to the server, there are also multiple possible implementations for the server to obtain the reference text sequence and the candidate text sequence to be verified.

[0166] For example, in one possible implementation, to reduce the complexity of obtaining candidate and reference text sequences on the server side, enabling the server to efficiently determine the candidate and reference text sequences and improve the flexibility of text sequence transmission between the server and the client, the client can also send a target text sequence and verification indication information. Correspondingly, the server can obtain the target text sequence and verification indication information sent by the client. As mentioned earlier, the verification indication information is used to indicate at least one candidate word to be verified in the target text sequence. Based on this, the server can determine the reference text sequence and the candidate text sequence to be verified based on the target text sequence and the verification indication information.

[0167] In this application, the target text sequence may include only the candidate text sequence, or it may include both the reference text sequence and the candidate text sequence. Based on this, it can be determined whether the target text sequence contains the reference text sequence based on the verification indication information, and different methods are used to obtain the reference text sequence for the two cases: whether the target text sequence includes the reference text sequence or not.

[0168] For example, in one possible scenario, to reduce data transmission, the client may transmit only the candidate text sequence. Correspondingly, in response to the verification indication that each word in the target text sequence is a candidate word, the server can identify the target text sequence as a candidate text sequence to be verified, thus obtaining the reference text sequence identified by the server and corresponding to the inference task. In this possible scenario, if the target text sequence only includes candidate text sequences, it means the server has already obtained the prompt words and cached at least one word that has been verified before the current time, i.e., the server has cached the reference text sequence. Therefore, the cached reference text sequence corresponding to the inference task can be directly obtained from the server.

[0169] Furthermore, in this possible scenario, in order to accurately obtain the reference text sequence referenced by the client for inferring the candidate text sequence from the server side after obtaining the candidate text sequence sent by the client next time, the server can also cache the reference text sequence and the corrected candidate text sequence as the reference text sequence corresponding to the inference task in the server after determining the corrected candidate text sequence.

[0170] Specifically, if the server receives an inference completion indication from the client for the inference task, it indicates that the inference for that task has been completed, and there is no need to cache the cached data related to that inference task. Based on this, in response to receiving the inference completion indication from the client for the inference task, the server can also delete the reference text sequence corresponding to that inference task.

[0171] In another possible scenario, to reduce server-side data caching, the client can send both a reference text sequence and a candidate text sequence each time. Correspondingly, in response to the verification indication that some terms in the target text sequence are candidate terms, the server can determine the reference text sequence and the candidate text sequence to be verified from the target text sequence based on the verification indication. For example, based on the verification indication, a candidate text sequence consisting of at least one candidate term in the target text sequence is determined, and the text sequence in the target text sequence other than this candidate text sequence is determined as the reference text sequence.

[0172] In the above embodiments of this application, the specific form of the verification indication information can be varied. To more intuitively and flexibly indicate the candidate words to be verified, the verification indication information may include at least one of: the number of verified words and the number of words to be verified. The number of verified words represents the number of words in the target text sequence that do not require verification, that is, the number of words belonging to the reference text sequence. The number of words to be verified is the number of words that need to be verified by the server, that is, the number of candidate words belonging to the candidate text sequence.

[0173] In practical applications, the verification indication information can include both the number of verified tokens and the number of tokens to be verified, or it can include only the number of verified tokens or the number of tokens to be verified.

[0174] Based on this, the verification indication information indicating that each word in the target text sequence is a candidate word can include at least one of the following:

[0175] The number of verified lexical units is zero;

[0176] The number of tokens to be verified is the total number of tokens in the target text sequence.

[0177] If the verification indication information indicates that the number of verified tokens is zero, it means that the target text sequence does not contain the reference text sequence. Therefore, the target text sequence only includes candidate text sequences, and the target text sequence can be identified as a candidate text sequence.

[0178] Correspondingly, if the number of tokens to be verified is the same as the total number of tokens in the target text sequence, it also indicates that the target text sequence only includes candidate text sequences.

[0179] Similarly, the verification indication information indicates that some lexical units in the target text sequence are candidate lexical units, including at least one of the following:

[0180] The number of verified lexical units is not zero;

[0181] The number of lexical units to be verified is less than the total number of lexical units in the target text sequence.

[0182] Understandably, if the number of verified lexical units is not zero, or if the number of lexical units to be verified is less than the total number of lexical units in the target text sequence, it indicates that the target text sequence includes both candidate text sequences and reference text sequences. In this case, considering that when the client performs task inference based on the model, the candidate lexical units to be verified are inferred based on the verified lexical units, and each lexical unit is inferred sequentially.

[0183] Therefore, in the target text sequence, verified terms must precede candidate terms to be verified. Thus, the server can determine the first number of terms in the target text sequence, ranked from front to back, as reference text terms based on the number of verified terms. For example, if the number of verified terms is 5, then the first 5 terms in the target text sequence can be identified as terms in the reference text sequence. Correspondingly, the terms following the reference text sequence in the target text sequence are considered candidate terms in the candidate text sequence.

[0184] Similarly, the server can also determine the second-to-last number of terms in the target text sequence as candidate terms in the candidate text sequence based on the number of terms to be verified. For example, if the number of terms to be verified is 7, then the last 7 terms in the target text sequence are determined as candidate terms in the candidate text sequence. Correspondingly, terms outside the candidate text sequence in the target text sequence can be determined as terms in the reference text sequence.

[0185] To facilitate understanding of the scheme in this application, the following explanation uses one scenario as an example. The scenario is that the client sends the prompt word and the candidate text sequence to the server only the first time it infers the candidate text sequence; and in subsequent inferences, the client only transmits the candidate text sequence to the server.

[0186] like Figure 4 This illustration shows a flowchart of the data processing method provided in this application. The method in this embodiment may include:

[0187] S401, in response to the detection of a task request, the client calls the first model to perform the target number of inferences based on the prompt words in the task request, and obtains the inferred first candidate text sequence.

[0188] As explained earlier, the client can only infer one word per call to the first model, while the server can verify multiple words with a single call to the second model. To reduce the number of times the server calls the second model, the candidate text sequence generated by the client can include multiple words. Furthermore, considering the relatively low accuracy of words in the client-inferred candidate text sequence, the number of words inferred by the client each time should not be too high. Therefore, the application can pre-set the number of iterative inference calls to the first model by the client each time, i.e., the target number. This target number is also the maximum limit length of words in the candidate text sequence generated by the client, i.e., the target sequence length mentioned earlier.

[0189] In this process, each time the client calls the first model, it performs a forward propagation. The client can determine a candidate word based on the candidate probability distribution output by the first model. For example, based on a prompt word, the first model is called to output a candidate probability distribution. Based on the candidate probability distribution, the next candidate word after the prompt word can be determined. After the first model is called a second time to output a candidate probability distribution, the second candidate word can be determined based on the second output candidate probability distribution, and so on.

[0190] For example, if the target number is 6, the client can call the first model 6 times consecutively based on the prompt word. Through the 6 candidate probability distributions output by the first model in 6 forward propagations, 6 candidate words can be generated in sequence, so the candidate text sequence includes 6 candidate words.

[0191] To facilitate differentiation, the candidate text sequence inferred by the client based on the prompt words for the first time is called the second candidate text sequence, and the candidate text sequence inferred for the second time is also called the second candidate text sequence.

[0192] S402, the client combines the prompt word and the first candidate text sequence into a target text sequence, and sends the target text sequence and verification instruction information to the server.

[0193] For example, a client can send an inference request to the server, which includes a target text sequence consisting of a prompt word and a first candidate text sequence, and verification instructions. Furthermore, to enable the server to distinguish between different inference tasks from different clients, the inference request may also include a unique identifier for the inference task indicated by the prompt word.

[0194] S403, the server responds to the verification indication information indicating that some words in the target text sequence are candidate words, and determines the prompt word and the first candidate text sequence from the target text sequence based on the verification indication information.

[0195] In one possible scenario, the verification indication information may include at least one of the following: the number of verified tokens and the number of tokens to be verified.

[0196] For example, the `num_tokens_computed` field can represent the number of verified tokens, while the `num_tokens_to_be_verified` field represents the number of tokens to be verified. At least one of these fields can be included in the inference request. Assuming the prompt word contains 5 tokens, and the length of the first candidate text sequence (i.e., the number of candidate tokens it contains) is 6, then the value of `num_tokens_computed` (number of verified tokens) is 5, and the value of `num_tokens_to_be_verified` (number of tokens to be verified) is 6.

[0197] Based on this, the server determines the first 5 tokens in the target text sequence as prompt words based on at least one of num_tokens_computed (number of verified tokens) and num_tokens_to_be_verified (number of tokens to be verified); and determines the last 6 tokens in the target text sequence as candidate tokens to be verified, thus obtaining the first candidate text sequence.

[0198] S404, the server calls the second model to process the prompt word and the first candidate text sequence, and obtains the first comprehensive probability distribution output by the second model.

[0199] The second model has a larger number of parameters than the first model.

[0200] The first comprehensive probability distribution includes the probability distribution of the position of each candidate word in the first candidate text sequence within the vocabulary. Of course, the first comprehensive probability distribution may also include the probability distribution of the position of at least one word in the prompt word within the vocabulary.

[0201] For ease of distinction, the comprehensive probability distribution generated in step S404 is referred to as the first comprehensive probability distribution, while the comprehensive probability distribution generated by the server in subsequent steps is referred to as the second comprehensive probability distribution. The specific implementation of the server generating the comprehensive probability distribution can be found in the previous related introduction, and will not be repeated here.

[0202] S405, the server determines the corrected first candidate text sequence based on the probability distribution of each candidate word in the first candidate text sequence, and caches the prompt word and the corrected first candidate text sequence as the reference text sequence corresponding to the inference task.

[0203] For example, the corrected first candidate text sequence is appended to the prompt word, and the resulting text sequence is cached as a reference text sequence corresponding to the unique identifier of the reasoning task.

[0204] S406, the server returns the corrected first candidate text sequence to the client.

[0205] This embodiment illustrates the example where the server only returns the first candidate text sequence to the client. However, if the server returns the prompt word and the reference text sequence concatenated from the first candidate text sequence to the client, the same principle applies.

[0206] It should be noted that the order in which the server caches the reference text sequence and sends the first candidate text sequence to the client can be interchanged or performed synchronously, without any restriction.

[0207] S407, if the client determines, based on the modified first candidate text sequence, that the reasoning task has not yet been completed, the prompt word and the modified first candidate text sequence are combined into a reference text sequence.

[0208] In this embodiment, for ease of understanding, the different target text sequences transmitted by the client to the client in different instances are illustrated using the example of the client determining inference tasks that have not yet been completed based on the corrected first candidate text sequence. Of course, if the client determines inference tasks that have been completed based on the corrected first candidate text sequence, then the inference result can be determined based on the corrected first candidate text sequence, and the specific details will not be elaborated further.

[0209] S408, the client calls the first model to perform the target number of inferences based on the reference text sequence, and obtains the inferred second candidate text sequence.

[0210] The number of candidate words in the second candidate text sequence is no greater than the target number. For example, if the second candidate text sequence does not include an end marker, the number of candidate words in the second candidate text sequence is the target number.

[0211] S409, the client takes the second candidate text sequence as the target text sequence and sends the target text sequence and verification instruction information to the server.

[0212] In this scenario, once the client has already sent the prompt words to the server, after the client infers the candidate text sequence, the client only needs to send the candidate text sequence and verification instruction information to the server.

[0213] In step S409, the verification indication information is used to indicate that each word in the target text sequence is a candidate word to be verified.

[0214] S410, in response to the verification indication information indicating that each word in the target text sequence is a candidate word, the server determines the target text sequence as the second candidate text sequence to be verified and obtains the cached reference text sequence corresponding to the inference task.

[0215] For example, the verification instruction information includes the number of verified tokens and the number of tokens to be verified. Suppose the length of the second candidate text sequence is 6, the number of verified tokens is 0, and the number of tokens to be verified is 6. Then, after the server obtains the target text sequence, if it confirms that the length of the target text sequence is 6, it means that the target text sequence is a candidate text sequence.

[0216] The cached reference text sequence consists of the aforementioned prompt words and the corrected first candidate text sequence.

[0217] S411, the server calls the second model to process the reference text sequence and the second candidate text sequence, and obtains the second comprehensive probability distribution output by the second model.

[0218] The second comprehensive probability distribution includes at least the probability distribution of the position of each candidate word in the second candidate text sequence within the vocabulary.

[0219] S412, the server determines the corrected second candidate text sequence based on the probability distribution of each candidate word in the second candidate text sequence, concatenates the corrected second candidate text sequence with the reference text sequence, and caches the concatenated text sequence as the reference text sequence corresponding to the inference task.

[0220] The modified second candidate text sequence needs to be appended to the end of the reference text sequence so that the reference text sequence and the modified second candidate text sequence are combined into a new reference text sequence.

[0221] S413, the server returns the corrected second candidate text sequence to the client.

[0222] S414, if the client determines the reasoning to complete the reasoning task based on the corrected second candidate text sequence, it determines the reasoning result corresponding to the reasoning task based on the reference text sequence and the corrected second candidate text sequence.

[0223] It is understood that this embodiment illustrates the inference task by having the client complete two candidate text sequences through two pre-inference steps. If, based on the corrected second candidate text sequence, the client determines that the inference task has not yet been completed, the client can combine the reference text sequence and the corrected second candidate text sequence into an updated reference text sequence, return to step S408, and perform subsequent operations until the inference task is determined to be completed.

[0224] Specifically, after step S414, since the client has confirmed the completion of the inference task, it can also send an inference completion indication to the server. This inference completion indication indicates the end of the inference task; for example, it carries a unique identifier for the inference task. Correspondingly, in response to this inference completion indication, the server can delete the cached reference text sequence corresponding to the inference task.

[0225] To intuitively understand the benefits of this application, combined with Figure 5 To explain, Figure 5 A comparative example diagram is shown, illustrating existing solutions for handling reasoning tasks and the solution presented in this application.

[0226] exist Figure 5 The interaction diagram to the left of the middle arrow is an example of the interaction between the client and the server (i.e., the server side) when processing inference tasks. As can be seen, in the current solution, after receiving the hints for the inference task, the client sends the hints to the server. Based on the hints, the server may need to call the second model deployed on the server multiple times to perform task inference and finally determine the inference result corresponding to the inference task. Then, the server directly returns the inference result to the client to complete the inference task.

[0227] exist Figure 5 The interaction diagram to the right of the arrow is an example of the interaction between the client and the server in this application. As shown in the diagram, after receiving the prompt words for the inference task, the client uses the first model to pre-infer predicted words based on the prompt words. These predicted words, or candidate text sequences, can include at least one candidate word element predicted by the client and requiring verification. In the first iteration, the client sends the prompt words and predicted words to the server. The server, based on the prompt words, calls the second model once to verify the predicted words using the probability distribution output by the second model. This involves correcting each word element in the predicted words and returning the verification result (i.e., the corrected candidate text sequence) to the client.

[0228] Based on this, the client will continue to use the first model to infer and predict words based on the prompt words and the verified lexical units in the verification results returned by the server. Then, the client only needs to send the inferred predicted words to the server. The server only needs to verify the current predicted words and return the verification results to the client. This process is repeated until the inference task is completed.

[0229] contrast Figure 5 As can be seen from the interaction diagrams to the left and right of the middle arrow, in the current solutions, the reasoning process of the reasoning task is executed on the server side. In order to deduce each word in the reasoning result in sequence, the server must continuously call the second model for reasoning, resulting in a relatively large number of calls and runs of the second model.

[0230] In the solution of this application, the client performs pre-inference based on the prompt words, and the server only needs to verify the predicted words inferred by the client. Each time a predicted word is verified, the second model only needs to be called once, which greatly reduces the number of times the server calls and runs the second model.

[0231] To provide a more intuitive understanding of the number of times the client calls the first model to perform inference and the number of times the server calls the second model during the task inference process, a specific inference task example is used below.

[0232] Figure 6 This diagram illustrates an example of an interaction between the client and server in performing a reasoning task in this application.

[0233] exist Figure 6 In the process, the client receives the prompt: Please name a provincial capital city and its famous attractions.

[0234] Assume that the target sequence length N (i.e. the number of targets mentioned earlier) of the candidate text sequence inferred by the first model each time the client calls it is 10.

[0235] exist Figure 6 To facilitate differentiation, the terms in the reference text sequence that do not require verification are represented in non-italic font, while the terms in the candidate text sequence to be verified are represented in italics. Furthermore, the verified candidate words or generated correct terms determined after the server has corrected the candidate terms are represented in bold italics.

[0236] Depend on Figure 6 It can be seen that, based on the prompt word, the client can determine the candidate text sequence "Nanjing: West Lake; Xiamen: Drum" by calling the first model to perform 10 forward propagations.

[0237] Based on this, the client will send the prompt word and the candidate text sequence "Nanjing: West Lake" to the server.

[0238] The server only needs to call the second model once to perform forward propagation to obtain a comprehensive probability distribution. This comprehensive probability distribution includes at least the probability distribution of each candidate word in "Nanjing: West Lake" corresponding to the word position in the vocabulary. Based on this, the server verifies each candidate word sequentially according to the order of the words in the candidate text sequence, based on the probability distribution corresponding to each candidate word in the candidate text sequence. For example, if it verifies that "West" is incorrect and the correct word corresponding to the position of "West" should be "Center", the server will replace "West" with "Center" and delete the word "Lake; Xiamen: Drum" that was originally located after "West", and return the corrected candidate text sequence "Nanjing: Center" to the client.

[0239] Based on the prompt and "Nanjing: Zhong", the client will repeatedly call the first model to perform reasoning, resulting in: Please name a provincial capital city and its famous attractions: Nanjing: Sun Yat-sen Mausoleum; Xi'an: Terracotta Warriors <eos>The candidate text sequence is set as "mountains; Xi'an: Terracotta Warriors". <eos>Send to the server.

[0240] The server combined the previously determined reference text sequence: "Please name a provincial capital city and its famous attractions: Nanjing: Zhong" with the candidate text sequence "Mountains; Xi'an: Terracotta Warriors" <eos>The second model is invoked once to output the comprehensive probability distribution. If, based on this comprehensive probability distribution, it is determined that all candidate words in the candidate text sequence are correct, then the candidate text sequence can be determined as the corrected candidate text sequence: "Mountains; Xi'an: Terracotta Warriors". <eos>".

[0241] The client received the corrected candidate text sequence: "mountains; Xi'an: Terracotta Warriors" <eos>"Afterwards, it was determined that the corrected candidate text sequence contained an end marker:" <eos>If the reasoning in the task is complete, the corrected candidate text sequence is concatenated with "Please name a provincial capital and its famous attractions: Nanjing: Zhong", resulting in "Please name a provincial capital and its famous attractions: Nanjing: Sun Yat-sen Mausoleum; Xi'an: Terracotta Warriors". <eos>And determined that the reasoning result of "Please name a provincial capital city and its famous attractions" was "Nanjing: Sun Yat-sen Mausoleum; Xi'an: Terracotta Warriors".

[0242] As can be seen from the above interaction process, since the first model on the client side is relatively small in size, the resources and time required for multiple calls to the first model to perform pre-inference of the candidate text sequence are relatively small. After the candidate text sequence inferred by the client is sent to the server, the server only needs to call the second model once to complete the verification process of the candidate text sequence and obtain the corrected candidate text sequence. Furthermore, since the number of times the client infers candidate text sequences is relatively small, the number of times the server calls and runs the second model is also greatly reduced, thereby reducing the resources and time consumed by running the second model and improving the inference efficiency of the task.

[0243] Understandably, in Figure 4 In this embodiment, the example illustrates that the client only returns the prompt (i.e., the reference text sequence) and the candidate text sequence to the server the first time it infers a candidate text sequence. In practical applications, to reduce the amount of data that the server needs to cache, the client can also send the reference text sequence and the candidate text sequence to the server each time. Accordingly, the server can directly extract the reference text sequence and the candidate text sequence from the target text sequence.

[0244] In this situation, Figure 4 Step S409 can be replaced by: the client sending the reference text sequence and the second candidate text sequence as the target text sequence to the server. Correspondingly, step S410 can be replaced by the server responding to the verification indication that some words in the target text sequence are candidate words, and determining the reference text sequence and the second candidate text sequence to be verified from the target text sequence. For details on how to identify the reference text sequence and the second candidate text sequence from the target text sequence, please refer to the previous related introduction, which will not be repeated here. In this case, Figure 4 The client and client operations in other steps besides S409 and S410 also apply to this situation, and will not be elaborated further.

[0245] It is understood that in the above embodiments, during the process of the server calling the second model to process the reference text sequence and the candidate text sequence, the server needs to use the second model to generate a key matrix (K matrix) and a value matrix (V matrix). In order to reduce the amount of computation on the server side, the server side can also cache the key matrix and value matrix corresponding to the historically verified reference text sequence.

[0246] Based on this, the server can call the second model to process the reference text sequence and the candidate text sequence as follows: in response to the cached first key matrix and first value matrix corresponding to the historically generated reference text sequence, the server calls the second model to process the reference text sequence and the candidate text sequence based on the first key matrix and first value matrix corresponding to the reference text sequence, and generates the comprehensive probability distribution output by the second model.

[0247] Furthermore, after the server determines the corrected candidate text sequence, it can also determine the second key matrix and the second value matrix generated by the second model corresponding to the reference text sequence and the corrected candidate text sequence, and cache the second key matrix and the second value matrix. Based on this, when the server subsequently obtains the candidate text sequence again, it can use the second key matrix and the second value matrix as the first key matrix and the first value matrix, respectively, and perform operations based on the first key matrix and the first value matrix to call the second model to process the reference text sequence and the candidate text sequence, thereby reducing the data computation required for the server to run the second model.

[0248] In particular, the server can also delete the cached second key matrix and second value matrix in response to receiving an inference end instruction sent by the client for the inference task, so as to reduce the consumption of cache resources.

[0249] Furthermore, this application also provides a server. For example... Figure 7 The diagram shows a schematic of the composition structure of a server provided in this application, which includes at least: a processor 701, a memory 702, and a sending module 703.

[0250] The memory 702 is used to store the computer programs required for the processor to perform operations.

[0251] Processor 701 is configured to execute a computer program stored in the memory to perform the following operations:

[0252] Obtain a reference text sequence and a candidate text sequence to be verified. The candidate text sequence is a text sequence inferred by the client based on the reference text sequence using a first model. The candidate text sequence includes at least one candidate word. The reference text sequence includes prompt words for indicating the inference task and words verified by the server.

[0253] The second model is invoked to process the reference text sequence and the candidate text sequence to obtain the comprehensive probability distribution output by the second model. The comprehensive probability distribution includes the probability distribution of the position of each candidate word in the candidate text sequence in the vocabulary. The parameter magnitude of the second model is greater than that of the first model.

[0254] Based on the probability distribution of each candidate word in the candidate text sequence, the corrected candidate text sequence is determined.

[0255] The sending module 703 is used to return the corrected candidate text sequence to the client as a verified reference text sequence.

[0256] The specific operations performed by the processor 701 and the sending module 703 can be found in the relevant operations performed on the server side in the preceding data processing method, and will not be repeated here.

[0257] Of course, the server may also include a receiving module 704 for receiving candidate text sequences sent by the client, etc. For details, please refer to the previous introduction, which will not be repeated here.

[0258] Furthermore, this application also provides a client device, such as... Figure 8 This paper illustrates a schematic diagram of the composition structure of a client device provided in this application. The client device includes at least: a processor 801, a transmitting module 802, and a receiving module 803.

[0259] The processor 801 is configured to, in response to detecting a task request, determine a prompt word in the task request as a reference text sequence corresponding to a reasoning task, the prompt word being used to indicate the reasoning task; and, based on the reference text sequence, invoke a first model to perform task reasoning to obtain a reasoned candidate text sequence, the candidate text sequence including at least one candidate word.

[0260] The sending module 802 is used to send at least the candidate text sequence to the server;

[0261] The receiving module 803 is used to obtain the corrected candidate text sequence returned by the server. The corrected candidate text sequence is determined by the server based on the candidate text sequence using a second model. The parameter magnitude of the second model is greater than that of the first model.

[0262] The processor 801 is further configured to: if, based on the corrected candidate text sequence, determine that the reasoning task has not yet been completed, use the reference text sequence and the corrected candidate text sequence as the updated reference text sequence, and return to the operation of calling the first model to perform task reasoning; if, based on the corrected candidate text sequence, determine that the reasoning task has been completed, determine the reasoning result corresponding to the reasoning task based on the reference text sequence and the corrected candidate text sequence.

[0263] The specific operations performed by the processor, the sending module, and the receiving module can be found in the relevant introduction on the client side of the previous data processing method, and will not be repeated here.

[0264] Understandably, the client device may also include a memory 804 for storing computer programs required by the processor 801 to perform operations.

[0265] Furthermore, the client device may also include a display unit 805 for outputting inference results.

[0266] Of course, the client device may also include input units such as a keyboard, mouse, or touchscreen, as well as more or fewer other components, without any specific limitations.

[0267] This application also provides a computer program product, including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the data processing methods provided in this application.

[0268] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the data processing methods provided in this application.

[0269] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0270] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0271] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0272] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).< / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos> < / eos>

Claims

1. A data processing method applied to a server for task inference, comprising: Obtain a reference text sequence and a candidate text sequence to be verified. The candidate text sequence is a text sequence inferred by the client based on the reference text sequence using a first model. The candidate text sequence includes at least one candidate word. The reference text sequence includes prompt words for indicating the inference task and words verified by the server. The second model is invoked to process the reference text sequence and the candidate text sequence to obtain the comprehensive probability distribution output by the second model. The comprehensive probability distribution includes the probability distribution of the position of each candidate word in the candidate text sequence in the vocabulary. The parameter magnitude of the second model is greater than that of the first model. Based on the probability distribution corresponding to each candidate word in the candidate text sequence, the corrected candidate text sequence is determined; The corrected candidate text sequence is returned to the client as a verified reference text sequence.

2. The data processing method according to claim 1, wherein determining the corrected candidate text sequence based on the probability distribution corresponding to each candidate word in the candidate text sequence comprises: Based on the probability distribution of each candidate word in the candidate text sequence, each candidate word in the candidate text sequence is corrected in sequence according to the order of the candidate words in the candidate text sequence to obtain the corrected candidate text sequence.

3. The data processing method according to claim 1, wherein obtaining the reference text sequence and the candidate text sequence to be verified includes: Obtain the target text sequence and verification indication information sent by the client, wherein the verification indication information is used to indicate the at least one candidate word character to be verified in the target text sequence; Based on the target text sequence and the verification indication information, a reference text sequence and a candidate text sequence to be verified are determined.

4. The data processing method according to claim 3, wherein determining the reference text sequence and the candidate text sequence to be verified based on the target text sequence and the verification indication information comprises: In response to the verification indication information indicating that each word in the target text sequence is a candidate word, the target text sequence is determined as a candidate text sequence to be verified, and the reference text sequence determined by the server and corresponding to the reasoning task is obtained. In response to the verification indication information indicating that some words in the target text sequence are candidate words, a reference text sequence and a candidate text sequence to be verified are determined from the target text sequence based on the verification indication information.

5. The data processing method according to claim 4, wherein the verification indication information includes: At least one of the number of verified lexical units and the number of lexical units to be verified; The verification indication information indicates that each word in the target text sequence is a candidate word, including at least one of the following: The number of verified lexical units is zero; The number of lexical units to be verified is the total number of lexical units in the target text sequence; The verification indication information indicates that some words in the target text sequence are candidate words, including at least one of the following: The number of verified lexical units is not zero; The number of lexical units to be verified is less than the total number of lexical units in the target text sequence.

6. A data processing method applied to a client for task reasoning, comprising: In response to the detection of a task request, the prompt words in the task request are identified as the reference text sequence corresponding to the reasoning task, and the prompt words are used to indicate the reasoning task; Based on the reference text sequence, the first model is invoked to perform task inference to obtain the inferred candidate text sequence, wherein the candidate text sequence includes at least one candidate word; At least the candidate text sequence should be sent to the server; Obtain the corrected candidate text sequence returned by the server. The corrected candidate text sequence is determined by the server based on the candidate text sequence using a second model. The parameter magnitude of the second model is greater than that of the first model. If, based on the corrected candidate text sequence, it is determined that the reasoning task has not yet been completed, the reference text sequence and the corrected candidate text sequence are used as the updated reference text sequence, and the operation of calling the first model to perform the task reasoning is returned. If the reasoning required to complete the reasoning task is determined based on the modified candidate text sequence, the reasoning result corresponding to the reasoning task is determined based on the reference text sequence and the modified candidate text sequence.

7. The data processing method according to claim 6, wherein sending at least the candidate text sequence to the server comprises: The reference text sequence and the candidate text sequence are combined into a target text sequence; The target text sequence and verification indication information are sent to the server, wherein the verification indication information is used to indicate at least one candidate word to be verified in the candidate text sequence.

8. The data processing method according to claim 7, wherein combining the reference text sequence and the candidate text sequence into a target text sequence comprises: In response to the fact that the reference text sequence only includes the prompt word, the reference text sequence and the candidate text sequence are combined into a target text sequence; In response to the reference text sequence including prompt words and a modified historical candidate text sequence, the candidate text sequence is determined as the target text sequence.

9. A server, comprising: Processor, memory, and transmission module; The memory is used to store the computer programs required for the processor to perform operations; The processor is configured to execute a computer program stored in the memory to perform the following operations: Obtain a reference text sequence and a candidate text sequence to be verified. The candidate text sequence is a text sequence inferred by the client based on the reference text sequence using a first model. The candidate text sequence includes at least one candidate word. The reference text sequence includes prompt words for indicating the inference task and words verified by the server. The second model is invoked to process the reference text sequence and the candidate text sequence to obtain the comprehensive probability distribution output by the second model. The comprehensive probability distribution includes the probability distribution of the position of each candidate word in the candidate text sequence in the vocabulary. The parameter magnitude of the second model is greater than that of the first model. Based on the probability distribution corresponding to each candidate word in the candidate text sequence, the corrected candidate text sequence is determined; The sending module is used to return the corrected candidate text sequence to the client as a verified reference text sequence.

10. A client device, comprising: Processor, transmitting module, and receiving module; The processor is configured to, in response to detecting a task request, determine the prompt words in the task request as a reference text sequence corresponding to the inference task, wherein the prompt words are used to indicate the inference task; Based on the reference text sequence, the first model is invoked to perform task inference to obtain the inferred candidate text sequence, wherein the candidate text sequence includes at least one candidate word; The sending module is used to send at least the candidate text sequence to the server; The receiving module is used to obtain the corrected candidate text sequence returned by the server. The corrected candidate text sequence is determined by the server based on the candidate text sequence using a second model. The parameter magnitude of the second model is greater than that of the first model. The processor is further configured to: if, based on the corrected candidate text sequence, determine that the reasoning task has not yet been completed, use the reference text sequence and the corrected candidate text sequence as the updated reference text sequence, and return to the operation of calling the first model to perform task reasoning; if, based on the corrected candidate text sequence, determine that the reasoning task has been completed, determine the reasoning result corresponding to the reasoning task based on the reference text sequence and the corrected candidate text sequence.

Images