Data processing method and apparatus, electronic device, and computer program product

Abstract

The present disclosure relates to the technical field of artificial intelligence, and provides a data processing method and apparatus, an electronic device, and a computer program product. The method comprises: acquiring a plurality of pieces of question information to be responded to; according to the plurality of pieces of question information, determining, from among a plurality of draft models of a target system, a target draft model corresponding to question information in the plurality of pieces of question information; and when the plurality of pieces of question information are processed by means of a large language model in the target system, using the target draft model to perform speculative decoding, so as to obtain response information corresponding to the plurality of pieces of question information. The present disclosure solves the technical problem in the related art that when a large language model performs data processing, an existing draft model cannot adapt to different types of data processing requests, resulting in relatively low data processing efficiency.

Description

Data processing methods and apparatus, electronic equipment and computer program products Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a data processing method and apparatus, electronic equipment, and computer program product. Background Technology

[0002] Speculative decoding is a novel decoding acceleration technique in the field of Large Language Model (LLM) inference optimization. Unlike traditional autoregressive decoding, speculative decoding can predict and generate multiple subsequent tokens for parallel verification, ultimately achieving the effect of generating multiple characters (tokens) after one iteration. In existing technologies, large language models use a fixed draft model for speculative decoding during data processing. However, if the average probability of the draft tokens proposed by the draft model being accepted is low, speculative decoding will not only fail to improve performance but will also continuously waste computational resources, leading to increased request response time and a significant reduction in the effective token throughput of the inference engine. Furthermore, if the request type distribution changes, the existing draft model may become unable to adapt to the current request type, resulting in a lower token acceptance rate and consequently affecting the efficiency of responding to the request.

[0003] Regarding the issue that existing draft models cannot adapt to different types of data processing requests when performing data processing on large language models, resulting in relatively low data processing efficiency, no effective solution has yet been proposed. Summary of the Invention

[0004] This disclosure provides a data processing method and apparatus, electronic device, and computer program product to at least solve the technical problem in the related art where, when processing data in a large language model, the existing draft model cannot adapt to different types of data processing requests, resulting in relatively low data processing efficiency.

[0005] According to one aspect of the present disclosure, a data processing method is provided, comprising: acquiring multiple question information to be answered; determining a target draft model corresponding to question information in the multiple question information from multiple draft models of a target system based on the multiple question information; and performing speculative decoding using the target draft model when processing the multiple question information through a large language model in the target system to obtain answer information corresponding to the multiple question information.

[0006] Further, determining the target draft model corresponding to the problem information in the multiple problem information from multiple draft models of the target system based on the multiple problem information includes: classifying the problem information in the multiple problem information to obtain target category information corresponding to the problem information in the multiple problem information, and determining the target draft model from multiple draft models based on the target category information and the current load state of the target system; or, using a machine learning algorithm to predict the draft model corresponding to the problem information in the multiple problem information based on the multiple problem information and the current load state, so as to determine the target draft model from multiple draft models.

[0007] Furthermore, before determining the target draft model corresponding to the problem information in the multiple draft models of the target system based on the multiple problem information, the method further includes: detecting whether there is any unprocessed problem information in the waiting queue of the target system, and obtaining a judgment result; and determining the current load state of the target system based on the judgment result.

[0008] Furthermore, after determining the target draft model corresponding to the question information in the multiple question information from multiple draft models of the target system based on the multiple question information, the method further includes: determining the initial number of speculative characters of the target draft model based on the multiple question information and the current load state of the target system, wherein the initial number of speculative characters is the number of response characters output by the target draft model at one time; summing the initial number of speculative characters to obtain the total batch corresponding to the multiple question information; and determining whether the initial number of speculative characters needs to be adjusted to obtain the target number of speculative characters based on the total batch and a preset batch threshold.

[0009] Furthermore, when processing the multiple question information using the large language model in the target system, speculative decoding using the target draft model to obtain the response information corresponding to the multiple question information includes: reasoning about the target question information in the multiple question information using the target draft model to obtain multiple first response characters of a target inferred character count; determining, using the large language model, whether there is a target response character among the multiple first response characters that meets a preset condition; if there is a target response character among the multiple first response characters that meets the preset condition, then determining whether the target response character can generate the response information for the target question information; if the target response character can generate the response information for the target question information, then determining the response information based on the target response character.

[0010] Further, determining whether there is a target response character that meets preset conditions among the plurality of first response characters using the large language model includes: reasoning about the plurality of first response characters using the large language model to obtain a first generation probability value of the first response character among the plurality of first response characters; obtaining a second generation probability value of the first response character among the plurality of first response characters corresponding to the target draft model; and determining whether there is a target response character that meets preset conditions among the plurality of first response characters based on the first generation probability value and the second generation probability value.

[0011] Further, determining whether there is a target response character satisfying a preset condition among the plurality of first response characters based on the first generation probability value and the second generation probability value includes: determining whether the first first response character among the plurality of first response characters satisfies the preset condition based on the first generation probability value and the second generation probability value; if the first first response character does not satisfy the preset condition, then determining that there is no target response character satisfying the preset condition among the plurality of first response characters; if the first first response character satisfies the preset condition, then repeating the step of determining whether the second first response character among the plurality of first response characters satisfies the preset condition based on the first generation probability value and the second generation probability value, until determining whether the last first response character among the plurality of first response characters satisfies the preset condition.

[0012] Further, determining whether the first first reply character among the plurality of first reply characters satisfies the preset condition based on the first generation probability value and the second generation probability value includes: if the first generation probability value is greater than the second generation probability value, then determining that the first first reply character satisfies the preset condition; if the first generation probability value is less than the second generation probability value, then calculating the ratio between the first generation probability value and the second generation probability value to obtain a target ratio; and determining whether the first first reply character among the plurality of first reply characters satisfies the preset condition based on the target ratio and a random number.

[0013] Further, after determining whether there is a target response character satisfying a preset condition among the plurality of first response characters using the large language model, the method further includes: if there is no target response character satisfying the preset condition among the plurality of first response characters, then obtaining a first probability distribution of the large language model when generating a first generation probability value of the first first response character among the plurality of first response characters, wherein the first probability distribution is composed of generation probability values ​​corresponding to response characters in the character library; obtaining a second probability distribution corresponding to the target draft model, and sampling response characters in the character library to generate a second response character based on the difference probability distribution between the first probability distribution and the second probability distribution; concatenating the target question information and the second response character to obtain concatenated target question information; and repeatedly executing the step of processing the concatenated target question information through the target draft model until the response information is obtained.

[0014] Further, after determining whether the target response character can generate the response information for the target question information, the method further includes: if the target response character cannot generate the response information for the target question information, then obtaining a first probability distribution of the large language model when generating the first generation probability value of the last response character in the target response character, wherein the first probability distribution is composed of the generation probability values ​​corresponding to the response characters in the character library; obtaining a second probability distribution corresponding to the target draft model, and sampling the response characters in the character library to generate a second response character based on the difference probability distribution between the first probability distribution and the second probability distribution; concatenating the target question information, the target response character, and the second response character to obtain the concatenated target question information; and repeatedly executing the step of processing the concatenated target question information through the target draft model until the response information is obtained.

[0015] Further, determining whether the target response character can generate the response information for the target question includes: determining whether the target response character includes an end character to obtain a first determination result, and determining whether the target response character can generate the response information for the target question based on the first determination result; or, determining whether the target response character includes a preset character input by the target object to obtain a second determination result, and determining whether the target response character can generate the response information for the target question based on the second determination result; or, determining whether the number of the target response characters is equal to a preset character threshold to obtain a third determination result, and determining whether the target response character can generate the response information for the target question based on the third determination result.

[0016] Furthermore, after concatenating the target question information, the target response character, and the second response character to obtain the concatenated target question information, the method further includes: obtaining the current load state of the target system; calculating the character acceptance rate of the target draft model corresponding to the question information in the plurality of question information to obtain a plurality of character acceptance rates; if the current load state is a first state and the difference between the plurality of character acceptance rates is less than or equal to a first threshold, then updating the first draft model to the target draft model, wherein the first draft model is the draft model among the plurality of draft models whose computational cost is less than a preset cost; if the current load state is a second state, then updating the second draft model to the target draft model, wherein the second draft model is the draft model among the plurality of draft models whose computational character acceptance rate is higher than a second threshold.

[0017] According to another aspect of the embodiments of this disclosure, a data processing method is also provided, comprising: acquiring multiple question information to be answered sent by a client; determining a target draft model corresponding to the question information in the multiple question information from multiple draft models on a cloud server based on the multiple question information; when processing the multiple question information through a large language model on the cloud server, using the target draft model for speculative decoding to obtain the answer information corresponding to the multiple question information; and returning the answer information to the client.

[0018] According to another aspect of the embodiments of this disclosure, a data processing apparatus is also provided, the apparatus comprising: a first acquisition unit configured to acquire multiple question information to be answered; a first determination unit configured to determine, based on the multiple question information, a target draft model corresponding to question information in the multiple question information from multiple draft models of a target system; and a processing unit configured to, when processing the multiple question information through a large language model in the target system, perform speculative decoding using the target draft model to obtain answer information corresponding to the multiple question information.

[0019] Further, the determining unit includes: a classification subunit, configured to classify the problem information among the plurality of problem information to obtain target category information corresponding to the problem information among the plurality of problem information, and determine the target draft model from the plurality of draft models based on the target category information and the current load state; or, a prediction subunit, configured to predict the draft model corresponding to the problem information among the plurality of problem information based on the plurality of problem information and the current load state using a machine learning algorithm, so as to determine the target draft model from the plurality of draft models.

[0020] Furthermore, the device further includes: a detection unit, configured to detect whether there is any pending problem information in the waiting queue of the target system before determining the target draft model corresponding to the problem information in the multiple draft models of the target system based on the multiple problem information, and obtain a judgment result; and a second determination unit, configured to determine the current load state of the target system based on the judgment result.

[0021] Furthermore, the apparatus further includes: a third determining unit, configured to, after determining the target draft model corresponding to the question information in the multiple question information from multiple draft models of the target system based on the multiple question information, determine the initial speculative character count of the target draft model based on the multiple question information and the current load state of the target system, wherein the initial speculative character count is the number of response characters output by the target draft model at one time; a calculation unit, configured to sum the initial speculative character count to obtain the total batch corresponding to the multiple question information; and an adjustment unit, configured to determine whether the initial speculative character count needs to be adjusted based on the total batch and a preset batch threshold to obtain the target speculative character count.

[0022] Further, the processing unit includes: a reasoning subunit, configured to reason about the target question information among the plurality of question information using the target draft model to obtain a plurality of first response characters of the target predicted character count; a first judgment subunit, configured to determine, using the large language model, whether there is a target response character among the plurality of first response characters that meets a preset condition; a second judgment subunit, configured to determine whether the target response character can generate response information for the target question information if there is a target response character among the plurality of first response characters that meets the preset condition; and a fourth determination unit, configured to determine the response information based on the target response character if the target response character can generate response information for the target question information.

[0023] Further, the first judgment subunit includes: a reasoning module, configured to reason about the plurality of first response characters through the large language model to obtain a first generation probability value of the first response character among the plurality of first response characters; an acquisition module, configured to acquire a second generation probability value of the first response character among the plurality of first response characters corresponding to the target draft model; and a first judgment module, used to determine whether there is a target response character among the plurality of first response characters that meets preset conditions based on the first generation probability value and the second generation probability value.

[0024] Further, the judgment module includes: a judgment submodule, configured to determine whether the first first reply character among the plurality of first reply characters satisfies the preset condition based on the first generation probability value and the second generation probability value; a determination submodule, configured to determine that if the first first reply character does not satisfy the preset condition, there is no target reply character among the plurality of first reply characters that satisfies the preset condition; and an execution submodule, configured to repeatedly execute the step of determining whether the second first reply character among the plurality of first reply characters satisfies the preset condition based on the first generation probability value and the second generation probability value, until it is determined whether the last first reply character among the plurality of first reply characters satisfies the preset condition.

[0025] Further, the determination submodule includes: a determination submodule, configured to determine that the first first response character satisfies the preset condition if the first generation probability value is greater than the second generation probability value; a calculation submodule, configured to calculate the ratio between the first generation probability value and the second generation probability value to obtain a target ratio if the first generation probability value is less than the second generation probability value; and a determination submodule, configured to determine whether the first first response character among the plurality of first response characters satisfies the preset condition based on the target ratio and a random number.

[0026] Furthermore, the device further includes: a second acquisition unit, configured to, after determining whether there is a target response character satisfying a preset condition among the plurality of first response characters through the large language model, if there is no target response character satisfying the preset condition among the plurality of first response characters, acquire a first probability distribution when the large language model generates a first generation probability value of the first first response character among the plurality of first response characters, wherein the first probability distribution is composed of generation probability values ​​corresponding to response characters in the character library; a third acquisition unit, configured to acquire a second probability distribution corresponding to the target draft model, and sample response characters in the character library to generate a second response character based on the difference probability distribution between the first probability distribution and the second probability distribution; a first concatenation unit, configured to concatenate the target question information and the second response character to obtain concatenated target question information; and a first execution unit, configured to repeatedly execute the step of processing the concatenated target question information through the target draft model until the response information is obtained.

[0027] Furthermore, the device further includes: a third acquisition unit, configured to, after determining whether the target response character can generate response information for the target question information, if the target response character cannot generate response information for the target question information, acquire a first probability distribution of the large language model when generating the first generation probability value of the last response character in the target response character, wherein the first probability distribution is composed of generation probability values ​​corresponding to response characters in the character library; a fourth acquisition unit, configured to acquire a second probability distribution corresponding to the target draft model, and sample response characters in the character library to generate a second response character based on the difference probability distribution between the first probability distribution and the second probability distribution; a second concatenation unit, configured to concatenate the target question information, the target response character, and the second response character to obtain concatenated target question information; and a second execution unit, configured to repeatedly execute the step of processing the concatenated target question information through the target draft model until the response information is obtained.

[0028] Further, the second judgment subunit includes: a second judgment module, configured to judge whether the target response character includes an end character, obtain a first judgment result, and judge whether the target response character can generate the response information for the target question information based on the first judgment result; or, a third judgment module, configured to judge whether the target response character includes a preset character input by the target object, obtain a second judgment result, and judge whether the target response character can generate the response information for the target question information based on the second judgment result; or, a fourth judgment module, configured to judge whether the number of the target response characters is equal to a preset character threshold, obtain a third judgment result, and judge whether the target response character can generate the response information for the target question information based on the third judgment result.

[0029] Furthermore, the device further includes: a fifth acquisition unit, configured to, after concatenating the target question information, the target response character, and the second response character to obtain concatenated target question information, acquire the current load state of the target system and calculate the character acceptance rate of the target draft model corresponding to the question information among the multiple question information to obtain multiple character acceptance rates; a first update unit, configured to update the first draft model to the target draft model if the current load state is a first state and the difference between the multiple character acceptance rates is less than or equal to a first threshold, wherein the first draft model is a draft model among the multiple draft models whose computational cost is less than a preset cost; and a second update unit, configured to update the second draft model to the target draft model if the current load state is a second state, wherein the second draft model is a draft model among the multiple draft models whose computational character acceptance rate is higher than a second threshold.

[0030] According to another aspect of the present invention, an electronic device is also provided, comprising: a memory storing an executable program; and a processor configured to run the program, wherein the program, when running, executes the data processing method described in any one of the preceding embodiments.

[0031] According to another aspect of the present invention, a computer program product is also provided, the computer program product comprising a stored computer program that, when executed by a processor, implements the data processing method described in any one of the preceding embodiments.

[0032] In this embodiment, the method involves acquiring multiple question information to be answered; determining a target draft model corresponding to the question information from multiple draft models of the target system based on the multiple question information; and using the target draft model for speculative decoding when processing the multiple question information through the large language model of the target system to obtain the answer information corresponding to the multiple question information. This solves the problem in related technologies where existing draft models cannot adapt to different types of data processing requests when processing data through the large language model, resulting in low data processing efficiency. By using different draft models based on multiple question information, the effective token acceptance rate is improved, and different draft models can better adapt to changes in question information, thereby achieving the technical effect of improving the efficiency of question information processing. Attached Figure Description

[0033] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:

[0034] Figure 1 is a hardware structure block diagram of a computer terminal provided according to Embodiment 1 of this disclosure;

[0035] Figure 2 is a flowchart of a data processing method provided according to Embodiment 1 of this disclosure;

[0036] Figure 3 is a schematic diagram of speculative decoding provided according to Embodiment 1 of this disclosure;

[0037] Figure 4 is a schematic diagram of speculative decoding provided according to Embodiment 1 of this disclosure;

[0038] Figure 5 is a schematic diagram of the data processing architecture provided according to Embodiment 1 of this disclosure;

[0039] Figure 6 is a flowchart of a data processing method provided according to Embodiment 2 of this disclosure;

[0040] Figure 7 is a schematic diagram of a data processing apparatus provided according to Embodiment 3 of the present disclosure;

[0041] Figure 8 is a structural block diagram of an electronic device provided according to Embodiment 4 of this disclosure. Detailed Implementation

[0042] To enable those skilled in the art to better understand the present disclosure, the technical solutions of the present disclosure will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present disclosure.

[0043] The terms "first," "second," etc., used in this disclosure, claims, and accompanying drawings are for distinguishing similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0044] First, some nouns or terms that appear in the description of the embodiments of this disclosure shall be interpreted as follows:

[0045] Large Language Model (LLM): A large language model typically refers to a computational model trained using deep learning techniques, especially neural networks, that is capable of understanding and generating natural language. It usually contains a large number of parameters and is able to capture complex patterns and contextual relationships in language. It is widely used in natural language processing tasks such as text generation, translation, and question answering systems.

[0046] Inference engine: A software component or system responsible for reasoning and decision-making based on specific rules and knowledge bases. Inference engines process input information and use model reasoning to generate conclusions or answers; they are commonly used in expert systems, intelligent question-answering assistants, and other fields.

[0047] In the field of natural language processing, a token refers to a lexical unit. A token is the smallest semantic unit in text, which can be a word, subword, or character. Large language models typically segment the input text into tokens for processing and output generation.

[0048] Speculative decoding: Speculative decoding is a novel decoding acceleration technique in the field of Large Language Model (LLM) inference optimization. Unlike the traditional autoregressive decoding process, speculative decoding can predict and generate multiple subsequent tokens for parallel verification, ultimately achieving the effect of generating multiple tokens after one iteration.

[0049] Speculation length: refers to the length of the subsequent inferred text based on current information during speculative reasoning. Normally, LLM (Limited Memory Management) deterministically generates one subsequent token through autoregressive decoding, while speculative decoding can infer the generation of k subsequent tokens; the later the token is, the lower its probability of acceptance.

[0050] Draft Model: A draft model is a proposal model responsible for generating draft tokens during speculative reasoning. A draft model does not necessarily need to be a parameterized model; it can be an LLM with fewer parameters, an algorithm that searches and matches prompt word phrases, or even a specially structured model appended to the target LLM model.

[0051] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0052] Example 1

[0053] According to embodiments of this disclosure, a data processing method is also provided, wherein the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0054] The method embodiment provided in this disclosure can be executed in a mobile terminal, computer terminal, or similar computing device. Figure 1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a data processing method. As shown in Figure 1, the computer terminal (or mobile device) 10 may include a processor set 102 (the processor set 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, and the processor set 102 may include a processor set, shown in Figure 1 as 102a, 102b, ..., 102n), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a cursor control device, a keyboard, a display, an input / output interface (I / O interface), a Universal Serial Bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that the structure shown in Figure 1 is schematic and does not limit the structure of the above-described electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in Figure 1, or have a different configuration than shown in Figure 1.

[0055] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuitry are generally referred to herein as "data processing circuitry". This data processing circuitry may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in embodiments of this disclosure, the data processing circuitry serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0056] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the data processing method in this embodiment of the present disclosure. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby implementing the above-mentioned data processing method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0057] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0058] The display may be a touchscreen LCD display that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0059] Under the aforementioned operating environment, this disclosure provides a data processing method as shown in Figure 2. Figure 2 is a flowchart of the data processing method according to Embodiment 1 of this disclosure. The data processing method includes:

[0060] Step S201: Obtain information on multiple questions that need to be answered.

[0061] Optionally, users can input multiple questions to be answered through the front-end interface. These multiple questions can correspond to different users. For example, if there are n requests from different users, then n questions to be answered can be retrieved. The questions can be queries, such as "What's the weather like today?" or creative questions, such as "Write a short essay on a certain theme."

[0062] Step S202: Based on multiple problem information, determine the target draft model corresponding to the problem information in the multiple problem information from multiple draft models of the target system.

[0063] Optionally, after obtaining information on multiple questions to be answered, based on this information and the current load status of the target system, a target draft model corresponding to the question information in the multiple draft models of the target system is determined. For example, the multiple draft models may include small LLMs and n-gram drafts. For requests with higher creative requirements, small LLMs can be used to obtain higher-quality draft tokens at a higher generation cost, achieving a higher average acceptance rate; while for document retrieval-type QA requests, n-gram draft models, which obtain draft tokens by searching for and matching n-gram phrases in prompt words, can be used to generate slightly lower-quality draft tokens at a negligible cost, achieving a faster speculative decoding rate and a higher acceptance rate.

[0064] In an optional embodiment, when determining the target draft model corresponding to the problem information among multiple problem information, the load of the target system can also be considered. If the current load is relatively light and can withstand more invalid computational overhead, a draft model with a lower cost can be selected. If the current load is relatively heavy, a draft model with a higher acceptance rate can be selected.

[0065] In an alternative embodiment, the category of the problem information (e.g., creative category or document retrieval category) and the load of the target system can be taken into account to select a more suitable draft model for the problem information.

[0066] In step S203, when processing the multiple question information through the large language model in the target system, speculative decoding is performed using the target draft model to obtain the response information corresponding to the multiple question information.

[0067] Optionally, after determining the target draft model mentioned above, multiple question information is processed based on the target draft model and the large language model in the target system to obtain the response information corresponding to the multiple question information.

[0068] In an optional embodiment, when a large language model is required to process multiple question information, speculative decoding is performed using a target draft model, i.e., predicting and generating multiple subsequent tokens. Then, the large language model performs parallel scoring, and the multiple speculative tokens are sequentially accepted with probability to obtain the response information corresponding to multiple question information.

[0069] In summary, by using different draft models based on multiple question information, the effective token acceptance rate was improved. Furthermore, different draft models can better adapt to changes in question information, thereby achieving the technical effect of improving the efficiency of question information processing.

[0070] To improve the accuracy of selecting draft models, in the data processing method provided in Embodiment 1 of this disclosure, determining the target draft model corresponding to the problem information in the multiple draft models of the target system based on multiple problem information includes: classifying the problem information in the multiple problem information to obtain target category information corresponding to the problem information in the multiple problem information, and determining the target draft model from the multiple draft models based on the target category information and the current load state of the target system; or, using a machine learning algorithm to predict the draft model corresponding to the problem information in the multiple problem information based on the multiple problem information and the current load state, so as to determine the target draft model from the multiple draft models.

[0071] Optionally, the following steps are used to determine the target draft model corresponding to the problem information in the multiple problem information from multiple draft models of the target system: Classifying the problem information in the multiple problem information, for example, dividing the problem information into categories with higher creativity requirements and categories targeting document retrieval. This classification of multiple problem information can be done using a deep learning model. After obtaining the categories corresponding to the problem information, the target draft model corresponding to the problem information is determined based on the category of the problem information (e.g., creativity category or document retrieval category) and the load of the target system. A mapping table can be constructed, that is, a mapping table between the category and load of the problem information and the corresponding target draft model, and then the draft model corresponding to the problem information is selected according to the mapping table.

[0072] Optionally, a machine learning algorithm can also be trained to predict the draft models corresponding to the problem information from multiple problem information sets based on multiple problem information sets and the current load state, so as to determine the target draft model from multiple draft models. For example, a training sample set can be constructed based on multiple sample problems, corresponding load states, and the actual selected draft models. The machine learning algorithm can be trained using this training sample set so that the trained machine learning algorithm can predict the draft models.

[0073] Based on the nature of the problem information and the current workload, a more suitable draft model can be selected for the problem information, thereby improving the efficiency of subsequent processing of the problem information.

[0074] Determining the load status is crucial. Therefore, in the data processing method provided in Embodiment 1 of this disclosure, before determining the target draft model corresponding to the problem information in the multiple draft models of the target system based on multiple problem information, the method further includes: detecting whether there is any problem information to be processed in the waiting queue of the target system, and obtaining a judgment result; and determining the current load status based on the judgment result.

[0075] Optionally, the current load status can be determined by detecting whether there are pending issues in the waiting queue of the target system. For example, the current load status is divided into a first state and a second state. If there are no pending issues in the waiting queue, it means that the current load status of the target system is relatively light, which is the first state. If there are pending issues in the waiting queue, it means that the current load status of the target system is relatively heavy, which is the second state.

[0076] By checking whether there are pending issues in the waiting queue of the target system, the current load of the target system can be accurately assessed, thereby improving the accuracy of the subsequent determination of the target draft model.

[0077] In an alternative embodiment, the current load status can also be determined by the amount of video memory occupied in the target system.

[0078] To further improve the efficiency of processing problem information, in the data processing method provided in Embodiment 1 of this disclosure, after determining the target draft model corresponding to the problem information from multiple draft models of the target system based on multiple problem information, the method further includes: determining the initial number of speculative characters of the target draft model based on the multiple problem information and the current load state of the target system, wherein the initial number of speculative characters is the number of response characters output by the target draft model at one time; summing the initial number of speculative characters to obtain the total batch corresponding to the multiple problem information; and determining whether the initial number of speculative characters needs to be adjusted to obtain the target number of speculative characters based on the total batch and a preset batch threshold.

[0079] Optionally, the initial predicted character count for the target draft model is determined based on multiple question information and the current load status of the target system. For example, if the question information is a highly creative question and the current load status is heavy, the initial predicted character count can be set relatively small, such as 3. If the question information is a query-type question and the current load status is relatively light, the initial predicted character count can be set relatively large, such as 5. In an optional embodiment, the initial predicted character count can also be obtained by predicting the character count based on the question information and the current load status of the target system using a deep learning model.

[0080] After obtaining the initial estimated number of characters, the total batch size corresponding to multiple problem messages is calculated by summing these initial estimated character counts. A batch refers to the amount of data that needs to be processed simultaneously in this data processing phase. Therefore, to avoid the target system's performance being insufficient to support all the current problem messages, a batch threshold can be set based on the performance of the server in the target system (such as graphics card model, video memory size, CPU model, memory size, and other hardware characteristics).

[0081] Determine the relationship between the total batch size and the preset batch threshold. Based on this relationship, determine whether the initial number of predicted characters needs to be adjusted. If the total batch size is greater than the preset batch threshold, adjust the initial number of predicted characters to obtain the target number of predicted characters.

[0082] In an optional embodiment, if there are n question messages from different users, and the estimated length of each question message (i.e., the initial estimated character count mentioned above) is k, then the current total batch size can be considered to be n x k. If the initial estimated character counts set for the question messages in the n question messages are inconsistent, then the total batch size can be considered to be k_1 + k_2 + ... + k_n. When the total batch size reaches a preset threshold k_sum_threshold (i.e., the batch threshold mentioned above), k_1 to k_n need to be sorted, for example, from largest to smallest. Then, the initial estimated character count of the first-ranked message is selected and reduced by 1. It is then determined whether the current total batch size is less than or equal to the threshold. If the requirement is still not met, the initial estimated character count of the first-ranked message is repeatedly selected and reduced by 1 until the requirement is met. It should be noted that the total batch size usually exceeds the threshold due to the arrival of new question messages, so the currently running iteration steps are not affected, and the next iteration steps will take effect.

[0083] By following the steps above, the number of speculative characters can be set more reasonably, thereby ensuring that the output token is increased without affecting the overall data processing efficiency.

[0084] Obtaining response information for multiple questions is crucial. Therefore, in the data processing method provided in Embodiment 1 of this disclosure, when processing the multiple questions using a large language model in the target system, speculative decoding using a target draft model is employed to obtain response information corresponding to the multiple questions. This includes: reasoning about the target question information among the multiple questions using the target draft model to obtain multiple first response characters of a target speculative character count; determining, using the large language model, whether there is a target response character among the multiple first response characters that meets preset conditions; if there is a target response character among the multiple first response characters that meets preset conditions, determining whether the target response character can generate response information for the target question information; and if the target response character can generate response information for the target question information, determining the response information based on the target response character.

[0085] Optionally, the aforementioned multiple question information is input into the target draft model. The target draft model infers from the target question information to obtain multiple first response characters of the target predicted character count. For example, if the target predicted character count is 3, then the number of first response characters is 3, and the target draft model outputs 3 tokens (i.e., the aforementioned multiple first response characters) at a time. The target question information can be any one of the multiple question information.

[0086] After obtaining the multiple first response characters, the large language model is used to determine whether there is a target response character that meets the preset conditions. For example, the large language model can score the multiple first response characters, and then the scores can be used to determine whether the multiple first response characters meet the preset conditions. The preset conditions may be whether the score corresponding to the first response character is higher than a threshold.

[0087] If, among multiple initial response characters, there is a target response character that meets preset conditions, it can be determined whether the target response character can generate a response to the target question information; that is, whether the current target response character can answer the target question information. If the target response character can generate a response to the target question information, the response information is determined directly based on the target response character and returned to the user.

[0088] Speculative decoding of LLM using the target draft model can effectively improve the efficiency of responding to question information.

[0089] To improve the accuracy of determining whether multiple first response characters meet preset conditions, the data processing method provided in Embodiment 1 of this disclosure, by using a large language model, determines whether there is a target response character among the multiple first response characters that meets preset conditions, including: reasoning about the multiple first response characters using the large language model to obtain a first generation probability value of the first response character among the multiple first response characters; obtaining a second generation probability value of the first response character among the multiple first response characters corresponding to the target draft model; and determining whether there is a target response character among the multiple first response characters that meets preset conditions based on the first generation probability value and the second generation probability value.

[0090] Optionally, the first generation probability value of the first response character among multiple first response characters can be obtained by reasoning through a large language model. For example, the k presumably generated tokens can be converted into k+1 parallel traditional autoregressive inference requests, assuming that the preceding tokens of the current token are accepted. Then, these requests are simultaneously handed over to the original target LLM for parallel inference to obtain the probability distribution of each request generating the next token.

[0091] For example, as shown in Figure 3, the question information is AB, the corresponding k is 3, and the three generated tokens are 123. The four parallel traditional autoregressive inference requests can be AB, AB1, AB12, and AB123. These four traditional autoregressive inference requests are input into a large language model for parallel inference to obtain the probability distribution of the next token being generated. Specifically, the large language model infers the first generation probability value of character 1 from AB, the first generation probability value of character 2 from AB1, and the first generation probability value of character 3 from AB12. Furthermore, if 123 is considered to meet preset requirements, the large language model can infer a new response character from AB123.

[0092] After obtaining the first generation probability value, multiple first response characters can be judged based on the first generation probability value. The second generation probability value of the first response character among the multiple first response characters corresponding to the target draft model can also be obtained, that is, the second generation probability value of the target draft model when inferring multiple first response characters.

[0093] Finally, based on the first generation probability value and the second generation probability value, it is determined whether there is a target response character among the multiple first response characters that meets the preset conditions. In an optional embodiment, the target generation probability value of the first response character can be determined based on the first generation probability value and the second generation probability value. For example, the target generation probability value can be obtained by weighted summing of the first generation probability value and the second generation probability value. This target generation probability value is used to characterize the probability that the first response character is the current output character. If the target generation probability value is higher than a set threshold, then the first response character is the target response character that meets the preset requirements.

[0094] Compared to using the generation probability value of a single model for evaluation, using the first generation probability value and the second generation probability value output by two models can more accurately assess the probability that the first response character is the current output character, thereby improving the accuracy of subsequent responses to the question information.

[0095] To further improve the accuracy of the target response character, in the data processing method provided in Embodiment 1 of this disclosure, determining whether there is a target response character that meets a preset condition among a plurality of first response characters based on a first generation probability value and a second generation probability value includes: determining whether the first first response character among a plurality of first response characters meets the preset condition based on the first generation probability value and the second generation probability value; if the first first response character does not meet the preset condition, then determining that there is no target response character that meets the preset condition among a plurality of first response characters; if the first first response character meets the preset condition, then repeating the step of determining whether the second first response character among a plurality of first response characters meets the preset condition based on the first generation probability value and the second generation probability value, until determining whether the last first response character among a plurality of first response characters meets the preset condition.

[0096] Optionally, since the probability of a token being accepted decreases (i.e., the probability of it meeting the preset conditions decreases) as it appears later in the list of k tokens, the first token has a higher probability of being accepted because the information used to speculate on it is all deterministic. The kth token, on the other hand, has a lower probability of being accepted because the information used to speculate on it contains k-1 pieces of speculative information. Therefore, if any token is not accepted, all subsequent speculative tokens do not need to be verified and are also not accepted. That is, the first first response character is first evaluated based on its first and second generation probability values ​​to determine whether it is accepted, i.e., whether it meets the preset conditions. If the first first response character is not accepted, then all subsequent first response characters do not need to be evaluated; that is, none of the multiple first response characters meet the target response conditions.

[0097] If the first first reply character meets the preset condition, the step of judging whether the second first reply character among multiple first reply characters meets the preset condition based on the first generation probability value and the second generation probability value is repeated until the last first reply character among multiple first reply characters meets the preset condition.

[0098] By following the steps above, once the current response character does not meet the preset conditions, no further judgment will be made on all subsequent speculative tokens, which can effectively improve the accuracy of the judgment on the response character.

[0099] In the data processing method provided in Embodiment 1 of this disclosure, determining whether the first first reply character among a plurality of first reply characters meets a preset condition based on a first generation probability value and a second generation probability value includes: if the first generation probability value is greater than the second generation probability value, then determining that the first first reply character meets the preset condition; if the first generation probability value is less than the second generation probability value, then calculating the ratio between the first generation probability value and the second generation probability value to obtain a target ratio; and determining whether the first first reply character among a plurality of first reply characters meets the preset condition based on the target ratio and a random number.

[0100] Optionally, based on the first generation probability value and the second generation probability value, it is determined whether the first first response character among multiple first response characters meets the preset conditions. This is mainly based on the probability acceptance of the speculative decoding verification algorithm. Assuming that for the token at position 1, the generation probability of LLM is q and the generation probability of the target draft model is p, in the process of top-1 greedy decoding (a text decoding strategy of LLM), this token will be accepted with a probability of min(1, q / p).

[0101] Therefore, if the first generation probability value is greater than the second generation probability value, LLM is more confident that the ratio between the two is greater than 1, and the first response character can be directly determined to be accepted, i.e., the preset condition is met. If the first generation probability value is less than the second generation probability value, it is accepted with probability q / p, i.e., the target ratio is obtained by calculating the ratio between the first and second generation probability values. Finally, based on the target ratio and a random number, it is determined whether the first response character among multiple first response characters meets the preset condition. For example, if the target ratio is greater than the random number, the first response character meets the preset condition; if the target ratio is less than or equal to the random number, the first response character does not meet the preset condition.

[0102] In an optional embodiment, if the first response character does not meet the preset conditions, a token can be sampled from the entire logits distribution of (qp). This can effectively ensure that the probability of acceptance and rejection in both cases is consistent with the adoption probability of LLM.

[0103] By determining whether the first response character is accepted by comparing the first generation probability value with the second generation probability value, the rationality and accuracy of the judgment on the first response character are improved.

[0104] If, among multiple first response characters, there is no target response character that meets the preset conditions, the data processing method provided in Embodiment 1 of this disclosure further includes: obtaining a first probability distribution of the first generation probability value of the first first response character among multiple first response characters generated by the large language model, wherein the first probability distribution is composed of the generation probability values ​​corresponding to the response characters in the character library; obtaining a second probability distribution corresponding to the target draft model, and sampling the response characters in the character library to generate a second response character based on the difference probability distribution between the first probability distribution and the second probability distribution; concatenating the target question information and the second response character to obtain the concatenated target question information; and repeatedly executing the step of processing the concatenated target question information through the target draft model until the response information is obtained.

[0105] Optionally, if no target response character satisfying the preset conditions exists among the multiple first response characters, i.e., no target response character is accepted among the multiple first response characters, then the first probability distribution of the first generation probability value of the first first response character generated by the large language model among the multiple first response characters is obtained, i.e., the logits distribution of the large language model generating the next token, and the second probability distribution corresponding to the target draft model is obtained. The first probability distribution consists of the generation probability values ​​corresponding to all possible response characters in the character library.

[0106] Then, based on the difference probability distribution between the first probability distribution and the second probability distribution, the response characters in the character library are sampled to generate the second response character. That is, based on the difference between the first probability distribution and the second probability distribution, a new probability distribution (the difference probability distribution mentioned above) is obtained. The response characters in the character library are sampled using the new probability distribution to generate the second response character, and the second response character is concatenated to the end of the question information to form a new decoding request (that is, the concatenated target question information mentioned above). The next round of speculative decoding continues, that is, the steps of processing the concatenated target question information through the target draft model are repeated until the response information is obtained.

[0107] Furthermore, in the data processing method provided in Embodiment 1 of this disclosure, after determining whether the target response character can generate response information for the target question information, the method further includes: if the target response character cannot generate response information for the target question information, obtaining the first probability distribution of the last response character in the generation of the target response character by the large language model, wherein the first probability distribution is composed of the generation probability values ​​corresponding to the response characters in the character library; obtaining the second probability distribution corresponding to the target draft model, and sampling the response characters in the character library to generate the second response character based on the difference probability distribution between the first probability distribution and the second probability distribution; concatenating the target question information, the target response character, and the second response character to obtain the concatenated target question information; and repeatedly executing the step of processing the concatenated target question information through the target draft model until the response information is obtained.

[0108] If the current target response character cannot answer the target question information, then the first probability distribution of the large language model when generating the first generation probability value of the last response character in the target response character and the second probability distribution corresponding to the target draft model are obtained. Then, based on the difference distribution formed by the difference probability distribution between the first probability distribution and the second probability distribution, the response characters in the character library are sampled to generate a second response character, and the second response character is concatenated to the end of the question information to form a new decoding request (i.e. the concatenated target question information mentioned above). The next round of speculative decoding continues, that is, the steps of processing the concatenated question information through the target draft model are repeated until the response information is obtained.

[0109] In an optional embodiment, the question information is AB, and the corresponding k is 3. If the three tokens generated by the draft model are 123, then the four parallel traditional autoregressive inference requests can be AB, AB1, AB12, and AB123. These four traditional autoregressive inference requests are input into a large language model for parallel inference to obtain the probability distribution of the request generating the next token. That is, the large language model infers the first generation probability value of character 1 from AB, the first generation probability value of character 2 from AB1, and the first generation probability value of character 3 from AB12. Based on the first generation probability, it is determined whether to accept the three tokens generated by the draft model. And if 3 is accepted, a new response character can be obtained by inferring from AB123 through the large language model.

[0110] As shown in Figure 4, if the three tokens output by the draft model are CDE, and if all draft tokens are accepted, then based on the large language model, the intermediate results of the reasoning of ABCDE are sampled to obtain a new token, as shown in F in ABCDEF in Figure 4.

[0111] For example, if the three tokens output by the draft model are C12, then the first draft token is accepted. Then, a new token is obtained by sampling the intermediate result when reasoning about ABC based on the large language model to obtain the first generation probability value of character 1, as shown in D of ABCD in Figure 4.

[0112] For example, if the draft model outputs 3 tokens as 123, then the first draft token in the corresponding 123 is not accepted. Then, a new token is obtained by sampling the intermediate result when reasoning about AB based on the large language model to obtain the first generation probability value of character 1, as shown in C in ABC in Figure 4.

[0113] Through the above speculative decoding process, a maximum of k+1 tokens can be generated in parallel in one speculative decoding, which is k+1 times that of traditional autoregressive decoding technology. Therefore, it can effectively improve the efficiency of answering questions.

[0114] The data processing method provided in Embodiment 1 of this disclosure for determining whether a target response character can generate a response to the target question includes the following steps: determining whether the target response character includes an end character to obtain a first determination result, and determining whether the target response character can generate a response to the target question based on the first determination result; or, determining whether the target response character includes a preset character input by the target object to obtain a second determination result, and determining whether the target response character can generate a response to the target question based on the second determination result; or, determining whether the number of target response characters is equal to a preset character threshold to obtain a third determination result, and determining whether the target response character can generate a response to the target question based on the third determination result.

[0115] Optionally, the following methods can be used to determine whether the target response characters are sufficient to generate a response to the target question: The first method includes determining whether the target response characters include a terminator. If the target response characters contain a terminator, then the target response characters can generate a response to the target question. The second method includes determining whether the target response characters include a preset character input by the target user. The preset character refers to the terminator indicated by the user. Therefore, if the target response characters include the preset character input by the target user, then the target response characters can generate a response to the target question. The third method includes determining whether the number of target response characters is equal to a preset character threshold. The character threshold can be set by the user according to their needs. If the number of target response characters is equal to the preset character threshold, then the target response characters can generate a response to the target question.

[0116] The three methods described above allow for a more flexible assessment of whether the target problem information has been resolved.

[0117] Regarding the selection of draft models, since the draft generation cost and acceptance rate of different requests in the same batch may be different, the following strategy can be used for global scheduling during concurrent processing: Obtain the current load state of the target system, calculate the character acceptance rate of the target draft model corresponding to the problem information in multiple problem information, and obtain multiple character acceptance rates; if the current load state is the first state, and the difference between the multiple character acceptance rates is less than or equal to the first threshold, then update the first draft model to the target draft model, where the first draft model is the draft model with a computational cost less than the preset cost among the multiple draft models; if the current load state is the second state, then update the second draft model to the target draft model, where the second draft model is the draft model with a computational character acceptance rate higher than the second threshold among the multiple draft models.

[0118] Optionally, after one inference, the current load state of the target system can be obtained, i.e., a first state or a second state, where the first state indicates a light load and the second state indicates a heavy load. Then, the character acceptance rate of the target draft model is obtained, i.e., the proportion of response characters output by the target draft model that meet preset conditions. If the current load state is the first state, and the difference between multiple character acceptance rates is less than or equal to a first threshold, then the cost of each round of draft token generation for requests in the same batch should be based on the higher cost of the entire batch. At the same time, since some requests have a very low draft token acceptance rate, it means that these requests need more iterations to complete, which will lead to a decrease in the processing efficiency of these requests (generating drafts at a higher cost and requiring more rounds). Therefore, in this case, if the load is light and can tolerate more invalid computational overhead, then the draft model with a lower draft generation cost is used for unified scheduling, that is, the first draft model is updated to the target draft model.

[0119] If the load is heavy, we should try to ensure that each calculation brings effective token throughput. Therefore, we should choose the model with the higher average draft acceptance rate as the unified draft model for the current batch and current step, that is, update the second draft model to the target draft model.

[0120] If the load is light and the difference between the acceptance rates of multiple characters is higher than the first threshold, then the target draft model remains unchanged, that is, different draft models are used for different question information.

[0121] Alternatively, instead of using the above strategy, the user can specify a fixed scheduling strategy with priorities, such as the given priority first strategy: given a priority, the model with the higher priority that appears in the same batch is used as the unified draft model for the current step of the current batch.

[0122] In an optional embodiment, the data processing architecture is shown in Figure 5, which mainly includes a speculative decoding inference engine. The inference engine consists of a scheduler, a draft model pool, a draft verification and acceptance module, and a target LLM. The specific operation process includes: Step 1: The user sends an inference request to the inference engine; Step 2: The scheduler in the inference engine determines the draft model selection and inference length for the current request based on the current load and the content of the user's request, i.e., schedules the draft model selection and inference length, and sends the results to the draft model pool; Step 3: The draft model pool generates corresponding draft tokens for each request according to the scheduler's requirements, i.e., transmits the draft tokens, and sends the results to the target LLM for parallel scoring, while also sending the results to the draft verification receiving module for verification; Step 4: The target LLM performs parallel scoring on the drafts and sends the results to the draft verification receiving module, i.e., transmits the scoring results; Step 5: The draft verification receiving module verifies and accepts the draft tokens sequentially, and returns the results to the scheduler for the next round of iteration, i.e., transmits the accepted results and continues iterating; Step 6: The scheduler determines whether the current request has been completed. If it has been completed, it jumps to Step 7; if it has not been completed, it jumps to Step 2; Step 7: The response is returned to the user, ending the interaction.

[0123] In the data processing method provided in Embodiment 1 of this disclosure, multiple question information to be answered is obtained; based on the multiple question information, a target draft model corresponding to the question information in the multiple question information is determined from multiple draft models of the target system; when processing the multiple question information through the large language model of the target system, the target draft model is used for speculative decoding to obtain the answer information corresponding to the multiple question information. This method solves the problem in related technologies where existing draft models cannot adapt to different types of data processing requests when processing data through the large language model, resulting in low data processing efficiency. By using different draft models according to multiple question information, the effective token acceptance rate is improved, and different draft models can better adapt to changes in question information, thereby achieving the technical effect of improving the efficiency of question information processing.

[0124] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0125] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this disclosure.

[0126] Example 2

[0127] According to an embodiment of this disclosure, a data processing method is also provided, as shown in FIG6, the data processing method comprising:

[0128] Step S601: Obtain information on multiple pending questions sent by the client;

[0129] Step S602: Based on multiple question information, determine the target draft model corresponding to the question information in the multiple question information from multiple draft models in the cloud server; when processing multiple question information through the large language model in the cloud server, use the target draft model for speculative decoding to obtain the response information corresponding to the multiple question information.

[0130] Step S603: Return the response information to the client.

[0131] The process of responding to questions on the cloud server is the same as the specific method provided in Example 1, and will not be repeated here.

[0132] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0133] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solutions of this disclosure, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this disclosure.

[0134] Example 3

[0135] According to an embodiment of this disclosure, a data processing apparatus for implementing the above-described data processing method is also provided, as shown in FIG7. The apparatus includes: a first acquisition unit 701, a first determination unit 702, and a processing unit 703.

[0136] The first acquisition unit 701 is configured to acquire information on multiple questions that need to be answered;

[0137] The first determining unit 702 is configured to determine, based on multiple problem information, a target draft model corresponding to the problem information in the multiple problem information from multiple draft models of the target system;

[0138] The processing unit 703 is configured to perform speculative decoding using the target draft model when processing multiple question information through the large language model in the target system, so as to obtain the response information corresponding to the multiple question information.

[0139] In the data processing method provided in Embodiment 3 of this disclosure, the first acquisition unit 701 acquires multiple question information to be answered; the first determination unit 702 determines the target draft model corresponding to the question information in the multiple question information from multiple draft models of the target system based on the multiple question information; when the processing unit 703 processes the multiple question information through the large language model of the target system, it uses the target draft model for speculative decoding to obtain the answer information corresponding to the multiple question information. This solves the problem in the related technology that when processing data through the large language model, the existing draft models cannot adapt to different types of data processing requests, resulting in low efficiency in data processing. By using different draft models according to multiple question information, the effective token acceptance rate is improved, and different draft models can better adapt to changes in question information, thereby achieving the technical effect of improving the efficiency of question information processing.

[0140] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the determining unit includes: a classification subunit, configured to classify the problem information in multiple problem information to obtain target category information corresponding to the problem information in multiple problem information, and determine a target draft model from multiple draft models based on the target category information and the current load state of the target system; or, a prediction subunit, configured to predict the draft models corresponding to the problem information in multiple problem information based on multiple problem information and the current load state using a machine learning algorithm, so as to determine the target draft model from multiple draft models.

[0141] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the apparatus further includes: a detection unit, configured to detect whether there is any pending problem information in the waiting queue of the target system before determining the target draft model corresponding to the problem information in the multiple draft models of the target system based on multiple problem information, and obtain a judgment result; and a second determination unit, configured to determine the current load state based on the judgment result.

[0142] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the apparatus further includes: a third determining unit, configured to, after determining the target draft model corresponding to the question information in the multiple question information from multiple draft models of the target system based on multiple question information, determine the initial speculative character count of the target draft model based on the multiple question information and the current load state of the target system, wherein the initial speculative character count is the number of response characters output by the target draft model at one time; a calculation unit, configured to sum the initial speculative character count to obtain the total batch corresponding to the multiple question information; and an adjustment unit, configured to determine whether the initial speculative character count needs to be adjusted based on the total batch and a preset batch threshold to obtain the target speculative character count.

[0143] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the processing unit includes: a reasoning subunit, configured to reason about the target question information in multiple question information through a target draft model to obtain multiple first response characters of the target predicted character number; a first judgment subunit, configured to determine whether there is a target response character that meets a preset condition among the multiple first response characters through a large language model; a second judgment subunit, configured to determine whether the target response character can generate response information for the target question information if there is a target response character that meets the preset condition among the multiple first response characters; and a fourth determination unit, configured to determine the response information based on the target response character if the target response character can generate response information for the target question information.

[0144] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the first judgment subunit includes: a reasoning module, configured to reason about multiple first response characters through a large language model to obtain a first generation probability value of the first response character among the multiple first response characters; an acquisition module, configured to acquire a second generation probability value of the first response character among the multiple first response characters corresponding to the target draft model; and a first judgment module, configured to determine whether there is a target response character that meets preset conditions among the multiple first response characters based on the first generation probability value and the second generation probability value.

[0145] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the judgment module includes: a judgment submodule, configured to determine whether the first first reply character among a plurality of first reply characters satisfies a preset condition based on a first generation probability value and a second generation probability value; a determination submodule, configured to determine that if the first first reply character does not satisfy the preset condition, there is no target reply character among the plurality of first reply characters that satisfies the preset condition; and an execution submodule, configured to repeatedly execute the step of determining whether the second first reply character among a plurality of first reply characters satisfies the preset condition based on the first generation probability value and the second generation probability value, until it is determined whether the last first reply character among a plurality of first reply characters satisfies the preset condition.

[0146] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the determination submodule includes: a determination submodule, configured to determine that the first first response character satisfies a preset condition if the first generation probability value is greater than the second generation probability value; a calculation submodule, configured to calculate the ratio between the first generation probability value and the second generation probability value to obtain a target ratio if the first generation probability value is less than the second generation probability value; and a judgment submodule, configured to determine whether the first first response character among a plurality of first response characters satisfies the preset condition based on the target ratio and a random number.

[0147] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the apparatus further includes: a second acquisition unit, configured to, after determining whether there is a target response character satisfying a preset condition among a plurality of first response characters through a large language model, if there is no target response character satisfying the preset condition among the plurality of first response characters, acquire a first probability distribution when the large language model generates a first generation probability value of the first first response character among the plurality of first response characters, wherein the first probability distribution is composed of generation probability values ​​corresponding to response characters in the character library; a third acquisition unit, configured to acquire a second probability distribution corresponding to the target draft model, and sample response characters in the character library to generate a second response character based on the difference probability distribution between the first probability distribution and the second probability distribution; a first splicing unit, configured to splice the target question information and the second response character to obtain spliced ​​target question information; and a first execution unit, configured to repeatedly execute the step of processing the spliced ​​target question information through the target draft model until response information is obtained.

[0148] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the apparatus further includes: a third acquisition unit, configured to, after determining whether the target response character can generate response information for the target question information, if the target response character cannot generate response information for the target question information, acquire a first probability distribution of the first generation probability value of the last response character in the target response character generated by the large language model, wherein the first probability distribution is composed of the generation probability values ​​corresponding to the response characters in the character library; a fourth acquisition unit, configured to acquire a second probability distribution corresponding to the target draft model, and sample the response characters in the character library to generate a second response character based on the difference probability distribution between the first probability distribution and the second probability distribution; a second splicing unit, configured to splice the target question information, the target response character, and the second response character to obtain spliced ​​target question information; and a second execution unit, configured to repeatedly execute the step of processing the spliced ​​target question information through the target draft model until response information is obtained.

[0149] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the second judgment subunit includes: a second judgment module, configured to judge whether the target reply character includes an end character, obtain a first judgment result, and judge whether the target reply character can generate reply information for the target question information based on the first judgment result; or, a third judgment module, configured to judge whether the target reply character includes a preset character input by the target object, obtain a second judgment result, and judge whether the target reply character can generate reply information for the target question information based on the second judgment result; or, a fourth judgment module, configured to judge whether the number of target reply characters is equal to a preset character threshold, obtain a third judgment result, and judge whether the target reply character can generate reply information for the target question information based on the third judgment result.

[0150] Optionally, in the data processing apparatus provided in Embodiment 3 of this disclosure, the apparatus further includes: a fifth acquisition unit, configured to, after concatenating the target question information, the target response character, and the second response character to obtain the concatenated target question information, acquire the current load state of the target system and calculate the character acceptance rate of the target draft model corresponding to the question information among the multiple question information to obtain multiple character acceptance rates; a first update unit, configured to, if the current load state is a first state and the difference between the multiple character acceptance rates is less than or equal to a first threshold, update the first draft model to the target draft model, wherein the first draft model is a draft model among the multiple draft models whose computational cost is less than a preset cost; and a second update unit, configured to, if the current load state is a second state, update the second draft model to the target draft model, wherein the second draft model is a draft model among the multiple draft models whose calculated character acceptance rate is higher than a second threshold.

[0151] The first acquisition unit 701, the first determination unit 702, and the processing unit 703 described above correspond to steps S201 to S203 in Embodiment 1. The three units and their corresponding steps implement the same instances and application scenarios, but are not limited to the content disclosed in Embodiment 1. These modules, as part of the device, can run on the computer terminal 10 provided in Embodiment 1.

[0152] The preferred embodiments involved in the above embodiments of this disclosure are the same as the solutions, application scenarios and implementation processes provided in Embodiment 1, but are not limited to the solutions provided in Embodiment 1.

[0153] Example 4

[0154] Embodiments of this disclosure can provide an electronic device, which can be any electronic device in a group of electronic device terminals. Optionally, in this embodiment, the aforementioned electronic device can also be replaced with a terminal device such as a mobile terminal.

[0155] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0156] In this embodiment, the computer terminal can execute the program code corresponding to the steps in the data processing method provided in any of the above method embodiments.

[0157] Optionally, FIG8 is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG8, the electronic device 80 may include: one or more (one shown in FIG8) processors 802 and memory 804. The electronic device 80 may also include a memory controller for controlling and managing the memory 804; the electronic device 80 may also include a peripheral interface for connecting to a radio frequency module, an audio module, and a display screen, etc.

[0158] The memory may be configured to store software programs and modules, such as the program instructions / modules corresponding to the data processing method and apparatus in this embodiment. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the aforementioned data processing method. The memory may include high-speed random access memory (RAM) and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, which can be connected to the electronic device 80 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks (LANs), mobile communication networks, and combinations thereof.

[0159] The processor can access the information and application programs stored in the memory via the transmission device, as well as the program code corresponding to the steps in the data processing method provided in any of the above method embodiments.

[0160] It will be understood by those skilled in the art that the structure shown in FIG8 is illustrative, and the electronic device 80 may also be a smartphone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile internet device (MID), a PAD, or other terminal device. FIG8 does not limit the structure of the aforementioned electronic device. For example, the electronic device 80 may also include more or fewer components (such as a network interface, a display device, etc.) than shown in FIG8, or have a different configuration than shown in FIG8.

[0161] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0162] Example 5

[0163] Embodiments of this disclosure also provide a computer program product. Optionally, in this embodiment, the computer program product may be configured to store the program code executed by the data processing method provided in Embodiment 1.

[0164] Optionally, in this embodiment, the computer program product may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0165] In the above embodiments of this disclosure, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0166] In the several embodiments provided in this disclosure, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are illustrative; for example, the division of units is a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0167] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0169] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0170] The above description represents the preferred embodiments of this disclosure. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this disclosure, and these improvements and modifications should also be considered within the scope of protection of this disclosure. Industrial applicability

[0171] In this embodiment, the method involves acquiring multiple question information to be answered; determining a target draft model corresponding to the question information from multiple draft models of the target system based on the multiple question information; and using the target draft model for speculative decoding when processing the multiple question information through the large language model of the target system to obtain the answer information corresponding to the multiple question information. This solves the problem in related technologies where existing draft models cannot adapt to different types of data processing requests when processing data through the large language model, resulting in low data processing efficiency. By using different draft models based on multiple question information, the effective token acceptance rate is improved, and different draft models can better adapt to changes in question information, thereby achieving the technical effect of improving the efficiency of question information processing.

Claims

1. A data processing method, comprising: Retrieve information on multiple pending questions; Based on the aforementioned multiple problem information, a target draft model corresponding to the problem information in the multiple problem information is determined from the multiple draft models of the target system; When processing the multiple question information through the large language model in the target system, speculative decoding is performed using the target draft model to obtain the response information corresponding to the multiple question information.

2. The method according to claim 1, wherein, Based on the aforementioned problem information, determining the target draft model corresponding to each problem information from multiple draft models of the target system includes: The problem information in the plurality of problem information is classified to obtain target category information corresponding to the problem information in the plurality of problem information, and the target draft model is determined from the plurality of draft models based on the target category information and the current load state of the target system; or, The target draft model is determined from the multiple draft models by using machine learning algorithms to predict the draft models corresponding to the multiple problem information based on the multiple problem information and the current load status.

3. The method according to claim 1, wherein, Before determining the target draft model corresponding to the problem information in the multiple problem information from multiple draft models of the target system based on the multiple problem information, the method further includes: The system detects whether there are any pending issues in the waiting queue of the target system and obtains a judgment result. Based on the judgment result, the current load status of the target system is determined.

4. The method according to claim 1, wherein, After determining the target draft model corresponding to the problem information in the multiple problem information from multiple draft models of the target system based on the multiple problem information, the method further includes: Based on the multiple question information and the current load status of the target system, the initial number of speculative characters of the target draft model is determined, wherein the initial number of speculative characters is the number of response characters output by the target draft model at one time; The total batch corresponding to the multiple question information is obtained by summing the initial predicted character counts. Based on the total number of batches and the preset batch threshold, it is determined whether the initial number of predicted characters needs to be adjusted to obtain the target number of predicted characters.

5. The method according to claim 1, wherein, When processing the multiple question information using the large language model in the target system, speculative decoding is performed using the target draft model to obtain the response information corresponding to the multiple question information, including: By reasoning about the target question information in the multiple question information through the target draft model, multiple first response characters of the target inferred character number are obtained; Using the large language model, it is determined whether there is a target response character among the plurality of first response characters that meets the preset conditions; If among the plurality of first response characters there is a target response character that meets the preset conditions, then it is determined whether the target response character can generate response information for the target question information; If the target response character can generate response information for the target question, then the response information is determined based on the target response character.

6. The method according to claim 5, wherein, Determining whether a target response character that meets preset conditions exists among the plurality of first response characters using the large language model includes: By reasoning about the plurality of first response characters using the large language model, a first generation probability value of the first response character among the plurality of first response characters is obtained; Obtain the second generation probability value of the first response character among the plurality of first response characters corresponding to the target draft model; Based on the first generation probability value and the second generation probability value, it is determined whether there is a target response character among the plurality of first response characters that meets the preset conditions.

7. The method according to claim 6, wherein, Based on the first generation probability value and the second generation probability value, determining whether there is a target response character among the plurality of first response characters that meets preset conditions includes: Based on the first generation probability value and the second generation probability value, determine whether the first first response character among the plurality of first response characters satisfies the preset condition; If the first first reply character does not meet the preset condition, then it is determined that there is no target reply character among the plurality of first reply characters that meets the preset condition; If the first first reply character satisfies the preset condition, the step of determining whether the second first reply character among the plurality of first reply characters satisfies the preset condition based on the first generation probability value and the second generation probability value is repeated until it is determined whether the last first reply character among the plurality of first reply characters satisfies the preset condition.

8. The method according to claim 7, wherein, Determining whether the first first reply character among the plurality of first reply characters satisfies the preset condition based on the first generation probability value and the second generation probability value includes: If the first generation probability value is greater than the second generation probability value, then the first response character is determined to satisfy the preset condition; If the first generation probability value is less than the second generation probability value, then the ratio between the first generation probability value and the second generation probability value is calculated to obtain the target ratio. Based on the target ratio and the random number, determine whether the first first reply character among the plurality of first reply characters satisfies the preset condition.

9. The method according to claim 5, wherein, After determining, through the large language model, whether there exists a target response character among the plurality of first response characters that meets preset conditions, the method further includes: If there is no target response character that meets the preset condition among the plurality of first response characters, then obtain the first probability distribution of the large language model when generating the first generation probability value of the first first response character among the plurality of first response characters, wherein the first probability distribution is composed of the generation probability values ​​corresponding to the response characters in the character library; Obtain the second probability distribution corresponding to the target draft model, and sample the reply characters in the character library to generate the second reply character based on the difference probability distribution between the first probability distribution and the second probability distribution; The target question information and the second response character are concatenated to obtain the concatenated target question information; Repeat the steps of processing the spliced ​​target question information through the target draft model until the response information is obtained.

10. The method according to claim 5, wherein, After determining whether the target response character can generate a response to the target question information, the method further includes: If the target response character cannot generate a response to the target question information, then obtain the first probability distribution of the large language model when generating the first generation probability value of the last response character in the target response character, wherein the first probability distribution is composed of the generation probability values ​​corresponding to the response characters in the character library; Obtain the second probability distribution corresponding to the target draft model, and sample the reply characters in the character library to generate the second reply character based on the difference probability distribution between the first probability distribution and the second probability distribution; The target question information, the target response character, and the second response character are concatenated to obtain the concatenated target question information; Repeat the steps of processing the spliced ​​target question information through the target draft model until the response information is obtained.

11. The method according to claim 5, wherein, Determining whether the target response characters can generate response information for the target question includes: Determine whether the target response characters include a terminator to obtain a first determination result, and based on the first determination result, determine whether the target response characters can generate response information for the target question; or, Determine whether the target response character includes the preset character input by the target object to obtain a second determination result, and based on the second determination result, determine whether the target response character can generate response information for the target question; or, Determine whether the number of target response characters is equal to a preset character threshold to obtain a third determination result, and based on the third determination result, determine whether the target response characters can generate response information for the target question information.

12. The method according to claim 10, wherein, After concatenating the target question information, the target response character, and the second response character to obtain the concatenated target question information, the method further includes: Obtain the current load status of the target system, calculate the character acceptance rate of the target draft model corresponding to the problem information among the multiple problem information, and obtain multiple character acceptance rates; If the current load state is the first state and the difference between the acceptance rates of the multiple characters is less than or equal to the first threshold, then the first draft model is updated to the target draft model, wherein the first draft model is the draft model among the multiple draft models whose computational cost is less than the preset cost. If the current load state is the second state, then the second draft model is updated to the target draft model, wherein the second draft model is the draft model among the plurality of draft models whose calculated character acceptance rate is higher than the second threshold.

13. A data processing method, comprising: Retrieve information on multiple pending questions sent by the client; Based on the multiple question information, a target draft model corresponding to the question information in the multiple question information is determined from multiple draft models on the cloud server; when processing the multiple question information through the large language model in the cloud server, the target draft model is used for speculative decoding to obtain the response information corresponding to the multiple question information; The response information is returned to the client.

14. A data processing apparatus, comprising: The first acquisition unit is configured to acquire information on multiple questions that need to be answered; The first determining unit is configured to determine, based on the plurality of problem information, a target draft model corresponding to the problem information in the plurality of problem information from a plurality of draft models of the target system; The processing unit is configured to perform speculative decoding using the target draft model when processing the multiple question information through the large language model in the target system, so as to obtain the response information corresponding to the multiple question information.

15. An electronic device comprising: Memory, which stores executable programs; A processor is configured to run the program, wherein the program executes the following data processing method during runtime: acquiring multiple question information to be answered; determining a target draft model corresponding to the question information in the multiple question information from multiple draft models of the target system based on the multiple question information; and performing speculative decoding using the target draft model when processing the multiple question information through a large language model in the target system to obtain the answer information corresponding to the multiple question information.

16. The electronic device according to claim 15, wherein, The processor is also configured to run a program corresponding to the following data processing method: classifying the problem information in the plurality of problem information to obtain target category information corresponding to the problem information in the plurality of problem information, and determining the target draft model from a plurality of draft models based on the target category information and the current load state of the target system; or, using a machine learning algorithm to predict the draft models corresponding to the problem information in the plurality of problem information based on the plurality of problem information and the current load state, so as to determine the target draft model from a plurality of draft models.

17. The electronic device according to claim 15, wherein, The processor is also configured to run a program corresponding to the following data processing method: before determining the target draft model corresponding to the problem information in the multiple draft models of the target system based on the multiple problem information, detecting whether there is any problem information to be processed in the waiting queue of the target system, and obtaining a judgment result; and determining the current load state of the target system based on the judgment result.

18. The electronic device according to claim 15, wherein, The processor is further configured to run a program corresponding to the following data processing method: after determining the target draft model corresponding to the question information in the multiple question information from multiple draft models of the target system based on the multiple question information, determining the initial speculative character count of the target draft model based on the multiple question information and the current load state of the target system, wherein the initial speculative character count is the number of response characters output by the target draft model at one time; summing the initial speculative character count to obtain the total batch corresponding to the multiple question information; and determining whether the initial speculative character count needs to be adjusted to obtain the target speculative character count based on the total batch and a preset batch threshold.

19. The electronic device according to claim 15, wherein, The processor is also configured to run a program corresponding to the following data processing method: reasoning about the target question information in the plurality of question information through the target draft model to obtain a plurality of first response characters of the target inferred character number; determining, through the large language model, whether there is a target response character among the plurality of first response characters that meets a preset condition; if there is a target response character among the plurality of first response characters that meets the preset condition, then determining whether the target response character can generate response information for the target question information; If the target response character can generate response information for the target question, then the response information is determined based on the target response character.

20. A computer program product comprising a stored computer program that, when executed by a processor, implements the data processing method according to any one of claims 1 to 13.

Images