A reply method, device and system based on a generative language model

Abstract

The application discloses a reply method, device and system based on a generative language model, belonging to the field of automatic reply. After a user inputs an instruction text, a plurality of generative language models are used to obtain a candidate output token respectively, then the product of the generation probability of each candidate output token and the preset weight of the corresponding generative language model is taken as the final target generation probability, and the candidate output token with the highest target generation probability is taken as the target output token. The text corresponding to the target output token is taken as the output of all generative language models, so that each token in the final reply text is obtained by comprehensively outputting all generative language models. Without further training of each generative language model, the cost is greatly reduced, and even if the candidate output token of a certain generative language model is incorrect, it can be eliminated in combination with the candidate output token of other generative language models, greatly increasing the accuracy of the final reply.

Description

TECHNICAL FIELD

[0001] The present application relates to an automatic reply system, in particular, to a reply method, device and system based on a generative language model. BACKGROUND

[0002] At present, generative large language models represented by GPT have become the mainstream model in the field of artificial intelligence, especially natural language processing. However, due to the probability in the generation process of the generative large language model, the "hallucination" problem is inevitable.

[0003] This phenomenon is one of the manifestations of the "hallucination" problem of large models. There are usually two solutions to this problem. One is to adjust the model generation parameters to reduce randomness, but this can only solve a small part of the generation errors caused by random sampling. If the token with the highest prediction probability at the current position is not the correct answer, this solution will fail. The second is to increase the model parameter quantity and the pre-training data quantity. Current researchers generally believe that the knowledge and ability of large language models come from pre-training, and are proportional to the parameter quantity and pre-training data quantity. Therefore, by increasing the model parameter quantity and pre-training data quantity in the pre-training stage, the knowledge and ability of the model can be improved, thereby fundamentally alleviating the "hallucination" problem. However, this solution requires a huge training cost in the early stage, and the effect cannot be predicted in advance. SUMMARY

[0004] In order to overcome the shortcomings of the prior art, the present application provides a reply method, device and system based on a generative language model, to solve the problem that the existing methods for alleviating the "hallucination" problem of generative language models either cannot effectively alleviate the problem or have high training costs and uncertain training effects.

[0005] The technical scheme adopted by the present application to solve its technical problems is:

[0006] In a first aspect, a reply method based on a generative language model is provided, the generative language model including a plurality of, any two generative language models being independent of each other, and the method comprising:

[0007] determining a predicted candidate output token and a generation probability of the candidate output token of each generative language model based on a user input instruction text;

[0008] multiplying the generation probability of the candidate output token by a preset weight of the corresponding generative language model to obtain a target generation probability of the candidate output token;

[0009] taking the candidate output token with the highest target generation probability as a target output token;

[0010] obtaining a reply text based on the target output token.

[0011] Further, the determining the candidate output token of each generative language model based on the instruction text input by the user comprises:

[0012] obtaining the instruction text input by the user;

[0013] encoding the instruction text into an input sequence of tokens by using the tokenizer corresponding to each generative language model;

[0014] obtaining a plurality of predicted tokens by performing a forward propagation in the corresponding generative language model based on the input sequence;

[0015] taking the token with the maximum probability in the probability distribution of the plurality of predicted tokens as the candidate output token of the generative language model.

[0016] Further, the obtaining the reply text based on the target output token comprises:

[0017] decoding the target output token by using the tokenizer in the generative language model corresponding to the target output token to obtain a target string;

[0018] adopting the target output token of each generative language model obtained by encoding the target string by using each generative language model, and adding the target output token to the input sequence of the corresponding generative language model;

[0019] determining the next candidate output token and the generation probability of the next candidate output token of each generative language model based on the input sequence;

[0020] multiplying the generation probability of the next candidate output token by the preset weight of the corresponding generative language model to obtain the target generation probability of the next candidate output token;

[0021] taking the next candidate output token with the highest target generation probability as the next target output token;

[0022] if the currently generated target output token satisfies the preset stopping condition, taking the text corresponding to the target output token sequence when the stopping condition is satisfied as the reply text.

[0023] Further, if the currently generated target output token does not satisfy the preset stopping condition, repeating the step of obtaining the next target output token.

[0024] Further, the preset stop condition is that the generated target output token is a stop token of the corresponding generative language model.

[0025] Further, the preset stop condition is that the length of the target output token sequence reaches a maximum generation length.

[0026] Further, the method further comprises:

[0027] determining a category of the instruction text based on the instruction text input by the user;

[0028] determining a preset weight of each generative language model based on the category.

[0029] Further, each generative language model is loaded on a GPU, and any two generative language models are loaded on different GPUs.

[0030] In a second aspect, a reply device based on generative language models is provided, the generative language models include a plurality of generative language models, and any two generative language models are independent of each other, and the device comprises:

[0031] a candidate output token determination module configured to determine a predicted candidate output token of each generative language model and a generation probability of the candidate output token based on an instruction text input by a user;

[0032] a target generation probability calculation module configured to multiply the generation probability of the candidate output token by a preset weight of the corresponding generative language model to obtain a target generation probability of the candidate output token;

[0033] a target output token determination module configured to take the candidate output token with the highest target generation probability as a target output token;

[0034] a reply text acquisition module configured to obtain a reply text based on the target output token.

[0035] In a third aspect, a reply system based on generative language models is provided, and the system comprises:

[0036] at least one processor and at least one memory;

[0037] the memory stores executable instructions of the processor;

[0038] the processor is configured to execute the above method.

[0039] Advantages:

[0040] The technical scheme of the application provides a reply method, device and system based on a generative language model. After a user inputs an instruction text, a plurality of generative language models are used to respectively predict a next token, a token with the maximum generation probability is taken as a candidate output token, a product of the generation probability of each candidate output token and a preset weight of the corresponding generative language model is taken as a final target generation probability, and a candidate output token with the highest target generation probability is taken as a target output token. That is, the advantages of the plurality of generative language models are integrated to obtain a target output token, and the text corresponding to the target output token is taken as the output of all generative language models, so that each token in the final reply text is obtained by integrating the outputs of all generative language models. The application scheme does not need to further train each generative language model, greatly reduces the cost, and even if the candidate output token of a certain generative language model is incorrect, it can be eliminated in combination with the candidate output tokens of other generative language models, greatly increasing the accuracy of the final reply. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0042] Figure 1 is a reply method flowchart based on a generative language model provided by the embodiments of the present application;

[0043] Figure 2 is a specific reply method flowchart based on a generative language model provided by the embodiments of the present application;

[0044] Figure 3 is a reply device structure schematic diagram based on a generative language model provided by the embodiments of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical scheme and advantages of the present application more clear, the technical scheme of the present application will be described in detail below in combination with the drawings and embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0046] First of all, it needs to be pointed out that in large language models, tokens are the basic building blocks of text processing. They represent a discrete element in the text, which can be a word, a character, a subword or a character. The role of token is to split the text into the smallest processable unit for subsequent text analysis and application.

[0047] In large language models, tokens have the following characteristics and roles:

[0048] Model input and output: the input and output of large language models are processed in token units. Whether it is input text or generated text, it is represented and analyzed by token.

[0049] In addition, the way a generative language model generates a reply is as follows: for the text input by the user, it generates one token based on the tokenizer (in Chinese, it is called a tokenizer, which is based on a preset word table to divide sentences into small word blocks (tokens)). Then, based on the input token sequence, one or more output tokens are predicted, and the token with the highest probability is selected as the final input token. Then, based on the input token sequence and the generated output token, the next token is predicted until the final stop. Then, the output token sequence at the stop of the tokenizer is converted into text.

[0050] However, the current single generative language model has the problem of inaccurate output. The reason is that either the generative language model has fewer parameters, in which case no matter how the parameters are trained or adjusted, the output is incorrect. Or the generative language model is trained less, but to achieve correct output, the training cost is too high.

[0051] Based on the above problems, the present application provides an output method based on multiple generative language models, the generative language models include multiple, any two generative language models are independent of each other, that is, each generative language model is different. Referring to Figure 1 , the method comprises:

[0052] S11: based on the instruction text input by the user, determining the predicted candidate output token of each generative language model and the generation probability of the candidate output token;

[0053] Specifically, instruction text input by a user is acquired; the instruction text is encoded into an input sequence of tokens by using a tokenizer corresponding to each generative language model; one forward propagation is performed in the corresponding generative language model based on the input sequence, to obtain a plurality of predicted tokens; and the token with the highest probability in the probability distribution of the plurality of predicted tokens is taken as a candidate output token of the generative language model.

[0054] S12: The generation probability of the candidate output token is multiplied by a preset weight of the corresponding generative language model, to obtain a target generation probability of the candidate output token.

[0055] S13: The candidate output token with the highest target generation probability is taken as a target output token.

[0056] S14: A reply text is obtained based on the target output token.

[0057] Specifically, the tokenizer in the generative language model corresponding to the target output token is used to decode the target output token to obtain a target string; the target output token of each generative language model is obtained by encoding the target string by using each generative language model, and the target output token is added to the input sequence of the corresponding generative language model; because the target output token obtained at this time is only a token recognizable by one generative language model, and due to different vocabularies, the target output token may not be recognized by other generative language models, the target output token needs to be decoded into text, and then encoded into a target output token recognizable by each generative language model according to the vocabulary of each generative language model.

[0058] Based on the input sequence, a next candidate output token predicted by each generative language model and a generation probability of the next candidate output token are determined; the generation probability of the next candidate output token is multiplied by a preset weight of the corresponding generative language model, to obtain a target generation probability of the next candidate output token; the next candidate output token with the highest target generation probability is taken as a next target output token; if the currently generated target output token satisfies a preset stopping condition, the text corresponding to the sequence of target output tokens at the time when the stopping condition is satisfied is taken as a reply text. If the currently generated target output token does not satisfy the preset stopping condition, the step of obtaining the next target output token is repeated.

[0059] The preset stop condition is that the generated target output token is a stop token of the corresponding generative language model. Alternatively, the stop condition is that the length of the target output token sequence reaches the maximum generation length. It should be noted that after obtaining the next target output token, the next target output token is decoded into text, and then encoded by using the vocabulary of other generative language models, and then whether to stop is determined according to the encoded token.

[0060] The step of obtaining the next target output token is repeated, specifically, the newly obtained target output token is added to the current input sequence, then candidate output tokens and corresponding generation probabilities of each generative language model are obtained based on the current input sequence, and the next target output token is obtained based on the candidate output tokens and the corresponding generation probabilities.

[0061] As an optional implementation of an embodiment of the present application, each generative language model is loaded on a GPU, and GPUs loaded by any two generative language models are different. That is, each generative language model is loaded on a separate GPU, so that data processing can be performed in parallel, greatly improving processing speed.

[0062] In one embodiment, the preset weight of each generative language model in the present application is set to a fixed value according to the user's own needs. However, since each generative language model has different focuses, its accuracy in responding to different types of problems is different. For example, a generative language model has an accuracy of 90% when responding to class A problems, but only 70% when responding to class B problems. Therefore, if a fixed value is used, it cannot adapt to all situations.

[0063] Therefore, in another embodiment of the present application, the classification of the instruction text input by the user is determined, and the preset weight of each generative language model is determined based on the classification. That is, each generative language model uses different preset weights when facing different classified instruction texts. In this way, the final accuracy can be improved when facing different classified texts.

[0064] To more clearly illustrate the above scheme, the present application provides a specific reply method based on a generative language model, which comprises the following steps:

[0065] 1. Model parallel loading: independently load multiple generative language models participating in combination on different GPUs to fully utilize the high-performance characteristics of multi-card parallel computing.

[0066] 2. Candidate output token generation: For the same instruction text input by the user, all generative language models in step 1 respectively and in parallel encode it into a sequence of tokens using the corresponding tokenizer, and perform a forward propagation based on the sequence, taking the maximum probability in the predicted token probability distribution as the candidate output token.

[0067] 3. Target output token determination: For the candidate output token of each generative language model in step 2, multiply its generation probability by the weight coefficient preset for each model, compare the adjusted generation probability, and take the maximum probability as the target output token of this round.

[0068] 4. Token alignment: Due to the different vocabularies of different generative language models, tokens cannot be shared, so the target output token in step 3 needs to be decoded into the actual string represented by its corresponding generative language model tokenizer, and then encoded by all generative language models in step 1 and added to the token sequence in step 2.

[0069] 5. Iterative generation: Repeat steps 2 to 4 until the generated target output token is the stop marker (such as EOS, etc.) of its corresponding generative language model, or the maximum generation length is reached.

[0070] 6. Output result: At this time, decoding the token sequence generated by any generative language model will correspond to the same text, and outputting this text as the final output result of the user's output instruction in step S22.

[0071] DETAILED DESCRIPTIONThe steps are as shown in Figure 2

[0072] 1. Initialize each of the multiple generative language models participating in the combination. In order to fully utilize the parallel characteristics, each generative language model should be loaded onto a different GPU. At the same time, set the weight coefficient w for each generative language model in advance, with a default value of 1.0.

[0073] 2. Input the user's input instruction text I into each model. According to the general processing flow of generative large language models, for each generative language model m, first encode the input instruction text I into a token sequence S m .

[0074] 3. From the first token generation round, for each generative language model m, based on the current token sequence S m , use the greedy method to generate the candidate token T m, and its corresponding probability is denoted as P m ;

[0075] 4. For each generative language model m, multiply the probability P m by the pre-defined weight coefficient w m of the generative language model m to obtain

[0076] 5. Compare the adjusted probabilities of all models Take the token corresponding to the maximum value as the preferred token of this round, denoted as T.

[0077] 6. Decode the preferred token T through the tokenizer of model m into the actual string s it represents;

[0078] 7. Encode the string s again by all models, and add the encoded token to the respective sequence Sm;

[0079] 8. Repeat steps 3-7 until the generated token T of this round is the stop token (such as EOS, etc.) of its source model m, or the maximum generation length is reached.

[0080] 9. Take the token sequence S of any model, decode it as the final output text O of the user instruction text I.

[0081] The reply method based on the generative language model provided in the application fully utilizes the different knowledge reserves and capabilities of multiple generative language models, and combines multiple models through probability weighting in the token generation stage, thereby mutually making up for the respective limitations and weaknesses of individual generative language models and alleviating the "hallucination" problem commonly seen when only using a single parameter small generative language model. Compared with the prior art, the application has the following advantages:

[0082] 1. Any number of generative language models can be supported to participate in combination, so the combination scale can be flexibly adjusted according to resources and needs;

[0083] 2. The generative language models participating in the combination can be any parameter scale and any base Transformers architecture causal language model, such as LLaMA, Bloom, ChatGLM, etc., and the greater the difference between the models, the better;

[0084] 3. The generative language models participating in the combination do not require additional training steps, thereby avoiding huge pre-training resources and time investment;

[0085] 4. The method adopts a probability weighting combination method, so the priority of different models can be flexibly adjusted through self-defined weight distribution.

[0086] 5. The combined generative language models can be calculated in parallel, and the combined system generates faster than a single generative language model with the same number of parameters.

[0087] Based on the same inventive concept, the present application provides a reply device based on a generative language model, wherein the generative language model includes multiple, any two generative language models are independent of each other, wherein each generative language model is loaded on a GPU, and any two generative language models are loaded on different GPUs. Figure 3 As shown, the device includes:

[0088] The candidate output token determination module 31 is used to determine the predicted candidate output tokens and the generation probability of the candidate output tokens for each generative language model based on the instruction text input by the user. Specifically, the instruction text input by the user is obtained; the instruction text is encoded into an input sequence of tokens using the tokenizer corresponding to each generative language model; a forward propagation is performed in the corresponding generative language model based on the input sequence to obtain multiple predicted tokens; and the one with the highest probability among the multiple predicted tokens is used as the candidate output token of the generative language model.

[0089] The target generation probability calculation module 32 is used to multiply the generation probability of the candidate output token by the preset weight of the corresponding generative language model to obtain the target generation probability of the candidate output token.

[0090] In one embodiment, the target generation probability calculation module 32 determines the classification of the instruction text based on the instruction text input by the user; and determines the preset weight of each generative language model based on the classification.

[0091] The target output token determination module 33 is configured to select the candidate output token with the highest target generation probability as the target output token.

[0092] The reply text acquisition module 34 is configured to obtain the reply text based on the target output token.

[0093] Specifically, the reply text obtaining module 34 decodes the target output token in the tokenizer of the generative language model corresponding to the target output token to obtain a target string; encodes the target string in the target output token of each generative language model to obtain the target output token of each generative language model, and adds the target output token to the input sequence of the corresponding generative language model; determines the predicted next candidate output token and the generation probability of the next candidate output token of each generative language model based on the input sequence; multiplies the generation probability of the next candidate output token by the preset weight of the corresponding generative language model to obtain the target generation probability of the next candidate output token; takes the next candidate output token with the highest target generation probability as the next target output token; if the currently generated target output token satisfies the preset stopping condition, takes the text corresponding to the target output token sequence when the stopping condition is satisfied as the reply text. If the currently generated target output token does not satisfy the preset stopping condition, repeat the step of obtaining the next target output token.

[0094] The preset stopping condition is that the generated target output token is a stop token of the corresponding generative language model, or the length of the target output token sequence reaches the maximum generation length.

[0095] The reply device based on the generative language model provided in the embodiments of the present application, after the user inputs the instruction text, predicts the next token of the instruction text by using multiple generative language models, takes the token with the maximum generation probability as the candidate output token, then takes the product of the generation probability of each candidate output token and the preset weight of the corresponding generative language model as the final target generation probability, and takes the candidate output token with the highest target generation probability as the target output token. That is, the advantages of multiple generative language models are combined to obtain a target output token, and the text corresponding to the target output token is taken as the output of all generative language models, so that each token in the final reply text is obtained by combining the outputs of all generative language models. The present application does not need to further train each generative language model, greatly reducing the cost, and even if the candidate output token of a generative language model is incorrect, it can be eliminated by combining the candidate output tokens of other generative language models, greatly increasing the accuracy of the final reply.

[0096] Based on the same inventive concept, the present application provides a reply system based on a generative language model, comprising:

[0097] at least one processor and at least one memory;

[0098] The memory stores executable instructions of the processor;

[0099] The processor is configured to perform the reply method based on the generative language model provided by the above-mentioned embodiments.

[0100] The reply system based on the generative language model provided by the embodiments of the present application stores the executable instructions of the processor through the memory. When the executable instructions are executed, the processor can predict the next token of the user input instruction text by using multiple generative language models, take the token with the maximum generation probability as the candidate output token, then take the product of the generation probability of each candidate output token and the preset weight of the corresponding generative language model as the final target generation probability, and take the candidate output token with the highest target generation probability as the target output token. That is, the advantages of multiple generative language models are integrated to obtain a target output token, and the text corresponding to the target output token is taken as the output of all generative language models, so that each token in the final reply text is obtained by integrating the outputs of all generative language models. The present application scheme does not need to further train each generative language model, greatly reducing the cost, and even if the candidate output token of a certain generative language model is incorrect, it can be eliminated in combination with the candidate output token of other generative language models, greatly increasing the accuracy of the final reply.

[0101] It can be understood that the same or similar parts in the above-mentioned embodiments can be mutually referred to, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0102] It should be noted that in the description of the present application, the terms "first", "second" and the like are only for the purpose of description and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "multiple" is at least two.

[0103] Any process or method descriptions in flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for performing specific logic functions or steps in the process, and the various embodiments of the present application include additional implementations in which the order of steps can differ from those shown or discussed, including a process performed substantially concurrently or in reverse order, depending upon the functionality involved, as will be understood by those having ordinary skill in the art of the present application.

[0104] It should be understood that each of the elements of the present application can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and in another embodiment, any of the following technologies known in the art or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0105] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment methods can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, and when executed, include one or a combination of steps of the method embodiments.

[0106] In addition, each functional unit in each embodiment of the present application can be integrated into one processing module, or each unit can be physically present separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of a software function module. The integrated module, if realized in the form of a software function module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0107] The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0108] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0109] Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A reply method based on a generative language model, characterized in that: The generative language models include a plurality of generative language models, and any two generative language models are independent of each other. The method includes: Determining, based on the instruction text input by the user, a candidate output token predicted by each generative language model and a generation probability of the candidate output token; Multiplying the generation probability of the candidate output token by the preset weight of the corresponding generative language model to obtain the target generation probability of the candidate output token; The candidate output token with the highest target generation probability is used as the target output token; The reply text is obtained based on the target output token.

2. The method according to claim 1, wherein: The step of determining a candidate output token for each generative language model based on the instruction text input by the user includes: Get the command text entered by the user; Encode the instruction text into an input sequence of tokens using the tokenizer corresponding to each generative language model; Perform a forward propagation in the corresponding generative language model based on the input sequence to obtain multiple predicted tokens; The token with the highest probability among the multiple predicted token probability distributions is used as the candidate output token of the generative language model.

3. The method according to claim 2, wherein: The step of obtaining a reply text based on the target output token includes: Decoding the target output token using a tokenizer in a generative language model corresponding to the target output token to obtain a target string; respectively using each generative language model to encode the target character string, and adding the target output token to the input sequence of the corresponding generative language model; Determining, based on the input sequence, a next candidate output token predicted by each generative language model and a generation probability of the next candidate output token; Multiplying the generation probability of the next candidate output token by the preset weight of the corresponding generative language model to obtain the target generation probability of the next candidate output token; The next candidate output token with the highest target generation probability is used as the next target output token; If the currently generated target output token meets the preset stop condition, the text corresponding to the target output token sequence when the stop condition is met will be used as the reply text.

4. The method according to claim 3, wherein: If the currently generated target output token does not meet the preset stop condition, the steps to obtain the next target output token are repeated.

5. The method according to claim 3, wherein: The preset stopping condition is: the generated target output token is the stop mark of the corresponding generative language model.

6. The method according to claim 3, wherein: The preset stopping condition is: the target output token sequence length reaches the maximum generation length.

7. The method according to claim 1, characterized in that Also includes: Determining a classification of the instruction text based on the instruction text input by the user; A preset weight for each generative language model is determined based on the classification.

8. The method according to claim 1, wherein: Each generative language model is loaded on a GPU, and any two generative language models are loaded on different GPUs.

9. A reply device based on a generative language model, characterized in that: The generative language model includes a plurality of generative language models, and any two generative language models are independent of each other. The device includes: A candidate output token determination module is used to determine the candidate output token predicted by each generative language model and the generation probability of the candidate output token based on the instruction text input by the user; A target generation probability calculation module is used to multiply the generation probability of the candidate output token by the preset weight of the corresponding generative language model to obtain the target generation probability of the candidate output token; A target output token determination module is used to select the candidate output token with the highest target generation probability as the target output token; The reply text acquisition module is used to obtain the reply text based on the target output token.

10. A response system based on a generative language model, characterized in that: include: at least one processor and at least one memory; The memory stores executable instructions of the processor; The processor is configured to execute the method according to any one of claims 1 to 8.

Images