Large language model text generation method and device, equipment and storage medium

Abstract

This invention relates to the field of deep learning technology and discloses a method, apparatus, computer device, and storage medium for generating text using a large language model. In the process of generating response text based on question text using a large language model, each time the large language model generates a first response word, the target vocabulary range for the next response word is determined based on the first response word. Then, after the large language model generates the vocabulary probability distribution for the next response word, the second response word is determined from the target vocabulary range based on the vocabulary probability distribution. It is evident that this invention determines the vocabulary range for the next response word using the previous response word, and then constrains the generation of the next response word based on the vocabulary range. This avoids inconsistencies in format between the next and previous response words. Compared to existing technologies, this allows for the constraint of the large language model to output high-quality generated text even when the large language model has poor convergence.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a method, apparatus, device, and storage medium for generating text from a large language model. Background Technology

[0002] Large Language Model (LLM) is a natural language processing model based on deep learning technology. In question answering tasks, LLM can predict the next word based on existing words, and then predict the next word based on newly generated words and existing words, and so on, until a complete text is generated.

[0003] However, when the large language model is limited by the quality of the training samples, resulting in poor model convergence, the text generation ability of the large language model will be greatly affected. A common situation is that it generates words that are not in a specific format. For example, "logic" appears under the first-level discipline "mathematics", but "logic" is a second-level discipline of "philosophy". In this case, "logic" is a word that is not in the feature format compared to "mathematics".

[0004] Therefore, there is an urgent need for a method that can still output high-quality generated text even when the large language model has poor convergence. Summary of the Invention

[0005] This invention provides a method, apparatus, computer device, and storage medium for generating text from a large language model, in order to solve the problem that it is difficult to output high-quality generated text when the large language model has poor convergence.

[0006] A method for text generation from a large language model, the method comprising: Obtain the problem text; The question text is input into a large language model so that when the large language model generates a first response word based on the question text, the target vocabulary range of the next response word is determined according to the first response word. After the large language model generates the probability distribution of the next response word, the second response word is determined from the target vocabulary range according to the probability distribution of the vocabulary. The second response word is the next response word of the first response word. If the second response word is not the end response word, the second response word is used as the new first response word, and the process returns to the target step: determine the target vocabulary range for the next response word based on the first response word; When the second response word is the closing response word, the response text corresponding to the question text is obtained.

[0007] Optionally, in the above method, determining the second response word from the target vocabulary range based on the vocabulary probability distribution, wherein the second response word is the next response word after the first response word, includes: Based on the target words in the target vocabulary range, the probability distribution of the vocabulary is cleaned to retain the word probabilities corresponding to the target words in the probability distribution of the vocabulary. The target word with the highest probability is selected as the second response word.

[0008] Optionally, in the above method, determining the target vocabulary range for the next response term based on the first response term includes: The first response word is input into the state machine to determine the target vocabulary range corresponding to the next response word according to the preset state transition rules in the state machine.

[0009] Optionally, in the above method, determining the target vocabulary range for the next response term based on the first response term includes: The first response word is input into the state machine to determine the range of the blocked word list corresponding to the next response word according to the preset state transition rules in the state machine; The target vocabulary range is obtained by removing the blocked words from the original vocabulary range.

[0010] Optionally, the state transition rule in the above method is obtained in the following way: The status text was retrieved. The state text is input into the rule converter to obtain the state diagram corresponding to the state text; The state diagram is input into the state machine to obtain the state transition rules.

[0011] Optionally, in the above method, the step of inputting the state text into a rule converter to obtain the rule state diagram corresponding to the state text includes: After the status text is input into the rule converter, the rule segmenter in the rule converter performs word segmentation on the status text according to the preset word segmentation rules to obtain multiple status words. Each state word is used to construct a state, thereby obtaining the vocabulary range corresponding to the state word; A state graph is generated based on the vocabulary range and the corresponding state words.

[0012] Optionally, the method described above can determine whether the second response word is a closing response word by means of the following: Determine whether the second response word is an end response word within the preset end word list; If the second response word is not an end response word in the range of the end word list, then the second response word is determined to be the end response word.

[0013] A large language model text generation device, the device comprising: The problem text acquisition module is used to acquire the problem text; The vocabulary range determination module is used to input the question text into the large language model, so that when the large language model generates a first response word based on the question text, it determines the target vocabulary range of the next response word based on the first response word. The response word determination module is used to determine a second response word from the target vocabulary range based on the vocabulary probability distribution generated by the large language model after the large language model generates the next response word; the second response word is the next response word of the first response word. The text generation loop module is used to take the second response word as the first response word when the second response word is not the end response word, and return to the execution target step: determine the target vocabulary range of the next response word based on the first response word; The response text generation module is used to obtain the response text when the second response word is the ending response word.

[0014] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the large language model text generation method as described in any of the preceding claims.

[0015] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the large language model text generation method as described in any of the preceding claims.

[0016] The aforementioned large language model text generation method, apparatus, computer equipment, and storage medium, in the process of generating response text based on question text using a large language model, determine the target vocabulary range for the next response word based on the first response word when the large language model generates a first response word. Then, after the large language model generates the vocabulary probability distribution for the next response word, it determines the second response word from the target vocabulary range based on the vocabulary probability distribution. If the second response word is not the ending response word, it is used as the first response word, and the target vocabulary range for the next response word is determined based on the first response word. When the second response word is the ending response word, the response text is obtained. It can be seen that this invention determines the vocabulary range for the next response word based on the previous response word, and then constrains the generation of the next response word based on the vocabulary range. This avoids inconsistencies in format between the next and previous response words. Compared with existing technologies, this can constrain the large language model to output high-quality generated text even when the large language model has poor convergence. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of an implementation of a large language model text generation method disclosed in an embodiment of the present invention; Figure 2 This is a partial implementation flowchart of a large language model text generation method disclosed in an embodiment of the present invention; Figure 3 This is another implementation flowchart of a large language model text generation method disclosed in an embodiment of the present invention; Figure 4 This is a partial implementation flowchart of a large language model text generation method disclosed in an embodiment of the present invention; Figure 5 This is a partial implementation flowchart of a large language model text generation method disclosed in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a state diagram disclosed in an embodiment of the present invention; Figure 7 This is a partial implementation flowchart of a large language model text generation method disclosed in an embodiment of the present invention; Figure 8 This is a partial implementation flowchart of a large language model text generation method disclosed in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a large language model text generation device disclosed in an embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of a computer device disclosed in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0023] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0024] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0025] This invention discloses a method, apparatus, computer device, and storage medium for generating text using a large language model. In the process of generating response text based on question text using a large language model, each time the large language model generates a first response word, the target vocabulary range for the next response word is determined based on the first response word. Then, after the large language model generates the vocabulary probability distribution for the next response word, a second response word is determined from the target vocabulary range based on the vocabulary probability distribution. If the second response word is not the ending response word, it is used as the first response word, and the target vocabulary range for the next response word is determined based on the first response word. When the second response word is the ending response word, the response text is obtained. It can be seen that this invention determines the vocabulary range for the next response word based on the previous response word, and then constrains the generation of the next response word based on the vocabulary range. This avoids inconsistencies in format between the next and previous response words. Compared with existing technologies, this can constrain the large language model to output high-quality generated text even when the large language model has poor convergence. Specific embodiments are described below.

[0026] like Figure 1 The diagram shows a flowchart of a large language model text generation method disclosed in this embodiment of the invention. This method is applicable to electronic devices with data processing capabilities, such as mobile phones, tablets, laptops, personal computers, or servers. The method constrains the text generation process of the large language model through a state machine, ensuring that the large language model always generates text in the same format, thereby improving the text generation quality of the large language model. Based on this, the method in this embodiment includes the following steps: S101: The problem text has been obtained.

[0027] The question text is the specific content of the question the user wants to ask, which may include, but is not limited to, questions from various disciplines or technical fields. The language of the question text may include, but is not limited to, Chinese, English, and other languages.

[0028] In practice, users can enter the question they want to ask on the search page of the large language model output on the electronic device. When the user clicks the search function button on the search page, the text of the question the user wants to ask is obtained.

[0029] S102: Input the question text into the large language model so that when the large language model generates the first response word based on the question text each time, it determines the target vocabulary range of the next response word based on the first response word.

[0030] In this specific implementation, a target vocabulary range corresponding to each possible first response word can be pre-set. Once the first response word is determined, the corresponding target vocabulary range can be matched based on it. This target vocabulary range is used to constrain the generation of the next response word. This embodiment includes, but is not limited to, setting the state mapping relationship between the first response word and the target vocabulary range using a state machine.

[0031] S103: After generating the vocabulary probability distribution of the next response word in the large language model, determine the second response word from the target vocabulary range based on the vocabulary probability distribution.

[0032] The second response is the response word following the first response.

[0033] As is understandable, the target vocabulary range refers to the range of words that can be generated by the next response word. In other words, the second response word must be a word within the target vocabulary range, and the second response word is determined from the target vocabulary range. The vocabulary probability distribution refers to all possible words for the next response word predicted by the large language model based on the first response word and all words preceding the first response word, as well as the distribution of the corresponding word probabilities.

[0034] In a specific implementation, this embodiment can determine the second response word from the target vocabulary range based on the vocabulary probability distribution through the following steps, such as... Figure 2 As shown: S201: Clean the probability distribution of the word list based on the target words within the target word list range, so as to retain the word probabilities corresponding to the target words in the word list probability distribution.

[0035] S202: Select the target word with the highest probability as the second response word.

[0036] It should be understood that the second response word can only be a target word within the target vocabulary range. Therefore, the probability distribution of the vocabulary can be cleaned according to the target vocabulary range to remove invalid words, that is, to remove words other than target words in the vocabulary probability distribution, in order to retain the word probability corresponding to each target word in the vocabulary probability distribution. Then, the target words are sorted according to the size of the word probability, and the target word with the highest word probability is selected as the second response text. In this way, the second response word can be constrained to be selected only from the target vocabulary range to avoid inconsistencies in the format of the second response word and the first response word.

[0037] S104: If the second response word is not the closing response word, use the second response word as the new first response word and return to the execution target step: determine the target vocabulary range of the next response word based on the first response word.

[0038] S105: When the second response word is the closing response word, obtain the response text corresponding to the question text.

[0039] If the second response word is not the ending response word, it means that the large language model has not yet generated a complete text. Therefore, the large language model needs to continue generating text. Thus, the second response word is used as the new first response word, and the generation of the second response word continues based on the new first response word. This process is repeated until the generated second response word is the ending response word, at which point the response text corresponding to the question text is obtained. At this point, the large language model outputs the generated response text to the user. Thus, in this embodiment, the response text corresponding to the question text can be obtained when the second response word is the ending response word.

[0040] In summary, this embodiment discloses a method for generating text using a large language model. During the process of generating response text based on question text using a large language model, each time the large language model generates a first response word, the target vocabulary range for the next response word is determined based on the first response word. Then, after the large language model generates the vocabulary probability distribution for the next response word, a second response word is determined from the target vocabulary range based on the vocabulary probability distribution. If the second response word is not the ending response word, it is used as the first response word, and the target vocabulary range for the next response word is determined based on the first response word. When the second response word is the ending response word, the response text is obtained. It can be seen that this invention determines the vocabulary range for the next response word based on the previous response word, and then constrains the generation of the next response word based on the vocabulary range. This avoids inconsistencies in format between the next and previous response words. Compared with existing technologies, this method can constrain the large language model to output high-quality generated text even when the large language model has poor convergence.

[0041] based on Figure 1In its specific implementation, step S102 in this embodiment can be achieved through the following steps, such as... Figure 3 As shown: S106: Input the first response word into the state machine to determine the target vocabulary range corresponding to the next response word according to the preset state transition rules in the state machine.

[0042] In this specific implementation, the state machine in this embodiment is pre-input with state transition rules. After the first response word is input into the state machine, the state machine determines the target vocabulary range corresponding to the next response word based on the state transition rules. The target vocabulary range varies depending on the first response word. Therefore, after obtaining the first response word each time, it is necessary to input the first response word into the state machine to obtain the target vocabulary range corresponding to the next response word in real time.

[0043] For example, taking the search for "Where is Huaxian?" as an example, "Huaxian" is the first response term. Based on the first response term, the target word range corresponding to the determined next response term will only contain words such as "Anyang, Northern Henan, Henan, China, Asia", but will not contain words such as "Guangdong, Shenzhen, Nanjing". Therefore, the target word range of the next response term can be determined based on the first response term, and then the generation of the next response term can be constrained based on the target word range.

[0044] In summary, this embodiment establishes state transition rules through a state machine, and constrains the text generation process of the large language model by generating a target vocabulary range based on the state transition rules. This eliminates the need for complex calculations or filtering, which helps improve the text generation efficiency of the large language model.

[0045] In one embodiment, step S102 in this embodiment can also be implemented by the following steps, such as... Figure 4 As shown: S401: Input the first response word into the state machine to determine the range of the block list corresponding to the next response word according to the preset state transition rules in the state machine.

[0046] S402: Remove the blocked words from the blocked word list from the original word list range to obtain the target word list range.

[0047] It should be understood that the blocked words in the blocked word list are words that cannot appear in the next response word. After the first response word is input into the state machine, the state machine determines the blocked word list range corresponding to the next response word according to the state transition rules. The corresponding blocked word list range varies depending on the first response word. Therefore, after obtaining the first response word each time, it is necessary to input the first response word into the state machine to obtain the blocked word list range corresponding to the next response word in real time. The blocked word list range is set according to actual needs. Furthermore, the original word list range value in this embodiment is at least a word list range containing all commonly used words. After removing the blocked words from the blocked word list range from the original word list range, the target word list range is obtained. It should be noted that in this embodiment, after obtaining the blocked word list range corresponding to the next response word of the first response word, the blocked words in the blocked word list range can be directly removed from the word list probability distribution of the next response word.

[0048] For example, if a user does not want "Jackson" to appear after "Michael", then when "Michael" is the first response word, the list of blocked words for the next response word should at least include "Jackson".

[0049] based on Figure 1 In the specific implementation, the state transition rules in this embodiment are obtained in the following way, such as Figure 5 As shown: S501: Status text obtained.

[0050] S502: Input the status text into the rule converter to obtain the status diagram corresponding to the status text.

[0051] S503: Input the state diagram into the state machine to obtain the state transition rules.

[0052] In this embodiment, the rule converter is used to convert the state text input by the user to obtain the state diagram corresponding to the state text, and uses the state diagram as the input of the state machine. After receiving the state diagram, the state machine generates the corresponding state transition rules.

[0053] In a specific implementation, different state texts can be input into the rule converter to generate different state diagrams. After inputting the different state diagrams into the state machine, different state transition rules are generated in the state machine.

[0054] Furthermore, step S502 in this embodiment can be implemented through the following steps, specifically as follows: Figure 6 As shown: S601: After the status text is input into the rule converter, the rule segmenter in the rule converter performs word segmentation processing on the status text according to the preset word segmentation rules to obtain multiple status words.

[0055] The segmentation rules of the rule-based word segmenter can be set according to the user's actual needs, thereby dividing the status text into independent status words.

[0056] In one embodiment, the status text entered by the user can be as follows: Var: {"src_name""[a]", "tgt_name":"[b]", "formula":"lambda x:x"} / / Source variable a, target variable b Main: {"type":"preset", "tag_name":"first-level discipline", "end_str":["|"]} {"type":"trie", "trie_dict": {"|": {"str":"[sep]", "next": null}}} {"type":"str", "end_str":["|"]} {"type":"trie", "trie_dict": {"|": {"str":"[sep]", "next": null}}} {"type":"str", "end_str":["|"], "var":"[a]"} {"type":"trie", "trie_dict": {"|": {"str":"[sep]", "next": null}}} {"type":"str", "end_str": null, "var":"[b]"} StrSub: {"pattern":"Michael", "target_str"Jackson", "type":"ban"} The system consists of seven main process states and one prohibited text string state, arranged from top to bottom. After the rule segmenter segments the state text, it produces a string in the following format: "[First-level subject] [sep][str1] [sep][str2] [sep][str3]". The texts in str2 and str3 are the same. The last prohibited text string state means that the text "Jackson" cannot be generated after the text "Michael".

[0057] S602: Perform state construction for each state word to obtain the vocabulary range corresponding to the state word.

[0058] In this specific implementation, it can be determined whether each state word has a preset vocabulary range in the state repository. If the preset vocabulary range exists in the state repository, the corresponding vocabulary range is retrieved from the state repository based on the state word. If the preset vocabulary range does not exist in the state repository, a corresponding vocabulary range is constructed for the state word.

[0059] It should be noted that the above is only a part of the method for constructing the vocabulary range corresponding to the state words in this embodiment. This embodiment does not specifically limit the actual application method of constructing state words.

[0060] S603: Generate a state graph based on the vocabulary range and the corresponding state words.

[0061] Specifically, in this embodiment, after determining the vocabulary range and the corresponding state words, a selected state diagram drawing tool is used to draw a state diagram based on the vocabulary range and the corresponding state words. For example... Figure 7 The diagram shown is a structural illustration of a state diagram in this embodiment. The state diagram contains two types of nodes: restricted nodes (circular nodes in the diagram) and prefix tree nodes (square nodes in the diagram). Different nodes correspond to different vocabulary ranges and state words. For prefix tree nodes, the output content is either enumerable or fixed; for example, Henan Province only contains a few specific prefecture-level cities, while [sep] only contains fixed text. The output of restricted nodes is not enumerable and is often determined by adding constraints through rules, such as the string not containing specific languages ​​or needing to conform to specific rules.

[0062] In summary, this embodiment provides an intuitive way to visualize states and transitions between states through state diagrams, making complex system behaviors easier to understand and analyze. Furthermore, when the state transition rules in the state machine need to be updated, the state transition rules can be quickly updated by modifying the state diagram.

[0063] based on Figure 1 In the specific implementation, this embodiment can determine whether the second response word is a closing response word in the following way, such as Figure 8 As shown: S801: Determine whether the second response word is an end response word within the preset end word list.

[0064] S802: If the second response word is not a closing response word in the closing word list, then the second response word is determined to be a closing response word.

[0065] The terminology for the second response word is a pre-defined list of ending response words. Each time a new second response word is predicted, it is compared with these pre-defined words to determine if it matches the pre-defined list. If the second response word matches, the large language model has generated a complete dialogue, and it no longer needs to generate a new word. If the second response word does not match, the model has not generated a complete dialogue. In this case, the model continues to predict new second response words based on all previously predicted words until it outputs an ending response word. The final response text is composed of all the generated response words.

[0066] In this specific implementation, the ending response word in this embodiment includes, but is not limited to, one or more of the following as stop words: special markers, punctuation marks, and dates. Special markers may include things like <|im_end|> and <|endoftext|>, punctuation marks may include periods, exclamation marks, and question marks, and dates may include the newspaper date after generating the newspaper summary.

[0067] It should be noted that the above are only some manifestations of the closing response in this embodiment, and the specific manifestation of the closing response in this embodiment is not limited.

[0068] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0069] like Figure 9 The diagram shown is a structural schematic of a large language model text generation device disclosed in this invention. This device is suitable for electronic devices with data processing capabilities, such as mobile phones, tablets, laptops, personal computers, or servers. The method in this embodiment may specifically include the following steps: The problem text acquisition module 901 is used to acquire the problem text; The vocabulary range determination module 902 is used to input the question text into the large language model so that when the large language model generates a first response word based on the question text, it determines the target vocabulary range of the next response word based on the first response word. The response word determination module 903 is used to determine the second response word from the target vocabulary range based on the vocabulary probability distribution after the large language model generates the vocabulary probability distribution of the next response word. The second response word is the next response word of the first response word. The text generation loop module 904 is used to take the second response word as the first response word when the second response word is not the end response word, and return to the execution target step: determine the target vocabulary range of the next response word based on the first response word; The response text generation module 905 is used to obtain the response text when the second response word is the closing response word.

[0070] In summary, the large language model text generation device disclosed in this embodiment, during the process of generating response text based on question text by the large language model, determines the target vocabulary range of the next response word based on the first response word when the large language model generates a first response word, and then determines the second response word from the target vocabulary range based on the vocabulary probability distribution after the large language model generates the next response word. If the second response word is not the ending response word, the second response word is used as the first response word, and the target vocabulary range of the next response word is determined based on the first response word. When the second response word is the ending response word, the response text is obtained. It can be seen that this invention determines the vocabulary range of the next response word by using the previous response word, and then constrains the generation of the next response word based on the vocabulary range, thereby avoiding inconsistencies in the format of the next response word compared to the previous response word. Compared with the prior art, this can constrain the large language model to output high-quality generated text even when the large language model has poor convergence.

[0071] In one embodiment, the response word determination module 903 is used for: Based on the target words in the target vocabulary range, the probability distribution of the vocabulary is cleaned to retain the word probabilities corresponding to the target words in the vocabulary probability distribution; Choose the target word with the highest probability as the second response word.

[0072] In one embodiment, the vocabulary range determination module 902 is used for: The first response word is input into the state machine to determine the target vocabulary range corresponding to the next response word according to the preset state transition rules in the state machine.

[0073] In one embodiment, the vocabulary range determination module 902 is also used for: The first response word is input into the state machine to determine the range of the blocked word list corresponding to the next response word according to the preset state transition rules in the state machine. The target vocabulary range is obtained by removing the blocked words from the original vocabulary range.

[0074] In one embodiment, the state transition rule is obtained as follows: The status text was retrieved. Input the state text into the rule converter to obtain the state diagram corresponding to the state text; The state diagram is input into the state machine to obtain the state transition rules.

[0075] In one embodiment, the vocabulary range determination module 902 is also used for: After the status text is input into the rule converter, the rule segmenter in the rule converter segments the status text according to the preset segmentation rules to obtain multiple status words. Each state word is constructed separately to obtain the vocabulary range corresponding to the state word; A state graph is generated based on the vocabulary range and the corresponding state words.

[0076] In one embodiment, it is determined whether the second response word is a closing response word in the following manner: Determine whether the second response word is a closing response word within the preset closing word list; If the second response word is not a term in the terminology list, then the second response word is determined to be a terminology for ending responses.

[0077] For specific limitations regarding the large language model text generation device, please refer to the limitations regarding the large language model text generation method mentioned above, which will not be repeated here. Each module in the aforementioned large language model text generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0078] This application also discloses a computer device, which can be a server, and its internal structure diagram can be as follows. Figure 10 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a large language model text generation method.

[0079] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps: Obtain the problem text; The question text is input into a large language model so that when the large language model generates the first response word based on the question text, the target vocabulary range of the next response word is determined based on the first response word. After generating the vocabulary probability distribution of the next response word in the large language model, the second response word is determined from the target vocabulary range based on the vocabulary probability distribution. The second response word is the next response word of the first response word. If the second response is not the closing response, the second response is used as the new first response, and the process returns to the target step: determine the target vocabulary range for the next response based on the first response; When the second response word is the closing response word, the response text corresponding to the question text is obtained.

[0080] This embodiment discloses a computer device. When the computer program is executed by a processor, during the process of generating response text based on question text using a large language model, each time the large language model generates a first response word, it determines the target vocabulary range for the next response word based on the first response word. Then, after the large language model generates the vocabulary probability distribution for the next response word, it determines the second response word from the target vocabulary range based on the vocabulary probability distribution. If the second response word is not the ending response word, it is used as the first response word, and the target vocabulary range for the next response word is determined based on the first response word. When the second response word is the ending response word, the response text is obtained. It can be seen that this invention determines the vocabulary range for the next response word based on the previous response word, and then constrains the generation of the next response word based on the vocabulary range. This avoids inconsistencies in format between the next and previous response words. Compared with the prior art, this can constrain the large language model to output high-quality generated text even when the large language model has poor convergence.

[0081] In one embodiment, determining a second response word from a target vocabulary range based on a vocabulary probability distribution, wherein the second response word is the next response word after the first response word, includes: Based on the target words in the target vocabulary range, the probability distribution of the vocabulary is cleaned to retain the word probabilities corresponding to the target words in the vocabulary probability distribution; Choose the target word with the highest probability as the second response word.

[0082] In one embodiment, determining the target vocabulary range for the next response term based on the first response term includes: The first response word is input into the state machine to determine the target vocabulary range corresponding to the next response word according to the preset state transition rules in the state machine.

[0083] In one embodiment, determining the target vocabulary range for the next response term based on the first response term includes: The first response word is input into the state machine to determine the range of the blocked word list corresponding to the next response word according to the preset state transition rules in the state machine. The target vocabulary range is obtained by removing the blocked words from the original vocabulary range.

[0084] In one embodiment, the state transition rule is obtained as follows: The status text was retrieved. Input the state text into the rule converter to obtain the state diagram corresponding to the state text; The state diagram is input into the state machine to obtain the state transition rules.

[0085] In one embodiment, the state text is input into a rule converter to obtain a rule state diagram corresponding to the state text, including: After the status text is input into the rule converter, the rule segmenter in the rule converter segments the status text according to the preset segmentation rules to obtain multiple status words. Each state word is constructed separately to obtain the vocabulary range corresponding to the state word; A state graph is generated based on the vocabulary range and the corresponding state words.

[0086] In one embodiment, it is determined whether the second response word is a closing response word in the following manner: Determine whether the second response word is a closing response word within the preset closing word list; If the second response word is not a term in the terminology list, then the second response word is determined to be a terminology for ending responses.

[0087] This application also discloses a computer-readable storage medium that, when executed by a processor in a computer device, enables the computer device to perform various steps of any embodiment of a large language model text generation method disclosed in this invention. The computer-readable storage medium may be non-volatile or volatile.

[0088] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain the problem text; The question text is input into a large language model so that when the large language model generates the first response word based on the question text, the target vocabulary range of the next response word is determined based on the first response word. After generating the vocabulary probability distribution of the next response word in the large language model, the second response word is determined from the target vocabulary range based on the vocabulary probability distribution. The second response word is the next response word of the first response word. If the second response is not the closing response, the second response is used as the new first response, and the process returns to the target step: determine the target vocabulary range for the next response based on the first response; When the second response word is the closing response word, the response text corresponding to the question text is obtained.

[0089] This embodiment discloses a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, during the generation of response text based on question text using a large language model, each time the large language model generates a first response word, it determines the target vocabulary range for the next response word based on the first response word. Then, after the large language model generates the vocabulary probability distribution for the next response word, it determines the second response word from the target vocabulary range based on the vocabulary probability distribution. If the second response word is not the ending response word, it is used as the first response word, and the target vocabulary range for the next response word is determined based on the first response word. If the second response word is the ending response word, the response text is obtained. Therefore, this invention determines the vocabulary range for the next response word based on the previous response word, and then constrains the generation of the next response word based on the vocabulary range. This avoids inconsistencies in format between the next and previous response words. Compared to existing technologies, this allows for the constraint of the large language model to output high-quality generated text even when the large language model has poor convergence.

[0090] In one embodiment, determining a second response word from a target vocabulary range based on a vocabulary probability distribution, wherein the second response word is the next response word after the first response word, includes: Based on the target words in the target vocabulary range, the probability distribution of the vocabulary is cleaned to retain the word probabilities corresponding to the target words in the vocabulary probability distribution; Choose the target word with the highest probability as the second response word.

[0091] In one embodiment, determining the target vocabulary range for the next response term based on the first response term includes: The first response word is input into the state machine to determine the target vocabulary range corresponding to the next response word according to the preset state transition rules in the state machine.

[0092] In one embodiment, determining the target vocabulary range for the next response term based on the first response term includes: The first response word is input into the state machine to determine the range of the blocked word list corresponding to the next response word according to the preset state transition rules in the state machine. The target vocabulary range is obtained by removing the blocked words from the original vocabulary range.

[0093] In one embodiment, the state transition rule is obtained as follows: The status text was retrieved. Input the state text into the rule converter to obtain the state diagram corresponding to the state text; The state diagram is input into the state machine to obtain the state transition rules.

[0094] In one embodiment, the state text is input into a rule converter to obtain a rule state diagram corresponding to the state text, including: After the status text is input into the rule converter, the rule segmenter in the rule converter segments the status text according to the preset segmentation rules to obtain multiple status words. Each state word is constructed separately to obtain the vocabulary range corresponding to the state word; A state graph is generated based on the vocabulary range and the corresponding state words.

[0095] In one embodiment, it is determined whether the second response word is a closing response word in the following manner: Determine whether the second response word is a closing response word within the preset closing word list; If the second response word is not a term in the terminology list, then the second response word is determined to be a terminology for ending responses.

[0096] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0097] Any references to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory.

[0098] By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0100] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for generating text from a large language model, characterized in that, The method includes: Obtain the problem text; The question text is input into a large language model so that when the large language model generates a first response word based on the question text, the target vocabulary range of the next response word is determined according to the first response word. After the large language model generates the probability distribution of the next response word, the second response word is determined from the target vocabulary range according to the probability distribution of the vocabulary. The second response word is the next response word of the first response word. If the second response word is not the end response word, the second response word is used as the new first response word, and the process returns to the target step: determine the target vocabulary range for the next response word based on the first response word; When the second response word is the closing response word, the response text corresponding to the question text is obtained.

2. The large language model text generation method as described in claim 1, characterized in that, The step of determining the second response word from the target vocabulary range based on the vocabulary probability distribution includes: Based on the target words in the target vocabulary range, the probability distribution of the vocabulary is cleaned to retain the word probabilities corresponding to the target words in the probability distribution of the vocabulary. The target word with the highest probability is selected as the second response word.

3. The large language model text generation method as described in claim 1, characterized in that, The step of determining the target vocabulary range for the next response term based on the first response term includes: The first response word is input into the state machine to determine the target vocabulary range corresponding to the next response word according to the preset state transition rules in the state machine.

4. The large language model text generation method as described in claim 1, characterized in that, The step of determining the target vocabulary range for the next response term based on the first response term includes: The first response word is input into the state machine to determine the range of the blocked word list corresponding to the next response word according to the preset state transition rules in the state machine; The target vocabulary range is obtained by removing the blocked words from the original vocabulary range.

5. The large language model text generation method as described in claim 3, characterized in that, The state transition rules are obtained in the following way: The status text was retrieved. The state text is input into the rule converter to obtain the state diagram corresponding to the state text; The state diagram is input into the state machine to obtain the state transition rules.

6. The large language model text generation method as described in claim 5, characterized in that, The step of inputting the state text into the rule converter to obtain the rule state diagram corresponding to the state text includes: After the status text is input into the rule converter, the rule segmenter in the rule converter performs word segmentation on the status text according to the preset word segmentation rules to obtain multiple status words. Each state word is used to construct a state, thereby obtaining the vocabulary range corresponding to the state word; A state graph is generated based on the vocabulary range and the corresponding state words.

7. The large language model text generation method as described in claim 1, characterized in that, Determine whether the second response word is a closing response word using the following method: Determine whether the second response word is an end response word within the preset end word list; If the second response word is not an end response word in the range of the end word list, then the second response word is determined to be the end response word.

8. A large language model text generation device, characterized in that, The device includes: The problem text acquisition module is used to acquire the problem text; The vocabulary range determination module is used to input the question text into the large language model, so that when the large language model generates a first response word based on the question text, it determines the target vocabulary range of the next response word based on the first response word. The response word determination module is used to determine a second response word from the target vocabulary range based on the vocabulary probability distribution generated by the large language model after the large language model generates the next response word; the second response word is the next response word of the first response word. The text generation loop module is used to take the second response word as the first response word when the second response word is not the end response word, and return to the execution target step: determine the target vocabulary range of the next response word based on the first response word; The response text generation module is used to obtain the response text when the second response word is the ending response word.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the large language model text generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the large language model text generation method as described in any one of claims 1 to 7.

Images