Text generation method and device based on pre-trained language model, equipment, medium

Abstract

The present disclosure provides a text generation method and device based on a pre-trained language model, equipment, and a medium, which can be applied to the technical field of text generation. The method comprises: inputting target question text into a pre-trained language model to generate a target path; generating an uncertainty value based on the conditional probability of each step in a logical step sequence; in response to determining that the uncertainty value is greater than a predetermined threshold, correcting at least one step in the logical step sequence using the pre-trained language model to obtain a corrected logical step sequence; and processing the target question text according to the corrected logical step sequence using the pre-trained language model to generate a target reply text, ensuring the correctness of each step in the logical step sequence, improving reasoning efficiency, reducing the consumption of computing resources, and improving the robustness and explainability of the target reply text.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of text generation, and more particularly to a text generation method and device based on a pre-trained language model, an apparatus, a medium. BACKGROUND

[0002] With the rapid development of inference ability of large language models, the application prospect of large language models in the fields of mathematical reasoning, complex task decision-making, scientific exploration, intelligent control, etc. is increasingly broad. In the application process of related large language models, when a large language model processes a reasoning task with a long reasoning chain, if a deviation occurs in the initial reasoning step, the subsequent reasoning is often layered on the basis of the error, causing the overall reasoning trajectory to deviate from the correct direction, and thus a correct reasoning result cannot be obtained. SUMMARY

[0003] In view of the above problems, the present disclosure provides a text generation method and device based on a pre-trained language model, an apparatus, and a medium.

[0004] According to a first aspect of the present disclosure, a text generation method based on a pre-trained language model is provided, comprising: inputting the above-mentioned target problem text into the pre-trained language model to generate a target path; wherein the target path indicates a logical step sequence and a conditional probability for obtaining an initial answer text from the target problem text, and the target path includes M steps, M being an integer greater than or equal to 1; based on the conditional probability of each step in the logical step sequence, a perplexity value is generated, the perplexity value representing the credibility of the (m+1)th step inferred from the mth step, m being an integer greater than or equal to 1 and less than or equal to M-1; in response to determining that the perplexity value is greater than a predetermined threshold, at least one step in the logical step sequence is corrected using the pre-trained language model to obtain a corrected logical step sequence; and the pre-trained language model is used to process the target problem text according to the corrected logical step sequence to generate a target answer text.

[0005] According to an embodiment of the present disclosure, the mth step is composed of N reasoning words, N being an integer greater than or equal to 1, and the conditional probability of the mth step is determined by the following method: for the first reasoning word in the N reasoning words, a first conditional probability is obtained based on a preset start token, the first conditional probability being the probability of outputting the first reasoning word; for the nth reasoning word in the N reasoning words, an nth conditional probability is obtained based on the first n-1 reasoning words to obtain N-1 conditional probabilities, n being an integer greater than 1 and less than or equal to N, the nth probability being the probability of outputting the nth reasoning word under the condition that the first n-1 reasoning words are determined; and the first probability to the Nth probability are used to generate the conditional probability of the mth step.

[0006] According to an embodiment of the present disclosure, the generating the perplexity value based on the conditional probability of each step in the sequence of logical steps includes: performing logarithmic operations on the first conditional probability and the N-1 conditional probabilities to obtain N inference word conditional probabilities; obtaining an average inference word conditional probability based on the N inference word conditional probabilities; and generating the perplexity value based on the average inference word conditional probability.

[0007] According to an embodiment of the present disclosure, the mth step includes mth inference thinking data, mth inference action data, and mth inference observation data, and the correcting at least one step in the sequence of logical steps by using the pre-trained language model to obtain a corrected sequence of logical steps includes: inputting the mth step into the pre-trained language model to obtain reflection data, the reflection data representing an evaluation and improvement of the mth step by the pre-trained language model; determining the mth inference thinking data based on the reflection data by using the pre-trained language model, the mth inference thinking data representing logical reasoning of the mth inference step; determining the mth inference action data based on the mth inference thinking data by using the pre-trained language model, the mth inference action data representing an execution action of the mth inference step; and obtaining the mth inference observation data based on the mth inference action data by using the pre-trained language model, the mth inference observation data being an execution result of the mth inference action data.

[0008] According to an embodiment of the present disclosure, the method further includes: in response to determining that the perplexity value is less than or equal to a predetermined threshold, processing the target question text according to the sequence of logical steps to generate an initial answer text.

[0009] According to an embodiment of the present disclosure, the pre-trained language model is trained by: obtaining sample question texts and first sample labels, the first sample labels indicating a sequence of logical steps for obtaining a correct answer text from the sample question texts; inputting the sample question texts into an initial language model to obtain a sample sequence of logical steps; generating an inference loss value based on the sample sequence of inference steps and the first sample labels according to a first loss function; and adjusting a first low-rank adaptive matrix of the initial language model based on the inference loss value to obtain the pre-trained language model, the first low-rank adaptive matrix being used to adjust parameters of the initial language model that are applicable to inference.

[0010] According to an embodiment of the present disclosure, sample correction data and a sample correction step sequence are obtained; the sample problem text, the sample correction data, and the sample correction step sequence are input into the initial language model to obtain actual correction data and an actual correction reasoning step sequence; a correction loss value is generated based on a second loss function according to the sample correction data and the actual correction data; and a second low-rank adaptive matrix of the initial language model is adjusted based on the correction loss value to obtain a pre-trained language model, the second low-rank adaptive matrix being used to adjust parameters of the initial language model that are applicable to reflection.

[0011] A second aspect of the present disclosure provides a text generation device of a pre-trained language model, comprising: a target path generation module configured to input the target problem text into the pre-trained language model to generate a target path; wherein the target path indicates a logical step sequence and a conditional probability for obtaining an initial reply text from the target problem text, and the target path comprises M steps, M being an integer greater than or equal to 1; a perplexity value generation module configured to generate a perplexity value based on the conditional probability of each step in the logical step sequence, the perplexity value representing the reliability of reasoning from an mth step to an m+1th step, m being an integer greater than or equal to 1 and less than or equal to M-1; a correction module configured to, in response to determining that the perplexity value is greater than a predetermined threshold, correct at least one step in the logical step sequence using the pre-trained language model to obtain a corrected logical step sequence; and a processing module configured to process the target problem text according to the corrected logical step sequence using the pre-trained language model to generate a target reply text.

[0012] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory configured to store one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0013] A fourth aspect of the present disclosure further provides a computer-readable storage medium having stored thereon a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the method.

[0014] A fifth aspect of the present disclosure further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the method.

[0015] According to the embodiments of the present disclosure, by judging the perplexity value generated based on the conditional probability of each step in the generated logical step sequence, in the case that the perplexity value is greater than a predetermined threshold, the pre-trained language model is used to correct at least one step in the logical step sequence, thereby ensuring the correctness of each step in the logical step sequence, and then using the pre-trained language model to process the target problem text according to the corrected logical step sequence to generate a correct target reply text, which optimizes the target path, reduces the error steps in the logical step sequence, thereby improving the reasoning efficiency, reducing the consumption of computing resources, and improving the robustness and explainability of the target reply text. BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0017] Figure 1 A diagram showing an application scenario of text generation based on a pre-trained language model according to embodiments of the present disclosure is shown;

[0018] Figure 2 A flowchart showing a text generation method based on a pre-trained language model according to embodiments of the present disclosure is shown;

[0019] Figure 3 A structural diagram of a transformer module according to embodiments of the present disclosure is shown;

[0020] Figure 4 A diagram showing the correction of a logical step sequence according to embodiments of the present disclosure is shown;

[0021] Figure 5 A structural block diagram of a text generation device based on a pre-trained language model according to embodiments of the present disclosure is shown;

[0022] Figure 6 A block diagram of an electronic device suitable for implementing a text generation method based on a pre-trained language model according to embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0023] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it is to be understood that these descriptions are only exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that one or more embodiments can be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques have been omitted to avoid unnecessarily obscuring the concept of the present disclosure.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the term "includes" and tautological derivatives thereof means that the named features, steps, operations and / or components are present, but does not exclude the presence or addition of one or more other features, steps, operations or components.

[0025] All terms used herein, including technical and scientific terms, have the meaning commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of the specification, and should not be interpreted in an idealized or overly formal way.

[0026] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted that the meaning is the same as "at least one of the group consisting of A, B, and C" or "at least one of A or B or C" unless specifically stated otherwise.

[0027] Large language models often need to perform multi-step reasoning according to complex input information when reasoning on high complexity tasks, and gradually derive intermediate conclusions until the final result is obtained. In order to improve the coherence and interpretability of the reasoning process, a chain thinking reasoning mechanism is proposed, which guides the model to generate intermediate reasoning steps, effectively promoting the step-by-step solution of complex problems.

[0028] However, the reasoning mechanism in the related art has significant deficiencies in actual application: when dealing with reasoning tasks with long reasoning chains, once the initial reasoning step deviates, the subsequent reasoning is often layered on the wrong basis, causing the overall reasoning trajectory to deviate from the correct direction, and thus unable to obtain the correct reasoning result.

[0029] Therefore, an embodiment of the present disclosure provides a text generation method based on a pre-trained language model, comprising: inputting a target problem text into a pre-trained language model to generate a target path, wherein the target path indicates a sequence of logical steps and conditional probabilities for obtaining an initial answer text from the target problem text, and the target path includes M steps, M being an integer greater than or equal to 1; based on the conditional probability of each step in the sequence of logical steps, generating a perplexity value, the perplexity value representing the credibility of the (m+1)th step inferred from the mth step, m being an integer greater than or equal to 1 and less than or equal to M-1; in response to determining that the perplexity value is greater than a predetermined threshold, correcting at least one step in the sequence of logical steps using the pre-trained language model to obtain a corrected sequence of logical steps; and processing the target problem text according to the corrected sequence of logical steps using the pre-trained language model to generate a target answer text.

[0030] Figure 1 An application scenario diagram of text generation based on a pre-trained language model according to an embodiment of the present disclosure is shown.

[0031] As shown in Figure 1 The application scenario 100 according to this embodiment can include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 can include various connection types, such as wired, wireless communication links, or fiber optic cables, and the like.

[0032] A user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, and the like. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, and the like (only as examples).

[0033] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with display screens and support for web browsing, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, and the like.

[0034] The server 105 can be a server that provides various services, such as a background management server that provides support for websites browsed by a user using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (only as an example). The background management server can analyze and process received user requests and the like, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests, and the like) to the terminal device.

[0035] It should be noted that the text generation method based on the pre-trained language model provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the text generation apparatus based on the pre-trained language model provided in the embodiments of the present disclosure can generally be arranged in the server 105. The text generation method based on the pre-trained language model provided in the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105. Accordingly, the text generation apparatus based on the pre-trained language model provided in the embodiments of the present disclosure can also be arranged in a server or a server cluster different from the server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or the server 105.

[0036] It should be understood that Figure 1 the number of terminal devices, networks, and servers in the above-mentioned scenario is only illustrative. Any number of terminal devices, networks, and servers can be provided according to implementation needs.

[0037] The text generation method based on the pre-trained language model of the disclosed embodiments will be described in detail below based on the scenario described above. Figure 1 Figures 2-4 The text generation method based on the pre-trained language model of the disclosed embodiments will be described in detail below based on the scenario described above.

[0038] Figure 2 A flowchart of the text generation method based on the pre-trained language model according to the embodiments of the present disclosure is shown.

[0039] As shown in Figure 2 the text generation method based on the pre-trained language model of the embodiments includes operations S210-S240.

[0040] In operation S210, the target question text is input into the pre-trained language model to generate a target path.

[0041] The target path indicates a sequence of logical steps and conditional probabilities for obtaining an initial answer text from the target question text, and the target path includes M steps, where M is an integer greater than or equal to 1.

[0042] According to the embodiments of the present disclosure, the pre-trained language model described above can be any large language model, for example, it can be a LLAMA3 model, which has 8 billion parameters and uses 32 layers of transformer modules, and each layer of the transformer module is composed of core components as shown in Figure 3 .

[0043] Figure 3 A structural diagram of the transformer module according to the embodiments of the present disclosure is shown.

[0044] As​Figure 3 As shown, the transformer module first receives an input token embedding, which is an initial representation of the input sequence, converting each token into a fixed-dimensional vector. The token embedding is then normalized by an RMS normalization to stabilize the training process. The normalized vector is then passed through a group query attention layer to calculate attention weights, which identify the relevance of different parts of the sequence. The output of the attention layer is again passed through an RMS normalization layer to further stabilize the data. The normalized data is then passed to the feedforward network portion, which is first passed through a SwiGLU activation function to increase the non-linear representation capacity of the model. A residual connection is provided after the attention layer and the feedforward network layer, which adds the input directly to the output to help alleviate the gradient vanishing problem in deep networks.

[0045] According to an embodiment of the present application, a pre-trained large language model is used to first generate a detailed reasoning plan (i.e., a target path) for the target question text, which is used to decompose the task and determine the information retrieval target and reasoning steps.

[0046] For example, for a target question text of "Who is older, A or B?" (A and B are both people whose birth dates can be retrieved on the Internet), the information retrieval target is the birth dates of A and B, and a time comparison is made on the birth dates of A and B to determine the answer to the question.

[0047] In operation S220, a confusion value is generated based on the conditional probability of each step in the sequence of logical steps.

[0048] wherein the confusion value represents the reliability of the inference of the m+1th step from the mth step, and m is an integer greater than or equal to 1 and less than or equal to M-1.

[0049] In operation S230, in response to determining that the confusion value is greater than a predetermined threshold, at least one step in the sequence of logical steps is corrected using a pre-trained language model to obtain a corrected sequence of logical steps.

[0050] According to an embodiment of the present disclosure, in the reasoning process, a confusion value is calculated for each step. When the confusion value of a step is higher than a predetermined threshold, it indicates that the step has a high degree of uncertainty or potential error, which triggers a reflection mechanism immediately, generates reflection content, and corrects the step based on the reflection content, thereby obtaining a corrected sequence of logical steps.

[0051] According to an embodiment of the present disclosure, the reflection content described above can be generated for a logical step in the sequence of logical steps whose confusion value is greater than a predetermined threshold and the logical steps after the logical step.

[0052] For example, if the confusion value of logical step 2 is greater than a preset threshold, then reflection content can be generated based on logical step 2 and subsequent logical step 3.

[0053] Figure 4 A schematic diagram illustrating a modification of the sequence of logical steps according to an embodiment of the present disclosure is shown.

[0054] like Figure 4 As shown, the solid circles represent the sequence of logical steps generated by the pre-trained language model, while the dashed circles represent the corrected sequence of logical steps. The sequence includes logical step 1, logical step 2, and logical step 3. For demonstration purposes, only the perplexity value calculation for logical step 2 is shown in the diagram; in practice, perplexity values ​​should be calculated for logical steps 1, 2, and 3 separately. It can be seen that if the perplexity value of logical step 2 exceeds a predetermined threshold, logical step 2 will be reconsidered and corrected to the corrected logical step 2, thus yielding the corrected logical step 3.

[0055] According to embodiments of this disclosure, if the confusion value is greater than a preset threshold, the entire inference step sequence can be revised to obtain revised logic step 1, logic step 2, and logic step 3. This disclosure does not limit this, and the aforementioned preset threshold can be, for example, 6.

[0056] In operation S240, a pre-trained language model is used to process the target question text according to the corrected logical step sequence, and the target response text is generated.

[0057] According to embodiments of this disclosure, by determining the perplexity value generated based on the conditional probability of each step in the generated logical step sequence, and if the perplexity value is greater than a predetermined threshold, a pre-trained language model is used to correct at least one step in the logical step sequence, thereby ensuring the correctness of each step in the logical step sequence. Then, the pre-trained language model is used to process the target question text according to the corrected logical step sequence to generate the correct target response text, thereby optimizing the target path, reducing erroneous steps in the logical step sequence, improving reasoning efficiency, reducing the consumption of computing resources, and improving the robustness and interpretability of the target response text.

[0058] According to embodiments of this disclosure, the method further includes: in response to determining that the confusion value is less than or equal to a predetermined threshold, processing the target question text according to a logical step sequence to generate an initial response text.

[0059] According to the embodiments of this disclosure, if the confusion value of each logical step in the logical step sequence is less than or equal to a preset threshold, it can be considered that the model is confident enough in the current reasoning result. Then, the target question text can be processed according to the previously generated logical step sequence to generate the initial response text.

[0060] According to the embodiment of the present disclosure, the mth step includes mth reasoning thinking data, mth reasoning action data and mth reasoning observation data, the pre-trained language model is used to correct at least one step in the logical step sequence, and a corrected logical step sequence is obtained, including: inputting the mth step into the pre-trained language model to obtain reflection data, the reflection data representing the evaluation and improvement of the pre-trained language model on the mth step; determining the mth reasoning thinking data based on the reflection data by using the pre-trained language model, the mth reasoning thinking data representing the logical reasoning of the mth reasoning step; determining the mth reasoning action data based on the mth reasoning thinking data by using the pre-trained language model, the mth reasoning action data representing the execution action of the mth reasoning step; and obtaining the mth reasoning observation data based on the mth reasoning action data by using the pre-trained language model, the mth reasoning observation data being the execution result of the mth reasoning action data.

[0061] According to the embodiment of the present disclosure, the ReAct (Reasoning and Acting) reasoning framework can be used to generate the target path, in which process, the model alternately generates “thinking-action-observation” triplets to constitute the logical step sequence. Among them, the thinking part is the internal reasoning content, the action represents the execution steps such as “search” and “find”, and the observation is the feedback result of determining the action. In order to facilitate description, the following will use thinking to refer to reasoning thinking data, use action to refer to reasoning action data, and use observation to refer to reasoning observation data.

[0062] For example, for the target question text "What are the commonalities between City A and City B?", the plan is "Search for relevant information about City A and City B to find their commonalities", and the target path is "Thought 1: I need to understand the historical, cultural, and economic characteristics of City A and City B; Action 1: Search for basic information about City A; Observation 1: City A is a historically rich city known for its rich cultural heritage and active economic activities. Thought 2: City A has many characteristics, and I need to do the same search for City B to compare. Action 2: Search for basic information about City B. Observation 2: City B also has a rich history and culture and is an economically active city. Thought 3: Both cities have rich history and culture, but I need more specific commonalities. Action 3: Search for common historical events or cultural activities between City A and City B. Observation 3: No specific common historical events or cultural activities were found. Thought 4: It may be necessary to look for commonalities from other perspectives, such as geographical location or economic cooperation. Action 4: Search for the geographical location and economic cooperation between City A and City B. Observation 4: City A and City B are located in the same geographical region and have economic cooperation projects. Thought 5: A commonality has been found, but there may be other commonalities that have not been discovered. Action 6: Complete, the commonalities between City A and City B are that they are located in the same geographical region and have economic cooperation." Assuming that reflection is triggered at step 3, the reflection content can be "Should search more directly for common characteristics of City A and City B, rather than searching for all information broadly."

[0063] The revised plan is "Search for common characteristics of City A and City B, such as geographical location, economic cooperation, cultural activities, etc.", and the revised sequence of logical steps can be "Thought 1: I need to quickly identify the common characteristics of City A and City B in order to more effectively answer the question. Action 1: Search for the geographical location of City A and City B. Observation 1: City A and City B are both located in the same geographical region. Thought 2: Geographical location is a commonality, and I also need to check for other common characteristics. Action 2: Search for the economic cooperation between City A and City B. Observation 2: City A and City B have economic cooperation projects. Thought 3: Two commonalities have been found, geographical location and economic cooperation, which should be enough to answer the question. Action 3: Complete, the commonalities between City A and City B are that they are located in the same geographical region and have economic cooperation."

[0064] According to the embodiments of the present disclosure, the reflection data is obtained by the pre-trained language model, and further, the institutionalized reasoning process is generated according to the reflection data, so that the model can proceed in a cycle of "thinking-action-observation", and since each step has clear reasoning logic and execution actions, it is convenient for users to understand, thereby improving the reasoning ability and interpretability of the model, and at the same time, making the large language model more intelligent and efficient in the processing of complex tasks.

[0065] According to the embodiments of the present disclosure, the mth step consists of N reasoning words, N is an integer greater than or equal to 1, and the conditional probability of the mth step is determined by the following method: for the first reasoning word in the N reasoning words, the first conditional probability is obtained based on the preset starting mark, and the first conditional probability is the probability of outputting the first reasoning word; for the nth reasoning word in the N reasoning words, the nth conditional probability is obtained based on the first n-1 reasoning words, to obtain N-1 conditional probabilities, n is an integer greater than 1 and less than or equal to N, and the nth probability is the probability of outputting the nth reasoning word in the case of determining the first n-1 reasoning words; and the conditional probability of the mth step is generated based on the first probability to the Nth probability.

[0066] For example, assuming that the reasoning step 1 is "searching for basic information of city A", the reasoning step can be split into multiple reasoning words, separated by semicolons, which are "searching; city A; of; basic; information" respectively. For the first reasoning word, i.e. "searching", the first conditional probability P(search | <start>The first conditional probability is calculated using P(city A | search). Further, for the second inference word, the second conditional probability can be calculated using P(city A | search), and so on, until the fifth conditional probability is generated, finally obtaining the conditional probability of inference step 1 (the product of the first conditional probability to the fifth conditional probability).

[0067] According to an embodiment of this disclosure, the above-mentioned generation of perplexity value based on the conditional probability of each step in the logical step sequence includes: performing a logarithmic operation on the first conditional probability and N-1 conditional probabilities to obtain N inference word conditional probabilities; obtaining the average inference word conditional probability based on the N inference word conditional probabilities; and generating perplexity value based on the average inference word conditional probability.

[0068] The confusion value can be calculated using the following formula (1).

[0069] (1)

[0070] Where PPL(m) represents the perplexity value at the m-th step, and N represents the N reasoning words. This represents the conditional probability of the model for the nth inference word.

[0071] According to an embodiment of this disclosure, the pre-trained language model is trained by the following method: obtaining sample question text and a first sample label, the first sample label indicating the logical step sequence from the sample question text to obtain the correct answer text; inputting the sample question text into an initial language model to obtain a sample logical step sequence; generating an inference loss value based on a first loss function, according to the sample inference step sequence and the first sample label; adjusting the first low-rank adaptive matrix of the initial language model based on the inference loss value to obtain a pre-trained language model, the first low-rank adaptive matrix being used to adjust the parameters of the initial language model suitable for inference.

[0072] According to embodiments of this disclosure, the sample question text could be, for example, "What do city A and city B have in common?", and the corresponding first sample label could be the reasoning logic sequence shown above, which will not be repeated here. The language model is fine-tuned using a reasoning dataset to enable the model to generate a reasonable and executable reasoning plan based on the input question, thereby completing the initial generation of the reasoning logic sequence. The training objective is to maximize the conditional probability from the question to the reasoning plan and the reasoning logic sequence. The aforementioned first loss function can be an autoregressive log-likelihood function.

[0073] The inference loss value can be calculated using the following formula (2).

[0074] (2)

[0075] in, denotes a reasoning loss value, x denotes a sample question text, and y denotes a sample logical step sequence j denotes a jth sample reasoning word in a sample logical step sequence, denotes a probability of generating a sample reasoning word y j under the premise of given a sample question text x and the first to the j-1th sample reasoning word.

[0076] According to the embodiments of the present disclosure, the method further comprises: obtaining sample correction data and a sample correction step sequence; inputting the sample question text, the sample correction data, and the sample correction step sequence into the initial language model to obtain actual correction data and an actual correction reasoning step sequence; generating a correction loss value based on the second loss function according to the sample correction data and the actual correction data; and adjusting the second low-rank adaptive matrix of the initial language model based on the correction loss value to obtain the pre-trained language model, the second low-rank adaptive matrix being used to adjust parameters of the initial language model applicable to reflection.

[0077] The reasoning loss value can be calculated by the following formula (3).

[0078] ; (3)

[0079] wherein, denotes a correction loss value, x denotes a sample question text, y denotes a sample logical step sequence, and r j denotes a jth sample reflection reasoning word in a sample correction step sequence, denotes a probability of generating a sample reflection reasoning word r j under the premise of given a sample question text x, a sample logical step sequence y, and the first to the j-1th sample reflection reasoning word.

[0080] According to the embodiments of the present disclosure, the training efficiency can be improved and the parameter update cost can be reduced by using the low-rank adaptive matrix, and the first low-rank adaptive matrix and the second low-rank adaptive matrix are used in the reasoning and reflection training stages respectively, so as to realize modular low-rank parameter adaptation, thereby reducing the training overhead while maintaining the stability of the language model performance. Specifically, the weights of the linear projection units in the grouped query attention layer of the language model can be adjusted, so that the training process is more efficient.

[0081] According to the embodiments of the present disclosure, the method has a significant effect in balancing performance and computational cost. Under the experimental condition that the preset threshold of the perplexity value is 6, the F1 value of the model on the HotpotQA task reaches the optimum, effectively improving the accuracy and stability of the answer in multi-hop reasoning, and avoiding unnecessary repeated calculation in the case of low perplexity.

[0082] According to the embodiment of the present disclosure, the inference loss value is generated based on the sample inference step sequence and the first sample label by the first loss function, and then the first low-rank adaptive matrix is adjusted based on the inference loss value, so that the language model has the inference plan generation capability. The correction loss value is generated based on the sample correction data and the actual correction data by the second loss function, and then the second low-rank adaptive matrix is adjusted based on the correction loss value, so that the language model has the inference optimization capability. Finally, the pre-trained language model with multi-stage inference capability is obtained, which can improve the correctness and explainability of the inference result, and can more efficiently obtain the problem result in a complex inference task.

[0083] Based on the above text generation method based on the pre-trained language model, the present disclosure further provides a text generation device based on a pre-trained language model. The following will be described in detail Figure 5 The device is described in detail.

[0084] Figure 5 The structure block diagram of the text generation device based on the pre-trained language model according to the embodiment of the present disclosure is shown.

[0085] As Figure 5 shown, the text generation device based on the pre-trained language model of the embodiment 500 includes a target path generation module 510, a perplexity value generation module 520, a correction module 530, and a processing module 540.

[0086] The target path generation module 510 is configured to input the target problem text into the pre-trained language model to generate a target path. The target path indicates a logical step sequence and a conditional probability for obtaining an initial reply text from the target problem text. The target path includes M steps, and M is an integer greater than or equal to 1. In an embodiment, the target path generation module 510 can be configured to perform the operation S210 described above, and details are not repeated here.

[0087] The perplexity value generation module 520 is configured to generate a perplexity value based on the conditional probability of each step in the logical step sequence. The perplexity value represents the credibility of inferring the m+1th step from the mth step, and m is an integer greater than or equal to 1 and less than or equal to M-1. In an embodiment, the perplexity value generation module 520 can be configured to perform the operation S220 described above, and details are not repeated here.

[0088] The correction module 530 is configured to correct at least one step in the logical step sequence using the pre-trained language model in response to determining that the perplexity value is greater than a predetermined threshold, to obtain a corrected logical step sequence. In an embodiment, the correction module 530 can be configured to perform the operation S230 described above, and details are not repeated here.

[0089] The processing module 540 is used to process the target question text according to the corrected logical step sequence using a pre-trained language model, and generate the target response text. In one embodiment, the processing module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0090] According to embodiments of this disclosure, any plurality of modules among the target path generation module 510, confusion value generation module 520, correction module 530, and processing module 540 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules may be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the target path generation module 510, confusion value generation module 520, correction module 530, and processing module 540 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the target path generation module 510, confusion value generation module 520, correction module 530, and processing module 540 may be implemented at least partially as a computer program module that can perform corresponding functions when the computer program module is run.

[0091] Figure 6 A block diagram of an electronic device suitable for implementing a text generation method based on a pre-trained language model, according to an embodiment of the present disclosure, is shown.

[0092] like Figure 6 As shown, an electronic device 600 according to an embodiment of this disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0093] In the RAM 603, various programs and data required for the operation of the electronic device 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other via the bus 604. The processor 601 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 602 and / or the RAM 603. It should be noted that the programs can also be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 can also perform various operations of the method flow according to the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0094] According to an embodiment of the present disclosure, the electronic device 600 can further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 can further include one or more of the following components connected to the input / output (I / O) interface 605: an input part 606 including a keyboard, a mouse, and the like; an output part 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage part 608 including a hard disk, and the like; and a communication part 609 including a network interface card such as a LAN card, a modem, and the like. The communication part 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as necessary. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 610 as necessary, so that a computer program read therefrom is installed in the storage part 608 as necessary.

[0095] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0096] According to an embodiment of the present disclosure, the computer readable storage medium can be a nonvolatile computer readable storage medium, for example, can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer readable storage medium can include one or more memories, such as the ROM 602 and / or the RAM 603 described above, and / or one or more memory storage devices other than the ROM 602 and the RAM 603.

[0097] Embodiments of the present disclosure also include a computer program product, which includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the text generation method based on the pre-trained language model provided by the embodiments of the present disclosure.

[0098] The above-described functions defined in the system / device of the embodiments of the present disclosure are performed when the computer program is executed by the processor 601. According to an embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0099] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of signals on network media. The computer program is installed and executed through the communication part 609 and / or installed from the detachable medium 611. The program codes contained in the computer program can be transmitted by any appropriate network media, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0100] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609 and / or installed from the detachable medium 611. When the computer program is executed by the processor 601, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0101] According to embodiments of the present disclosure, program code of a computer program for performing the computer program provided by embodiments of the present disclosure can be written in any combination of one or more programming languages, and specifically, can be implemented using high-level procedural and / or object-oriented programming language, and / or assembly / machine language. Programming languages include, but are not limited to, Java, C++, python, "C" language, or similar programming languages. Program code can execute entirely on a user's computing device, partly on a user device, partly on a remote computing device, or entirely on a remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0102] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams or block diagrams can represent a module, a procedure, or a portion of code, which comprises one or more executable instructions for implementing the specific logical functions specified for the block. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block of the block diagrams or flow diagrams, and combinations of blocks in the block diagrams or flow diagrams, can be implemented by dedicated-circuitry-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0103] Those skilled in the art can understand that the features described in various embodiments of the present disclosure can be combined and / or integrated in various combinations, even if such combinations are not explicitly described in the present disclosure. In particular, the features described in various embodiments of the present disclosure can be combined and / or integrated in various combinations without departing from the spirit and teachings of the present disclosure. All such combinations and / or integrations are within the scope of the present disclosure.

[0104] The above describes embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.< / start>

Claims

1. A text generation method based on a pre-trained language model, characterized by, The method includes: The target question text is input into a pre-trained language model to generate a target path, wherein the target path indicates the logical step sequence and conditional probability of obtaining the initial response text from the target question text, and the target path includes M steps, where M is an integer greater than or equal to 1; Based on the conditional probability of each step in the logical step sequence, a perplexity value is generated. The perplexity value represents the credibility of reasoning from the m-th step to the (m+1)-th step, where m is an integer greater than or equal to 1 and less than or equal to M-1. In response to determining that the confusion value is greater than a predetermined threshold, at least one step in the logical step sequence is corrected using the pre-trained language model to obtain a corrected logical step sequence. Using the pre-trained language model, the target question text is processed according to the corrected logical step sequence to generate the target response text; The m-th step consists of N inference words, where N is an integer greater than or equal to 1. The conditional probability of the m-th step is determined by the following method: For the first inference word among the N inference words, a first conditional probability is obtained based on a preset starting marker. The first conditional probability is the probability of outputting the first inference word. For the nth inference word among the N inference words, the nth conditional probability is obtained based on the first n-1 inference words, so as to obtain N-1 conditional probabilities, where n is an integer greater than 1 and less than or equal to N, and the nth probability is the probability of outputting the nth inference word given the first n-1 inference words; The conditional probability of the m-th step is generated based on the first probability to the Nth probability; The generation of perplexity values ​​based on the conditional probability of each step in the logical step sequence includes: Perform a logarithmic operation on the first conditional probability and the N-1 conditional probabilities to obtain N inference word conditional probabilities; The average conditional probability of the inference word is obtained based on the conditional probabilities of the N inference words. The perplexity value is generated based on the average inference word conditional probability. The m-th step includes m-th reasoning thinking data, m-th reasoning action data, and m-th reasoning observation data. The step of using the pre-trained language model to correct at least one step in the logical step sequence to obtain a corrected logical step sequence includes: The m-th step is input into the pre-trained language model to obtain reflective data, which characterizes the pre-trained language model's evaluation and improvement of the m-th step; Using the pre-trained language model, the m-th reasoning thinking data is determined based on the reflection data, and the m-th reasoning thinking data represents the logical reasoning of the m-th reasoning step; Using the pre-trained language model, the m-th reasoning action data is determined based on the m-th reasoning thinking data, and the m-th reasoning action data represents the execution action of the m-th reasoning step; Using the pre-trained language model, the m-th inference observation data is obtained based on the m-th inference action data, and the m-th inference observation data is the execution result of the m-th inference action data; The pre-trained language model was obtained through the following method: obtaining a sample question text and a first sample label, the first sample label indicating a logical step sequence for obtaining a correct answer text from the sample question text; inputting the sample question text into an initial language model to obtain a sample logical step sequence; generating an inference loss value based on the first sample label and the sample inference step sequence according to a first loss function; adjusting a first low-rank adaptive matrix of the initial language model based on the inference loss value to obtain the pre-trained language model, the first low-rank adaptive matrix being used to adjust parameters of the initial language model suitable for inference.

2. The method of claim 1, wherein, The method further comprises: in response to determining that the perplexity value is less than or equal to a predetermined threshold, processing the target question text according to the logical step sequence to generate an initial answer text.

3. The method of claim 1, wherein, The method further comprises: obtaining sample correction data and a sample correction step sequence; inputting the sample question text, the sample correction data and the sample correction step sequence into the initial language model to obtain actual correction data and an actual correction inference step sequence; generating a correction loss value based on the sample correction data and the actual correction data according to a second loss function; adjusting a second low-rank adaptive matrix of the initial language model based on the correction loss value to obtain the pre-trained language model, the second low-rank adaptive matrix being used to adjust parameters of the initial language model suitable for reflection.

4. A text generation apparatus based on a pre-trained language model, characterized by, The device comprises: a target path generation module configured to input a target question text into a pre-trained language model to generate a target path, wherein the target path indicates a logical step sequence and a conditional probability for obtaining an initial answer text from the target question text, and the target path comprises M steps, M being an integer greater than or equal to 1; a perplexity value generation module configured to generate a perplexity value based on the conditional probability of each step in the logical step sequence, the perplexity value representing the reliability of inferring the m+1th step from the mth step, m being an integer greater than or equal to 1 and less than or equal to M-1; a correction module configured to, in response to determining that the perplexity value is greater than a predetermined threshold, correct at least one step in the logical step sequence using the pre-trained language model to obtain a corrected logical step sequence; a processing module configured to process the target question text according to the corrected logical step sequence using the pre-trained language model to generate a target answer text; The mth step is composed of N inference words, N being an integer greater than or equal to 1, and the device further comprises: a first conditional probability determination module configured to obtain a first conditional probability based on a preset starting mark for a first inference word in the N inference words, the first conditional probability being the probability of outputting the first inference word; an nth conditional probability determination module configured to obtain an nth conditional probability based on the first n-1 inference words for an nth inference word in the N inference words to obtain N-1 conditional probabilities, n being an integer greater than 1 and less than or equal to N, the nth probability being the probability of outputting the nth inference word under the condition that the first n-1 inference words are determined. The probability generation module is configured to generate a conditional probability of the mth step based on the first probability to the Nth probability. The perplexity value generation module comprises: The operation sub-module is configured to perform logarithmic operation on the first conditional probability and the N-1 conditional probabilities to obtain N inference word conditional probabilities. The average sub-module is configured to obtain an average inference word conditional probability based on the N inference word conditional probabilities. The perplexity value generation sub-module is configured to generate the perplexity value based on the average inference word conditional probability. The mth step comprises mth inference thinking data, mth inference action data and mth inference observation data, and the correction module comprises: The input sub-module is configured to input the mth step into the pre-trained language model to obtain reflection data, the reflection data representing evaluation and improvement of the pre-trained language model on the mth step. The thinking data determination sub-module is configured to determine mth inference thinking data based on the reflection data by using the pre-trained language model, the mth inference thinking data representing logical inference of the mth inference step. The action data determination sub-module is configured to determine mth inference action data based on the mth inference thinking data by using the pre-trained language model, the mth inference action data representing execution action of the mth inference step. The observation data determination sub-module is configured to obtain mth inference observation data based on the mth inference action data by using the pre-trained language model, the mth inference observation data being an execution result of the mth inference action data. The pre-trained language model is trained by the following modules: The sample acquisition module is configured to acquire sample question text and a first sample label, the first sample label indicating a logical step sequence for obtaining a correct answer text from the sample question text. The model input module is configured to input the sample question text into an initial language model to obtain a sample logical step sequence. The loss value generation module is configured to generate an inference loss value based on a first loss function and according to the sample inference step sequence and the first sample label. The adjustment module is configured to adjust a first low-rank adaptive matrix of the initial language model based on the inference loss value to obtain the pre-trained language model, the first low-rank adaptive matrix being used to adjust parameters of the initial language model applicable to inference.

5. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-3.

6. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instructions are executed by the processor to implement the steps of the method according to any one of claims 1-3.

Images