Natural language model generation method and data processing method

Abstract

The present application discloses a method for generating a natural language model and a data processing method. The method includes: obtaining a sample question and obtaining a set of reasoning paths corresponding to the sample question, wherein the set of reasoning paths at least includes: a reference reasoning path and a pair of reasoning path branches; constructing a target training set based on the sample question and the set of reasoning paths; training the initial natural language model based on the target training set to obtain a target natural language model, wherein the target natural language model is used to answer the question information input by the target object. The present application solves the technical problem in the related art that the accuracy of the natural language model in answering questions is relatively low due to the poor reasoning ability of the natural language model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a method for generating a natural language model and a method for processing data. Background Art

[0002] Natural language models demonstrate amazing problem-solving capabilities through step-by-step reasoning. However, they often make mistakes when dealing with more complex problems, which can affect subsequent step-by-step reasoning paths. The prior art lacks methods for training the reasoning model of natural language models, resulting in low accuracy in answering questions with natural language models.

[0003] With respect to the problem in the above-mentioned related technologies that the accuracy of natural language models in answering questions is relatively low due to the poor reasoning ability of natural language models, no effective solution has been proposed so far. Summary of the invention

[0004] The embodiments of the present application provide a method for generating a natural language model and a data processing method, so as to at least solve the technical problem in the related art that the accuracy of the natural language model in answering questions is relatively low due to the poor reasoning ability of the natural language model.

[0005] According to one aspect of an embodiment of the present application, a method for generating a natural language model is provided, comprising: obtaining the sample question, and obtaining a set of reasoning paths corresponding to the sample question, wherein the reasoning path set at least includes: a reference reasoning path and a reasoning path branch pair, the reference reasoning path includes true answer information for answering the sample question, the reasoning path branch pair includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the true answer information, and the second reasoning path branch is a path branch that cannot lead to the true answer information; constructing the target training set based on the sample question and the reasoning path set; training an initial natural language model based on the target training set to obtain the target natural language model.

[0006] Furthermore, obtaining the set of reasoning paths corresponding to the sample question includes: generating a thinking chain example based on the sample question and the thinking chain prompt, wherein the thinking chain example includes: the sample question and the thinking chain reasoning path corresponding to the sample question, and the thinking chain reasoning path includes real answer information for answering the sample question; processing the sample question and the thinking chain example through the initial natural language model to determine whether the reference reasoning path corresponding to the sample question can be obtained; when the reference reasoning path corresponding to the sample question can be obtained, obtaining the reasoning path branch pair based on the sample question, the reference reasoning path and the thinking chain example; obtaining the reasoning path set based on the reference reasoning path and the reasoning path branch pair.

[0007] Furthermore, the sample question and the thought chain example are processed by the initial natural language model to determine whether the reference reasoning path corresponding to the sample question can be obtained, including: processing the sample question and the thought chain example by the initial natural language model to obtain a first reasoning path; and determining whether the reference reasoning path corresponding to the sample question can be obtained based on whether the true answer information can be inferred by the first reasoning path.

[0008] Furthermore, based on the sample problem, the reference reasoning path and the thinking chain example, the reasoning path branch pair is obtained, including: based on the reference reasoning path, a reference reasoning sub-path set is obtained; based on the sample problem, the reference reasoning sub-path set and the thinking chain example, the reasoning path branches are sampled and generated with the last reasoning step in the reference reasoning sub-path of the reference reasoning sub-path set as the starting point, to obtain multiple reasoning path branches corresponding to the reference reasoning sub-path of the reference reasoning sub-path set; based on the multiple reasoning path branches, the first reasoning path branch and the second reasoning path branch are obtained; based on the first reasoning path branch and the second reasoning path branch, the reasoning path branch pair is obtained.

[0009] Furthermore, the initial natural language model is trained according to the target training set to obtain the target natural language model, including: processing the target training set through the initial natural language model to obtain a target prediction reasoning path; obtaining a target loss function according to the target prediction reasoning path; and training the initial natural language model according to the target loss function to obtain the target natural language model.

[0010] Furthermore, the target training set is processed by the initial natural language model to obtain a target prediction reasoning path, including: processing the sample question by the initial natural language model to obtain a second reasoning path; processing the sample question and a set of reference reasoning sub-paths of the reference reasoning path by the initial natural language model to obtain a third reasoning path; and obtaining the target prediction reasoning path based on the second reasoning path and the third reasoning path.

[0011] Furthermore, based on the target prediction reasoning path, obtaining the target loss function includes: calculating based on the second reasoning path and the reference reasoning path to obtain a reference path loss function; calculating based on the third reasoning path and the reasoning path branch pair to obtain a branch loss function; obtaining the target loss function based on the reference path loss function and the branch loss function.

[0012] Furthermore, calculations are performed based on the third reasoning path and the reasoning path branch to obtain a branch loss function, including: calculations are performed based on the third reasoning path and the first reasoning path branch to obtain a first loss function; calculations are performed based on the third reasoning path and the second reasoning path branch to obtain a second loss function; calculations are performed based on the first loss function and the second loss function to obtain the branch loss function.

[0013] According to another aspect of an embodiment of the present application, a data processing method is provided, including: obtaining question information input by a target object; performing reasoning analysis on the question information through a target natural language model to obtain a target reasoning path, wherein the target natural language model is obtained according to any of the above-mentioned natural language model generation methods, and the target reasoning path consists of logical steps of deriving from the question information to answer information corresponding to the question information, and the target natural language model is obtained from a target training set, and the target training set consists of sample questions and a set of reasoning paths corresponding to the sample questions; and returning the target reasoning path to the target object.

[0014] Furthermore, the question information is reasoned and analyzed through the target natural language model to obtain a target reasoning path, including: reasoning and analyzing the question information through the target natural language model to generate multiple candidate reasoning paths, wherein the candidate reasoning paths include answer information corresponding to the question information; and determining the target reasoning path from the multiple candidate reasoning paths through the target natural language model.

[0015] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining question information sent by a client; performing reasoning analysis on the question information through a target natural language model in a cloud server to obtain a target reasoning path, wherein the target reasoning path is composed of logical steps of deriving from the question information to answer information corresponding to the question information, the target natural language model is obtained from a target training set, and the target training set is composed of sample questions and a set of reasoning paths corresponding to the sample questions; and returning the target reasoning path to the client.

[0016] According to another aspect of an embodiment of the present application, a device for generating a natural language model is also provided, including: a first acquisition unit, used to acquire sample questions, and acquire a set of reasoning paths corresponding to the sample questions, wherein the reasoning path set includes at least: a reference reasoning path and a reasoning path branch pair, the reference reasoning path includes real answer information for answering the sample questions, the reasoning path branch pair includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the real answer information, and the second reasoning path branch is a path branch that cannot lead to the real answer information; a construction unit, used to construct the target training set based on the sample questions and the reasoning path set; a training unit, used to train the initial natural language model based on the target training set to obtain the target natural language model, wherein the target natural language model is used to answer the question information input by the target object.

[0017] Furthermore, the first acquisition unit includes: a generating subunit, used to generate a thinking chain example based on the sample question and the thinking chain prompt, wherein the thinking chain example includes: the sample question and the thinking chain reasoning path corresponding to the sample question, and the thinking chain reasoning path includes the real answer information of answering the sample question; a processing subunit, used to process the sample question and the thinking chain example through the initial natural language model to determine whether the reference reasoning path corresponding to the sample question can be obtained; a first determining subunit, used to obtain the reasoning path branch pair based on the sample question, the reference reasoning path and the thinking chain example when the reference reasoning path corresponding to the sample question can be obtained; a second determining subunit, used to obtain the reasoning path set based on the reference reasoning path and the reasoning path branch pair.

[0018] Furthermore, the processing subunit includes: a processing module, used to process the sample question and the thinking chain example through the initial natural language model to obtain a first reasoning path; a judgment module, used to judge whether the reference reasoning path corresponding to the sample question can be obtained based on whether the first reasoning path can be inferred to obtain the true answer information.

[0019] Furthermore, the first determination subunit includes: a first determination module, used to obtain a reference reasoning sub-path set based on the reference reasoning path; a sampling module, used to sample and generate reasoning path branches based on the sample problem, the reference reasoning sub-path set and the thinking chain example, taking the last reasoning step in the reference reasoning sub-path of the reference reasoning sub-path set as the starting point, to obtain multiple reasoning path branches corresponding to the reference reasoning sub-path of the reference reasoning sub-path set; a second determination module, used to obtain the first reasoning path branch and the second reasoning path branch based on the multiple reasoning path branches; a third determination module, used to obtain the reasoning path branch pair based on the first reasoning path branch and the second reasoning path branch.

[0020] Furthermore, the training unit includes: a processing subunit, used to process the target training set through the initial natural language model to obtain a target prediction reasoning path; a third determination subunit, used to obtain a target loss function based on the target prediction reasoning path; and a training subunit, used to train the initial natural language model based on the target loss function to obtain the target natural language model.

[0021] Furthermore, the processing sub-unit includes: a first processing module, used to process the sample question through the initial natural language model to obtain a second reasoning path; a second processing module, used to process the sample question and a reference reasoning sub-path set of the reference reasoning path through the initial natural language model to obtain a third reasoning path; and a fourth determination module, used to obtain the target prediction reasoning path based on the second reasoning path and the third reasoning path.

[0022] Furthermore, the third determination subunit includes: a first calculation module, used to perform calculations based on the second reasoning path and the reference reasoning path to obtain a reference path loss function; a second calculation module, used to perform calculations based on the third reasoning path and the reasoning path branch pair to obtain a branch loss function; and a fifth determination module, used to obtain the target loss function based on the reference path loss function and the branch loss function.

[0023] Furthermore, the second calculation module includes: a first calculation submodule, used to perform calculations based on the third reasoning path and the first reasoning path branch to obtain a first loss function; a second calculation submodule, used to perform calculations based on the third reasoning path and the second reasoning path branch to obtain a second loss function; and a third calculation submodule, used to perform calculations based on the first loss function and the second loss function to obtain the branch loss function.

[0024] According to another aspect of an embodiment of the present invention, a data processing device is also provided, including: a second acquisition unit, used to acquire question information input by a target object; an inference unit, used to perform inference analysis on the question information through a target natural language model to obtain a target reasoning path, wherein the target natural language model is obtained according to any of the above-mentioned natural language model generation methods, and the target reasoning path consists of logical steps of deriving from the question information to answer information corresponding to the question information; and a return unit, used to return the target reasoning path to the target object.

[0025] Furthermore, the reasoning unit includes: a reasoning subunit, used to perform reasoning analysis based on the question information through the target natural language model to generate multiple candidate reasoning paths, wherein the candidate reasoning paths include answer information corresponding to the question information; and a fourth determination subunit, used to determine the target reasoning path from the multiple candidate reasoning paths through the target natural language model.

[0026] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising a memory storing an executable program; and a processor for running the program, wherein when the program is run, the method for generating a natural language model or the data processing method described above is executed.

[0027] According to another aspect of an embodiment of the present invention, a computer program product is also provided, which includes a stored computer program, and when the computer program is executed by a processor, the method for generating a natural language model or the data processing method described in any one of the above items is implemented.

[0028] In an embodiment of the present application, the following steps are adopted: obtaining a sample question, and obtaining a set of reasoning paths corresponding to the sample question, wherein the reasoning path set includes at least: a reference reasoning path and a reasoning path branch pair, the reference reasoning path includes real answer information for answering the sample question, the reasoning path branch pair includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the real answer information, and the second reasoning path branch is a path branch that cannot lead to the real answer information; constructing the target training set based on the sample question and the reasoning path set; training an initial natural language model based on the target training set to obtain the target natural language model, wherein the target natural language model is used to answer the question information input by the target object, which solves the technical problem in the related art that the accuracy of the natural language model in answering questions is relatively low due to the poor reasoning ability of the natural language model.

[0029] In this scheme, reasoning analysis is performed based on sample questions to obtain a corresponding set of reasoning paths. The initial natural language model is trained using sample questions and the set of reasoning paths corresponding to the sample questions to obtain a target natural language model. This can effectively explore the potential solution space of a given problem and enhance the inherent reasoning ability of the natural language model, thereby achieving the effect of improving the accuracy of the natural language model in answering questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0031] Figure 1 is a hardware structure block diagram of a computer terminal provided according to Embodiment 1 of the present application;

[0032] Figure 2 is a flowchart of a method for generating a natural language model according to Embodiment 1 of the present application;

[0033] Figure 3 is a flowchart of determining a target natural language model according to Embodiment 1 of the present application;

[0034] Figure 4 is a flow chart of generating a reasoning path according to the first embodiment of the present application;

[0035] Figure 5 is a flow chart of a data processing method provided according to Embodiment 2 of the present application;

[0036] Figure 6 is a flow chart of a data processing method provided according to Embodiment 3 of the present application;

[0037] Figure 7 is a schematic diagram of a device for generating a natural language model according to Embodiment 4 of the present application;

[0038] Figure 8 is a schematic diagram of a data processing device provided according to Embodiment 5 of the present application;

[0039] Fig. 9 This is a structural block diagram of an electronic device provided according to Embodiment 6 of the present application. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0042] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards in the relevant regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0043] Example 1

[0044] According to an embodiment of the present application, a method for generating a natural language model is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] The method embodiment provided in the first embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 The hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for generating a natural language model is shown. Figure 1 As shown, the computer terminal (or mobile device) 10 may include a processor set 102 (the processor set 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA, and the processor set 102 may include a processor set, Figure 1 102a, 102b, ..., 102n are used to illustrate), a memory 104 for storing data, and a transmission module 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0046] It should be noted that the one or more processors 102 and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0047] The memory 104 can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the method for generating a natural language model in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the method for generating a natural language model described above is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0048] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0049] The display may be a touch screen type liquid crystal display, which may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

[0050] Under the above operating environment, this application provides Figure 2 The method for generating the natural language model shown. Figure 2 1 is a flow chart of a method for generating a natural language model according to Embodiment 1 of the present application. The method for generating a natural language model includes:

[0051] Step S201, obtain a sample question, and obtain a reasoning path set corresponding to the sample question, wherein the reasoning path set includes at least: a reference reasoning path and a reasoning path branch pair, the reference reasoning path includes true answer information for answering the sample question, the reasoning path branch pair includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the true answer information, and the second reasoning path branch is a path branch that cannot lead to the true answer information.

[0052] Optionally, obtain sample questions. It should be noted that there are multiple sample questions. For example, assuming that the natural language model needs to go through multiple reasoning steps S1, S2, ..., Sn to get the final answer A, then the reasoning path P is defined as the sequence of these steps: P = (S1, S2, ..., Sn), where each Si is a natural language sentence and the last step Sn should contain the answer A. Obtain the reference reasoning path corresponding to the sample question. It should be noted that the reference reasoning path is the reasoning path that can obtain the real answer information corresponding to the question information, that is, the above reference reasoning path includes the real answer information for answering the sample question.

[0053] Obtain the reasoning path branch pair corresponding to the sample question. It should be noted that the reasoning path branch pair includes a first reasoning path branch and a second reasoning path branch. The first reasoning path branch is a path branch leading to the true answer information, and the second reasoning path branch is a path branch that cannot lead to the true answer information. In an optional embodiment, the reasoning path branch pair is obtained based on the reference reasoning path corresponding to the sample question. By training the natural language model with reference to the reasoning path and the first reasoning path branch and the second reasoning path branch, the diversified reasoning paths of the model are explored, which can effectively improve the reasoning ability of the model.

[0054] Step S202, constructing a target training set based on the sample questions and the reasoning path set;

[0055] Step S203, training the initial natural language model according to the target training set to obtain a target natural language model, wherein the target natural language model is used to answer the question information input by the target object.

[0056] Optionally, a target training set is constructed through sample questions, reference reasoning paths, and reasoning path branches, and then the initial natural language model is trained through the target training set to obtain a target natural language model.

[0057] It should be noted that the initial natural language model is a natural language model that has undergone basic training and has certain reasoning capabilities. The initial natural language model is fine-tuned through the above steps S201 to S203 to obtain the target natural language model, that is, the above steps S201 to S203 are downstream tasks.

[0058] To summarize, by performing reasoning analysis based on sample questions to obtain the corresponding set of reasoning paths, and training the initial natural language model through sample questions and the set of reasoning paths corresponding to the sample questions to obtain the target natural language model, we can effectively explore the potential solution space of given problems and enhance the inherent reasoning ability of the natural language model, thereby achieving the effect of improving the accuracy of the natural language model in answering questions.

[0059] In order to improve the efficiency of obtaining a set of reasoning paths, in the method for generating a natural language model provided in Example 1 of the present application, obtaining a set of reasoning paths corresponding to sample questions includes: generating a thinking chain example based on the sample question and the thinking chain prompt, wherein the thinking chain example includes: a sample question and a thinking chain reasoning path corresponding to the sample question, and the thinking chain reasoning path includes real answer information for answering the sample question; processing the sample question and the thinking chain example through the initial natural language model to determine whether a reference reasoning path corresponding to the sample question can be obtained; when the reference reasoning path corresponding to the sample question can be obtained, obtaining a reasoning path branch pair based on the sample question, the reference reasoning path and the thinking chain example; obtaining a reasoning path set based on the reference reasoning path and the reasoning path branch pair.

[0060] Optionally, a thought chain example is generated based on the sample question and the thought chain. For example, given a question Q, a thought chain prompt is used to generate a reasoning path. The thought chain example can be composed of m real examples, each of which is composed of a question and its corresponding thought chain reasoning path. It should be noted that the thought chain prompt is a training method for the model to learn how to reason and solve problems step by step, rather than relying solely on memory and pattern recognition.

[0061] Then, the sample questions and thought chain examples are input into the initial natural language model to determine whether a reference reasoning path can be obtained through the initial natural language model.

[0062] In an optional embodiment, the sample question and the thought chain example are processed by an initial natural language model to determine whether a reference reasoning path corresponding to the sample question can be obtained, including: processing the sample question and the thought chain example by an initial natural language model to obtain a first reasoning path; judging whether a reference reasoning path corresponding to the sample question can be obtained based on whether the first reasoning path can be inferred to obtain true answer information.

[0063] After the sample question and the thought chain example are input into the initial natural language model, the first reasoning path is generated by the initial natural language model. If the final answer of the generated path is correct, the path is judged to be correct, that is, the reference reasoning path corresponding to the sample question can be obtained. If the correct path is not obtained after multiple attempts, it is considered that the problem is far beyond the ability of the model and the sample problem is removed from the training set.

[0064] After determining that a reference reasoning path corresponding to the sample problem can be obtained, a pair of reasoning path branches is obtained based on the sample problem, the reference reasoning path and the thinking chain example. For example, given a sample problem, a partial path of the reference reasoning path and a thinking chain example, temperature sampling is used to obtain a variety of branches from the current point of the reasoning path to obtain the above-mentioned pair of reasoning path branches.

[0065] Finally, a set of reasoning paths is obtained based on the reference reasoning paths and the reasoning path branch pairs.

[0066] In summary, by deriving the correct reasoning steps from the basic model as the reference reasoning path, we avoid manually obtaining the true reasoning path annotations, and obtain the reasoning path branch pairs from each step of the reference reasoning path, thereby improving the overall reasoning ability of the large language model.

[0067] How to obtain the above-mentioned reasoning branches is also crucial. Therefore, in the natural language model generation method provided in Example 1 of the present application, based on sample questions, reference reasoning paths and thinking chain examples, obtaining a reasoning path branch pair includes: based on the reference reasoning path, obtaining a reference reasoning sub-path set; based on the sample questions, the reference reasoning sub-path set and the thinking chain example, taking the last reasoning step in the reference reasoning sub-path of the reference reasoning sub-path set as the starting point to sample and generate reasoning path branches, and obtaining multiple reasoning path branches corresponding to the reference reasoning sub-path of the reference reasoning sub-path set; based on the multiple reasoning path branches, obtaining a first reasoning path branch and a second reasoning path branch; based on the first reasoning path branch and the second reasoning path branch, obtaining a reasoning path branch pair.

[0068] Optionally, according to the reference reasoning path, a reference reasoning sub-path set is obtained, that is, the first few steps of the reasoning path generated according to the reference reasoning path (S1, S2, ..., Si−1) are constructed as a reference reasoning sub-path set. For example, the reference reasoning path includes (S1, S2, S3, S4), and the corresponding reference reasoning sub-path set includes: reference reasoning sub-path 1 (S1), reference reasoning sub-path 2 (S1, S2) and reference reasoning sub-path 3 (S1, S2, S3).

[0069] After obtaining the above-mentioned reference reasoning subpath set, based on the sample problem, the reference reasoning subpath set and the thought chain example, the reasoning path branch is sampled and generated with the last reasoning step in the reference reasoning subpath of the reference reasoning subpath set as the starting point. For example, given the problem Q, the thought chain example D and the first few steps (S1, S2, ..., Si−1) of the reference reasoning path, temperature sampling is used to obtain diversified branches from the current point. Each branch Bi should contain the content from the current step to the final step, that is, multiple reasoning path branches corresponding to the above-mentioned reference reasoning subpath are obtained.

[0070] The goal of generating multiple reasoning path branches is to obtain a favorable branch Bi+ (i.e., the first reasoning path branch mentioned above) and an unfavorable branch Bi- (i.e., the second reasoning path branch mentioned above), wherein the favorable branch leads to the correct final answer, while the unfavorable branch does not. Therefore, starting from each step Si, multiple branches are iteratively sampled and each branch is verified until a favorable branch and an unfavorable branch are obtained, thereby forming a reasoning branch pair (Bi+, Bi-) (i.e., the reasoning path branch pair mentioned above). However, if a branch pair cannot be formed after sampling a maximum of ten branches, the problem is removed from the training set. This ensures that the training data contains only those problems that the model can solve by comparing favorable and unfavorable branches.

[0071] Through the above steps, the potential solution space of a given problem can be effectively explored, and the overall reasoning ability of the large language model can be improved through reasoning path branching.

[0072] In order to improve the reasoning ability of the target natural language model, in the natural language model generation method provided in Example 1 of the present application, the initial natural language model is trained according to the target training set to obtain the target natural language model, including: processing the target training set through the initial natural language model to obtain a target prediction reasoning path; obtaining a target loss function based on the target prediction reasoning path; training the initial natural language model according to the target loss function to obtain the target natural language model.

[0073] Optionally, the target training set is input into an initial natural language model, and the initial natural language model performs reasoning analysis on sample questions in the target training set to obtain a target prediction reasoning path.

[0074] In an optional embodiment, the target training set is processed through the initial natural language model to obtain a target prediction reasoning path, including: processing the sample question through the initial natural language model to obtain a second reasoning path; processing the sample question and a reference reasoning sub-path set of the reference reasoning path through the initial natural language model to obtain a third reasoning path; and obtaining the target prediction reasoning path based on the second reasoning path and the third reasoning path.

[0075] After the target training set is input into the initial natural language model, the initial natural language model performs reasoning analysis on the sample question to obtain a second reasoning path. The initial natural language model then performs reasoning analysis on the sample question and a set of reference reasoning sub-paths of the reference reasoning path to obtain a third reasoning path. Finally, based on the second reasoning path and the third reasoning path, the above-mentioned target prediction reasoning path is obtained.

[0076] Then, according to the target prediction reasoning path, the target loss function is obtained, for example, the target prediction reasoning path and the reference reasoning path or reasoning path branch are used to calculate the loss to obtain the above target loss function. Finally, the initial natural language model is trained by the target loss function to obtain the target natural language model.

[0077] By calculating the loss between the target prediction reasoning path and the reference reasoning path or the reasoning path branch, it is possible to provide comparative feedback on the reasoning path to reward the correct branch and punish the wrong branch, thereby enhancing the reasoning ability of the model.

[0078] In order to improve the accuracy of calculating the loss function, in the natural language model generation method provided in Example 1 of the present application, the target loss function is obtained based on the target prediction reasoning path, including: calculating based on the second reasoning path and the reference reasoning path to obtain the reference path loss function; calculating based on the third reasoning path and the reasoning path branch pair to obtain the branch loss function; obtaining the target loss function based on the reference path loss function and the branch loss function.

[0079] Optionally, the reference path loss function is calculated using the second reasoning path and the reference reasoning path, for example, the reference path loss function is calculated using a standard language modeling loss. It should be noted that the standard language modeling loss is mainly used to evaluate the difference between the sequence output by the model and the true loss, such as the common cross entropy loss.

[0080] Then, the branch loss function is calculated based on the third reasoning path and the reasoning path branch pair. For example, the third reasoning path and the reasoning path branch pair are similarly calculated and summed to obtain the above branch loss function. For example, the branch loss function can also be calculated by the logarithmic odds ratio between the favorable branch Bi+ and the unfavorable branch Bi−.

[0081] Finally, by referring to the path loss function and the branch loss function, the target loss function is obtained. For example, the target loss function is calculated using the following formula:

[0082]

[0083] in, is the target loss function mentioned above, is the reference path loss function mentioned above, is the branch loss function mentioned above, is the hyperparameter weight.

[0084] The reference path loss function and the branch loss function provide comparative feedback of the reasoning path, thereby enhancing the reasoning ability of the model.

[0085] In order to improve the accuracy of branch loss, in the natural language model generation method provided in Example 1 of the present application, calculation is performed based on the third reasoning path and the reasoning path branch to obtain a branch loss function, including: calculating based on the third reasoning path and the first reasoning path branch to obtain a first loss function; calculating based on the third reasoning path and the second reasoning path branch to obtain a second loss function; calculating based on the first loss function and the second loss function to obtain a branch loss function.

[0086] Optionally, given the input problem Q and the first i-1 steps P1:i−1 of the reference path, the ratio between the probability of generating the branch and the probability of not generating the branch is used as the loss function of the reasoning path branch. For example, the probability of generating the branch can be determined by the difference between the third reasoning path and the first reasoning path branch, that is, the first loss function is calculated by the third reasoning path and the first reasoning path branch, and the second loss function is calculated by the third reasoning path and the second reasoning path branch.

[0087] Then, the loss is calculated by the branch of step i. It is calculated by the log-odds ratio between the favorable branch Bi+ and the unfavorable branch Bi−, that is, the branch loss of the i-th step is obtained by the ratio of the first loss function and the second loss function of the i-th step.

[0088] Finally, the above branch loss function is obtained by summing up the branch losses corresponding to each step in the reasoning path. For example, the branch loss function is shown in the following formula:

[0089]

[0090] The reasoning path includes n reasoning steps, and logσ is the logarithmic form of the sigmoid function.

[0091] By calculating the branch loss, the natural language model can learn more effectively from the explored reasoning paths, thereby improving the reasoning ability.

[0092] In an optional embodiment, the following may be used: Figure 3 The flowchart shown in the figure obtains the above-mentioned target natural language model, which mainly includes three stages: generation stage, exploration stage and optimization stage. The generation stage is to derive the correct reasoning steps from the basic model as the reference reasoning path. The exploration stage is to gradually create branches from each step of the reference reasoning path, thereby obtaining multiple favorable and unfavorable reasoning branches, which will be used to provide comparative feedback to the model. The optimization stage optimizes the natural language model according to the reference reasoning path and the explored branches to enhance the intrinsic reasoning ability of the basic model.

[0093] In an optional embodiment, the following may be used: Figure 4 The reference reasoning path and reasoning branch path are shown as follows: Given a question Q, a reasoning path is generated using a chain of thoughts prompt. The chain of thoughts example can consist of m real examples, each of which consists of a question and its corresponding reasoning path. Let M be the base model, and sample the reference reasoning path P by prompting the model with a chain of thoughts demonstration D and a given question Q. If the generated path has the correct answer in the end, the path is judged to be correct. If no suitable path is obtained after multiple attempts, it is considered that the problem is far beyond the ability of the model and is removed from the training set. In order to consider the potential errors that may occur in each reasoning step, multiple branches are explored at each step. Specifically, given the question Q, the chain of thoughts demonstration D, and the first few steps (S1, S2, ..., Si−1) of the generated reasoning path, a variety of branches are obtained from the current point of the reasoning path using temperature sampling. Each branch Bi should contain the content from the current step to the final step, and obtain a favorable branch Bi+ and an unfavorable branch Bi-, where the favorable branch leads to the correct final answer, while the unfavorable branch does not. To this end, starting from each step Si, multiple branches are iteratively sampled and each branch is verified until a favorable branch and an unfavorable branch are obtained, thereby forming an inference branch pair (Bi+, Bi-), and finally obtaining a reference inference path and an inference branch path.

[0094] In the method for generating a natural language model provided in Example 1 of the present application, a sample question is obtained, and a set of reasoning paths corresponding to the sample question is obtained, wherein the reasoning path set at least includes: a reference reasoning path and a reasoning path branch pair, the reference reasoning path includes real answer information for answering the sample question, the reasoning path branch pair includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the real answer information, and the second reasoning path branch is a path branch that cannot lead to the real answer information; the target training set is constructed based on the sample question and the reasoning path set; the initial natural language model is trained based on the target training set to obtain the target natural language model, wherein the target natural language model is used to answer the question information input by the target object, thereby solving the technical problem in the related art that the accuracy of the natural language model in answering questions is relatively low due to the poor reasoning ability of the natural language model.

[0095] In this scheme, reasoning analysis is performed based on sample questions to obtain a corresponding set of reasoning paths. The initial natural language model is trained using sample questions and the set of reasoning paths corresponding to the sample questions to obtain a target natural language model. This can effectively explore the potential solution space of a given problem and enhance the inherent reasoning ability of the natural language model, thereby achieving the effect of improving the accuracy of the natural language model in answering questions.

[0096] Example 2

[0097] According to an embodiment of the present application, a data processing method is also provided, such as Figure 5 As shown, the data processing method includes:

[0098] Step S501, obtaining question information input by the target object.

[0099] Optionally, the user (i.e., the target object mentioned above) can input the question information through the user interface. It should be noted that the question information mentioned above is a question that requires multi-step reasoning, such as a math word problem. For example, given a question Q posed in the form of natural language text, the goal is to generate a final answer A in the form of natural language text.

[0100] Step S502, perform reasoning analysis on the question information through the target natural language model to obtain a target reasoning path, wherein the target natural language model is obtained according to the natural language model generation method provided in Example 1 of the present application, the target reasoning path consists of logical steps of deriving from the question information to the answer information corresponding to the question information, the target natural language model is obtained from the target training set, and the target training set consists of sample questions and a set of reasoning paths corresponding to the sample questions.

[0101] Optionally, the above question information is input into a target natural language model, and the target natural language model is used to perform reasoning analysis on the question information to obtain the above target reasoning path.

[0102] For example, assuming that the target natural language model needs to go through multiple reasoning steps S1, S2, ..., Sn to obtain the final answer A, the reasoning path P is defined as a sequence of these steps: P = (S1, S2, ..., Sn), where each Si is a natural language sentence and the last step Sn should contain the answer A.

[0103] It should be noted that the above-mentioned target natural language model is trained through sample questions and a set of reasoning paths corresponding to the sample questions. The inherent reasoning ability of the natural language model can be enhanced through sample questions and a set of reasoning paths corresponding to the sample questions.

[0104] Step S503: Return the target reasoning path to the target object.

[0105] Optionally, the target reasoning path output by the natural language model is returned to the target object.

[0106] To summarize, after receiving the question information, the target natural language model will perform reasoning analysis based on the question information to obtain the corresponding target reasoning path. The target natural language model is obtained by training with sample questions and the set of reasoning paths corresponding to the sample questions. This can enhance the inherent reasoning ability of the natural language model, thereby achieving the effect of improving the accuracy of the natural language model's answers to questions.

[0107] In order to improve the accuracy of reply information, in the data processing method provided in Example 2 of the present application, the question information is inferred and analyzed through the target natural language model to obtain the target reasoning path, including: performing reasoning analysis based on the question information through the target natural language model to generate multiple candidate reasoning paths, wherein the candidate reasoning paths include reply information corresponding to the question information; and determining the target reasoning path from the multiple candidate reasoning paths through the target natural language model.

[0108] Optionally, after the question information is input into the target natural language model, the target natural language model understands the meaning and context of the question information, and generates possible reasoning paths (i.e., the multiple candidate reasoning paths mentioned above) based on the meaning and context. It should be noted that these paths are logical steps from known questions to answers.

[0109] After obtaining multiple candidate reasoning paths, the target natural language model determines the target reasoning path from the multiple candidate reasoning paths based on the relevance, logical consistency and probability of the candidate reasoning paths.

[0110] By performing reasoning analysis on question information through the target natural language model to generate candidate reasoning paths, and obtaining the target reasoning path from the candidate reasoning paths, the accuracy of answering question information can be effectively improved.

[0111] In the data processing method provided in the first embodiment of the present application, the question information input by the target object is obtained; the question information is reasoned and analyzed by the target natural language model to obtain the target reasoning path, wherein the target reasoning path is composed of logical steps derived from the question information to the answer information corresponding to the question information, and the target natural language model is obtained by the target training set, and the target training set is composed of sample questions and a set of reasoning paths corresponding to the sample questions; the target reasoning path is returned to the target object, which solves the technical problem in the related art that the accuracy of the natural language model in answering questions is relatively low due to the poor reasoning ability of the natural language model. In this scheme, after receiving the question information, the target natural language model will perform reasoning analysis based on the question information to obtain the corresponding target reasoning path, and the target natural language model is obtained by training the sample questions and the set of reasoning paths corresponding to the sample questions, which can enhance the inherent reasoning ability of the natural language model, thereby achieving the effect of improving the accuracy of the natural language model in answering questions.

[0112] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0114] Example 3

[0115] According to an embodiment of the present application, a data processing method is also provided, such as Figure 6 As shown, the data processing method includes:

[0116] Step S601, obtaining question information sent by the client;

[0117] Step S602: In the cloud server, the question information is analyzed by reasoning through a target natural language model to obtain a target reasoning path, wherein the target natural language model is obtained according to the method for generating a natural language model disclosed in the first embodiment, the target reasoning path is composed of logical steps for deriving from the question information to the answer information corresponding to the question information, the target natural language model is obtained from a target training set, and the target training set is composed of sample questions and a set of reasoning paths corresponding to the sample questions;

[0118] Step S603: Return the target reasoning path to the client.

[0119] It should be noted that the specific method for processing the problem information in the cloud server is the same as that in implementation 1 and will not be repeated here.

[0120] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0121] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0122] Example 4

[0123] According to an embodiment of the present application, a natural language model generation device for implementing the above-mentioned natural language model generation method is also provided. Figure 7 As shown, the device includes: a first acquisition unit 701, a construction unit 702 and a training unit 703.

[0124] The first acquisition unit 701 is used to acquire a sample question and acquire a set of reasoning paths corresponding to the sample question, wherein the set of reasoning paths at least includes: a reference reasoning path and a pair of reasoning path branches, the reference reasoning path includes true answer information for answering the sample question, the pair of reasoning path branches includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the true answer information, and the second reasoning path branch is a path branch that cannot lead to the true answer information;

[0125] A construction unit 702, configured to construct the target training set according to the sample question and the reasoning path set;

[0126] The training unit 703 is used to train the initial natural language model according to the target training set to obtain the target natural language model, wherein the target natural language model is used to answer the question information input by the target object.

[0127] In the natural language model generation device provided in Example 4 of the present application, a sample question is obtained through a first acquisition unit 701, and a reasoning path set corresponding to the sample question is obtained, wherein the reasoning path set at least includes: a reference reasoning path and a reasoning path branch pair, the reference reasoning path includes real answer information for answering the sample question, the reasoning path branch pair includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the real answer information, and the second reasoning path branch is a path branch that cannot lead to the real answer information; a construction unit 702 constructs the target training set based on the sample question and the reasoning path set; a training unit 703 trains the initial natural language model based on the target training set to obtain the target natural language model, wherein the target natural language model is used to answer the question information input by the target object, which solves the technical problem in the related art that the accuracy of the natural language model in answering questions is relatively low due to the poor reasoning ability of the natural language model. In this solution, after receiving the question information, the target natural language model will perform reasoning analysis based on the question information to obtain the corresponding target reasoning path. The target natural language model is obtained by training with sample questions and a set of reasoning paths corresponding to the sample questions. This can enhance the inherent reasoning ability of the natural language model, thereby achieving the effect of improving the accuracy of the natural language model's answers to questions.

[0128] Optionally, in the natural language model generation device provided in Example 4 of the present application, the first acquisition unit includes: a generation subunit, which is used to generate a thinking chain example based on the sample question and the thinking chain prompt, wherein the thinking chain example includes: the sample question and the thinking chain reasoning path corresponding to the sample question, and the thinking chain reasoning path includes real answer information for answering the sample question; a processing subunit, which is used to process the sample question and the thinking chain example through the initial natural language model to determine whether the reference reasoning path corresponding to the sample question can be obtained; a first determination subunit, which is used to obtain the reasoning path branch pair based on the sample question, the reference reasoning path and the thinking chain example when the reference reasoning path corresponding to the sample question can be obtained; a second determination subunit, which is used to obtain the reasoning path set based on the reference reasoning path and the reasoning path branch pair.

[0129] Optionally, in the natural language model generation device provided in Example 4 of the present application, the processing subunit includes: a processing module, used to process the sample question and the thinking chain example through the initial natural language model to obtain a first reasoning path; a judgment module, used to judge whether the reference reasoning path corresponding to the sample question can be obtained based on whether the first reasoning path can be inferred to obtain the true answer information.

[0130] Optionally, in the natural language model generation device provided in Example 4 of the present application, the first determination sub-unit includes: a first determination module, used to obtain a reference reasoning sub-path set based on the reference reasoning path; a sampling module, used to sample and generate reasoning path branches based on the sample problem, the reference reasoning sub-path set and the thinking chain example, taking the last reasoning step in the reference reasoning sub-path of the reference reasoning sub-path set as the starting point, to obtain multiple reasoning path branches corresponding to the reference reasoning sub-path of the reference reasoning sub-path set; a second determination module, used to obtain the first reasoning path branch and the second reasoning path branch based on the multiple reasoning path branches; and a third determination module, used to obtain the reasoning path branch pair based on the first reasoning path branch and the second reasoning path branch.

[0131] Optionally, in the natural language model generation device provided in Example 4 of the present application, the training unit includes: a processing subunit, used to process the target training set through the initial natural language model to obtain a target prediction reasoning path; a third determination subunit, used to obtain a target loss function based on the target prediction reasoning path; and a training subunit, used to train the initial natural language model based on the target loss function to obtain the target natural language model.

[0132] Optionally, in the natural language model generation device provided in Example 4 of the present application, the processing subunit includes: a first processing module, used to process the sample question through the initial natural language model to obtain a second reasoning path; a second processing module, used to process the sample question and a reference reasoning sub-path set of the reference reasoning path through the initial natural language model to obtain a third reasoning path; and a fourth determination module, used to obtain the target prediction reasoning path based on the second reasoning path and the third reasoning path.

[0133] Optionally, in the natural language model generation device provided in Example 4 of the present application, the third determination subunit includes: a first calculation module, used to perform calculations based on the second reasoning path and the reference reasoning path to obtain a reference path loss function; a second calculation module, used to perform calculations based on the third reasoning path and the reasoning path branch pairs to obtain a branch loss function; and a fifth determination module, used to obtain the target loss function based on the reference path loss function and the branch loss function.

[0134] Optionally, in the natural language model generation device provided in Example 4 of the present application, the second calculation module includes: a first calculation sub-module, used to perform calculations based on the third reasoning path and the first reasoning path branch to obtain a first loss function; a second calculation sub-module, used to perform calculations based on the third reasoning path and the second reasoning path branch to obtain a second loss function; a third calculation sub-module, used to perform calculations based on the first loss function and the second loss function to obtain the branch loss function.

[0135] It should be noted that the first acquisition unit 701, the construction unit 702 and the training unit 703 described above correspond to steps S201 to S203 in the first embodiment, and the three units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the first embodiment. It should be noted that the above modules as part of the device can be run in the computer terminal 10 provided in the first embodiment.

[0136] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0137] Example 5

[0138] According to an embodiment of the present application, a data processing device for implementing the above data processing method is also provided. Figure 8 As shown, the device includes: a second acquisition unit 801, an inference unit 802 and a return unit 803.

[0139] The second acquisition unit 801 is used to acquire question information input by the target object;

[0140] The inference unit 802 is used to perform inference analysis on the question information through a target natural language model to obtain a target inference path, wherein the target natural language model is obtained according to the method for generating a natural language model provided in an embodiment of the present application, the target inference path consists of logical steps of deriving from the question information to the answer information corresponding to the question information, the target natural language model is obtained from a target training set, and the target training set consists of sample questions and a set of inference paths corresponding to the sample questions;

[0141] The returning unit 803 is used to return the target reasoning path to the target object.

[0142] In the data processing device provided in the fifth embodiment of the present application, the question information input by the target object is obtained by the second acquisition unit 801; the reasoning unit 802 performs reasoning analysis on the question information through the target natural language model to obtain the target reasoning path, wherein the target reasoning path is composed of logical steps derived from the question information to the answer information corresponding to the question information, and the target natural language model is obtained from the target training set, and the target training set is composed of sample questions and the reasoning path set corresponding to the sample questions; the return unit 803 returns the target reasoning path to the target object, which solves the technical problem in the related art that the accuracy of the natural language model answering questions is relatively low due to the poor reasoning ability of the natural language model. In this scheme, after receiving the question information, the target natural language model will perform reasoning analysis based on the question information to obtain the corresponding target reasoning path, and the target natural language model is obtained by training the sample questions and the reasoning path set corresponding to the sample questions, which can enhance the inherent reasoning ability of the natural language model, thereby achieving the effect of improving the accuracy of the natural language model answering questions.

[0143] Optionally, in the data processing device provided in Example 5 of the present application, the reasoning unit includes: a reasoning sub-unit, used to perform reasoning analysis based on question information through a target natural language model to generate multiple candidate reasoning paths, wherein the candidate reasoning paths include answer information corresponding to the question information; and a fourth determination sub-unit, used to determine a target reasoning path from multiple candidate reasoning paths through a target natural language model.

[0144] It should be noted that the second acquisition unit 801, the reasoning unit 802 and the return unit 803 described above correspond to steps S501 to S503 in the second embodiment, and the three units and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in the first embodiment. It should be noted that the above modules, as part of the device, can be run in the computer terminal 10 provided in the first embodiment.

[0145] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in Example 1, as well as the application scenario and implementation process, but is not limited to the scheme provided in Example 1.

[0146] Example 6

[0147] The embodiment of the present application may provide an electronic device, which may be any electronic device in an electronic device terminal group. Optionally, in this embodiment, the electronic device may also be replaced by a terminal device such as a mobile terminal.

[0148] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0149] In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the data processing method: obtain sample questions, and obtain a set of reasoning paths corresponding to the sample questions, wherein the reasoning path set at least includes: a reference reasoning path and a reasoning path branch pair, the reference reasoning path includes real answer information for answering the sample questions, the reasoning path branch pair includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the real answer information, and the second reasoning path branch is a path branch that cannot lead to the real answer information; construct a target training set based on the sample questions and the reasoning path set; train the initial natural language model based on the target training set to obtain a target natural language model, wherein the target natural language model is used to answer the question information input by the target object.

[0150] The above-mentioned electronic device can execute the program code of the following steps in the data processing method: obtaining a set of reasoning paths corresponding to sample questions includes: generating a thinking chain example based on the sample questions and thinking chain prompts, wherein the thinking chain example includes: the sample question and the thinking chain reasoning path corresponding to the sample question, and the thinking chain reasoning path includes the real answer information of answering the sample question; processing the sample question and the thinking chain example through the initial natural language model to determine whether a reference reasoning path corresponding to the sample question can be obtained; when the reference reasoning path corresponding to the sample question can be obtained, obtaining a reasoning path branch pair based on the sample question, the reference reasoning path and the thinking chain example; obtaining a reasoning path set based on the reference reasoning path and the reasoning path branch pair.

[0151] The above-mentioned electronic device can execute the program code of the following steps in the data processing method: processing the sample question and the thinking chain example through the initial natural language model, and judging whether the reference reasoning path corresponding to the sample question can be obtained, including: processing the sample question and the thinking chain example through the initial natural language model to obtain the first reasoning path; judging whether the reference reasoning path corresponding to the sample question can be obtained based on whether the first reasoning path can infer the true answer information.

[0152] The above-mentioned electronic device can execute the program code of the following steps in the data processing method: based on the sample problem, the reference reasoning path and the thinking chain example, obtaining the reasoning path branch pair includes: based on the reference reasoning path, obtaining a reference reasoning sub-path set; based on the sample problem, the reference reasoning sub-path set and the thinking chain example, taking the last reasoning step in the reference reasoning sub-path of the reference reasoning sub-path set as the starting point to sample and generate reasoning path branches, and obtaining multiple reasoning path branches corresponding to the reference reasoning sub-path of the reference reasoning sub-path set; based on the multiple reasoning path branches, obtaining a first reasoning path branch and a second reasoning path branch; based on the first reasoning path branch and the second reasoning path branch, obtaining a reasoning path branch pair.

[0153] The above-mentioned electronic device can execute the program code of the following steps in the data processing method: training the initial natural language model according to the target training set to obtain the target natural language model includes: processing the target training set through the initial natural language model to obtain the target prediction reasoning path; obtaining the target loss function according to the target prediction reasoning path; training the initial natural language model according to the target loss function to obtain the target natural language model.

[0154] The above-mentioned electronic device can execute the program code of the following steps in the data processing method: processing the target training set through the initial natural language model to obtain the target prediction reasoning path including: processing the sample question through the initial natural language model to obtain the second reasoning path; processing the sample question and the reference reasoning sub-path set of the reference reasoning path through the initial natural language model to obtain the third reasoning path; obtaining the target prediction reasoning path based on the second reasoning path and the third reasoning path.

[0155] The above-mentioned electronic device can execute the program code of the following steps in the data processing method: obtaining the target loss function based on the target prediction reasoning path includes: calculating based on the second reasoning path and the reference reasoning path to obtain the reference path loss function; calculating based on the third reasoning path and the reasoning path branch pair to obtain the branch loss function; obtaining the target loss function based on the reference path loss function and the branch loss function.

[0156] The above-mentioned electronic device can execute the program code of the following steps in the data processing method: calculating based on the third reasoning path and the reasoning path branch to obtain the branch loss function includes: calculating based on the third reasoning path and the first reasoning path branch to obtain the first loss function; calculating based on the third reasoning path and the second reasoning path branch to obtain the second loss function; calculating based on the first loss function and the second loss function to obtain the branch loss function.

[0157] The above-mentioned electronic device can execute the program code of the following steps in the data processing method: obtaining question information input by the target object; performing reasoning analysis on the question information through the target natural language model to obtain a target reasoning path, wherein the target reasoning path is composed of logical steps from the question information to the answer information corresponding to the question information, the target natural language model is obtained from the target training set, and the target training set is composed of sample questions and a set of reasoning paths corresponding to the sample questions; and returning the target reasoning path to the target object.

[0158] The above-mentioned electronic device can execute the program code of the following steps in the data processing method: performing reasoning analysis on the question information through the target natural language model to obtain the target reasoning path including: performing reasoning analysis based on the question information through the target natural language model to generate multiple candidate reasoning paths, wherein the candidate reasoning paths include answer information corresponding to the question information; determining the target reasoning path from multiple candidate reasoning paths through the target natural language model.

[0159] Optionally, Fig. 9 is a structural block diagram of an electronic device according to an embodiment of the present application. Fig. 9 As shown, the electronic device 90 may include: one or more ( Fig. 9 Only one is shown in the figure) processor 902, memory 904. The electronic device 90 may also include a storage controller, through which the memory 904 is controlled and managed; the electronic device 90 may also include a peripheral interface, through which the radio frequency module, audio module and display screen are connected.

[0160] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the data processing method and device in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned data processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the electronic device 70 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: obtain sample questions and obtain a set of reasoning paths corresponding to the sample questions, wherein the reasoning path set at least includes: a reference reasoning path and a reasoning path branch pair, the reference reasoning path includes real answer information for answering the sample questions, the reasoning path branch pair includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the real answer information, and the second reasoning path branch is a path branch that cannot lead to the real answer information; construct a target training set based on the sample questions and the reasoning path set; train the initial natural language model based on the target training set to obtain a target natural language model, wherein the target natural language model is used to answer the question information input by the target object.

[0162] Optionally, the processor may also execute program code for the following steps: obtaining a set of reasoning paths corresponding to sample questions, including: generating a thinking chain example based on the sample question and the thinking chain prompt, wherein the thinking chain example includes: the sample question and the thinking chain reasoning path corresponding to the sample question, and the thinking chain reasoning path includes the real answer information for answering the sample question; processing the sample question and the thinking chain example through the initial natural language model to determine whether a reference reasoning path corresponding to the sample question can be obtained; when the reference reasoning path corresponding to the sample question can be obtained, obtaining a pair of reasoning path branches based on the sample question, the reference reasoning path and the thinking chain example; obtaining a set of reasoning paths based on the reference reasoning path and the pair of reasoning path branches.

[0163] Optionally, the processor may also execute program code of the following steps: processing sample questions and thought chain examples through an initial natural language model, and determining whether a reference reasoning path corresponding to the sample question can be obtained, including: processing sample questions and thought chain examples through an initial natural language model to obtain a first reasoning path; determining whether a reference reasoning path corresponding to the sample question can be obtained based on whether the first reasoning path can infer true answer information.

[0164] Optionally, the processor may also execute program code for the following steps: based on sample questions, reference reasoning paths and thinking chain examples, obtaining a pair of reasoning path branches including: based on the reference reasoning path, obtaining a reference reasoning sub-path set; based on the sample questions, the reference reasoning sub-path set and thinking chain examples, taking the last reasoning step in the reference reasoning sub-path of the reference reasoning sub-path set as the starting point to sample and generate reasoning path branches, obtaining multiple reasoning path branches corresponding to the reference reasoning sub-path of the reference reasoning sub-path set; based on the multiple reasoning path branches, obtaining a first reasoning path branch and a second reasoning path branch; based on the first reasoning path branch and the second reasoning path branch, obtaining a pair of reasoning path branches.

[0165] Optionally, the processor may also execute the program code of the following steps: training the initial natural language model according to the target training set to obtain the target natural language model, including: processing the target training set through the initial natural language model to obtain a target prediction reasoning path; obtaining a target loss function according to the target prediction reasoning path; training the initial natural language model according to the target loss function to obtain the target natural language model.

[0166] Optionally, the processor may also execute program code of the following steps: processing the target training set through an initial natural language model to obtain a target prediction reasoning path including: processing the sample question through an initial natural language model to obtain a second reasoning path; processing the sample question and a reference reasoning sub-path set of the reference reasoning path through an initial natural language model to obtain a third reasoning path; and obtaining a target prediction reasoning path based on the second reasoning path and the third reasoning path.

[0167] Optionally, the processor may also execute the following program code steps: obtaining a target loss function based on the target prediction reasoning path, including: calculating based on the second reasoning path and the reference reasoning path to obtain a reference path loss function; calculating based on the third reasoning path and the reasoning path branch pair to obtain a branch loss function; obtaining a target loss function based on the reference path loss function and the branch loss function.

[0168] Optionally, the processor may also execute the program code of the following steps: performing calculations based on the third reasoning path and the reasoning path branch to obtain a branch loss function, including: performing calculations based on the third reasoning path and the first reasoning path branch to obtain a first loss function; performing calculations based on the third reasoning path and the second reasoning path branch to obtain a second loss function; performing calculations based on the first loss function and the second loss function to obtain a branch loss function.

[0169] Optionally, the processor may also execute the following program code: obtaining question information input by the target object; performing reasoning analysis on the question information through a target natural language model to obtain a target reasoning path, wherein the target reasoning path is composed of logical steps for deriving from the question information to the answer information corresponding to the question information, the target natural language model is obtained from a target training set, and the target training set is composed of sample questions and a set of reasoning paths corresponding to the sample questions; and returning the target reasoning path to the target object.

[0170] Optionally, the processor may also execute program code for the following steps: performing reasoning analysis on the question information through a target natural language model to obtain a target reasoning path, including: performing reasoning analysis based on the question information through a target natural language model to generate multiple candidate reasoning paths, wherein the candidate reasoning paths include answer information corresponding to the question information; and determining a target reasoning path from multiple candidate reasoning paths through a target natural language model.

[0171] It can be understood by those skilled in the art that Fig. 9 The structure shown is for illustration only, and the electronic device 90 may also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), or a PAD. Fig. 9 The structure of the electronic device is not limited. Fig. 9 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Fig. 9 Different configurations are shown.

[0172] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0173] Example 7

[0174] The embodiment of the present application also provides a computer program product. Optionally, in this embodiment, the computer program product can be used to store the program code executed by the method for generating a natural language model provided in the first embodiment and the program code executed by the data processing method provided in the second embodiment.

[0175] Optionally, in this embodiment, the computer program product may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0176] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0177] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0181] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc., which can store program code.

[0182] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for generating a natural language model, characterized in that: include: Obtain a sample question, and obtain a set of reasoning paths corresponding to the sample question, wherein the set of reasoning paths at least includes: a reference reasoning path and a pair of reasoning path branches, the reference reasoning path includes true answer information for answering the sample question, the pair of reasoning path branches includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the true answer information, and the second reasoning path branch is a path branch that cannot lead to the true answer information; Constructing a target training set according to the sample problem and the reasoning path set; Training the initial natural language model according to the target training set to obtain a target natural language model, wherein the target natural language model is used to answer question information input by the target object; Wherein, obtaining a set of reasoning paths corresponding to the sample problem includes: Generate a thinking chain example according to the sample question and the thinking chain prompt, wherein the thinking chain example includes: the sample question and the thinking chain reasoning path corresponding to the sample question, and the thinking chain reasoning path includes the real answer information of answering the sample question; Processing the sample question and the thought chain example through the initial natural language model to determine whether the reference reasoning path corresponding to the sample question can be obtained; In the case where the reference reasoning path corresponding to the sample question can be obtained, the reasoning path branch pair is obtained according to the sample question, the reference reasoning path and the thinking chain example; The reasoning path set is obtained according to the reference reasoning path and the reasoning path branch pairs.

2. The method according to claim 1, characterized in that Processing the sample question and the thought chain example by using the initial natural language model to determine whether the reference reasoning path corresponding to the sample question can be obtained includes: Processing the sample question and the thought chain example through the initial natural language model to obtain a first reasoning path; Based on whether the first reasoning path can be used to infer the true answer information, it is determined whether the reference reasoning path corresponding to the sample question can be obtained.

3. The method according to claim 1, characterized in that According to the sample question, the reference reasoning path and the thought chain example, the reasoning path branch pair is obtained, including: According to the reference reasoning path, a reference reasoning sub-path set is obtained; Based on the sample problem, the reference reasoning sub-path set and the thought chain example, a reasoning path branch is generated by sampling with the last reasoning step in the reference reasoning sub-path of the reference reasoning sub-path set as the starting point, so as to obtain a plurality of reasoning path branches corresponding to the reference reasoning sub-path of the reference reasoning sub-path set; According to the multiple reasoning path branches, the first reasoning path branch and the second reasoning path branch are obtained; The reasoning path branch pair is obtained according to the first reasoning path branch and the second reasoning path branch.

4. The method according to claim 1, characterized in that: Training the initial natural language model according to the target training set to obtain the target natural language model includes: Processing the target training set through the initial natural language model to obtain a target prediction reasoning path; According to the target prediction reasoning path, a target loss function is obtained; The initial natural language model is trained according to the target loss function to obtain the target natural language model.

5. The method according to claim 4, characterized in that The target training set is processed by the initial natural language model to obtain a target prediction reasoning path, which includes: Processing the sample question through the initial natural language model to obtain a second reasoning path; Processing the sample question and the reference reasoning sub-path set of the reference reasoning path through the initial natural language model to obtain a third reasoning path; The target prediction reasoning path is obtained according to the second reasoning path and the third reasoning path.

6. The method according to claim 5, characterized in that According to the target prediction reasoning path, the target loss function is obtained including: Calculating according to the second reasoning path and the reference reasoning path to obtain a reference path loss function; Calculating according to the third reasoning path and the reasoning path branch pair to obtain a branch loss function; The target loss function is obtained according to the reference path loss function and the branch loss function.

7. The method according to claim 6, characterized in that Calculating according to the third reasoning path and the reasoning path branch pair to obtain a branch loss function includes: Calculating according to the third reasoning path and the first reasoning path branch to obtain a first loss function; Calculating according to the third reasoning path and the second reasoning path branch to obtain a second loss function; The branch loss function is obtained by calculating according to the first loss function and the second loss function.

8. A data processing method, characterized in that: include: Get the question information input by the target object; Performing reasoning analysis on the question information through a target natural language model to obtain a target reasoning path, wherein the target natural language model is obtained according to the method for generating a natural language model according to any one of claims 1 to 7, and the target reasoning path consists of logical steps of deriving from the question information to the answer information corresponding to the question information; The target reasoning path is returned to the target object.

9. The method according to claim 8, characterized in that The target natural language model is used to perform reasoning analysis on the question information, and the target reasoning path includes: Performing reasoning analysis based on the question information through the target natural language model to generate multiple candidate reasoning paths, wherein the candidate reasoning paths include answer information corresponding to the question information; A target reasoning path is determined from the multiple candidate reasoning paths using the target natural language model.

10. A data processing method, characterized in that: include: Get the question information sent by the client; In the cloud server, the question information is reasoned and analyzed by a target natural language model to obtain a target reasoning path, wherein the target reasoning path consists of logical steps from the question information to the answer information corresponding to the question information, and the target natural language model is obtained from a target training set, and the target training set consists of sample questions and a set of reasoning paths corresponding to the sample questions, wherein the target natural language model is obtained according to the method for generating a natural language model according to any one of claims 1 to 7. The target inference path is returned to the client.

11. A device for generating a natural language model, characterized in that: include: A first acquisition unit is used to acquire a sample question and acquire a set of reasoning paths corresponding to the sample question, wherein the set of reasoning paths at least includes: a reference reasoning path and a pair of reasoning path branches, the reference reasoning path includes true answer information for answering the sample question, the pair of reasoning path branches includes a first reasoning path branch and a second reasoning path branch, the first reasoning path branch is a path branch leading to the true answer information, and the second reasoning path branch is a path branch that cannot lead to the true answer information; A construction unit, used to construct a target training set according to the sample problem and the reasoning path set; A training unit, configured to train an initial natural language model according to the target training set to obtain a target natural language model, wherein the target natural language model is used to answer question information input by a target object; Among them, the first acquisition unit includes: a generating subunit, which is used to generate a thinking chain example based on the sample question and the thinking chain prompt, wherein the thinking chain example includes: the sample question and the thinking chain reasoning path corresponding to the sample question, and the thinking chain reasoning path includes the real answer information of answering the sample question; a processing subunit, which is used to process the sample question and the thinking chain example through the initial natural language model to determine whether the reference reasoning path corresponding to the sample question can be obtained; a first determining subunit, which is used to obtain the reasoning path branch pair based on the sample question, the reference reasoning path and the thinking chain example when the reference reasoning path corresponding to the sample question can be obtained; a second determining subunit, which is used to obtain the reasoning path set based on the reference reasoning path and the reasoning path branch pair.

12. A data processing device, characterized in that: include: A second acquisition unit is used to acquire question information input by a target object; an inference unit, configured to perform inference analysis on the question information through a target natural language model to obtain a target inference path, wherein the target natural language model is obtained according to the method for generating a natural language model according to any one of claims 1 to 7, the target inference path is composed of logical steps of deriving from the question information to answer information corresponding to the question information, the target natural language model is obtained from a target training set, and the target training set is composed of sample questions and a set of inference paths corresponding to the sample questions; A returning unit is used to return the target reasoning path to the target object.

13. An electronic device, characterized in that: include: A memory storing an executable program; A processor for running the program, wherein the program, when running, executes the method for generating a natural language model described in any one of claims 1 to 7, or the data processing method described in any one of claims 8 to 10.

14. A computer program product, characterized in that The computer program product includes a stored computer program, and when the computer program is executed by a processor, the method for generating a natural language model described in any one of claims 1 to 7, or the data processing method described in any one of claims 8 to 10 is implemented.

Images