Question answering method, system, device, medium and product based on pre-training model

Abstract

The present invention discloses a question-answering method, system, device, medium and product based on a pre-trained model, and relates to the field of artificial intelligence technology. For mathematical question-answering, a relatively professional question-answering dataset corresponding to mathematical reasoning is provided to train a more professional mathematical reasoning model. An additional correction layer is added after the result output of the attention layer and the multi-expert feedforward layer of the pre-trained model to complete a correction for the corresponding initial result, thereby improving the accuracy of the pre-trained model in the mathematical reasoning process. In the process of constructing the pre-trained model, the erroneous results of the first result are corrected by the proxy engine to realize the correction process on the data set, and the model parameters of the initial pre-trained model are fine-tuned by the reinforcement learning process, so that the model can perform mathematical calculations and process analysis more accurately, thereby improving the accuracy of the model in mathematical reasoning.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a question-answering method, system, device, medium and product based on a pre-training model. Background Art

[0002] With the continuous increase of computing resources, the parameters of the corresponding pre-trained models are iteratively upgraded, which significantly improves the reasoning ability of the pre-trained models.

[0003] Traditional pre-trained models have achieved good results in natural language reasoning. However, since mathematical reasoning requires high precision and logic, the accuracy of the corresponding output results of traditional pre-trained models in mathematical reasoning that uses logical reasoning processes to solve mathematical problems is relatively low. For example, it is impossible to correctly compare the sizes of 9.11 and 9.9.

[0004] Therefore, how to improve the accuracy of pre-trained models in mathematical reasoning is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] The purpose of the present invention is to provide a question-answering method, system, device, medium and product based on a pre-trained model to solve the technical problem of low accuracy of the corresponding pre-trained model for mathematical reasoning problems.

[0006] In order to solve the above technical problems, the present invention provides a question answering method based on a pre-training model, comprising:

[0007] Obtain the question-answering dataset corresponding to the mathematical reasoning questions;

[0008] Input the question-answering dataset into the attention layer of the pre-trained model to obtain a first initial result; input the first initial result into the correction layer to obtain a second initial result; input the second initial result into the multi-expert feedforward layer to obtain a third initial result; input the third initial result into the correction layer to obtain the response information of the pre-trained model;

[0009] Outputting the response information;

[0010] Among them, the construction process of the pre-trained model includes: inputting a first question and answer data set into the initial pre-trained model to obtain a first result; correcting erroneous results in the first result through a proxy engine to obtain a second question and answer data set; performing reinforcement learning processing on the first question and answer information output by the initial pre-trained model corresponding to the second question and answer data set to feed back to the initial pre-trained model, and fine-tuning the model parameters of the initial pre-trained model to obtain the pre-trained model.

[0011] On the one hand, the process of obtaining the question-answering dataset includes:

[0012] Obtain the third question-answering dataset;

[0013] Inputting the third question-answering dataset into the initial pre-training model to obtain a second result;

[0014] The proxy engine is used to correct erroneous results in the second result to obtain the question and answer data set.

[0015] On the other hand, the proxy engine corrects the erroneous results in the first results to obtain a second question-answering dataset, including:

[0016] Perform non-greedy sampling on the first result to obtain multiple second results;

[0017] Screening a target second result smaller than a threshold value from among the plurality of second results, wherein the threshold value is used to characterize a result quality standard corresponding to the second result; the target second result is an erroneous result;

[0018] Correcting the target second result by the proxy engine to obtain a third result;

[0019] The results other than the target second result among the third result and the second result are used as the second question-answering question data set.

[0020] On the other hand, the correction layer in the pre-trained model includes a gating model and an inference engine; wherein the inference engine is used to perform logical correction and rule constraints on the input result data.

[0021] On the other hand, the reasoning engine is at least a logic reasoning engine, a geometric reasoning engine, and an external formula tool engine; the reasoning engine is composed of a basic model; the logic reasoning engine is composed of a basic model of a reward model;

[0022] Correspondingly, the correction processing of the first initial result in the correction layer includes:

[0023] Screening and classifying the first initial result through a gating model to determine a corresponding target inference engine;

[0024] The first initial result is distributed to the target reasoning engine for reasoning analysis and correction to obtain a second initial result.

[0025] On the other hand, the first initial result is screened and classified through the gating model to determine the corresponding target inference engine, including:

[0026] Performing channel classification on the first initial result by using the gating model; wherein the channel classification is in a one-to-many relationship with the type of the inference engine;

[0027] Merge according to the target inference engine determined by channel classification;

[0028] The data corresponding to the classified first initial result is input into the corresponding merged target reasoning engine.

[0029] On the other hand, the first initial result is screened and classified through the gating model to determine the corresponding target inference engine, including:

[0030] Performing channel classification on the first initial result by using the gating model; wherein the channel classification is in a one-to-one correspondence with the type of the inference engine;

[0031] The data corresponding to the classified first initial result is input into the corresponding target reasoning engine.

[0032] On the other hand, the first question and answer information output by the initial pre-training model corresponding to the second question and answer data set is subjected to reinforcement learning processing to be fed back to the initial pre-training model, including:

[0033] Inputting the first question-answer information into the reward model to obtain a fifth result;

[0034] Evaluate the initial pre-training model according to the fifth result and feedback the corresponding target model parameters;

[0035] The reward data corresponding to the fifth result is optimized through the proximal strategy optimization training model to feed back to the initial pre-training model.

[0036] On the other hand, the first question-and-answer information is input into the reward model to obtain a fifth result, including:

[0037] Inputting the first question-answer information into the instruction reward model to obtain a first sub-result;

[0038] Input the first question-answer information into the supervised reward model to obtain the second sub-result;

[0039] The first sub-result and the second sub-result are used as the fifth result.

[0040] On the other hand, the question-and-answer data set includes at least data sets corresponding to an arithmetic dimension, a textual mathematics dimension, a geometric reasoning dimension, and a theorem proof dimension.

[0041] To solve the above technical problems, the present invention also provides a question-answering system based on a pre-training model, comprising a plurality of computing nodes;

[0042] Among them, each computing node deploys a pre-trained model;

[0043] The computing node is used to input the question-answering dataset into the attention layer of the pre-training model to obtain a first initial result; input the first initial result into the correction layer to obtain a second initial result; input the second initial result into the multi-expert feedforward layer to obtain a third initial result; input the third initial result into the correction layer to obtain the response information of the pre-training model;

[0044] Outputting the response information obtained by each computing node;

[0045] Among them, the construction process of the pre-trained model includes: inputting a first question and answer data set into the initial pre-trained model to obtain a first result; correcting erroneous results in the first result through a proxy engine to obtain a second question and answer data set; performing reinforcement learning processing on the first question and answer information output by the initial pre-trained model corresponding to the second question and answer data set to feed back to the initial pre-trained model, and fine-tuning the model parameters of the initial pre-trained model to obtain the pre-trained model.

[0046] On the one hand, it is applied to distributed model reasoning systems.

[0047] In order to solve the above technical problems, the present invention also provides a question-answering device based on a pre-training model, comprising:

[0048] Memory for storing computer programs;

[0049] A processor is used to implement the steps of the question-answering method based on the pre-training model as described above when executing the computer program.

[0050] In order to solve the above technical problems, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the question-answering method based on the pre-training model as described above are implemented.

[0051] In order to solve the above technical problems, the present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-mentioned question-answering method based on the pre-training model.

[0052] The present invention provides a question-answering method based on a pre-training model, which obtains a question-answering dataset corresponding to a mathematical reasoning problem; inputs the question-answering dataset into the attention layer of the pre-training model to obtain a first initial result; inputs the first initial result into the correction layer to obtain a second initial result; inputs the second initial result into the multi-expert feedforward layer to obtain a third initial result; inputs the third initial result into the correction layer to obtain response information of the pre-training model; and outputs the response information; wherein the construction process of the pre-training model includes: inputting the first question-answering dataset into the initial pre-training model to obtain a first result; correcting erroneous results in the first result through a proxy engine to obtain a second question-answering dataset; performing reinforcement learning processing on the second question-answering dataset corresponding to the first question-answering information output by the initial pre-training model to feed back to the initial pre-training model, and performing model parameter fine-tuning processing on the initial pre-training model to obtain the pre-training model.

[0053] The beneficial effect of the present invention is that it provides a relatively professional mathematical reasoning question and answer dataset corresponding to the question and answer of mathematical reasoning problems, so as to train a more professional mathematical reasoning model. An additional correction layer is added after the result output of the attention layer and the multi-expert feedforward layer of the pre-trained model to complete a correction for the corresponding initial result, thereby improving the accuracy of the pre-trained model in the mathematical reasoning process. In the process of constructing the pre-trained model, the erroneous results of the first result are corrected by the proxy engine to realize the correction process on the data set, and the model parameters of the initial pre-trained model are fine-tuned by the reinforcement learning process, so that the model can perform mathematical calculations and process analysis more accurately, thereby improving the accuracy of the model in mathematical reasoning.

[0054] Secondly, the correction process of the question-and-answer dataset fine-tunes the dataset and improves the quality of the dataset, so as to improve the accuracy of the first initial result obtained later. The proxy engine corrects the wrong results, completes the fine-tuning process of the second question-and-answer dataset, and improves the prediction accuracy of the dataset in the subsequent initial pre-training model. The correction process of the correction layer improves the accuracy of the pre-training model in the mathematical reasoning process. The correction layer is set based on the attention layer and the multi-expert feedforward layer, so that each output result is corrected, and the accuracy of the subsequent result prediction is improved. In the case of multiple target reasoning engines, multiple target reasoning engines are merged to obtain a reasoning engine, and the first initial result is processed to obtain the second initial result, thereby improving the flexibility of data processing. In the case of one target reasoning engine, it is assigned to the designated target reasoning engine for processing to obtain the second initial result, further improving the accuracy of data correction. Through the fine-tuning strategy of the reinforcement learning processing method, the pre-training model can perform mathematical calculations and process analysis more accurately, thereby improving the accuracy of data reasoning.

[0055] In addition, the present invention also provides a question-answering system, device, medium and product based on a pre-trained model, which has the same beneficial effects as the above-mentioned question-answering method based on a pre-trained model. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A flowchart of a question-answering method based on a pre-training model provided in an embodiment of the present invention;

[0058] Figure 2 An architectural diagram of a pre-training model provided in an embodiment of the present invention;

[0059] Figure 3 A schematic diagram of the architecture of a correction layer provided by an embodiment of the present invention;

[0060] Figure 4 A schematic diagram of an artificial intelligence agent fine-tuning strategy provided by an embodiment of the present invention;

[0061] Figure 5 A schematic diagram of another pre-training model provided by an embodiment of the present invention;

[0062] Figure 6 A structural diagram of a question-answering device based on a pre-trained model provided in an embodiment of the present invention;

[0063] Figure 7 A structural diagram of a question-answering device based on a pre-trained model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0065] The core of the present invention is to provide a question-answering method, system, device, medium and product based on a pre-trained model to solve the technical problem of low accuracy of the corresponding pre-trained model for mathematical reasoning problems.

[0066] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0067] The performance of traditional large models is getting closer and closer to the level of human capabilities, and its accuracy in natural language reasoning is over 90%, but its ability in mathematical reasoning problems is still insufficient, and its prediction accuracy is low. The question-answering method based on the pre-training model provided by the present invention can solve the above technical problems.

[0068] Figure 1 A flowchart of a question-answering method based on a pre-training model provided in an embodiment of the present invention, such as Figure 1 As shown, the method includes:

[0069] S11: Obtain a question-answering dataset corresponding to mathematical reasoning questions;

[0070] S12: Input the question-answering dataset into the attention layer of the pre-trained model to obtain a first initial result; input the first initial result into the correction layer to obtain a second initial result; input the second initial result into the multi-expert feedforward layer to obtain a third initial result; input the third initial result into the correction layer to obtain the response information of the pre-trained model;

[0071] S13: Output response information;

[0072] Among them, the construction process of the pre-trained model includes: inputting the first question and answer data set into the initial pre-trained model to obtain a first result; correcting the erroneous results in the first result through the proxy engine to obtain a second question and answer data set; performing reinforcement learning processing on the first question and answer information output by the initial pre-trained model corresponding to the second question and answer data set to feed back to the initial pre-trained model, and fine-tuning the model parameters of the initial pre-trained model to obtain a pre-trained model.

[0073] Specifically, mathematical reasoning problems are problems that require logical thinking and mathematical knowledge to solve, covering various mathematical fields, such as algebra, geometry, etc. Mathematical reasoning corresponds to the requirements of high precision and logic, and requires a dataset of question-and-answer questions on mathematical reasoning problems. The pre-trained model generates mathematics-related questions by learning a large amount of text data, including mathematics textbooks, tutorials, papers, and problem answers, which are used to solve mathematical problems. The question-and-answer dataset in step S11 covers not only one dimension of mathematical reasoning, but also multiple dimensions of mathematical reasoning.

[0074] In some embodiments, the question-and-answer data set includes at least data sets corresponding to an arithmetic dimension, a textual mathematics dimension, a geometric reasoning dimension, and a theorem proving dimension.

[0075] It is understandable that the main factors affecting model performance are the model architecture, fine-tuning strategy and the dataset used for training or fine-tuning. Therefore, the dimensions of the dataset corresponding to the question and answer questions need to cover more. In conventional solutions, the accuracy cannot be improved due to the low coverage of the dataset used for mathematical reasoning.

[0076] The question-answering dataset includes large-scale text data and public data, as well as datasets of multiple dimensions in mathematical reasoning, such as datasets of arithmetic with data enhancement, datasets of text data questions, datasets related to geometric principles, geometric theorems, and formula analysis in the geometric reasoning dimension, and datasets of corresponding proof theorems. There is no limitation here, and you can set it according to the actual situation.

[0077] It should be noted that, in addition to solving question-answering problems related to mathematical reasoning, this embodiment can also solve question-answering problems corresponding to conventional natural language reasoning, which will not be explained in detail here.

[0078] The pre-trained model is a machine learning model with large-scale parameters and complex computing structure. It learns common features such as language and vision by pre-training on large amounts of data, so as to handle various complex tasks.

[0079] In step S12, the question-and-answer dataset is input into the attention layer of the pre-trained model. It should be noted that the pre-trained model in this embodiment includes an attention layer, a multi-expert feedforward layer and a correction layer. The attention layer and the multi-expert feedforward layer are located in the transformer, and each encoder thereof is usually composed of multiple sublayers, each of which includes an attention layer and a multi-expert feedforward layer. The self-attention mechanism is a core component, which allows the model to dynamically allocate attention weights between different parts of the sequence, so as to better understand the context of the sequence. The encoder and decoder layers of the Transformer model are usually stacked, which means that the model can gradually build more complex feature representations through multiple levels of learning. This stacked structure allows each layer to further abstract and learn the high-level features of the data based on the previous layer.

[0080] It is worth noting that the pre-trained model does not include other deep learning pooling layers and convolution layers, mainly because the pre-trained model processes text data, and the convolution layer and pooling layer are used for image processing. The question-answering questions in this embodiment require understanding the context of the entire sentence or document in order to accurately predict the next word or generate text. The attention layer processes text data, allowing the model to consider all words in the entire sequence when processing each word, providing an effective way to capture long-distance dependencies, which is impossible for convolution layers and pooling layers.

[0081] The first initial result is obtained in the attention layer, and the first initial result is input and output to the correction layer. The correction layer here performs logical correction and rule constraints on the input data to obtain the correct result, improve the accuracy of the model in data reasoning, and can also be a process of correcting the input data.

[0082] After the second initial result is input into the multi-expert feedforward layer to obtain the third initial result, it is further input into the correction layer for correction, and the output result is used as the response information of the pre-trained model, and then the response information is output. Here, a correction layer is added to the initial results of multiple layers to perform correction processing, and the result obtained is then input into the next layer. The specific data processing process of the attention layer and the multi-expert feedforward layer here is the same as the conventional processing process, or it can be different. In different cases, specific processing methods can be added based on specific mathematical reasoning question-and-answer datasets for processing, which is not limited here.

[0083] The multi-expert feed-forward layer combines multiple expert networks and a gating network. In this architecture, each expert network is responsible for processing different aspects of the input data, while the gating network is responsible for deciding which experts should be activated to process specific input data. This can be applied to a variety of tasks such as text classification, sentiment analysis, question-answering systems, machine translation, dialogue systems, and text generation. Its core advantage is that it can be pre-trained with less computation, which means that the scale of the model or dataset can be expanded under the same computational budget.

[0084] Regarding the correction layer, after the initial result is input, it is identified to the corresponding reasoning engine for processing. The correction layer includes multiple reasoning engines. The reasoning engine is a key component in the artificial intelligence system, which is used to simulate the human thinking process and automatically derive new conclusions and information through a given knowledge base and rules. The working principle of the reasoning engine is based on logical reasoning and inference. It can work in two ways: forward reasoning (starting from known facts) and backward reasoning (starting from the goal). For example, the reasoning engine can classify texts, such as assigning news articles to different subject categories. By analyzing the emotional tendency in the text, it is determined whether the user's comments are positive or negative. In the intelligent question-and-answer system, the reasoning engine can understand the user's questions and retrieve relevant information from the knowledge base to provide accurate answers. Therefore, the specific processing process of the corresponding reasoning engine is not limited. The reasoning engine corresponding to the mathematical question-and-answer question in the present invention can design a logical reasoning engine, a formula reasoning engine, a geometric reasoning engine, etc., to form an engine feature in mathematical reasoning.

[0085] Figure 2 An architecture diagram of a pre-training model provided by an embodiment of the present invention, such as Figure 2As shown in the figure, the model architecture includes the attention layer, the multi-expert feedforward layer and the correction layer in sequence. The correction layer needs to be connected to both the attention layer and the multi-expert feedforward layer, and will be constrained by the correction layer. When the input corpus (initial result) enters the multi-expert feedforward layer, the gating model will divide the corpus into channels. The corpus for natural language reasoning is sent to expert model 1, the corpus for geometric reasoning is sent to expert model 2, and the corpus for mathematical logic reasoning is sent to expert model 3, and so on. The reasoning engine is processed in the correction layer to realize logical correction and rule constraints on the input results, and further realizes the correction of the output results of the attention layer and the multi-expert feedforward layer, thereby improving the accuracy of the output results.

[0086] Regarding the construction process of the pre-trained model, it should be noted that in the present invention, while the pre-trained model is constructed based on the training set for correction, the first question and answer information obtained by inputting the corrected second question and answer data set into the initial pre-trained model is processed through the fine-tuning strategy of reinforcement learning processing, and fed back to the initial pre-trained model to play the role of fine-tuning the model parameters to obtain the pre-trained model.

[0087] The fine-tuning strategy in the present invention can be a fine-tuning process for the corresponding artificial intelligence agent, namely Agent-(Artificial Intelligence, AI) fine-tuning. In the field of artificial intelligence, an agent refers to an intelligent entity that can perform tasks and make decisions in a specific environment. Fine-tuning is a transfer learning technology that allows the use of models pre-trained on large data sets and further training on new tasks or data sets to improve the performance of the model on specific tasks. Specifically, the learning model used in the fine-tuning strategy is different from the model in the pre-trained model of the present invention. The learning model is only used for the use of the fine-tuning strategy, which is an evaluation optimization of the initial pre-trained model to obtain model parameters and implement the fine-tuning process of replacing the model parameters of the pre-initial model.

[0088] The proxy engine refers to software or services used in the network, which acts as an intermediary between the client and the target server to process network requests and responses. The pre-trained model is used to process and understand text data. This engine can integrate different tools and services, such as search engines, database retrieval, or application programming interface (API) calls to enhance the reasoning and generation capabilities of the model. The correction work performed by the proxy engine in the present invention is to obtain the second question-answering dataset by fine-tuning the first result with poor quality.

[0089] It should be noted that the fine-tuning process of the present invention is reflected in the correction performed on the question and answer data set input into the correction layer of the pre-trained model in step S11; in the process of constructing the pre-trained model, the agent engine is used for correction corresponding to the fine-tuning of the data set; in the process of constructing the model, the model parameters are fine-tuned through reinforcement learning processing.

[0090] The embodiment of the present invention provides a question-answering method based on a pre-training model, obtaining a question-answering dataset corresponding to a mathematical reasoning problem; inputting the question-answering dataset into the attention layer of the pre-training model to obtain a first initial result; inputting the first initial result into the correction layer to obtain a second initial result; inputting the second initial result into the multi-expert feedforward layer to obtain a third initial result; inputting the third initial result into the correction layer to obtain the response information of the pre-training model; outputting the response information; wherein the construction process of the pre-training model includes: inputting the first question-answering dataset into the initial pre-training model to obtain a first result; correcting the erroneous results in the first result through the proxy engine to obtain a second question-answering dataset; performing reinforcement learning processing on the first question-answering information output by the initial pre-training model corresponding to the second question-answering dataset to feed back to the initial pre-training model, and performing model parameter fine-tuning processing on the initial pre-training model to obtain a pre-training model. For mathematical question-answering, a question-answering dataset corresponding to more professional mathematical reasoning is provided to train a more professional mathematical reasoning model. An additional correction layer is added after the output of the results of the attention layer and the multi-expert feedforward layer of the pre-trained model to complete a correction for the corresponding initial result, thereby improving the accuracy of the pre-trained model in the mathematical reasoning process. In the process of building the pre-trained model, the proxy engine is used to correct the erroneous results of the first result, and the correction process is implemented on the data set. The model parameters of the initial pre-trained model are fine-tuned through reinforcement learning processing, so that the model can perform mathematical calculations and process analysis more accurately, thereby improving the accuracy of the model in mathematical reasoning.

[0091] In some embodiments, the process of obtaining the question and answer dataset includes:

[0092] Obtain the third question-answering dataset;

[0093] Inputting the third question-answering dataset into the initial pre-training model to obtain a second result;

[0094] The proxy engine is used to correct the erroneous results in the second result to obtain a question-and-answer dataset.

[0095] Specifically, the question and answer dataset in this embodiment is obtained through correction processing. The correction process here is the same as the construction process of the pre-trained model corresponding to the correction process of the first question and answer dataset. The proxy engine corrects the erroneous results in the second result of the third question and answer dataset input into the initial training model to obtain the question and answer dataset. The specific correction process can be the same as the proxy engine correction process in the above embodiment, which will not be discussed here.

[0096] The correction process of the question-and-answer dataset provided in this embodiment fine-tunes the dataset and improves the quality of the dataset, so as to improve the accuracy of the first initial result obtained subsequently.

[0097] In some embodiments, the second question-answering dataset is obtained by correcting the erroneous results in the first result through the proxy engine, including:

[0098] Perform non-greedy sampling on the first result to obtain multiple second results;

[0099] Screening a target second result smaller than a threshold value from among the plurality of second results, wherein the threshold value is used to characterize a result quality standard corresponding to the second result; the target second result is an erroneous result;

[0100] Correcting the target second result by the proxy engine to obtain a third result;

[0101] The results of the third result and the second result except the target third result are used as the second question-answering question data set.

[0102] Specifically, in this embodiment, the first result is non-greedy sampled to obtain multiple second results. Non-greedy sampling is based on outputting multiple answers to a question, which is different from outputting one answer to a question. Therefore, multiple different answers will be output, and the proxy engine will be called for the results of poor quality to make preliminary corrections to the wrong steps.

[0103] The result with poor quality is to select the target second result that is less than the threshold from multiple second results. The corresponding threshold here is the quality standard for measuring the errors and correct results of the second results, and the target second result is the wrong result. The target second result is corrected by the proxy engine to obtain the third result. The third result and the correct second result are used as the second question and answer dataset. It should be noted here that the correction process can be a repeated process or a one-time correction process, which is not limited here. Repeating the correction process can further enrich the corpus and improve the fine-tuning effect.

[0104] In this embodiment, the proxy engine is used to correct the erroneous results, complete the fine-tuning process of the second question-and-answer data set, and improve the prediction accuracy of the data set in the subsequent initial pre-training model.

[0105] In some embodiments, the correction layer within the pre-trained model includes a gating model and an inference engine; wherein the inference engine is used to perform logical correction and rule constraints on the input result data.

[0106] Figure 3 A schematic diagram of the architecture of a correction layer provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown in the figure, the correction layer includes a gating model and multiple reasoning engines. The reasoning engine can perform logical correction and rule constraints on the received input content to improve the accuracy of the model in these mathematical reasoning aspects. Some engines are trained with relevant corpus of the corresponding engine using a training method similar to the reward model, and the other part is formed by linking external tools and engines.

[0107] The correction layer first filters the input content from the attention layer and the multi-expert feedforward layer through the gating model, dispatches it to the specified engine for processing, and outputs the corrected content. It should be noted that the specific model of the gating model here is not limited, and it mainly plays a screening role to correct or adjust the behavior of the model. For example, through the gating mechanism, the model can selectively focus on certain parts of the input data when processing a specific task, while ignoring other irrelevant parts. This method can help the model capture key information more accurately in complex decision-making tasks, thereby improving the performance and accuracy of the model.

[0108] In some embodiments, the reasoning engine is at least a logic reasoning engine, a geometric reasoning engine, and an external formula tool engine; the reasoning engine is composed of a basic model; the logic reasoning engine is composed of a basic model of a reward model;

[0109] Correspondingly, the correction process of the first initial result in the correction layer includes:

[0110] Screening and classifying the first initial result through a gating model to determine a corresponding target inference engine;

[0111] The first initial result is distributed to the target reasoning engine for reasoning analysis and correction to obtain the second initial result.

[0112] It can be understood that the reasoning engines are composed of basic models, such as the logical reasoning engine, geometric reasoning engine, formula reasoning engine, symbolic reasoning engine, etc. The reasoning engine here is based on the reasoning characteristics corresponding to the mathematical reasoning aspect. The external formula tool engine is a reasoning engine corresponding to the formula query tool. The logical reasoning engine is a reasoning engine based on the logical characteristics of mathematical reasoning problems. The geometric reasoning engine is a reasoning engine based on the geometric characteristics of mathematical reasoning problems.

[0113] The correction process takes the first initial result output by the attention layer as an example. The first initial result is screened through the gating model of the correction layer to determine the corresponding target inference engine. The first initial result is assigned to the target inference engine for reasoning analysis and correction to obtain the second initial result.

[0114] The reasoning engine in this embodiment is the same as the conventional reasoning engine. In this embodiment, the second initial result is obtained by the reasoning engine analysis mainly based on the mathematical reasoning characteristic analysis. The correction process for the third initial result is the same as the correction process for the first initial result, which will not be discussed here.

[0115] The correction process of the correction layer in this embodiment improves the accuracy of the pre-trained model in the mathematical reasoning process. The correction layer is set based on the attention layer and the multi-expert feedforward layer, so that the result of each output is corrected, thereby improving the accuracy of subsequent result predictions.

[0116] In some embodiments, screening and classifying the first initial result through a gating model to determine a corresponding target reasoning engine includes:

[0117] Channel classification is performed on the first initial result by using a gating model; wherein the channel classification is in a one-to-many relationship with the type of the inference engine;

[0118] Merge according to the target inference engine determined by channel classification;

[0119] The data corresponding to the classified first initial result is input into the corresponding merged target reasoning engine.

[0120] Specifically, the screening process of the first initial result through the gating model is to perform channel classification to classify and determine the corresponding target reasoning engine. The classification process here can be classified by feature recognition or keyword recognition, etc. It should be noted that the corresponding reasoning engine for the first initial result can include multiple types, such as both a logical reasoning engine and a geometric reasoning engine. Such a relationship requires the target reasoning engines to be merged, and the data corresponding to the first initial result is jointly input into the merged target reasoning engine for processing.

[0121] In this embodiment, when there are multiple target inference engines, the multiple target inference engines are merged to obtain one inference engine, and the first initial result is processed to obtain the second initial result, thereby improving the flexibility of data processing.

[0122] In some other embodiments, screening and classifying the first initial result through a gating model to determine a corresponding target inference engine includes:

[0123] Channel classification is performed on the first initial result by using a gating model; wherein the channel classification is in a one-to-many relationship with the type of the inference engine;

[0124] Inputting the data corresponding to the classified first initial result into the target reasoning engine after channel classification to obtain the corresponding second sub-initial result; wherein there are multiple target reasoning engines;

[0125] The second initial sub-results are combined to obtain the second initial sub-results.

[0126] Specifically, in the above embodiment, the target engines are merged to process the first initial result. In this embodiment, the first initial result is input into the corresponding target reasoning engine to obtain multiple second sub-initial results, and the multiple second sub-initial results are subsequently merged to obtain the second initial sub-result. The merging process here is not a simple splicing, but an algorithm calculation based on the weights or features corresponding to each second sub-initial result to obtain the second initial result, which is not limited here and can be calculated according to actual conditions and specific algorithms.

[0127] In this embodiment, when there are multiple target reasoning engines, the second sub-initial results processed by the target reasoning engines are merged to obtain the second initial result, so that the final second initial result contains the training of the corresponding target reasoning engine, thereby improving the diversity and flexibility of data processing methods.

[0128] In some other embodiments, screening and classifying the first initial result through a gating model to determine a corresponding target inference engine includes:

[0129] Channel classification is performed on the first initial result by using a gating model; wherein the channel classification is in a one-to-one correspondence with the type of the inference engine;

[0130] The data corresponding to the classified first initial result is input into the corresponding target reasoning engine.

[0131] Specifically, when there is only one target inference engine, that is, the channel classification and the type of inference engine are in a one-to-one correspondence, the data corresponding to the classified first initial result needs to be input into the corresponding target inference engine to obtain the second initial result.

[0132] When there is only one corresponding target inference engine in this embodiment, it is assigned to the designated target inference engine for processing to obtain a second initial result, thereby further improving the accuracy of data correction.

[0133] In some embodiments, the first question and answer information output by the initial pre-training model corresponding to the second question and answer dataset is subjected to reinforcement learning processing to be fed back to the initial pre-training model, including:

[0134] Inputting the first question-answer information into the reward model to obtain a fifth result;

[0135] According to the fifth result, the initial pre-trained model is evaluated and fed back the corresponding target model parameters;

[0136] The reward data corresponding to the fifth result is optimized through the proximal strategy optimization training model to feed back to the initial pre-training model.

[0137] Figure 4 A schematic diagram of an artificial intelligence agent fine-tuning strategy provided by an embodiment of the present invention, such as Figure 4 As shown in the figure, the agent fine-tuning in the first step inputs the first question-answering dataset into the agent engine, and fine-tuning is achieved through correction to obtain the second question-answering dataset. Reinforcement learning is performed in the second and third steps. The reinforcement learning method is to improve the mathematical reasoning ability of the pre-trained model through the reinforcement learning from Evol-InstructFeedback (RLEIF) method based on evolutionary instruction feedback. It mainly includes supervised fine-tuning, instruction reward model, process supervision reward model and proximal policy optimization training model. The reward model here includes the instruction reward model (IRM) and the supervised reward model (PRM). IRM is used to give the correct reward and punishment mechanism according to the output results of the model, and PRM is used to supervise and correct the problem-solving process of the model, and AI is used to judge the score of IRM and optimize the process of PRM; finally, the proximal policy optimization (PPO) training model is used to allow the model to continuously update the correct problem-solving process, thereby improving the model's ability in mathematical reasoning.

[0138] In some embodiments, inputting the first question-and-answer information into the reward model to obtain a fifth result includes:

[0139] Inputting the first question-answer information into the instruction reward model to obtain a first sub-result;

[0140] Input the first question-answer information into the supervised reward model to obtain the second sub-result;

[0141] The first sub-result and the second sub-result are taken as the fifth result.

[0142] Specifically, the first question-and-answer information is input into the instruction reward model and the supervision reward model respectively, and the first sub-result and the second sub-result are obtained correspondingly, and the two results are used as the fifth result. Then, the fifth result is evaluated and fed back to the initial pre-trained model to obtain the corresponding target model parameters, and the target model parameters are used in the initial pre-trained model. Then, the reward data corresponding to the fifth result is optimized through the proximal strategy optimization training model to be fed back to the initial pre-trained model, and finally the pre-trained model is obtained.

[0143] In this embodiment, the fine-tuning strategy of the reinforcement learning processing method is used to enable the pre-trained model to perform mathematical calculations and process analysis more accurately, thereby improving the accuracy of data reasoning.

[0144] Figure 5 A schematic diagram of another pre-training model provided in an embodiment of the present invention is shown in FIG. Figure 5 As shown, the question-answering method based on the pre-trained model of the present invention is also applicable to the network model of the matching task and the corresponding general pre-trained model, and is also applicable to the fine-tuning strategy for the artificial intelligence agent.

[0145] Furthermore, the present invention also provides a question-answering system based on a pre-trained model, comprising a plurality of computing nodes;

[0146] Among them, each computing node deploys a pre-trained model;

[0147] A computing node, used for inputting the question-answering dataset into the attention layer of the pre-trained model to obtain a first initial result; inputting the first initial result into the correction layer to obtain a second initial result; inputting the second initial result into the multi-expert feedforward layer to obtain a third initial result; inputting the third initial result into the correction layer to obtain the response information of the pre-trained model;

[0148] Output the response information obtained by each computing node;

[0149] Among them, the construction process of the pre-trained model includes: inputting the first question and answer data set into the initial pre-trained model to obtain a first result; correcting the erroneous results in the first result through the proxy engine to obtain a second question and answer data set; performing reinforcement learning processing on the first question and answer information output by the initial pre-trained model corresponding to the second question and answer data set to feed back to the initial pre-trained model, and fine-tuning the model parameters of the initial pre-trained model to obtain a pre-trained model.

[0150] Specifically, computing nodes in computer science usually refer to servers or computers dedicated to performing computing tasks in a distributed computing environment. They are key components in distributed systems, data centers, supercomputers or cloud computing platforms, responsible for processing data and performing various computing operations. The premise of computing nodes is that pre-trained models need to be deployed. In this embodiment, a computing node can obtain response information through a question-and-answer method based on a pre-trained model, and the response information of each computing node can be output.

[0151] In some embodiments, a question answering system based on a pre-trained model is applied to a distributed model reasoning system.

[0152] A distributed model inference system is a system that processes model inference tasks in parallel on multiple computing nodes. It can significantly improve the speed and throughput of model inference, especially when processing large machine learning models. This system achieves distributed utilization of computing resources by distributing different parts of the model or different data batches to multiple computing nodes.

[0153] For an introduction to a question-answering system based on a pre-trained model provided by the present invention, please refer to the above-mentioned method embodiment, and the present invention will not be repeated here. It has the same beneficial effects as the above-mentioned question-answering method based on a pre-trained model.

[0154] The above describes in detail various embodiments corresponding to the question-answering method based on the pre-training model. On this basis, the present invention also discloses a question-answering device based on the pre-training model corresponding to the above method. Figure 6 The structure diagram of a question-answering device based on a pre-trained model provided by an embodiment of the present invention. Figure 6 As shown, the device includes:

[0155] An acquisition module 11 is used to acquire a question-answering dataset corresponding to a mathematical reasoning question;

[0156] The input module 12 is used to input the question-answering dataset into the attention layer of the pre-trained model to obtain a first initial result; input the first initial result into the correction layer to obtain a second initial result; input the second initial result into the multi-expert feedforward layer to obtain a third initial result; input the third initial result into the correction layer to obtain the response information of the pre-trained model;

[0157] Output module 13, used for outputting response information;

[0158] Among them, the construction process of the pre-trained model includes: inputting the first question and answer data set into the initial pre-trained model to obtain a first result; correcting the erroneous results in the first result through the proxy engine to obtain a second question and answer data set; performing reinforcement learning processing on the first question and answer information output by the initial pre-trained model corresponding to the second question and answer data set to feed back to the initial pre-trained model, and fine-tuning the model parameters of the initial pre-trained model to obtain a pre-trained model.

[0159] Since the embodiments of the device part correspond to the above embodiments, please refer to the description of the embodiments of the method part for the embodiments of the device part, and will not be repeated here.

[0160] For an introduction to a question-and-answer device based on a pre-trained model provided by the present invention, please refer to the above-mentioned method embodiment, and the present invention will not be repeated here. It has the same beneficial effects as the above-mentioned question-and-answer method based on a pre-trained model.

[0161] Figure 7 A structural diagram of a question-answering device based on a pre-training model provided in an embodiment of the present invention, such as Figure 7 As shown, the device comprises:

[0162] A memory 21, used for storing computer programs;

[0163] The processor 22 is used to implement the steps of the question-answering method based on the pre-trained model when executing the computer program.

[0164] The question-answering device based on the pre-trained model provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0165] Among them, the processor 22 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 22 may be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 22 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 22 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 22 may also include an AI processor, which is used to process computing operations related to machine learning.

[0166] The memory 21 may include one or more computer-readable storage media, which may be non-transitory. The memory 21 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 21 is at least used to store the following computer program 211, wherein, after the computer program is loaded and executed by the processor 22, it can implement the relevant steps of the question-answering method based on the pre-trained model disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 21 may also include an operating system 212 and data 213, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 212 may include Windows, Unix, Linux, etc. Data 213 may include but is not limited to data involved in the question-answering method based on the pre-trained model, etc.

[0167] In some embodiments, the question-answering device based on the pre-trained model may also include a display screen 23 , an input / output interface 24 , a communication interface 25 , a power supply 26 , and a communication bus 27 .

[0168] Those skilled in the art can understand that Figure 7 The structure shown in does not constitute a limitation on the question-answering device based on the pre-trained model, and may include more or fewer components than those shown in the figure.

[0169] The processor 22 implements the question-answering method based on the pre-training model provided in any of the above embodiments by calling the instructions stored in the memory 21.

[0170] For an introduction to a question-and-answer device based on a pre-trained model provided by the present invention, please refer to the above-mentioned method embodiment. The present invention will not be repeated here. It has the same beneficial effects as the above-mentioned question-and-answer method based on a pre-trained model.

[0171] Furthermore, the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor 22, the steps of the question-answering method based on the pre-trained model are implemented.

[0172] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0173] For an introduction to a computer-readable storage medium provided by the present invention, please refer to the above-mentioned method embodiment, and the present invention will not go into details here. It has the same beneficial effects as the above-mentioned question-answering method based on the pre-training model.

[0174] Furthermore, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned question-answering method based on the pre-trained model.

[0175] For an introduction to a computer program product provided by the present invention, please refer to the above method embodiment, and the present invention will not be repeated here. It has the same beneficial effects as the above-mentioned question-answering method based on the pre-training model.

[0176] The above is a detailed introduction to a question-answering method, system, device, medium and product based on a pre-trained model provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the present invention.

[0177] It should also be noted that, in this specification, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A question-answering method based on a pre-trained model, characterized in that: include: Obtain the question-answering dataset corresponding to the mathematical reasoning questions; Input the question-answering dataset into the attention layer of the pre-trained model to obtain the first initial result; Inputting the first initial result into a correction layer to obtain a second initial result; Inputting the second initial result into a multi-expert feed-forward layer to obtain a third initial result; Inputting the third initial result into the correction layer to obtain the response information of the pre-trained model; wherein an additional correction layer is added after the output of the results of the attention layer and the multi-expert feedforward layer; the correction layer performs logical correction and rule constraints on the input data to obtain the correct result; Outputting the response information; Among them, the construction process of the pre-trained model includes: inputting a first question and answer data set into the initial pre-trained model to obtain a first result; correcting erroneous results in the first result through a proxy engine to obtain a second question and answer data set; performing reinforcement learning processing on the first question and answer information output by the initial pre-trained model corresponding to the second question and answer data set to feed back to the initial pre-trained model, and fine-tuning the model parameters of the initial pre-trained model to obtain the pre-trained model.

2. The question-answering method based on the pre-training model according to claim 1, characterized in that: The process of obtaining the question-answering dataset includes: Obtain the third question-answering dataset; Inputting the third question-answering dataset into the initial pre-training model to obtain a second result; The proxy engine is used to correct erroneous results in the second result to obtain the question and answer data set.

3. The question-answering method based on the pre-training model according to claim 1, characterized in that: Correcting the erroneous results in the first results by using a proxy engine to obtain a second question-answering dataset, including: Perform non-greedy sampling on the first result to obtain multiple second results; Screening a target second result smaller than a threshold value from among the plurality of second results, wherein the threshold value is used to characterize a result quality standard corresponding to the second result; the target second result is an erroneous result; Correcting the target second result by the proxy engine to obtain a third result; The results other than the target second result among the third result and the second result are used as the second question-answering question data set.

4. The question-answering method based on a pre-training model according to claim 1, characterized in that: The correction layer in the pre-trained model includes a gating model and an inference engine; wherein the inference engine is used to perform logical correction and rule constraints on the input result data.

5. The question-answering method based on the pre-training model according to claim 4, characterized in that: The reasoning engine is at least a logic reasoning engine, a geometric reasoning engine, and an external formula tool engine; the reasoning engine is composed of a basic model; the logic reasoning engine is composed of a basic model of a reward model; Correspondingly, the correction processing of the first initial result in the correction layer includes: Screening and classifying the first initial result through a gating model to determine a corresponding target inference engine; The first initial result is distributed to the target reasoning engine for reasoning analysis and correction to obtain a second initial result.

6. The question-answering method based on the pre-training model according to claim 5, characterized in that: The first initial result is screened and classified through the gating model to determine the corresponding target inference engine, including: Performing channel classification on the first initial result by using the gating model; wherein the channel classification is in a one-to-many relationship with the type of the inference engine; Merge according to the target inference engine determined by channel classification; The data corresponding to the classified first initial result is input into the corresponding merged target reasoning engine.

7. The question-answering method based on a pre-training model according to claim 5, characterized in that: The first initial result is screened and classified by the gating model to determine a corresponding target inference engine, including: Performing channel classification on the first initial result by using the gating model; wherein the channel classification is in a one-to-one correspondence with the type of the inference engine; The data corresponding to the classified first initial result is input into the corresponding target reasoning engine.

8. The question-answering method based on a pre-training model according to claim 1, characterized in that: Performing reinforcement learning processing on the first question and answer information output by the initial pre-training model corresponding to the second question and answer dataset to feed back to the initial pre-training model, including: Inputting the first question-answer information into the reward model to obtain a fifth result; Evaluate the initial pre-training model according to the fifth result and feedback the corresponding target model parameters; The reward data corresponding to the fifth result is optimized through the proximal strategy optimization training model to feed back to the initial pre-training model.

9. The question-answering method based on the pre-training model according to claim 8, characterized in that: Inputting the first question-and-answer information into the reward model obtains a fifth result, including: Inputting the first question-answer information into the instruction reward model to obtain a first sub-result; Input the first question-answer information into the supervised reward model to obtain the second sub-result; The first sub-result and the second sub-result are used as the fifth result.

10. The question-answering method based on a pre-training model according to claim 1, characterized in that: The question-and-answer dataset includes at least datasets corresponding to an arithmetic dimension, a textual mathematics dimension, a geometric reasoning dimension, and a theorem proof dimension.

11. A question-answering system based on a pre-trained model, characterized in that: including multiple computing nodes; Among them, each computing node deploys a pre-trained model; The computing node is used to input the question-answering dataset into the attention layer of the pre-training model to obtain a first initial result; input the first initial result into the correction layer to obtain a second initial result; input the second initial result into the multi-expert feedforward layer to obtain a third initial result; input the third initial result into the correction layer to obtain the response information of the pre-training model; wherein an additional correction layer is added after the result output of the attention layer and the multi-expert feedforward layer; the correction layer performs logical correction and rule constraints on the input data to obtain the correct result; Outputting the response information obtained by each computing node; Among them, the construction process of the pre-trained model includes: inputting a first question and answer data set into the initial pre-trained model to obtain a first result; correcting erroneous results in the first result through a proxy engine to obtain a second question and answer data set; performing reinforcement learning processing on the first question and answer information output by the initial pre-trained model corresponding to the second question and answer data set to feed back to the initial pre-trained model, and fine-tuning the model parameters of the initial pre-trained model to obtain the pre-trained model.

12. The question-answering system based on the pre-training model according to claim 11, characterized in that: Applied to distributed model reasoning systems.

13. A question-answering device based on a pre-trained model, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the question-answering method based on a pre-training model as described in any one of claims 1 to 10 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the question-answering method based on the pre-training model as described in any one of claims 1 to 10 are implemented.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the question-answering method based on the pre-training model described in any one of claims 1 to 10 are implemented.

Images