Alignment thinking chain-based science answering method and system

Abstract

The invention discloses a science answering method and system based on an aligned thinking chain, and the method comprises the following steps: inputting a science question into a known LLM, generating a natural language reasoning step under a zero sample training condition, and recognizing the native style of the natural language reasoning step as a local style; compiling a few-sample prompt example according to the local style, wherein the few-sample prompt example comprises questions, reasoning steps and answers; modifying the format and reasoning steps of the few-sample prompt example according to the local style, so that the few-sample prompt example is aligned with the local style; optimizing the prompt example retriever based on the few-sample prompt example to obtain an optimized prompt example retriever; and inputting a new science question into the optimized prompt example searcher, and performing answering in combination with the LLM reasoning path to generate an answer. By aligning the native style of the language model, unifying the inference step format and enhancing the retrieval method, the manual workload is reduced, and the performance and reliability of the existing science question answering system are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a method and system for solving science problems based on an aligned thinking chain. Background Art

[0002] As the amount of computation and data in language models continues to grow, one problem is that unique capabilities emerge when models become large. The ability to learn in context (ICL), i.e. solve a corresponding task based on a given few examples, is something that language models are particularly good at when they scale to a certain size. Some researchers have found that large models significantly outperform small models in multi-step reasoning tasks (which is considered a manifestation of emergent capabilities), while large models show very limited performance gains on tasks such as sentiment classification. Moreover, in multi-step reasoning, small number of prompts methods begin to outperform full training set fine-tuning, even on the same large model. These characteristics have made multi-step reasoning tasks such as math problem solving and commonsense reasoning attract a lot of attention from researchers. This work takes an important step towards multi-step reasoning by showing the key role of CoT text style in large language models.

[0003] Existing science problem-solving systems usually rely on large-scale language models (LLMs) and use chain-of-thought (CoT) techniques to enhance reasoning capabilities. These systems are mainly implemented through the following methods:

[0004] Chain-of-Thought (CoT) prompts: This method significantly improves the performance of multi-step reasoning tasks by adding intermediate reasoning steps to the prompts.

[0005] Complex Chained Thinking (Complex CoT): This method selects examples containing the most complex reasoning steps, further improving the performance of multi-step reasoning tasks.

[0006] Self-Consistency: Improve the accuracy of the answer by sampling different reasoning paths and taking majority vote.

[0007] Retrieval enhancement methods: For example, Efficient Prompt Retriever (EPR) uses an unsupervised retriever to obtain a set of candidate examples and optimizes the retrieval model through contrastive learning to select the most similar and relevant examples.

[0008] Although existing systems have made some progress in solving science problems, there are still some problems:

[0009] Limitations of manually written examples: In existing systems, examples of chain thinking prompts are usually written manually, which may not fully conform to the language style of LLM and limit the learning ability of LLM.

[0010] Inconsistency in intermediate reasoning steps: Manually written chain thinking examples may have inconsistencies in format and steps, which will affect the model's reasoning ability.

[0011] Unstable performance of retrieval methods: Retrieval-based methods may not always provide the most relevant examples, thus affecting the overall performance of the system.

[0012] Errors and irrationalities in multi-step reasoning: In the process of generating multi-step reasoning, errors and irrationalities may occur, which need to be manually modified and corrected.

[0013] Therefore, how to provide a scientific answer method and system that can significantly improve retrieval performance and reasoning reliability is a technical problem that needs to be solved urgently by technical personnel in this field. Summary of the invention

[0014] In view of the above research status and existing problems, the present invention provides a science problem-solving method and system based on aligned thinking chains, which aligns the native style of the language model, unifies the format of reasoning steps, enhances the retrieval method, reduces manual workload, and significantly improves the performance and reliability of the existing science problem-solving system.

[0015] The present invention first provides a method for solving science problems based on an aligned thinking chain, comprising the following steps:

[0016] S1: Input science questions into known LLMs and generate natural language reasoning steps under zero-shot training conditions to identify the native style of natural language reasoning steps as the local style;

[0017] S2: write a few-shot prompt example according to the local style, wherein the few-shot prompt example includes a question, a reasoning step, and an answer;

[0018] S3: modifying the format and reasoning steps of the few-shot prompt example according to the local style to align it with the local style;

[0019] S4: optimizing the prompt example retriever based on the few-sample prompt example to obtain an optimized prompt example retriever;

[0020] S5: Input a new science question into the optimized prompt example retriever, solve it in combination with the LLM reasoning path, and generate an answer.

[0021] Preferably, the optimized prompt example retriever is used to select the prompt example with the highest similarity to the new science question for prompting.

[0022] Preferably, the step of acquiring the LLM reasoning path includes:

[0023] Set sampling strategies to generate different reasoning paths;

[0024] The answer is determined based on a majority voting mechanism.

[0025] Preferably, the alignment step in S2 includes:

[0026] S31: Input the question in the newly compiled prompt example into the LLM, obtain the generated natural language reasoning steps and answers, and identify the erroneous content in the reasoning steps and answers of the prompt example that do not conform to the local style;

[0027] S32: among all the identified error contents, select the first error in the current error content as the starting point for adjustment;

[0028] S33: modifying the first error to make it conform to the local style and updating the newly edited prompt example;

[0029] S34: Submitting the updated new prompt example to the LLM for processing, and only submitting it to the modified content position, and the LLM generates the remaining reasoning steps and answers;

[0030] S35: Identify whether there are erroneous contents in the reasoning steps and answers of the updated newly compiled prompt example that are inconsistent with the reasoning steps and answers of the remaining part generated by the LLM in S4. If yes, repeat S32-S35 and modify only one error each time; if no, the updated newly compiled prompt example is stored as a few-sample prompt example.

[0031] The present invention provides a science answering system based on aligned thinking chains, comprising: a zero-sample native style detection module, a few-sample prompt example generation module, a reasoning step format unification module, a retrieval enhancement module and a system integration module; wherein,

[0032] The zero-shot native style detection module inputs a science question into a known LLM, generates a natural language reasoning step under a zero-shot training condition, and identifies the native style of its natural language reasoning step as a local style;

[0033] The few-sample prompt example generation module is used to write a few-sample prompt example according to the local style, and the few-sample prompt example includes a question, a reasoning step and an answer;

[0034] The reasoning step format unification module is used to modify the format and reasoning steps of the few-shot prompt example according to the local style to align it with the local style;

[0035] The retrieval enhancement module is used to optimize the prompt example retriever based on the few-sample prompt example to obtain an optimized prompt example retriever;

[0036] The system integration module is used to integrate the zero-sample native style detection module, the few-sample prompt example generation module, the reasoning step format unification module, and the retrieval enhancement module, input new science questions into the optimized prompt example retriever, solve them in combination with the LLM reasoning path, and generate answers.

[0037] Preferably, the optimized prompt example retriever is used to select the prompt example with the highest similarity to the new science question for prompting.

[0038] Preferably, it also includes a self-consistency module for setting a sampling strategy to generate different reasoning paths; and determining the answer based on a majority voting mechanism.

[0039] Preferably, the few-sample prompt example generation module includes an error content identification unit, a modification unit, and a submission unit:

[0040] The error content identification unit inputs the question in the newly compiled prompt example into the LLM, obtains the generated natural language reasoning steps and answers, and identifies the error content in the reasoning steps and answers of the prompt example that do not conform to the local style;

[0041] The modification unit selects the first error in the current error content from among all the identified error contents as a starting point for adjustment; modifies the first error to make it conform to the local style and updates the newly edited prompt example;

[0042] The submission unit submits the updated newly compiled prompt example to the LLM for processing, and only submits it to the content position where the modification is completed, and the LLM generates the reasoning steps and answers of the remaining part; the error content identification unit identifies whether there is error content in the reasoning steps and answers of the updated newly compiled prompt example that does not conform to the reasoning steps and answers of the remaining part generated by the LLM. If so, the modification unit is called again to modify the updated newly compiled prompt example, and only one error is modified each time, until the error content identification unit identifies the error-free content, and obtains the final prompt example as the few-sample prompt example.

[0043] The present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the science solution method based on aligned thinking chains are implemented.

[0044] The present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the science solution method based on aligned thinking chains are implemented.

[0045] Compared with the prior art, it has the following beneficial effects:

[0046] The present invention solves the problem of unnatural or incoherent model reasoning due to the inconsistency between the chain thinking prompt examples and the native style of LLM by aligning the prompt examples with the native style of LLM, thereby improving the naturalness and consistency of reasoning.

[0047] The present invention improves the performance of the retrieval-based prompting method by rewriting the CoT in the multi-step reasoning dataset and optimizing the text style of the prompting example, making the retrieved examples more relevant and similar. For example, in the retrieval-enhanced generation (RAG) method, the performance is improved by 3.6%, thereby improving the overall performance of the retrieval enhancement method.

[0048] The present invention effectively solves the problem of logical errors that are prone to occur in multi-step reasoning tasks by enhancing the logical error detection capability of LLM, and improves the reliability of the answer, which is very important for ensuring the correctness and logical consistency of the answer.

[0049] The present invention can significantly improve the accuracy of multi-step reasoning tasks by using the method of the present invention, solving the performance bottleneck of low accuracy of existing systems in such tasks. Compared with traditional chain thinking (CoT) and complex chain thinking (Complex CoT) methods, the performance of the method of the present invention is significantly improved. For example, on GPT-3.5-turbo, the accuracy is improved by 3.2% on average.

[0050] The present invention focuses on modifying the CoT text style in few-shot examples, can be easily integrated with other ICL techniques, and further improves the performance of LLM.

[0051] The present invention effectively reduces manual workload and significantly improves the performance and reliability of existing science question answering systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work.

[0053] Figure 1 A principle comparison diagram of the science problem-solving method based on the alignment thinking chain provided by an embodiment of the present invention and the traditional method;

[0054] Figure 2 A schematic diagram of an example of an alignment thought chain prompt provided by an embodiment of the present invention;

[0055] Figure 3 A structural block diagram of a science answering system based on an aligned thinking chain provided by an embodiment of the present invention;

[0056] Figure 4 : Schematic diagram of the comparison results of the answer accuracy of the method of the present invention and the complex CoT when sampling different reasoning paths on GSM8k. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Existing science problem-solving systems generally use chain-of-thought (CoT) technology to enhance reasoning capabilities based on large-scale language models (LLMs). Traditional chain-of-thought prompt examples are usually manually designed, which is inconsistent with the native style of LLM, resulting in unnatural or incoherent situations in the model during reasoning. Retrieval-based methods often lack effectiveness in prompt example selection. In addition, existing systems are prone to logical errors in multi-step reasoning tasks, affecting the accuracy of the final answer. Based on this, the present invention provides a science problem-solving method, system, electronic device and storage medium based on aligned thinking chains, aiming to effectively improve the performance and reliability of science problem-solving systems.

[0059] Before introducing this application, the technical terms involved in this article are first explained:

[0060] Prompt: When communicating with the large language model, you need to send instructions to the AI. It can be a text description, such as "solve the equation x^2+3x-4=0" when you communicate with the large language model. It can also be a parameter description in a certain format. For example, if you want the large language model to draw a picture in a certain format, you need to describe the relevant drawing parameters.

[0061] Large language model (LLM): refers to a generative deep neural network model based on the Transformer structure, which has super semantic understanding and multi-round dialogue capabilities. Common large language models include GT4 and large language models developed by other companies.

[0062] The large language model used in the present invention can be an existing large language model, such as GPT-4, Instruct-GPT, GLM and other large language models.

[0063] Chain of Thought (CoT): In the study and application of Large Language Models (LLMs), it is used to describe the process of how a model gradually derives a solution.

[0064] Complex Chain Thinking (Complex CoT): refers to a multi-level, multi-step thinking and reasoning process used to solve complex problems. This way of thinking requires not only the ability to understand the problem itself, but also the ability to decompose the problem, identify key information, develop strategies, perform multi-step reasoning, and finally draw reasonable conclusions.

[0065] The solution of the present invention can be implemented on a computer device with data query capability and executed by the computer device, wherein the computer device can be a terminal or a server. The terminal can be a terminal device such as a smart phone, a tablet computer, a laptop computer, a touch screen, a game console, a personal computer (PC), a personal digital assistant (PDA), etc. The server can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution network services, and big data and artificial intelligence platforms, but is not limited to this.

[0066] The first aspect of the embodiment of the present invention discloses a method for solving science problems based on an alignment thinking chain, such as Figure 1As shown, the following steps are included: S1: input science questions into known LLM, and generate natural language reasoning steps under zero-sample training conditions to identify the native style of its natural language reasoning steps as the local style; S2: write a few-sample prompt example according to the local style, and the few-sample prompt example includes questions, reasoning steps and answers; S3: modify the format and reasoning steps of the few-sample prompt example according to the local style to align them with the local style, and ensure that the reasoning steps and answer formats of these examples are consistent with the native style of LLM; S4: optimize the prompt example retriever based on the few-sample prompt example to obtain the optimized prompt example retriever; S5: input the new science question into the optimized prompt example retriever, solve it in combination with the LLM reasoning path, and generate the answer.

[0067] Figure 1 As shown in , the left part A in the figure indicates that among the few thought chain prompt samples in the existing system, the CoT is either from the original dataset or hand-made, and its CoT text is "manual style". LLM is like imitating the tone of others when prompted with "manual style" examples. The right part B in the figure indicates that the CoT text style generated by the embodiment of the present invention under zero-sample training conditions is regarded as "local style". In order to obtain correct and formatted "native style" CoTs, the embodiment of the present invention proposes an aligned thought chain (AlignedCoT) prompt.

[0068] In one embodiment, the optimized prompt example retriever is used to select the prompt example with the highest similarity to the new science question for prompting.

[0069] In one embodiment, the step of obtaining the LLM reasoning path includes:

[0070] Set sampling strategies to generate different reasoning paths;

[0071] The answer is determined based on a majority voting mechanism.

[0072] In one embodiment, a machine learning method called few-shot learning is used to optimize the prompt example retriever. It should be noted that the selection of few-shot prompt examples usually considers the following factors:

[0073] Complexity of the problem: Choose a problem that is relatively difficult for traditional machine learning models, but can produce good results through few-shot learning.

[0074] Data availability: Choose a dataset that contains enough categories and instances for effective training and evaluation in few-shot learning.

[0075] Application scenarios: Choose problems that are relevant to real-world application scenarios, which can better illustrate the potential of few-shot learning in solving real-world problems.

[0076] In one embodiment, Figure 2 As shown, the aligned thought chain prompt includes three steps: detection, refinement and formatting, specifically:

[0077] S31: Input the questions in the newly compiled prompt example into the LLM, obtain the generated natural language reasoning steps and answers, identify the erroneous content in the reasoning steps and answers of the prompt example that do not conform to the local style, and implement the detection step; the erroneous content in this step may include misaligned formats, inconsistency with the reasoning steps used by the LLM, or differences in language expression, etc.

[0078] S32: Among all the identified error contents, select the first error in the current error content as the starting point for adjustment; the first error in this step is usually the most obvious problem in the sample or the problem that needs to be corrected immediately.

[0079] S33: Modify the first error to make it conform to the local style and update the new prompt example;

[0080] S34: Submit the updated new prompt example to LLM for processing, and only submit it to the modified content position. LLM generates the remaining reasoning steps and answers, which helps LLM to continue to generate content consistent with the style based on the adjusted example;

[0081] S35: Identify whether there are erroneous contents in the reasoning steps and answers of the updated newly compiled prompt examples that are inconsistent with the reasoning steps and answers of the remaining parts generated by LLM in S4. If so, repeat S32-S35 and modify only one error each time, so as to ensure that the entire example is gradually aligned with the native style of LLM and realize the refinement process; if not, the updated newly compiled prompt examples are stored as few-sample prompt examples to realize the final formatting.

[0082] It should be noted that the few-sample prompt examples refer to the optimization of the prompt example retriever using a smaller number of prompt examples.

[0083] Figure 2 In the figure, the changes in red are manually annotated. Each time, the first error is corrected, and then the LLM is queried to complete the text behind the last error.

[0084] The main results of the first aspect of the embodiments of the present invention are shown in Tables 1 and 2. All methods use greedy decoding (ie, the temperature is set to 0). The research results show that the method proposed by the present invention significantly improves the reasoning ability of LLM. The technical solution of the standard prompt is handcrafted by humans and has no intermediate reasoning steps, while the technical solution of the CoT prompt includes manually designed intermediate steps. The complex CoT technical solution serves as a strong baseline, which consists of examples with the most complex intermediate reasoning steps. In complex prompts, the intermediate steps are taken from the source dataset, and if the source data has no intermediate steps, they are manually annotated. AddSub and SingleEQ are not used as benchmarks for GPT-4 because GPT-4 has already performed close to 100% accuracy on these datasets using CoT prompts. Using the method of the present invention, it can be observed that the average + is improved by 3.2% and 1.7% on the experimental benchmarks of GPT-3.5-turbo and GPT-4, respectively.

[0085] like Figure 4 As shown, the self-consistent method is compared with the complex CoT, sampling multiple inference paths with the temperature set to 0.7. The present invention evaluates the accuracy of the answer of gpt-3.5-turbo.

[0086] Table 1 Comparison of answer accuracy using or not using the method of the present invention on GPT-3.5-Turbo

[0087]

[0088] Table 2 Comparison of accuracy with or without the method of the present invention on GPT-4

[0089]

[0090] Here, Δ represents the performance improvement.

[0091] Combined with the above experiments, the impact of each step in the thought chain alignment of the present invention is analyzed in detail: detection, refinement and formatting. The experiment uses a complex CoT and is conducted on GSM8K. The experiment is conducted by not using the thought chain alignment steps and by using the thought chain alignment steps step by step.

[0092] Table 4 Ablation study of different steps of AlignedCoT. Baseline model: GPT-3.5-Turbo (OpenAI 2022).

[0093]

[0094] The results are shown in Table 4. Complex CoT is equivalent to the case where these three steps are not used. AlignedCoT can also effectively improve performance without adopting all three steps. When only the first step (detection) is performed, the performance improvement over the baseline is +0.7%. When refinement or formatting steps are used on the basis of the detection step, the performance can be further improved. In particular, when all three steps are deployed simultaneously, the method of the present invention can be 2.5% higher than the complex CoT. In general, the three steps in the method of the present invention can make LLM perform better in multi-step reasoning, and they work best when deployed in combination.

[0095] Based on the same inventive idea as the method described in the first aspect of the embodiment, Figure 3 As shown, the second aspect of the embodiment of the present invention provides a science answering system based on aligned thinking chain, including: a zero-sample native style detection module, a few-sample prompt example generation module, a reasoning step format unification module, a retrieval enhancement module and a system integration module; wherein the zero-sample native style detection module inputs a science question to a known LLM, generates a natural language reasoning step under a zero-sample training condition, and identifies its natural language reasoning step native style as a local style; the few-sample prompt example generation module is used to write a few-sample prompt example according to the local style, and the few-sample prompt example includes a question, a reasoning step and an answer;

[0096] The reasoning step format unification module is used to modify the format and reasoning steps of the few-sample prompt examples according to the local style to align them with the local style; the retrieval enhancement module is used to optimize the prompt example retriever based on the few-sample prompt examples to obtain the optimized prompt example retriever; the system integration module is used to integrate the zero-sample native style detection module, the few-sample prompt example generation module, the reasoning step format unification module, and the retrieval enhancement module, input new science problems into the optimized prompt example retriever, and solve them in combination with the LLM reasoning path to generate answers.

[0097] In one embodiment, the optimized prompt example retriever is used to select the prompt example with the highest similarity to the new science question for prompting.

[0098] In one embodiment, a self-consistency module is further included, which is used to set a sampling strategy to generate different reasoning paths; and determine the answer based on a majority voting mechanism.

[0099] In one embodiment, the few-sample prompt example generation module includes an error content identification unit, a modification unit, and a submission unit:

[0100] The error content identification unit inputs the questions in the newly compiled prompt example into the LLM, obtains the generated natural language reasoning steps and answers, and identifies the error contents in the reasoning steps and answers of the prompt example that do not conform to the local style;

[0101] The modification unit selects the first error in the current error content from among all the identified error contents as the starting point for adjustment; modifies the first error to make it conform to the local style and updates the newly edited prompt example;

[0102] The submission unit submits the updated newly compiled prompt example to the LLM for processing, and only submits it to the modified content position, and the LLM generates the remaining reasoning steps and answers; the error content identification unit identifies whether there is error content in the reasoning steps and answers of the updated newly compiled prompt example that does not conform to the reasoning steps and answers of the remaining part generated by the LLM. If so, the modification unit is called again to modify the updated newly compiled prompt example, and only one error is modified each time, until the error content identification unit identifies the error-free content, and obtains the final prompt example as the few-sample prompt example.

[0103] In one embodiment, the system integration module is also used to perform system-level optimization to ensure that the modules work together efficiently, test and verify system performance, and make necessary adjustments and improvements.

[0104] Based on the same inventive idea as the method described in the first aspect of the embodiment, the third aspect of the embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the steps of a science problem-solving method based on aligned thinking chains are implemented when the processor executes the program.

[0105] Based on the same inventive idea as the method described in the first aspect of the embodiment, the fourth aspect of the embodiment of the present invention provides a non-transitory computer-readable storage medium on which a computer program is stored, which implements the steps of a science problem-solving method based on aligned thinking chains when the computer program is executed by a processor.

[0106] The electronic device and non-transitory computer-readable storage medium and method provided in the embodiments of the present invention correspond one to one. Therefore, the electronic device and the non-transitory computer-readable storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the device and the medium will not be repeated here.

[0107] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0108] The above is a detailed introduction to a method and system for solving science problems based on aligned thinking chains provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

[0109] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for solving science problems based on the alignment of thinking chains, characterized in that: The steps include: S1: Input science questions into known LLMs and generate natural language reasoning steps under zero-shot training conditions to identify the native style of natural language reasoning steps as the local style; S2: write a few-shot prompt example according to the local style, wherein the few-shot prompt example includes a question, a reasoning step, and an answer; S3: modifying the format and reasoning steps of the few-shot prompt example according to the local style to align it with the local style; S4: Based on the few-shot prompt examples, detect the native text style of the large language model, and convert the prompt example retriever into the native style of the large language model; S5: Input a new science question into the optimized prompt example retriever, solve it in combination with the LLM reasoning path, and generate an answer.

2. The method for solving science problems based on the alignment thinking chain according to claim 1 is characterized in that: The optimized prompt example retriever is used to select the prompt example with the highest similarity to the new science question for prompting.

3. The method for solving science problems based on the alignment thinking chain according to claim 1 is characterized in that: The steps of obtaining the LLM reasoning path include: Set sampling strategies to generate different reasoning paths; The answer is determined based on a majority voting mechanism.

4. The method for solving science problems based on the alignment thinking chain according to claim 1 is characterized in that: The alignment step in S3 includes: S31: Input the question in the newly compiled prompt example into the LLM, obtain the generated natural language reasoning steps and answers, and identify the erroneous content in the reasoning steps and answers of the prompt example that do not conform to the local style; S32: among all the identified error contents, select the first error in the current error content as the starting point for adjustment; S33: modifying the first error to make it conform to the local style and updating the newly edited prompt example; S34: Submitting the updated new prompt example to the LLM for processing, and only submitting it to the modified content position, and the LLM generates the remaining reasoning steps and answers; S35: Identify whether there are erroneous contents in the reasoning steps and answers of the updated newly compiled prompt example that are inconsistent with the reasoning steps and answers of the remaining part generated by the LLM in S4. If yes, repeat S32-S35 and modify only one error each time; if no, the updated newly compiled prompt example is stored as a few-sample prompt example.

5. A science answering system based on aligned thinking chains, characterized in that: include: Zero-shot native style detection module, few-shot prompt example generation module, reasoning step format unification module, retrieval enhancement module and system integration module; among them, The zero-shot native style detection module inputs a science question into a known LLM, generates a natural language reasoning step under a zero-shot training condition, and identifies the native style of its natural language reasoning step as a local style; The few-sample prompt example generation module is used to write a few-sample prompt example according to the local style, and the few-sample prompt example includes a question, a reasoning step and an answer; The reasoning step format unification module is used to modify the format and reasoning steps of the few-shot prompt example according to the local style to align it with the local style; The retrieval enhancement module is used to optimize the prompt example retriever based on the few-sample prompt example to obtain an optimized prompt example retriever; The system integration module is used to integrate the zero-sample native style detection module, the few-sample prompt example generation module, the reasoning step format unification module, and the retrieval enhancement module, input new science questions into the optimized prompt example retriever, solve them in combination with the LLM reasoning path, and generate answers.

6. The science answering system based on aligned thinking chain according to claim 5 is characterized in that: The optimized prompt example retriever is used to select the prompt example with the highest similarity to the new science question for prompting.

7. The science answering system based on aligned thinking chain according to claim 5 is characterized in that: It also includes a self-consistency module for setting sampling strategies to generate different reasoning paths; the answer is determined based on a majority voting mechanism.

8. The science answering system based on aligned thinking chain according to claim 5 is characterized in that: The few-sample prompt example generation module includes an error content identification unit, a modification unit, and a submission unit: The error content identification unit inputs the question in the newly compiled prompt example into the LLM, obtains the generated natural language reasoning steps and answers, and identifies the error content in the reasoning steps and answers of the prompt example that do not conform to the local style; The modification unit selects the first error in the current error content from among all the identified error contents as the starting point for adjustment; Modify the first error to make it conform to the local style and update the newly edited prompt example; The submitting unit submits the updated new prompt example to the LLM for processing, and only submits it to the content position where the modification is completed, and the LLM generates the reasoning steps and answers for the remaining part; The error content identification unit identifies whether there are error contents in the reasoning steps and answers of the updated newly compiled prompt example that are inconsistent with the reasoning steps and answers of the remaining part generated by the LLM. If so, the modification unit is called again to modify the updated newly compiled prompt example, and only one error is modified each time until the error content identification unit identifies no error content, and obtains the final prompt example as the minority sample prompt example.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the science solution method based on aligned thinking chains as described in any one of claims 1 to 4 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the science solution method based on aligned thinking chains as described in any one of claims 1 to 4 are implemented.

Images