Problem solving method, device, equipment and storage medium

Abstract

The present disclosure provides a problem processing method, device, equipment and storage medium, which relate to the fields of artificial intelligence and big data technology, and in particular to the field of knowledge question answering technology. The specific implementation scheme is: in response to the problem to be processed input by the user, the problem to be processed is disassembled into several sub-problems; according to the solution order and logical relationship between the several sub-problems, the solution steps of the several sub-problems are determined; according to the problem type of the problem to be processed, the corresponding problem processing engine is called to process several sub-problems according to the solution steps, and the engine execution result is obtained; based on the solution steps of the several sub-problems and the engine execution result, a prompt word is generated to guide the large language model to process the problem to be processed; the large language processing model is used to process the problem to be processed based on the prompt word, and the problem processing result is obtained.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence and big data technologies, specifically to the field of knowledge question answering technology, and in particular to a question processing method, device, equipment and storage medium. Background Art

[0002] In people's daily lives and corporate production activities, reasoning and calculation problems are highly purposeful and professional, and the results of the calculations are often used as the basis for subsequent decision-making, so the tolerance for computational errors is low; at the same time, in the current wave of industrial intelligent upgrading, the accuracy of reasoning calculations is also the basis for higher-level intelligent decision-making and the key to promoting the transformation of all walks of life towards a higher level of intelligence.

[0003] However, existing problem-solving methods based on large models or prompt words are limited by the computing power of the large language model itself, and have low accuracy in complex reasoning calculations and professional vertical calculations. At the same time, due to the high cost of updating model knowledge, they are unable to meet the computing power required to introduce new knowledge, especially for timeliness issues. Summary of the invention

[0004] The present disclosure provides a problem handling method, apparatus, device and storage medium.

[0005] According to a first aspect of the present disclosure, a problem handling method is provided, the method comprising:

[0006] In response to a pending problem input by a user, decomposing the pending problem into a plurality of sub-problems;

[0007] Determining the steps for solving the plurality of sub-problems according to the solving order and logical relationship between the plurality of sub-problems;

[0008] According to the problem type of the problem to be processed, calling the corresponding problem processing engine to process the plurality of sub-problems according to the solving steps, and obtaining the engine execution result;

[0009] Based on the solving steps of the plurality of sub-problems and the engine execution results, generating prompt words for guiding the large language model to process the problem to be processed;

[0010] The large language processing model is used to process the problem to be processed based on the prompt word to obtain a problem processing result.

[0011] Further, the large language processing model is used to process the problem to be processed based on the prompt word to obtain a problem processing result, including:

[0012] Using the large language processing model to process the problem to be processed based on the prompt word to obtain a preliminary processing result;

[0013] Performing an evaluation based on the prompt word and the preliminary processing result to obtain an evaluation result;

[0014] Based on the evaluation result and the problem to be processed, a correction prompt word for the preliminary processing result is determined, and the correction prompt word is input into the large language processing model, so as to use the large language processing model to correct the preliminary processing result based on the correction prompt word and the problem to be processed, so as to obtain the problem processing result.

[0015] Further, according to the problem type of the problem to be processed, the corresponding problem processing engine is called to process the plurality of sub-problems according to the solving steps to obtain the engine execution result, including:

[0016] If the problem type of the problem to be processed is a factual knowledge query problem, then the search engine is called to search for answers to the plurality of sub-problems according to the solving steps to obtain the search engine execution result.

[0017] The query data source of the factual knowledge query question includes at least structured data and unstructured data.

[0018] Further, according to the problem type of the problem to be processed, the corresponding problem processing engine is called to process the plurality of sub-problems according to the solving steps to obtain the engine execution result, including:

[0019] If the problem type of the problem to be processed is an operator calculation problem, the calculation engine is called to calculate the plurality of sub-problems according to the solving steps to obtain the calculation engine execution result.

[0020] Furthermore, based on the solving steps of the plurality of sub-problems and the engine execution result, a prompt word for guiding the large language model to process the problem to be processed is generated, including:

[0021] Splicing the solution steps of the plurality of sub-problems and the engine execution results to obtain a splicing result;

[0022] The splicing result is converted into question reasoning steps and question processing prompt information to obtain the prompt word.

[0023] According to a second aspect of the present disclosure, a problem handling device is provided, the device comprising:

[0024] A large language processing module, used for responding to a problem to be processed input by a user, and breaking down the problem to be processed into a plurality of sub-problems;

[0025] A thought chain planning module, used to determine the solution steps of the plurality of sub-problems according to the solution order and logical relationship between the plurality of sub-problems;

[0026] A central control service module, for invoking a corresponding problem processing engine to process the plurality of sub-problems according to the problem type to be processed and obtain an engine execution result according to the solution steps;

[0027] A prompt word construction module, used for generating prompt words for guiding the large language model to process the problem to be processed based on the solution steps of the plurality of sub-problems and the engine execution result;

[0028] The large language processing module is used to process the problem to be processed based on the prompt word to obtain a problem processing result.

[0029] Furthermore, the large language processing module is also used to process the problem to be processed based on the prompt word to obtain a preliminary processing result;

[0030] The device also includes:

[0031] An evaluation module, used for performing evaluation based on the prompt word and the preliminary processing result to obtain an evaluation result;

[0032] A reflection module is used to determine a correction prompt word for the preliminary processing result based on the evaluation result and the problem to be processed, and input the correction prompt word into the large language processing model, so as to use the large language processing model to correct the preliminary processing result based on the correction prompt word and the problem to be processed to obtain the problem processing result.

[0033] Furthermore, the device comprises:

[0034] A search engine is used to search for answers to the plurality of sub-questions according to the solving steps if the type of the problem to be processed is a factual knowledge query question, and obtain a search engine execution result.

[0035] The query data source of the factual knowledge query question includes at least structured data and unstructured data.

[0036] Furthermore, the device also includes:

[0037] A computing engine is used to calculate the plurality of sub-problems according to the solving steps to obtain a computing engine execution result if the problem type of the problem to be processed is an operator computing problem.

[0038] Furthermore, the prompt word construction module is also used for:

[0039] Splicing the solution steps of the plurality of sub-problems and the engine execution results to obtain a splicing result;

[0040] The splicing result is converted into question reasoning steps and question processing prompt information to obtain the prompt word.

[0041] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0042] at least one processor; and

[0043] a memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any one of the methods described above.

[0045] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute any one of the methods.

[0046] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the method described in the first aspect.

[0047] According to the technology disclosed in the present invention, the decomposition and processing of complex problems can be achieved, and the advantages of the problem processing engine and the large language model can be used to improve the efficiency and accuracy of problem processing. Specifically, the method can simplify complex problems into small problems that are easy to understand and solve by disassembling the problem to be processed to obtain several sub-problems, thereby reducing the difficulty and complexity of the problem. By determining the solution steps and logical relationships of several sub-problems, the structure and process of the problem can be clarified, and the solvability and operability of the problem can be improved. By calling the corresponding problem processing engine according to the problem type of the problem to be processed, the appropriate problem processing method can be selected for different types of problems, thereby improving the pertinence and effectiveness of problem processing. By generating prompt words for guiding the large language model to process the problem to be processed, the execution results of the problem processing engine can be used to provide useful information and guidance for the large language model, thereby improving the quality and accuracy of problem processing. By using the large language model to process the problem to be processed based on the prompt words, the powerful ability of the large language model can be used to conduct in-depth analysis and solution of the problem, thereby improving the level and innovation of problem processing.

[0048] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0050] Figure 1 is a flow chart of a problem handling method provided according to an embodiment of the present disclosure;

[0051] Figure 2 is a flow chart of an optional problem handling method that can implement the embodiments of the present disclosure;

[0052] Figure 3 is a flow chart of an optional problem handling method provided according to an embodiment of the present disclosure;

[0053] Figure 4 is a flow chart of an optional problem handling method provided according to an embodiment of the present disclosure;

[0054] Figure 5 is a flow chart of an optional problem handling method provided according to an embodiment of the present disclosure;

[0055] Figure 6 is a schematic diagram of a framework of an optional problem handling device provided according to an embodiment of the present disclosure;

[0056] Figure 7 is a schematic diagram of a framework of an optional problem handling device provided according to an embodiment of the present disclosure;

[0057] Figure 8 The present invention is a block diagram of an electronic device for implementing a problem handling method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0059] In people's daily lives and corporate production activities, reasoning and calculation problems are highly purposeful and professional, and the results of the calculations are often used as the basis for subsequent decision-making, so the tolerance for computational errors is low; at the same time, in the current wave of industrial intelligent upgrading, the accuracy of reasoning calculations is also the basis for higher-level intelligent decision-making and the key to promoting the transformation of all walks of life towards a higher level of intelligence.

[0060] For example, a user asks: "Who was the player with the most points in the NBA Finals the year Xiao M was born?" You need to first query the specific year Xiao M was born, then find the list of players participating in the NBA Finals that year, and finally sum up the points scored by each player in each game to get the result.

[0061] When solving the above problems, the methods used in the prior art all have problems and defects to a certain extent, such as:

[0062] Calculation based on the large language model itself: This method mainly relies on the ability of the large language model itself to answer the questions to be processed, and builds high-quality data to train the model to improve the calculation accuracy of the large language model itself. However, it is essentially a one-way decoding model in the form of autoregression, and the subsequent results are a prediction process. For complex problems to be processed, it is usually necessary to combine the context and various types of knowledge for comprehensive calculation. The accuracy of the calculation using the large language model alone is low, especially when it involves professional operators, such as eight-character calculation, cross-year date difference calculation, etc.

[0063] Calculation based on prompt words: It improves the accuracy of calculation to a certain extent, but due to the limitation of the large model itself, the accuracy is low in complex reasoning calculations and professional vertical calculations. At the same time, due to the high cost of updating model knowledge, the computing power required to introduce new knowledge, especially time-sensitive knowledge, cannot be met.

[0064] To solve the above problems, the present disclosure provides a method, device, equipment and storage medium for problem processing, which are applied to the fields of artificial intelligence and big data technology, especially knowledge question answering technology, so as to decompose and process complex problems, take advantage of the problem processing engine and large language model, and improve the efficiency and accuracy of problem processing.

[0065] Figure 1 is a flow chart of a problem handling method provided according to an embodiment of the present disclosure, such as Figure 1 As shown, the present disclosure provides a problem solving method, comprising the following method steps:

[0066] S101, in response to a pending problem input by a user, decomposing the pending problem into a plurality of sub-problems;

[0067] S102, determining the solution steps of the plurality of sub-problems according to the solution order and logical relationship between the plurality of sub-problems;

[0068] S103, according to the problem type of the problem to be processed, calling the corresponding problem processing engine to process the above-mentioned several sub-problems according to the above-mentioned solving steps, and obtaining the engine execution result;

[0069] S104, based on the steps of solving the above-mentioned several sub-problems and the above-mentioned engine execution results, generating prompt words for guiding the large language model to process the above-mentioned problem to be processed;

[0070] S105, using the large language processing model to process the problem to be processed based on the prompt words to obtain a problem processing result.

[0071] The problem-solving method provided by the embodiments of the present disclosure can be applied to various occasions where complex problems need to be analyzed and solved, such as education, scientific research, consulting, entertainment, etc. The method solves the problem that traditional problem-solving methods are difficult to solve complex problems, such as being unable to use the powerful capabilities of large language models to disassemble the structure of the problem and thus being unable to generate effective prompt words.

[0072] The disclosed solution is specifically based on a large language model, relying on the powerful generation and generalization capabilities of the large language model. Specifically, the large language model is used to receive the pending problems input by the user, and after the thinking chain planning module decomposes the above pending problems to obtain several sub-problems, the large language model is used to determine the solution order and logical relationship between the several sub-problems, that is, the solution order of the several sub-problems and the logical relationship between the several sub-problems. Finally, the large language model is used to process the pending problems according to the prompt words generated by other modules, and finally the problem processing results are obtained.

[0073] It should be noted that the problems to be solved in the present disclosure do not only refer to mathematical calculations. The problems to be solved in actual production activities usually involve the comprehensive application of factual knowledge, logical reasoning, and computing ability.

[0074] Specifically, this method solves the following technical problems:

[0075] Since traditional problem-solving methods usually handle problems as a whole, ignoring the internal structure and composition of the problem, the difficulty and complexity of problem handling increases, and the efficiency and accuracy of problem handling decreases. In the example disclosed in this disclosure, for example, after a large language model receives a piece of text input by a user and extracts the problem to be handled contained in the text, it decomposes the problem to be handled into several sub-problems, which can clearly show the internal structure and composition of the problem, reduce the difficulty and complexity of problem handling, and improve the efficiency and accuracy of problem handling.

[0076] After that, currently, the problem is usually classified according to its surface features, ignoring the essential features and internal laws of the problem, resulting in reduced pertinence and effectiveness of problem handling, and reduced quality and accuracy of problem handling. In this regard, the example disclosed herein analyzes the essential features, internal logic, laws, etc. of the problem to be handled in the text, determines the solution steps and logical relationships of several sub-problems, and can reveal the essential features and internal laws of the problem, thereby improving the pertinence and effectiveness of problem handling, and improving the quality and accuracy of problem handling.

[0077] An optional example is, for example, the text input by the user is "At present, the enrollment age of junior high school students tends to be lower than that of their parents' generation. In what year did the generation born in 1987 go to middle school?", then the large language model determines that the problem to be processed in the text is: "In what year did the generation born in 1987 go to middle school?", "In what year do people born in 1987 generally go to middle school?", then the expression result of the thinking chain planning module can use the following code to represent the solution steps of several sub-problems of the above problem to be processed:

[0078] x1 = set(person, birth year, 1987)

[0079] x2 = search(middle school,age)

[0080] x3 = compute(year calculation, [x1, x2])

[0081] Among them, set means: value setting, search means: calling the search engine for query, and compute means: calling the compute engine for calculation.

[0082] Furthermore, in the disclosed example, the large language model can also determine the problem type of the problem to be processed in the text, and by calling the corresponding problem processing engine according to the problem type of the problem to be processed, such as calling a computing engine or a retrieval engine, the problem of not being able to call a suitable problem processing engine is solved. According to the diversity and differences of the problems, a suitable problem processing engine can be selected, thereby improving the adaptability and flexibility of problem processing, and improving the level and innovation of problem processing.

[0083] In the example disclosed in this disclosure, in order to avoid ignoring the limitations and deficiencies of the problem processing engine due to directly using the execution results of the problem processing engine for processing, the execution results of the problem processing engine can be used to provide effective information and guidance for the large language model, and generate prompt words for guiding the large language model to process the pending problems, so as to improve the depth and breadth of problem processing, and improve the integrity and comprehensiveness of problem processing. Finally, using the prompt words determined by the large language model based on the above method to process the pending problems can effectively utilize the powerful capabilities of the large language model to deeply analyze and solve the received user problems.

[0084] In an optional example, still taking the text input by the user as "Currently, the enrollment age of junior high school students tends to be lower than that of their parents' generation. About what year did the generation born in 1987 go to middle school?", the solution steps (thinking chain code) of several sub-problems obtained by the thinking chain planning module and the engine execution results obtained by calling the retrieval / computing engine by the central control service module can be spliced ​​into the following splicing results:

[0085] x1 = set(person, birth year, 1987)

[0086] #Generally speaking, if children start primary school at the age of 6-7, they will start secondary school at the age of 12-13.

[0087] x2 = search(middle school,age)

[0088] x3 = compute(year calculation, [x1, x2])

[0089] The above concatenation results can be converted into the following question reasoning steps and question processing prompt information, so that the obtained prompt words can be input into the large language model to calculate the problem to be processed:

[0090] Problem reasoning steps:

[0091] 1. The person’s birth year is 1987.

[0092] 2. The age of entering middle school can be obtained from the following: Generally, people enter middle school at the age of 12-13. People enter middle school from the sixth grade of primary school.

[0093] 3. Calculate the year based on the person’s birth year and the age at which they started high school.

[0094] 4. If you cannot draw a conclusion, apologize and refuse to answer, and be careful not to draw the wrong conclusion.

[0095] Problem solving tips:

[0096] Based on the above information, answer the question and complete the dialogue: When will people born in 1987 go to middle school?

[0097] By using the above examples of the present disclosure, it is possible to decompose and process complex problems, and take advantage of the problem processing engine and large language model to improve the efficiency and accuracy of problem processing. Specifically, the method decomposes the problem to be processed in the user input text into several sub-problems, which can simplify complex problems into small problems that are easy to understand and solve, thereby reducing the difficulty and complexity of the problem.

[0098] By determining the solution steps and logical relationships of several sub-problems, the structure and process of the problem can be clarified, and the solvability and operability of the problem can be improved. Then, according to the type of the problem to be processed, the corresponding problem processing engine is called, and the appropriate problem processing method can be selected for different types of problems to improve the pertinence and effectiveness of problem processing. After generating prompt words to guide the large language model to process the problem to be processed, the execution results of the problem processing engine can be used to provide useful information and guidance for the large language model to improve the quality and accuracy of problem processing. Finally, by using the large language model to process the problem to be processed based on the prompt words, the powerful capabilities of the large language model can be used to conduct in-depth analysis and solution of the problem, improving the level and innovation of problem processing.

[0099] In one example, Figure 2 is a flowchart of an optional problem handling method provided according to an embodiment of the present disclosure, such as Figure 2 As shown, the above large language processing model is used to process the above problem to be processed based on the above prompt words to obtain the problem processing results, including:

[0100] S201, using the large language processing model to process the problem to be processed based on the prompt words to obtain a preliminary processing result;

[0101] S202, performing evaluation based on the prompt words and the preliminary processing results to obtain an evaluation result;

[0102] S203, based on the above-mentioned evaluation results and the above-mentioned problems to be processed, determine the correction prompt words of the above-mentioned preliminary processing results, and input the above-mentioned correction prompt words into the above-mentioned large language processing model, so as to use the above-mentioned large language processing model to correct the above-mentioned preliminary processing results based on the above-mentioned correction prompt words and the above-mentioned problems to be processed, so as to obtain the above-mentioned problem processing results.

[0103] Since the current mainstream large language models are all one-way decoding models based on transformers, the natural task is to predict the subsequent output results based on the previous input, and their single output results naturally lack a comprehensive understanding of contextual information. In order to improve the accuracy of its final problem processing, the evaluation module and the reflection module are introduced in this public example, and the large language model is interacted again by returning the modified prompt words based on the candidates to enhance the accuracy of the answer.

[0104] In the evaluation module, the answer to the problem processed by the large language model at the beginning is taken as the preliminary processing result and concatenated with the prompt word, and then the large language model is accessed again to evaluate the answer. For example, the prompt word and the preliminary processing result can be input into the large language processing model to evaluate the preliminary processing result using the large language processing model to obtain the evaluation result.

[0105] The following is an example of content that the assessment module uses to assess the prompt words:

[0106] You are a serious and responsible reasoning and calculation evaluation expert. You will receive a [question] and [candidate answer]. Please judge the answer according to the following steps:

[0107] Step 1: Please read the question and answer carefully and rate the factual correctness of each statement in your answer on a scale of 0-2 and give your reasons.

[0108] Step 2: Please read the question and answer carefully and rate the correctness of the result in your answer related to the question to be addressed on a scale of 0-2 and give your reasons.

[0109] Step 3: Please read the question and answer carefully and evaluate the completeness and coherence of the reasoning in your answer on a scale of 0-2, and give your reasons.

[0110] By adopting the above embodiment, the steps of evaluating and correcting the preliminary processing results are added, and the process of processing the pending problem based on the prompt words by the large language processing model can be improved to improve the quality and accuracy of the problem processing results.

[0111] In one example, a large language processing model is used to obtain prompt words according to the above method steps to analyze and solve the problem to be processed given in the user input text, and obtain a preliminary problem processing result, such as an answer, a solution, a suggestion, etc.

[0112] Afterwards, an evaluation module is used to perform an evaluation based on the prompt words and the preliminary processing results to obtain an evaluation result, such as a score, a grade, a feedback, etc., which is used to indicate the quality, correctness, rationality, etc. of the preliminary processing results.

[0113] In one example, the reflection module is connected with the evaluation module, and a large language model is constructed based on the evaluation results output by the evaluation module, the problems to be processed, and the preliminary processing results, and the correction prompt words are input into the large language processing model. The preliminary processing results are corrected based on the correction prompt words and the problems to be processed, and finally the problem processing results are obtained.

[0114] The following is an example of a correction prompt output by the reflection module:

[0115] You are a serious and responsible reasoning expert. You will receive a [question], [candidate answer] and [evaluation result]. Please revise the [candidate answer] based on the [question] and [evaluation result].

[0116] The following is the input information:

[0117] [Question]: XXXX

[0118] [Candidate answer]: XXXX

[0119] [Evaluation results]: XXXX

[0120] In the above example, an evaluation is performed based on the prompt word and the preliminary processing result to obtain the evaluation result; based on the evaluation result and the problem to be processed, the revised prompt word of the preliminary processing result is determined, and then the preliminary processing result is revised based on the revised prompt word and the problem to be processed by using the above large language processing model, which is an optimization adjustment of the preliminary processing result to obtain the problem processing result. For example, a modified answer, an optimized solution, an improved suggestion, etc. can be obtained, so that the problem processing result obtained in this way better meets the requirements and purpose of the problem.

[0121] In this example, evaluating the preliminary processing results can reveal the deficiencies and errors in the preliminary processing results, and improve the reliability and correctness of the problem processing results. Correcting the preliminary processing results can improve the advantages and rationality of the preliminary processing results, and improve the optimization and innovation of the problem processing results. Specifically, using the large language processing model for evaluation and correction can utilize the powerful capabilities of the large language processing model to improve the level and quality of the problem processing results.

[0122] By using the above examples of the present disclosure, it is possible to optimize and improve the process of a large language model processing a problem to be processed, and utilize the self-evaluation and self-correction capabilities of the large language model to improve the quality and accuracy of the problem processing results.

[0123] In an optional example, Figure 3 is a flowchart of an optional problem handling method provided according to an embodiment of the present disclosure, such as Figure 3 As shown, according to the problem type of the above-mentioned problem to be processed, the corresponding problem processing engine is called to process the above-mentioned several sub-problems according to the above-mentioned solution steps to obtain the engine execution results, including:

[0124] S301, if the problem type of the above-mentioned problem to be processed is a factual knowledge query problem, then call the search engine to search for answers to the above-mentioned several sub-problems according to the above-mentioned solving steps to obtain the search engine execution result.

[0125] In the above example, the query data source of the above factual knowledge query problem includes at least: structured data and unstructured data.

[0126] In the example disclosed herein, if the type of the problem to be processed is a factual knowledge query problem, the search engine is called to search for answers to several sub-questions according to the solution steps to obtain a search engine execution result. This method is a process for factual knowledge query problems, that is, using a search engine to search for several sub-questions according to the solution steps, obtaining relevant information from the query data source, and obtaining a search engine execution result, such as a text, a chart, a link, etc., which is used to represent the answer or reference to the sub-question.

[0127] In the example disclosed herein, the selection of the query data source is to use the search engine to query from different types of data sources according to the characteristics and requirements of the sub-questions to improve the coverage and quality of the query. Among them, structured data refers to data organized according to certain rules and formats, such as databases, tables, lists, etc.; unstructured data refers to data without fixed rules and formats, such as web pages, documents, pictures, etc.

[0128] In one example, the retrieval engine is mainly used to solve the problem of high cost of updating model knowledge. This module is responsible for: factual knowledge query. The main query sources include two categories, one is structured data with higher accuracy; the other is unstructured data with higher coverage, such as documents, web pages, etc.; during specific execution, structured data is queried first, and unstructured data is queried without results; the reason for distinguishing the two types of data is mainly that, under normal circumstances, the quality of structured data within the enterprise is higher, and the knowledge retrieved is more accurate.

[0129] For example, for the statement in the above example: x2 = search (middle school, age), after execution, the search engine execution result based on the general search is: "Generally speaking, if a child starts primary school at the age of 6-7, then the age for middle school is 12-13 years old."

[0130] Using the above example, calling the search engine to retrieve the answer to the sub-question can quickly obtain relevant information and improve the speed and efficiency of problem processing. By querying from structured data and unstructured data, different types of data sources can be widely covered to improve the scope and quality of problem processing. It can realize the processing of factual knowledge query problems and use the fast and accurate capabilities of the search engine to improve the efficiency and accuracy of problem processing.

[0131] In an optional example, Figure 4 is a flowchart of an optional problem handling method provided according to an embodiment of the present disclosure, such as Figure 4As shown, according to the problem type of the above-mentioned problem to be processed, the corresponding problem processing engine is called to process the above-mentioned several sub-problems according to the above-mentioned solution steps to obtain the engine execution results, including:

[0132] S401, if the problem type of the problem to be processed is an operator calculation problem, then call the calculation engine to calculate the above-mentioned several sub-problems according to the above-mentioned solution steps to obtain the calculation engine execution result.

[0133] Optionally, the above example is the processing of an operator calculation problem, that is, using a calculation engine to calculate several sub-problems according to the solution steps, obtaining relevant data from the calculation data source, and obtaining a calculation engine execution result, such as a number, a formula, a graph, etc., which is used to represent the calculation result or demonstration of the sub-problem.

[0134] Among them, operator calculation problems refer to problems involving operations, transformations, derivations, and other processes in the fields of mathematics, physics, chemistry, etc., such as solving equations, derivative integrals, balancing reactions, etc.

[0135] In one example, the calculation engine is mainly used to solve the problem of low accuracy of large model calculations for professional vertical operators (such as calculations of birth dates and sunrise and sunset times, which are currently poorly internalized by large language models). This problem is solved by the built-in calculation operators implemented by the system (such operators that are currently poorly internalized by large language models), while general calculations, such as basic arithmetic and comparisons, are solved by constructing prompt words.

[0136] At the same time, because the built-in functions have strict format requirements on the input, such as the date requirement yyyy year MM month dd day HH hour mm minute ss second, and the user's input may vary, we can use the generalization of the large language model here to normalize the format of several sub-problems in the thinking chain planning module.

[0137] Example 1: x1 = compute (eight-character calculation, [November 15, 2003 08:24:33])

[0138] This calculation belongs to professional operator calculation. The built-in operator is called and the output result is as follows: Yichou year Dinghai month Wuwu day Bingchen hour

[0139] Example 2: x1 = set(person, birthyear, 1987)

[0140] x2 = search(middle school,age)

[0141] x3 = compute(year calculation, [x1, x2])

[0142] As shown in the search engine example, the search result of x2 is a natural language, and the built-in operator cannot be called, so no calculation is performed.

[0143] By using the above example, the operator calculation problem can be processed, and the accuracy and precision of the problem processing can be improved by using the precise and efficient capabilities of the calculation engine. Specifically, by calling the calculation engine to calculate the sub-problem, relevant data can be accurately obtained, and the correctness and precision of the problem processing can be improved.

[0144] As an alternative example, Figure 5 is a flowchart of an optional problem handling method provided according to an embodiment of the present disclosure, such as Figure 5 As shown, based on the solution steps of the above several sub-problems and the above engine execution results, prompt words are generated to guide the large language model to process the above problems to be processed, including:

[0145] S501, combining the solution steps of the above-mentioned several sub-problems and the above-mentioned engine execution results to obtain a combined result;

[0146] S502, converting the above splicing results into question reasoning steps and question processing prompt information to obtain the above prompt words.

[0147] The process of generating prompt words for guiding the large language model to process the problem to be processed based on the solution steps of several sub-problems and the engine execution results has been improved, and a step of converting the splicing results has been added to improve the quality and accuracy of problem processing.

[0148] In the above example, by splicing the solution steps of several sub-problems and the engine execution results, that is, splicing the solution steps of several sub-problems and the engine execution results together in a certain order and format, a splicing result, such as a text, a list, a table, etc., is obtained to represent the overall structure and process of the problem.

[0149] Afterwards, the concatenation results are converted into question reasoning steps and question processing prompt information to obtain prompt words. For example, the concatenation results can be semantically converted, that is, a prompt word, such as a sentence, a paragraph, a dialogue, etc., is generated based on the concatenation results using a large language processing model to represent the question reasoning steps and processing prompt information, providing useful information and guidance for the large language model.

[0150] Through the above examples, the splicing results are converted into question reasoning steps and question processing prompt information, which can improve the semantics and readability of the prompt words and make the prompt words more consistent with the logic and purpose of the question. Using a large language processing model for conversion can improve the creativity and flexibility of the prompt words; by generating prompt words, the quality and accuracy of question processing can be improved, making the problem processing results more consistent with the requirements and expectations of the problem.

[0151] The problem handling method provided by the disclosed solution can be implemented based on a problem handling system or device. Figure 6 is a flow chart of an optional problem handling device provided according to an embodiment of the present disclosure, such as Figure 6 As shown, an optional architecture diagram of the device may include the following modules: a large language model, a thought chain planning module, a central control service module, a search engine, a calculation engine, a prompt word construction module, an evaluation module, and a reflection module, wherein:

[0152] The disclosed solution is specifically based on a large language model, relying on the powerful generation and generalization capabilities of the large language model. Specifically, the large language model is used to receive the pending problems contained in the text input by the user, and after the thinking chain planning module disassembles the above pending problems to obtain several sub-problems, the large language model is used to determine the solution order and logical relationship between the several sub-problems, that is: the solution order of several sub-problems, and the logical relationship between several sub-problems. Finally, the large language model is used to process the pending problems according to the prompt words generated by other modules, and finally the problem processing results are obtained.

[0153] After using the large language model to determine the solution order and logical relationship between several sub-problems, the thinking chain planning module is used to decompose the pending problem input by the user, decomposing a complex pending problem into several small problems, and planning the solution steps of several sub-problems (which can be understood as reasoning calculation steps) based on the solution order and logical relationship between several sub-problems generated by the large language model.

[0154] For example, if the large language model extracts a question to be processed contained in a text input by the user, which is "Who was the player with the most points in the NBA Finals in the year when Xiao M was born?", it is necessary to first query the specific year when Xiao M was born, then solve the list of players participating in the NBA Finals of that year, and finally sum up the scores of each player in each game to get the result, thus obtaining the solution steps of several independent sub-problems.

[0155] Different from the thinking chain that directly adopts natural language, the embodiment of the present disclosure adopts an agreed code code to represent the solution steps of several sub-problems. The predetermined code is a customized code that can be understood by the central control service module, which mainly expresses the solution steps in a structured manner, thereby isolating the thinking chain planning module, the central control service module (such as the search engine, the calculation engine) and the prompt word construction module.

[0156] In an optional example, if the problem to be processed contained in the text input by the user is extracted as: When will the generation born in 1987 go to middle school, the expression result of the thinking chain planning module can use the following code to represent the solution steps of several sub-problems obtained by decomposing the problem to be processed:

[0157] x1 = set(person, birth year, 1987)

[0158] x2 = search(middle school,age)

[0159] x3 = compute(year calculation, [x1, x2])

[0160] Among them, set means: value setting, search means: calling the search engine for query, and compute means: calling the compute engine for calculation.

[0161] The central control service module is responsible for understanding the output of the thinking chain planning module, and calling the retrieval engine or calculation engine to perform knowledge query or calculation according to the solution steps of several sub-problems planned by the thinking chain planning module, and outputting the engine execution results to the prompt word construction module.

[0162] The search engine is mainly used to solve the problem of high cost of updating model knowledge and is responsible for factual knowledge query. The main query sources include two categories: one is structured data with higher accuracy; the other is unstructured data with higher coverage, such as documents, web pages, etc.; during specific execution, structured data is queried first, and unstructured data is queried without results; the reason for distinguishing the two types of data is mainly that, under normal circumstances, the quality of structured data within the enterprise is higher, and the knowledge retrieved is more accurate.

[0163] For example, for the statement in the above example: x2 = search (middle school, age), after execution, the search engine execution result based on the general search is: "Generally speaking, if a child starts primary school at the age of 6-7, then the age for middle school is 12-13 years old."

[0164] The calculation engine is mainly used to solve the problem of low accuracy of large model calculations for professional vertical operators (such as calculations of birth dates and sunrise and sunset times, which are currently poorly internalized by large language models). This problem is solved by the built-in calculation operators implemented by the system (such operators that are currently poorly internalized by large language models). For general calculations, such as basic arithmetic and comparison, this problem is solved by constructing prompt words.

[0165] At the same time, because the built-in functions have strict format requirements on the input, such as the date requirement yyyy year MM month dd day HH hour mm minute ss second, and the user's input may vary, we can use the generalization of the large language model here to normalize the format of several sub-problems in the thinking chain planning module.

[0166] Example 1: x1 = compute (eight-character calculation, [November 15, 2003 08:24:33])

[0167] This calculation belongs to professional operator calculation. The built-in operator is called and the output result is as follows: Yichou year Dinghai month Wuwu day Bingchen hour

[0168] Example 2: x1 = set(person, birthyear, 1987)

[0169] x2 = search(middle school,age)

[0170] x3 = compute(year calculation, [x1, x2])

[0171] As shown in the search engine example, the search result of x2 is a natural language, and the built-in operator cannot be called, so no calculation is performed.

[0172] The prompt word construction module is mainly based on concatenating the prompt words of the output of the above process to generate prompt words for guiding the large language model to process the problem to be processed, inputting the prompt words into the large language model for calculation and processing, and replying the problem processing results to the user in the form of natural language.

[0173] In an optional example, as shown in Table 1 below, if the problem to be processed is: When will the generation born in 1987 go to middle school, then the solution steps (thinking chain code) of several sub-problems obtained by the thinking chain planning module and the engine execution results obtained by calling the retrieval / computation engine by the central control service module can be spliced ​​into the following splicing results:

[0174] x1 = set(person, birth year, 1987)

[0175] #Generally speaking, if children start primary school at the age of 6-7, they will start secondary school at the age of 12-13.

[0176] x2 = search(middle school,age)

[0177] x3 = compute(year calculation, [x1, x2])

[0178] The above concatenation results can be converted into the following question reasoning steps and question processing prompt information, so that the obtained prompt words can be input into the large language model to calculate the problem to be processed:

[0179] Problem reasoning steps:

[0180] 1. The birth year of the person (not a specific person but a type of person, used to infer social phenomena) is 1987.

[0181] 2. The age of entering middle school can be obtained from the following: Generally, people enter middle school at the age of 12-13. People enter middle school from the sixth grade of primary school.

[0182] 3. Calculate the year based on the person’s birth year and the age at which they started high school.

[0183] 4. If you cannot draw a conclusion, apologize and refuse to answer, and be careful not to draw the wrong conclusion.

[0184] Problem solving tips:

[0185] Based on the above information, answer the following questions and complete the dialogue: When will the generation born in 1987 go to middle school?

[0186] Table 1

[0187]

[0188]

[0189] In an optional example, the above prompt words may also include a question answering method (optional), for example: 1. Answer the question directly based on the reasoning conclusion. If the reasoning conclusion is empty, perform reasoning calculations based on the reasoning steps to obtain the reasoning conclusion. If the conclusion cannot be deduced, apologize and refuse to answer, and be careful not to give a wrong conclusion. 2. Provide additional introduction based on the reasoning steps and expanded knowledge.

[0190] Since the current mainstream large language models are all one-way decoding models based on transformers, the natural task is to predict the output of the next token based on the previous input, and the result of its single output naturally lacks a comprehensive understanding of contextual information. In order to improve the accuracy of its final problem processing, the evaluation module and the reflection module are introduced in this public example, which interact with the large language model again based on candidate returns and prompt words to enhance the accuracy of the answer.

[0191] In the evaluation module, the answer to the problem processed by the large language model at the beginning is taken as the preliminary processing result and concatenated with the prompt word, and then the large language model is accessed again to evaluate the answer, that is, the prompt word and the preliminary processing result are input into the large language processing model, so as to evaluate the preliminary processing result using the large language processing model and obtain the evaluation result.

[0192] The following are examples of prompt words that the assessment module uses for assessments:

[0193] You are a serious and responsible reasoning and calculation evaluation expert. You will receive a [question] and [candidate answer]. Please judge the answer according to the following steps:

[0194] Step 1: Please read the question and answer carefully and rate the factual correctness of each statement in your answer on a scale of 0-2 and give your reasons.

[0195] Step 2: Please read the question and answer carefully and rate the correctness of the result in your answer related to the question to be addressed on a scale of 0-2 and give your reasons.

[0196] Step 3: Please read the question and answer carefully and evaluate the completeness and coherence of the reasoning in your answer on a scale of 0-2, and give your reasons.

[0197] The following is the input information:

[0198] [Question]: XXXX

[0199] [Candidate answer]: XXXX

[0200] In one example, the reflection module is connected with the evaluation module, and based on the evaluation results, pending issues and preliminary processing results output by the evaluation module, a prompt word input large language model is further constructed. The large language processing model is used to correct the preliminary processing results based on the evaluation results and pending issues, and finally the problem processing results are obtained.

[0201] The following are examples of reflection prompts used in the reflection module:

[0202] You are a serious and responsible reasoning expert. You will receive a [question], [candidate answer] and [evaluation result]. Please revise the [candidate answer] based on the [question] and [evaluation result].

[0203] The following is the input information:

[0204] [Question]: XXXX

[0205] [Candidate answer]: XXXX

[0206] [Evaluation results]: XXXX

[0207] The problem solving method provided in this disclosure can effectively improve the reasoning and computing capabilities of large language models:

[0208] 1. The overall system is based on the memory expression of the thinking chain code, which isolates the step planning, retrieval / computation engine call and prompt word construction, and executes them separately, effectively utilizing the characteristics of each sub-technology and significantly improving the end-to-end computing effect of the large language model.

[0209] 2. Based on thought chain planning, complex problems can be broken down into multiple independent sub-problems, thereby guiding the large language model to perform reasoning calculations and improve accuracy.

[0210] 3. The retrieval engine can effectively solve the knowledge updating problem of large language models.

[0211] 4. The built-in operators based on the computing engine can effectively improve the complex and professional computing capabilities of large language models.

[0212] 5. Based on the prompt word construction, we can use the results of the retrieval and calculation of some of the aforementioned sub-questions to generate more accurate prompt words and improve the final calculation effect of the large language model.

[0213] 6. Based on the evaluation module and reflection module, the first response produced is evaluated, reflected and corrected, which can overcome the autoregressive defects of the large language model and improve the final calculation accuracy.

[0214] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0215] According to an embodiment of the present disclosure, Figure 7 is a schematic diagram of a framework of a problem handling device provided according to an embodiment of the present disclosure, such as Figure 7 As shown, the present disclosure further provides a problem handling device, the problem handling device 700 comprising:

[0216] The large language processing module 701 is used to respond to the problem to be processed input by the user and decompose the problem to be processed into a plurality of sub-problems;

[0217] The thought chain planning module 702 is used to determine the solution steps of the above-mentioned several sub-problems according to the solution order and logical relationship between the above-mentioned several sub-problems;

[0218] The central control service module 703 is used to call the corresponding problem processing engine according to the problem type of the problem to be processed to process the above-mentioned several sub-problems according to the above-mentioned solution steps, and obtain the engine execution result;

[0219] A prompt word construction module 704 is used to generate prompt words for guiding the large language model to process the problem to be processed based on the solution steps of the above-mentioned several sub-problems and the above-mentioned engine execution results;

[0220] The large language processing module 701 is used to process the problem to be processed based on the prompt words to obtain a problem processing result.

[0221] It should be noted that the device embodiments in the embodiments of the present disclosure can refer to the above method embodiments, which will not be described in detail.

[0222] According to one or more embodiments of the present disclosure, the above-mentioned large language processing module is also used to process the above-mentioned problem to be processed based on the above-mentioned prompt words to obtain a preliminary processing result.

[0223] According to one or more embodiments of the present disclosure, the above-mentioned device also includes: an evaluation module, which is used to evaluate based on the above-mentioned prompt words and the above-mentioned preliminary processing results to obtain evaluation results; a reflection module, which is used to determine the correction prompt words of the above-mentioned preliminary processing results based on the above-mentioned evaluation results and the above-mentioned problems to be processed, and input the above-mentioned correction prompt words into the above-mentioned large language processing model, so as to use the above-mentioned large language processing model to correct the above-mentioned preliminary processing results based on the above-mentioned correction prompt words and the above-mentioned problems to be processed, so as to obtain the above-mentioned problem processing results.

[0224] According to one or more embodiments of the present disclosure, the above-mentioned device includes:

[0225] A search engine is used to search for answers to the above-mentioned sub-questions according to the above-mentioned solution steps if the question type of the above-mentioned question to be processed is a factual knowledge query question, and obtain the search engine execution result,

[0226] The query data source for the above-mentioned factual knowledge query problem includes at least: structured data and unstructured data.

[0227] According to one or more embodiments of the present disclosure, the above device further includes:

[0228] A computing engine is used to calculate the above-mentioned several sub-problems according to the above-mentioned solving steps if the problem type of the above-mentioned problem to be processed is an operator computing problem, and obtain the computing engine execution result.

[0229] According to one or more embodiments of the present disclosure, the prompt word construction module is further used to:

[0230] Splicing the solution steps of the above several sub-problems and the above engine execution results to obtain a splicing result;

[0231] The above concatenation results are converted into question reasoning steps and question processing prompt information to obtain the above prompt words.

[0232] It should be noted that the device embodiments in the embodiments of the present disclosure can refer to the above method embodiments, which will not be described in detail.

[0233] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0234] According to an embodiment of the present disclosure, the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute any one of the above methods.

[0235] According to an embodiment of the present disclosure, the present disclosure provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0236] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0237] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0238] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0239] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the problem handling method. For example, in some embodiments, the problem handling method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the problem handling method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the problem handling method in any other appropriate manner (e.g., by means of firmware).

[0240] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0241] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0242] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0243] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0244] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0245] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0246] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0247] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for solving a problem, the method comprising: In response to a pending problem input by a user, decomposing the pending problem into a plurality of sub-problems; Determining the steps for solving the plurality of sub-problems according to the solving order and logical relationship between the plurality of sub-problems; According to the problem type of the problem to be processed, calling the corresponding problem processing engine to process the plurality of sub-problems according to the solving steps, and obtaining the engine execution result; Based on the solving steps of the plurality of sub-problems and the engine execution result, generating prompt words for guiding the large language model to process the problem to be processed; Using a large language processing model to process the problem to be processed based on the prompt word to obtain a problem processing result; Wherein, the problem to be processed is processed based on the prompt word using a large language processing model to obtain a problem processing result, including: Using the large language processing model to process the problem to be processed based on the prompt word to obtain a preliminary processing result; wherein the preliminary processing result includes: an answer, a solution or a suggestion; Performing an evaluation based on the prompt word and the preliminary processing result to obtain an evaluation result; wherein the evaluation result includes: a score, a grade or a feedback; Based on the evaluation result and the problem to be processed, a correction prompt word for the preliminary processing result is determined, and the correction prompt word is input into the large language processing model, so as to use the large language processing model to correct the preliminary processing result based on the correction prompt word and the problem to be processed, so as to obtain the problem processing result; wherein the problem processing result includes: a modified answer, an optimized solution or an improved suggestion.

2. The method according to claim 1, wherein: According to the problem type of the problem to be processed, the corresponding problem processing engine is called to process the plurality of sub-problems according to the solving steps to obtain the engine execution result, including: If the problem type of the problem to be processed is a factual knowledge query problem, then the search engine is called to search for answers to the plurality of sub-problems according to the solving steps to obtain the search engine execution result. The query data source of the factual knowledge query question includes at least structured data and unstructured data.

3. The method according to claim 1, wherein: According to the problem type of the problem to be processed, the corresponding problem processing engine is called to process the plurality of sub-problems according to the solving steps to obtain the engine execution result, including: If the problem type of the problem to be processed is an operator calculation problem, the calculation engine is called to calculate the plurality of sub-problems according to the solving steps to obtain the calculation engine execution result.

4. The method according to any one of claims 1 to 3, wherein: Based on the solving steps of the plurality of sub-problems and the engine execution results, a prompt word is generated for guiding the large language model to process the problem to be processed, including: Splicing the solution steps of the plurality of sub-problems and the engine execution results to obtain a splicing result; The splicing result is converted into question reasoning steps and question processing prompt information to obtain the prompt word.

5. A problem handling device, comprising: A large language processing module, used for responding to a problem to be processed input by a user, and breaking down the problem to be processed into a plurality of sub-problems; A thought chain planning module, used to determine the solution steps of the plurality of sub-problems according to the solution order and logical relationship between the plurality of sub-problems; A central control service module, for invoking a corresponding problem processing engine to process the plurality of sub-problems according to the problem type to be processed and obtain an engine execution result according to the solution steps; A prompt word construction module, used for generating prompt words for guiding the large language model to process the problem to be processed based on the solution steps of the plurality of sub-problems and the engine execution result; The large language processing module is used to process the problem to be processed based on the prompt word to obtain a problem processing result; The large language processing module is further used to process the problem to be processed based on the prompt word to obtain a preliminary processing result; wherein the preliminary processing result includes: an answer, a solution or a suggestion; The device also includes: An evaluation module, configured to perform evaluation based on the prompt word and the preliminary processing result to obtain an evaluation result; wherein the evaluation result includes: a score, a grade or a feedback; A reflection module is used to determine a correction prompt word for the preliminary processing result based on the evaluation result and the problem to be processed, and input the correction prompt word into the large language processing model, so as to use the large language processing model to correct the preliminary processing result based on the correction prompt word and the problem to be processed to obtain the problem processing result; wherein the problem processing result includes: a modified answer, an optimized solution or an improved suggestion.

6. The device according to claim 5, wherein: The device also includes: A search engine is used to search for answers to the plurality of sub-questions according to the solving steps if the type of the problem to be processed is a factual knowledge query question, and obtain a search engine execution result. The query data source of the factual knowledge query question includes at least structured data and unstructured data.

7. The device according to claim 5, wherein: The device also includes: A computing engine is used to calculate the plurality of sub-problems according to the solving steps to obtain a computing engine execution result if the problem type of the problem to be processed is an operator computing problem.

8. The device according to any one of claims 5 to 7, wherein: The prompt word building module is also used for: Splicing the solution steps of the plurality of sub-problems and the engine execution results to obtain a splicing result; The splicing result is converted into question reasoning steps and question processing prompt information to obtain the prompt word.

9. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

11. A computer program product, comprising a computer program, which implements the steps of the method according to any one of claims 1 to 4 when executed by a processor.

Images