A cause analysis generation method and device, a storage medium and an equipment

Abstract

The application discloses a mistake reason analysis generation method and device, a storage medium and equipment. The method comprises the following steps: first, obtaining a target answer text; determining a target question text and a standard answer text corresponding to the target answer text; then, based on pre-constructed question information of the target question text and a problem solving method set, performing answer analysis on the target answer text to generate target answer information corresponding to the target answer text; wherein, the problem solving method set is pre-iteratively generated based on the target question text and the standard answer text; then, the target answer information, the standard answer information and the question information are input into a preset large language model combined with a first prompt to obtain a mistake reason analysis result for the target answer text output by the model; wherein, the standard answer information is pre-analytically generated based on the question information and the standard answer text. Therefore, the generation efficiency and accuracy of the mistake reason analysis result can be improved, and the interactive experience of the user is improved.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method, apparatus, storage medium and device for generating error cause analysis. Background Technology

[0002] With the rapid development of information technologies such as artificial intelligence and the Internet of Things, the application scenarios of human-computer interaction are becoming increasingly widespread. Various large language models (LLMs) are appearing in people's lives and work, such as Chat Generative Pre-trained Transformer (ChatGPT), smart speakers, and smart TVs. They can provide intelligent interactive functions for many application scenarios, such as data analysis, to assist users in completing various behavioral intentions.

[0003] Currently, when using Large Language Models (LLMs) to generate error analysis, the common approach is to directly design prompts, inputting the question, standard answer, and student answer into the LLM, and then using the task description to allow the model to generate error analysis in one go. However, this places very high demands on the model's capabilities, often making it difficult to generate accurate error analysis and correction guidance, thus reducing the user's interactive experience. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, storage medium, and device for error cause analysis, which can improve the efficiency and accuracy of error cause analysis results generation, thereby enhancing the user's interactive experience.

[0005] This application provides a method for generating error cause analysis, including:

[0006] Obtain the target response text to be parsed; and determine the target question text and standard response text corresponding to the target response text;

[0007] Based on the pre-constructed set of question information and problem-solving approaches for the target question text, the answer analysis is performed on the target answer text to generate target answer information corresponding to the target answer text; the set of problem-solving approaches is pre-generated iteratively based on the target question text and standard answer text;

[0008] The target answer information, standard answer information, and question information, combined with the first prompt instruction, are input into a preset large language model to obtain the error analysis results of the target answer text output by the large language model; the standard answer information is generated in advance based on the question information and the standard answer text.

[0009] In one possible implementation, the question information of the target question text is generated as follows:

[0010] Using the first preset prompt instruction and the Large Language Model (LLM), the question analysis and standard problem-solving approach are generated based on the target question text and the standard answer text.

[0011] Using the second preset prompt instruction and the Large Language Model (LLM), a set of candidate problem-solving approaches is iteratively generated based on the standard problem-solving approach.

[0012] The large language model (LLM) is used to sort the different problem-solving approaches in the candidate problem-solving approach set, and K problem-solving approaches that meet the preset conditions are selected from the sorting results to form a problem-solving approach set; and the problem-solving approach set, the target problem text, and the problem parsing are used to form the problem information of the target problem text; where K is a positive integer greater than 0;

[0013] The large language model LLM is obtained by training language rules and patterns using a large-scale language dataset through autoregressive generation.

[0014] In one possible implementation, the step of using a second preset prompt instruction and a large language model (LLM) to iteratively generate a set of candidate problem-solving approaches based on the standard problem-solving approach includes:

[0015] Using the second preset prompt instruction and the Large Language Model (LLM), candidate problem-solving approaches are generated based on the standard problem-solving approach.

[0016] Calculate the similarity between the candidate solution approach and the standard solution approach, and determine whether the similarity is lower than a preset threshold. If so, the candidate solution approach and the standard solution approach constitute an initial set of candidate solution approaches.

[0017] The second preset prompt instruction and the Large Language Model (LLM) are reused to generate updated candidate problem-solving approaches based on the standard problem-solving approach. The problem-solving approaches in the candidate problem-solving approach set are then iteratively updated based on the similarity calculation results between the updated candidate problem-solving approaches and all problem-solving approaches in the initial candidate problem-solving approach set.

[0018] In one possible implementation, the similarity is text similarity; the preset threshold is 0.4.

[0019] In one possible implementation, the standard response information is generated as follows:

[0020] Using the third preset prompt instruction and the Large Language Model (LLM), the standard answer text is analyzed based on the question information and standard problem-solving approach of the target question text, generating the standard answer analysis result corresponding to the standard answer text; and the standard answer information is constructed using the standard answer analysis result, the standard answer text, and the standard problem-solving approach.

[0021] In one possible implementation, the step of analyzing the target answer text based on a pre-constructed set of question information and problem-solving approaches from the target question text, and generating target answer information corresponding to the target answer text, includes:

[0022] Using the second prompt instruction and the Large Language Model (LLM), based on the question information of the target question text and the target answer text, the most matching problem-solving approach and its sequence number are selected from the set of problem-solving approaches;

[0023] Based on the question information of the target question text and the most matching solution approach and its sequence number, the target answer text is analyzed to generate the target answer analysis result corresponding to the target answer text; and the target answer information corresponding to the target answer text is constructed by using the target answer analysis result, the target answer text and the most matching solution approach.

[0024] In one possible implementation, the step of inputting the target answer information, standard answer information, and question information, combined with the first prompt instruction, into a preset large language model to obtain the error analysis results of the large language model for the target answer text includes:

[0025] The problem-solving approach set is deleted from the problem information to obtain updated problem information. The updated problem information, the target answer information, and the standard answer information are then combined with the first prompt instruction and input into a preset large language model to obtain the error analysis results of the target answer text output by the large language model.

[0026] This application also provides an error cause analysis and generation apparatus, including:

[0027] The acquisition unit is used to acquire the target answer text to be parsed; and to determine the target question text and standard answer text corresponding to the target answer text;

[0028] The first generation unit is used to perform answer analysis on the target answer text based on the question information and solution approach set of the pre-constructed target question text, and generate target answer information corresponding to the target answer text; the solution approach set is generated in advance iteratively based on the target question text and standard answer text;

[0029] The input unit is used to input the target answer information, standard answer information, and question information, combined with the first prompt instruction, into a preset large language model to obtain the error analysis results of the target answer text output by the large language model; the standard answer information is generated in advance based on the question information and the standard answer text.

[0030] In one possible implementation, the device further includes:

[0031] The second generation unit is used to generate a question analysis and a standard problem-solving approach based on the target question text and the standard answer text, using the first preset prompt instruction and the Large Language Model (LLM).

[0032] The third generation unit is used to iteratively generate a set of candidate problem-solving approaches based on the standard problem-solving approach, using the second preset prompt instructions and the Large Language Model (LLM).

[0033] The constitutive unit is used to sort the different problem-solving approaches in the candidate problem-solving approach set using the large language model LLM, and select K problem-solving approaches that meet preset conditions from the sorting results to form a problem-solving approach set; and to use the problem-solving approach set, the target problem text, and the problem parsing to form the problem information of the target problem text; where K is a positive integer greater than 0;

[0034] The large language model LLM is obtained by training language rules and patterns using a large-scale language dataset through autoregressive generation.

[0035] In one possible implementation, the third generation unit includes:

[0036] The first generation subunit is used to generate candidate problem-solving approaches based on the standard problem-solving approach by utilizing the second preset prompt instructions and the Large Language Model (LLM).

[0037] The calculation subunit is used to calculate the similarity between the candidate problem-solving approach and the standard problem-solving approach, and to determine whether the similarity is lower than a preset threshold. If so, the candidate problem-solving approach and the standard problem-solving approach are combined to form an initial set of candidate problem-solving approaches.

[0038] The update subunit is used to reuse the second preset prompt instruction and the large language model LLM. Based on the standard problem-solving approach, it generates an updated candidate problem-solving approach and iteratively updates the problem-solving approaches in the candidate problem-solving approach set based on the similarity calculation results between the updated candidate problem-solving approach and all problem-solving approaches in the initial candidate problem-solving approach set.

[0039] In one possible implementation, the similarity is text similarity; the preset threshold is 0.4.

[0040] In one possible implementation, the device further includes:

[0041] The fourth generation unit is used to utilize the third preset prompt instruction and the Large Language Model (LLM) to perform answer analysis on the standard answer text based on the question information and standard problem-solving approach of the target question text, generate the standard answer analysis result corresponding to the standard answer text, and use the standard answer analysis result, the standard answer text, and the standard problem-solving approach to construct the standard answer information.

[0042] In one possible implementation, the first generation unit includes:

[0043] The filtering subunit is used to select the most matching problem-solving approach and its sequence number from the set of problem-solving approaches based on the question information of the target question text and the target answer text, using the second prompt instruction and the large language model LLM.

[0044] The second generation subunit is used to perform answer analysis on the target answer text based on the question information of the target question text and the most matching solution approach and its sequence number, and generate the target answer analysis result corresponding to the target answer text; and use the target answer analysis result, the target answer text and the most matching solution approach to construct the target answer information corresponding to the target answer text.

[0045] In one possible implementation, the input unit is specifically used for:

[0046] The problem-solving approach set is deleted from the problem information to obtain updated problem information. The updated problem information, the target answer information, and the standard answer information are then combined with the first prompt instruction and input into a preset large language model to obtain the error analysis results of the target answer text output by the large language model.

[0047] This application embodiment also provides an error cause analysis and generation device, including: a processor, a memory, and a system bus;

[0048] The processor and the memory are connected via the system bus;

[0049] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described error cause analysis generation methods.

[0050] This application also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described error cause analysis generation methods.

[0051] This application also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described error cause analysis and generation methods.

[0052] This application provides an error analysis generation method, apparatus, storage medium, and device. First, it acquires the target answer text to be analyzed; then, it determines the target question text and standard answer text corresponding to the target answer text; next, based on a pre-constructed set of question information and problem-solving approaches for the target question text, it performs answer analysis on the target answer text to generate target answer information corresponding to the target answer text; wherein, the set of problem-solving approaches is pre-iterated iteratively generated based on the target question text and standard answer text; then, it inputs the target answer information, standard answer information, and the question information, combined with a first prompt instruction, into a pre-set large language model to obtain the error analysis results for the target answer text output by the large language model; wherein, the standard answer information is pre-analyzed and generated based on the question information and standard answer text.

[0053] As can be seen, since this application first generates the question information and standard answer information of the target question text offline, and iteratively generates a set of problem-solving ideas, and then generates the target answer text online based on this data, it not only alleviates the problem caused by multiple solutions to a single question and improves the accuracy of the error analysis results, but also greatly improves the efficiency of error analysis results generation through this decoupled online and offline processing method, thereby improving the user's interactive experience. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 A flowchart illustrating a method for generating error cause analysis according to an embodiment of this application;

[0056] Figure 2 This is a schematic diagram illustrating the overall process of error cause analysis generation provided in the embodiments of this application;

[0057] Figure 3 This is a schematic diagram illustrating the process of iteratively generating a set of candidate problem-solving approaches based on standard problem-solving approaches, using a second preset prompt instruction and a large language model (LLM) as provided in the embodiments of this application.

[0058] Figure 4 Example diagrams of student answer texts, standard answers, and questions provided for embodiments of this application;

[0059] Figure 5 This is a schematic diagram of the composition of an error cause analysis and generation device provided in an embodiment of this application. Detailed Implementation

[0060] Currently, the most direct method for generating error analysis based on Large Language Models (LLMs) is to design a prompt, inputting the question, standard answer, and student answer together into the LMM. Then, through the task description, the model generates error analysis in one go. However, this places very high demands on the model's capabilities and often makes it difficult to generate accurate error analysis and correction guidance. Although the model's understanding of the task scenario can be improved by optimizing the prompt or adding examples (In-context Learning, ICL), the improvement in understanding answers to math-related questions and math-related reasoning abilities is not significant.

[0061] Furthermore, commonly used Chain of Thought (COT) or Tree of Thought (TOT) methods have certain requirements regarding the task scenario: 1) the output target can be broken down into steps; 2) the preceding generation steps have a logical recursive relationship with subsequent generation steps. Additionally, common COT and TOT methods have two other problems: 1) Common COT or TOT methods require multiple calls to the large language model in a pipeline manner, which can be very slow when there are many steps; 2) In error analysis tasks, such as those related to mathematics, when student answers are highly divergent and the question may have multiple solutions, simply comparing the answer text with the standard answer is insufficient to determine whether the student's answer is incorrect and what the errors are, thus affecting the generation of subsequent correction guidance.

[0062] It is evident that current error analysis generation methods based on Large Language Model (LLM) mainly suffer from the following three problems:

[0063] (1) Generating error analysis in a single prompt requires extremely high model capabilities and often yields poor results.

[0064] (2) Although the model can generate intermediate reasoning processes in a step-by-step manner, the currently common COT method has problems such as incompatibility with error analysis generation tasks and low reasoning efficiency.

[0065] (3) In tasks such as math-related error analysis, the divergence of students' answers and the existence of multiple solutions to a problem will affect the model's ability to generate the final error analysis.

[0066] To address the aforementioned shortcomings, this application provides a method for generating error analysis results. First, the target answer text to be analyzed is obtained; then, the target question text and standard answer text corresponding to the target answer text are determined; next, based on a pre-constructed set of question information and problem-solving strategies for the target question text, the answer text is analyzed to generate target answer information corresponding to the target answer text; wherein, the set of problem-solving strategies is pre-generated iteratively based on the target question text and standard answer text; then, the target answer information, standard answer information, and the question information, combined with a first prompt instruction, are input into a pre-defined large language model to obtain the error analysis results for the target answer text output by the large language model; wherein, the standard answer information is pre-generated based on the question information and standard answer text.

[0067] As can be seen, since this application first generates the question information and standard answer information of the target question text offline, and iteratively generates a set of problem-solving ideas, and then generates the target answer text online based on this data, it not only alleviates the problem caused by multiple solutions to a single question and improves the accuracy of the error analysis results, but also greatly improves the efficiency of error analysis results generation through this decoupled online and offline processing method, thereby improving the user's interactive experience.

[0068] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0069] First Embodiment

[0070] See Figure 1This is a flowchart illustrating a method for error analysis provided in this embodiment. The method includes the following steps:

[0071] S101: Obtain the target answer text to be parsed; and determine the target question text and standard answer text corresponding to the target answer text.

[0072] In this embodiment, any answer text input by the user in intelligent interactive software or device such as a large language model (LLM) that requires error analysis is defined as the target answer text to be analyzed. Furthermore, this embodiment does not limit the language type of the target answer text; for example, the target answer text can be Chinese text or English text. This embodiment also does not limit the length of the target answer text; that is, the target question text can be sentence text (i.e., a collection of words) or passage text (i.e., a collection of sentences). This embodiment also does not limit the method of obtaining the target answer text; for example, the target answer text can be the answer text obtained by scanning or photographing student exam papers, assignments, etc., and then recognizing them using Optical Character Recognition (OCR).

[0073] Large Language Models (LLMs) are deep learning-based language models that can generate new language expressions, such as text, sentences, paragraphs, and even articles, based on input text content. LLMs utilize large-scale language datasets and are trained on language rules and patterns through autoregressive generation. They can simulate human commands to generate language expressions (such as text data). Specifically, when generating new text data, LLMs predict the probability of the next language unit based on previously generated content until complete text data is generated.

[0074] Furthermore, after obtaining the target question text input by the user, the system can determine the target question title text and standard answer text corresponding to the target answer text based on the text content of the target question text. It can also determine the standard answer information and the question information of the target question title text, which are pre-generated offline based on the target question title text and standard answer text. Figure 2 As shown, this is used to execute subsequent steps S102-S103.

[0075] Specifically, one possible implementation is that the generation of the question information for the target problem text can be as follows: First, using a first preset prompt instruction and a Large Language Model (LLM), a question analysis and a standard solution approach are generated based on the target problem text and the standard answer text. Then, using a second preset prompt instruction and the LLM, a candidate solution approach set is iteratively generated based on the standard solution approach. Next, the LLM is used to sort the different solution approaches in the candidate solution approach set, and K solution approaches that meet preset conditions (the specific content is not limited) are selected from the sorting results to form a solution approach set. The question information for the target problem text is then constructed using the solution approach set, the target problem text, and the question analysis; where K is a positive integer greater than 0.

[0076] In this implementation, it's important to note that understanding the question is crucial for both problem generation and answer grading, as it's key to determining whether the answer meets the requirements. Therefore, to enhance the understanding and effectiveness of the Large Language Model (LLM) in handling multiple-solution scenarios and improve the efficiency and accuracy of error analysis, this application pre-generates a set of solution approaches for the target problem. Unlike current methods that directly generate solutions based on the target problem, this application uses an iterative rewriting approach to generate a set of candidate solutions. Then, the Large Language Model (LLM) selects the Top K solutions to construct the final set of solutions. Figure 3 As shown.

[0077] In practical applications, when generating question analysis and standard problem-solving approaches using the first preset prompt instruction and the Large Language Model (LLM) based on the target question text and the standard answer text, the target question text is represented as P. The standard answer text is represented as S, where S = {s1, s2, ..., s...} n ), s i Let represent the i-th response step in the standard response text, and let n represent the total number of response steps. Let the target response text be represented as U, where U = {u1, u2, ..., u...} m ), u i Let m represent the i-th answer step in the target answer text, and let m represent the total number of answer steps. The formulas for generating the question analysis and standard problem-solving approach are as follows:

[0078] (P a O s = LLM(PromptPS(P,S))

[0079] Among them, PromptPS() represents the first preset prompt instruction used by the Large Language Model (LLM) built based on the target question text and the standard answer text to generate question parsing and standard problem-solving ideas;

[0080] P a O s These represent the problem analysis and the standard solution approach, respectively. The key to constructing PromptPS() is to first output the problem analysis P based on the target problem text. a This includes, but is not limited to, the requirements of the target question and the content it tests, and then, based on the target question and the standard answer text, as well as the generated question analysis P... a Output the standard solution approach corresponding to the standard answer text.

[0081] For example, here is an example of PromptPS():

[0082] "You are now a senior, meticulous, and kind elementary school math teacher. Next, based on the given [Problem] and [Standard Answer], please first understand the problem and output the corresponding [Problem Analysis], including your understanding of the problem, its requirements, and the content and skills it tests. Then, based on the known information, output the [Solution Strategy] corresponding to the [Standard Answer] in points. The specific output format is: [Problem Analysis] XXX [Solution Strategy] 1. XXX; 2. XXX...\nGiven:\n[Problem] {Placeholder}\n[Standard Answer] {Placeholder}\nOutput:"

[0083] Then, based on the target question, standard answer text, generated question analysis, and standard solution approach, the system utilizes a second preset prompt and a large language model (LLM) to understand the target question and standard solution approach. It rewrites the standard answer approach to generate candidate solutions and calculates the similarity between the candidate solutions and the standard solution approach (the specific content is not limited, such as text similarity). It then determines whether this similarity is below a preset threshold (the specific value is not limited, such as 0.4). If so, the candidate solutions and the standard solution approach form an initial candidate solution approach set. The second preset prompt and the large language model (LLM) are then used repeatedly to generate updated candidate solutions based on the standard solution approach. The similarity between the updated candidate solutions and all solutions in the initial candidate solution approach set is used to iteratively update the solution approaches in the candidate solution approach set, thus iteratively generating a candidate solution approach set. The specific iterative generation process is as follows:

[0084] (1) Define the candidate solution set as: O T ={O s}, P aThis still indicates the explanation of the question.

[0085] (2) Based on the problem text, problem analysis, and standard problem-solving approaches, the following formulas are generated by rewriting the second preset hints and the Large Language Model (LLM) to produce different problem-solving approaches:

[0086] O i =LLM(PromptO(P, P) a S, O s ))

[0087] Where PromptO represents the second preset hint instruction for constructing a rewritten solution approach; O i This represents the different problem-solving approaches generated.

[0088] For example, an example of PromptO is as follows:

[0089] "You are now a seasoned and meticulous elementary school math teacher. Next, based on the given [Problem], [Problem Analysis], [Standard Answer], and [Solution Approach], please first understand the requirements of the problem and the corresponding solution approach, and then provide another solution approach that will also yield the correct answer. The specific output format should be: [New Solution Approach] 1.XXX; 2.XXX;..."

[0090] (3) Determine the different candidate solutions generated. i With the set of candidate solutions O T The similarity of existing problem-solving approaches is used to determine the similarity of approaches. If the similarity is lower than a threshold (e.g., 0.4), the approach is added to the set, and step (2) is repeated. If the similarity is higher than the threshold, the iteration ends, and a candidate problem-solving approach set O is obtained. T .

[0091] Next, the Large Language Model (LLM) is used to analyze the candidate problem-solving approach set O. T Different problem-solving approaches are scored, sorted, and numbered. The top K solutions with the highest scores are then selected to form the final set of problem-solving approaches O.

[0092] In this way, by rewriting and expanding different problem-solving approaches based on standard problem-solving approaches, a combination of problem-solving approaches can be constructed. On the one hand, this reduces the difficulty of model generation and improves the generation effect. On the other hand, it can also expand the diversity of problem-solving approaches and provide basic information reference for solving the current problem of "multiple solutions to one problem".

[0093] Furthermore, we can utilize the problem-solving approach set O, the target problem text P, and the problem analysis P. a The question information P that constitutes the target question text I That is, P I =(P, P) a(O), and this information can be generated offline, such as Figure 2 As shown.

[0094] Another optional implementation is that the standard answer information can be generated by using a third preset prompt instruction and a large language model (LLM) to analyze the standard answer text based on the question information and standard problem-solving approach in the target question text, generating a standard answer analysis result corresponding to the standard answer text. The standard answer information is then constructed using the standard answer analysis result, the standard answer text, and the standard problem-solving approach.

[0095] In this implementation, it's important to note that the generation of error analysis (including but not limited to error location, error type, and correction guidance) involves determining the correctness of the target answer, and the information from the standard answer is crucial in specific error analysis. Although the problem-solving approach set generated through the above implementation includes a summary and overview of the overall approach in the standard answer text, the presence of numerous mathematical symbols and formulas in the standard answer text makes it difficult for the model to achieve a detailed understanding of the answering process. Therefore, a third preset prompt instruction can be used to allow the Large Language Model (LLM) to analyze each step in the standard answer text based on the question information, describing in text form what each step in the standard answer text does. This helps the model better understand the details of the answering steps in the standard answer text. The specific formula is as follows:

[0096] S a =LLM(PromptSA(P I ,S))

[0097] Among them, P I The question information that represents the target question text; S a This indicates the results of the standard answer analysis; PromptSA indicates a third preset prompt instruction built based on the question information and the standard answer text, used to analyze the steps of answering the standard answer text.

[0098] For example, the following is an example of PromptSA:

[0099] "You are now a seasoned and meticulous elementary school math teacher. Next, based on the given [Problem], [Problem Analysis], [Solution Approach], and [Standard Answer], please first understand the requirements of the problem and the corresponding solution approach. Then, referring to the specific solution approach, concisely explain what each step in the [Standard Answer] did. Output the results in JSON format: [Standard Answer Analysis] {1: XXX, 2: xxx}"

[0100] The key points in constructing PromptSA include: 1) having the model provide the sequence number of the solution approach that best matches the standard answer text, allowing the model to focus on a specific solution approach rather than the entire set, since the standard solution approach O corresponding to the standard answer text has already been generated. s Therefore, O is directly given in PromptSA. s 1) The corresponding sequence number in the set is sufficient; 2) The input of the standard answer text is a list of answer steps, allowing the model to provide the corresponding answer analysis for each step; 3) When providing the standard answer analysis results, it is necessary to combine the existing problem-solving ideas, and the answer step analysis should not be separated from the overall problem-solving ideas.

[0101] Furthermore, the results of the standard response analysis can be used to analyze S. a Standard problem-solving approach O s Together with the standard response text S, they constitute the standard response information S. I S I =(S, S) a O s Furthermore, this information can be generated offline, such as... Figure 2 As shown.

[0102] S102: Based on the pre-constructed set of question information and solution ideas of the target question text, perform answer analysis on the target answer text and generate the target answer information corresponding to the target answer text; wherein, the set of solution ideas is generated in advance iteratively based on the target question text and the standard answer text.

[0103] In this embodiment, after obtaining the target answer text and determining the target question text and standard answer text corresponding to the target answer text through step S101, the target answer text can be further analyzed based on the question information and problem-solving ideas set of the pre-constructed target question text to generate target answer information corresponding to the target answer text, which is then used to execute the subsequent step S103.

[0104] Specifically, one possible implementation is to first utilize the second prompt instruction (prompt) and the Large Language Model (LLM) to select the most matching solution and its index from the set of solution approaches based on the question information in the target question text and the target answer text. Then, based on the question information in the target question text and the selected most matching solution and its index, the answer text is analyzed to generate a target answer analysis result. Finally, the target answer analysis result, the target answer text, and the most matching solution approach are used to construct the target answer information corresponding to the target answer text.

[0105] In this implementation, it should be noted that in practical applications, the target response (such as student responses) is highly divergent, mainly reflected in the following three points:

[0106] 1) The problem-solving approach in the target response text (such as student response text) may be inconsistent with that in the standard response text.

[0107] 2) In the process of solving a partial problem, it is uncertain whether to use separate formulas or comprehensive formulas. For example, if the standard answer text is: A=(B+C)*D, the target answer text (such as the student's answer) may be: T=B+C, A=T*D.

[0108] 3) The order of the problem-solving process in the target answer (e.g., student answer) text may be inconsistent with that in the standard answer text. This may be because there are two or more parallel and independent intermediate results that need to be calculated.

[0109] Therefore, in the process of generating error analysis (including but not limited to error location, error type, and correction guidelines), the Large Language Model (LLM) needs to have the following three capabilities:

[0110] 1) Be able to understand the problem-solving approach in the target response text (such as student responses), even if it is inconsistent with the standard response text.

[0111] 2) Be able to understand the meaning of each step in the target response (e.g., student response) text, even if it may be inconsistent with the standard response text.

[0112] 3) Be able to understand certain equivalent problem-solving steps in the target response text (such as student response text) and the standard response text.

[0113] For these three issues, the previously generated question analysis, problem-solving strategy sets, and standard answer analysis results can provide the Large Language Model (LLM) with very detailed question and standard answer information. However, since the target answer (e.g., student's answer) text, like the standard answer text, may also contain many mathematical symbols and formulas, it is difficult for the model to understand the detailed answer process of the target answer (e.g., student's answer). Therefore, the second prompt instruction can be used to allow the Large Language Model (LLM) to base its understanding on the question information P of the target question text. I Given the target response text U, select the solution approach O from the set of solution approaches O that best matches the target response text U. U The sequence number allows the model to focus on a single problem-solving approach and enables the idol to analyze each step in the target response text, describing in text form what each step in the target response text did. This helps the model better understand the details of the response steps in the target response text. The specific formula is as follows:

[0114]

[0115] in, O represents the solution approach that best matches the target response text U. U The serial number; U a This indicates the target response analysis result corresponding to the target response text; PromptUA indicates the response based on question information P. I The second prompt instruction, constructed from the target response text U, is used to analyze the target response text U.

[0116] For example, here is an example of PromptUA:

[0117] "You are now a seasoned and meticulous elementary school math teacher. Based on the given [Problem], [Problem Analysis], [Solution Approach], and [Student Response], please first understand the requirements of the problem and the student's response. First, provide the [Solution Approach Number] that most closely matches the student's response. Then, referring to the specific solution approach, concisely describe what each step in the [Student Response] did. The output format should be: [Solution Approach Number] XXX [Student Response Analysis] {1: XXX, 2: xxx}"

[0118] The key points of constructing PromptUA include: 1) First, select the solution approach from the set of solutions that best matches the text of the current target problem; 2) When providing the answer analysis, it is necessary to combine the existing solution approach, and the analysis of the answer steps should not be separated from the overall solution approach. It can be seen that the key points of constructing PromptUA are similar to those of PromptSA, the difference being that the sequence number of the solution approach is known in PromptSA.

[0119] Furthermore, the results of the target response analysis can be used. a The target response text U and the best matching problem-solving approach O. U The target response information U that constitutes the target response text I , that is U I =(U, U a O U This information can be generated online, such as... Figure 2 As shown.

[0120] S103: Input the target answer information, standard answer information, and question information, along with the first prompt instruction, into the preset large language model to obtain the error analysis results of the target answer text output by the large language model; wherein, the standard answer information is generated based on the question information and the standard answer text in advance.

[0121] In this embodiment, the target response information U corresponding to the target response text is generated through step S102. I Then, the target response information U can be further... I Standard answer information S I And the question information P I The first prompt instruction is incorporated and input into a pre-defined large language model (LLM). The LLM then outputs error analysis results for the target response text, including but not limited to the error location, error type, and corresponding correction guidance in the target response text.

[0122] Specifically, one possible implementation is that, due to the standard response information S I and target response information U I The solution already includes the corresponding problem-solving approach. s and O U Therefore, we can start with the information P in the question. I Delete the set of solutions O to obtain the updated problem information. Right now, Then update the question information Target response information U I and standard response information S I Combined with the first prompt instruction, the data is input into a pre-defined large language model to obtain the error analysis result R of the target response text, as shown in the following formula:

[0123]

[0124] Where R represents the error analysis result of the target answer text, including but not limited to whether there are errors in the target answer text, the location of the errors, the type of errors, and corresponding analysis and correction guidance;

[0125] PromptR represents the first prompt instruction for generating error message parsing, constructed based on updated question information, standard answer information, and target answer information.

[0126] For example, here is an example of PromptR:

[0127] "You are now a seasoned and meticulous elementary school math teacher. Based on the given question information, standard answer information, and student responses, please use simple and easy-to-understand language to identify errors in the students' answers and provide specific guidance for correction. The question information includes: [Question] and [Question Analysis], used to understand the question's requirements and the content being tested; the standard answer information includes: [Standard Answer], [Standard Answer Solution Approach], and [Standard Answer Analysis], used to understand the solution approach corresponding to the given standard answer and the content of each step in the answer; the student response information includes: [Student Response], [Student Response Solution Approach], and [Student Response Analysis], used to understand the specific student's solution approach and the meaning of each step. Based on the question information, carefully compare the differences between the student's response and the standard answer, identify the errors in the student's response, and provide specific guidance for correction. The output format is: [Error Location] (X, X) [Error Type] XXX [Correction Guidance] XXX"

[0128] The key points for building PromptR include: 1) clearly highlighting S I U I 1) The information in each field enables the large language model LLM to clearly understand the meaning of each field; 2) The large language model LLM first determines whether there are errors in the target answer text. If there are errors, it continues to output the error location, error type, and corresponding analysis and correction guidance. It is necessary to constrain the model to only point out the first error for repeated types of errors. For example, in a continuous answer step or equation, if the calculation of the first step is wrong, the calculation of the subsequent steps will be wrong, and it is not necessary to point it out again; 3) It is necessary to instruct the large language model LLM not to give the correct answer directly when giving correction guidance, so as to make it more conducive to users to think independently to obtain the correct answer.

[0129] In this way, by scanning or photographing student exam papers, assignments, etc., and using OCR to obtain the student's answer text, the above steps S101-103 can be executed to automatically correct the student's answer in conjunction with the standard answer, while pointing out the errors in the student's answer and providing error analysis and correction guidance to guide the student in answering.

[0130] For example: Suppose the student's answer text, the standard answer, and the question are as follows: Figure 4As shown, the prompt instruction for the input large language model LLM can be: "Now you are a senior, rigorous, and kind elementary school math teacher. Next, please find the errors in the students' answers and provide specific correction guidance in simple and easy-to-understand language based on the given [question], [standard answer], and [student answer]. The specific output format is: [error location] (X,X)\n [error type] XXX\n [correction guidance] XXX\n [question] Now you are a senior, rigorous, and kind elementary school math teacher. Next, please find the errors in the students' answers and provide specific correction guidance in simple and easy-to-understand language based on the given [question], [standard answer], and [student answer]. The specific output format is: [error location] (X,X)[error type] XXX[correction guidance] XXX[standard answer] ['Solution:(1)(216-96)\\\\div5',

[0131] '=120\\\\div5','=24(pages)','Answer: Read an average of 24 pages per day.','(2)96\\\\div(8-5)','=96\\\\div3',

[0132] =32 (pages)','Answer: From the sixth day onwards, an average of 32 pages need to be read each day.'] [Student's answer] ['(1)216-96=120 (pages)','120\\\\div5=24 (pages)','Answer: An average of 24 pages need to be read each day','8-5=3 (days)','216\\\\div3=72 (pages)','Answer: An average of 72 pages need to be read each day.']". The Large Language Model (LLM) outputs the following error analysis: "[Error Location] (4,5) [Error Type] Incorrect thinking [Correction Guidance] Please review the question carefully again. The solution to question (2) should be to calculate the average number of pages read each day for the remaining 96 pages to be read within 3 days, instead of dividing the 216 pages into 3 days for reading."

[0133] In summary, the error analysis generation method provided in this embodiment first obtains the target answer text to be analyzed; and determines the target question text and standard answer text corresponding to the target answer text; then, based on the pre-constructed question information and problem-solving approach set of the target question text, the answer text is analyzed to generate target answer information corresponding to the target answer text; wherein, the problem-solving approach set is pre-iterated based on the target question text and standard answer text; next, the target answer information, standard answer information, and the question information, combined with the first prompt instruction, are input into a preset large language model to obtain the error analysis result of the large language model for the target answer text; wherein, the standard answer information is pre-analyzed based on the question information and standard answer text.

[0134] As can be seen, since this application first generates the question information and standard answer information of the target question text offline, and iteratively generates a set of problem-solving ideas, and then generates the target answer text online based on this data, it not only alleviates the problem caused by multiple solutions to a single question and improves the accuracy of the error analysis results, but also greatly improves the efficiency of error analysis results generation through this decoupled online and offline processing method, thereby improving the user's interactive experience.

[0135] Second Embodiment

[0136] This embodiment will introduce an error cause analysis and generation device; please refer to the above method embodiment for related content.

[0137] See Figure 5 This is a schematic diagram of the composition of an error cause analysis and generation device provided in this embodiment. The device 500 includes:

[0138] The acquisition unit 501 is used to acquire the target answer text to be parsed; and to determine the target question text and standard answer text corresponding to the target answer text;

[0139] The first generation unit 502 is used to perform answer analysis on the target answer text based on the question information and solution approach set of the pre-constructed target question text, and generate target answer information corresponding to the target answer text; the solution approach set is pre-iterated based on the target question text and standard answer text.

[0140] The input unit 503 is used to input the target answer information, standard answer information, and question information, combined with the first prompt instruction, into a preset large language model to obtain the error analysis result of the large language model for the target answer text; the standard answer information is generated in advance based on the question information and the standard answer text.

[0141] In one implementation of this embodiment, the apparatus further includes:

[0142] The second generation unit is used to generate a question analysis and a standard problem-solving approach based on the target question text and the standard answer text, using the first preset prompt instruction and the Large Language Model (LLM).

[0143] The third generation unit is used to iteratively generate a set of candidate problem-solving approaches based on the standard problem-solving approach, using the second preset prompt instructions and the Large Language Model (LLM).

[0144] The constitutive unit is used to sort the different problem-solving approaches in the candidate problem-solving approach set using the large language model LLM, and select K problem-solving approaches that meet preset conditions from the sorting results to form a problem-solving approach set; and to use the problem-solving approach set, the target problem text, and the problem parsing to form the problem information of the target problem text; where K is a positive integer greater than 0;

[0145] The large language model LLM is obtained by training language rules and patterns using a large-scale language dataset through autoregressive generation.

[0146] In one implementation of this embodiment, the third generation unit includes:

[0147] The first generation subunit is used to generate candidate problem-solving approaches based on the standard problem-solving approach by utilizing the second preset prompt instructions and the Large Language Model (LLM).

[0148] The calculation subunit is used to calculate the similarity between the candidate problem-solving approach and the standard problem-solving approach, and to determine whether the similarity is lower than a preset threshold. If so, the candidate problem-solving approach and the standard problem-solving approach are combined to form an initial set of candidate problem-solving approaches.

[0149] The update subunit is used to reuse the second preset prompt instruction and the large language model LLM. Based on the standard problem-solving approach, it generates an updated candidate problem-solving approach and iteratively updates the problem-solving approaches in the candidate problem-solving approach set based on the similarity calculation results between the updated candidate problem-solving approach and all problem-solving approaches in the initial candidate problem-solving approach set.

[0150] In one implementation of this embodiment, the similarity is text similarity; the preset threshold is 0.4.

[0151] In one implementation of this embodiment, the apparatus further includes:

[0152] The fourth generation unit is used to utilize the third preset prompt instruction and the Large Language Model (LLM) to perform answer analysis on the standard answer text based on the question information and standard problem-solving approach of the target question text, generate the standard answer analysis result corresponding to the standard answer text, and use the standard answer analysis result, the standard answer text, and the standard problem-solving approach to construct the standard answer information.

[0153] In one implementation of this embodiment, the first generation unit 502 includes:

[0154] The filtering subunit is used to select the most matching problem-solving approach and its sequence number from the set of problem-solving approaches based on the question information of the target question text and the target answer text, using the second prompt instruction and the large language model LLM.

[0155] The second generation subunit is used to perform answer analysis on the target answer text based on the question information of the target question text and the most matching solution approach and its sequence number, and generate the target answer analysis result corresponding to the target answer text; and use the target answer analysis result, the target answer text and the most matching solution approach to construct the target answer information corresponding to the target answer text.

[0156] In one implementation of this embodiment, the input unit 503 is specifically used for:

[0157] The problem-solving approach set is deleted from the problem information to obtain updated problem information. The updated problem information, the target answer information, and the standard answer information are then combined with the first prompt instruction and input into a preset large language model to obtain the error analysis results of the target answer text output by the large language model.

[0158] Furthermore, embodiments of this application also provide an error cause analysis and generation device, including: a processor, a memory, and a system bus;

[0159] The processor and the memory are connected via the system bus;

[0160] The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described implementations of the error cause analysis generation method.

[0161] Furthermore, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described implementations of the error cause analysis generation method.

[0162] Furthermore, this application embodiment also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described error cause analysis generation methods.

[0163] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0164] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0165] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating error cause analysis, characterized in that, include: Obtain the target response text to be parsed; And determine the target question text and standard answer text corresponding to the target answer text; Based on the pre-constructed set of question information and problem-solving ideas of the target question text, the answer text of the target answer is analyzed to generate the target answer information corresponding to the target answer text; The set of problem-solving approaches is generated iteratively in advance based on the target problem text and the standard answer text. The method for generating the problem-solving approach set includes: generating a problem analysis and a standard problem-solving approach based on the target problem text and the standard answer text; generating candidate problem-solving approaches by rewriting the standard problem-solving approach, and then determining the similarity between the candidate problem-solving approach and the standard problem-solving approach; filtering the candidate problem-solving approaches based on the similarity, iteratively executing the above steps, and forming a problem-solving approach set by K problem-solving approaches that meet preset conditions; where K is a positive integer greater than 0. The target answer information, standard answer information, and question information, combined with the first prompt instruction, are input into a preset large language model to obtain the error analysis results of the target answer text output by the large language model; the standard answer information is generated in advance based on the question information and the standard answer text.

2. The method according to claim 1, characterized in that, The question information of the target question text is generated in the following way: Using the first preset prompts and the large language model, the question analysis and standard problem-solving approach are generated based on the target question text and the standard answer text. Using the second preset prompts and the large language model, a set of candidate problem-solving approaches is iteratively generated based on the standard problem-solving approach. The large language model is used to sort the different problem-solving approaches in the candidate problem-solving approach set, and K problem-solving approaches that meet the preset conditions are selected from the sorting results to form a problem-solving approach set; and the problem-solving approach set, the target problem text, and the problem analysis are used to form the problem information of the target problem text. The large language model is obtained by training language rules and patterns using a large-scale language dataset through autoregressive generation.

3. The method according to claim 2, characterized in that, The step of using the second preset prompt instruction and the large language model to iteratively generate a set of candidate problem-solving approaches based on the standard problem-solving approach includes: Using the second preset prompts and the large language model, candidate problem-solving approaches are generated based on the standard problem-solving approach. Calculate the similarity between the candidate solution approach and the standard solution approach, and determine whether the similarity is lower than a preset threshold. If so, the candidate solution approach and the standard solution approach constitute an initial set of candidate solution approaches. The second preset prompt instruction and the large language model are reused to generate updated candidate problem-solving approaches based on the standard problem-solving approach. The problem-solving approaches in the candidate problem-solving approach set are then iteratively updated based on the similarity calculation results between the updated candidate problem-solving approaches and all problem-solving approaches in the initial candidate problem-solving approach set.

4. The method according to claim 3, characterized in that, The similarity is text similarity; the preset threshold is 0.

4.

5. The method according to claim 2, characterized in that, The standard response information is generated in the following way: Using the third preset prompt instruction and the large language model, the standard answer text is analyzed based on the question information and standard problem-solving approach of the target question text, generating the standard answer analysis result corresponding to the standard answer text; and the standard answer information is constructed using the standard answer analysis result, the standard answer text, and the standard problem-solving approach.

6. The method according to claim 1, characterized in that, The method involves analyzing the target answer text based on a pre-constructed set of question information and solution strategies, generating target answer information corresponding to the target answer text, including: Using the second prompt instruction and the large language model, based on the question information of the target question text and the target answer text, the most matching problem-solving approach and its sequence number are selected from the set of problem-solving approaches; Based on the question information of the target question text and the most matching solution approach and its sequence number, the target answer text is analyzed to generate the target answer analysis result corresponding to the target answer text; and the target answer information corresponding to the target answer text is constructed by using the target answer analysis result, the target answer text and the most matching solution approach.

7. The method according to any one of claims 1-6, characterized in that, The step of inputting the target answer information, standard answer information, and question information, combined with the first prompt instruction, into a preset large language model to obtain the error analysis results of the large language model for the target answer text, includes: The problem-solving approach set is deleted from the problem information to obtain updated problem information. The updated problem information, the target answer information, and the standard answer information are then combined with the first prompt instruction and input into a preset large language model to obtain the error analysis results of the target answer text output by the large language model.

8. An error cause analysis and generation device, characterized in that, include: The acquisition unit is used to acquire the target response text to be parsed. And determine the target question text and standard answer text corresponding to the target answer text; The first generation unit is used to perform answer analysis on the target answer text based on the question information and problem-solving ideas set of the pre-constructed target question text, and generate target answer information corresponding to the target answer text; The set of problem-solving approaches is generated iteratively in advance based on the target problem text and the standard answer text. The method for generating the problem-solving approach set includes: generating a problem analysis and a standard problem-solving approach based on the target problem text and the standard answer text; generating candidate problem-solving approaches by rewriting the standard problem-solving approach, and then determining the similarity between the candidate problem-solving approach and the standard problem-solving approach; filtering the candidate problem-solving approaches based on the similarity, iteratively executing the above steps, and forming a problem-solving approach set by K problem-solving approaches that meet preset conditions; where K is a positive integer greater than 0. The input unit is used to input the target answer information, standard answer information, and question information, combined with the first prompt instruction, into a preset large language model to obtain the error analysis result of the large language model for the target answer text; the standard answer information is generated in advance based on the question information and the standard answer text.

9. A fault cause analysis and generation device, characterized in that, include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method described in any one of claims 1-7.

Images