Diagnosis method and device for problem solving process, electronic equipment and storage medium

Abstract

The invention discloses a diagnosis method and device for a problem solving process, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence or other related technical fields, and the diagnosis method for the problem solving process comprises the steps: receiving a problem solving image, carrying out the preprocessing of the image, and inputting the image into a target recognition model, dividing the problem solving image into N problem solving regions through a target recognition model, and configuring region labels; performing spatial logic analysis on each problem solving area to obtain a spatial logic analysis result; calling a content recognition engine corresponding to the problem solving area to perform content recognition to obtain a content recognition result; and constructing a problem solving process chain based on the spatial logic analysis result and the content identification result, inputting the problem solving process chain into a diagnosis model, and outputting a diagnosis result. According to the method and the device, the technical problems that the diagnosis on the answer input by the user is emphasized, the effective diagnosis on the problem solving process is lacked and the error reason cannot be accurately positioned in the related technology are solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology or other related technical fields. Specifically, it relates to a diagnostic method, apparatus, electronic device, and storage medium for problem-solving processes. Background Technology

[0002] With advancements in technology and shifts in educational philosophies, the application of artificial intelligence (AI) in education is receiving increasing attention. AI has demonstrated tremendous potential in assisting user learning and improving teaching quality. However, traditional AI tutoring systems have significant shortcomings and limitations, particularly in handling complex handwritten problem-solving processes and providing deeply personalized tutoring.

[0003] In related technologies, most online education tutoring tools and platforms focus on providing answer analysis for standardized questions, requiring users to input structured text, such as printed mathematical formulas or clearly stated learning problems, and emphasizing the identification of answers and outputting diagnostic results to users. However, the answer-based diagnostic method ignores the problem-solving process, making it difficult to effectively diagnose the user's problem-solving process. The output results are limited and cannot accurately locate the cause of the error.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a diagnostic method, apparatus, electronic device, and storage medium for the problem-solving process, to at least address the technical problem in related technologies that focus on diagnosing user-inputted answers but lack effective diagnosis of the problem-solving process and cannot accurately pinpoint the cause of errors.

[0006] According to one aspect of the present invention, a method for diagnosing a problem-solving process is provided, comprising: receiving a problem-solving image corresponding to the problem-solving process sent by a user terminal, and preprocessing the problem-solving image; inputting the preprocessed problem-solving image into a target recognition model, dividing the problem-solving image into N problem-solving regions through the target recognition model, and configuring a region label for each problem-solving region, wherein the target recognition model is a pre-built machine learning model for image recognition, and N is a positive integer; performing spatial logic parsing on each problem-solving region to obtain a spatial logic parsing result; calling a content recognition engine corresponding to the problem-solving region based on the region label, performing content recognition on the problem-solving region through the content recognition engine to obtain a content recognition result; constructing a problem-solving process chain based on the spatial logic parsing result and the content recognition result, inputting the problem-solving process chain into a diagnostic model, outputting a diagnostic result, generating a diagnostic report based on the diagnostic result, and returning the diagnostic report to the user terminal, wherein the diagnostic model is a machine learning model built based on a symbolic computation engine and a large language model.

[0007] Further, the step of inputting the preprocessed solution image into the target recognition model, dividing the solution image into N solution regions by the target recognition model, and configuring a region label for each solution region includes: inputting the preprocessed solution image into the target recognition model, extracting image features by the target recognition model, and identifying different elements in the solution image based on the image features; generating bounding boxes for the same elements within a preset region range to obtain N solution regions; identifying the element category within each solution region, and configuring a region label for each solution region based on the element category, wherein the region label includes at least one of the following: formula label, text label, image label, table label, and modification trace label.

[0008] Further, the step of performing spatial logic parsing for each of the problem-solving regions to obtain the spatial logic parsing result includes: calling a symbol detection model to identify logical connection symbols between and within each of the problem-solving regions, and recording the position information of the logical connection symbols, wherein the symbol detection model is a pre-built machine learning model for detecting logical connection symbols; identifying the logical relationship between elements based on the logical connection symbols and the position information; constructing a directed graph based on the logical relationship, and sequentially reconstructing each of the problem-solving regions and the interior of each problem-solving region based on the directed graph and a preset priority rule, and obtaining the spatial logic parsing result based on the reconstructed problem-solving regions.

[0009] Furthermore, the step of performing content recognition on the problem-solving area using the content recognition engine to obtain the content recognition result includes: identifying content features from the problem-solving area using the content recognition engine; and, based on the content features, calling the knowledge base within the content recognition engine to perform content matching to obtain the content recognition result.

[0010] Furthermore, the step of constructing a problem-solving process chain based on the spatial logic parsing result and the content recognition result includes: associating the spatial logic parsing result and the content recognition result to obtain an association result; sorting the problem-solving content according to the logical order recorded in the spatial logic parsing result based on the association result, and constructing the problem-solving process chain based on the sorting result.

[0011] Further, the step of inputting the problem-solving process chain into the diagnostic model and outputting diagnostic results includes: inputting the problem-solving process chain into the diagnostic model, identifying computational errors and derivation logic errors in the problem-solving process chain through the symbolic computation engine in the diagnostic model to obtain a first diagnostic result; identifying conceptual errors, strategic errors, and content omissions in the problem-solving process chain through the large language model in the diagnostic model to obtain a second diagnostic result; associating the first diagnostic result and the second diagnostic result, and calling a pre-built knowledge graph to determine the root cause error based on the association result; and constructing the diagnostic result based on the association result and the root cause error.

[0012] Furthermore, the step of generating a diagnostic report based on the diagnostic results includes: annotating the erroneous content in the solution image based on the diagnostic results to obtain an annotated solution image; generating guidance information based on the diagnostic results and searching for knowledge points based on the diagnostic results; and generating the diagnostic report based on the annotated solution image, the guidance information, and the knowledge points.

[0013] According to another aspect of the present invention, a diagnostic device for a problem-solving process is also provided, comprising: a receiving unit, configured to receive a problem-solving image corresponding to the problem-solving process sent by a user terminal, and preprocess the problem-solving image; a partitioning unit, configured to input the preprocessed problem-solving image into a target recognition model, divide the problem-solving image into N problem-solving regions through the target recognition model, and configure a region label for each problem-solving region, wherein the target recognition model is a pre-built machine learning model for image recognition, and N is a positive integer; a parsing unit, configured to perform spatial logic parsing for each problem-solving region to obtain a spatial logic parsing result; a recognition unit, configured to call a content recognition engine corresponding to the problem-solving region based on the region label, perform content recognition on the problem-solving region through the content recognition engine, and obtain a content recognition result; and a diagnostic unit, configured to construct a problem-solving process chain based on the spatial logic parsing result and the content recognition result, input the problem-solving process chain into a diagnostic model, output a diagnostic result, generate a diagnostic report based on the diagnostic result, and return the diagnostic report to the user terminal, wherein the diagnostic model is a machine learning model constructed based on a symbolic computation engine and a large language model.

[0014] Further, the partitioning unit includes: a first extraction module, used to input the preprocessed solution image into the target recognition model, extract image features through the target recognition model, and identify different elements in the solution image based on the image features; a first generation module, used to generate bounding boxes for the same elements within a preset area, to obtain N solution regions; and a first configuration module, used to identify the element category within each solution region, and configure the region label for each solution region based on the element category, wherein the region label includes at least one of the following: formula label, text label, image label, table label, and modification trace label.

[0015] Further, the parsing unit includes: a first identification module, used to call a symbol detection model to identify logical connection symbols between and within each of the problem-solving regions, and record the position information of the logical connection symbols, wherein the symbol detection model is a pre-built machine learning model for detecting logical connection symbols; a second identification module, used to identify the logical relationship between elements based on the logical connection symbols and the position information; and a first parsing module, used to construct a directed graph based on the logical relationship, and to sequentially reconstruct each of the problem-solving regions and the interior of each problem-solving region based on the directed graph and a preset priority rule, and to obtain the spatial logic parsing result based on the reconstructed problem-solving regions.

[0016] Furthermore, the recognition unit includes: a third recognition module, used to recognize content features from the problem-solving area through the content recognition engine; and a first matching module, used to perform content matching by calling the knowledge base inside the content recognition engine based on the content features, and obtain the content recognition result.

[0017] Furthermore, the diagnostic unit includes: a first association module, used to associate the spatial logic parsing result and the content recognition result to obtain an association result; and a first construction module, used to sort the problem-solving content according to the logical order recorded in the spatial logic parsing result based on the association result, and construct the problem-solving process chain based on the sorting result.

[0018] Further, the diagnostic unit includes: a fourth identification module, used to input the problem-solving process chain into the diagnostic model, and identify calculation errors and derivation logic errors in the problem-solving process chain through the symbolic computation engine in the diagnostic model to obtain a first diagnostic result; a fifth identification module, used to identify conceptual errors, strategic errors, and content omissions in the problem-solving process chain through the large language model in the diagnostic model to obtain a second diagnostic result; a first determination module, used to associate the first diagnostic result and the second diagnostic result, and call a pre-built knowledge graph to determine the root cause error based on the association result; and a second construction module, used to construct the diagnostic result based on the association result and the root cause error.

[0019] Furthermore, the diagnostic unit also includes: a first annotation module, used to annotate the erroneous content in the solution image based on the diagnostic results, to obtain an annotated solution image; a second generation module, used to generate guidance information based on the diagnostic results, and to find knowledge points based on the diagnostic results; and a third generation module, used to generate the diagnostic report based on the annotated solution image, the guidance information, and the knowledge points.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform any of the above-described problem-solving processes for diagnostic purposes.

[0021] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the diagnostic method of any of the above-described problem-solving processes.

[0022] In this application, the following steps are performed: receiving a problem-solving image corresponding to the problem-solving process sent by the user terminal, preprocessing the problem-solving image, inputting the preprocessed problem-solving image into a target recognition model, dividing the problem-solving image into N problem-solving regions through the target recognition model, and configuring a region label for each problem-solving region. Here, the target recognition model is a pre-built machine learning model for image recognition, where N is a positive integer. Then, spatial logic parsing is performed on each problem-solving region to obtain the spatial logic parsing result. Based on the region label, the content recognition engine corresponding to the problem-solving region is called to perform content recognition on the problem-solving region to obtain the content recognition result. Finally, a problem-solving process chain is constructed based on the spatial logic parsing result and the content recognition result, and the problem-solving process chain is input into a diagnostic model to output the diagnostic result. Based on the diagnostic result, a diagnostic report is generated and returned to the user terminal. Here, the diagnostic model is a machine learning model built based on a symbolic computation engine and a large language model.

[0023] In this application, a solution image uploaded by a user is received, and the image is divided into multiple solution regions. These regions are then further diagnosed to provide targeted diagnosis for different content. Spatial logic analysis is performed on each solution region to reconstruct the logical sequence of the solution steps, thus rebuilding the thought process and ensuring the logical coherence of the solution chain. For different solution regions, different content recognition engines are used to accurately identify the content within each region. The solution chain is constructed by combining the spatial logic analysis results and the content recognition results. A multi-dimensional deep diagnosis is performed using a hybrid model based on symbolic computation and large language models to accurately locate error positions and causes, generating personalized diagnostic reports. This improves the targeting and effectiveness of error diagnosis, enabling precise location of error causes. This solves the technical problem in related technologies that focus on diagnosing user-input answers but lack effective diagnosis of the solution process and cannot accurately locate error causes. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 This is a flowchart of an optional problem-solving process diagnostic method according to an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of a diagnostic process for an optional problem-solving procedure according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of a diagnostic device for an optional problem-solving process according to an embodiment of the present invention;

[0028] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) that performs a diagnostic method for solving a problem according to an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] To facilitate understanding of the present invention by those skilled in the art, some terms or nouns involved in the various embodiments of the present invention are explained below:

[0032] Optical Character Recognition (OCR) is an artificial intelligence technology primarily used to convert handwritten or printed text in scanned or photographed documents or images into a computer-readable and editable text format.

[0033] LaTeX is a document preparation system and markup language widely used for typesetting and producing scientific, mathematical, engineering, and technical documents.

[0034] Large Language Model (LLM) is a type of deep learning model in the field of natural language processing.

[0035] Artificial intelligence (AI) is a field of research that studies the intelligent behavior exhibited by computer systems.

[0036] It should be noted that the diagnostic method and apparatus for the problem-solving process in this application can be used in the field of artificial intelligence technology when diagnosing the problem-solving process based on artificial intelligence, or in any field other than the field of artificial intelligence technology when diagnosing the problem-solving process based on artificial intelligence. This application does not limit the application field of the diagnostic method and apparatus for the problem-solving process.

[0037] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, and displayed data) involved in this application are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of such data all comply with the relevant laws, regulations, and standards of the relevant regions, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse. For example, this system has interfaces with relevant users or organizations. Before obtaining relevant information, a request to obtain the information needs to be sent to the aforementioned user or organization through the interface, and the relevant information is obtained only after receiving consent from the aforementioned user or organization.

[0038] It should be noted that in this application, when collecting and analyzing customer information, users are provided with corresponding operation entry points to choose whether to agree to or reject the automated decision-making results; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0039] The following embodiments of the present invention can be applied to various diagnostic systems / applications / devices for problem-solving processes. The present invention transforms unstructured images into ordered, analyzable problem-solving step chains through layout analysis, logical reconstruction, and content recognition. It achieves automated analysis of the user's thought process, breaking through the limitation of traditional tools that can only process standard text, and greatly expanding the application scenarios and depth of AI tutoring. Combining symbolic computation with multidimensional error diagnosis using large language models, it can distinguish between computational errors (such as addition, subtraction, multiplication, and division errors) and conceptual / logical errors (such as incorrect formula usage, logical inconsistencies, and missing steps), achieving accurate attribution of error types. Based on the diagnostic results, it generates guided dialogue-based interactive tutoring, not directly providing the correct answer, but generating heuristic questions based on the user's error type to guide the user to independently discover and correct errors.

[0040] The present invention will now be described in detail with reference to various embodiments.

[0041] Example 1

[0042] According to an embodiment of the present invention, an embodiment of a diagnostic method for a problem-solving process is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0043] Figure 1 This is a flowchart of an optional problem-solving process diagnostic method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0044] Step S101: Receive the solution image corresponding to the solution process sent by the user terminal, and preprocess the solution image.

[0045] The embodiment of this invention is a problem-solving process diagnostic system. In step S101 above, the user takes and uploads images of the problem-solving process using a mobile device. These images can be handwritten problem-solving steps on paper, containing various types of information such as mathematical formulas, text descriptions, and diagrams. The system's front-end module is responsible for receiving the image data and temporarily storing it in the system server's cache area, ready for subsequent processing.

[0046] To enhance the quality of the problem-solving images, the system performs a series of preprocessing operations. These include: image enhancement, employing adaptive histogram equalization and contrast adjustment to improve image clarity and make handwritten strokes more visible; noise removal, using algorithms such as Gaussian filtering and median filtering to remove noise from the image and prevent interference from the shooting environment from affecting the recognition results; rotation and layout correction, using techniques such as Hough transform to detect tilt angles in the image and perform rotation correction to ensure the image content is a planar view; simultaneously, perspective transformation algorithms are used to correct layout distortion caused by different shooting angles of handheld devices, creating favorable conditions for subsequent content recognition; and binarization, converting the image into a black and white binary image to further enhance the contrast between handwriting and background and reduce interference factors in the recognition stage.

[0047] Step S102: Input the preprocessed solution image into the target recognition model. The target recognition model divides the solution image into N solution regions and assigns a region label to each solution region.

[0048] In step S102 above, the preprocessed solution image is input into a deep learning object recognition model. This model is a pre-built machine learning model for image recognition, which extracts high-level visual features from the image using a convolutional neural network architecture. These features encompass information such as lines, shapes, and textures in the image, and are key to identifying different elements. Based on the extracted features, the object recognition model further performs object detection, identifying different types of solution elements in the image, such as mathematical formulas, text descriptions, charts, tables, or modification marks. The model classifies these elements into one of the aforementioned categories by setting a specific classifier. For elements of the same type detected in the image, the model generates bounding boxes. These bounding boxes accurately delineate the range of each solution region, ensuring that the content of each region can be processed independently in subsequent steps, thus obtaining N solution regions, where N is a positive integer. Simultaneously, a region label is assigned to each divided solution region according to the identified elements; the region label indicates the type of identified element.

[0049] Further, the steps of inputting the preprocessed solution image into the target recognition model, dividing the solution image into N solution regions by the target recognition model, and configuring a region label for each solution region include: inputting the preprocessed solution image into the target recognition model, extracting image features by the target recognition model, and identifying different elements in the solution image based on the image features; generating bounding boxes for the same elements within a preset region to obtain N solution regions; identifying the element category within each solution region, and configuring a region label for each solution region based on the element category, wherein the region label includes at least one of the following: formula label, text label, image label, table label, and modification trace label.

[0050] Specifically, the preprocessed solution images are fed into an object recognition model, which, based on deep learning technology, automatically extracts key features from the image through a multi-layered structure of a convolutional neural network. These features cover the shape, size, and position of the handwriting, as well as background information of the image, providing rich visual cues for subsequent element recognition.

[0051] By analyzing the extracted image features, the target recognition model can intelligently identify various elements in problem-solving images, including mathematical formulas, text descriptions, charts, tables, and possible traces of modification. After training on a large number of handwritten problem-solving samples, the model can recognize subtle differences and underlying logical relationships among these elements.

[0052] For a predefined problem-solving region in the image, the object recognition model clusters elements of the same type and generates precise bounding boxes for each category. These bounding boxes not only accurately identify the range of each problem-solving element but also ensure that different types of problem-solving elements can be clearly distinguished, facilitating subsequent independent processing. Through the generation of bounding boxes, the problem-solving image is automatically divided into N problem-solving regions, where N is a positive integer representing the total number of identifiable elements in the image. Each problem-solving region is an independent unit of information and can contain, but is not limited to, formulas, text, charts, tables, or modification marks.

[0053] Based on the analysis of the target recognition model, the system can determine the specific category of elements within each problem-solving area. For example, a mathematical formula will be recognized as a formula element, while a text description will be considered a text element. After identifying the element category, the system assigns corresponding area labels to each problem-solving area, such as formula labels, text labels, image labels, table labels, and modification trace labels. These labels clearly mark the content type of each area, enabling the subsequent content recognition engine to make targeted calls, effectively improving recognition efficiency and accuracy.

[0054] Step S103: Perform spatial logic analysis on each solution area to obtain the spatial logic analysis result.

[0055] In step S103 above, the focus is on spatial logic analysis of each problem-solving area to determine the logical relationships between and within each area, and the correct order of the problem-solving process. The system first performs a preliminary spatial sorting of the divided problem-solving areas, typically following human reading habits—from top to bottom and from left to right. This establishes the basic framework of the problem-solving process. Subsequently, the system introduces an additional logical symbol detection mechanism to specifically identify symbols representing logical connections between steps, such as arrows, sequence numbers, and logical words (e.g., "therefore," "but," etc.). These symbols are important clues guiding the problem-solving process. Based on the detected logical symbols, the system corrects the preliminary spatial sorting results to ensure that the relationships between steps conform to the user's problem-solving logic. For example, if an arrow is detected pointing from one "formula area" to another "text description area," the system will adjust the order of these two areas to ensure that the formula is displayed first, followed by the description of other content. Based on the detected logical symbols, the system corrects the preliminary spatial sorting results to ensure that the relationships between steps conform to the user's problem-solving logic. For example, if an arrow is detected pointing from one "formula area" to another "text description area", the system will adjust the order of the two areas to ensure that the formula is displayed first, followed by its description.

[0056] Through the above steps, the logical order of the problem-solving process is reconstructed, spatially adjacent but logically unrelated elements are correctly separated, while the connection between logically closely connected elements is strengthened.

[0057] Furthermore, the steps for performing spatial logic analysis on each problem-solving region to obtain the spatial logic analysis result include: calling a symbol detection model to identify logical connection symbols between and within each problem-solving region, and recording the position information of the logical connection symbols. Here, the symbol detection model is a pre-built machine learning model for detecting logical connection symbols; identifying the logical relationships between elements based on the logical connection symbols and position information; constructing a directed graph based on the logical relationships, and sequentially reconstructing each problem-solving region and the interior of each problem-solving region based on the directed graph and preset priority rules, and obtaining the spatial logic analysis result based on the reconstructed problem-solving regions.

[0058] Specifically, during spatial logic parsing, a pre-built symbol detection model is invoked. This model is a deep learning model specifically designed to identify logical connection symbols, such as arrows, ordinal numbers, and logical connectors (e.g., "therefore," "thus," etc.). These symbols are crucial for understanding the logical relationships in the problem-solving steps. The symbol detection model runs on each problem-solving region, accurately identifying logical connection symbols between and within regions, and recording the precise location information of these symbols in the image. This location information provides coordinate references for subsequent construction of logical relationships. Based on the detected logical connection symbols and their locations, the system constructs a mapping of logical relationships between elements. For example, when the model detects an arrow pointing from region A to region B, the system interprets it as "the content of region A is a precondition for the content of region B." Through this mapping, the system constructs a directed graph with problem-solving regions as nodes and logical connection symbols as edges. Each node represents a problem-solving region, and each edge represents a logical dependency between two regions.

[0059] The system analyzes the directed graph using preset priority rules to determine the final logical order between elements. Priority rules include ordinal guidance, dependency indication (arrow pointing), and spatial proximity as supplementary criteria. Based on the directed graph analyzed by the priority rules, the system reconstructs the order, ultimately obtaining a spatial logic parsing result that conforms to the user's problem-solving logic. This result clearly demonstrates the correct order and logical structure of the problem-solving steps, providing an ordered reference for subsequent content recognition.

[0060] Step S104: Based on the region label, call the content recognition engine corresponding to the problem-solving region, and use the content recognition engine to perform content recognition on the problem-solving region to obtain the content recognition result.

[0061] In step S204 above, based on the configured region labels, the system invokes the matching content recognition engine to perform detailed content recognition on each problem-solving region to obtain accurate text and formula descriptions of the problem-solving steps. The system checks the region label of each problem-solving region and determines which type of content recognition engine to invoke based on the element type indicated by the label. For example, a problem-solving region with a formula label will trigger an OCR engine specifically for mathematical formulas. Once the engine is invoked, it immediately performs content recognition on the problem-solving region. For formula regions, the recognition engine will attempt to convert the formulas in the image into structured LaTeX code, etc.; for text regions, the engine will convert handwritten or printed text into an editable text format. The system binds the recognized content with the specific location information and logical sequence number of the problem-solving region, forming a three-in-one data package of "location-content-logical sequence number". These data packages constitute a structured representation of the problem-solving process, facilitating subsequent in-depth analysis.

[0062] Furthermore, the steps for obtaining content recognition results by performing content recognition on the problem-solving area through a content recognition engine include: identifying content features from the problem-solving area through the content recognition engine; and, based on the content features, calling the knowledge base inside the content recognition engine to perform content matching to obtain the content recognition results.

[0063] Specifically, the content recognition engine first runs a deep learning model, such as a convolutional neural network, to extract key content features from the problem-solving area. These features may include the shape of strokes, the structure of characters, the layout of formula elements, and the texture of images, reflecting the details and complexity of the problem content. The extracted content features are further analyzed to determine the specific type of elements in the problem-solving area. For example, the engine can identify the presence of mathematical formulas, text descriptions, or images in a certain area. The content recognition engine maintains an internal knowledge base containing a large number of predefined content templates and rules, covering various common problem-solving element forms and expressions. Based on the content features of the problem-solving area, the engine queries its internal knowledge base to find the content template or rule with the highest degree of matching with the extracted features. For example, for a mathematical formula, the engine will try to match the closest formula structure; for a text description, it will try to identify the closest semantic expression. Once a match is found, the engine generates content recognition results, including the recognized LaTeX code of the mathematical formula, the text content of the text description, and the descriptive tags of the image. These results directly reflect the original meaning and expression of the elements in the problem-solving area.

[0064] In the steps described above, deep feature recognition and precise matching with the internal knowledge base ensure that the recognition results for each element in the problem-solving area are both accurate and comprehensive. This process overcomes the diversity and complexity of the problem content, providing high-quality data input for subsequent intelligent diagnosis.

[0065] Step S105: Construct a problem-solving process chain based on the spatial logic parsing results and content recognition results, input the problem-solving process chain into the diagnostic model, output the diagnostic results, generate a diagnostic report based on the diagnostic results, and return the diagnostic report to the user terminal.

[0066] Step S105 focuses on integrating the spatial logic parsing results with the content recognition results into a problem-solving process chain. It then utilizes a diagnostic model that integrates a symbolic computation engine and a large language model for intelligent diagnosis, generating a detailed diagnostic report. This diagnostic model is a machine learning model built upon the symbolic computation engine and the large language model. Based on the spatial logic parsing results, the system connects the elements in the content recognition results in a logical order, forming an ordered problem-solving process chain. This chain includes not only the recognized formulas and text but also the logical relationships between the problem-solving steps, such as causal relationships and conditional relationships. The system performs preliminary verification of the constructed problem-solving process chain to ensure the continuity and integrity of the elements within the chain, eliminating any logical breaks or redundant elements caused by recognition errors.

[0067] The diagnostic model receives a pre-constructed problem-solving process chain as input. First, it uses a symbolic computation engine for precise mathematical verification, checking the accuracy of calculations and logical derivations. Then, through a large language model, it performs semantic analysis on the language descriptions and conceptual applications of the problem-solving process, ensuring that the process conforms to disciplinary norms and cognitive logic. The diagnostic model integrates the results of symbolic computation and language understanding to generate a detailed diagnostic report. This report includes the diagnostic status of each step, such as calculation errors, conceptual misunderstandings, or logical flaws, and may indicate the specific location and possible causes of these errors. Finally, the diagnostic model integrates the results of symbolic computation and language understanding to generate a detailed diagnostic report. This report includes the diagnostic status of each step, such as calculation errors, conceptual misunderstandings, or logical flaws, and may indicate the specific location and possible causes of these errors. The diagnostic report is packaged and sent back to the user's client, where the user can view the report through a client application to receive feedback and guidance on their problem-solving process.

[0068] Furthermore, the steps for constructing a problem-solving process chain based on the spatial logic parsing results and content recognition results include: associating the spatial logic parsing results and content recognition results to obtain association results; sorting the problem-solving content according to the logical order recorded in the spatial logic parsing results based on the association results; and constructing a problem-solving process chain based on the sorting results.

[0069] Specifically, the system binds the location information of the problem-solving region marked in the spatial logic parsing results with the text descriptions, formula expressions, and image labels identified in the content recognition results, forming a set. Each element carries its spatial coordinates and content information in the original image. For each problem-solving region, the system verifies whether its content recognition result matches the expectations in the spatial logic parsing results. For example, if the spatial logic parsing result predicts that a certain region is a formula region, it needs to confirm whether the content recognition result is indeed a mathematical formula to ensure data consistency.

[0070] Based on the logical order (such as sequence number, arrow direction, etc.) recorded in the spatial logic parsing results, the system reorganizes the associated content recognition results to ensure the logical coherence of the problem-solving steps. Even if some steps are spatially adjacent but logically discontinuous in the original image, they will be sorted according to the correct logical relationship. After completing the logical order sorting, the system connects these sorted problem-solving contents into a directed graph, where nodes represent problem-solving steps and edges represent the logical relationships between steps. This directed graph is the problem-solving process chain, which completely represents the logical flow of the problem-solving process, facilitating in-depth analysis by the subsequent diagnostic module.

[0071] Through the above steps, we can ensure a high degree of synergy between the content recognition results and the spatial logic parsing results, eliminate parsing errors caused by factors such as image shooting angle, handwriting style, or blurred handwriting, and construct a problem-solving process chain that accurately reflects the user's problem-solving logic and content details.

[0072] Furthermore, the steps of inputting the problem-solving process chain into the diagnostic model and outputting diagnostic results include: inputting the problem-solving process chain into the diagnostic model, identifying computational errors and logical derivation errors in the problem-solving process chain through the symbolic computation engine in the diagnostic model to obtain a first diagnostic result; identifying conceptual errors, strategic errors, and content omissions in the problem-solving process chain through the large language model in the diagnostic model to obtain a second diagnostic result; associating the first and second diagnostic results, and calling a pre-built knowledge graph based on the association result to determine the root cause error; and constructing a diagnostic result based on the association result and the root cause error.

[0073] Specifically, each mathematical expression or logical deduction in the problem-solving process chain is submitted to the symbolic computation engine, which is responsible for verifying the accuracy of each calculation and the logical consistency of the derivation. For example, the engine checks whether steps such as algebraic transformations, integral operations, and equation solving conform to mathematical rules. The engine identifies unreasonable computational steps or logical jumps, such as directly taking the square root of a negative number or unjustifiably eliminating variables on both sides of an equation. Through this mechanism, computational errors and logical flaws are accurately identified. The large language model deeply analyzes the textual descriptions in the problem-solving process chain to identify any misunderstandings of basic concepts, such as confusing theorems, ignoring assumptions, or misinterpreting the meaning of technical terms. The model evaluates the appropriateness of the chosen problem-solving strategy, such as whether the optimal solution path was selected, and whether there is overcomplication or neglect of simplification techniques. Furthermore, it can identify potentially omitted key steps in the solution.

[0074] The system performs correlation analysis between the first diagnostic result from the symbolic computation engine and the second diagnostic result from the large language model to ensure that computational errors correspond to errors in conceptual understanding, problem-solving strategies, etc., avoiding omissions or duplicate diagnoses. Based on the results of the correlation analysis, the system calls upon a pre-built knowledge graph to match the identified error types with specific knowledge points in the graph, determining which specific knowledge point or problem-solving skill is the root cause of the error. Finally, the system integrates the first and second diagnostic results and constructs a comprehensive diagnostic report based on the root cause error information. The report clearly indicates the location and type of the error, as well as the related knowledge points and skill deficiencies.

[0075] The dual-engine diagnostic approach of the symbolic computation engine and the large language model aims to ensure that all types of errors in the problem-solving process can be accurately captured through multi-level and multi-angle error identification and analysis. This is not limited to simple calculation errors, but also includes deeper conceptual biases and strategic mistakes, providing comprehensive data support for subsequent personalized tutoring and error correction.

[0076] Furthermore, the steps for generating a diagnostic report based on the diagnostic results include: annotating the incorrect content in the solution image based on the diagnostic results to obtain an annotated solution image; generating guidance information based on the diagnostic results and finding knowledge points based on the diagnostic results; and generating a diagnostic report based on the annotated solution image, guidance information, and knowledge points.

[0077] Specifically, based on the error location information in the diagnostic results, the system uses specific visualization tools to highlight the specific areas where errors occur on the original solution image. For example, if the diagnostic results indicate a conceptual error in step 4, the system will add a red border or a highlight layer around the corresponding solution area in the solution image to visually display the error location. After highlighting the errors, the system generates an annotated solution image, which visually displays the location of all errors based on the user-submitted original solution image. This image allows users to easily see which parts need correction at a glance.

[0078] Furthermore, based on the error type and cause in the diagnostic results, the system constructs a series of guiding questions or prompts to help users understand the nature of the error. For example, when faced with a conceptual error, the system might generate a question that prompts deeper thinking: "Please consider whether we need to apply the concept of vectors in this context?" The guiding information aims to stimulate users' independent thinking rather than providing direct answers, thereby promoting the development of learning abilities and critical thinking.

[0079] Simultaneously, the system utilizes the error attribution in the diagnostic results, combined with a pre-built subject knowledge graph, to identify core knowledge points related to the error. For example, if the error is related to the concept of "conservation of momentum," the system will locate the corresponding "conservation of momentum" node in the knowledge graph. The system generates links or indexes pointing to relevant knowledge points, facilitating users to further explore the root cause of the error and solidify related concepts.

[0080] Finally, the system integrates the annotated solution images, guidance information, and knowledge point links into an intuitive diagnostic report. The report is clearly structured, first presenting the incorrectly annotated images, then listing detailed information for each error, including the error type, attribution analysis, and guidance questions, and finally providing knowledge point links. The diagnostic report is packaged and sent to the user's client application via the network. Users can directly view and use the report within the application to correct errors and learn new knowledge.

[0081] By accurately labeling errors, generating guiding questions, and searching for relevant knowledge points, the system achieves efficient conversion of intelligent diagnostic results and personalized feedback.

[0082] Through the above steps, the system receives the problem-solving image corresponding to the problem-solving process sent by the user, preprocesses the problem-solving image, and inputs the preprocessed problem-solving image into the target recognition model. The target recognition model divides the problem-solving image into N problem-solving regions and configures a region label for each problem-solving region. The target recognition model is a pre-built machine learning model for image recognition, where N is a positive integer. Then, spatial logic parsing is performed on each problem-solving region to obtain the spatial logic parsing result. Based on the region label, the corresponding content recognition engine is called to perform content recognition on the problem-solving region to obtain the content recognition result. Finally, a problem-solving process chain is constructed based on the spatial logic parsing result and the content recognition result, and the problem-solving process chain is input into the diagnostic model to output the diagnostic result. A diagnostic report is generated based on the diagnostic result and returned to the user. The diagnostic model is a machine learning model built based on a symbolic computation engine and a large language model.

[0083] In this embodiment, the system receives a problem-solving image uploaded by the user, divides the image into regions to obtain multiple problem-solving areas, and further diagnoses these areas. This allows for targeted diagnosis of different content. Spatial logic analysis is performed on each problem-solving area to reconstruct the logical sequence of the problem-solving steps, thus rebuilding the thought process of the problem-solving process and ensuring the logical coherence of the problem-solving process chain. For different problem-solving areas, different content recognition engines are invoked to accurately identify the content within the area. The problem-solving process chain is constructed by combining the spatial logic analysis results and the content recognition results. Multi-dimensional deep diagnosis is performed through a hybrid model based on symbolic computation and large language models to accurately locate the error location and cause, generating a personalized diagnostic report. This improves the targeting and effectiveness of error diagnosis, enabling precise location of error causes. This solves the technical problem in related technologies that focus on diagnosing user-input answers but lack effective diagnosis of the problem-solving process and cannot accurately locate the cause of errors.

[0084] The following describes in detail another optional implementation method.

[0085] Figure 2 This is a schematic diagram of a diagnostic process for an optional problem-solving procedure according to an embodiment of the present invention, such as... Figure 2 As shown, the diagnostic process for solving the problem includes:

[0086] Step 1, Begin;

[0087] Step two, the user uploads a solution image;

[0088] Users capture image data of the problem-solving process using a user terminal (such as a mobile app or a dedicated application on a tablet) and upload it to the diagnostic process system server. To ensure the accuracy of subsequent recognition, the system first performs a series of preprocessing operations on the received raw images. These operations include, but are not limited to:

[0089] Image enhancement employs algorithms such as adaptive histogram equalization to adjust the contrast and brightness of the image, making the text clearer;

[0090] Noise removal uses methods such as Gaussian filtering or median filtering to remove noise from images caused by shooting environment or equipment limitations.

[0091] Viewpoint correction and layout correction: The tilt angle of text lines in the image is detected by algorithms such as Hough transform and rotation correction is performed; the perspective transformation algorithm is used to correct image distortion caused by tilted shooting, ensuring that the draft content is presented as a regular planar view.

[0092] Binarization converts a grayscale image into a black-and-white binary image, separating the handwriting from the background and highlighting key content.

[0093] Step 3: Analyze the problem-solving process, using deep learning-based layout analysis and content structuring;

[0094] The preprocessed, unordered problem-solving images are transformed into structured data that computers can understand and analyze. This step can be broken down into the following three stages:

[0095] Content block identification (corresponding to the problem-solving areas mentioned above) employs a deep learning model based on object detection to scan and analyze the problem-solving image. This model can accurately identify different content regions in the image, obtaining multiple problem-solving regions, and assigning a classification label to each region (corresponding to the region labels mentioned above), such as "formula blocks" (containing mathematical formulas, equations, chemical equations, etc.), "text blocks" (containing textual descriptions of problem-solving strategies), "graph blocks" (containing geometric figures, function graphs, circuit diagrams, force analysis diagrams, etc., to assist in problem-solving), and "tampering marks."

[0096] To accurately reproduce the user's problem-solving thought process, a spatial-logical sorting algorithm is employed to handle common issues such as jumps and non-linear writing in problem-solving images. This algorithm, based on initial spatial sorting (following conventional reading habits from top to bottom and left to right), intelligently corrects errors using identified logical symbols. The specific implementation process is as follows:

[0097] Logical symbol recognition and localization utilizes a lightweight symbol detection model specifically designed to identify logical connection symbols between or within content blocks, such as arrows (→). The diagram includes symbols such as , ↓, etc., serial numbers (e.g., 1), 2), ①, ②, etc., and mathematical symbols (∴, ∵) representing "therefore" and "because", and records the type of these symbols and their precise position in the solution diagram.

[0098] The system constructs and corrects the sorting relationship graph based on the identified logical symbols, building a directed graph with content blocks as nodes and logical relationships as edges. It then adjusts the initial sorting results according to the following preset priority rules:

[0099] Strong constraint rules (serial numbers) have the highest priority. When a clear numerical or alphanumeric serial number is identified, the system will force the corresponding content blocks to be arranged in that serial number order.

[0100] Dependency rules (arrows / logic words): When arrows or other connectors are identified, the system places the content block where the arrow starts before the content block where the arrow ends, forming a local dependency.

[0101] For content blocks not covered by the above rules, the system will maintain the original spatial order.

[0102] Content recognition and structuring accurately identifies each block in an ordered sequence of content blocks. To improve the accuracy of recognizing mixed content, a differentiated OCR engine invocation strategy based on content block classification is adopted, which is divided into the following two stages:

[0103] The engine distribution mechanism distributes different content block images to the dedicated OCR engine best suited for processing that type of content, based on the content block's category tags.

[0104] For formula blocks, a dedicated mathematical formula OCR engine is invoked, which can accurately convert complex formula images into structured LaTeX code and other formats, fully preserving the two-dimensional structural information of the formula.

[0105] For text blocks, a mature general-purpose text OCR engine is used to efficiently recognize handwritten Chinese and English text.

[0106] The recognition results are normalized and correlated. Results returned by different engines (such as LaTeX code and plain text strings) are normalized. The system binds the recognized content with its corresponding image region coordinates and logical sequence number in the original image, ultimately forming a structured problem-solving step chain (corresponding to the problem-solving process chain mentioned above). This structure provides a complete and ordered input for subsequent intelligent diagnosis.

[0107] Step 4: Hybrid intelligent error diagnosis based on symbolic computation and large language models.

[0108] Through a multi-stage, dual-engine parallel diagnostic process, the structural problem-solving steps are analyzed in depth to achieve accurate error location and in-depth attribution.

[0109] For deterministic error diagnosis (based on a symbolic computation engine), the system inputs the formulas and derivations from the structured problem-solving steps into the symbolic computation engine. This engine, based on rigorous mathematical axioms and theorems, performs step-by-step mathematical equivalence verification. Its main tasks are: locating calculation errors, identifying mistakes in the calculation process (errors in addition, subtraction, multiplication, division, exponentiation, root extraction, etc.); verifying the derivation logic, checking whether the algebraic transformations and formula applications between steps are valid; and discovering logical contradictions, identifying expressions or inferences that do not conform to mathematical definitions. The engine's output is objective and accurate, providing irrefutable "factual evidence" for all deterministic errors.

[0110] Conceptual and Strategic Error Diagnosis (Based on Large Language Model): The system inputs the complete structured problem-solving step chain (including text and formulas) into a large language model (LLM) enhanced with mathematical knowledge. To guide the LLM in professional diagnosis, the system constructs a specific prompt that instructs the LLM to analyze from the following dimensions: diagnosing conceptual errors, identifying whether subject concepts are confused (e.g., confusing "median" with "mean" in mathematics), and whether the applicable conditions of theorems or formulas are incorrectly applied; evaluating problem-solving strategies, judging whether the user's chosen problem-solving method is appropriate and whether there are directional errors in the thinking process; and identifying missing steps, discovering whether any key steps have been omitted in the problem-solving process.

[0111] Multi-source diagnostic fusion and knowledge point association: The system intelligently fuses and deeply processes the diagnostic results of the two engines mentioned above: result fusion, matching and associating the "objective error points" found by the symbolic computation engine with the "attribution explanations" given by LLM; knowledge point association, accurately matching the root causes of errors analyzed by LLM (such as "errors in applying the law of conservation of energy" or "errors in applying the discriminant of the roots of a quadratic equation") with the system's pre-set subject knowledge graph to locate specific weak knowledge point nodes.

[0112] Generate an internal structured diagnostic report, integrating the analysis results from the two engines to generate an internal structured diagnostic report that includes the error location, error type, and related knowledge points.

[0113] Finally, the system integrates all analysis results to generate an internal structured diagnostic report for subsequent steps. Key fields in this report may include: error location, error content, error type (such as "calculation error" or "conceptual error"), detailed error analysis, and related knowledge point identifiers pointing to the corresponding nodes in the knowledge graph.

[0114] Step 5: Determine if there is an error. If yes, proceed to step 6. If no, output praise.

[0115] Step six: Generative dialogue guidance and interactive correction.

[0116] Error visualization annotation: The system visually marks the specific location of errors on the original solution image uploaded by the user, using methods such as highlighting and annotation.

[0117] The system generates a diagnostic report by combining the integrated diagnostic results. The report is easy to understand and clearly explains the error type, the cause of the error, and points out the relevant knowledge points involved.

[0118] The system uses heuristic dialogue guidance, meaning it doesn't directly provide the correct answer. Instead, it mimics a teacher's tone, generating a series of thought-provoking questions around the point of error to guide the conversation. For example, regarding an incorrect application of a formula, the system might ask, "Please double-check the conditions for applying this law; are they all met?"

[0119] Knowledge linking and reinforcement: During the conversation, the system will intelligently recommend relevant knowledge point explanations, micro-course videos, or similar practice questions based on the type of error, to help users consolidate their weak areas.

[0120] Step seven, end.

[0121] This invention transforms unstructured images into ordered, analyzable problem-solving steps through layout analysis, logical reconstruction, and content recognition. It automates the analysis of the user's thought process, overcoming the limitations of traditional tools that can only process standard text, and significantly expanding the application scenarios and depth of AI tutoring. Combining symbolic computation with multidimensional error diagnosis using a large language model, it can distinguish between computational errors (such as addition, subtraction, multiplication, and division errors) and conceptual / logical errors (such as incorrect formula usage, logical inconsistencies, and missing steps), achieving accurate attribution of error types. Based on the diagnostic results, it generates guided dialogue-based interactive tutoring that does not directly provide the correct answer but instead generates heuristic questions based on the user's error type, guiding the user to independently discover and correct their mistakes.

[0122] The following is a detailed description with reference to another embodiment.

[0123] Example 2

[0124] The diagnostic device for problem-solving process provided in this embodiment includes multiple implementation units, each of which corresponds to a specific implementation step in the above embodiment one. The specific implementation method and beneficial effects can be referred to the foregoing method embodiment, and will not be repeated here.

[0125] Figure 3This is a schematic diagram of an optional problem-solving process diagnostic device according to an embodiment of the present invention, such as... Figure 3 As shown, the diagnostic device for this problem-solving process may include: a receiving unit 31, a dividing unit 32, a parsing unit 33, an identification unit 34, and a diagnostic unit 35, wherein,

[0126] The receiving unit 31 is used to receive the solution image corresponding to the solution process sent by the user terminal, and to preprocess the solution image;

[0127] The segmentation unit 32 is used to input the preprocessed problem-solving image into the target recognition model, which divides the problem-solving image into N problem-solving regions and configures a region label for each problem-solving region. The target recognition model is a pre-built machine learning model for image recognition, and N is a positive integer.

[0128] Analysis unit 33 is used to perform spatial logic analysis on each problem-solving area to obtain the spatial logic analysis result;

[0129] The recognition unit 34 is used to call the content recognition engine corresponding to the problem-solving area based on the area label, and to perform content recognition on the problem-solving area through the content recognition engine to obtain the content recognition result;

[0130] The diagnostic unit 35 is used to construct a problem-solving process chain based on the spatial logic parsing results and content recognition results, input the problem-solving process chain into the diagnostic model, output the diagnostic results, generate a diagnostic report based on the diagnostic results, and return the diagnostic report to the user terminal. The diagnostic model is a machine learning model built based on a symbolic computing engine and a large language model.

[0131] The diagnostic device for the above-mentioned problem-solving process receives the problem-solving image corresponding to the problem-solving process sent by the user terminal through the receiving unit 31 and preprocesses the problem-solving image; the segmentation unit 32 inputs the preprocessed problem-solving image into the target recognition model, which divides the problem-solving image into N problem-solving regions and configures a region label for each problem-solving region. The target recognition model is a pre-built machine learning model for image recognition, and N is a positive integer; the parsing unit 33 performs spatial logic parsing on each problem-solving region to obtain the spatial logic parsing result; the recognition unit 34 calls the content recognition engine corresponding to the problem-solving region based on the region label, and performs content recognition on the problem-solving region to obtain the content recognition result; the diagnostic unit 35 constructs a problem-solving process chain based on the spatial logic parsing result and the content recognition result, inputs the problem-solving process chain into the diagnostic model, outputs the diagnostic result, generates a diagnostic report based on the diagnostic result, and returns the diagnostic report to the user terminal. The diagnostic model is a machine learning model built based on a symbolic computation engine and a large language model.

[0132] In this embodiment, the system receives a problem-solving image uploaded by the user, divides the image into regions to obtain multiple problem-solving areas, and further diagnoses these areas. This allows for targeted diagnosis of different content. Spatial logic analysis is performed on each problem-solving area to reconstruct the logical sequence of the problem-solving steps, thus rebuilding the thought process of the problem-solving process and ensuring the logical coherence of the problem-solving process chain. For different problem-solving areas, different content recognition engines are invoked to accurately identify the content within the area. The problem-solving process chain is constructed by combining the spatial logic analysis results and the content recognition results. Multi-dimensional deep diagnosis is performed through a hybrid model based on symbolic computation and large language models to accurately locate the error location and cause, generating a personalized diagnostic report. This improves the targeting and effectiveness of error diagnosis, enabling precise location of error causes. This solves the technical problem in related technologies that focus on diagnosing user-input answers but lack effective diagnosis of the problem-solving process and cannot accurately locate the cause of errors.

[0133] Furthermore, the partitioning unit includes: a first extraction module, used to input the preprocessed solution image into the target recognition model, extract image features through the target recognition model, and identify different elements in the solution image based on the image features; a first generation module, used to generate bounding boxes for the same elements within a preset area, resulting in N solution regions; and a first configuration module, used to identify the element category within each solution region and configure a region label for each solution region based on the element category, wherein the region label includes at least one of the following: formula label, text label, image label, table label, and modification trace label.

[0134] Furthermore, the parsing unit includes: a first identification module, used to call a symbol detection model to identify logical connection symbols between and within each problem-solving region, and record the position information of the logical connection symbols. The symbol detection model is a pre-built machine learning model for detecting logical connection symbols. A second identification module is used to identify the logical relationships between elements based on logical connection symbols and position information. A first parsing module is used to construct a directed graph based on logical relationships, and to sequentially reconstruct each problem-solving region and the interior of each problem-solving region based on the directed graph and preset priority rules, and to obtain spatial logical parsing results based on the reconstructed problem-solving regions.

[0135] Furthermore, the recognition unit includes: a third recognition module, used to recognize content features from the problem-solving area through a content recognition engine; and a first matching module, used to perform content matching based on the content features by calling the knowledge base inside the content recognition engine to obtain the content recognition result.

[0136] Furthermore, the diagnostic unit includes: a first association module, used to associate the spatial logic parsing results and the content recognition results to obtain an association result; and a first construction module, used to sort the problem-solving content according to the logical order recorded in the spatial logic parsing results based on the association result, and construct a problem-solving process chain based on the sorting result.

[0137] Furthermore, the diagnostic unit includes: a fourth identification module, used to input the problem-solving process chain into the diagnostic model, and identify calculation errors and derivation logic errors in the problem-solving process chain through the symbolic computation engine in the diagnostic model to obtain a first diagnostic result; a fifth identification module, used to identify conceptual errors, strategic errors, and content omissions in the problem-solving process chain through the large language model in the diagnostic model to obtain a second diagnostic result; a first determination module, used to associate the first diagnostic result and the second diagnostic result, and call a pre-built knowledge graph to determine the root cause error based on the association result; and a second construction module, used to construct a diagnostic result based on the association result and the root cause error.

[0138] Furthermore, the diagnostic unit also includes: a first annotation module, used to annotate the incorrect content in the solution image based on the diagnostic results, to obtain an annotated solution image; a second generation module, used to generate guidance information based on the diagnostic results, and to find knowledge points based on the diagnostic results; and a third generation module, used to generate a diagnostic report based on the annotated solution image, guidance information, and knowledge points.

[0139] The diagnostic device for the above problem-solving process may also include a processor and a memory. The receiving unit 31, the dividing unit 32, the parsing unit 33, the recognizing unit 34, the diagnostic unit 35, etc. are all stored in the memory as program units. The processor executes the above program units stored in the memory to realize the corresponding functions.

[0140] The processor described above contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and by adjusting kernel parameters, errors in the problem-solving process can be identified and diagnosed.

[0141] The aforementioned memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0142] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein the diagnostic method controls the device where the computer-readable storage medium is located to perform any of the above-described problem-solving processes when the computer program is running.

[0143] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the diagnostic method of any of the above-described problem-solving processes.

[0144] According to another aspect of the present invention, a computer program product is also provided, the computer program product including a computer program, wherein the computer program, when executed by a processor, implements the diagnostic method of any of the above-described problem-solving processes.

[0145] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program with the following method steps: receiving a problem-solving image corresponding to the problem-solving process sent by a user terminal, and preprocessing the problem-solving image; inputting the preprocessed problem-solving image into a target recognition model, dividing the problem-solving image into N problem-solving regions through the target recognition model, and configuring a region label for each problem-solving region, wherein the target recognition model is a pre-built machine learning model for image recognition, and N is a positive integer; performing spatial logic parsing for each problem-solving region to obtain the spatial logic parsing result; calling the content recognition engine corresponding to the problem-solving region based on the region label, performing content recognition on the problem-solving region through the content recognition engine to obtain the content recognition result; constructing a problem-solving process chain based on the spatial logic parsing result and the content recognition result, inputting the problem-solving process chain into a diagnostic model, outputting a diagnostic result, generating a diagnostic report based on the diagnostic result, and returning the diagnostic report to the user terminal, wherein the diagnostic model is a machine learning model built based on a symbolic computation engine and a large language model.

[0146] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) that performs a diagnostic method for solving a problem, according to an embodiment of the present invention. Figure 4 As shown, an electronic device may include one or more processors ( Figure 4 The processor, denoted by 402a, 402b, ..., 402n, can include, but is not limited to, a processing device such as a microprocessor (MCU) or a programmable logic device (FPGA), and a memory 404 for storing data. In addition, it may include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports in the I / O interface), a network interface, a keyboard, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown.

[0147] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0148] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0149] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0150] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0151] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0152] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0153] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for diagnosing the problem-solving process, characterized in that, include: Receive the solution image corresponding to the solution process sent by the user terminal, and preprocess the solution image; The preprocessed solution image is input into the target recognition model, which divides the solution image into N solution regions and assigns a region label to each solution region. The target recognition model is a pre-built machine learning model for image recognition, and N is a positive integer. For each of the problem-solving regions, spatial logic analysis is performed to obtain the spatial logic analysis results; Based on the region label, the content recognition engine corresponding to the problem-solving region is invoked, and the content recognition engine performs content recognition on the problem-solving region to obtain the content recognition result. Based on the spatial logic parsing results and the content recognition results, a problem-solving process chain is constructed, and the problem-solving process chain is input into the diagnostic model to output the diagnostic results. A diagnostic report is generated based on the diagnostic results and returned to the user terminal. The diagnostic model is a machine learning model built based on a symbolic computing engine and a large language model.

2. The method according to claim 1, characterized in that, The steps of inputting the preprocessed solution image into the target recognition model, dividing the solution image into N solution regions by the target recognition model, and configuring a region label for each solution region include: The preprocessed solution image is input into the target recognition model, the target recognition model extracts image features, and the different elements in the solution image are identified based on the image features; Generate bounding boxes for identical elements within a preset region to obtain N solution regions; Identify the element categories within each of the problem-solving regions, and configure region labels for each of the problem-solving regions based on the element categories, wherein the region labels include at least one of the following: formula labels, text labels, image labels, table labels, and modification trace labels.

3. The method according to claim 1, characterized in that, The steps for performing spatial logic analysis on each of the problem-solving regions to obtain the spatial logic analysis results include: The symbol detection model is invoked to identify logical connection symbols between and within each of the problem-solving regions, and the position information of the logical connection symbols is recorded. The symbol detection model is a pre-built machine learning model for detecting logical connection symbols. Based on the logical connection symbol and the location information, the logical relationship between the identification elements is recognized. A directed graph is constructed based on the logical relationship, and each of the problem-solving regions and the interior of the problem-solving regions are sequentially reconstructed based on the directed graph and preset priority rules. The spatial logic parsing result is obtained based on the reconstructed problem-solving regions.

4. The method according to claim 1, characterized in that, The steps for obtaining content recognition results by performing content recognition on the problem-solving area using the content recognition engine include: The content recognition engine identifies content features from the problem-solving area. Based on the content features, the knowledge base inside the content recognition engine is invoked to perform content matching, and the content recognition result is obtained.

5. The method according to claim 1, characterized in that, The steps for constructing a problem-solving process chain based on the spatial logic parsing results and the content recognition results include: The spatial logic parsing result and the content recognition result are correlated to obtain the correlation result; Based on the association results, the solution content is sorted according to the logical order recorded in the spatial logic parsing results, and the solution process chain is constructed based on the sorting results.

6. The method according to claim 1, characterized in that, The steps for inputting the problem-solving process chain into the diagnostic model and outputting the diagnostic results include: The problem-solving process chain is input into the diagnostic model, and the symbolic computation engine in the diagnostic model identifies calculation errors and derivation logic errors in the problem-solving process chain to obtain a first diagnostic result. The diagnostic model identifies conceptual errors, strategic errors, and content omissions in the problem-solving process chain using the large language model, thus obtaining a second diagnostic result. The first diagnostic result and the second diagnostic result are correlated, and the root cause error is determined by calling a pre-built knowledge graph based on the correlation result; The diagnostic results are constructed based on the correlation results and the root cause errors.

7. The method according to claim 6, characterized in that, The steps for generating a diagnostic report based on the diagnostic results include: Based on the diagnostic results, the incorrect content is marked in the solution image to obtain the marked solution image; Based on the diagnostic results, guidance information is generated, and knowledge points are searched based on the diagnostic results; The diagnostic report is generated based on the annotated solution image, the guidance information, and the knowledge points.

8. A diagnostic device for problem-solving processes, characterized in that, include: The receiving unit is used to receive the solution image corresponding to the solution process sent by the user terminal, and to preprocess the solution image; A segmentation unit is used to input the preprocessed problem-solving image into the target recognition model, and to divide the problem-solving image into N problem-solving regions through the target recognition model, and to configure a region label for each problem-solving region. The target recognition model is a pre-built machine learning model for image recognition, and N is a positive integer. The parsing unit is used to perform spatial logic parsing on each of the problem-solving regions to obtain spatial logic parsing results; The identification unit is used to call the content recognition engine corresponding to the problem-solving area based on the area label, and to perform content recognition on the problem-solving area through the content recognition engine to obtain the content recognition result. The diagnostic unit is used to construct a problem-solving process chain based on the spatial logic parsing results and the content recognition results, input the problem-solving process chain into the diagnostic model, output the diagnostic results, generate a diagnostic report based on the diagnostic results, and return the diagnostic report to the user terminal. The diagnostic model is a machine learning model built based on a symbolic computing engine and a large language model.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform a diagnostic method for the problem-solving process as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the diagnostic method for the problem-solving process as described in any one of claims 1 to 7.

Images