Image correction method and image correction system

Abstract

Embodiments of the present specification provide an image correction method and an image correction system, wherein the image correction method comprises: obtaining an initial question figure, a vector figure marking result, an answer text and a rendered question figure, wherein the vector figure marking result is obtained by marking a target figure element in the initial question figure as a vector figure based on the answer text, and the rendered question figure is obtained by rendering the target figure element based on the vector figure marking result; comparing the initial question figure and the rendered question figure to extract the target figure element; performing consistency verification on the target figure element based on the answer text to obtain a verification result; and adjusting the rendered question figure based on the verification result to obtain an image output result. The method actively and efficiently perceives errors existing in the target figure element, and adjusts the rendered question figure based on the verification result, thereby filling the gap of the lack of drawing quality control mechanism in the existing automatic drawing system.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of large model technology, and in particular to an image correction method and an image correction system. Background Technology

[0002] With the rapid development of computer graphics and artificial intelligence technologies, automated drawing has not only become possible but is also gradually maturing and being applied in multiple fields. Advances in computer graphics have provided a solid theoretical and technical foundation for image processing, modeling, and rendering, making high-precision, high-efficiency graphics generation a reality. Currently, automated drawing systems can automatically generate the required graphics based on basic parameters or natural language descriptions provided by users, greatly improving work efficiency in industries such as design, education, and manufacturing.

[0003] However, automated drawing systems still suffer from problems such as coordinate offset, size deviation, and drawing errors, resulting in inaccurate graphic positions and sizes, which affects the actual effect of automated drawing. Existing technologies lack effective correction mechanisms, making it difficult to detect and handle these problems in a timely manner. Summary of the Invention

[0004] In view of this, embodiments of this specification provide an image correction method. One or more embodiments of this specification also relate to an image correction system, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, an image correction method is provided, comprising: The process involves acquiring an initial question image, vector graphic marking results, answer text, and a rendered question image. The vector graphic marking results are obtained by marking the target graphic elements in the initial question image with vector graphics based on the answer text, and the rendered question image is obtained by rendering the target graphic elements based on the vector graphic marking results. The initial question image and the rendered question image are compared to extract the target graphic elements. Based on the answer text, the target graphic elements are validated for consistency to obtain the validation results. Based on the validation results, the rendered question image is adjusted to obtain the image output result.

[0006] According to a second aspect of the embodiments of this specification, an image correction system is provided, comprising: The content acquisition module is used to acquire the initial question image, vector graphic marking results, answer text, and rendered question image. The vector graphic marking results are obtained by vector graphic marking the target graphic elements in the initial question image based on the answer text, and the rendered question image is obtained by rendering the target graphic elements based on the vector graphic marking results. The reflection and correction module is used to compare the initial question image and the rendered question image to extract the target graphic elements; perform consistency verification on the target graphic elements based on the answer text to obtain the verification result; and adjust the rendered question image based on the verification result to obtain the image output result.

[0007] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the above-described image correction method.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the image correction method described above.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the image correction method described above.

[0010] The image correction method provided in one or more embodiments of this specification acquires an initial question image, vector graphic marking results, answer text, and a rendered question image. The vector graphic marking results are obtained by marking target graphic elements in the initial question image using vector graphics based on the answer text, and the rendered question image is obtained by rendering the target graphic elements based on the vector graphic marking results. The method compares the initial question image and the rendered question image to extract target graphic elements. Based on the answer text, consistency verification is performed on the target graphic elements to obtain a verification result. Based on the verification result, the rendered question image is adjusted to obtain the image output result. This method can accurately acquire the rendered target graphic elements and perform automated consistency verification based on them. It proactively and efficiently detects errors in the target graphic elements and adjusts the rendered question image based on the verification result, filling the gap in existing automated drawing systems that lack a drawing quality control mechanism and improving the accuracy and reliability of automated drawing. Attached Figure Description

[0011] Figure 1 A flowchart illustrating an image correction method provided in one embodiment of this specification; Figure 2 This diagram illustrates an embodiment of the multimodal problem-solving method provided in this specification, applied to the correction of bar chart completion problems. Figure 3 A flowchart illustrating the processing steps of an image correction method provided in one embodiment of this specification; Figure 4 This is a schematic diagram of the structure of an image correction device provided in one embodiment of this specification; Figure 5 This is a structural block diagram of a computing device provided for one embodiment of this specification. Detailed Implementation

[0012] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0013] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items. The term “at least one” in one or more embodiments of this application means “one or more,” and “a plurality of” means “two or more.” The term “comprising” is an open-ended description and should be understood as “including but not limiting,” and may include other content in addition to what has been described.

[0014] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0015] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0016] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0017] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0018] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0019] Multimodal: Multimodal refers to data from multiple different sources or formats. These data sources can include text, images, audio, video, and even more complex data types. In machine learning, "multimodal learning" refers to models being designed and trained to simultaneously process and utilize these input data from different types of data.

[0020] Scalable Vector Graphics (SVG) is a vector graphics format that can be scaled without loss and supports the drawing of various graphic elements, including rectangles, circles, lines, and text.

[0021] Graphing problems: These are math problems that require drawing to solve or display the results, including but not limited to bar chart completion (e.g., completing statistical charts based on data, such as completing missing bars), plotting points and connecting lines in a coordinate system (e.g., plotting points or connecting lines in a Cartesian coordinate system), connecting lines (e.g., connecting multiple points to form a specific shape), circling problems (e.g., circling a specific area or object in an image), drawing geometric figures (e.g., drawing various geometric figures, such as triangles, quadrilaterals, etc.), and statistical charts (e.g., generating various statistical charts, such as pie charts, line charts, etc.).

[0022] Region of Interest (ROI): refers to a specific area in an image that needs to be processed or drawn.

[0023] Embedded computing devices: Computer systems designed to perform one or more specific tasks. Compared to personal mainframes, they are less powerful, but have significant advantages in terms of size and power consumption. They are commonly used in various electronic and mechanical control systems.

[0024] This specification provides an image correction method, and also relates to an image correction method system, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail in the following embodiments.

[0025] It is worth noting that the image correction method provided in the embodiments of this specification is generally executed by the server. However, in other embodiments of this specification, the client may also have similar functions to the server, thereby executing the image correction method provided in the embodiments of this specification. In other embodiments, the image correction method provided in the embodiments of this specification may also be executed jointly by the client and the server.

[0026] While existing multimodal problem-solving systems based on SVG drawing can visualize drawing problems, a key issue exists: the SVG code generated by the model may have drawing offsets, leading to inaccurate drawing results.

[0027] Specific issues include: (1) Coordinate offset: The starting point or key position of the drawing deviates from the expected coordinates, such as the starting point of the bar chart not being at 0; (2) Size deviation: The size of the drawn figure does not meet the requirements of the question, such as the height of the bar chart exceeding or falling short of the expected value; (3) Drawing error: Although students can understand the meaning of the question, the inaccuracy of the drawing result will affect the completeness and professionalism of the solution to the question; (4) Lack of correction mechanism: The existing system cannot automatically detect and correct drawing errors, requiring manual intervention.

[0028] These issues make the explanations, answers, and drawing results of the drawing questions less than perfect, affecting the teaching effectiveness and user experience.

[0029] See Figure 1 , Figure 1 A flowchart of an image correction method provided in one embodiment of this specification includes the following steps: Step 102: Obtain the initial question graphic, vector graphic marking result, answer text, and rendered question graphic. The vector graphic marking result is obtained by performing vector graphic marking on the target graphic elements in the initial question graphic based on the answer text, and the rendered question graphic is obtained by rendering the target graphic elements based on the vector graphic marking result.

[0030] The question graph is a graphical representation of the question within the input image, used to provide supplementary descriptions of the question's content, offer key information, or illustrate a solution approach. For example, the question graph can be a geometric figure (such as triangles, circles, or other polygons), a chart or table (such as a bar chart, line graph, or statistical table), a function graph (such as a quadratic function curve or trigonometric function waveform), a logic diagram (such as a flowchart or mind map), etc. This specification does not limit the types of graphs used in the embodiments.

[0031] The initial question graphic is the original graphic representation of the input image corresponding to the question text, used to provide the visual context and initial state of the question. For example, for a bar chart completion question, the initial question graphic might be a statistical chart missing some bars; for a geometric figure drawing question, the initial question graphic might be a geometric sketch with partial dimensions marked.

[0032] For example, a user takes a picture of a bar chart completion question in a math exam paper to obtain an input image. The input image includes the question text "Complete the bar chart based on the data in the table below" and an initial bar chart containing only coordinate axes and bars from 2015 to 2017. The horizontal axis is labeled with the year, the vertical axis is labeled with the sales amount (ten thousand yuan), and the missing part is marked with a dashed box to indicate the area to be completed.

[0033] For example, suppose there is a math statistics problem that requires students to calculate the height of a missing bar chart and complete the bar chart. The initial problem graph could be a graph with a missing part of the bar chart.

[0034] Accordingly, the question text is the text data in the input image that describes the question, used to express the problem to be solved, to provide auxiliary descriptions of the problem's content, or to provide a solution approach, etc. For example, when the question type of the graph problem is to complete a bar chart, the question text is "There are 45 people with scores greater than 60 and less than 75. Complete the bar chart"; when the question type of the graph problem is to draw a geometric figure, the question text is "In the given triangle ABC, construct the angle bisector of ∠A". The question text usually includes text, numbers, and symbols, and appears in a combination of text, numbers, and symbols. In some cases, it may also include codes or encodings and other semantic content, which are not limited in this embodiment of the specification.

[0035] Continuing with the above example, the question text could also provide data from an existing bar chart and ask students to calculate the height of the missing bars and complete the bar chart.

[0036] Vector graphic annotation results in a structured graphic output generated after vector graphic annotation of the initial question graphic according to the question requirements. Specifically, it can be generated by determining the content to be annotated in the initial question graphic based on the operational requirements proposed by the question text. Vector graphic annotation can be performed using text, XML text, etc., and vector graphics can highlight the position, name, or attributes of the target graphic elements.

[0037] Vector graphic markup results can be calculated based on the width and height of the target bar chart for filling in missing bars and the initial title graphic, and are used to describe the vector graphic markup content of the target bar chart. For example, SVG code generated based on the SVG standard.

[0038] The answer text is a complete description of the solution steps and final result generated after the reasoning process is completed. It can be generated by parsing and solving the initial problem image and corresponding problem text. It can be used as the answer to the problem to directly solve it, or as the parsing of the problem to further instruct on how to perform vector graphic labeling and generate the vector graphic labeling result.

[0039] refer to Figure 2 , Figure 2 This is a schematic diagram illustrating an embodiment of a multimodal problem-solving method provided in this specification, applied to the correction of bar chart completion problems. Figure 2 As shown: The answer text could be: "There are 45 people with scores greater than 60 and less than 75. The coordinates of the bottom left corner of the bar chart are x="0.721" y="0.215", width: width="0.042", height: height="0.181".

[0040] The rendered question graphic represents the final visual solution, containing a complete representation of both the original question graphic and the target graphic elements. For example, a complete statistical chart with completed bars, data labels, and proportion explanations. In one possible implementation, the rendered question graphic is a graphic obtained by rendering vector graphics markings on the initial question graphic. The rendered question graphic is used to answer the question, serving as part or all of the answer; or to highlight key content on the initial question graphic to emphasize important information and facilitate question processing.

[0041] The target graphic element can be a vector graphic rendered in the initial question graphic. Continuing the previous example, the bottom left corner has coordinates of x="0.720" y="0.192", the bottom right corner has coordinates of x="0.761" y="0.192", the top left corner has coordinates of x="0.720" y="0.377", and the top right corner has coordinates of x="0.761" y="0.377".

[0042] Rendering target graphic elements onto an initial title graphic is a technical process of blending vector graphic annotations with the initial title graphic for display. For example, an SVG annotation layer is overlaid on the original bitmap using a graphics processing engine, employing appropriate blending modes and transparency settings.

[0043] The process of obtaining the initial question image, vector graphic labeling results, answer text, and rendered question image can be implemented in several ways. One possible implementation is to obtain the initial question image, vector graphic labeling results, answer text, and rendered question image output by the multimodal problem-solving module. Another possible implementation is to obtain the initial question image, vector graphic labeling results, answer text, and rendered question image stored in the sample dataset. Yet another possible implementation is to obtain the generated initial question image, vector graphic labeling results, answer text, and rendered question image from the network.

[0044] Step 104: Compare the initial question image and the rendered question image to extract the target image elements.

[0045] The target graphic elements are vector graphics rendered within the initial question's graphic. These graphic objects can be geometric shapes (such as circles and rectangles), lines, paths, or more complex combinations. For questions requiring modification or supplementation of the graphics (such as completing a bar chart or drawing a geometric figure), the target graphic elements can provide specific modification guidelines or directly provide an answer to the question. For example, in a bar chart completion question, by determining the specific location and size of the missing bars, it guides how to fill in these gaps.

[0046] To extract target graphic elements by comparing the initial and rendered question images, one possible approach is to use a large language model to compare the initial and rendered question images. Another possible approach is to perform feature point matching on the initial and rendered question images, and based on the feature point matching results, call a visual model to compare the initial and rendered question images to extract target graphic elements. Yet another possible approach is to perform vector graphics parsing on the initial and rendered question images to generate initial and rendered question vector graphics, and then use a vector graphics comparison algorithm to compare the initial and rendered question vector graphics to extract target graphic elements.

[0047] Continuing from the above Figure 2 For example, comparing the initial question image and the rendered question image, the target graphic elements are extracted. An example of the initial question image could be a bar chart with a width of 0.042 and a height of 'a' displayed for scores greater than 45 and less than 60, and a bar chart with a width of 0.042 and a height of 'b' displayed for scores greater than 75 and less than 90, with a 0.042-width space between the two bar charts. The rendered question image could be a bar chart with the following coordinates within the two bar charts: bottom left corner (x="0.720" y="0.192"), bottom right corner (x="0.761" y="0.192"), top left corner (x="0.720" y="0.377"), and top right corner (x="0.761" y="0.377"). The comparison can be between the two graphs mentioned above. By comparing them, the target graphic element can be extracted from the rendered question graph as a bar graph with the coordinates of the lower left corner point x="0.720" y="0.192", the lower right corner point x="0.761" y="0.192", the upper left corner point x="0.720" y="0.377", and the upper right corner point x="0.761" y="0.377".

[0048] Vector graphics parsing is the process of parsing an image using vector graphic tags to generate a vector graphic file (e.g., using SVG parsing to generate an XML-formatted SVG file corresponding to the image).

[0049] Extracting target graphic elements allows for precise consistency verification of each element, ensuring that its position, size, shape, and other attributes meet expectations. Identifying and locating specific graphic elements helps to quickly discover and correct any deviations or errors, improving the overall drawing quality.

[0050] Step 106: Based on the answer text, perform consistency verification on the target graphic elements and obtain the verification results.

[0051] Consistency verification involves comparing and validating the target graphic elements. It can include verification of attributes, positions, dimensions, and other dimensions to confirm whether these graphic elements accurately reflect the requirements of the problem and the solution logic, and to identify any drawing errors such as coordinate offsets, size deviations, incomplete drawings, or inconsistent formats.

[0052] There are several ways to perform consistency verification on target graphic elements based on the answer text. One possible approach is to parse the answer text using rules to obtain the corresponding structured text data, and then perform consistency verification on the target graphic elements based on this structured text data. Another possible approach is to perform vector graphics parsing on the target graphic elements to obtain their vector representations, and then perform consistency verification on these vector representations based on the answer text to obtain the verification result. Yet another possible approach is to perform rule parsing on the answer text to obtain the corresponding structured text data, and then perform vector graphics parsing on the target graphic elements to obtain their vector representations, and finally perform consistency verification on these vector representations based on the structured text data to obtain the verification result.

[0053] Continuing from the above Figure 2 For example, the answer text is "Draw a bar chart with a width of 0.042 and a height of 0.181 between the coordinates of the bars for scores greater than 45 and less than 60 and scores greater than 75 and less than 90". The target graphic element is "a bar chart with coordinates of the bottom left corner x="0.721" y="0.215", the bottom right corner x="0.763" y="0.215", the top left corner x="0.721" y="0.396", and the top right corner x="0.763" y="0.396". Verification shows that the two are consistent.

[0054] By comparing the theoretical explanation in the answer text with the actual rendered target graphic elements, the system can automatically identify whether the target graphic elements are consistent with the answer text, thereby obtaining a verification result of whether the target graphic elements are drawn correctly. Based on the verification result, it can determine whether the target graphic elements need to be corrected, thus building an automated question-answering system with error perception and feedback correction capabilities.

[0055] Step 108: Based on the verification results, adjust the rendered question graphic to obtain the image output result.

[0056] The image output is a visual graphic data that meets the requirements of the question after adjusting the rendered question graphic. The image output is characterized by high accuracy and reliability. It is the final product of the automated drawing system after a full process of "understanding-generation-verification-correction", and is used to present the solution process or answer of the question completely and correctly.

[0057] There are several ways to adjust the rendered question graphic based on the verification results to obtain the image output result. One possible implementation is to render the target question graphic in the initial question graphic based on the verification results to obtain the target vector graphic marking result.

[0058] Another possible implementation is to adjust the vector graphic marking results based on the verification results, and then render the target question graphic in the initial question graphic based on the vector graphic marking results to obtain the target vector graphic marking results.

[0059] Another possible implementation is to adjust the rendered question graphics based on the verification results using a large language model to obtain the image output result.

[0060] Continuing from the above Figure 2 For example, the image output can be a bar chart icon that includes multiple bar bars, where the rendered image is a bar chart with the coordinates of the bottom left corner being x="0.721" y="0.215", the bottom right corner being x="0.763" y="0.215", the top left corner being x="0.721" y="0.396", and the top right corner being x="0.763" y="0.396".

[0061] In one optional embodiment of this specification, the verification result includes a consistency result and an error type. Step 108 includes: If the consistency result is inconsistent, perform error detection on the target graphic elements to determine the error type; Based on the error type, the rendered question graphic is adjusted to obtain the image output result.

[0062] The verification result is obtained by performing a consistency verification on the target graphic elements, indicating the judgment result of whether the answer text and the rendered graphic match at the semantic and geometric levels. The value can be "consistent" or "inconsistent". Consistent means that the answer text and the target graphic elements are consistent, indicating that the rendered question graphic is correct and no additional correction is needed. Inconsistent means that there may be errors in the rendered question graphic and adjustments are required.

[0063] Error detection is the process of locating deviations or anomalies in target graphic elements by comparing the expected logic with the actual rendering result. Error detection can be a comprehensive analysis based on at least one of the vector graphic markings in the answer text and the rendered question graphic. Using the answer text as the correct standard, and combining graphic comparison and consistency verification mechanisms, error detection determines which of several error types exists in the rendered question graphic, such as coordinate offset, size error, missing element, or logical inconsistency. Based on this, it identifies the error type of the target graphic element, providing a basis for subsequent graphic adjustments and quality optimization.

[0064] Error types refer to various deviations or anomalies that may occur during the graphic generation, rendering, and verification process. Error types include at least one of the following: coordinate offset, size deviation, drawing integrity, and format consistency. Specifically, the handling methods differ depending on the error type. For example, for coordinate offset errors, the deviation of the target graphic element from its expected position is identified; for size deviation errors, the difference between the size of the target graphic element and the requirements of the question is identified; drawing integrity checks identify any missing parts among all elements that need to be drawn; and format consistency checks ensure that the format and content of the vector graphic markup results conform to specifications. In the case of SVG code, this can involve checking whether the SVG code format conforms to specifications and identifying any errors.

[0065] There are several ways to perform error detection on target graphic elements and determine the error type. One possibility is to perform error detection on the target graphic elements based on the initial question graphic and / or answer text to determine the error type. Another possibility is to perform error detection on the rendered question graphic to determine the error range, and then perform error detection on the target graphic elements based on the error range to determine the error type.

[0066] Continuing from the above Figure 2 For example, for a bar chart with the coordinates of the bottom left corner point as x="0.720" y="0.192", the bottom right corner point as x="0.761" y="0.192", the top left corner point as x="0.720" y="0.377", and the top right corner point as x="0.761" y="0.377", the error type can be determined as coordinate offset.

[0067] In the embodiments of this specification, when the consistency result is inconsistent, error detection is performed on the target graphic element to determine the error type. Based on the error type, the rendered question graphic is adjusted to obtain the image output result. First, the error type is determined, and then the rendered question graphic is adjusted based on the error type. According to different error types, corresponding adjustment strategies are adopted to avoid indiscriminate or guesswork adjustments to the entire graphic, thereby improving the accuracy and efficiency of error correction.

[0068] In one optional embodiment of this specification, the rendered question graphic is adjusted based on the error type to obtain the image output result, including: Based on the error type, perform error analysis on the rendered question graphics to obtain the error analysis content; Based on the error analysis, the rendered question graphics are adjusted to obtain the image output result.

[0069] Error analysis is the process of further analyzing the specific manifestations of an error based on its type. This process includes, but is not limited to: locating the specific position of the error (e.g., coordinates, layers, connectors, etc.); analyzing the deviation between the error and the expected specifications (e.g., 5° angular deviation, a coordinate deviation of a value, etc.); determining whether the error triggers a chain reaction of errors (e.g., one node offset causing multiple line offsets); and deriving adjustment parameters or rules that can be used for graphic correction, etc. Error analysis is used to accurately locate the error position and cause, thereby clarifying the method for correcting the error.

[0070] Error analysis content is a set of descriptive information generated based on error analysis of target graphic elements. This content is the result of in-depth analysis of the specific manifestations, spatial location, deviation parameters, scope of impact, and executable correction rules of the error, based on the already determined error type. The error analysis content includes the specific error location, error content, and / or suggested modifications.

[0071] An example of error analysis could be that, if the error type is determined to be coordinate deviation, the error analysis would be performed on a bar graph showing the deviation of the target graphic element in the rendered question graphic from its expected position. The bar graph would have coordinates x="0.720" y="0.192", x="0.761" y="0.192", x="0.720" y="0.377" for the bottom left corner, and x="0.761" y="0.377" for the top left corner. The error analysis would then determine that the top left corner is shifted upwards, and the bottom right corner is shifted upwards.

[0072] There are several ways to perform error analysis on the rendered question image based on error type to obtain the error analysis content. One possible approach is to perform error analysis on the rendered question image based on both the error type and the answer text. Another possible approach is to parse the rendered question image using a pre-trained graph neural network to obtain the corresponding text representation, and then perform error analysis on the text representation based on the error type to obtain the error analysis content. Yet another possible approach is to perform error analysis on the target graphic elements within the rendered question image based on the error type to obtain the error analysis content.

[0073] This specification's embodiments perform error analysis on the rendered question graphics based on error type to obtain error analysis content; based on the error analysis content, the rendered question graphics are adjusted to obtain the image output result. Analyzing the rendered question graphics based on error type is more targeted and can obtain accurate error analysis content for each type. Adjustments based on the error analysis content significantly improve the quality and accuracy of the image output result.

[0074] In one optional embodiment of this specification, the rendered question graphic is adjusted based on the error analysis content to obtain the image output result, including: Based on the error analysis, the vector graphic marking results are adjusted to obtain the target vector graphic marking results; Based on the target vector graphic marking results, the target graphic elements are rendered to obtain the image output results.

[0075] The target vector graphics labeling result is structured data obtained after error correction. The target vector graphics labeling result is obtained by adjusting the errors in the vector graphics labeling result based on error analysis, therefore, the target vector graphics labeling result contains structured data identical to the answer expressed in the answer text. Rendering the target graphic elements based on the target vector graphics labeling result can obtain the correct image output result.

[0076] Rendering is the process of converting abstract, structured data models (such as vector graphics, 3D models, scene descriptions, etc.) into visual image outputs (usually bitmap images, such as PNG, JPG, etc.). In the embodiments of this specification, rendering can be the process of rendering SVG data to obtain visual image outputs.

[0077] There are several ways to adjust the vector graphics labeling results based on error analysis to obtain the target vector graphics labeling result. One possible approach is to determine a correction strategy based on the error analysis content, and then adjust the vector graphics labeling results based on the correction strategy to obtain the target vector graphics labeling result. Another possible approach is to generate vector graphics error variables based on the error analysis content, and then adjust the vector graphics labeling results based on these error variables to obtain the target vector graphics labeling result. Yet another possible approach is to call a large language model and adjust the vector graphics labeling results based on the error analysis content to obtain the target vector graphics labeling result.

[0078] Continuing from the above Figure 2 For example, the existing vector graphics markup results are as follows: <ai-svg roi="0.598,0.025 1.000,0.463"> <rect id="1" x="0.720" y="0.192" width="0.041" height="0.185" caption="分数大于等于60分且小于70分的有45人" / > < / ai-svg> ; The error analysis revealed a coordinate offset; the top-left corner of the erroneous graphic was shifted upwards relative to the correct graphic. Based on this error analysis, the vector graphic markings were adjusted to obtain the correct target vector graphic markings: <ai-svg roi="0.598,0.028 1.000,0.463"> <rect id="1" x="0.721" y="0.215" width="0.042" height="0.181" caption="分数大于等于60分且小于70分的有45人" / > < / ai-svg> .

[0079] This specification's embodiments adjust the vector graphic marking results based on error analysis to obtain target vector graphic marking results; based on the target vector graphic marking results, the target graphic elements are rendered to obtain image output results. Adjusting the vector graphic marking results used to render the target graphic elements ensures the correctness of the vector graphic markings. Simultaneously, the image output results obtained based on the vector graphic markings are used to output correct image output results, thereby achieving automated correction of the problem-solving tool's output content.

[0080] In one optional embodiment of this specification, after adjusting the vector graphic marking results based on the error analysis content to obtain the target vector graphic marking results, the method further includes: Integrate error analysis content and target vector graphic markers to obtain text output results.

[0081] The text output is a comprehensive document or data structure that integrates error analysis content and target vector graphic markers. The text output comprehensively reflects the error analysis and processing of the rendered problem graphics, providing a clear and detailed solution path. It is used to output visual documentation to users, improving the interpretability of the correction process.

[0082] There are several ways to integrate error analysis content and target vector graphic markers to obtain text output results. One possible method is to integrate the answer text, render the question graphics, error analysis content, and target vector graphic markers to obtain text output results.

[0083] Continuing from the above Figure 2 For example, the error analysis content is: coordinate offset; the top-left corner of the erroneous graphic is shifted upwards relative to the correct graphic. The target vector graphic is labeled as: <ai-svg roi="0.598,0.028 1.000,0.463"> <rect id="1" x="0.721" y="0.215" width="0.042" height="0.181" caption="分数大于等于60分且小于70分的有45人" / > < / ai-svg> By integrating the error analysis content and the target vector graphic markers, the text output result is obtained: [Drawing Error Analysis] rect id="1" Drawing command="45 people scored between 60 and 70 points" The top left corner point is shifted upwards; the bottom right corner point is shifted upwards.

[0084] [SVG Correction Results] <ai-svg roi="0.598,0.028 1.000,0.463"> <rect id="1" x="0.721" y="0.215" width="0.042" height="0.181" caption="分数大于等于60分且小于70分的有45人" / > < / ai-svg> ".

[0085] This specification's embodiments integrate error analysis content and target vector graphic markers to obtain text output results. This generates a complete description of the error resolution process, facilitating the analysis and recording of the error analysis process, enhancing its interpretability, and contributing to improved efficiency in automated error correction.

[0086] In one optional embodiment of this specification, the error type is at least one of coordinate offset, dimension deviation, drawing integrity, and format consistency. Error analysis is performed on the rendered title graphic based on the error type to obtain the error analysis content, including: Based on the error type, the target analysis rule is determined from the preset error type analysis rules; Error analysis is performed on the rendered question graphics based on the target analysis rules to obtain the error analysis content.

[0087] Error type analysis rules are templated rules for the systematic analysis of errors. Through predefined, structured logic and standards, error type analysis rules classify, locate, attribute, and quantify errors, thus providing a basis for subsequent correction, optimization, and automation. Error type analysis rules are related to error types; different error types correspond to different error type analysis rules.

[0088] For example, for coordinate offset type errors, the error analysis rule is to identify the deviation of the target graphic element from the expected position; for size deviation type errors, the error analysis rule is to identify the difference between the size of the target graphic element and the requirements of the question; for drawing integrity detection, the error analysis rule is to identify the missing parts among all elements that need to be drawn; for format consistency detection, the error analysis rule is to detect errors in the format and content of the vector graphic markup results. In the case of vector graphic markup results being SVG code, it can be to detect whether the SVG code format conforms to the specification and to detect erroneous content.

[0089] Error analysis of the rendered question graphic based on target analysis rules can be performed in several ways to obtain error analysis content. One possible approach is to extract target graphic elements from the rendered question graphic, and then perform error analysis on these target graphic elements based on target analysis rules to obtain error analysis content. Another possible approach is to perform error analysis on the vector graphic markup results based on target analysis rules to obtain vector graphic error analysis content, and then perform error analysis on the rendered question graphic based on this vector graphic error analysis content to obtain the final error analysis content.

[0090] Continuing from the above Figure 2 For example, the target graphic element extracted from the rendered question graphic is a bar chart with the following coordinates: bottom left corner x="0.720" y="0.192", bottom right corner x="0.761" y="0.192", top left corner x="0.720" y="0.377", top right corner x="0.761" y="0.377". The answer text indicates that the graph is plotted based on the bottom left corner coordinates x="0.721" y="0.215", width: 0.042, height: 0.181. The error type is determined to be coordinate offset. At this point, the target analysis rule is determined to be to identify the deviation of the target graphic element from the expected position. Therefore, the analysis of the deviation of the target graphic element from the expected position is performed on the target graphic element.

[0091] This specification's embodiments are based on error types, determining target analysis rules from preset error type analysis rules; based on the target analysis rules, error analysis is performed on the rendered question graphics to obtain error analysis content. By determining target analysis rules for corresponding error types based on error types, and performing targeted diagnosis based on these rules, it possesses high error location specificity and analysis guidance, accurately and efficiently identifying specific errors in rendered graphics, providing a reliable basis for subsequent automatic correction and quality improvement.

[0092] In one optional embodiment of this specification, the above method further includes: If the consistency result is consistent, the rendered question graphic is determined to be the image output result.

[0093] If the consistency result is consistent, it means that there are no errors in rendering the question image. At this point, it can be determined that no correction is needed for the rendered question image. The rendered question image can be set as the image output result and output.

[0094] In the embodiments of this specification, when the consistency result is consistent, the rendered question graphic is determined to be the image output result. This avoids unnecessary detection, correction or re-rendering of the graphic that already meets the expectations, thereby reducing computational overhead, improving processing efficiency, and preventing the introduction of potential distortion or error due to excessive intervention.

[0095] In one optional embodiment of this description, the target graphic elements are extracted by comparing the initial question graphic and the rendered question graphic, including: Using a large language model, the initial question image and the rendered question image are compared to extract the target image elements; Based on the answer text, consistency verification is performed on the target graphic elements to obtain the verification results, including: Using a large language model, based on the answer text, consistency verification is performed on the target graphic elements to obtain the verification results; Based on the verification results, the rendered question image is adjusted to obtain the image output results, including: Using a large language model, the rendered question graphics are adjusted based on the verification results to obtain the image output.

[0096] The Large Language Model (MLM) is a pre-trained natural language processing model with both natural language understanding and visual understanding capabilities. Based on deep learning technology and trained on massive amounts of text, the MLM can understand the grammatical structure, contextual relationships, and deep semantics of natural language. It can deeply understand natural language and attempt to generate coherent, context-relevant answers based on input content, logical reasoning, and domain context. In this specification, the MLM has the ability to process text and image features, and can perform reasoning based on these features to determine the question analysis results and target graphic elements, outputting the answer text. With its powerful semantic understanding and reasoning capabilities, the MLM can accurately parse question text and extract target graphic elements and deep analysis results. Compared to traditional methods, it is more intelligent, flexible, and robust, significantly improving the automation level and user experience of the text-image understanding system.

[0097] In practical applications, the above steps can be performed based on the prompt words and a large language model. Specifically, comparison prompt words are constructed based on the initial and rendered question images, and then input into the large language model. The large language model compares the initial and rendered question images to extract target graphic elements. Verification prompt words are constructed based on the answer text and target graphic elements, and then input into the large language model. The large language model performs consistency verification on the target graphic elements based on the answer text to obtain the verification results. Adjustment prompt words are constructed based on the verification results, and then input into the large language model. The large language model adjusts the rendered question image based on the verification results to obtain the image output result.

[0098] This embodiment guides a large language model through prompts to sequentially compare the initial and rendered problem images, verify the consistency between the answer text and target graphic elements, and adjust the rendered problem image based on the verification results, thus achieving automated correction of the solution. By fully leveraging the advantages of the large language model in semantic parsing, structured generation, and logical reasoning, the automatic correction method improves the accuracy of the image output and the interpretability of the problem-solving reasoning, significantly enhancing the automated processing capability and reliability of answer generation in complex visual-language tasks for intelligent problem-solving systems.

[0099] The following is in conjunction with the appendix Figure 3 Taking the image correction method provided in this specification applied to the correction of a bar chart completion question as an example, the image correction method will be further explained. Among other things, Figure 3 A flowchart illustrating the processing steps of an image correction method provided in one embodiment of this specification includes the following steps: Step 302: Obtain the bar chart completion question image, wherein the bar chart completion question image includes the bar chart completion question text and the initial bar chart.

[0100] This step aims to provide structured input data for the multimodal problem-solving process. Specifically, a digital image containing a bar chart completion problem is acquired via an image acquisition device or storage system. This image must simultaneously contain a text area describing the problem requirements and an initial bar chart area serving as the basis for solving the problem. For example, a photo of a math problem taken by a user might contain the text "Complete the sales bar chart for 2018-2020 based on the data in the table below" and an initial bar chart containing only data from 2015-2017.

[0101] Step 304: Perform an image size check on the image for completing the bar chart; and / or, perform an image format check on the image for completing the bar chart.

[0102] This step focuses on controlling the quality of the bar chart completion problem image. By checking the resolution and aspect ratio of the image, we ensure the clarity of subsequent processing. We also verify image format compatibility to guarantee correct parsing. For example, we check if the image resolution meets the minimum requirement of 800×600 pixels and confirm that the image format is a supported format such as JPEG, PNG, or BMP.

[0103] Step 306: Analyze the completed question text in the bar chart according to the preset question type rules to obtain the question analysis results; based on the initial bar chart, perform graphic element matching on the question analysis results to determine the target graphic elements.

[0104] This step transforms the process from understanding the question to extracting key elements. Natural language processing (NLP) is used to analyze the question requirements, identify the data categories and corresponding relationships that need to be completed, and, combined with the existing structure of the initial bar chart, locate the specific graphic components that need to be added or modified. For example, the analysis identifies that data bars for the three years 2018-2020 need to be completed, and based on the coordinate system of the initial bar chart, the target graphic elements are determined to be three rectangular bars at specific locations.

[0105] Step 308: Based on the target graphic elements, analyze the results of the question analysis to generate a reasoning chain; execute the reasoning chain to obtain the reasoning chain output and the answer text.

[0106] This step completes the derivation process from graphic elements to the answer. By constructing a logically rigorous reasoning process and clarifying the order of data calculation or graphic completion, it ultimately generates text output containing a detailed reasoning process and a clear conclusion. For example, the generated reasoning chain is: "1. Analyze the existing data growth trend; 2. Calculate the average annual growth rate; 3. Predict the values ​​for 2018-2020; 4. Determine the height of the rectangular column." After execution, the answer text is obtained: "2018: 120 units, 2019: 135 units, 2020: 150 units."

[0107] Step 310: Based on the answer text, determine the marking parameters of the target graphic element; using the preset code rules for vector graphic elements, generate codes for the target graphic elements in the initial bar chart based on the marking parameters of the target graphic element to obtain vector graphic marking codes.

[0108] This step converts the text answer into graphic markers. Based on the answer, the visual characteristic parameters of the graphic elements to be highlighted are determined, and corresponding drawing instruction codes are generated according to vector graphics standards, providing a foundation for subsequent visualization rendering. For example, the coordinates and dimensions of three rectangular pillars are determined based on the values ​​in the answer text, generating corresponding SVG drawing code. The information in the drawing code can include the position, size, color, and label text of the rectangular pillars.

[0109] Step 312: Execute the vector graphic markup code to overlay and render the target graphic element on the initial bar chart to obtain a rendered enhanced image; from the rendered enhanced image, crop out the image area where the target graphic element is located to obtain the rendered title graphic.

[0110] This step completes the visualization and optimized output of the vector graphic markup code. By executing the vector graphic markup code, the completed graphic elements are precisely overlaid and rendered onto the initial bar chart, and key areas are extracted to highlight the important parts. For example, by executing the generated SVG code, highlighted rectangular bars are added to the initial bar chart, and the core area containing the completed rectangular bars and their annotations is intelligently extracted to form the final rendered title graphic.

[0111] Step 314: Obtain the initial bar chart, vector graphic marking results, answer text, and rendered question graphic. The vector graphic marking results are obtained by performing vector graphic marking on the target graphic elements in the initial bar chart based on the answer text. The rendered question graphic is obtained by rendering the target graphic elements based on the vector graphic marking results.

[0112] The initial question graphic is the original graphic representation of the input image corresponding to the question text, used to provide the visual context and initial state of the question. For example, for a bar chart completion question, it might be a statistical chart with missing bars.

[0113] The vector graphic markup result is a structured graphic output generated after the initial question graphic is marked with vector graphics according to the question requirements. It can be based on SVG code generated by the SVG standard, used to describe the position, name, or attributes of the target graphic elements.

[0114] The answer text is a complete description of the problem-solving steps and the final result generated after the reasoning process is completed. It is used to guide how to perform vector graphic labeling and generate vector graphic labeling results.

[0115] The rendered question graphic is a graphic obtained by rendering the target graphic elements based on the vector graphic markup results. It is used to answer the question or to highlight key content on the initial question graphic. By comparing the initial bar chart and the rendered question graphic, the target graphic elements in the rendered question graphic are determined based on the differences between the initial bar chart and the rendered question graphic. The target graphic elements are the graphic elements that have been rendered in the rendered question graphic.

[0116] In this step, we first obtain the initial bar chart, vector graphic markers, answer text, and rendered question graphic as a foundation. This step aims to determine the rendered question graphic used for correction, the initial bar chart used for comparison to obtain the target graphic elements, the answer text used to determine if the target graphic elements are consistent (whether errors have occurred), and the vector graphic markers used for correction in sync with the rendered question graphic.

[0117] The question text is: Complete the sales bar chart for 2018-2020 based on the data in the table below. The initial bar chart only contains data from 2015-2017. The vector graphics markup result is generated by determining the coordinates and dimensions of three rectangular bars based on the values ​​in the answer text, and generating the corresponding SVG drawing code. The answer text is the text description of the corresponding SVG drawing code, such as "the position, width, and height of the sales bar chart for 2018, the position, width, and height of the sales bar chart for 2019, and the position, width, and height of the sales bar chart for 2020".

[0118] By acquiring this information, the system can provide the necessary data support for subsequent comparisons and verifications. This step is fundamental to ensuring that subsequent steps can be executed accurately.

[0119] Step 316: Compare the initial bar chart and the rendered question graphic to extract the target graphic elements.

[0120] By comparing the initial bar chart with the rendered problem graph, we can identify the graphic elements that need to be adjusted or supplemented. This helps to identify deviations or errors in the drawing and prepares us for the next step of consistency verification.

[0121] Step 318: Based on the answer text, perform consistency verification on the target graphic elements and obtain the verification results.

[0122] Consistency verification is a verification process that includes multiple dimensions such as attributes, positions, and sizes, to confirm whether these graphic elements accurately reflect the requirements of the question and the solution logic, and whether there are any drawing errors (such as coordinate offsets, size deviations, etc.).

[0123] This step ensures consistency between the target graphic elements and the answer text. If inconsistencies are found, further investigation is needed to identify the specific error type in order to facilitate subsequent corrections.

[0124] Step 320: Based on the verification results, adjust the rendered question graphic to obtain the image output result.

[0125] Based on the verification results, the system will make necessary adjustments to the rendered problem image to generate a final image that meets the requirements of the problem and has no obvious errors. This approach improves the accuracy and reliability of the automated drawing system while also enhancing the user experience.

[0126] This specification's embodiments acquire an initial bar chart, vector graphic marking results, answer text, and a rendered question graphic. The vector graphic marking results are obtained by marking target graphic elements in the initial question graphic based on the answer text, and the rendered question graphic is obtained by rendering the target graphic elements based on the vector graphic marking results. The initial question graphic and the rendered question graphic are compared to extract the target graphic elements. Based on the answer text, the initial bar chart is validated for consistency, and the validation result is obtained. Based on the validation result, the rendered question graphic is adjusted to obtain the image output result. This method accurately acquires the rendered target graphic elements and performs automated consistency validation based on them. It proactively and efficiently detects errors in the target graphic elements and adjusts the rendered question graphic based on the validation result. This overcomes the lack of a quality control mechanism in existing automated drawing systems, improving the accuracy and reliability of automated drawing.

[0127] Corresponding to the above method embodiments, this specification also provides embodiments of image correction apparatus. Figure 4 This is a schematic diagram of an image correction device provided in one embodiment of this specification. Figure 4 As shown, the device includes: The acquisition module 402 is configured to acquire an initial question graphic, vector graphic marking results, answer text, and a rendered question graphic. The vector graphic marking results are obtained by performing vector graphic marking on the target graphic elements in the initial question graphic based on the answer text, and the rendered question graphic is obtained by rendering the target graphic elements based on the vector graphic marking results.

[0128] Extraction module 404 is configured to compare the initial question image and the rendered question image to extract the target image elements.

[0129] The verification module 406 is configured to perform consistency verification on the target graphic elements based on the answer text and obtain the verification result.

[0130] Adjustment module 408 is configured to adjust the rendered question graphic based on the verification result to obtain the image output result.

[0131] Optionally, the adjustment module 408 is further configured to perform error detection on the target graphic elements and determine the error type when the consistency result is inconsistent; and adjust the rendered question graphic based on the error type to obtain the image output result.

[0132] Optionally, the adjustment module 408 is further configured to perform error analysis on the rendered question graphic based on the error type to obtain the error analysis content; and to adjust the rendered question graphic based on the error analysis content to obtain the image output result.

[0133] Optionally, the adjustment module 408 is further configured to adjust the vector graphic marking results based on the error analysis content to obtain the target vector graphic marking results; and to render the target graphic elements based on the target vector graphic marking results to obtain the image output results.

[0134] Optionally, the image correction device also includes an integration module configured to integrate error analysis content and target vector graphic markers to obtain text output results.

[0135] Optionally, the adjustment module 408 is further configured to determine the target analysis rule based on the error type from the preset error type analysis rules; and to perform error analysis on the rendered question graphic based on the target analysis rule to obtain the error analysis content.

[0136] Optionally, the image correction apparatus further includes a determining module configured to determine the rendered title graphic as the image output result if the consistency result is consistent.

[0137] Optionally, the extraction module 404 is further configured to use a large language model to compare the initial question image and the rendered question image to extract the target image elements.

[0138] Optionally, the verification module 406 is further configured to use a large language model to perform consistency verification on the target graphic elements based on the answer text, and obtain the verification result.

[0139] Optionally, the adjustment module 408 is further configured to utilize a large language model to adjust the rendered question graphic based on the verification results, thereby obtaining the image output result.

[0140] This specification's embodiments acquire an initial question image, vector graphic marking results, answer text, and a rendered question image. The vector graphic marking results are obtained by marking target graphic elements in the initial question image using vector graphics based on the answer text. The rendered question image is obtained by rendering the target graphic elements based on the vector graphic marking results. The initial question image and the rendered question image are compared to extract the target graphic elements. Based on the answer text, consistency verification is performed on the target graphic elements to obtain the verification result. Based on the verification result, the rendered question image is adjusted to obtain the image output result. This method can accurately acquire the rendered target graphic elements and perform automated consistency verification based on them. It proactively and efficiently detects errors in the target graphic elements and adjusts the rendered question image based on the verification result. This overcomes the lack of a drawing quality control mechanism in existing automated drawing systems, improving the accuracy and reliability of automated drawing.

[0141] Figure 5This is a structural block diagram of a computing device provided according to one embodiment of this specification. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0142] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Networks (WLAN) interface, a Wi-MAX (World Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0143] In one embodiment of this specification, the above-described components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0144] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 500 can also be a mobile or stationary server.

[0145] The processor 520 is used to execute computer programs / instructions, which, when executed by the processor, implement the steps of the above-described image correction method.

[0146] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the image correction method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the image correction method described above.

[0147] An embodiment of this specification also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the image correction method described above.

[0148] The above is an illustrative embodiment of a computer-readable storage medium according to this invention. It should be noted that the technical solution of this storage medium and the technical solution of the image correction method described above belong to the same concept. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the image correction method described above.

[0149] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described image correction method.

[0150] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the image correction method described above belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the image correction method described above.

[0151] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0152] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0153] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0154] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0155] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An image correction method, characterized in that, include: The system acquires the question text, initial question image, vector graphic marking result, answer text, and rendered question image. The vector graphic marking result is obtained by performing vector graphic marking on the target graphic elements in the initial question image based on the answer text. The rendered question image is obtained by rendering the target graphic elements based on the vector graphic marking result. The initial question image is the original graphic representation of the question text in the input image, used to provide a visual context for the initial question. By comparing the initial question image and the rendered question image, the target graphic elements are extracted; Based on the answer text, the consistency of the target graphic elements is verified to obtain the verification result; Based on the verification results, the rendered question graphic is adjusted to obtain the image output result.

2. The method according to claim 1, characterized in that, The verification results include consistency results and error types; The process of adjusting the rendered question graphic based on the verification result to obtain the image output result includes: If the consistency result is inconsistent, error detection is performed on the target graphic element to determine the error type; Based on the error type, the rendered question graphic is adjusted to obtain the image output result.

3. The method according to claim 2, characterized in that, The process of adjusting the rendered question graphic based on the error type to obtain the image output result includes: Based on the error type, perform error analysis on the rendered question graphic to obtain the error analysis content; Based on the error analysis, the rendered title graphic is adjusted to obtain the image output result.

4. The method according to claim 3, characterized in that, The process of adjusting the rendered question graphic based on the error analysis to obtain the image output result includes: Based on the error analysis, the vector graphic marking results are adjusted to obtain the target vector graphic marking results; Based on the target vector graphic marking results, the target graphic elements are rendered to obtain the image output results.

5. The method according to claim 4, characterized in that, After adjusting the vector graphics marking results based on the error analysis content to obtain the target vector graphics marking results, the method further includes: By integrating the error analysis content and the target vector graphic markup, a text output result is obtained.

6. The method according to claim 2, characterized in that, The error type is at least one of coordinate offset, dimension deviation, drawing integrity, and format consistency. The error analysis performed on the rendered question graphic based on the error type yields the following error analysis content: Based on the error type, a target analysis rule is determined from the preset error type analysis rules; Based on the target analysis rules, error analysis is performed on the rendered question graphic to obtain the error analysis content.

7. The method according to claim 2, characterized in that, Also includes: If the consistency result is consistent, the rendered title graphic is determined to be the image output result.

8. The method according to any one of claims 1-7, characterized in that, The step of comparing the initial question image and the rendered question image to extract the target image elements includes: Using a large language model, the target graphic elements are extracted by comparing the initial question graphic with the rendered question graphic. The process of performing consistency verification on the target graphic elements based on the answer text to obtain the verification result includes: Using the large language model, based on the answer text, the consistency of the target graphic elements is verified to obtain the verification result; The process of adjusting the rendered question graphic based on the verification result to obtain the image output result includes: Using the large language model and based on the verification results, the rendered question graphic is adjusted to obtain the image output result.

9. An image correction system, characterized in that, Includes a content acquisition module and a reflection and correction module; The content acquisition module is used to acquire the question text, the initial question graphic, the vector graphic marking result, the answer text, and the rendered question graphic. The vector graphic marking result is obtained by performing vector graphic marking on the target graphic elements in the initial question graphic based on the answer text. The rendered question graphic is obtained by rendering the target graphic elements based on the vector graphic marking result. The initial question graphic is the original graphic representation of the question text in the input image, used to provide a visual context for the initial question. The reflection and correction module is used to compare the initial question image and the rendered question image to extract the target graphic elements; based on the answer text, to perform consistency verification on the target graphic elements to obtain the verification result; and based on the verification result, to adjust the rendered question image to obtain the image output result.

10. The system according to claim 9, characterized in that, It also includes a text summary module; The text summary module is used to integrate error detection results and target vector graphic markers to obtain text output results.

11. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, The device stores a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

13. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 8.

Images