An automatic error question collection method based on video image collection

Abstract

The application discloses an automatic wrong question collection method based on video image collection and belongs to the technical field of image processing, which specifically comprises the following steps: acquiring continuous video frames of homework or test papers through a video acquisition device, and extracting a question area image through pre-processing; performing text detection and recognition on the question area image, extracting question content and answer content, and constructing a data pair; matching the data pair with a standard answer library, identifying a wrong question area in combination with correction trace features; performing knowledge point labeling and difficulty grading on the question in the area, classifying and collecting based on the labeling result, generating a structured wrong question set, and outputting; and the application realizes automatic identification, classification and collection of wrong questions, improves wrong question arrangement efficiency, and is suitable for wrong question management scenes of various homework and test papers.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically a method for automatically collecting incorrect questions based on video image acquisition. Background Technology

[0002] In the educational process, compiling incorrect answers is an important means to help students identify and fill knowledge gaps and improve learning efficiency. Traditional methods of compiling incorrect answers mainly rely on manual organization, requiring students to manually copy down the wrong questions and record the reasons for the errors. This is not only time-consuming and laborious, but also prone to omissions and improper organization.

[0003] While some existing technologies offer automated error collection tools, most rely solely on scanning the question text and comparing it to the standard answer to identify errors, presenting significant limitations. Firstly, these tools struggle with deep semantic analysis of the question text, failing to accurately grasp the subject knowledge points and logical structure being tested. Secondly, they neglect the temporal characteristics of students' answering process, such as correction traces, pause durations, and answer order, which largely reflect students' thought processes and the root causes of errors. Furthermore, existing tools employ relatively crude error clustering methods, unable to accurately cluster errors based on their specific characteristics, nor effectively link errors to subject knowledge point maps or dynamically update label weights, thus failing to meet the needs of personalized teaching and precise tutoring.

[0004] For example, Chinese Patent No. CN115841670B discloses an image recognition-based homework error collection system, which includes a database module, a data acquisition module, a data processing module, and a data verification module. The system acquires image information uploaded by the user and identifies the text outlines corresponding to the answers in the images based on the image information. The text outlines are compared with outline patterns stored in the database to determine the text corresponding to the text outlines. When the overlap between the text outlines and outline patterns is low, the system considers the related text of adjacent text outlines. After adjusting the overlap comparison threshold, the system re-acquires the overlap comparison results between the text outlines and the outline patterns corresponding to the related text, thereby improving the recognition rate of the text corresponding to the text outlines.

[0005] The main shortcomings of existing technologies are: they rely on static image input, making it impossible to extract incorrect questions from continuous video in real time; the recognition accuracy of correction marks is insufficient, and they are easily affected by handwriting or layout; and they lack an automated knowledge point classification and difficulty leveling mechanism, resulting in low reuse value of the incorrect question set. Therefore, this application proposes an automatic incorrect question collection method based on video image acquisition to solve the above problems. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention proposes an automatic error collection method based on video image acquisition. The method involves acquiring continuous video frames from assignments or exam papers using a video acquisition device, preprocessing them to extract the question area image, performing text detection and recognition on the question area image to extract the question content and answer content, and constructing data pairs. These data pairs are then matched against a standard answer database, and error areas are identified by combining grading trace features. The questions in this area are then labeled with knowledge points and categorized by difficulty level. Based on the labeling results, the errors are classified and collected, generating a structured error collection and outputting it. This application achieves automatic identification, classification, and collection of error questions, improving the efficiency of error handling, and is applicable to error management scenarios for various assignments and exam papers.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] An automatic error collection method based on video image acquisition includes:

[0009] A series of video frames containing homework or test papers are acquired using a video capture device, and the video frames are preprocessed to obtain an image of the question area.

[0010] Perform text detection and recognition on the image of the question area to extract the question content and the corresponding answer content, and construct question-answer data pairs;

[0011] The question-answer data pairs are matched with a preset standard answer database, and the areas of incorrect questions are identified by combining the marking traces in the question area image.

[0012] The questions corresponding to the incorrect question areas are labeled with knowledge points and graded by difficulty. Based on the labeling results, they are classified and collected to generate a structured incorrect question set and output it.

[0013] Specifically, the video frames are preprocessed to obtain the title area image, including:

[0014] Inter-frame stabilization is performed on consecutive video frames, and image shift caused by shooting shake is eliminated by feature point matching to obtain a stable video frame sequence.

[0015] For the stable video frame sequence, an adaptive threshold segmentation algorithm is used to extract the paper region in the video frame to obtain the initial paper region image;

[0016] A perspective transformation correction is performed on the initial paper area image to obtain a rectangular page image with orthographic projection;

[0017] Based on the rectangular page image of the orthographic projection, the title segmentation line is located by edge detection and Hough transform, and independent title region images are segmented according to the title segmentation line.

[0018] Specifically, perspective transformation correction is performed on the initial paper area image to obtain a rectangular page image with orthographic projection, including:

[0019] Edge detection is performed on the initial paper area image to identify the coordinates of the four vertices of the paper area, and the spatial distortion parameters of the vertices are calculated based on the spatial positional relationship of the four vertices.

[0020] Based on the spatial distortion parameters, a perspective transformation matrix is ​​constructed. The perspective transformation matrix is ​​then used to map the tilted initial paper area image onto a two-dimensional frontal plane to obtain the transformed image.

[0021] The transformed image is then subjected to grayscale equalization to obtain a rectangular page image with orthographic projection.

[0022] Specifically, text detection and recognition are performed on the image of the question area to extract the question content and the corresponding answer content, including:

[0023] For the question area image, the EAST text detection model is used to locate the text lines in the question area. The question stem text and the answer area text are distinguished by the position and color features of the text lines, thus obtaining the question stem text lines and the answer area text lines.

[0024] The CRNN model is used to perform character recognition on the above question stem text lines and answer area text lines respectively, converting the image text corresponding to the question stem text lines into editable question stem text data, and converting the image text corresponding to the answer area text lines into editable answer content text data.

[0025] The system performs semantic error correction on the identified question text data and answer text data, and, in conjunction with a pre-set subject terminology dictionary, corrects any incorrectly identified characters or formula symbols, ultimately obtaining accurate question content and corresponding answer content.

[0026] Specifically, the step of identifying the area of ​​incorrect questions by combining the marking traces in the question area image includes:

[0027] Red or blue correction marks in the question area image are extracted by color space threshold segmentation; the red or blue correction marks include crosses, wavy lines, and annotation text.

[0028] Calculate the spatial overlap between the correction marks and the answer area. When the overlap exceeds a preset threshold, mark the area as a suspected wrong answer.

[0029] The semantic similarity between the answers to suspected incorrect questions and the standard answers is calculated. If the similarity is lower than the acceptable threshold, the question is confirmed as incorrect.

[0030] Specifically, the semantic similarity calculation between the answers to suspected incorrect questions and the standard answers includes:

[0031] Identify the types of questions suspected of being incorrect, and categorize them as objective questions, subjective questions, or formula-based questions;

[0032] For suspected incorrect questions that are judged to be objective questions, the answer content is extracted, the answer is compared with the corresponding standard answer at the character level, the character consistency rate is calculated, and it is used as the semantic similarity of the objective question;

[0033] For questions suspected of being wrong and identified as subjective questions, their answers and corresponding standard answers are obtained and input into a pre-trained BERT model. The BERT model converts the answers into semantic vectors and the standard answers into standard semantic vectors. The cosine similarity between the semantic vectors and the standard semantic vectors is calculated and used as the semantic similarity of the subjective questions.

[0034] For suspected incorrect questions identified as formula-related questions, the formula text in the identified answer is converted into a LaTeX format answer formula. At the same time, the formula text in the corresponding standard answer is converted into a LaTeX format standard formula. Syntax trees are constructed based on the LaTeX format answer formula and the LaTeX format standard formula, respectively. The structural similarity is calculated by comparing the structural differences between the two syntax trees, and this is used as the semantic similarity of the formula-related questions.

[0035] Specifically, the questions corresponding to the aforementioned incorrect question areas are marked with knowledge points and graded by difficulty, including:

[0036] Extract the question text corresponding to the incorrect question area, input the question text corresponding to the incorrect question area into a preset knowledge point classification model, and output the knowledge point label corresponding to the question text; the classification model is obtained by training through subject knowledge point corpus.

[0037] Based on the error types of incorrect questions and combined with the error rate data of the same questions over the years, the difficulty coefficient of the question is calculated according to the preset calculation rules; the error types of the incorrect questions include conceptual errors, calculation errors, and logical errors;

[0038] Integrate knowledge point tags and difficulty levels to generate multi-dimensional attribute tags for incorrect questions that include both knowledge point and difficulty information.

[0039] Specifically, the knowledge point classification model includes a text embedding layer, an attention mechanism layer, and a classification output layer, wherein:

[0040] The text embedding layer receives the input question text and converts each word in the question text into a fixed-dimensional word vector through a preset word embedding algorithm, forming a word vector sequence corresponding to the question text.

[0041] The attention mechanism layer receives the word vector sequence output by the text embedding layer, calculates and assigns weights to each word vector in the sequence, and obtains a word vector sequence after weight adjustment.

[0042] The classification output layer receives the weighted word vector sequence output by the attention mechanism layer, performs feature extraction and classification calculation on the weighted word vector sequence through a fully connected network, outputs the probability distribution corresponding to the knowledge point label, and selects the label with the highest probability as the knowledge point classification result of the question text.

[0043] Specifically, the preset calculation rule refers to a specific method for calculating the difficulty coefficient using a quantitative formula, which combines the error type weights of incorrect questions with historical error rate data, including:

[0044] Based on the differences in knowledge mastery difficulty reflected by different error types, a fixed weight value is pre-set for each error type;

[0045] Convert the error rate of this question over the years into a standardized value between 0 and 1;

[0046] The weight values ​​and the standardized values ​​of the error rates of this question over the years are calculated according to a preset ratio, and then the two results are added together to obtain the difficulty coefficient.

[0047] The calculated difficulty coefficient is mapped to a difficulty level.

[0048] Specifically, the process of classifying and aggregating the annotation results to generate a structured set of incorrect questions includes:

[0049] A tree-shaped classification directory is constructed based on the knowledge point tags of the incorrect questions, forming a preliminary classification system divided by knowledge points; the tree-shaped classification directory takes the major subject category as the root node and each sub-knowledge point as the child node, and incorrect questions with the same knowledge point tags are assigned to the corresponding child node directory;

[0050] Within each of the above knowledge point sub-node directories, extract the difficulty coefficient of the incorrect questions, arrange the incorrect questions in ascending order of difficulty coefficient, and form a graded practice sequence;

[0051] For each incorrect question in the graded practice sequence, extract the key error steps and the corresponding correct explanations, associate the key error steps and correct explanations with the corresponding questions in the incorrect question set, and generate a document of incorrect questions with explanations.

[0052] The critical error step refers to the erroneous answer section located based on the correction traces;

[0053] The corresponding correct analysis refers to the analysis content matched from the standard answer database;

[0054] The document containing the analyzed incorrect questions includes the question content, the answer details, key incorrect steps, and the correct explanation.

[0055] Compared with the prior art, the beneficial effects of the present invention are:

[0056] 1. This invention proposes an automatic error collection method based on video image acquisition. Through video image acquisition and intelligent processing technology, it automates the entire process of error collection, accurately extracting question regions from continuous video frames and efficiently identifying errors by combining text recognition and correction trace analysis. This eliminates the need for manual screening and entry of each question, significantly reducing the time cost of error organization and improving the efficiency and convenience of error management. Simultaneously, through structured collection using knowledge point annotation and difficulty grading, the generated error collection not only forms a systematic classification according to knowledge points but also constructs a graded practice sequence based on difficulty, providing learners with targeted references for their weaknesses. This process retains key error information and correct explanations of the errors and provides support for personalized learning path planning through data analysis, helping to improve the accuracy and effectiveness of learning.

[0057] 2. This invention proposes an automatic error collection method based on video image acquisition, which supports real-time video stream acquisition and adapts to dynamic shooting needs in various scenarios such as classrooms and homes through inter-frame processing and perspective correction; it integrates text semantic comparison and correction trace recognition to improve the accuracy of error judgment, and is especially suitable for handwritten answers and teacher correction scenarios; it realizes automatic annotation of knowledge points and difficulty classification to generate a structured error collection. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of an automatic error collection method based on video image acquisition according to the present invention;

[0059] Figure 2 This is a flowchart illustrating the principle of an automatic error collection method based on video image acquisition according to the present invention. Detailed Implementation

[0060] Example 1

[0061] Please see Figure 1 and Figure 2 The present invention provides an embodiment of an automatic error collection method based on video image acquisition, the method comprising steps S1 to S4, including the following steps:

[0062] S1: Acquire continuous video frames containing homework or test papers through video acquisition equipment, preprocess the video frames to obtain the question area image;

[0063] S2: Perform text detection and recognition on the question area image, extract the question content and the corresponding answer content, and construct question-answer data pairs;

[0064] S3: Match the question-answer data pairs with a preset standard answer database, and identify the areas of questions with errors by combining the marking traces in the question area image;

[0065] S4: Mark the knowledge points and classify the difficulty level of the questions corresponding to the wrong question area, classify and collect them based on the marking results, generate a structured wrong question set and output it.

[0066] Furthermore, the method also includes:

[0067] The structured error collection is updated regularly, and the knowledge point classification model is updated through incremental learning.

[0068] Based on the user's record of incorrect answers, a personalized analysis report of weak knowledge points is generated;

[0069] It supports the export and sharing of incorrect question sets in formats including PDF, Word, or JSON format adapted to educational platforms.

[0070] In summary, this application is applicable to the automatic collection of incorrect answers from student assignments, test papers, workbooks, and other paper-based materials, and is particularly suitable for the following scenarios:

[0071] During classroom exercises, teachers use mobile devices to photograph students' work while circulating, identifying incorrect answers in real time and calculating the class's error rate.

[0072] For homework, students use tablets or mobile phones to take pictures of the completed assignments, which automatically generate a personal collection of incorrect questions;

[0073] After the exam papers are graded, the exam video is processed in batches using a scanner to quickly generate a report analyzing incorrect answers categorized by knowledge point.

[0074] Example 2

[0075] In this embodiment, the video frame is preprocessed to obtain the title area image, including:

[0076] S1.1: Perform inter-frame stabilization on consecutive video frames, and eliminate image shift caused by shooting shake through feature point matching to obtain a stable video frame sequence;

[0077] Furthermore, the specific steps of S1.1 include:

[0078] (1) Read the continuous video frames input, arrange them into a frame sequence in chronological order, and select an initial reference frame from the frame sequence. Usually, the first frame or the visually most stable frame is selected. In this invention, the frame with the smallest motion is selected as the first frame by calculating the motion amplitude of the first few frames, which is used as the alignment reference for all subsequent frames. In this invention, the core of this step is to establish a stable anchor point to ensure that the offset of subsequent frames is corrected based on this reference frame.

[0079] (2) Perform feature point detection on the reference frame and each subsequent frame. The specific process is as follows:

[0080] The fast corner detection algorithm compares the gray values ​​of neighboring pixels around each pixel. If the gray values ​​of multiple consecutive pixels differ from the current pixel by more than a threshold, they are identified as key points. These key points are usually areas with drastic gray value changes, such as edges and corners in the image. The algorithm has strong anti-interference capabilities. The fast corner detection algorithm is existing technology in this field and is not an inventive solution of this application. It will not be described in detail here.

[0081] Based on the gray-level distribution around the keypoint, a rotation-invariant binary descriptor is calculated. The calculation of the rotation-invariant binary descriptor involves determining the direction by the gray-level centroid and then performing gray-level contrast encoding on the neighboring pixels. This ensures that keypoints at the same physical location in different frames have similar descriptors, which facilitates subsequent matching.

[0082] (3) Match feature points between the reference frame and the current frame;

[0083] (4) Based on the effective matching feature point pairs, calculate the transformation matrix that can describe the offset of the current frame relative to the reference frame. Since the shooting shake is mainly manifested as rigid motion such as translation, rotation, and scaling, an affine transformation matrix is ​​usually used. The affine transformation matrix is ​​a 2×3 matrix that includes translation amount, rotation angle, and scaling factor. Its principle is to minimize the error between the coordinates of the current frame feature points after matrix transformation and the coordinates of the corresponding points in the reference frame by using the least squares method, that is, to make the transformed current frame feature points as close as possible to the position in the reference frame.

[0084] Specifically, a system of equations is established for the coordinate differences of each matching pair, and the matrix parameters that minimize the overall error are solved to finally obtain a transformation matrix that can completely describe the offset of the current frame.

[0085] (5) Apply the calculated transformation matrix to all pixels of the current frame, perform geometric transformation on the current frame through reverse mapping, align the feature points of the current frame with the reference frame, and eliminate the offset caused by jitter.

[0086] During the transformation process, for pixels that exceed the range of the original image after transformation, an edge filling strategy is adopted, that is, copying the edge pixel values ​​or cropping, to ensure the integrity of the output image.

[0087] (6) For each subsequent frame in the video sequence, repeat (3) to (5): based on the reference frame, detect feature points, match, calculate the transformation matrix, and correct the offset. If the video is too long or the jitter mode changes later, the reference frame can be dynamically updated. For example, after processing a certain number of frames, the current stable frame is set as the new reference frame to avoid excessive cumulative error.

[0088] (7) After the above processing, all frames are aligned with the reference frame, the irregular offset caused by jitter is eliminated, and finally a continuous and stable video frame sequence is formed.

[0089] Furthermore, feature points of the reference frame and the current frame are matched, specifically including:

[0090] (1) Hamming distance is used to measure the similarity between the descriptor of each feature point in the current frame and the descriptors of all feature points in the reference frame. The smaller the Hamming distance, the higher the similarity. The formula for calculating the Hamming distance is the prior art in this field and is not an inventive solution of this application. It will not be elaborated here.

[0091] (2) Initially select the candidate matching pairs with the highest similarity, wherein each feature point in the current frame corresponds to the feature point in the reference frame that is closest to it;

[0092] (3) Meanwhile, since jitter or noise may cause incorrect matching, a random sampling consensus algorithm is used for further screening, including: randomly selecting a small number of matching pairs to fit an initial transformation model, counting the interior points that conform to the model, i.e. matching pairs with errors within the threshold, repeating this process multiple times, and retaining the matching pair corresponding to the model with the most interior points as the final effective feature point correspondence. The random sampling consensus algorithm is the prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0093] In summary, this invention compares the degree of difference between two feature point descriptors, such as the Hamming distance of binary descriptors (i.e., the number of different bits in the two descriptors). The smaller the difference, the higher the probability that they are the same physical point. The matrix parameters are determined by minimizing the positional difference between the transformed feature points of the current frame and the corresponding points of the reference frame. Essentially, this makes the transformed current frame fit the structure of the reference frame as closely as possible, thereby offsetting the offset caused by jitter. Spatial correspondence is found through cross-frame matching, and then the positional deviation is corrected by mathematical transformation, ultimately achieving stable alignment between frames.

[0094] S1.2: For the stable video frame sequence, an adaptive threshold segmentation algorithm is used to extract the paper region in the video frame to obtain an initial paper region image. The adaptive threshold segmentation algorithm is prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0095] S1.3: Perform perspective transformation correction on the initial paper area image to obtain a rectangular page image with orthographic projection;

[0096] S1.4: Based on the rectangular page image of the orthographic projection, the title segmentation line is located by edge detection and Hough transform, and independent title area images are segmented according to the title segmentation line.

[0097] Furthermore, the specific steps in S1.4 include:

[0098] (1) Preprocess the rectangular page image of the orthographic projection to reduce noise interference and enhance the accuracy of subsequent edge detection, including:

[0099] Converting a colored rectangular page image to a grayscale image simplifies the image data by preserving pixel brightness information and removing color channels, making subsequent processing more efficient.

[0100] A Gaussian blur algorithm is used to smooth grayscale images. The grayscale value of each pixel is weighted and averaged using a Gaussian kernel of a preset size or a weighted sliding window. Pixels at the center of the window have a high weight, while pixels at the edges have a low weight. This weakens random noise in the image, such as paper texture and glare from the shooting process, while preserving as much realistic edge information as possible. The Gaussian blur algorithm is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0101] (2) Edge detection is performed on the preprocessed grayscale image. The core of this process is to identify regions in the image where the grayscale values ​​change drastically. In this application, these regions typically correspond to lines, text edges, or title dividing lines on paper. The specific process includes:

[0102] The Sobel operator is used to calculate the gray-level gradients of the image in the horizontal and vertical directions, respectively. The magnitude of the gradient reflects the strength of the edge, and the direction of the gradient indicates the direction of the edge. The Sobel operator is prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0103] The gradient direction of each pixel is determined, and only pixels with the largest local gray-level change in the gradient direction are retained, while pixels with the largest non-local gray-level change are removed, so that the edge changes from a wide width to a thin line.

[0104] Two thresholds are set: a high threshold and a low threshold. Pixels with gradient values ​​exceeding the high threshold are judged as strong edges, and pixels with gradient values ​​below the low threshold are judged as non-edges and are directly removed. Pixels with gradient values ​​between the two thresholds need to be further judged, including: if they are adjacent to strong edge pixels, they are retained as weak edges, otherwise they are removed. Finally, a binary image containing only clear edges is obtained, where edge pixels are white and the background is black.

[0105] (3) Map the edge image to Hough space. In image space, a straight line can be described by polar coordinate equations. That is, the coordinates of each point are related to the angle and distance of the straight line. The core of Hough transformation is to convert each edge pixel in image space into a corresponding curve in Hough space. This curve represents all possible straight lines that pass through the pixel.

[0106] (4) Set an accumulator, i.e. a counter, in the Hough space to count the points that each curve passes through. If multiple edge pixels correspond to the same straight line, the curves in the Hough space will intersect at the same point. The accumulator count at that point will be significantly higher than that at non-intersecting points. The higher the count, the more edge pixels the corresponding straight line contains, and the more likely it is to be a real straight line.

[0107] (5) Set a counting threshold, extract the lines corresponding to the points in the accumulator that exceed the threshold, and then filter out the horizontal and vertical lines according to the angle of the lines. Finally, obtain the coordinates of all possible question dividing lines, such as the Y-axis coordinate of the horizontal line and the X-axis coordinate of the vertical line.

[0108] (6) Based on the located horizontal and vertical dividing lines, determine the boundaries of each question and divide it into independent areas, including:

[0109] Sort the horizontal dividing lines from low to high according to the Y-axis coordinate, and sort the vertical dividing lines from left to right according to the X-axis coordinate to clarify the spatial distribution of the lines on the page;

[0110] For horizontal dividing lines, the area between two adjacent lines is usually the vertical range of a question; for vertical dividing lines, the area between two adjacent lines is usually the horizontal range. Combining horizontal and vertical dividing lines, the area of ​​each question is defined as a rectangular area below the upper horizontal dividing line, above the lower horizontal dividing line, to the right of the left vertical dividing line, and to the left of the right vertical dividing line.

[0111] If any area has only horizontal dividing lines and no vertical dividing lines, then the left and right edges of the page are used as the left and right boundaries; if there are multiple dense horizontal or vertical dividing lines, then redundant lines with too small a spacing are removed by calculating the line spacing.

[0112] Based on the defined rectangular boundaries of each question, the corresponding area is cropped from the orthographically projected rectangular page image to obtain an independent question area image.

[0113] In this invention, the entire process starts with an orthographically projected rectangular page image. First, clear edge contours are extracted through preprocessing and edge detection. Then, the straight lines in the edges are transformed from image space to parameter space through Hough transform for identification and filtering. Finally, the boundary of each question is determined based on the coordinates of the dividing line to complete the region segmentation, thereby achieving the goal of accurately extracting independent question regions from the complete page.

[0114] Perform perspective transformation correction on the initial paper area image to obtain a rectangular page image with orthographic projection, including:

[0115] S1.3.1: Perform edge detection on the initial paper area image, identify the coordinates of the four vertices of the paper area, and calculate the spatial distortion parameters of the vertices based on the spatial positional relationship of the four vertices. The edge detection uses the Canny edge detection algorithm, which is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0116] S1.3.2: Construct a perspective transformation matrix based on the spatial distortion parameters, and use the perspective transformation matrix to map the tilted initial paper area image onto a two-dimensional frontal plane to obtain the transformed image;

[0117] Furthermore, the specific steps in S1.3.2 include:

[0118] (1) Determine the vertex coordinates; the vertex coordinates include the initial vertex and the target vertex;

[0119] Initial vertices: Obtain the pixel coordinates of the four identified vertices from the extracted initial paper area image, denoted as A, B, C, and D, corresponding to the four corners of the paper in a tilted state. The coordinates are based on the top left corner of the image as the origin, with the horizontal axis as the X-axis and the vertical axis as the Y-axis. The pixel coordinates reflect the distortion caused by the tilt of the paper. For example, a rectangular piece of paper may appear as a trapezoid or an irregular quadrilateral in the image, and the relative positions of the four vertices deviate from the right angle relationship of the rectangle.

[0120] Target vertices: Set the coordinates of the four vertices of the ideal two-dimensional plane, that is, the rectangle that the paper should present after correction, and set them as A', B', C', and D'. Usually, they are set according to the approximate proportion of the initial paper. For example, set A' to (0,0), B' to (W,0), C' to (W,H), and D' to (0,H) to form a standard rectangle, that is, the perspective transformation matrix, where W represents the target width and H represents the target height.

[0121] (2) For each pair of corresponding vertices, such as A and A', assume the initial coordinates are The target coordinates are According to the principle of perspective transformation, the coordinates before and after the transformation satisfy the condition that the target coordinates are the normalized result of the initial coordinates after transformation by the perspective transformation matrix. Simply put, it involves using the parameters of the perspective transformation matrix to... Convert to Furthermore, this transformation requires a one-to-one correspondence between four pairs of vertices, where x and y represent the initial X-axis and Y-axis coordinates, respectively. and These represent the transformed X-axis and Y-axis coordinate values, respectively.

[0122] (3) By establishing such a correspondence for each of the four pairs of vertices, we can obtain 8 equations, of which each pair of vertices contributes 2 equations. For example, for vertices A and A', correspond At that time, the equation will contain 8 parameters of the perspective transformation matrix and , , , The relationship, and so on, includes, and These represent the X-axis and Y-axis coordinates of vertex A, respectively. and These represent the X-axis and Y-axis coordinates of vertex A', respectively.

[0123] (4) By solving the system of equations consisting of the above 8 equations, the 8 parameters of the perspective transformation matrix are obtained. Since the vertex coordinates in the actual image may contain noise, the direct solution may have deviation. Therefore, it is necessary to optimize by the least squares method. The least squares method is the prior art in this field and is not an inventive solution of this application. It will not be described in detail here.

[0124] (5) Using the solved perspective transformation matrix, perform coordinate transformation on each pixel of the initial paper area image. The specific process is as follows:

[0125] Traverse each pixel of the target frontal plane image, i.e. the rectangular image to be generated, and calculate the original coordinates of the pixel in the initial tilted image by using the inverse of the perspective transformation matrix;

[0126] Repeat the above operation for all pixels of the target frontal image, rearrange the pixels in the initial tilted image according to the rules of the perspective transformation matrix, fill them into the target rectangular area, and finally obtain the transformed image with tilt distortion eliminated.

[0127] Since the calculated original coordinates may be decimals rather than integer pixel positions, the pixel value at that position needs to be determined by bilinear interpolation: by finding the gray values ​​of the four integer pixels around the original coordinates, the final value is calculated based on the distance weight, ensuring the smoothness of the target image.

[0128] In this invention, the entire process starts with the difference between the initial tilted vertex and the target normal plane vertex. The perspective transformation matrix is ​​solved by establishing a correspondence, and then the matrix is ​​used to reposition the pixels of the tilted image to the normal plane, ultimately realizing the mapping from the distorted state to the normal state. Each step takes the output of the previous step as input, realizing vertex coordinates → matrix parameters → mapped pixels → complete image, forming a coherent distortion correction logic, ensuring that the tilted paper image is accurately converted into a flat rectangular image.

[0129] S1.3.3: Perform grayscale equalization processing on the transformed image to obtain a rectangular page image of orthographic projection. The grayscale equalization processing is prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0130] Perform text detection and recognition on the image of the question area to extract the question content and the corresponding answer content, including:

[0131] S2.1: For the question area image, the EAST text detection model is used to locate the text lines in the question area. The question stem text and the answer area text are distinguished by the position and color features of the text lines, and the question stem text lines and answer area text lines are obtained.

[0132] Among them, EAST (Efficient and Accurate Scene Text Detector) is a scene text detection model whose core advantage is end-to-end detection: it directly outputs the bounding boxes of text lines from the input image without intermediate steps, thus achieving high speed and high accuracy.

[0133] Furthermore, the specific steps of S2.1 include:

[0134] (1) Preprocess the image of the question area to ensure that it meets the input requirements of the EAST model, including: first converting the image to RGB format, since the EAST model usually receives three-channel color images, then normalizing the size, and finally standardizing the pixel values ​​to make the input data distribution consistent with the model training data and improve detection stability.

[0135] (2) Input the preprocessed question region image into the EAST model, and calculate the bounding boxes of all text lines through the model, specifically including:

[0136] The backbone network in the EAST model performs multiple convolutions and downsampling on the input image to generate feature maps at different scales. In these feature maps, low-level features preserve the edges and stroke details of the text, which are suitable for detecting small-sized text, while high-level features contain more abstract semantic information, which are suitable for detecting large-sized text or text in complex backgrounds.

[0137] By fusing feature maps of different scales through a feature pyramid network, the fused feature map contains both detailed information and semantic information, which can more accurately locate text lines of different sizes and directions. The feature pyramid network is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0138] The feature maps are input to and output from two branches of the layer:

[0139] The score map branch predicts the probability that each pixel is text through convolution operations, generating a text score map where the probability value is between 0 and 1, and the closer it is to 1, the more likely it is to be text.

[0140] Geometric shape branch: Similarly, the bounding box parameters corresponding to each text pixel are predicted through convolution operations to generate a geometric shape map;

[0141] Extracting text line bounding boxes: Threshold filtering is performed on the score image, and then adjacent text pixels are merged into complete text line bounding boxes according to the bounding box parameters corresponding to these pixels in the geometric shape image. For example, the bounding boxes corresponding to pixels belonging to the same line of text are merged into a whole rectangle. Finally, the bounding boxes of all text lines in the question area image are obtained. Each bounding box is represented by four vertex coordinates and surrounds a line of text.

[0142] (3) Using the bounding box information and pixel color features of the text lines, distinguish the question stem from the answer area, including:

[0143] The coordinates of the bounding boxes of all text lines were calculated. The origin was set to the top-left corner of the question area image, with X representing the horizontal direction and Y representing the vertical direction. The center coordinates of each text line were calculated. Typically, the question stem text is positioned higher or to the left in the question area, while the answer area text is positioned lower or to the right. By comparing the relative positions of the center coordinates of the text lines, they were initially divided into two categories: the question stem text (including the question description and the question statement), and the answer area text (including the student's handwritten answers).

[0144] Extract the color features of the text lines. The question stem text is mostly printed, typically pure black, while the answer area text is mostly handwritten, typically blue or black. The specific steps are as follows:

[0145] For each pixel within the bounding box of a text line, convert it from the RGB color space to the HSV color space;

[0146] The distribution of hue (H), saturation (S), and brightness (V) values ​​of pixels within the bounding box is statistically analyzed.

[0147] Based on the color distribution characteristics, the initially divided text lines are verified a second time: if the pixels within the bounding box of the text line are mainly black printed features, then it is confirmed as the question stem text line; if the pixels are mainly blue or handwritten black features, then it is confirmed as the answer area text line.

[0148] By combining the initial division based on location features and the secondary verification based on color features, misjudgments were eliminated, and all lines of text in the question stem and the text lines in the answer area were finally determined.

[0149] In this invention, the entire process starts with the image of the question area. First, it adapts the input to the EAST model through preprocessing. Then, it uses the feature extraction, fusion and prediction of the EAST model to locate the bounding boxes of all text lines. Finally, by analyzing the position coordinates and color distribution of the text lines, it distinguishes the question stem and the text lines in the answer area, forming a complete logical chain from image to text line type.

[0150] S2.2: Using the CRNN model, character recognition is performed on the above-mentioned question stem text lines and answer area text lines respectively, converting the image text corresponding to the question stem text lines into editable question stem text data, and converting the image text corresponding to the answer area text lines into editable answer content text data;

[0151] Among them, CRNN (Convolutional Recurrent Neural Network) includes convolutional layers, recurrent layers, and transcriptional layers. It is suitable for character recognition tasks and can directly identify ordered character sequences from continuous image pixels without manually segmenting individual characters.

[0152] Furthermore, the specific steps of S2.2 include:

[0153] (1) The images of the question stem text line and the answer area text line are preprocessed to ensure that they meet the input requirements of CRNN. ​​CRNN is a prior art in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0154] (2) Input the preprocessed text line image into the convolutional layer of the CRNN to extract the feature sequence;

[0155] (3) Input the one-dimensional feature sequence output by the convolutional layer into the recurrent layer of the CRNN to process the order dependency between characters;

[0156] (4) By connecting the temporal classification loss function through CTC (Connectionist Temporal Classification), the probability distribution output by the recurrent layer is converted into the final character sequence to obtain editable text, thus solving the problem of mismatch between the feature sequence length and the character sequence length;

[0157] (5) Repeat steps (1)-(4) above for the image of the question stem text line and the image of the answer area text line respectively;

[0158] After the question stem text is recognized by CRNN, editable question stem text data is output.

[0159] After the text lines in the answer area are recognized by CRNN, the editable answer content text data is output.

[0160] In this invention, the entire process begins with a preprocessed image of text lines. First, a convolutional layer converts the image into a one-dimensional sequence containing visual features. Then, a bidirectional LSTM captures the sequential dependencies between characters, outputting the character probability at each position. Finally, CTC transcription yields the final character sequence. By performing this process on the text lines of the question stem and the answer area respectively, the text in the image can be converted into editable text data.

[0161] S2.3: Perform semantic error correction on the identified question stem text data and answer content text data, and combine it with the preset subject terminology dictionary to correct the characters or formula symbols that are incorrectly identified, so as to obtain the accurate question content and corresponding answer content.

[0162] Furthermore, the specific steps in S2.3 include:

[0163] (1) Preprocess the question stem text data and answer content text data obtained by CRNN recognition to remove interference information;

[0164] (2) Construct a subject-specific terminology dictionary and establish reference standards for error correction;

[0165] (3) Compare the preprocessed text units with the subject terminology dictionary and mark units that may have errors. Specifically, for each text unit after splitting, first compare it with the standard writing in the basic terminology database. If it matches completely, such as acceleration, it is determined to be a reliable unit. If it does not match, such as the degree of acceleration, it is marked as a suspicious unit.

[0166] (4) For the marked suspicious units, multiple candidate correction words are generated by combining the subject terminology dictionary and the context. The specific methods include: querying the error case library for historical errors with similar glyphs to the suspicious units and directly generating the corresponding correct terms as candidates;

[0167] (5) Select the option that best fits the context and subject logic from the candidate correction words to complete the error correction.

[0168] Furthermore, the process of constructing a subject-specific terminology dictionary includes:

[0169] (1) Store core concepts, commonly used symbols, and formula templates by subject category, and label the standard writing, common error forms, and occurrence scenarios of each term;

[0170] (2) Record common collocations between terms and grammatical rules for formula symbols;

[0171] (3) Collect historical identification error data, label the error type and correction plan, and use it as a reference for targeted error correction.

[0172] In this invention, the entire process starts with the original text data after recognition. First, noise is removed and the data is split into units through preprocessing. Then, suspicious error units are located by relying on a subject terminology dictionary to generate targeted candidate correction words. Finally, the optimal results are selected through semantic coherence, subject rules, etc., forming a complete error correction logic from recognition results to accurate text, providing a reliable data foundation for subsequent error analysis.

[0173] The method of identifying erroneous question areas by combining the marking trace features in the question area image includes:

[0174] S3.1: Extract red or blue correction marks from the question area image by color space threshold segmentation; the red or blue correction marks include crosses, wavy lines, and annotation text;

[0175] S3.2: Calculate the spatial overlap between the correction marks and the answer area. When the spatial overlap exceeds a preset threshold, mark the area as a suspected wrong answer.

[0176] Furthermore, the specific steps in S3.2 include:

[0177] (1) Obtain the binary image of the correction mark obtained by color space threshold segmentation and the coordinate range of the region in the question image. In the binary image, the pixel value corresponding to the correction mark is 1, indicating that there is a mark, and the background pixel value is 0, indicating that there is no mark.

[0178] (2) Obtain the binary image of the answer area located by text detection and the coordinate range of the area in the question image. In the binary image, the pixel value corresponding to the answer area is 1, indicating that there is answer content, and the background pixel value is 0, indicating that there is no answer content.

[0179] It is necessary to ensure that the coordinates of the two regions are based on the same coordinate system to avoid calculation errors caused by different coordinate references.

[0180] (3) Locate the overlapping area between the correction mark area and the answer area in the image through coordinate operations, including:

[0181] Calculate the coordinate intersection: Compare the coordinates of the bounding rectangles of the two regions to determine the boundary of the overlapping region. Specifically, the X-axis coordinate of the top-left corner of the overlapping region is the larger of the X-axis coordinates of the top-left corners of the two regions, and the Y-axis coordinate of the top-left corner is the larger of the Y-axis coordinates of the top-left corners of the two regions; the X-axis coordinate of the bottom-right corner of the overlapping region is the smaller of the X-axis coordinates of the bottom-right corners of the two regions, and the Y-axis coordinate of the bottom-right corner is the smaller of the Y-axis coordinates of the bottom-right corners of the two regions.

[0182] Determine if there is an intersection: If the X-axis coordinate of the upper left corner of the calculated overlapping area is greater than the X-axis coordinate of the lower right corner, or the Y-axis coordinate of the upper left corner is greater than the Y-axis coordinate of the lower right corner, it means that the two areas have no spatial overlap, and the overlap is directly determined to be 0; otherwise, the coordinate range is the spatial range in which the two areas may overlap.

[0183] (4) Based on the binary image and the intersection range, count the number of pixels in the overlapping part of the two regions and the total number of pixels covered by the two regions; the number of pixels in the overlapping part of the two regions refers to the intersection area; the total number of pixels covered by the two regions refers to the union area.

[0184] Calculate the intersection area: Within the defined overlapping coordinate range, check the corresponding positions of the binary image of the correction mark and the binary image of the answer area pixel by pixel. If the pixel value of the two images at that pixel position is 1, that is, it is both the correction mark and the answer content, then the pixel belongs to the intersection area. Count the total number of pixels that meet this condition, which is the area of ​​the intersection area.

[0185] Calculate the area of ​​the union: The union region is the sum of all pixels belonging to the correction mark region or the answer region. The calculation method is as follows: First, count the total number of pixels in the correction mark region, that is, the total number of pixels with a value of 1 in the binary image of the correction mark. Then, count the total number of pixels in the answer region, that is, the total number of pixels with a value of 1 in the binary image of the answer region. Finally, subtract the number of pixels in the intersection region. The result is the area of ​​the union region.

[0186] (5) The spatial overlap of two regions is quantified by the ratio of the intersection area to the union area. The higher the ratio, the more obvious the overlap between the correction marks and the answer area. The specific calculation logic is: divide the area of ​​the intersection area by the area of ​​the union area, and the resulting value is the spatial overlap.

[0187] (6) Determine whether to mark the area as a suspected wrong question based on the degree of overlap, including: comparing the calculated spatial overlap with a preset threshold. If the overlap is greater than or equal to the threshold, it indicates that the correction marks and the answer area overlap significantly, which is consistent with the common characteristics of wrong questions. Therefore, the answer area is marked as a suspected wrong question area. If the overlap is less than the threshold, it is determined to be a non-suspected wrong question area.

[0188] S3.3: The semantic similarity between the answer to a suspected incorrect question and the standard answer is calculated. When the similarity is lower than the qualified threshold, the question is confirmed as incorrect. The similarity calculation uses cosine similarity, which is existing technology in this field and is not an inventive solution of this application. It will not be elaborated here.

[0189] The semantic similarity calculation between the answers to suspected incorrect questions and the standard answers includes:

[0190] S3.3.1: Determine the type of question suspected of being wrong, and classify it into objective questions, subjective questions, or formula-based questions;

[0191] S3.3.2: For suspected incorrect questions that are judged to be objective questions, extract the answer from the answer content, compare the answer with the corresponding standard answer at the character level, calculate the character consistency rate, and use it as the semantic similarity of the objective question;

[0192] S3.3.3: For suspected incorrect questions that are judged to be subjective questions, obtain their answer content and corresponding standard answer, and input them into the pre-trained BERT model. The BERT model converts the answer content into an answer semantic vector and the standard answer into a standard semantic vector. The cosine similarity between the answer semantic vector and the standard semantic vector is calculated and used as the semantic similarity of the subjective question. The BERT model is existing technology in this field and is not an inventive solution of this application, so it will not be described in detail here.

[0193] S3.3.4: For suspected incorrect questions identified as formula-related questions, the formula text in the identified answer is converted into a LaTeX format answer formula. At the same time, the formula text in the corresponding standard answer is converted into a LaTeX format standard formula. A syntax tree is constructed based on the LaTeX format answer formula and the LaTeX format standard formula, respectively. The structural similarity is calculated by comparing the structural differences between the two syntax trees, and this is used as the semantic similarity of the formula-related questions.

[0194] The questions corresponding to the aforementioned incorrect question areas are marked with knowledge points and graded by difficulty, including:

[0195] S4.1: Extract the question text corresponding to the incorrect question area, input the question text corresponding to the incorrect question area into the preset knowledge point classification model, and output the knowledge point label corresponding to the question text; the classification model is obtained by training through subject knowledge point corpus;

[0196] S4.2: Based on the error type of the incorrect question and combined with the error rate data of the question over the years, calculate the difficulty coefficient of the question according to the preset calculation rules; the error type of the incorrect question includes conceptual error, calculation error, and logical error;

[0197] S4.3: Integrate knowledge point tags and difficulty coefficients to generate multi-dimensional attribute tags for incorrect questions that include knowledge point information and difficulty information.

[0198] The knowledge point classification model described in S4.1 includes a text embedding layer, an attention mechanism layer, and a classification output layer, wherein:

[0199] The text embedding layer receives the input question text and converts each word in the question text into a fixed-dimensional word vector through a preset word embedding algorithm, forming a word vector sequence corresponding to the question text.

[0200] The attention mechanism layer receives the word vector sequence output by the text embedding layer, calculates and assigns weights to each word vector in the sequence, and obtains a word vector sequence after weight adjustment.

[0201] The classification output layer receives the weighted word vector sequence output by the attention mechanism layer, performs feature extraction and classification calculation on the weighted word vector sequence through a fully connected network, outputs the probability distribution corresponding to the knowledge point label, and selects the label with the highest probability as the knowledge point classification result of the question text.

[0202] The preset calculation rule described in S4.2 refers to the specific method for calculating the difficulty coefficient through a quantitative formula, which combines the error type weights of incorrect questions with historical error rate data, including:

[0203] Based on the differences in knowledge mastery difficulty reflected by different error types, a fixed weight value is pre-set for each error type;

[0204] Convert the error rate of this question over the years into a standardized value between 0 and 1;

[0205] The weight values ​​and the standardized values ​​of the error rates of this question over the years are calculated according to a preset ratio, and then the two results are added together to obtain the difficulty coefficient.

[0206] The calculated difficulty coefficient is mapped to a difficulty level.

[0207] The process of classifying and grouping questions based on annotation results to generate a structured set of incorrect answers includes:

[0208] A1: Construct a tree-shaped classification directory based on the knowledge point tags of the incorrect questions to form a preliminary classification system divided by knowledge points; the tree-shaped classification directory takes the major subject category as the root node and each sub-knowledge point as the child node, and incorrect questions with the same knowledge point tags are assigned to the corresponding child node directory;

[0209] A2: Within each of the above knowledge point sub-node directories, extract the difficulty coefficient of the incorrect questions, arrange the incorrect questions in ascending order of difficulty coefficient, and form a graded practice sequence;

[0210] A3: For each incorrect question in the graded practice sequence, extract the key error steps and the corresponding correct explanations, associate the key error steps and correct explanations with the corresponding questions in the incorrect question set, and generate a document of incorrect questions with explanations.

[0211] The critical error step refers to the erroneous answer section located based on the correction traces;

[0212] The corresponding correct analysis refers to the analysis content matched from the standard answer database;

[0213] The document containing the analyzed incorrect questions includes the question content, the answer details, key incorrect steps, and the correct explanation.

[0214] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments under the guidance of the present invention without departing from the spirit and scope of the present invention. All of these variations are within the protection scope of the present invention.

[0215] If the technical solution disclosed herein involves personal information, the product using this technical solution has clearly informed the user of the personal information processing rules and obtained the user's voluntary consent before processing the personal information. If the technical solution disclosed herein involves sensitive personal information, the product using this technical solution has obtained the user's separate consent before processing the sensitive personal information, and also meets the requirement of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set up to inform users that they have entered the scope of personal information collection and that personal information will be collected. If an individual voluntarily enters the collection scope, it is deemed that they have agreed to the collection of their personal information; or on the personal information processing device, with clear signs / information informing users of the personal information processing rules, authorization is obtained from the individual through pop-up information or by asking the individual to upload their personal information; wherein, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

Claims

1. An automatic error collection method based on video image acquisition, characterized in that, include: A series of video frames containing homework or test papers are acquired using a video capture device, and the video frames are preprocessed to obtain an image of the question area. Perform text detection and recognition on the image of the question area to extract the question content and the corresponding answer content, and construct question-answer data pairs; The question-answer data pairs are matched with a preset standard answer database, and the areas of incorrect questions are identified by combining the marking traces in the question area image. The questions corresponding to the incorrect question areas are marked with knowledge points and graded by difficulty. Based on the marking results, they are classified and collected to generate a structured incorrect question set and output it. The method of identifying erroneous question areas by combining the marking trace features in the question area image includes: Red or blue correction marks in the question area image are extracted by color space threshold segmentation; the red or blue correction marks include crosses, wavy lines, and annotation text. Calculate the spatial overlap between the correction marks and the answer area. When the overlap exceeds a preset threshold, mark the area as a suspected wrong answer. The semantic similarity between the answers to suspected incorrect questions and the standard answers is calculated. If the similarity is lower than the acceptable threshold, the question is confirmed as incorrect.

2. The automatic error collection method based on video image acquisition as described in claim 1, characterized in that, The video frames are preprocessed to obtain the title area image, including: Inter-frame stabilization is performed on consecutive video frames, and image shift caused by shooting shake is eliminated by feature point matching to obtain a stable video frame sequence. For the stable video frame sequence, an adaptive threshold segmentation algorithm is used to extract the paper region in the video frame to obtain the initial paper region image; A perspective transformation correction is performed on the initial paper area image to obtain a rectangular page image with orthographic projection; Based on the rectangular page image of the orthographic projection, the title segmentation line is located by edge detection and Hough transform, and independent title region images are segmented according to the title segmentation line.

3. The automatic error collection method based on video image acquisition as described in claim 2, characterized in that, Perform perspective transformation correction on the initial paper area image to obtain a rectangular page image with orthographic projection, including: Edge detection is performed on the initial paper area image to identify the coordinates of the four vertices of the paper area, and the spatial distortion parameters of the vertices are calculated based on the spatial positional relationship of the four vertices. Based on the spatial distortion parameters, a perspective transformation matrix is ​​constructed. The perspective transformation matrix is ​​then used to map the tilted initial paper area image onto a two-dimensional frontal plane to obtain the transformed image. The transformed image is then subjected to grayscale equalization to obtain a rectangular page image with orthographic projection.

4. The automatic error collection method based on video image acquisition as described in claim 1, characterized in that, Perform text detection and recognition on the image of the question area to extract the question content and the corresponding answer content, including: For the question area image, the EAST text detection model is used to locate the text lines in the question area. The question stem text and the answer area text are distinguished by the position and color features of the text lines, thus obtaining the question stem text lines and the answer area text lines. The CRNN model is used to perform character recognition on the above question stem text lines and answer area text lines respectively, converting the image text corresponding to the question stem text lines into editable question stem text data, and converting the image text corresponding to the answer area text lines into editable answer content text data. The system performs semantic error correction on the identified question text data and answer text data, and, in conjunction with a pre-set subject terminology dictionary, corrects any incorrectly identified characters or formula symbols, ultimately obtaining accurate question content and corresponding answer content.

5. The automatic error collection method based on video image acquisition as described in claim 4, characterized in that, The semantic similarity calculation between the answers to suspected incorrect questions and the standard answers includes: Identify the types of questions suspected of being incorrect, and categorize them as objective questions, subjective questions, or formula-based questions; For suspected incorrect questions that are judged to be objective questions, the answer content is extracted, the answer is compared with the corresponding standard answer at the character level, the character consistency rate is calculated, and it is used as the semantic similarity of the objective question; For questions suspected of being wrong and identified as subjective questions, their answers and corresponding standard answers are obtained and input into a pre-trained BERT model. The BERT model converts the answers into semantic vectors and the standard answers into standard semantic vectors. The cosine similarity between the semantic vectors and the standard semantic vectors is calculated and used as the semantic similarity of the subjective questions. For suspected incorrect questions identified as formula-related questions, the formula text in the identified answer is converted into a LaTeX format answer formula. At the same time, the formula text in the corresponding standard answer is converted into a LaTeX format standard formula. Syntax trees are constructed based on the LaTeX format answer formula and the LaTeX format standard formula, respectively. The structural similarity is calculated by comparing the structural differences between the two syntax trees, and this is used as the semantic similarity of the formula-related questions.

6. The automatic error collection method based on video image acquisition as described in claim 1, characterized in that, The questions corresponding to the aforementioned incorrect question areas are marked with knowledge points and graded by difficulty, including: Extract the question text corresponding to the incorrect question area, input the question text corresponding to the incorrect question area into a preset knowledge point classification model, and output the knowledge point label corresponding to the question text; the classification model is obtained by training through subject knowledge point corpus. Based on the error types of incorrect questions and combined with the error rate data of the same questions over the years, the difficulty coefficient of the question is calculated according to the preset calculation rules; the error types of the incorrect questions include conceptual errors, calculation errors, and logical errors; Integrate knowledge point tags and difficulty levels to generate multi-dimensional attribute tags for incorrect questions that include both knowledge point and difficulty information.

7. The automatic error collection method based on video image acquisition as described in claim 6, characterized in that, The knowledge point classification model includes a text embedding layer, an attention mechanism layer, and a classification output layer, wherein: The text embedding layer receives the input question text and converts each word in the question text into a fixed-dimensional word vector through a preset word embedding algorithm, forming a word vector sequence corresponding to the question text. The attention mechanism layer receives the word vector sequence output by the text embedding layer, calculates and assigns weights to each word vector in the sequence, and obtains a word vector sequence after weight adjustment. The classification output layer receives the weighted word vector sequence output by the attention mechanism layer, performs feature extraction and classification calculation on the weighted word vector sequence through a fully connected network, outputs the probability distribution corresponding to the knowledge point label, and selects the label with the highest probability as the knowledge point classification result of the question text.

8. The automatic error collection method based on video image acquisition as described in claim 7, characterized in that, The preset calculation rule refers to a specific method for calculating the difficulty coefficient using a quantitative formula, which combines the error type weights of incorrect questions with historical error rate data, including: Based on the differences in knowledge mastery difficulty reflected by different error types, a fixed weight value is pre-set for each error type; Convert the error rate of this question over the years into a standardized value between 0 and 1; The weight values ​​and the standardized values ​​of the error rates of this question over the years are calculated according to a preset ratio, and then the two results are added together to obtain the difficulty coefficient. The calculated difficulty coefficient is mapped to a difficulty level.

9. The automatic error collection method based on video image acquisition as described in claim 1, characterized in that, The process of classifying and grouping questions based on annotation results to generate a structured set of incorrect answers includes: A tree-shaped classification directory is constructed based on the knowledge point tags of the incorrect questions, forming a preliminary classification system divided by knowledge points; the tree-shaped classification directory takes the major subject category as the root node and each sub-knowledge point as the child node, and incorrect questions with the same knowledge point tags are assigned to the corresponding child node directory; Within each of the above knowledge point sub-node directories, extract the difficulty coefficient of the incorrect questions, arrange the incorrect questions in ascending order of difficulty coefficient, and form a graded practice sequence; For each incorrect question in the graded practice sequence, extract the key error steps and the corresponding correct explanations, associate the key error steps and correct explanations with the corresponding questions in the incorrect question set, and generate a document of incorrect questions with explanations. The critical error step refers to the erroneous answer section located based on the correction traces; The corresponding correct analysis refers to the analysis content matched from the standard answer database; The document containing the analyzed incorrect questions includes the question content, the answer details, key incorrect steps, and the correct explanation.

Images