Automatic marking method and system for answer sheets

Abstract

The application relates to the field of image recognition, and particularly discloses an automatic marking method and system for a test paper, which comprises the following steps: recognizing the divided answer area, and determining according to the confidence: marking the area with a confidence greater than or equal to a first threshold value as a'recognized successful area' to enter a comparison link; otherwise, marking the area as a 'to-be-reasoned area'; recognizing the divided answer area of the 'to-be-reasoned area' and secondly determining according to the confidence: marking the area with a confidence greater than or equal to a second threshold value as a'reasoned area', otherwise, marking the area as a 'non-reasoned area'; the second threshold value is less than the first threshold value; correcting the'reasoned area' to obtain a'reasoned processing area' image; and recognizing the divided answer area of the'reasoned processing area' image and thirdly determining according to the confidence: marking the area with a confidence greater than or equal to the first threshold value as a'recognized successful area' to enter the comparison link.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image recognition, in particular to an automatic marking method and system for a test paper. BACKGROUND

[0002] In the prior art, the OCR recognition is easily affected by factors such as messy handwriting (e.g. illegible handwriting, writing errors), poor printing quality of the test paper (e.g. blurring, stains) and physical deformation (e.g. wrinkles, tilting), resulting in high recognition errors and thus reducing the accuracy of automatic marking, especially in the recognition of subjective questions and non-standard handwritten content, which requires a large amount of manual intervention, affecting the marking efficiency.

[0003] In addition, there are many areas in the test paper that have poor OCR recognition but have reasoning correction value (e.g. some testees write messy, but the content is close to the standard answer), and the existing technology cannot effectively identify the answers in these areas, which requires a large amount of manual intervention. The present application can particularly solve such problems. SUMMARY

[0004] The present application aims to overcome the defects of the prior art and provide an automatic marking method and system for a test paper to solve the problems raised in the background.

[0005] The automatic marking method for a test paper of the present application comprises the following steps:

[0006] The standard answer of the test paper is input, and the text features, logical associations and position information of the standard answer on the test paper are stored. The divided answer area is subjected to OCR recognition, and a confidence level is determined: if the confidence level is greater than or equal to a first threshold value, it is marked as a "recognition success area" and enters the comparison stage; otherwise, it is marked as a "to-be-reasoned area". The divided answer area of the "to-be-reasoned area" is subjected to OCR recognition and a second confidence level is determined: if the confidence level is greater than or equal to a second threshold value, it is marked as a "reasoning area", otherwise it is marked as a "no reasoning value area" and pushed to the manual comparison stage; the second threshold value is less than the first threshold value. For the "reasoning area", the handwritten text features are compared with the standard answer text features, and the handwritten logical associations are compared with the standard answer logical associations, and the "reasoning processing area" image is obtained by correction;

[0007] For the divided answer area of the "reasoning processing area" image, OCR recognition is performed and a third confidence level is determined: if the confidence level is greater than or equal to the first threshold value, it is marked as a "recognition success area" and enters the comparison stage; otherwise, it is marked as a "no reasoning value area" and pushed to the manual comparison stage; and for the "recognition success area" entering the comparison stage, automatic marking is performed.

[0008] Further, the pre-processing of the answer sheet before the divided answer area is subjected to OCR recognition includes:

[0009] The answer sheet filled out by the student is scanned and a picture is generated, the student number and the question number sequence information are associated through a two-dimensional code; the picture is pre-processed to retain the effective answer traces.

[0010] Further, the pre-processing of the answer sheet before the divided answer area is subjected to OCR recognition includes:

[0011] According to the position information recorded in the standard answer feature library, the answer areas of each question are divided on the pre-processed answer sheet picture; if there is deviation in the area, the question number text on the answer sheet is recognized to correct the boundary.

[0012] Further, the standard answer of the test question corresponding to the answer sheet is entered, and the text features, logical associations and position information of the standard answer on the answer sheet are stored, including: the standard answer of the test question corresponding to the answer sheet is entered, wherein the text content of each option, the stroke structure and morphological features of the characters, and the logical association of knowledge points between questions are entered for objective questions; the semantic association of core keywords and key sentences, the logical sequence of problem solving steps, and the stroke number and radical position features of the text are entered for subjective questions. At the same time, the position information of the area where the standard answer of each question is located is determined by scanning a blank answer sheet, including the corner coordinates of the area and the character spacing. These text features, logical associations and position information are uniformly stored to form a standard answer feature library.

[0013] Further, when the divided answer area is subjected to OCR recognition, the multi-dimensional feature matching degree is calculated synchronously as the confidence evaluation basis; first, the contour features of the characters are extracted, and the normalized cross-correlation value thereof with the standard character template is calculated; then the stroke structure features are analyzed, and the matching degrees of the stroke number and the position features of the cross points are counted; then the context semantic association degree is evaluated, and the cosine similarity of the recognized text and the expected content is calculated according to the question type and the expected semantics of the standard answer; finally, the confidence is obtained by weighted fusion of the three indexes.

[0014] Further, the answer area after re-dividing the "to-be-reasoned area" is subjected to OCR recognition again, and the multi-dimensional feature fusion is used to calculate the secondary confidence; first, the local stroke detail features of the handwritten characters are extracted, and the feature matching rate thereof with the corresponding characters in the standard answer text feature library is calculated; secondly, the position distribution features of the characters in the area are analyzed, and the layout matching degree is obtained by comparing with the character layout of the standard answer area; and thirdly, according to the type of the question to which the area belongs, a question type adaptation coefficient is introduced, and the secondary confidence is calculated by weighting the feature matching rate and the layout matching degree together.

[0015] Further, the pushing to the manual comparison link also includes calculating the manual processing priority of the "non-reasoning value area", so that the area with low confidence and high score is preferentially entered into the manual processing queue.

[0016] Further, the second threshold is set to be less than the first threshold, and the second threshold is set to be less than the first threshold in terms of the quality of the answer sheet recognition.

[0017] Further, when comparing the handwritten character features and the standard answer character features, the feature matching degree M of the handwritten character and the standard answer character is calculated, M=(number of matched stroke features+number of matched structure features) / (total number of stroke features+total number of structure features), wherein the matched stroke features include stroke length, inclination angle, and coincidence degree of start and end point coordinates, and the matched structure features include character overall contour overlap rate and relative position deviation rate of radicals and components; when comparing the handwritten logical association and the standard answer logical association, the option logical fit degree L1 of objective questions is calculated, L1=(number of features conforming to the standard answer option distribution rule+number of cross-question option association matches) / (total number of option features+total number of cross-question associations), and the semantic logical matching degree L2 of subjective questions is calculated, L2=(keyword semantic similarity average×keyword weight+step logical node matching degree×step weight), wherein the keyword semantic similarity is calculated by word vector cosine distance, and the step logical node matching degree is obtained by weighting the step order coincidence rate and the causal association degree; the comprehensive correction index kuy after correction is determined, kuy=α×M+β×L1 or kuy=α×M+β×L2.

[0018] In addition, an electronic device is disclosed, comprising:

[0019] at least one processor; and

[0020] a memory connected with the at least one processor in communication; wherein,

[0021] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the automatic marking method of the answer sheet.

[0022] The beneficial effects of the present application are:

[0023] The application introduces an auxiliary reasoning mechanism for the content with OCR recognition error (low confidence), which significantly improves the accuracy of OCR recognition in the scenarios of messy handwriting, test paper stains / wrinkles, etc., and effectively solves the problem of low accuracy of automatic marking due to high recognition error in the prior art. By multiple confidence determinations on the test card area, the "reasoning area" and the "area without reasoning value" are distinguished, only the latter is pushed to manual, which greatly reduces the demand for manual intervention; at the same time, by calculating the priority of manual processing, the area with low confidence and high score is preferentially put into the manual queue, which improves the efficiency of manual comparison. In the preprocessing step, through the association information of the two-dimensional code, image preprocessing and area division correction, the accurate extraction and area positioning of the effective answer trace are ensured. If there are many areas in the test card that are not good in OCR recognition but have reasoning correction value (such as the messy writing of some test takers, but the written content is very close to the standard answer), the application can especially solve such problems, and can find the area that is not good in recognition but has reasoning correction value. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 The workflow diagram of the automatic marking method of the test card of the application. DETAILED DESCRIPTION

[0025] The application discloses an automatic marking method of a test card, as shown in Figure 1, comprising steps: S100. entering the standard answer of the corresponding test question of the answer sheet, storing the text features, logical association and position information of the standard answer on the answer sheet as the basis for recognition and reasoning; S1001. preprocessing the answer sheet; S101. performing OCR recognition on the divided answer area, and determining according to the confidence: the area with confidence greater than or equal to the first threshold value is marked as "recognition successful area", and enters the comparison link; otherwise, it is marked as "to be reasoned area"; S102. performing OCR recognition on the divided answer area of "to be reasoned area" and determining according to the confidence; S103. The area with confidence greater than or equal to the second threshold value is marked as "reasoning area", otherwise it is marked as "no reasoning value area" and pushed to the artificial comparison link, and the second threshold value is less than the first threshold value; S104. For "reasoning area", through comparing the handwritten text features and the standard answer text features, comparing the handwritten logical association and the standard answer logical association, the "reasoning processing area" image is obtained; S105. For the "reasoning processing area" image, the divided answer area is recognized by OCR and determined according to the confidence: the area with confidence greater than or equal to the first threshold value is marked as "recognition successful area" and enters the comparison link; otherwise, it is marked as "no reasoning value area" and pushed to the artificial comparison link; S106. For "recognition successful area" entering the comparison link, automatic marking is performed. The automatic marking method of the answer sheet of the application can be realized by a computer program, and the computer program for implementing the method of the application can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the computer programs make the functions / operations specified in the flowchart and / or block diagram be implemented when executed by the processor. The computer program can be executed completely on the machine, partially on the machine, partially on the machine and partially on a remote machine, or completely on a remote machine or server.

[0026] For S100. entering the standard answer of the corresponding test question of the answer sheet, storing the text features, logical association and position information of the standard answer on the answer sheet as the basis for recognition and reasoning; in the implementation, the standard answer of the corresponding test question of the answer sheet is entered, wherein the text content, stroke structure and morphological features of each option and the knowledge point association logic between the questions need to be entered for objective questions; the semantic association of core keywords and key sentences, the logical order of problem solving steps, and the stroke number and radical position features of the text need to be entered for subjective questions. At the same time, the position information of the area where the standard answer of each question is located is determined by scanning the blank answer sheet, including the corner coordinates of the area and the character spacing. These text features, logical associations and position information are uniformly stored to form a standard answer feature library, which provides a reference standard for subsequent OCR recognition, error reasoning and area division.

[0027] For S101. The divided answer area is subjected to OCR recognition, and is judged according to the confidence: the label of the answer area with a confidence greater than or equal to a first threshold value is "recognition successful area", and enters the comparison link; otherwise, it is marked as "to be reasoned area". In implementation, when the divided answer area is subjected to OCR recognition, multi-dimensional feature matching degrees are calculated synchronously as the basis for confidence evaluation. Firstly, the contour features of characters are extracted, and the normalized cross-correlation values thereof with standard character templates are calculated, denoted as CF; then, stroke structure features are analyzed, and the matching degrees of stroke quantity and intersection position features are counted, denoted as SF; and finally, the contextual semantic correlation degree is evaluated, and the cosine similarity of the recognized text and the expected content is calculated in combination with the expected semantics of the standard answer according to the question type, denoted as LF. The final confidence C is obtained by weighted fusion of the three indexes, and the calculation formula is C = w1 x CF + w2 x SF + w3 x LF, wherein w1, w2 and w3 are weight coefficients, and w1 + w2 + w3 = 1, and the specific values are dynamically adjusted according to the question type. For objective questions, the weight of SF is increased; and for subjective questions, the weight of LF is increased. In this way, the quantitative evaluation of the accuracy of the OCR recognition result is realized.

[0028] For S102. The answer area after division of the "to be reasoned area" is subjected to OCR recognition and secondary judgment according to the confidence, and in implementation, the answer area after re-division of the "to be reasoned area" is subjected to OCR recognition again, and the secondary confidence is calculated by using the multi-dimensional feature fusion mode. Firstly, the local stroke detail features of the handwritten characters, such as stroke width variation and endpoint shape, are extracted, and the feature matching rate thereof with the corresponding characters in the standard answer text feature library is calculated, denoted as FM; secondly, the position distribution features of the characters in the area are analyzed, and the layout matching degree is obtained by comparing the character layout with the standard answer area, denoted as PM; and then, according to the type of the question (objective question / subjective question) to which the area belongs, a question type adaptation coefficient is introduced, and the feature matching rate and the layout matching degree are weighted and adjusted. The calculation formula of the secondary confidence C2 is C2 = ku1 x FM + ku2 x PM, wherein ku1 and ku2 are weight coefficients, and ku1 + ku2 = 1, and in the objective question, ku1 weight is higher to highlight the features of the characters, and in the subjective question, ku2 weight is increased to take into account the rationality of the writing layout.

[0029] For S103. The answer region after the division of the "to-be-reasoned region" is subjected to OCR recognition and secondary determination according to the confidence, and the region with a confidence greater than or equal to a second threshold value is marked as "a region capable of reasoning", otherwise it is marked as "a region without reasoning value", and is pushed to the manual comparison link. In implementation, when the region is marked as "a region without reasoning value", it is pushed to the manual comparison link, and the detailed information of the region is associated, including the original OCR recognition result, the corresponding question number, the standard answer feature, the region image, and the basis for determining "a region without reasoning value". The manual comparison interface displays these information for easy viewing by the reviewer. The manual processing priority of the region is calculated, and the calculation formula of the priority P is P = m x (1-C_max) + n x s, where C_max is the highest value (one or two confidence) of the region in the confidence determination, s is the score ratio of the question in the test paper, and m and n are weight coefficients and m+n=1. Through this formula, the region with low confidence and high score is given priority to enter the manual processing queue, and the efficiency of manual comparison is improved. The reviewer completes the judgment according to the displayed information.

[0030] It should be noted that the second threshold value is less than the first threshold value.

[0031] In implementation, the first threshold value is determined based on the confidence calculation result of the first OCR recognition, and the normalized cross-correlation value CF of the character contour and the standard template, the stroke structure matching degree SF, and the context semantic association degree LF are integrated to obtain C1 = w1 x CF + w2 x SF + w3 x LF, where w1, w2, and w3 are weights and the sum is 1, SF weight is focused on objective questions, and LF weight is focused on subjective questions. The second threshold value is determined based on the confidence calculation result of the second OCR recognition, and the matching rate FM of the local stroke details of the handwritten character and the standard answer feature and the character layout matching degree PM are combined to obtain C2 = ku1 x FM + ku2 x PM, where ku1 and ku2 are weights and the sum is 1, FM weight is highlighted for objective questions, and PM weight is considered for subjective questions. Since the initial recognition quality of the "to-be-reasoned region" is low, the second confidence calculation focuses more on the features that can be corrected by reasoning, so the second threshold value is set to be less than the first threshold value, and the second threshold value is set to be less than the first threshold value in terms of representing the recognition quality of the answer sheet, for example, the second threshold value is set to 85%, and the first threshold value is set to 90%. In terms of representing the recognition quality of the answer sheet, more regions with poor initial recognition but reasoning correction value can be screened out, while avoiding the inclusion of regions that cannot be corrected by reasoning. Therefore, if there are many regions with poor OCR recognition but reasoning correction value in the answer sheet (such as some testees with messy handwriting, but the content is close to the standard answer), the present application can especially solve such problems, and can find the region with poor recognition but reasoning correction value.

[0032] For S104. For "reasoning area", by comparing the characteristics of handwritten characters and standard answer character features, comparing the logical association of handwritten characters and standard answer character features, and correcting, get "reasoning processing area" image; In the implementation, specifically, when comparing the characteristics of handwritten characters and standard answer character features, calculate the feature matching degree M of handwritten characters and standard answer characters, M = (the number of matched stroke features + the number of matched structure features) / (the total number of stroke features + the total number of structure features), wherein the matched stroke features include stroke length, inclination angle, and coincidence degree of start and end point coordinates, and the matched structure features include the overlap rate of character overall contour and the relative position deviation rate of radical. When comparing the logical association of handwritten characters and standard answer character features, the objective question calculates the logical fit degree L1 of the options = (the number of features that meet the standard answer option distribution rule + the number of cross-question option association matches) / (the total number of option features + the total number of cross-question association), and the subjective question calculates the semantic logical matching degree L2 = (keyword semantic similarity average × keyword weight + step logical node matching degree × step weight), wherein the keyword semantic similarity is calculated by word vector cosine distance, and the step logical node matching degree is obtained by weighting the step sequence coincidence rate and the causal association degree. Comprehensive correction index kuy = α × M + β × L1 or kuy = α × M + β × L2, (objective question α = 0.7, β = 0.3; subjective question α = 0.4, β = 0.6), when kuy≥0.85, directly replace the content corresponding to the highest matching degree of the standard answer feature; when 0.6≤kuy<0.85, complete or correct the characters based on the feature difference (such as completing the missing strokes and correcting the radical); when kuy<0.6, keep the original handwritten features and mark the difference area. The image formed after the above correction is the "reasoning processing area" image.

[0033] S105. For "reasoning processing area" image, the answer area after division is recognized by OCR and judged by confidence three times:

[0034] The confidence greater than or equal to the first threshold value is marked as "recognition successful area" and enters the comparison link; otherwise, it is marked as "not having reasoning value area" and pushed to the artificial comparison link; in the implementation, specifically, the OCR recognition of the answer area after division of the "reasoning processing area" image and the confidence evaluation of the first time of the OCR recognition of the answer area after division are consistent, and the multi-dimensional feature matching degree is calculated as the confidence evaluation basis. First, extract the contour features of the characters, calculate the normalized cross-correlation value with the standard character template; then analyze the stroke structure features, count the matching degree of stroke number and intersection point position features; then evaluate the context semantic association degree, calculate the cosine similarity of the recognized text and the expected content combined with the question type and the expected semantics of the standard answer. The final confidence is obtained by weighted fusion of the three indexes.

[0035] S106. For the entering alignment link "identify successful area" automatic marking. In implementation, specifically, when the entering alignment link "identify successful area" is automatically marked, first call the corresponding alignment rules in the standard answer feature library according to the type of the question:

[0036] For objective questions (such as multiple choice questions, true or false questions), the student answers (such as options "A" and "√") recognized in the region are extracted and directly matched with the options in the standard answer (such as the standard answer is "B" and "×"). If they are completely consistent, it is determined to be correct, and a "√" mark is automatically generated next to the region, and the matching basis (such as "matching standard answer option B") is recorded; if they are not consistent, it is determined to be incorrect, and an "×" mark is generated, and the record is "not matching standard answer option B, the recognition result is A".

[0037] For subjective questions (such as short answer questions, calculation questions):

[0038] For example: Analyze the core scoring points of the standard answer, including key keywords (such as "photosynthesis" and "Pythagorean theorem"), semantic association (such as "cause → result" logic), and step logic (such as "simplify first, then substitute"). Extract the text content of the student's answer, compare the keyword matching degree (such as "photosynthesis" and "photosynthesis" partially match due to a mistake), semantic similarity (calculate the correlation between the core semantics of the student's answer and the standard answer through a word vector model), and step integrity (such as whether the mathematical question contains the necessary steps "formula substitution" and "result calculation"). If the student's answer covers more than 80% of the key scoring points and the semantics and step logic are consistent, it is determined to be correct, a "√" mark is generated, and the record is "contains the key word 'photosynthesis' and the step logic is consistent"; if it only covers part of the scoring points, it is determined to be partially correct according to the proportion (such as assigning a score of "3 / 5"); if the key information is missing or the logic is contradictory, it is determined to be incorrect, an "×" mark is generated, and the record is "missing core keyword'mitochondria'".

[0039] All determination results are associated with the corresponding question and student number in real time, store the automatic marking track (including alignment process data), and generate a preliminary marking report in the system for subsequent manual review.

[0040] Before dividing the answer area, S1001. Preprocessing of the answer sheet, preprocessing of the answer sheet includes:

[0041] The answer sheet of the student is scanned and an image is generated, the student number, the question number sequence information are associated through the two-dimensional code, and the effective answer trace is reserved after the image is preprocessed. In the implementation, the student's answer sheet is scanned in batches using a high-definition scanner to generate a high-resolution digital image (resolution not less than 300 dpi), and the image format is JPEG or PNG. During the scanning process, the system automatically identifies the preset two-dimensional code on the answer sheet, which contains the unique number of the student, the class information, the examination subject, the test paper number and the question number sequence information (such as the arrangement order of each question type in the test paper, the question number corresponding to the score interval). After the system analyzes the two-dimensional code information, it is bound with the answer sheet image generated by scanning to establish a one-to-one index relationship, so that each image can be accurately associated with the corresponding student and test paper batch.

[0042] The generated image is preprocessed in multiple steps: first, image correction is performed, the positioning line of the edge of the answer sheet or the positioning point of the four corners is recognized, the image is detected for whether there is tilt or wrinkle, if there is tilt, the image is rotated to a horizontal state, if there is slight wrinkle, local flattening is performed through elastic deformation correction to ensure that the text line remains horizontal; second, image enhancement is performed, sharpening is used to enhance the stroke edges of the fuzzy area, so that the handwriting is clearer; then, noise reduction processing is performed, median filtering is used to remove random noise points (such as paper stains, scanning noise points) in the image, while the detail features of the handwriting strokes are preserved; finally, edge cutting processing is performed, according to the effective answer area range recorded in the standard answer feature library, the irrelevant frame, header and footer area in the image is cut off, only the core area containing the answer trace is reserved.

[0043] Before the divided answer area is subjected to OCR recognition, S1001. The answer sheet is preprocessed, and the preprocessing of the answer sheet includes:

[0044] According to the position information recorded in the standard answer feature library, the answer areas of each question are divided on the preprocessed answer sheet image; if there is deviation in the area, the question number text on the answer sheet is recognized to correct the boundary. In the implementation, according to the position information of the answer area of each question recorded in the standard answer feature library, including the upper left corner and the lower right corner coordinates of each question answer area on the blank answer sheet, the width and height of the area, the arrangement line spacing and column spacing of characters in the area, these parameters are used as a template to the preprocessed student answer sheet image. Through coordinate mapping, the coordinate system of the student answer sheet image and the blank answer sheet template is aligned, and the answer area corresponding to each question is automatically framed according to the template parameters, realizing preliminary division.

[0045] If there is a regional deviation after division, such as the answer region on the student answer sheet not matching the template position due to printing offset or scanning distortion, start the question number text recognition correction. First, extract the text content around the periphery of each preliminary division region, identify the question number text (such as "1." "(2)" "third question") in it through OCR, and extract the character features (such as number shape, punctuation symbol style, and text font) of these question number texts. Compare them with the pre-stored question number text features in the standard answer feature library to determine the actual question number corresponding to the recognized question number.

[0046] According to the matching result, the standard position information corresponding to the question in the standard answer feature library is retrieved, and the deviation amount (including the x-axis offset in the horizontal direction and the y-axis offset in the vertical direction) of the recognized question number position on the student answer sheet from the standard position is calculated. Based on this deviation amount, the system corrects the boundary of the answer region corresponding to the question, and through the translation and scaling of the boundary box, the answer region accurately covers the student's answer content, ensuring that the region division is consistent with the actual question position.

[0047] In addition, the values of the weight coefficients w1, w2, and w3 are based on the dynamic iterative optimization method of the weight coefficients proposed in this application: by analyzing the "correlation between feature performance and recognition accuracy" in historical recognition samples, the weights of contour features (CF), stroke structure features (SF), and semantic correlation degree (LF) are adjusted in real time, so that the confidence calculation is more consistent with the actual recognition rules in different scenarios (such as handwriting sloppiness and test paper cleanliness).

[0048] The dynamic calculation model of the weight coefficient is:

[0049] ; i = 1, 2, 3, which includes:

[0050] Basic weight (wᵢ0): The initial value is pre-set according to the question type (such as objective questions: w 10 = 0.2, w 20 = 0.6, w 30 = 0.2; subjective questions: w 10 = 0.1, w 20 = 0.3, w 30 = 0.6), as the starting point of iteration; k and j are feature index variables, used to distinguish different feature dimensions (i.e. contour features CF, stroke structure features SF, and semantic correlation degree LF); when k = 1 or j = 1, it corresponds to contour features (CF); when k = 2 or j = 2, it corresponds to stroke structure features (SF); when k = 3 or j = 3, it corresponds to semantic correlation degree (LF);

[0051] Feature reliability (Rᵢ): used to quantify the contribution of the feature to the recognition accuracy, the calculation formula is: ; To correctly identify the mean of feature i in the sample; To incorrectly identify the mean of feature i in the sample; To correctly identify the standard deviation of feature i in the sample, the larger the value, the stronger the ability of feature i to distinguish between true and false;

[0052] Feature redundancy coefficient γ kj : used to measure the correlation between feature k and feature j (value 0~1), avoid repeated calculation, feature redundancy coefficient γ kj The calculation formula is ; (corr is the Pearson correlation coefficient, k and j represent two different feature dimensions, for example:

[0053] γ 12 , k=1 represents the contour feature, j=2 represents the stroke structure feature, γ 12 , that is, the absolute value of the correlation coefficient of the two;

[0054] γ 32 , k=3 represents the semantic correlation degree, j=2 represents the stroke structure feature, γ 32 , that is, the absolute value of the correlation coefficient of the two features, if CF and SF are highly correlated, γ 12 tends to 1, at this time, the one with higher weight will be weakened).

[0055] Select a number of historical answer sheet samples (covering different degrees of illegibility, degrees of stains, and types of questions), and label "correct / incorrect" labels and corresponding CF, SF, and LF values;

[0056] Substitute the above formula, and get the first version of the weight (w1, w2, w3) based on the first batch of training data;

[0057] After processing 500 new answer sheets, automatically recalculate Rᵢ and γ kj , update the weight coefficient, and ensure that the model adapts to new scenarios (such as differences in printing quality of different batches of test papers during the examination season).

[0058] This method relates the actual recognition accuracy through "feature reliability Rᵢ", for example:

[0059] When the test paper is heavily stained, the R1 of the contour feature CF will decrease (the contour is blurred due to stains), at this time w1 will automatically decrease to avoid invalid feature interference; when the semantic complexity of subjective questions is high, the R3 of LF will increase (semantic correlation is more important), w3 will increase accordingly, which conforms to the logic of manual marking.

[0060] Through γ kjSolve the feature overlap problem (such as stroke structure SF and contour CF are highly related, automatically reduce the weight of one side), avoid the defect of "feature repeated calculation leads to high confidence" in traditional model.

[0061] Interpretability: Weight value no longer depends on artificial pre-setting, but is based on the objective law of "feature performance → recognition result" in training data, and each round of weight adjustment can be traced back to the feature contribution of specific samples (such as the average R2 of SF in a batch of samples is 0.8, which corresponds to the basis for increasing w2).

[0062] Deeply bind "feature performance and actual recognition effect", which not only solves the basis of weight value, but also breaks through the scene limitation of existing fixed weight model, has originality and engineering practice value.

[0063] It can be understood that the present application discloses an electronic device, comprising:

[0064] At least one processor; and

[0065] The memory is in communication connection with the at least one processor; wherein,

[0066] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the automatic marking method of the answer sheet.

[0067] The present application discloses an automatic marking system of an answer sheet, comprising:

[0068] The standard answer feature library module is used for storing the text features, logical associations and position information of the standard answers of the test questions as the basis for recognition and reasoning;

[0069] The answer sheet preprocessing module is used for converting paper answer sheets into high-quality digital images and accurately dividing the answer areas;

[0070] The OCR recognition and confidence evaluation module is used for multi-round OCR recognition of the answer areas, calculating the confidence through multi-dimensional feature calculation, and determining the area type (successful recognition / to be reasoned / no reasoning value);

[0071] The reasoning correction module is used for feature and logical comparison of "areas that can be reasoned" to correct OCR recognition errors;

[0072] The automatic marking module is used for automatically determining the correctness of the "successfully recognized areas" and assigning scores.

[0073] Finally should be noted: the above only for the preferred embodiments of the present application, and is not intended to limit the present application, although with reference to the foregoing embodiments of the present application has been described in detail, for those skilled in the art, it still can be modified, or part of the technical features of the equivalent replacement, which is recorded in the foregoing embodiments of the technical solutions, any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application, should be included within the scope of the present application.

[0074] The above are the preferred embodiments of the present application and not limited to the scope of the present application, therefore: any equivalent changes made in accordance with the structure, shape, principle of the present application, should be included within the scope of the present application.

Claims

1. Method for the automatic marking of answer sheets, characterized in that, The method comprises the steps of: entering the standard answer of the corresponding test question of the answer sheet, storing the text features, logical association and position information of the standard answer on the answer sheet; performing OCR identification on the divided answer area, and determining according to the confidence: if the confidence is greater than or equal to the first threshold, mark it as "successful recognition area", and enter the comparison link; otherwise, mark it as "to be inferred area"; performing OCR identification on the divided answer area of the "to be inferred area" and performing secondary determination according to the confidence: if the confidence is greater than or equal to the second threshold, mark it as "inferred area", otherwise, mark it as "no inference value area", and push it to the manual comparison link; The second threshold is less than the first threshold; for the "inferred area", the handwritten text features are compared with the standard answer text features, and the handwritten logical association is compared with the standard answer logical association, to obtain the "inferred processing area" image through correction; For the "inferred processing area" image, perform OCR identification on the divided answer area and perform third determination according to the confidence: if the confidence is greater than or equal to the first threshold, mark it as "successful recognition area", and enter the comparison link; otherwise, mark it as "no inference value area", and push it to the manual comparison link; for the "successful recognition area" entering the comparison link, automatically correct.

2. The method of claim 1, wherein, Before performing OCR identification on the divided answer area, the answer sheet is also preprocessed, including: scanning the answer sheet of the student's answer and generating a picture, associating the student number and question number sequence information through a two-dimensional code; pre-processing the picture to retain the effective answer traces.

3. The method of automatically marking an answer sheet according to claim 1, wherein, Before performing OCR identification on the divided answer area, the answer sheet is also preprocessed, including: According to the position information recorded in the standard answer feature library, divide the answer area of each question on the pre-processed answer sheet picture; if there is deviation in the area, correct the boundary by identifying the question number text on the answer sheet.

4. The method of claim 1, wherein, Enter the standard answer of the corresponding test question of the answer sheet, store the text features, logical association and position information of the standard answer on the answer sheet, including: enter the standard answer of the corresponding test question of the answer sheet, record the text content of each option, the stroke structure and morphological features of the characters, and the knowledge point association logic between questions; enter the semantic association of core keywords and key sentences, the logical sequence of problem solving steps, as well as the stroke number and component position features of the text, at the same time, scan the blank answer sheet to determine the position information of the area where the standard answer of each question is located, including the corner coordinates of the area and the character spacing, unify the storage of these text features, logical associations and position information, and form a standard answer feature library.

5. The method of automatically marking answer sheets of claim 1, wherein, When performing OCR identification on the divided answer area, the multi-dimensional feature matching degree is calculated synchronously as the confidence evaluation basis; first, extract the contour features of the characters, and calculate the normalized cross-correlation value with the standard character template; Then analyze the stroke structure features, and calculate the matching degree of the stroke number and intersection position features; Then evaluate the context semantic association degree, calculate the cosine similarity between the recognized text and the expected content according to the question type and the expected semantics of the standard answer; and finally, the final confidence is obtained by weighted fusion of the three indicators.

6. The method of claim 1, wherein, The answer region after re-partitioning the "to-be-reasoned region" is subjected to OCR recognition again, and a multi-dimensional feature fusion method is used to calculate the secondary confidence; first, the local stroke detail features of the handwritten characters are extracted, and the feature matching rate with the corresponding characters in the standard answer text feature library is calculated; second, the position distribution features of the characters in the region are analyzed and compared with the character layout of the standard answer region to obtain the layout matching degree; and then according to the type of the region belonging to the question, a question type adaptation coefficient is introduced, and the secondary confidence is calculated by weighting the feature matching rate and the layout matching degree together.

7. The method of automatically grading answer sheets of claim 1 wherein, The pushing to the artificial comparison link also includes calculating the artificial processing priority of the "region without reasoning value", so that the regions with low confidence and high score are preferentially entered into the artificial processing queue.

8. The method of claim 1, wherein, The second threshold is set to be less than the first threshold, and the second threshold is set to be less than the first threshold in terms of the quality of the answer sheet recognition.

9. The method of automatically grading answer sheets of claim 1 wherein, When comparing the handwritten character features with the standard answer text features, the feature matching degree M of the handwritten characters and the standard answer characters is calculated, M = (number of matched stroke features + number of matched structure features) / (total number of stroke features + total number of structure features), wherein the matched stroke features include stroke length, inclination angle, and coincidence degree of start and end point coordinates, and the matched structure features include overlap rate of character overall contour and relative position deviation rate of radical and component; when comparing the handwritten logical association with the standard answer logical association, the option logical matching degree L1 of objective questions is calculated as L1 = (number of features conforming to the standard answer option distribution rule + number of cross-question option association matches) / (total number of option features + total number of cross-question associations), and the semantic logical matching degree L2 of subjective questions is calculated as L2 = (average keyword semantic similarity × keyword weight + step logical node matching degree × step weight), wherein the keyword semantic similarity is calculated by word vector cosine distance, and the step logical node matching degree is obtained by weighting the step order coincidence rate and the causal association degree; The comprehensive correction index is determined to form the image comprehensive correction index kuy = α × M + β × L1 or kuy = α × M + β × L2 after correction.

10. An electronic device comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the automatic marking method of the answer sheet of claim 1. ​

Images