Handwritten blank-filling question correcting method and device and related equipment
By character recognition and multimodal data comparison of the images of handwritten fill-in-the-blank questions, the problem of low accuracy of handwritten fill-in-the-blank questions in the prior art is solved, and the accuracy and efficiency of correction are improved.
Patent Information
- Application Number
- CN202311369455.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-20
- Publication Date
- 2025-04-29
AI Technical Summary
The existing automated correction methods have low accuracy when identifying handwritten fill-in-the-blank questions, especially when dealing with handwritten in irregular writing or sloppy handwriting, misjudgment and misjudgment are prone to occur.
By initializing the preset tool-type objects, we obtain target-type objects with character recognition capabilities and multimodal data comparison capabilities. This object uses the object to characterize the image of the handwritten fill-in-the-blank question, obtain the recognition results, and compare it with the preset answer text, and finally correct it based on the comparison results.
It improves the accuracy of identification of handwritten fill-in-the-blank questions, reduces misjudgments and misjudgments, and improves the accuracy of correcting answer sheets.
Smart Images
Figure CN119904876A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of electronic marking, and more specifically, to a method, device and related equipment for marking handwritten fill-in-the-blank questions. Background Art
[0002] In recent years, with the rapid development of Internet technology and artificial intelligence, the field of education has also begun to try to integrate various intelligent technologies into classroom teaching, curriculum design, student assessment, etc. Especially in the aspect of grading students' homework, automated grading technology is expected to reduce the workload of teachers and improve the efficiency and accuracy of grading.
[0003] As a common question type, fill-in-the-blank questions have an urgent need for automated grading. Existing automated grading methods are mainly targeted at printed fonts and standard handwritten fonts, but the recognition and grading effects of handwritten fill-in-the-blank questions are not ideal, and face great challenges.
[0004] First, the existing automated grading methods are limited by factors such as the writer's handwriting, strokes, and writing speed when recognizing handwritten characters, which greatly affects the recognition accuracy. Especially when dealing with handwriting that is not standardized or has sloppy handwriting, the recognition effect is poor, and it is easy to make misjudgments and missed judgments.
[0005] Secondly, handwritten fill-in-the-blank questions involve a variety of fonts, sizes, formats, etc., and existing automated grading methods often have difficulty adapting to these diversities. In addition, spaces and punctuation marks in fill-in-the-blank questions can easily interfere with recognition and grading.
[0006] Therefore, for the automated grading method of handwritten fill-in-the-blank questions, it is necessary to study a more accurate technology to solve the problem of poor recognition effect when processing handwritten fill-in-the-blank questions. Summary of the invention
[0007] In view of this, the first aspect of the present application provides a method for correcting handwritten fill-in-the-blank questions, comprising:
[0008] Initializing a preset tool class object to obtain a target class object, wherein the target class object is configured to have character recognition capability and multimodal data comparison capability;
[0009] Using the target class object to perform character recognition on a target image to obtain a recognition result, the target image is an image of a handwritten fill-in-the-blank answer sheet;
[0010] Compare the recognition result with a preset answer text to obtain a comparison result;
[0011] Based on the comparison result, the target image, the recognition result and the answer text are processed using the target class object to obtain a correction result.
[0012] Preferably, the process of initializing the preset tool class object includes:
[0013] Initialize an optical character recognition (OCR) model object, wherein the OCR model object is used to perform character recognition on an input image and output a character string;
[0014] Initialize a device object, the device object is used to perform basic operations;
[0015] Initialize a multimodal model object, which is used to calculate the similarity between text and image.
[0016] Preferably, the process of performing character recognition on the target image using the target class object to obtain a recognition result includes:
[0017] Convert the target image into an array of a preset type, and input the array into the initialized OCR model object;
[0018] The array is processed using the initialized OCR model object to obtain a recognition result, which is a null value or a character string.
[0019] Preferably, the process of comparing the recognition result with a preset answer text to obtain the comparison result includes:
[0020] Determining whether the recognition result is consistent with a preset answer text;
[0021] If so, the comparison result is determined to be consistent;
[0022] If not, the comparison result is determined to be inconsistent.
[0023] Preferably, based on the comparison result, the target image, the recognition result and the answer text are processed using the target class object to obtain the correction result, including:
[0024] If the comparison result is consistent, the correction result is determined to be correct;
[0025] If the comparison result is inconsistent, the target image, the recognition result and the answer text are processed using the initialized multimodal model object to obtain a correction result.
[0026] Preferably, the process of processing the target image, the recognition result and the answer text using the initialized multimodal model object to obtain the correction result includes:
[0027] Using the initialized multimodal model object, obtaining a first similarity between the recognition result and the target image, and obtaining a second similarity between the target image and the answer text;
[0028] If the first similarity is greater than the second similarity, determining the correction result as an error;
[0029] If the first similarity is less than the second similarity, the correction result is determined to be correct.
[0030] Preferably, the process of initializing the preset tool class object also includes:
[0031] Initialize a preprocessing object of the multimodal model, wherein the preprocessing object is used to embed images and texts into preset vectors;
[0032] The process of obtaining a first similarity between the recognition result and the target image using the initialized multimodal model object, and obtaining a second similarity between the target image and the answer text, comprises:
[0033] Embedding the recognition result and the target image into a first vector using the initialized preprocessing object;
[0034] embedding the target image and the answer text into a second vector using the preprocessing object;
[0035] The first vector and the second vector are input into the initialized multimodal model object to obtain a first similarity and a second similarity.
[0036] The second aspect of the present application provides a handwritten fill-in-the-blank question correction device, comprising:
[0037] An initialization unit, used to initialize a preset tool class object to obtain a target class object, wherein the target class object is configured to have a character recognition capability and a multimodal data comparison capability;
[0038] A character recognition unit, used to perform character recognition on a target image using the target class object to obtain a recognition result, wherein the target image is an image of a handwritten fill-in-the-blank answer sheet;
[0039] An answer comparison unit is used to compare the recognition result with a preset answer text to obtain a comparison result;
[0040] A result determination unit is used to process the target image, the recognition result and the answer text using the target class object based on the comparison result to obtain a correction result.
[0041] A third aspect of the present application provides a handwritten fill-in-the-blank question correction device, comprising: a memory and a processor;
[0042] The memory is used to store programs;
[0043] The processor is used to execute the program to implement the various steps of the above-mentioned handwritten fill-in-the-blank question correction method.
[0044] A fourth aspect of the present application provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, each step of the above-mentioned method for correcting handwritten fill-in-the-blank questions is implemented.
[0045] It can be known from the above technical solution that the present application first initializes the preset tool class object to obtain the target class object. Through the initialization operation, the tool class object can be adapted to the current application scenario. Among them, the target class object is configured to have character recognition capability and multimodal data comparison capability. Then, the target class object is used to perform character recognition on the target image to obtain a recognition result. Among them, the target image is an image of a handwritten fill-in-the-blank answer sheet, for example, an image obtained after scanning the handwritten fill-in-the-blank answer sheet. Then, the recognition result is compared with the preset answer text to obtain a comparison result. Among them, the comparison result may include that the recognition result is consistent with the answer text or that the recognition result is inconsistent with the answer text. Finally, based on the comparison result, the target class object is used to process the target image, the recognition result and the answer text to obtain a correction result. By reviewing the target class object, the misjudgment caused by character recognition errors is avoided to a certain extent, the accuracy of answer sheet correction is improved, and the recognition effect of handwritten fill-in-the-blank questions is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0047] Figure 1 A schematic diagram of a handwritten fill-in-the-blank question correction method disclosed in an embodiment of the present application;
[0048] Figure 2 An example of initializing the tool class object disclosed in the embodiment of the present application is given;
[0049] Figure 3 The handwritten characters to be recognized disclosed in the embodiments of the present application are illustrated;
[0050] Figure 4 The process of reviewing the results disclosed in the embodiments of the present application is illustrated;
[0051] Figure 5 The process of processing using a multimodal model object disclosed in the embodiment of the present application is illustrated;
[0052] Figure 6 An example of initializing the tool class object disclosed in the embodiment of the present application is given;
[0053] Figure 7 A schematic diagram of a handwritten fill-in-the-blank question correction device disclosed in an embodiment of the present application;
[0054] Figure 8 This is a schematic diagram of the handwritten fill-in-the-blank question correction device disclosed in an embodiment of the present application. DETAILED DESCRIPTION
[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0056] The following describes the handwritten fill-in-the-blank question correction method provided by the embodiment of the present application. Figure 1 The handwritten fill-in-the-blank question correction method provided in the embodiment of the present application may include the following steps:
[0057] Step S101, initializing a preset tool class object to obtain a target class object.
[0058] Among them, in order to realize the correction of handwritten fill-in-the-blank questions, a variety of capabilities are needed, such as character recognition capability, image data and text data comparison capability, etc. Due to the need to consider factors such as mobility and decoupling, this application integrates them into a tool class. Therefore, it needs to be initialized before use to adapt to different application scenarios. It can be understood that the target class object is an initialized tool class object, which is configured to have character recognition capability and multimodal data comparison capability.
[0059] Step S102: using the target class object to perform character recognition on the target image to obtain a recognition result.
[0060] The target image is an image of a handwritten fill-in-the-blank answer sheet, for example, it may be an image obtained by scanning the handwritten fill-in-the-blank answer sheet.
[0061] Step S103, comparing the recognition result with a preset answer text to obtain a comparison result.
[0062] For example, the comparison result can be obtained by determining whether the recognition structure is consistent with the answer text.
[0063] Step S104: Based on the comparison result, the target image, the recognition result and the answer text are processed using the target class object to obtain a correction result.
[0064] Exemplarily, the multimodal data comparison capability in the target class object is used to compare the comparison result with the target image, and the target image with the answer text, and to determine whether the target image is closer to the answer text or the recognition result. If the target image is closer to the answer text, the answer is confirmed to be correct; if the target image is closer to the recognition result, whether the answer is correct or wrong is determined based on the comparison result in step S103.
[0065] The present application first initializes the preset tool class object to obtain the target class object. Through the initialization operation, the tool class object can be adapted to the current application scenario. Among them, the target class object is configured to have character recognition capabilities and multimodal data comparison capabilities. Then, the target class object is used to perform character recognition on the target image to obtain a recognition result. Among them, the target image is an image of a handwritten fill-in-the-blank answer sheet, for example, an image obtained after scanning the handwritten fill-in-the-blank answer sheet. Then, the recognition result is compared with the preset answer text to obtain a comparison result. Among them, the comparison result may include that the recognition result is consistent with the answer text or that the recognition result is inconsistent with the answer text. Finally, based on the comparison result, the target class object is used to process the target image, the recognition result and the answer text to obtain a correction result. By reviewing the target class object, the misjudgment caused by character recognition errors is avoided to a certain extent, the accuracy of answer sheet correction is improved, and the recognition effect of handwritten fill-in-the-blank questions is improved.
[0066] In some embodiments of the present application, Figure 2 As shown, the process of initializing the preset tool class object in step S101 may include:
[0067] S1, initialize the OCR (Optical Character Recognition) model object.
[0068] The OCR model object is used to perform character recognition on the input image and output the corresponding string.
[0069] S2, initialize the device object.
[0070] The device object is used to perform basic operations. Specifically, the initialization of the device object is to specify what device is used for data processing of this class, such as cpu, cuda, etc.
[0071] S3, initialize the multimodal model object.
[0072] The multimodal model object is used to calculate the similarity between text and images.
[0073] In some embodiments of the present application, the process of performing character recognition on the target image using the target class object to obtain a recognition result in step S102 may include:
[0074] S1, converting the target image into an array of a preset type, and inputting the array into an initialized OCR model object.
[0075] Exemplarily, the target image is converted into a numpy.array, and after the conversion is completed, it is input into the initialized OCR model object.
[0076] S2, using the initialized OCR model object to process the array to obtain a recognition result.
[0077] It should be noted that the recognition result can be a null value or a string.
[0078] In some embodiments of the present application, the process of comparing the recognition result with the preset answer text in step S103 to obtain the comparison result may include:
[0079] S1, determining whether the recognition result is consistent with a preset answer text.
[0080] If yes, execute S2; if no, execute S3.
[0081] S2, determining the comparison result as consistent.
[0082] At this time, the comparison result may be directly adopted, and the comparison result of "consistent" may be added to the result set.
[0083] S3, determining the comparison result as inconsistent.
[0084] In practical applications, OCR has a good recognition effect on clear pictures. However, if the picture resolution is very poor or the strokes are connected or unclear, it is very likely to make mistakes. Figure 3 The handwritten word shown is "nationally", but due to resolution and connected strokes, OCR is interpreted as "rationally". To avoid misjudgment, Figure 4 As shown, further results inspection is required for cases where the comparison results are "inconsistent".
[0085] Based on this, in some embodiments of the present application, Figure 5 As shown, step S104 processes the target image, the recognition result and the answer text based on the comparison result using the target class object to obtain the correction result, which may include:
[0086] S1, determining whether the comparison result is consistent or inconsistent.
[0087] If yes, go to S2; if no, go to S3.
[0088] S2, determining the correction result as correct.
[0089] S3, using the initialized multimodal model object to process the target image, the recognition result and the answer text to obtain a correction result.
[0090] In some embodiments of the present application, the process of using the initialized multimodal model object in S2 to process the target image, the recognition result, and the answer text to obtain the correction result may include:
[0091] S21, using the initialized multimodal model object to obtain a first similarity between the recognition result and the target image, and to obtain a second similarity between the target image and the answer text.
[0092] S22: If the first similarity is greater than the second similarity, the correction result is determined to be wrong.
[0093] If the first similarity is greater than the second similarity, it means that the target image is closer to the recognition result. Therefore, the comparison result in step S103 is used to determine the correction result as an error.
[0094] S23: If the first similarity is less than the second similarity, the correction result is determined to be correct.
[0095] If the first similarity is smaller than the second similarity, it means that the target image is closer to the answer text, and therefore, the comparison result in step S103 is not adopted, and the correction result is corrected to be correct.
[0096] In some embodiments of the present application, Figure 6 As shown, the process of initializing the preset tool class object in step S101 may also include:
[0097] S4, initialize the preprocessing objects of the multimodal model.
[0098] The preprocessing object is used to embed images and texts into preset vectors so as to subsequently input them into the multimodal model object.
[0099] The process of obtaining the first similarity between the recognition result and the target image and the second similarity between the target image and the answer text using the initialized multimodal model object in S21 may include:
[0100] S211, using the initialized preprocessing object to embed the recognition result and the target image into a first vector.
[0101] S212: Embed the target image and the answer text into a second vector using the preprocessing object.
[0102] S213: Input the first vector and the second vector into the initialized multimodal model object to obtain a first similarity and a second similarity.
[0103] The handwritten fill-in-the-blank question correction device provided in an embodiment of the present application is described below. The handwritten fill-in-the-blank question correction device described below and the handwritten fill-in-the-blank question correction method described above can be referred to each other.
[0104] See also Figure 7 The handwritten fill-in-the-blank question correction device provided in the embodiment of the present application may include:
[0105] An initialization unit 21 is used to initialize a preset tool class object to obtain a target class object, wherein the target class object is configured to have a character recognition capability and a multimodal data comparison capability;
[0106] A character recognition unit 22, used to perform character recognition on a target image using the target class object to obtain a recognition result, wherein the target image is an image of a handwritten fill-in-the-blank answer sheet;
[0107] The answer comparison unit 23 is used to compare the recognition result with a preset answer text to obtain a comparison result;
[0108] The result determination unit 24 is used to process the target image, the recognition result and the answer text using the target class object based on the comparison result to obtain a correction result.
[0109] In some embodiments of the present application, the process of initializing the preset tool class object by the initialization unit 21 may include:
[0110] Initialize an optical character recognition (OCR) model object, wherein the OCR model object is used to perform character recognition on an input image and output a character string;
[0111] Initialize a device object, the device object is used to perform basic operations;
[0112] Initialize a multimodal model object, which is used to calculate the similarity between text and image.
[0113] In some embodiments of the present application, the character recognition unit 22 uses the target class object to perform character recognition on the target image to obtain a recognition result, which may include:
[0114] Convert the target image into an array of a preset type, and input the array into the initialized OCR model object;
[0115] The array is processed using the initialized OCR model object to obtain a recognition result, which is a null value or a character string.
[0116] In some embodiments of the present application, the process in which the answer comparison unit 23 compares the recognition result with a preset answer text to obtain the comparison result may include:
[0117] Determining whether the recognition result is consistent with a preset answer text;
[0118] If so, the comparison result is determined to be consistent;
[0119] If not, the comparison result is determined to be inconsistent.
[0120] In some embodiments of the present application, the result determination unit 24 processes the target image, the recognition result, and the answer text based on the comparison result using the target class object to obtain the correction result, which may include:
[0121] If the comparison result is consistent, the correction result is determined to be correct;
[0122] If the comparison result is inconsistent, the target image, the recognition result and the answer text are processed using the initialized multimodal model object to obtain a correction result.
[0123] In some embodiments of the present application, the result determination unit 24 processes the target image, the recognition result, and the answer text using the initialized multimodal model object to obtain a correction result, including:
[0124] Using the initialized multimodal model object, obtaining a first similarity between the recognition result and the target image, and obtaining a second similarity between the target image and the answer text;
[0125] If the first similarity is greater than the second similarity, determining the correction result as an error;
[0126] If the first similarity is less than the second similarity, the correction result is determined to be correct.
[0127] In some embodiments of the present application, the process of initializing the preset tool class object by the initialization unit 21 may further include:
[0128] Initialize a preprocessing object of the multimodal model, wherein the preprocessing object is used to embed images and texts into preset vectors;
[0129] The process in which the result determination unit 24 uses the initialized multimodal model object to obtain a first similarity between the recognition result and the target image, and obtains a second similarity between the target image and the answer text may include:
[0130] Embedding the recognition result and the target image into a first vector using the initialized preprocessing object;
[0131] embedding the target image and the answer text into a second vector using the preprocessing object;
[0132] The first vector and the second vector are input into the initialized multimodal model object to obtain a first similarity and a second similarity.
[0133] The handwriting fill-in-the-blank question correction device provided in the embodiment of the present application can be applied to handwriting fill-in-the-blank question correction equipment, such as a computer, etc. Optionally, Figure 8 The hardware structure diagram of the handwritten fill-in-the-blank question correction device is shown. Figure 8 The hardware structure of the handwritten fill-in-the-blank question correction device may include: at least one processor 31, at least one communication interface 32, at least one memory 33 and at least one communication bus 34.
[0134] In the embodiment of the present application, the number of the processor 31, the communication interface 32, the memory 33, and the communication bus 34 is at least one, and the processor 31, the communication interface 32, and the memory 33 communicate with each other through the communication bus 34;
[0135] The processor 31 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application, etc.;
[0136] The memory 33 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;
[0137] The memory 33 stores a program, and the processor 31 can call the program stored in the memory 33, and the program is used to:
[0138] Initializing a preset tool class object to obtain a target class object, wherein the target class object is configured to have character recognition capability and multimodal data comparison capability;
[0139] Using the target class object to perform character recognition on a target image to obtain a recognition result, the target image is an image of a handwritten fill-in-the-blank answer sheet;
[0140] Compare the recognition result with a preset answer text to obtain a comparison result;
[0141] Based on the comparison result, the target image, the recognition result and the answer text are processed using the target class object to obtain a correction result.
[0142] Optionally, the detailed functions and extended functions of the program may refer to the above description.
[0143] The embodiment of the present application further provides a storage medium, which may store a program suitable for execution by a processor, wherein the program is used to:
[0144] Initializing a preset tool class object to obtain a target class object, wherein the target class object is configured to have character recognition capability and multimodal data comparison capability;
[0145] Using the target class object to perform character recognition on a target image to obtain a recognition result, the target image is an image of a handwritten fill-in-the-blank answer sheet;
[0146] Compare the recognition result with a preset answer text to obtain a comparison result;
[0147] Based on the comparison result, the target image, the recognition result and the answer text are processed using the target class object to obtain a correction result.
[0148] Optionally, the detailed functions and extended functions of the program may refer to the above description.
[0149] In summary:
[0150] The present application first initializes the preset tool class object to obtain the target class object. Through the initialization operation, the tool class object can be adapted to the current application scenario. Among them, the target class object is configured to have character recognition capabilities and multimodal data comparison capabilities. Then, the target class object is used to perform character recognition on the target image to obtain a recognition result. Among them, the target image is an image of a handwritten fill-in-the-blank answer sheet, for example, an image obtained after scanning the handwritten fill-in-the-blank answer sheet. Then, the recognition result is compared with the preset answer text to obtain a comparison result. Among them, the comparison result may include that the recognition result is consistent with the answer text or that the recognition result is inconsistent with the answer text. Finally, based on the comparison result, the target class object is used to process the target image, the recognition result and the answer text to obtain a correction result. By reviewing the target class object, the misjudgment caused by character recognition errors is avoided to a certain extent, the accuracy of answer sheet correction is improved, and the recognition effect of handwritten fill-in-the-blank questions is improved.
[0151] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0152] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.
[0153] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for correcting handwritten fill-in-the-blank questions, characterized in that: include: Initializing a preset tool class object to obtain a target class object, wherein the target class object is configured to have character recognition capability and multimodal data comparison capability; Using the target class object to perform character recognition on a target image to obtain a recognition result, the target image is an image of a handwritten fill-in-the-blank answer sheet; Comparing the recognition result with a preset answer text to obtain a comparison result; Based on the comparison result, the target image, the recognition result and the answer text are processed using the target class object to obtain a correction result.
2. The method according to claim 1, characterized in that The process of initializing the preset tool class object includes: Initialize an optical character recognition (OCR) model object, wherein the OCR model object is used to perform character recognition on an input image and output a character string; Initialize a device object, the device object is used to perform basic operations; Initialize a multimodal model object, which is used to calculate the similarity between text and image.
3. The method according to claim 2, characterized in that The process of using the target class object to perform character recognition on the target image to obtain a recognition result includes: Convert the target image into an array of a preset type, and input the array into the initialized OCR model object; The array is processed using the initialized OCR model object to obtain a recognition result, which is a null value or a character string.
4. The method according to claim 1, characterized in that: The process of comparing the recognition result with a preset answer text to obtain a comparison result includes: Determining whether the recognition result is consistent with a preset answer text; If so, the comparison result is determined to be consistent; If not, the comparison result is determined to be inconsistent.
5. The method according to claim 2, characterized in that: Based on the comparison result, the target image, the recognition result and the answer text are processed by using the target class object to obtain the correction result, including: If the comparison result is consistent, the correction result is determined to be correct; If the comparison result is inconsistent, the target image, the recognition result and the answer text are processed using the initialized multimodal model object to obtain a correction result.
6. The method according to claim 5, characterized in that The process of using the initialized multimodal model object to process the target image, the recognition result and the answer text to obtain the correction result includes: Using the initialized multimodal model object, obtaining a first similarity between the recognition result and the target image, and obtaining a second similarity between the target image and the answer text; If the first similarity is greater than the second similarity, determining the correction result as an error; If the first similarity is less than the second similarity, the correction result is determined to be correct.
7. The method according to claim 6, characterized in that The process of initializing the preset tool class object also includes: Initialize a preprocessing object of the multimodal model, wherein the preprocessing object is used to embed images and texts into preset vectors; The process of obtaining a first similarity between the recognition result and the target image using the initialized multimodal model object, and obtaining a second similarity between the target image and the answer text, comprises: Embedding the recognition result and the target image into a first vector using the initialized preprocessing object; embedding the target image and the answer text into a second vector using the preprocessing object; The first vector and the second vector are input into the initialized multimodal model object to obtain a first similarity and a second similarity.
8. A handwritten fill-in-the-blank question correction device, characterized in that: include: An initialization unit, used to initialize a preset tool class object to obtain a target class object, wherein the target class object is configured to have a character recognition capability and a multimodal data comparison capability; A character recognition unit, used to perform character recognition on a target image using the target class object to obtain a recognition result, wherein the target image is an image of a handwritten fill-in-the-blank answer sheet; An answer comparison unit is used to compare the recognition result with a preset answer text to obtain a comparison result; A result determination unit is used to process the target image, the recognition result and the answer text using the target class object based on the comparison result to obtain a correction result.
9. A handwritten fill-in-the-blank question correction device, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the handwritten fill-in-the-blank question correction method according to any one of claims 1 to 7.
10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the handwritten fill-in-the-blank question marking method according to any one of claims 1 to 7 is implemented.