A method and device for automatically matching batch uploaded data
Through overall text recognition, regional text recognition after keywords, and word recognition probability matrix calculation of handwritten single-word images, the problems of high standardization requirements for image upload and limitations in the existing technology are solved, and the problem of high standards and limitations in applicable scenarios is realized in multiple scenarios. Effective answer sheet matching.
Patent Information
- Application Number
- CN202410729906.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-06
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2044-06-06
AI Technical Summary
The existing automatic answer sheet matching technology has high standardization requirements for image uploads and is limited in applicable scenarios, so it cannot be effectively matched in irregular photo upload scenarios.
Three recognition methods are used to match in sequence: overall text recognition, regional text recognition after keywords, and word recognition probability matrix calculation of handwritten single-word images. Matching through the shortest editing distance and matching probability, suitable for irregular and relatively standardized photography Upload the scene.
It realizes effective matching in relatively irregular photo upload scenes and relatively standard scan upload scenes, and is suitable for various types of photo upload scenes, improving the applicability and accuracy of matching.
Smart Images

Figure CN118470726B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of text recognition and matching, and in particular to a method and device for automatically matching batch uploaded data. Background Art
[0002] Currently, there are many situations where images need to be matched with corresponding data in batches. Take the test answer sheet as an example. The answer sheet records the candidate's handwritten name, student number and other information. After the answer sheet is submitted, it is necessary to take a photo of the answer sheet and upload it to the corresponding system, and match the answer sheet image with the corresponding candidate after marking.
[0003] Some of the automatic answer sheet matching solutions currently on the market require answer sheet templates to be prepared in advance, and the style of the answer sheet must be known in advance, and the name and student number area must be demarcated; some use deep learning layout analysis methods to locate the name and student number area, but no matter which method is used, there are the following problems: there are relatively high requirements for the standardization of image uploading, and the applicable scenarios are relatively limited. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide a method and device for automatic matching of batch uploaded data, so as to solve the problem that the recognition and matching method in the prior art has relatively high requirements on the standardization of image uploading and has relatively limited applicable scenarios.
[0005] According to a first aspect of an embodiment of the present invention, a method for automatically matching batch uploaded data is provided, comprising:
[0006] S1, obtaining an image to be uploaded, performing rotation correction on the image to be uploaded, intercepting a preset area of the rotation-corrected image, and performing overall text recognition on the intercepted preset area;
[0007] S2, comparing the overall text recognition result with a plurality of pre-stored items of information to be matched, if there is any item of information to be matched in the overall text recognition result, matching the image to be uploaded with the item of information to be matched, otherwise executing step S3;
[0008] S3, extracting the regional text recognition result after the preset keyword in the overall text recognition result, calculating the shortest edit distance between the regional text recognition result and each item information to be matched, if the shortest edit distance is less than the preset value, matching the image to be uploaded with the item information to be matched, otherwise executing step S4;
[0009] S4. Cut out the area image after the preset keyword from the preset area, segment the area image into single words, and obtain a plurality of handwritten single-word images; calculate the word recognition probability matrix of each handwritten single-word image, and calculate the average probability matrix of the area image; obtain the matching probability of the area image corresponding to each item information to be matched according to the average probability matrix, and if the maximum matching probability is greater than a preset threshold, match the image to be uploaded with the item information to be matched.
[0010] Preferably, the method for automatically matching batch uploaded data further includes:
[0011] If after executing step S4, the image to be uploaded does not match, then execute step S5;
[0012] S5. Integrate multiple items of information to be matched whose shortest edit distance in step S3 is the same and smaller than a preset value into a candidate set; calculate the replacement probability of each item of information to be matched in the candidate set and the regional image; if the maximum replacement probability among the replacement probabilities is greater than a preset threshold, match the image to be uploaded with the item of information to be matched.
[0013] Preferably, after extracting the regional text recognition results after the preset keyword in the overall text recognition results, step S3 further includes:
[0014] Determine whether the regional text recognition result has a value;
[0015] If not, it is concluded that the image to be uploaded fails to match.
[0016] Preferably, before extracting the regional text recognition results after the preset keyword in the overall text recognition results, step S3 further includes:
[0017] Detecting whether there is a preset keyword in the overall text recognition result;
[0018] If not, the image to be uploaded and the last image to be uploaded are matched to the same item information to be matched.
[0019] Preferably, the step S4 of segmenting the regional image into individual words to obtain a plurality of handwritten individual word images comprises:
[0020] Calculate and obtain a pixel histogram in the vertical direction of the regional image, and determine an initial segmentation position according to the number of characters in the name in the list to be matched;
[0021] Starting from each initial segmentation position, traverse leftward and rightward at the same time to find the minimum value position in the vertical pixel histogram, and use the minimum value position as the final segmentation point.
[0022] Preferably, before calculating and obtaining the pixel histogram in the vertical direction of the regional image, the method further includes:
[0023] The regional image is binarized, and blank images above, below, left and right of the image are intercepted through the distribution of pixel projection histograms in horizontal and vertical directions, so that the regional image is close to the actual text area.
[0024] Preferably, step S2 further includes:
[0025] If there are multiple items of information to be matched in the overall text recognition result, the item of information to be matched that appears first in the overall text recognition result is matched with the image to be uploaded.
[0026] According to a second aspect of an embodiment of the present invention, a device for automatically matching batch uploaded data is provided, including:
[0027] A main controller, and a memory connected to the main controller;
[0028] a memory in which program instructions are stored;
[0029] The main controller is used to execute program instructions stored in the memory to perform any of the above methods.
[0030] The technical solution provided by the embodiments of the present invention may have the following beneficial effects:
[0031] It is understandable that the technical solution provided by the present invention uses three recognition methods to match in sequence. First, the overall text recognition is compared and matched with multiple items of information to be matched; then the regional text recognition result after the keyword is used to calculate the shortest edit distance between it and each item of information to be matched, and then match it; finally, the word recognition probability matrix is calculated for the handwritten regional image, and the matching probability between it and each item of information to be matched is calculated. It is understandable that the technical solution shown in the present invention is applicable to both relatively non-standard photo upload scenarios and relatively standard scan upload scenarios, and is applicable to various types of photo upload matching scenarios.
[0032] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0034] Figure 1 The figure is a schematic diagram of the steps of a method for automatic matching of batch uploaded data according to an exemplary embodiment. DETAILED DESCRIPTION
[0035] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.
[0036] Embodiment 1
[0037] Figure 1 is a schematic diagram of the steps of a method for automatic matching of batch uploaded data according to an exemplary embodiment, see Figure 1 , providing a method for automatic matching of batch uploaded data, including:
[0038] S1. Acquire an image to be uploaded, perform rotation correction on the image to be uploaded, intercept a preset area of the rotation-corrected image, and perform overall text recognition on the intercepted preset area.
[0039] In specific practice, the technical solution shown in this embodiment can be applied to a variety of scenarios, such as answer sheets for exams, taking photos and uploading students' homework, etc. This embodiment is explained using the scenario of answer sheets for exams as an example.
[0040] First, the answer sheet image is obtained. For the corrected answer sheet image, it is considered that the area for filling in the name and student number is located in the upper quarter of the image. Therefore, the preset area is set to the upper quarter of the image, and the upper quarter of the image is captured. The entire text recognition is performed on the area to generate the overall text recognition result.
[0041] In actual applications, an external OCR recognition interface with position information can be used to perform overall text recognition in the upper quarter area. It should be noted that only recognizing the upper quarter area is to save recognition time. Therefore, the setting of the preset area can be adjusted according to actual conditions. The overall text recognition results obtained this time are text boxes one by one, each of which contains the text content of the corresponding area, and the total text content is the total text content in the preset area.
[0042] S2. Compare the overall text recognition result with a plurality of pre-stored items of information to be matched. If the overall text recognition result contains items of information to be matched, match the image to be uploaded with the items of information to be matched. Otherwise, execute step S3.
[0043] In specific practice, take the answer sheet scenario as an example. In this scenario, multiple items of information to be matched are the names of students to be matched, or / and the student ID numbers. The names or student ID numbers can be matched separately, or the name recognition matching and the student ID recognition matching can be performed at the same time. The overall text recognition result will be compared with the student information to be matched (name, student ID number). If the name or student ID number to be matched exists in the overall text recognition result, the match is successful, and the answer sheet image is matched to the student's name.
[0044] Furthermore, step S2 also includes: if there are multiple items of information to be matched in the overall text recognition result, matching the item of information to be matched that appears first in the overall text recognition result with the image to be uploaded.
[0045] It is understandable that in some scenarios, there may be multiple names in the overall text recognition result, such as collaborative work, etc. The setting of this preferred embodiment can cope with this situation and ensure accurate recognition.
[0046] In actual application, we first loop through the list of names to be matched and the list of student numbers to be matched. If the string to be matched is in the overall text recognition result, the match is successful and the data is assigned to the corresponding student name. If multiple strings to be matched are in the overall text recognition result, the longest match, first match principle is adopted, and the student who is the first to be matched and has the longest string to be matched is hit.
[0047] S3, extracting the regional text recognition results after the preset keywords in the overall text recognition results, calculating the shortest edit distance between the regional text recognition results and each item information to be matched, if the shortest edit distance is less than the preset value, matching the image to be uploaded with the item information to be matched, otherwise executing step S4.
[0048] In practice, the preset keywords can be "name", "student number", etc., and the text recognition result of the area after these keywords is the handwritten name or student number of the examinee. The name result is used to calculate the shortest edit distance with each name to be matched. If the shortest edit distance with one of the names to be matched appears, and the shortest edit distance is less than 2, then the student is matched. Otherwise, the same method is used to calculate the shortest edit distance of the student number. If the shortest edit distance appears and is less than one-third of the student number to be matched (it can be set according to actual conditions), then the student is matched.
[0049] Specific examples are listed for illustration: Since the name or student ID number is handwritten by the examinee, there may be cases where character recognition errors occur. For example, the handwritten characters should be "Zhang Sansan", but the data obtained from the character recognition result is "Zhang Saner". In such a situation, step S2 cannot match successfully. After step S3 calculates the shortest edit distance, it is found that the shortest edit distance between "Zhang Saner" and "Zhang Sansan", one of the names to be matched, is 1, and this value is less than 2, meeting the requirements. Therefore, it can be determined that the answer sheet matches "Zhang Sansan". The same applies to the student ID number.
[0050] Furthermore, after step S3 extracts the regional character recognition result after the preset keyword in the overall character recognition result, it further includes:
[0051] Determine whether the regional character recognition result has a value; if not, it is concluded that the image to be uploaded fails to match.
[0052] It can be understood that taking the above example as a scenario for illustration, if it is detected that there is no value after the keyword "name", it means that the examinee forgot to write the name. In this case, it is completely impossible to match successfully. Therefore, the matching and uploading are directly ended, and the image to be uploaded is classified as a failed match.
[0053] Furthermore, before step S3 extracts the regional character recognition result after the preset keyword in the overall character recognition result, it further includes:
[0054] Detect whether there is a preset keyword in the overall character recognition result;
[0055] If not, match the image to be uploaded to the same entry information to be matched as the previous image to be uploaded.
[0056] It can be understood that taking the above example as a scenario for illustration, some answer sheets have front and back sides. When taking pictures and uploading, usually the front side is taken and then the back side is taken immediately. Therefore, if it is detected that there is no keyword "name" in the overall character recognition result, it means that this side is the back side of the answer sheet and belongs to the same examinee as the front side of the answer sheet.
[0057] In practical applications, firstly, whether there is a "name" or "student number" keyword text block in the overall text recognition result is detected; if there is, the text block after the horizontal position of the keyword is taken. The specific detection method is to detect all text blocks in the overall text recognition result. If the text block contains the "name" keyword, and the name keyword is not at the end of the text block, it is considered that the name written by the actual student is in the same text block as the name keyword, and the area after the name keyword is the actual name area. The text recognition result is taken, and the corresponding picture is intercepted for use in step S4. If the name keyword is at the end of the text block, the position of the next text block is detected. If the position of the next text block is after the name keyword and the distance is less than a threshold, it is considered that the next text block is the actual student name area, and the recognition result is taken and the corresponding picture is intercepted for use in step S4. The student number area recognition result is also taken in the same way and the corresponding picture is intercepted. If the name and student number keywords are not detected, it is considered that this picture is the next page of the above picture, and it is defaulted to the previous student name. If the name and student number keywords are detected, but the actual name and student number area is not detected according to the above method, we think that it is possible that the student did not write the name and student number, and it is classified as unmatched.
[0058] For the intercepted name and student number recognition results, the shortest edit distance is calculated with the names and student numbers to be matched in the library in turn. The shortest edit distance is the minimum number of single-character modifications required to change the string to be matched into the recognition result string. If there is only one set of names and student numbers to be matched and the names and student numbers of the recognition results have the minimum replacement distance, and the minimum replacement distance is less than a threshold, the match is successful. If there are multiple names and student numbers with the same replacement distance, they are added to the candidate set for use in step S5, and the position where the replacement occurs is recorded.
[0059] S4. Cut out the area image after the preset keyword from the preset area, segment the area image into single words, and obtain a plurality of handwritten single-word images; calculate the word recognition probability matrix of each handwritten single-word image, and calculate the average probability matrix of the area image; obtain the matching probability of the area image corresponding to each item information to be matched according to the average probability matrix, and if the maximum matching probability is greater than a preset threshold, match the image to be uploaded with the item information to be matched.
[0060] In specific practice, a pre-trained handwritten single-word recognition model can be used to calculate the word recognition probability matrix of each handwritten single-word image, where the word recognition probability matrix is the probability that each character among a large number of characters is similar to the handwritten single-word image.
[0061] Furthermore, in step S4, the regional image is segmented into single words to obtain a plurality of handwritten single word images, including:
[0062] Binarize the region image, and intercept the blank images above, below, left and right of the image through the distribution of pixel projection histograms in horizontal and vertical directions, so that the region image is close to the actual text region;
[0063] Calculate and obtain a pixel histogram in the vertical direction of the regional image, and determine an initial segmentation position according to the number of characters in the name in the list to be matched;
[0064] Starting from each initial segmentation position, traverse leftward and rightward at the same time to find the minimum value position in the vertical pixel histogram, and use the minimum value position as the final segmentation point.
[0065] In practical applications, a handwritten Chinese character recognition model and a number recognition model are trained in advance. For the regional image after the preset keyword is intercepted from the preset area, the image is first binarized, and a morphological operation of first closing and then opening is performed. Then, the blank images above, below, left and right of the image are intercepted through the horizontal and vertical pixel projection histogram distribution to approximate the actual text area. Then, the image is cut according to the length of the name in the list to be matched through an improved projection method to obtain a single-word image. The specific method is as follows: first, the vertical pixel histogram of the image is calculated, and then the initial segmentation position is determined according to the number of characters in the name in the list to be matched. Taking a three-character name as an example, the entire image is first divided into three equal parts to determine two segmentation positions, and then each segmentation position is traversed to the left and right at the same time to find the minimum value position in the histogram to obtain the final segmentation point. Then, for the segmentation result, the trained handwritten Chinese character recognition model is used to calculate the probability matrix of each image, and then the average probability matrix is calculated. If the highest probability appears and the highest probability is greater than a certain threshold, the match is successful. If there is no successful match, the student number is calculated in the same way.
[0066] In another preferred embodiment, the method for automatically matching batch uploaded data further includes:
[0067] If after executing step S4, the image to be uploaded does not match, then execute step S5;
[0068] S5. Integrate multiple items of information to be matched whose shortest edit distance in step S3 is the same and smaller than a preset value into a candidate set; calculate the replacement probability of each item of information to be matched in the candidate set and the regional image; if the maximum replacement probability among the replacement probabilities is greater than a preset threshold, match the image to be uploaded with the item of information to be matched.
[0069] In actual applications, the following situation may occur: the item information to be matched includes "Zhang", "Li" and "Wang", and the regional text recognition result after the name extracted in step S3 is "X", and the shortest edit distance between "X" and "Zhang", "Li" and "Wang" is 1, so it is impossible to successfully match in step S3. However, this step can integrate "Zhang", "Li" and "Wang" into a candidate set, and calculate the replacement probability of each item information to be matched and the regional image. Assuming that the replacement probability of "Zhang" is 0.8, the replacement probability of "Li" is 0.2, and the replacement probability of "Wang" is 0.3, then the replacement probability of "Zhang" is the largest and is greater than the preset threshold of 0.7, then the image to be uploaded is matched with "Zhang".
[0070] Calculate the replacement probability of each candidate name and student number in the candidate set. The specific calculation method is to obtain the recognition probability matrix of the changed position, get the probability that the character at that position belongs to the candidate set, calculate multiple times for multiple positions and take the average as the replacement probability of the candidate name and student number.
[0071] Embodiment 2
[0072] A batch upload data automatic matching device is provided, comprising:
[0073] A main controller, and a memory connected to the main controller;
[0074] a memory in which program instructions are stored;
[0075] The main controller is used to execute program instructions stored in the memory to perform any of the above methods.
[0076] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.
[0077] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.
[0078] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.
[0079] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0080] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0081] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0082] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.
[0083] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0084] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.
Claims
1. A method for automatically matching batch uploaded data, characterized in that: include: S1, obtaining an image to be uploaded, performing rotation correction on the image to be uploaded, intercepting a preset area of the rotation-corrected image, and performing overall text recognition on the intercepted preset area; S2, comparing the overall text recognition result with a plurality of pre-stored items of information to be matched, if there is any item of information to be matched in the overall text recognition result, matching the image to be uploaded with the item of information to be matched, otherwise executing step S3; S3, extracting the regional text recognition result after the preset keyword in the overall text recognition result, calculating the shortest edit distance between the regional text recognition result and each item information to be matched, if the shortest edit distance is less than the preset value, matching the image to be uploaded with the item information to be matched, otherwise executing step S4; S4, intercepting the regional image after the preset keyword from the preset region, and performing single-word segmentation on the regional image to obtain a plurality of handwritten single-word images; Calculate the word recognition probability matrix of each handwritten single word image using the pre-trained handwritten single word recognition model, and calculate the average probability matrix of the regional image according to the word recognition probability matrix; The matching probability of the regional image corresponding to each item of information to be matched is obtained according to the average probability matrix. If the maximum matching probability is greater than a preset threshold, the image to be uploaded is matched with the item of information to be matched. The word recognition probability matrix is the probability that each character in all the preset characters is similar to the handwritten single word image.
2. The automatic matching method for batch uploaded data according to claim 1, characterized in that: Also includes: If after executing step S4, the image to be uploaded does not match, then execute step S5; S5. Integrate multiple items of information to be matched whose shortest edit distance in step S3 is the same and smaller than a preset value into a candidate set; calculate the replacement probability of each item of information to be matched in the candidate set and the regional image; if the maximum replacement probability among the replacement probabilities is greater than a preset threshold, match the image to be uploaded with the item of information to be matched.
3. The automatic matching method for batch uploaded data according to claim 1, characterized in that: After extracting the regional text recognition results after the preset keyword in the overall text recognition results, step S3 further includes: Determine whether the regional text recognition result has a value; If not, it is concluded that the image to be uploaded fails to match.
4. The automatic matching method for batch uploaded data according to claim 1, characterized in that: Before extracting the regional text recognition results after the preset keyword in the overall text recognition results, step S3 further includes: Detecting whether there is a preset keyword in the overall text recognition result; If not, the image to be uploaded and the last image to be uploaded are matched to the same item information to be matched.
5. The method for automatic matching of batch uploaded data according to claim 1, characterized in that: In step S4, the regional image is segmented into single words to obtain a plurality of handwritten single word images, including: Calculate and obtain a pixel histogram in the vertical direction of the regional image, and determine an initial segmentation position according to the number of characters in the name in the list to be matched; Starting from each initial segmentation position, traverse leftward and rightward at the same time to find the minimum value position in the vertical pixel histogram, and use the minimum value position as the final segmentation point.
6. The method for automatic matching of batch uploaded data according to claim 5, characterized in that: Before calculating and obtaining the pixel histogram in the vertical direction of the regional image, the method further includes: The regional image is binarized, and blank images above, below, left and right of the image are intercepted through the distribution of pixel projection histograms in horizontal and vertical directions, so that the regional image is close to the actual text area.
7. The automatic matching method for batch uploaded data according to claim 1, characterized in that: Step S2 also includes: If there are multiple items of information to be matched in the overall text recognition result, the item of information to be matched that appears first in the overall text recognition result is matched with the image to be uploaded.
8. A batch upload data automatic matching device, characterized in that: include: A main controller, and a memory connected to the main controller; a memory in which program instructions are stored; The main controller is used to execute program instructions stored in the memory and execute the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Signature identification method and equipment
CN111368632A
Image correction method and device and computer readable storage medium
CN111783770A
Answer sheet character string recognition method and device, terminal and computer storage medium
CN112507758A
Handwritten bill numerical value information identification method and system, medium and equipment
CN113903024A
Logistics address intelligent identification method, apparatus and device, and storage medium
CN114758340A
