Apparatus and method for processing information and program
The information processing apparatus effectively recognizes handwritten characters in entry fields without frames by forming regions based on pixel connection strength, addressing the limitations of existing OCR systems in handling unframed entry fields.
Patent Information
- Application Number
- JP2023083947
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-12-04
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing OCR systems struggle to accurately recognize handwritten characters when entry fields are not separated by frames, such as underlines or headings, as they rely on ruled lines for character detection.
An information processing apparatus that specifies pixel pieces from an image, forms regions based on pixel connection strength, and recognizes characters within these regions, using a system that includes a specifying means, forming means, and character recognition means to handle entry fields without frames.
Enables high-accuracy recognition of handwritten characters even when entry fields are not separated by frames, improving the reliability of character detection in forms without ruled lines.
Smart Images

Figure 2024167696000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]
[0002] OCR (Optical Character Recognition) exists as a tool to assist in the task of reading information from forms and inputting it into a system. Before recognizing characters, OCR involves a process called character detection, which detects areas on forms that contain characters.
[0003] Character detection has different characteristics for printed and handwritten characters. Handwritten characters have characteristics such as the fact that character sizes are uneven compared to printed characters, that entry fields may or may not have a box, and that when there is no box, the boundaries of the items must be estimated. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] Patent Publication No. 2021-60729 DISCLOSURE OF THEINVENTION [Problem to be solved by the invention]
[0005] The method in Patent Document 1 combines and divides handwritten characters based on the lines contained in the document, but in the case of handwritten text, there are cases where the entry fields are composed of only underlines or headings without any lines, and this method cannot be used in such cases.
[0006] Therefore, an object of the present invention is to provide a mechanism that can recognize handwritten characters with high accuracy even when entry columns are not separated by frames. [Means for solving the problem]
[0007] The information processing device of the present invention includes: a specifying means for specifying pixel pieces formed by consecutive pixels from an image; a forming means for forming an area by combining a plurality of pixel pieces based on the strength of connection between the pixel pieces identified by the identifying means; a character recognition means for recognizing characters included in each area formed by the forming means; The present invention is characterized by comprising: Effect of the Invention
[0008] According to the present invention, even if an entry field is not separated by a box, handwritten characters can be recognized with high accuracy. [Brief description of the drawings]
[0009] [Figure 1] 1 is a diagram illustrating an example of a system configuration of a table extraction system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a PC in an embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram illustrating an example of a functional configuration in an embodiment of the present invention. [Figure 4] 1A and 1B are diagrams showing an example of a handwritten image as an input and a character region estimation result as an output in an embodiment of the present invention. [Diagram 5] 5 is a flowchart showing a process flow of a continuous pixel detection unit and a character region estimation unit in the embodiment of the present invention. [Figure 6] 6A to 6C are diagrams illustrating an example of a process of a continuous pixel detection unit and a character region estimation unit according to an embodiment of the present invention. [Figure 7] 10 is a flowchart showing a flow of processing for character grouping in the embodiment of the present invention. [Figure 8] 1 is a flowchart showing a flow of processing for dividing a character area in an embodiment of the present invention. [Figure 9] 5A to 5C are diagrams illustrating an example of a process for dividing a character region according to an embodiment of the present invention. [Figure 10] 11A and 11B are diagrams illustrating an example of a character region estimation result according to an embodiment of the present invention. [Figure 11] Diagram showing Equation 1 DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] FIG. 1 is a diagram showing an example of a system configuration of a handwritten character extraction system according to an embodiment of the present invention.
[0011] A client terminal 101 that accepts operations by a user and a scanner 102 that scans forms and creates image files are connected via a communication path 100 .
[0012] The communication path 100 can take the form of a wired LAN, a wireless LAN, a USB, etc., depending on the physical interface of the scanner 102 .
[0013] The server device 103 has a function of storing image data scanned by the scanner 102 and performing OCR processing on the image data based on an instruction from the client terminal 101 . It should be noted that image data storage and OCR processing may be performed by the client terminal 101 instead of the server device 103 .
[0014] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device applicable as the client terminal 101 or the server device 103 of the present invention.
[0015] As shown in FIG. 2, the information processing device has a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a storage device 204, an input controller 205, an audio controller 206, a video controller 207, a memory controller 208, and a communication I / F controller 209 connected via a system bus 200.
[0016] The CPU 201 comprehensively controls each device and controller connected to the system bus 200 .
[0017] ROM 202 or external memory 213 holds the BIOS (Basic Input / Output System) and OS (Operating System), which are control programs executed by CPU 201, computer-readable and executable programs for realizing this information processing method, and various necessary data (including data tables).
[0018] The RAM 203 functions as a main memory, a work area, etc., for the CPU 201. The CPU 201 loads programs and the like required for executing processes from the ROM 202 or the external memory 213 into the RAM 203, and executes the loaded programs to realize various operations.
[0019] The input controller 205 controls input from an input device such as a keyboard 210 or a pointing device such as a mouse (not shown). If the input device is a touch panel, the user can give various instructions by pressing (touching with a finger or the like) icons, cursors, or buttons displayed on the touch panel.
[0020] The touch panel may be a touch panel capable of detecting positions touched by multiple fingers, such as a multi-touch screen.
[0021] The video controller 207 controls display on an external output device such as a display 212. The display also includes the display of a notebook computer integrated with the main body. Note that the external output device is not limited to a display, and may be, for example, a projector. In addition, for the device capable of receiving the above-mentioned touch operation, an input device is also provided.
[0022] The video controller 207 is capable of controlling a video memory (VRAM) for display control, and can use part of the RAM 203 as the video memory area, or can provide a separate dedicated video memory.
[0023] The memory controller 208 controls access to the external memory 213. As the external memory, an external storage device (hard disk) for storing a boot program, various applications, font data, user files, edit files, and various data, a flexible disk (FD), or a compact flash (registered trademark) memory connected to a PCMCIA card slot via an adapter, etc., can be used.
[0024] The communication I / F controller 209 connects and communicates with external devices via a network, and executes communication control processing on the network. For example, communication using TCP / IP, telephone lines such as ISDN, and communication using 4G and 5G lines for mobile phones are possible.
[0025] The CPU 201 enables display on the display 212 by, for example, executing a process of expanding (rasterizing) an outline font in a display information area in the RAM 203. The CPU 201 also enables a user to give instructions using a mouse cursor (not shown) or the like on the display 212.
[0026] Fig. 3 is a diagram showing an example of a functional configuration of the client terminal 101 of the present invention. The following description will be given on the assumption that the OCR process is executed by the client terminal 101, but as described in the description of Fig. 1, the OCR process may be executed by the server device 103 instead of the client terminal 101. In that case, each function will be provided in the server device 103.
[0027] The input reception unit 251 captures an image into the CPU 201 of the client terminal 101 via the scanner 102 or the server device 103 .
[0028] The image preprocessing unit 252 performs noise removal and binarization on the input image 401 .
[0029] The continuous pixel detection unit 253 detects continuous pixels (black) from within the binarized image, and generates a workpiece image 601 including pixel piece groups 602 in which rectangles containing each of the continuous pixels are filled in.
[0030] The character area estimation unit 254 groups pixel fragments 602 in the workpiece image 601 by character area item, and also divides character items that were originally separate and were combined in the process, using the information in the paper information 255. The paper information 255 includes rectangular information that indicates the approximate position of the item to be read.
[0031] The character recognition unit 256 performs character recognition on the character area estimated by the character area estimation unit 254, and outputs text.
[0032] The result output unit 257 outputs the coordinates of the character area estimated by the character area estimation unit 254 and the text read by the character recognition unit 256 to a file or a database.
[0033] 4 shows an example of an input handwritten image and an output character region estimation result. Assume that an input image 401 contains a handwritten character region 402. The object of the present invention is to group characters by category from this handwritten character region 402 and obtain character grouping results 451-456.
[0034] FIG. 5 is a flowchart showing the flow of processing by character area estimation unit 254.
[0035] In step S501, the input receiving unit 251 acquires a handwritten image captured by the scanner 102 or the like.
[0036] In step S502, the image preprocessing unit 252 binarizes the input image 401, inverts the black and white, and then performs noise removal processing to generate a binarized image.
[0037] FIG. 6 is a diagram showing an example of the process of the continuous pixel detection unit and the character region estimation unit.
[0038] In step S503, the continuous pixel detection unit 253 detects continuous pixels (black) from the binary image and generates a work image 601 including a pixel fragment group 602 that fills in a rectangle containing each continuous pixel (Figure 6-1). In step S504, the character area estimation unit 254 performs a grouping process to group the pixel piece groups 602 in the workpiece image 601 by item.
[0039] FIG. 7 is a flowchart showing the flow of the character grouping process.
[0040] In step S701, the processes of S702 to S705 are repeatedly performed on the pixel piece group 602 in the workpiece image 601.
[0041] In step S702, two pixel pieces are selected from the pixel piece group 602. The combination of these two pixel pieces is covered in the loop of step S701.
[0042] In step S703, the strength of connection between the two pixel pieces selected in step S702 is evaluated. The strength of connection is an index devised to evaluate the probability that the two pixel pieces belong to the same item. Figure 11 shows an example of a formula (Formula 1) for evaluating the strength of connection.
[0043] Here, F is an evaluation value that represents the strength of the connection, H1 and H2 are the sizes of the two selected pixel pieces, and d is the distance between the two pixel transformations. This is the formula for universal gravitation. It is similar to TIFF2024167696000002.tif810 and simulates the attractive force between pixel fragments.
[0044] In step S704, it is evaluated whether or not the connection value F exceeds a threshold value. If it exceeds the threshold value, the process proceeds to step S705, and if not, the process returns to step S701.
[0045] In step S705, if the connection value F exceeds a threshold, it is determined that the two pixel pieces belong to the same item, and a line connecting the items is drawn in the workpiece image 601. As a result, a pixel piece combined workpiece image 611 and a combined pixel piece group 612 are obtained (FIG. 6-2). In step S706, the contiguous pixels are re-evaluated again for the pixel piece combined workpiece image 611 to recognize the pixel pieces combined in the previous step.
[0046] In step S707, the bbox (bounding box) area that contains each of the consecutive pixels recognized in the previous step is obtained, and the rectangular area is filled. Each rectangular area is also expanded horizontally, and small pixel pieces that could not be combined in step S705 are combined. As a result, a combined workpiece image 621 and a group of combined pixel pieces 622-626 are obtained.
[0047] In step S505, character area estimation unit 254 also divides character items that were originally separate and were combined in the process of step S504, using information from paper information 255.
[0048] FIG. 8 is a flowchart showing the flow of processing for dividing a character region.
[0049] In step S801, the character area estimation unit 254 reads the item definition included in the paper information 255. In the item definition, a rectangle that serves as a guide for the item to be read is registered.
[0050] FIG. 9 is a diagram showing an example of the process of dividing a character region.
[0051] Figure 9-1 shows an example of a rectangle that indicates the coordinates of an item that a user wants to read in a character recognition system that is the subject of the present invention. Here, item registration rectangles 902-903 are registered (Figure 9-1).
[0052] What must be taken into consideration here is that because this invention targets frameless entry fields, the item registration rectangle is only a guide for filling in the text (the area in which characters are written is somewhat fixed, but the size of the characters is unknown), and the characters actually written may deviate from or extend beyond the rectangle, so the problem cannot be solved by simply cutting out the coordinates of the item registration rectangle and performing character recognition (Figure 9-2).
[0053] In step S802, the combined pixel fragment group 622-626 is obtained from the combined workpiece image 621.
[0054] In step S803, the processes of steps S804 to S806 are performed for each of the consecutive pixels.
[0055] In step S804, it is determined whether the currently focused continuous pixel piece spans multiple item registration rectangles. In the example of FIG. 9-3, it is shown that the combined pixel piece 622 spans item registration rectangles 902 and 903. If this is the case, the process proceeds to step S805, and if not, the process returns to S804 to evaluate the next combined pixel piece.
[0056] In step S805, if there is a combined pixel piece that straddles multiple item registration rectangles, a line is generated to divide the pixel piece into items. In the example of FIG. 9-4, a line is drawn between item registration rectangles 902 and 903, and this is set as division line 941. As a method for drawing the division line, it is possible to apply an image processing technique called a thinning algorithm. A work image for generating division lines 951 is created by drawing the item registration rectangles in white on a black background, and a thinned work image 952 is obtained by processing this image with the thinning algorithm. (FIG. 9-5)
[0057] In step S806, the difference between the combined pixel piece 622 and the division line 941 is calculated to obtain a divided workpiece image 961 and divided pixel pieces 962-963 (FIG. 9-6).
[0058] In step S807, rectangles corresponding to the regions of the divided pixel pieces 962-967 are extracted from the input image 401, and character grouping results 451-456 are obtained.
[0059] In step S506, the character recognition unit 256 performs character recognition on the areas of the character grouping results 451-456 obtained in step S505.
[0060] In step S507, result output unit 257 outputs the coordinates of the character area estimated by character area estimation unit 254 and the text read by character recognition unit 256 to a file or a database.
[0061] Although the embodiment has been described above, the present invention can be embodied, for example, as a system, an apparatus, a method, a program, or a recording medium, etc. Specifically, the present invention may be applied to a system composed of multiple devices, or may be applied to an apparatus composed of a single device.
[0062] Moreover, the program of the present invention is a program that enables a computer to execute the processing method of the flowchart shown in Fig. 3, and the storage medium of the present invention stores a program that enables a computer to execute the processing method of Fig. 3. The program of the present invention may be a program for each processing method of each device in Fig. 3.
[0063] As described above, it goes without saying that the object of the present invention can be achieved by supplying a recording medium on which a program that realizes the functions of the above-mentioned embodiments is recorded to a system or device, and having the computer (or CPU or MPU) of that system or device read and execute the program stored on the recording medium.
[0064] In this case, the program itself read out from the recording medium will realize the novel functions of the present invention, and the recording medium on which the program is recorded will constitute the present invention.
[0065] Examples of recording media for supplying the program include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon disks, etc.
[0066] Furthermore, it goes without saying that the functions of the above-mentioned embodiments are not only realized by the computer executing a program it has read, but also includes cases where an OS (operating system) running on the computer performs some or all of the actual processing based on the instructions of the program, and the functions of the above-mentioned embodiments are realized through that processing.
[0067] Furthermore, it goes without saying that this also includes cases where a program read from a recording medium is written into a memory provided on a function expansion board inserted into a computer or a function expansion unit connected to a computer, and then a CPU or the like provided on the function expansion board or function expansion unit performs some or all of the actual processing based on the instructions of the program code, thereby realizing the functions of the above-mentioned embodiments.
[0068] The present invention may be applied to a system consisting of multiple devices, or to an apparatus consisting of a single device. Needless to say, the present invention can also be applied to a case where the object is achieved by supplying a program to a system or apparatus. In this case, the effect of the present invention can be enjoyed by the system or apparatus by reading a recording medium storing a program for achieving the present invention into the system or apparatus.
[0069] Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or device can enjoy the effects of the present invention. Note that the present invention also includes configurations that combine the above-mentioned embodiments and their modified examples. [Explanation of symbols]
[0070] 100: Communication path 101: PC main unit 102: Scanner body 103: File server
Claims
1. A means for identifying pixel pieces formed by consecutive pixels from the image; a forming means for forming an area by combining a plurality of pixel pieces based on the strength of connection between the pixel pieces identified by the identifying means; a character recognition means for recognizing characters included in each area formed by the forming means; An information processing device comprising:
2. 2 . The information processing device according to claim 1 , wherein the forming means forms an area by combining a plurality of pixel pieces based on a strength of connection calculated based on the size of the pixel piece and the distance between the pixel pieces.
3. 3. The information processing method according to claim 2, wherein the forming means forms an area by combining a plurality of pixel pieces based on a value indicating a strength of connection calculated by the formula 1. (Formula 1) F: Value indicating the strength of the connection H: pixel size d: Distance between pixel pieces
4. 4. The information processing apparatus according to claim 3, wherein the specifying means specifies a rectangle containing the consecutively existing pixels as a pixel piece.
5. identifying pixel pieces formed by contiguous pixels from the image; a forming step of forming an area in which a plurality of pixel pieces are connected based on the strength of connection between the pixel pieces identified in the identifying step; a character recognition step of recognizing characters included in each area formed by the forming step; An information processing method comprising:
6. A program for causing a computer to function as each of the means according to any one of claims 1 to 4.
Citation Information
Patent Citations
Image processing system, image processing method, and program
JP2021060729A
