Character recognition apparatus, character recognition method, and program

Abstract

To provide a character recognition technique for properly recognizing a selection item with a check mark, even for multiple kinds of document forms which are composed of similar items and having different arrangements of items.SOLUTION: A character recognition apparatus configured to recognize multiple kinds of document forms composed of similar items and having different arrangements of items includes: a region specifying unit 1302 which specifies, on a document form having a plurality of selection items composed of pre-printed character strings and to be selected with check marks, a selection region to be read in a selection item based on a character recognition result of a character recognition unit 1301; and a mark determination unit 1303 which performs determination processing as to whether the selection item in the selection region is check marked or not, based on a document form image.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to information processing technology, and more particularly to character recognition technology for recognizing characters on forms. [Background technology]

[0002] Conventionally, there are character recognition devices that use character recognition technologies such as optical character recognition / optical character reading (sometimes abbreviated as OCR) to read characters on forms and the like. OCR (Optical Character Recognition) is a technology that reads information about each item, such as character strings, from an image of a form or other paper captured by a scanner or other device. Character recognition using OCR is primarily targeted at, for example, standard forms in which the position and attributes of each item to be recognized are specified. For example, each item, such as character strings, is recognized by comparing the form image with a form definition in which the coordinates and attributes of each item are registered.

[0003] Furthermore, in character recognition of forms using OCR, not only character strings on the form but also options with check marks may be recognized (see Patent Document 1). Patent Document 1 describes a technology for recognizing option boxes with marks from a set of multiple option boxes whose coordinates have been specified in advance. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-44086 Summary of the Invention [Problem to be solved by the invention]

[0005] As mentioned above, conventional OCR character recognition for forms is primarily targeted at standard forms, and requires a form definition for each type of form. This requires time-consuming work such as creating and registering the form definition before character recognition can begin. Furthermore, even when recognizing selectable items marked with check marks, it is necessary to register a form definition in advance that specifies the positions of the selectable items. However, forms are diverse, and it is difficult to define the positions of each item, including the selectable items, for all forms.

[0006] For this reason, for example, in the case of a form that is composed of similar items but has a different item arrangement, i.e., a semi-standard form, full-text recognition may be performed to recognize all characters in the form from the form image that has been read in. With full-text recognition, it is possible to read each item on the form and the characters entered on the form, even for forms other than standard forms, such as semi-standard forms.

[0007] However, if a check mark is placed on a selection item, such as a character string pre-printed on a form, there is a risk that the check mark will be recognized as noise during full-text recognition. In particular, when full-text recognition is performed using AI-OCR, which uses artificial intelligence (AI) technology, the accuracy of character recognition increases, but the check mark placed on the selection item is likely to be recognized as noise. As a result, there is a risk that the selection item with the check mark will not be properly recognized.

[0008] The object of the present disclosure is to provide a character recognition technology that can appropriately recognize the state in which a check mark has been added to a selected item, even for multiple types of forms that are composed of similar items but have different item arrangements. [Means for solving the problem]

[0009] A representative embodiment of the present disclosure is characterized by having the following configuration. A character recognition device of one embodiment includes a form image acquisition unit that acquires an image of a form, and a character recognition unit that recognizes characters in the form from the form image acquired by the form image acquisition unit, and is capable of recognizing multiple types of forms that are composed of similar items but have different item arrangements. When the form has multiple selection items composed of pre-printed character strings and selected by check marks, the character recognition device includes: an area identification unit that identifies a selection area in which the selection items are read based on character recognition results by the character recognition unit; and a mark determination unit that performs processing to determine, for the selection area, whether the selection item has a check mark applied to it based on the form image. [Effects of the Invention]

[0010] According to a representative embodiment of the present disclosure, a character recognition technology can be provided that can appropriately recognize the state in which a check mark is placed on a selected item, even for multiple types of forms that are composed of similar items but have different item arrangements. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating a schematic configuration of a system including a character recognition device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a configuration of a form recognition processing unit included in the character recognition device according to the embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of a form. [Figure 4] 1 is a flowchart illustrating an example of a character recognition method according to an embodiment. [Figure 5] 1 is a flowchart illustrating an example of a character recognition method according to an embodiment. [Figure 6] FIG. 10 is a diagram illustrating an example of a character recognition processing result according to the embodiment. [Figure 7] 1 is a flowchart illustrating an example of a character recognition method according to an embodiment. [Figure 8]1A and 1B are diagrams illustrating an example of a character recognition method according to an embodiment. [Figure 9] 1A and 1B are diagrams illustrating an example of a character recognition method according to an embodiment. [Figure 10] 1 is a flowchart illustrating an example of a character recognition method according to an embodiment. [Figure 11] 1A and 1B are diagrams illustrating an example of a character recognition method according to an embodiment. [Figure 12] 10A and 10B are diagrams illustrating a modified example of the character recognition method according to the embodiment. [Figure 13] 1A and 1B are diagrams illustrating an example of a character recognition method according to an embodiment. [Figure 14] FIG. 2 is a diagram illustrating an example of a GUI screen in the character recognition device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] In each drawing, the same parts are generally designated by the same reference numerals, and repeated explanations are omitted. The depiction of components in the drawings may not represent their actual positions, sizes, shapes, ranges, etc., in order to facilitate understanding of the invention.

[0013] For the sake of explanation, when describing processing by a program, the program, function, processing unit, etc. may be described as the main body, but the main hardware body for these is a processor, or a controller, device, computer, system, etc. that is configured with the processor, etc. A computer executes processing according to a program read into memory using resources such as memory and communication interfaces as appropriate through the processor. This realizes predetermined functions, processing units, etc. A processor is configured, for example, with semiconductor devices such as a CPU / MPU or GPU. Processing is not limited to software program processing, but can also be implemented using dedicated circuits. Dedicated circuits such as FPGAs, ASICs, and CPLDs can be used.

[0014] The program may be pre-installed as data on the target computer, or may be distributed as data from a program source to the target computer. The program source may be a program distribution server on a communication network or a non-transitory computer-readable storage medium, such as a memory card or disk. The program may be composed of multiple modules. The computer system may be composed of multiple devices. The computer system may be composed of a client-server system, a cloud computing system, an IoT system, etc. Various data and information may be composed of structures such as, but not limited to, tables and lists. Expressions such as identification information, identifiers, IDs, names, and numbers are interchangeable.

[0015] [Outline of the embodiment] A character recognition device according to an embodiment includes a form image acquisition unit that acquires an image of a form and a character recognition unit that recognizes characters in the form from the form image acquired by the form image acquisition unit, and is capable of recognizing multiple types of forms that are composed of similar items but have different item arrangements (hereinafter also referred to as semi-standard forms). When the form is composed of pre-printed character strings and has multiple selection items selected by check marks, the character recognition device according to an embodiment includes: an area identification unit that identifies a selection area in which the selection items are read based on the character recognition results by the character recognition unit; and a mark determination unit that performs processing to determine whether the selection items in the selection area have check marks added to them based on the form image.

[0016] The character recognition device of the embodiment is capable of performing character recognition processing on so-called semi-standard forms, which are forms that are composed of similar items, such as invoices, purchase orders, and delivery notes, but in which the arrangement of the items varies depending on the company or local government, for example.

[0017] When reading a semi-standard form, this character recognition device performs, for example, full-text recognition processing using OCR on an image of the form, and then performs area identification processing to identify a reading area (a field where character strings, etc. are read) from the item names, etc. on the form based on the results of the full-text recognition processing, where characters, etc. are read, and then performs processing to read character strings, etc. in the identified reading area. This area identification processing can also be considered processing to identify the layout of the form. This processing takes some time at the stage of recognizing the form layout, so the speed of the processing to read character strings, etc. may be slower than in the case of standard forms whose reading areas are pre-defined, but it is possible to properly recognize character strings, etc. even on semi-standard forms.

[0018] Furthermore, in the character recognition device of the embodiment, when a form has multiple selection items that are composed of pre-printed character strings and selected by check marks, a selection area in which the selection items are read is identified along with the reading area. Furthermore, for the identified selection area, a determination process is performed to determine whether or not a check mark is added to the selection item based on the form image, rather than the character recognition results by the character recognition unit. In other words, the determination process recognizes whether or not a check mark is added to the selection item based on the form image, rather than the character recognition results. This allows for appropriate recognition of whether or not a check mark is added to the selection item when performing character recognition processing on a so-called semi-standard form.

[0019] In the following, functions related to character recognition processing for semi-standard forms will be described as an explanation of the character recognition device of the embodiment. In particular, functions related to the processing for determining whether or not a check mark has been added to a selection item in a semi-standard form will be described in detail. However, the character recognition device of the embodiment can perform character recognition not only for semi-standard forms but also for various forms including standard forms.

[0020] The configuration of a character recognition device according to an embodiment will be described with reference to Figures 1 and 2. First, the overall configuration of a system including a character recognition device according to an embodiment will be described with reference to Figure 1.

[0021] [Overall system configuration] The system shown in Fig. 1 is a system for recognizing characters and the like in forms, and can also be called a character recognition system or a form recognition system. The system shown in Fig. 1 includes a character recognition device 1 and a scanner device 2, which are connected to each other via a communication means such as a LAN. In other words, the character recognition device 1 of the embodiment is an OCR device, and is configured by a computer.

[0022] User U1 operates the character recognition device 1 and the scanner device 2. User U1 reads a form (in other words, a paper sheet) 200 with the scanner device 2, and processes a form image file 4 corresponding to the form image obtained by the reading with the character recognition device 1 to obtain recognition result information.

[0023] The scanner device 2 is a device that optically scans the surface of a set document such as a form 200 to read an image. The scanner device 2 outputs a form image file 4 as a result of the reading. The form image file 4 is transferred from the scanner device 2 to the character recognition device 1 via a communication means such as a LAN. The character recognition device 1 receives and inputs the form image file 4 via the communication interface device 103, and stores it as read image data 122 in the storage device 102. The scanner device 2 may also be a multifunction device having a scanner function and a printer function.

[0024] [Character recognition device configuration] The character recognition device 1 is a computer on which character recognition software (including programs, databases, etc.) is implemented. The character recognition device 1 performs OCR processing, which is a character recognition process, on scanned image data 122 corresponding to the input form image file 4, and obtains character recognition result data 123 as the processing result output. In this case, by performing AI-OCR processing using artificial intelligence (AI) technology as the OCR processing, it is possible to improve the recognition accuracy of character strings, etc. on the form.

[0025] Furthermore, as will be described in detail later, when a form is composed of pre-printed character strings and has multiple selection items selected by check marks, the character recognition device 1 executes an area specification process to specify a selection area in which the selected items with check marks are read, based on the character recognition result data 123. Furthermore, for the specified selection area, a determination process is executed to determine whether or not the selected items have check marks, based on the form image.

[0026] The character recognition software is a computer program that causes the character recognition device 1, which is a computer, to execute processing corresponding to the character recognition method of the present disclosure. In the first embodiment, the program that is the character recognition software is included in the control program 121 stored in the storage device 102. The program of the first embodiment may be provided in a form that is stored in a storage medium such as a memory card, or may be downloaded from a server on a communication network, etc.

[0027] Character recognition device 1 is configured, for example, as a PC and software installed thereon, but is not limited to this and may be configured as a dedicated device such as an OCR device, a multifunction peripheral, etc. Also, in the system of embodiment 1, character recognition device 1 and scanner device 2 are configured as separate devices, but character recognition device 1 and scanner device 2 may also be configured as an integrated device.

[0028] More specifically, the character recognition device 1 comprises an arithmetic unit (including a processor, etc.) 101, a storage unit (including a memory, etc.) 102, a communication interface unit 103, an input / output interface unit 104, an input unit 105, an output unit 106, and an external storage unit 107, which are interconnected via an architecture such as a bus so that they can communicate with each other.

[0029] The arithmetic device 101 is configured with, for example, a CPU / GPU, a ROM, a RAM, etc., and corresponds to a controller that controls the entire device and each unit. The arithmetic device 101 has, as functional blocks realized by software program processing, a form image acquisition unit 11, a form definition unit 12, a form recognition processing unit 13, a screen display unit 14, a setting unit 15, etc. The arithmetic device 101 realizes each of the above functional blocks by reading a control program 121 from the storage device 102 into memory and executing processing in accordance with the control program 121.

[0030] The storage device 102 is configured with a nonvolatile storage device or the like, and stores various data and information handled by the arithmetic device 101, etc. The storage device 102 stores various data and information such as a control program 121, scanned image data 122, character recognition result data 123, form recognition result data 124, form definition data 125, dictionary data 126, and setting information 127.

[0031] The scanned image data 122 is, for example, a form image file 4, which is an image of a form 200 captured using the scanner device 2, stored in the storage device 102. The character recognition result data 123 is data created as a result of character recognition processing by the form recognition processing unit 13. The form recognition result data 124 is data created as a result of form recognition processing by the form recognition processing unit 13, and includes not only the result of the character recognition processing but also information such as the result of a determination process as to whether or not a check mark has been placed on a selection item in the form. This determination process will be described in detail later.

[0032] Form definition data 125 includes information such as the layout of a form defined by form definition unit 12, and can also be called a format file. Form definition data 125 is organized and stored in a database (DB), file system, etc. Dictionary data 126 is character information that is registered in advance to be used to identify character strings in character recognition processing. Setting information 127 includes system setting information, user setting information, etc. related to character recognition device 1. Note that if a DB server, storage device, etc. is connected to character recognition device 1, various data and information may be stored in these DB servers, storage devices, etc.

[0033] The communication interface device 103 is a part that performs communication processing via a predetermined communication interface with an external device such as the scanner device 2. The character recognition device 1 may exchange data and information with not only the scanner device 2 but also an external server, storage device, etc.

[0034] An input device 105, an output device 106, an external storage device 107, and the like are connected to the input / output interface device 104. Examples of the input device 105 include a keyboard, a mouse, an operation panel, and a microphone. A user U1 inputs instructions and the like through the input device 105. Examples of the output device 106 include a display, a printer, and a speaker. The input device 105 and the output device 106 may be touch panels.

[0035] Various types of information are displayed on the display screen of the output device 106 using a predetermined graphical user interface (GUI). Examples of information displayed on the display screen include recognition result information based on character recognition result data 123 and form recognition result data 124, and various setting information such as user information. User U1 can set and check the recognition result information and setting information through this screen. In other words, user U1 can perform various tasks related to form recognition processing through this screen.

[0036] The external storage device 107 may be, for example, a disk device or a memory card device. The external storage device 107 is capable of storing the form image file 4 and the like under the control of the arithmetic unit 101. The character recognition device 1 may read the form image file 4 and the like stored in the external storage device 107 into the storage device 102, or may store the character recognition result data 123 and the like stored in the storage device 102 in the external storage device 107.

[0037] [Configuration of the computing device] Next, the form image acquisition unit 11, the form definition unit 12, the form recognition processing unit 13, the screen display unit 14, and the setting unit 15 provided in the arithmetic device 101 will be described in more detail.

[0038] The form image acquisition unit 11 acquires the form image file 4 read by the scanner device 2 and stores it in the storage device 102 as read image data 122. Note that the form image acquisition unit 11 may read the form image file 4 stored in the external storage device 107, for example, and store it in the storage device 102 as read image data 122.

[0039] The form definition unit 12 defines a form based on, for example, input information by the user U1. That is, the form definition unit 12 creates and registers a form definition (which can also be called a template) corresponding to the form to be recognized based on, for example, input information by the user U1. The created and registered form definition is stored as form definition data 125 in, for example, the storage device 102.

[0040] The content of the form definition data 125 is not particularly limited, but includes, for example, the names (character strings) of items to be recognized on the form by the form recognition processing unit 13, attribute information of each item, the character type and specified character size (font size) of each item pre-printed on the form, etc. Furthermore, if the form has selection items (options) that are selected by check marks, the form definition data 125 also includes information on these selection items.

[0041] For semi-standard forms, for example, the layout of pre-printing may change, and the position of each item and selection item within the form cannot be identified. For this reason, position information (coordinates) of each of these items and selection items is not included in the form definition data 125. However, the area where the selection area where the selection item is read is located is included in the form definition data 125 as information on the relative position based on the corresponding item.

[0042] The form recognition processing unit 13 performs OCR processing (character recognition processing) on ​​the form image using the read image data 122 and the form definition data 125, creates character recognition result data 123, and stores it in the storage device 102. Furthermore, the form recognition processing unit 13 uses the read image data 122, the character recognition result data 123, and the form definition data 125 to create form recognition result data 124 including the results of a determination process as to whether or not a check mark has been placed on a selection item, and stores it in the storage device 102.

[0043] As described above, the recognition result information based on the character recognition result data 123 and the form recognition result data 124 is displayed on the display of the output device 106. The user U1 can check and correct the recognition result information on the display screen of the output device 106.

[0044] [Configuration of the form recognition processing section] As shown in FIG. 2, the form recognition processing unit 13 according to the first embodiment includes, for example, a character recognition unit 1301, an area specification unit 1302, and a mark determination unit 1303.

[0045] The character recognition unit 1301 performs character recognition processing (OCR processing) on ​​the form image using the scanned image data 122 and form definition data 125, and creates character recognition result data 123. When performing recognition processing on a semi-standard form, the character recognition unit 1301 acquires and references the scanned image data 122 and form definition data (form format) 125, and performs full-text recognition processing as character recognition processing for the form. For example, the character recognition unit 1301 extracts all character strings from the form image, performs character recognition processing (OCR processing) on ​​each extracted character string, and stores the recognition results in the storage device 102 as character recognition result data 123.

[0046] When performing recognition processing on a fixed form, the character recognition unit 1301 does not perform full-text recognition processing, but performs character recognition processing on a read area whose position has been specified in advance.

[0047] When performing recognition processing on a semi-standard form, the area specifying unit 1302 specifies a reading area using the character recognition result data 123 and the form definition data 125. Furthermore, when the form is configured with pre-printed character strings and has multiple selection items selected by check marks, the area specifying unit 1302 specifies a selection area in which these selection items are read.

[0048] Character recognition result data 123 includes information on each character string recognized by the character recognition process and the position coordinates of each character string. Furthermore, in form definition data 125, the areas in which the reading area and the selection area are located on a semi-standard form are defined as relative positions to the corresponding items. Therefore, area identification unit 1302 can identify the reading area and the selection area using character recognition result data 123 and form definition data 125.

[0049] As an example, the area identification unit 1302 uses the character recognition result data 123 to identify each item included in the document by keyword search using pre-set setting words, and identifies the corresponding reading area and selection area from the position of each item.

[0050] Furthermore, when a selection area is identified by the area identification unit 1302, the mark determination unit 1303 executes a process of determining whether or not a check mark has been added to the selection items pre-printed in the identified selection area based on the form image. Note that the processes performed by the character recognition unit 1301, area identification unit 1302, and mark determination unit 1303 that constitute the form recognition processing unit 13 will be described in more detail later.

[0051] [Example of a form] An example of a form will now be described with reference to Fig. 3. Form 200 shown in Fig. 3 is an example of a so-called semi-standard form, and form name 201 is "Customer Registration Form." This form 200 has items 202a to 202g named "Postal Code," "Address," "Name," "Contact Information," "Date of Birth," "Gender," and "Marital Status" as items 202 for reading character strings or the like. These character strings are pre-printed on the form.

[0052] Of these items 202 (202a to 202g), items 202a to 202e, "Postal Code," "Address," "Name," "Contact Information," and "Date of Birth," are items into which characters and numbers are entered in the form. In the form definition data 125, the layout of reading areas 203 (203a to 203e) for reading characters and numbers is defined for each of the items 202a to 202e. For example, the form definition data 125 defines that reading area 203b is to be placed around the corresponding item 202b (near the right side in FIG. 3).

[0053] Each of the items 202e to 202g, "Date of Birth," "Sex," and "Marital Status," has a plurality of selection items 204 pre-printed, which are selected by checking a check mark. For example, below the "Date of Birth" item 202e, four selection items 204a to 204d, which are the character strings "Meiji," "Taisho," "Showa," and "Heisei," are pre-printed. For item 202e, any one of the four selection items 204a to 204d is selected by checking a check mark for each of the four selection items 204a to 204d.

[0054] Similarly, two selection items 204e and 204f, which are the characters "male" and "female," are pre-printed on the side (right side in FIG. 3) of the "gender" item 202f. Two selection items 204g and 204h, which are the characters "married" and "not yet married," are pre-printed on the side (right side in FIG. 3) of the "married" item 202g.

[0055] That is, selection areas 205 (205a to 205c) are set in the vicinity (surroundings) of each of the items 202e to 202g, allowing a user to select multiple selection items 204 (204a to 204h) by placing a check mark. The form definition data 125 defines the areas in which these selection areas 205 are arranged. For example, the form definition data 125 defines that selection area 205a is to be arranged around the corresponding item 202e (in this example, near the bottom in FIG. 3).

[0056] 3 is an example of a semi-standard form, and therefore the form definition data 125 does not define the position of each item 202 within the form 200, as described above. The areas in which the reading area 203 and the selection area 205 are located do not have defined position coordinates, but are defined as areas based on the corresponding items 202 (relative positions to the items 202).

[0057] [Form reading procedure] Next, the procedure for the recognition process of a form by the character recognition device 1 of the first embodiment, in particular the procedures for the area specification process and the determination process, will be described with reference to Figures 4 to 14. Note that the following description is an example in which the form 200 shown in Figure 3 is to be recognized.

[0058] When recognition processing of a form (semi-standard form) by the character recognition device 1 of the embodiment is started, first, in step S01, the character recognition unit 1301 executes full-text recognition processing of the form 200, as shown in Fig. 4. More specifically, as shown in an example in Fig. 5, in step S011, the form definition data (format file) 125 and scanned image data 122 stored in the storage device 102 are acquired. As described above, the form definition data 125 is created and registered in advance by the form definition unit 12, and the scanned image data 122 is acquired by the form image acquisition unit 11 from the scanner device 2 or the like.

[0059] Next, in step S012, character strings are extracted from the form 200 using the scanned image data 122. This character string extraction targets all character strings written in the form 200. For example, in the case of the form 200, all character strings including the form name 201, each item 202, and selection items 204 are extracted.

[0060] Next, character recognition processing (OCR processing) is performed on the extracted character string (step S013), and the character recognition processing result is output (step S014). In this example, as the output of the character recognition processing result, character recognition result data 123 created from the character recognition processing result is stored in storage device 102. Note that in the character recognition device 1 of the embodiment, AI-OCR processing using AI technology is performed as the character recognition processing. This makes it possible to improve the recognition accuracy of character strings extracted from forms.

[0061] In this example, multiple candidate characters are output as the character recognition processing result for each extracted character string. More specifically, during character recognition in step S013, each extracted character string is segmented character by character, and multiple candidate characters are output for each segmented character (step S014). As an example, during character recognition processing, the likelihood (also called confidence) of each segmented character candidate character is determined, and multiple candidate characters are output as the character recognition processing result in descending order of likelihood. This completes the flow of the full-text recognition processing in step S01 shown in FIG. 4.

[0062] FIG. 6 is a diagram for explaining an example of a character recognition processing result. The character recognition processing result shown in FIG. 6 is an example of the result of performing character recognition processing on the character string of "date of birth" which is the item 202e from the read image data 122 of the form 200. As shown in FIG. 6, a plurality of candidate characters are respectively displayed corresponding to each of the cut-out characters "生", "年", "月", and "日" of "date of birth". More specifically, for each of the cut-out characters "生", "年", "月", and "日", the 1st candidate, 2nd candidate, 3rd candidate, etc. are shown in descending order of the value of the likelihood d. In this example, the likelihood d can also be referred to as the confidence level and is a value from 0 to 100. The method for calculating the likelihood d is not particularly limited, and existing techniques can be adopted, so the description here is omitted.

[0063] In the example of FIG. 6, for the cut-out characters "生" and "年", among the plurality of candidate characters, the correct characters "生" and "年" are recognized as the 1st candidates. For the cut-out character "月", among the plurality of candidate characters, the incorrect character "日" is recognized as the 1st candidate, and the correct character "月" is recognized as the 2nd candidate. Similarly, for the cut-out character "日", the incorrect character "臼" is recognized as the 1st candidate, and the correct character "日" is recognized as the 2nd candidate.

[0064] In this character recognition processing result, the name (character string) of the item 201e, "date of birth", may be misread (misrecognized) as "生年日臼". In order to suppress such misreading of the character string, in this example, the recognition result (reading result) of each character string is corrected using the dictionary data 126 stored in the storage device 102. That is, by referring to the character information of the dictionary data 126, an appropriate character string "date of birth" is output as the recognition result from the combinations of the plurality of candidate characters of the cut-out characters "生", "年", "月", and "日". Of course, the plurality of candidate characters for each cut-out character as shown in FIG. 6 may be displayed on the screen of the output device 106 so that the user U1 can select the correct candidate character.

[0065] Next, proceeding to step S02 in Fig. 4, the area identification unit 1302 executes an area identification process to identify the read area 203 and the selected area 205 of the form 200. More specifically, as shown in an example in Fig. 7, first, in step S021, the character recognition result data 123 and the form definition data 125 are acquired. Next, using the character recognition result data 123 and the form definition data 125, a keyword search is executed using a preset word (step S022). That is, a keyword search is performed using the preset word, and detection results of each item 202, etc. of the form 200 are acquired from character strings recognized by full-text recognition. Then, the position of each item 202, etc. within the form 200 is identified from the results of the keyword search.

[0066] As an example, the set word is "date of birth," and item 202e is searched for among all character strings recognized in form 200. If a single character string "date of birth" is obtained through a keyword search, that character string is recognized as each item 202e. Of course, this keyword search may also search for character strings other than item 202.

[0067] Incidentally, this keyword search uses character recognition result data 123 created as a result of full-text recognition. For this reason, if, for example, the extracted character string is misread or noise is mixed in during the full-text recognition process in step S01, there is a risk that each item 202, etc. will not be properly output as a search result during the keyword search.

[0068] However, in this example, in the full-text recognition process (step S01) as described above, a plurality of character candidates are output for the segmented characters of the extracted character string, and the recognition result of each character string is corrected using dictionary data 126, so that an appropriate character string is output as the recognition result. This makes it easier to output appropriate search results in the keyword search in step S022 as well.

[0069] Furthermore, when multiple candidate characters for each extracted character are output as the full-text recognition result as shown in Fig. 6, a keyword search may be performed using combinations of these candidate characters as search targets. This method also makes it easier to output appropriate search results.

[0070] Next, in step S023, the area identification unit 1302 identifies the selection area 205 together with the reading area 204 of the form 200. As an example, the identification of the selection area 205a corresponding to the item 202e will be described. As described above, the area in which the selection area 205a is located is defined in the form definition data 125 as a relative position with respect to the corresponding item 202e. For example, as shown in FIG. 8, the area 206 in which the selection area 205a is located is defined in the form definition data 125 as an area adjacent to the lower side of the character string "Date of Birth" in the item 202e. Furthermore, the position coordinates of the character string "Date of Birth" in the form 200 are identified by full-text recognition processing.

[0071] Therefore, the area identification unit 1302 can identify the selection area 205a of the form 200 from the character recognition result data 123, the form definition data 125, and the results of the keyword search. The area identification unit 1302 recognizes the character strings "Meiji," "Taisho," "Showa," and "Heisei" pre-printed in the area 206 below the character string "Date of Birth" in the item 202e as the selection items 204a to 204d, and identifies the range in which these selection items 204a to 204d are written as the selection area 205a.

[0072] Next, in step S024, the multiple selection items 204 in each selection area 205 are individually separated. In other words, each selection area 205 is divided into multiple character areas corresponding to each selection item 204. For example, as shown in FIG. 9, in the case of selection area 205a, selection area 205a is divided into four character areas corresponding to selection items 204a to 204d, "Meiji," "Taisho," "Showa," and "Heisei." This completes the flow of the area identification process in step S02 shown in FIG. 4.

[0073] 4, the mark determination unit 1303 performs a process of determining whether or not a check mark has been added to the selection item 204 for each selection area 205, based on the form image. For example, in the case of the selection area 205a corresponding to the "Date of Birth" item 202e, the mark determination unit 1303 performs a process of determining whether or not a check mark has been added to each of the four selection items 204a to 204d, which are the character strings "Meiji," "Taisho," "Showa," and "Heisei," based on the scanned image data 122 as the form image. In other words, the determination process determines which of the four selection items 204a to 204d has a check mark added.

[0074] Since the determination process for each of the selection items 204a to 204d is similar, the determination process for the selection item 204a, "Meiji," will be described here. As shown in FIG. 10, first, in step S031, the character size (sometimes called character component size) of the selection item 204a is obtained from the scanned image data 122. That is, the mark determination unit 1303 calculates the character size of the selection item 204a by image recognition processing, not character recognition processing (OCR processing). Hereinafter, the character size of the selection item 204 obtained (calculated) by the mark determination unit 1303 will be referred to as the "obtained character size."

[0075] Next, in step S032, the acquired character size of the selection item 204a is compared with the specified character size. Specifically, it is determined whether the acquired character size of the selection item 204a is larger than the specified character size. If the acquired character size of the selection item 204a is larger than the specified character size (step S032: Yes), the process proceeds to step S033, where it is determined that the selection item 204a has a check mark 207 (marked). In this example, the specified character size corresponds to the pre-print font size and is defined in the form definition data 125.

[0076] 11, when a check mark 207 is added to a selection item 204a, the range of the selection item 204a including the check mark 207 is recognized as a character range 208. That is, the mark determination unit 1303 calculates the acquired character size of the selection item 204a based on this character range 208. Therefore, when the check mark 207 is added to surround the outside of the selection item 204a as shown in FIG. 11, the acquired character size of the selection item 204a is larger than the specified character size. Therefore, by comparing the acquired character size of the selection item 204a with the specified character size, it can be determined that the selection item 204a has a check mark 207 added.

[0077] In this example, if the acquired character size is larger than the specified character size in either the vertical or horizontal direction of the selection item 204a, it is determined that a mark is present. Of course, it may also be determined that a mark is present if the acquired character size is larger than the specified character size in both the vertical and horizontal directions of the selection item 204a.

[0078] The shape of the check mark 207 is not limited to the oval shape surrounding the selection item 204a as shown in FIG. 11. FIG. 12 is a diagram showing modified examples of the check mark. The shape of the check mark 207 is not particularly limited and may be, for example, a circle (Modification 1), a rectangle (Modification 2), or a so-called tick mark (Modification 3). Even if the check mark 207 has a shape like Modifications 1 to 3, if the check mark 207 is placed outside the selection item 204a or if it extends from above the selection item 204a to the outside of the selection item 204a, the acquired character size of the selection item 204a will be larger than the specified character size. Therefore, regardless of the shape of the check mark 207, it can be appropriately determined that the selection item 204a has the check mark 207 by comparing the acquired character size with the specified character size.

[0079] In this example, the standard character size of the selection item 204 is used as the criterion for determining whether or not a check mark 207 is added, but for example, a first threshold value larger than the standard character size may be used as the criterion. That is, the mark determination unit 1303 may determine that a mark is added when the acquired character size of the selection item 204 is equal to or larger than the first threshold. This makes it possible to more accurately determine whether or not a check mark 207 is added to the selection item 204.

[0080] On the other hand, if the acquired character size of the selection item 204 is the same as or smaller than the specified character size in step S032 (step S032: No), it can be determined that the selection item 204 does not have a check mark 207 (no mark). However, even if the acquired character size of the selection item 204 is the same as the specified character size, the selection item 204 may have a check mark 207. For example, as shown in FIG. 13, if the check mark 207 is placed on the selection item 204a, that is, if the selection item 204a is placed within the pre-printed area, the acquired character size of the selection item 204a will be approximately the same as the specified character size.

[0081] Therefore, in this example, if the acquired character size of the selection item 204 is the same as or smaller than the specified character size (step S032: No), the process proceeds to step S034, where the likelihood d of the character string "Meiji" in the selection item 204 is evaluated. As described above, during the character recognition process, the likelihood d is calculated for each character on the form. In step S034, the likelihood d of the selection item (recognized character) 204 is evaluated based on the character recognition result data 123, which is the result of the character recognition process (OCR process). As an example, it is evaluated whether the likelihood d of each extracted character in the selection item 204 is smaller than a determination threshold dt. Thereafter, in step S035, it is determined whether the likelihood d of each extracted character in the selection item 204 is smaller than a predetermined determination threshold dt.

[0082] If the likelihood d of any one of the extracted characters in the selection item 204 is smaller than the determination threshold dt (step S035: Yes), the selection item 204 is determined to have a check mark 207 (marked) (step S036). If the selection item 204 has a check mark 207, components other than the original character increase, and the character-likelihood (likelihood) of the selected item decreases. Therefore, if the likelihood d is smaller than the determination threshold dt in step S035, it can be determined that the selection item 204 has a check mark 207. This makes it possible to more accurately determine whether the selection item 204 has a check mark 207. The method for evaluating the likelihood d of the selection item 204 in step S034 is not particularly limited, and for example, it may be evaluated whether the average value of the likelihoods d of the extracted characters is smaller than the determination threshold dt.

[0083] In step S035, if the likelihood d of the selection item 204 is equal to or greater than the determination threshold dt (step S035: No), it can be determined that the selection item 204 is not checked (unmarked). However, in this example, in this case, the process proceeds to step S037, where graphic recognition processing is performed to determine whether the selection item 204 is checked. In step S037, graphic recognition processing is performed on the selection item 204 using, for example, learning data for the selection item 204, which is a character string pre-printed on the form 200 with the check mark 207. The learning data is created by machine learning using, for example, deep learning. This makes it possible to more accurately determine whether the selection item 204 is checked. The above is the flow of the determination processing in step S03 shown in FIG. 4.

[0084] Incidentally, there is a risk that the use of graphic recognition processing to determine the presence or absence of a mark may increase the time required for the recognition process of the form. However, as in this example, by performing graphic recognition processing in a limited manner depending on the determination result of step S035, it is possible to more appropriately determine the presence or absence of a mark while preventing the recognition process of the form from taking too long.

[0085] In addition, in this example, the determination of the presence or absence of a mark by the graphic recognition process is performed in accordance with the determination result of the presence or absence of a mark based on the evaluation of the likelihood d, but it may be performed instead of the determination of the presence or absence of a mark based on the evaluation of the likelihood d. The determination of the presence or absence of a mark by the graphic recognition process may be performed instead of the determination of the presence or absence of a mark based on the evaluation of the likelihood d and the character size. Furthermore, the determination of the presence or absence of a mark based on the evaluation of the likelihood d is performed in accordance with the determination result of the presence or absence of a mark based on the character size, but it may be performed instead of the determination of the mark based on the character size.

[0086] 4, the form recognition processing unit 13 creates form recognition result data 124 including the result of the determination process and stores it in the storage device 102. The screen display unit 14 also displays the recognition result information based on the form recognition result data 124 stored in the storage device 102 on the display of the output device 106. The user U1 can check and correct the recognition result information on the display screen of the output device 106.

[0087] FIG. 14 is an example of a GUI screen showing the results of the determination process, displayed on the output device 106. As shown in FIG. 14, for the "Date of Birth" item 202e, for those of the selection items 204a-204d, "Meiji," "Taisho," "Showa," and "Heisei," that are determined to have check marks 207, "*" is displayed in the "Meiji," "Taisho," "Showa," and "Heisei" columns corresponding to the selection items 204a-204d. By checking this display, the user U1 can easily confirm which of the selection items 204a-204d has been selected. Furthermore, if no "*" is displayed or if two or more "*"s are displayed, an error message may be displayed.

[0088] Although the embodiments of the present disclosure have been specifically described above, they are not limited to the above-described embodiments and can be modified in various ways without departing from the spirit of the present disclosure. Except for essential components, components can be added, deleted, or replaced in each embodiment. Unless otherwise specified, each component can be singular or plural. A combination of each embodiment and its variations is also possible.

[0089] For example, in the above embodiment, an example was described in which the acquired character size of a selected item is compared with the specified character size to determine whether or not the selected item has a check mark, but the comparison target for the acquired character size is not limited to the specified character size. For example, the acquired character sizes of multiple selected items may be compared, and if the acquired character size of one selected item is larger than the other selected items, it may be determined that the selected item has a check mark.

[0090] In the above embodiment, for example, an example was described in which the layout of the selection area in the form definition data was defined based on the item name, but the criteria for the layout of the selection area are not particularly limited. For example, the layout of the selection area may be defined based on one of the selection items. In this case, the position of one of the selection items that serves as the basis is identified by a keyword search, and the selection area is identified from the position of the identified selection item. [Explanation of symbols]

[0091] 1...character recognition device (computer), 2...scanner device, 11...form image acquisition unit, 12...form definition unit, 13...form recognition processing unit, 14...screen display unit, 15...setting unit, 101...arithmetic unit, 102...storage device, 103...communication interface device, 104...input / output interface device, 105...input device, 106...output device, 107...external storage device, 200...form, 1301...character recognition unit, 1302...area identification unit, 1303...mark determination unit

Claims

1. a form image acquisition unit that acquires an image of a form; a character recognition unit that recognizes characters in the form from the form image acquired by the form image acquisition unit, A character recognition device capable of recognizing a plurality of types of forms that are configured with similar items but have different arrangements of the items, an area specifying unit that specifies a selection area in which the selection items are read based on the character recognition result by the character recognition unit when the form has a plurality of selection items configured with pre-printed character strings and selected by check marks; a mark determination unit that performs a process of determining whether or not the check mark is placed on the selection item in the selection area based on the form image, Character recognition device.

2. 2. The character recognition device according to claim 1, The mark determination unit the determination process includes acquiring a character size of the selection item from the form image, comparing the acquired character size with a specified character size of the selection item, and determining whether or not the selection item has been checked based on the result of the comparison. Character recognition device.

3. 3. The character recognition device according to claim 2, The mark determination unit acquiring a character size of the selection item from the form image, and determining that the selection item has the check mark if the acquired character size is larger than the specified character size; Character recognition device.

4. 3. The character recognition device according to claim 2, The mark determination unit acquiring a character size of the selection item from the form image, and determining that the selection item has the check mark if the acquired character size is equal to or greater than a first threshold value that is larger than the specified character size; Character recognition device.

5. 3. The character recognition device according to claim 2, The mark determination unit acquiring a character size of the selection item from the form image, and determining that the selection item is not checked if the acquired character size is equal to or smaller than the specified character size; Character recognition device.

6. 3. The character recognition device according to claim 2, The mark determination unit a character size of the selection item is acquired from the form image, and if the acquired character size is equal to or smaller than the specified character size, the determination process further includes determining whether or not the selection item has been checked based on the likelihood of the selection item acquired from the character recognition result by the character recognition unit. Character recognition device.

7. 7. The character recognition device according to claim 6, The mark determination unit If the likelihood of the selected item is lower than a preset threshold, the selected item is determined to have the check mark. Character recognition device.

8. 8. The character recognition device according to claim 7, The mark determination unit If the likelihood of the selected item is equal to or greater than a predetermined threshold, the method further determines whether or not the selected item has been checked using learning data of character strings pre-printed on the form with the check mark. Character recognition device.

9. 2. The character recognition device according to claim 1, The mark determination unit the determination process determines whether the check mark is placed on the selection item based on a likelihood of the selection item obtained from a character recognition result by the character recognition unit. Character recognition device.

10. 9. The character recognition device according to claim 8, The mark determination unit If the likelihood of the selected item is lower than a preset threshold, the selected item is determined to have the check mark. Character recognition device.

11. 2. The character recognition device according to claim 1, The mark determination unit In the determination process, learning data of character strings pre-printed on the form to which the check mark is added is used to determine whether or not the check mark is added to the selection item. Character recognition device.

12. a form image acquisition step of acquiring an image of the form; a character recognition step of recognizing characters of the form from the form image acquired in the form image acquisition step, A character recognition method capable of recognizing a plurality of types of forms that are composed of similar items but have different arrangements of the items, comprising: an area specifying step of specifying a selection area for reading the selection items based on the character recognition result in the character recognition step, when the form has a plurality of selection items configured by pre-printed character strings and selected by check marks; and a determination step of determining whether or not the check mark is placed on the selection item in the selection area based on the form image. Character recognition method.

13. A form image acquisition process for acquiring an image of the form; character recognition processing for recognizing characters of the form from the form image acquired by the form image acquisition processing, A program to be executed by a character recognition device capable of recognizing multiple types of forms that are composed of similar items but have different arrangements of the items, an area specification process for specifying a selection area for reading the selection items based on the character recognition result in the character recognition process when the form has a plurality of selection items configured with pre-printed character strings and selected by check marks; and causing the character recognition device to execute a determination process for determining whether or not the selected item has been checked based on the form image for the selected area. program.

Images