Form processing device, form processing method, and form processing program

Abstract

To provide a business form processing device, a business form processing method, and a business form processing program that can perform suitable business form processing regardless of the results of the ruled line extraction processing.SOLUTION: The business form processing device has a ruled line extraction processing unit that performs ruled line extraction processing on the business form, and a business form recognition processing unit that executes business form recognition processing in different ways in the case where the number of ruled lines extracted by the ruled line extraction processing is greater than or equal to a predetermined threshold number of lines and in the case where the number of ruled lines is less than the predetermined threshold number of lines. The business form recognition processing unit registers in advance a fixed character string candidate and a variable character string candidate that may be included in the business form, as business form recognition processing when the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is less than a predetermined threshold number, then compares the character string candidate read from the business form with the fixed character string candidate and the variable character string candidate.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a form processing device, a form processing method, and a form processing program. [Background technology]

[0002] Patent Document 1 describes a form processing device that processes forms. This form processing device includes an image reading unit, a ruled line extraction unit, a cell area recognition unit, a character string recognition unit, a line information acquisition unit, a character string attribute determination unit, and a content item acquisition unit.

[0003] The image reading unit acquires a form image by having an image reading device read the form. The ruled line extraction unit extracts vertical and horizontal ruled lines from the form image read by the image reading unit. The cell area recognition unit recognizes cell areas based on the vertical and horizontal ruled lines extracted by the ruled line extraction unit. The string recognition unit recognizes strings by performing character recognition processing for each cell area recognized by the cell area recognition unit. The line information acquisition unit acquires line information consisting of strings recognized by the string recognition unit. The string attribute determination unit determines the attributes of the strings for each piece of line information acquired by the line information acquisition unit. When two or more pieces of line information are acquired by the line information acquisition unit within the cell area recognized by the cell area recognition unit, the content item acquisition unit acquires the string recognized by the string recognition unit as a specific content item written on the form based on the attributes determined by the string attribute determination unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-018206 Summary of the Invention [Problem to be solved by the invention]

[0005] However, Patent Document 1 leaves room for improvement in that there is a risk of insufficient form processing when the ruled lines (vertical and horizontal ruled lines) of a form cannot be extracted properly, or when a form that does not have ruled lines (vertical and horizontal ruled lines) is being targeted.

[0006] In view of the above circumstances, an object of the present invention is to provide a form processing device, a form processing method, and a form processing program that can execute suitable form processing regardless of the result of the ruled line extraction process. [Means for solving the problem]

[0007] The form processing device of the present disclosure has a ruled line extraction processing unit that executes a ruled line extraction process on a form, and a form recognition processing unit that executes a form recognition process in different modes when the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is equal to or greater than a predetermined threshold and when the number is less than the predetermined threshold. and the form recognition processing unit, as the form recognition processing when the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is less than a predetermined threshold, pre-registers fixed character string candidates and variable character string candidates that may be included in the form, and compares the character string candidates read from the form with the fixed character string candidates and the variable character string candidates. It is characterized by:

[0008] The form processing method of the present disclosure includes a step of executing a ruled line extraction process on a form by a ruled line extraction processing unit, and a step of executing a ruled line extraction process on a form by a form recognition processing unit. The ruled line extraction process a step of executing a form recognition process in different modes when the number of extracted vertical and horizontal ruled lines is equal to or greater than a predetermined threshold value and when the number is less than the predetermined threshold value; In the step of executing the form recognition processing by the form recognition processing unit, when the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is less than a predetermined threshold, fixed character string candidates and variable character string candidates that may be included in the form are pre-registered, and the character string candidates read from the form are compared with the fixed character string candidates and the variable character string candidates. It is characterized by:

[0009] The form processing program of the present disclosure includes a step of executing a ruled line extraction process on a form by a ruled line extraction processing unit; The ruled line extraction process a step of executing a form recognition process in different modes when the number of extracted vertical and horizontal ruled lines is equal to or greater than a predetermined threshold value and when the number is less than the predetermined threshold value; and in the step of executing the form recognition processing by the form recognition processing unit, when the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is less than a predetermined threshold, a fixed character string candidate and a variable character string candidate that may be included in the form are pre-registered, and the character string candidate read from the form is compared with the fixed character string candidate and the variable character string candidate. The present invention is characterized in that the processor executes the following. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a form processing device, a form processing method, and a form processing program that can execute suitable form processing regardless of the result of the ruled line extraction process. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration including a form processing device. [Figure 2] FIG. 2 is a functional block diagram showing an example of the internal configuration of a form processing device. [Figure 3] FIG. 2 is a block diagram showing an example of a hardware configuration of a form processing device. [Figure 4] FIG. 2 is a functional block diagram showing an example of the internal configuration of a form recognition processing unit. [Figure 5] 10A and 10B are diagrams illustrating examples of fixed character string candidates and variable character string candidates included in a form. [Figure 6] 10A and 10B are diagrams illustrating examples of fixed character string candidates and variable character string candidates registered in a definition body. [Figure 7] 10 is a flowchart showing a first example of form processing according to the present embodiment. [Figure 8] 10 is a flowchart showing a second example of form processing according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] The form processing device, form processing method, and form processing program disclosed herein perform character recognition of predetermined items from image information of a form that has been optically read, for example. The form processing method and form processing program disclosed herein are realized by causing a processor included in the form processing device to execute various processing steps. Note that the processor may also be read as a computer (CPU: Central Processing Unit).

[0014] FIG. 1 is a diagram showing an example of a system configuration including a form processing device. As shown in FIG. 1, form processing system 1 includes an image reading device 2 and a form processing device 3. Image reading device 2 and form processing device 3 are connected to each other so that they can communicate with each other. Form processing system 1 is installed in various facilities, including financial institutions and public institutions, and is used for various form processing at the facilities. For example, in banks and the like, forms are read by an image reading device such as a scanner, and text information is obtained from the resulting form image through optical character reading (OCR) processing, thereby improving the efficiency of administrative processing such as data entry by bank employees.

[0015] The image reading device 2 is configured, for example, by an image scanner, and optically reads the form and transmits the reading results to the form processing device 3. The form processing device 3 is configured, for example, by a workstation, a server, a processor, a personal computer, and performs form processing, which will be described later, on the form reading results transmitted from the image reading device 2.

[0016] 2 is a functional block diagram showing an example of the internal configuration of a form processing device 3. As shown in FIG. 2, the form processing device 3 has a control unit 30, a scanner control unit 31, a ruled line extraction processing unit (ruled line reading processing unit) 32, a form recognition processing unit 33, and a recognition result display unit 34.

[0017] The control unit 30 controls a series of processes performed by the scanner control unit 31, the ruled line extraction processing unit 32, the form recognition processing unit 33, and the recognition result display unit 34. The scanner control unit 31 controls the image reading device 2 to acquire the written surface of the form as form image data.

[0018] The ruled line extraction processor 32 executes ruled line extraction processing on the form (form image data) acquired by the scanner controller 31. For vertical ruled lines, the ruled line extraction processor 32 extracts whether or not there are vertically consecutive pixels, i.e., vertical straight lines, and outputs the result as the vertical ruled line extraction result. For horizontal ruled lines, the ruled line extraction processor 32 extracts whether or not there are horizontally consecutive pixels, i.e., horizontal straight lines, and outputs the result as the horizontal ruled line extraction result. Note that the ruled line extraction processing by the ruled line extraction processor 32 may be successful or unsuccessful. Specifically, "the ruled line extraction processing is successful" is synonymous with the result that the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is equal to or greater than a predetermined threshold, and "the ruled line extraction processing fails" is synonymous with the result that the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is less than a predetermined threshold. If clear ruled lines appear on the form (form image data), the ruled line extraction process is likely to be successful, but if the ruled lines on the form (form image data) are unclear or do not exist at all, the ruled line extraction process is likely to fail. For example, if the ruled lines are worn or consist of dotted lines or short lines, they may be considered to be characters and may not be extracted as ruled lines.

[0019] The form recognition processing unit 33 executes a predetermined form recognition process on the form (form image data) that has been subjected to the ruled line extraction process by the ruled line extraction processing unit 32. More specifically, the form recognition processing unit 33 executes the form recognition process in different modes depending on whether the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing by the ruled line extraction processing unit 32 is equal to or greater than a predetermined threshold value or whether the number is less than the predetermined threshold value (when the ruled line extraction processing is successful or unsuccessful). Details of the form recognition process in each mode when the ruled line extraction processing is successful or unsuccessful will be described in detail later.

[0020] The recognition result display unit 34 displays the results of the form recognition processing performed by the form recognition processing unit 33 on the display 3D of the form processing device 3 (see FIG. 1).

[0021] Fig. 3 is a block diagram showing an example of the hardware configuration of a form processing device. As shown in Fig. 3, the form processing device 3 has a CPU (Central Processing Unit) 300, an input device 301, an output device 302, a ROM (Read Only Memory) 303, a RAM (Random Access Memory) 304, an interface (I / F) 305, and a bus 306 that connects these components.

[0022] The input device 301 includes, for example, a keyboard, joystick, light pen, mouse, touchpad, touch panel, trackball, etc., and inputs various data, signals, etc. The output device 302 includes, for example, various displays such as an LCD (Liquid Crystal Display), a printer, etc., and outputs images and other information. The ROM 303 stores a program for executing form processing in the form processing device 3, as well as control programs and table data for controlling and executing each function of the form processing device 3. The RAM 304 stores a frame buffer for the output device 302, some application programs, etc. The interface 305 is a unit for connecting to external devices, and includes a serial interface such as USB (Universal Serial Bus) or a parallel interface such as Ethernet (registered trademark). The CPU 300 comprehensively controls the above components.

[0023] Conventional document processing devices, including those described in Patent Document 1, require advance preparation of a definition body that stores ruled line information and the coordinates of items to be character-recognized based on the layout of the document to be recognized before performing document identification and character recognition. When performing document identification and character recognition, the system acquires ruled line information from the document and compares it with the definition body. If there is a match, the system performs character recognition using the coordinates of the items to be character-recognized registered in the definition body, and notifies the results. Thus, conventional document processing devices rely on the assumption that the document contains ruled line information and that the ruled line information can be reliably acquired. If this assumption is violated, document processing may be inadequate. For example, for documents without ruled lines, such as bank transfer slips, it is impossible to acquire ruled line information, and therefore it is not possible to prepare a definition body in advance, making it impossible to register or use the coordinates of items to be character-recognized.

[0024] The inventors have recognized the above-mentioned problems as important technical challenges and have come up with the idea of a form processing device, a form processing method, and a form processing program that can perform suitable form processing regardless of the results of the line extraction process, i.e., even when the form's line cannot be successfully extracted, or when a form without line is being processed.

[0025] Specifically, in this embodiment, the form recognition process is executed in different modes depending on whether the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is equal to or greater than a predetermined threshold or whether the number is less than the predetermined threshold (whether the ruled line extraction process is successful or unsuccessful).If the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is equal to or greater than a predetermined threshold (when the ruled line extraction process is successful), the form characteristics are registered in a definition body using ruled line information (vertical and horizontal attributes, position, length, etc.) as a key, and the position and attributes (character type, etc.) of character recognition items are registered as specified by the user, and the form recognition process is executed based on this registered information.

[0026] If the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is less than the predetermined threshold (if the ruled line extraction process fails), the system pre-registers fixed and variable string candidates that may be included in the form, and compares the string candidates read from the form with the fixed and variable string candidates to perform form recognition. The system also analyzes the string candidates read from the form and classifies them as either fixed or variable string candidates, thereby updating the form characteristics (particularly the fixed string candidates) registered in the definition body. The system then registers the positions and attributes (character type, etc.) of the character recognition items as specified by the user, and performs form recognition based on this registration information. In this way, for forms for which ruled line information cannot be obtained, character strings (fixed and variable strings) that can identify the form type are pre-registered. When actually performing form identification / character recognition, character strings are extracted from the relevant form and compared with the definition body (fixed and variable strings), allowing character recognition to be performed and the results to be notified. Even for reports where the lines cannot be obtained, the character recognition results can be notified by comparing them with the definition body (fixed character string, variable character string) and if they have been registered.

[0027] 4 is a functional block diagram showing an example of the internal configuration of the form recognition processing unit 33. As shown in FIG. 4, the form recognition processing unit 33 has a definition body comparison unit 33A, a character recognition processing unit 33B, a character string candidate allocation unit 33C, a definition body 33D, and a character recognition dictionary 33E.

[0028] The definition body comparison unit 33A refers to a definition body 33D in which fixed and variable character string candidates are pre-registered, and compares the character string candidate read from the form with the fixed and variable character string candidates registered in the definition body 33D. Fixed character string candidates may be interpreted as fixed wording, and variable character string candidates may be interpreted as variable wording. Fixed character string candidates (fixed wording) are character strings (candidates) that are always included with the same content in the same type of form, such as form titles like "Transfer Details" and "Local Government Employee Accident Compensation Fund," and recognition item names like "Bank Name" and "Branch Name." Variable character string candidates (variable wording) are character strings (candidates) that are included with different content in each form, even in the same type of form, such as specific branch names like "Kawasaki" and "Omiya," as well as specific monetary values. As a form recognition process in the event that the line extraction process by the line extraction processing unit 32 fails (when the number of vertical and horizontal lines extracted by the line extraction process is less than a predetermined threshold value), the definition body comparison unit 33A pre-registers fixed string candidates and variable string candidates that may be included in the form in the definition body 33D, and compares the string candidates read from the form with the fixed string candidates and variable string candidates.

[0029] Fig. 5 is a diagram showing an example of fixed string candidates (fixed wording) and variable string candidates (variable wording) included in a form. In Fig. 5, fixed string candidates (fixed wording) are written in the area surrounded by a dashed rectangular frame, and variable string candidates (variable wording) are written in the area surrounded by a dotted oval frame. Specifically, Fig. 5 shows the following fixed string candidates (fixed wording): "Transfer Details," "Local Government Officials Accident Compensation Fund," "Bank Name," "Branch Name," "Type (Abbreviation for Account Type)," "Account Number," "Account Holder," "Amount," "Number of Accounts," and "Remarks." Additionally, Figure 5 shows the variable string candidates (variable wording), including "Fujitsu" as the bank name, "Kawasaki, Omiya" as the branch name, "Fu (savings deposit), Tou (current deposit)" as the type (abbreviation of account type), "1234567, 0987654" as the account number, "Fujitsu Taro, Fujitsu Hanako" as the account holder, and "83,620 yen, 1,580 yen" as the amount.

[0030] 6A to 6C are diagrams showing examples of fixed string candidates (fixed text) and variable string candidates (variable text) registered in a definition body. As shown in FIG. 6A, one index is assigned to a certain type (one type) of form, and multiple fixed string candidates (fixed text) and variable string candidates (variable text) are linked to that one index. In this case, there may be no correlation between the fixed string candidates (fixed text) and the variable string candidates (variable text). As shown in FIG. 6B, for Index 1 corresponding to a certain form, the contents of the fixed string candidate (fixed text) (transfer details, local government employee accident compensation fund, bank branch, ..., remarks, ...) and the coordinates of the fixed string candidate (fixed text) ((200, 10, 400, 50), (Xs, Ys, Xe, Ye), ...) are linked. Similar information is registered for Index 2 and subsequent indexes corresponding to other forms. As shown in Figure 6C, the coordinates ((100, 200, 300, 400), (Xs, Ys, Xe, Ye), ...) of the variable character string candidate (variable text) are linked to Index 1 corresponding to a certain form. Similar information is registered for Index 2 and subsequent Indexes corresponding to another form. Note that although shown as blank (...) in Figures 6B and 6C, additional information such as character type may also be registered for the fixed character string candidate (fixed text) and variable character string candidate (variable text).

[0031] The character recognition processing unit 33B recognizes characters written on the form writing surface from the form image data acquired by the scanner control unit 31. At this time, a character recognition dictionary 33E is used to recognize the characters written on the form. The character recognition processing by the character recognition processing unit 33B can use known character recognition technology, so further detailed explanation will be omitted.

[0032] Next, the function of the character string candidate allocation unit 33C will be described. As described above, when determining the form type using characters on the form during pre-registration in the definition body 33D, it is necessary to distinguish between fixed character string candidates (fixed wording) and variable character string candidates (variable wording) and allocate them. For example, since fixed character string candidates (fixed wording) are often printed in a predetermined (constant) font that is easy to recognize, it is possible to automatically distinguish between fixed character string candidates (fixed wording) and variable character string candidates (variable wording) and allocate them by specifying the font to be used in advance or by using information on the character recognition results.

[0033] The character string candidate allocating unit 33C may allocate character string candidates read from a form to either fixed character string candidates or variable character string candidates using at least one of the size, arrangement, and attributes (character type, etc.) of the character string candidates read from the form as an index. For example, the larger the size of the character string candidate read from the form, the higher the likelihood that the character string candidate will be allocated to a fixed character string candidate, and the smaller the size of the character string candidate read from the form, the higher the likelihood that the character string candidate will be allocated to a variable character string candidate. Furthermore, the more conspicuous the arrangement of the character string candidate read from the form, such as at the top of the form, the higher the likelihood that the character string candidate will be allocated to a fixed character string candidate, and the more inconspicuous the arrangement of the character string candidate read from the form, such as at the bottom of the form, the higher the likelihood that the character string candidate will be allocated to a variable character string candidate. Furthermore, if a character string candidate read from a form is printed in a predetermined (constant) font that is easy to recognize, the character string candidate is more likely to be assigned to a fixed character string candidate, and if the character string candidate read from a form is handwritten, the character string candidate is more likely to be assigned to a variable character string candidate. In this way, it is preferable to assign the character string candidate read from a form to either a fixed character string candidate or a variable character string candidate by comprehensively taking into consideration the size, arrangement, and attributes (character type, etc.) of the character string candidate read from the form.

[0034] Alternatively, the character string candidate allocation unit 33C may allocate the character string candidates read from the form to either fixed character string candidates or variable character string candidates using at least one of the following indicators: the character string candidates read from the form are positioned at the top of the form; the character string candidates read from the form have a high degree of certainty; and the character string candidates read from the form are spaced at regular intervals.

[0035] Alternatively, the character string candidate allocation unit 33C may define A as the distance of a character string candidate read from a form from the top edge of the form, B as the certainty of the character string candidate read from the form, C as the degree of uniformity of the spacing between the character string candidates read from the form, and D as the degree of uniformity when comparing the left end / right end of a character string candidate read from a form with the left end / right end of another character string candidate, and define N1, N2, N3, and N4 as the same or different coefficient values, and allocate the character string candidate read from the form to a fixed character string candidate if A×N1+B×N2+C×N3-D×N4 is equal to or greater than a predetermined threshold, and allocate the character string candidate read from the form to a variable character string candidate if A×N1+B×N2+C×N3-D×N4 is less than the predetermined threshold. There is a degree of freedom in how the specified threshold value is set, and various design changes are possible. For example, the threshold value may be set based on statistical information or externally specified information, taking into account the units (scales) of A, B, C, and D and the system environment.

[0036] In this case, N1, N2, N3, and N4 may be set to coefficient values that satisfy N1>N2>N3>N4, so that the distance A of the character string candidate read from the form from the top edge of the form is weighted most heavily, the certainty B of the character string candidate read from the form is weighted second most heavily, the degree of uniformity C of the spacing of the character string candidates read from the form is weighted third most heavily, and the degree of uniformity D when comparing the left and right ends of the character string candidate read from the form with the left and right ends of other character string candidates is weighted least heavily. This is because the inventor's extensive research revealed that the order of influence on the likelihood that a character string candidate read from the form corresponds to a fixed character string candidate is A>B>C>D. Of course, there is a degree of freedom in the manner in which the coefficient values of N1, N2, N3, and N4 are weighted, and various design modifications are possible.

[0037] Here, the distance A from the top of the form to the character string candidate read from the form may be, for example, the distance from the top of the form to any point on the character string candidate (top, bottom, vertical middle, etc.). Furthermore, the unit (scale) of the distance A from the top of the form to the character string candidate read from the form may be, for example, cm (centimeter) or mm (millimeter).

[0038] Furthermore, the certainty B of the character string candidate read from the form may be interpreted as a certainty factor. The certainty B of the character string candidate read from the form is parameterized as an index of the certainty (uncertainty) that the character string candidate read from the form matches the fixed character string candidate pre-registered in the definition body 33D through analysis using, for example, artificial intelligence (AI), machine learning, deep learning, or the like.

[0039] Furthermore, the degree of uniformity C of the spacing between character string candidates read from a form may be parameterized, for example, by sampling the spacing between multiple character string candidates, setting the spacing between one of the character string candidates as the reference spacing, and setting the spacing between the other character string candidates as the reference spacing, using as an index whether the reference spacing falls within a predetermined range of the reference spacing (for example, 0.9 × reference spacing < reference spacing < 1.1 × reference spacing). Alternatively, the degree of uniformity C of the spacing between character string candidates read from a form may be parameterized, using as an index the spacing between each character in the same character string candidate. Alternatively, the degree of uniformity C of the spacing between character string candidates read from a form may be parameterized, for example, using as an index the average character spacing between multiple character string candidates.

[0040] Furthermore, the degree of consistency D when comparing the left and right ends of a character string candidate read from a form with those of other character string candidates can be interpreted as the degree of difference when comparing the left and right ends of the character string candidate read from a form with those of other character string candidates. The higher the degree of consistency D when comparing the left and right ends of a character string candidate read from a form with those of other character string candidates, the more likely it is a variable character string candidate. The lower the degree of consistency D when comparing the left and right ends of a character string candidate read from a form with those of other character string candidates, the more likely it is a fixed character string candidate. This is because fixed character strings are printed centered in the first row as recognition item names in table format, while variable character strings such as amounts and account numbers are often printed left-justified or right-justified, meaning that the beginning and end of variable character strings tend to have the same coordinates across multiple lines. Taking Figure 5 as an example, the left end of the character string "Amount" on the form matches "Local Public Servant Accident Compensation Fund," but the right end does not match any other character strings. On the other hand, the left end of the string "83,620 yen" on the form does not match any other strings, but the right end matches "1,580 yen." In the example in Figure 5, the third item onwards has been omitted, but in an actual form, dozens of items may be printed / written, and this matches the left end of the specific amount from the third item onwards. From the above, the lower the degree of consistency when comparing the left and right ends of other string candidates, the more likely it is to be determined to be a fixed string.

[0041] Furthermore, when it is determined that the reports are of the same type at the time of pre-registration (when registering in definition body 33D), if the fixed string candidate on the report is significantly different from the registered fixed string candidate, it may be determined that the fixed string candidate pre-registered in definition body 33D is (appropriately) a variable string candidate, and the registration content of definition body 33D may be updated.

[0042] Next, we will explain how to compare (during recognition) the character string candidates read from the form with the character string candidates (fixed character string candidates, variable character string candidates) pre-registered in the definition body 33D. First, all characters are extracted from the form to be recognized. At this time, there is a possibility of misreading / rejection depending on the state of the form / characters. Next, all indexes corresponding to all forms are searched for indexes that match the fixed character string candidates (fixed wording). At this time, depending on the operation, it is possible to switch between searching for indexes that match all fixed character string candidates (fixed wording), or searching for indexes that match the most fixed character string candidates (fixed wording). Finally, character recognition is performed using the coordinates of the variable character string candidates (variable wording) linked to the index (form type) that is determined to match.

[0043] In addition to the components illustrated and described in FIGS. 2 to 4, the form processing device 3 may also include, for example, an item position identification unit, a specific item processing unit, a cell area recognition unit, a character string recognition unit, a line information acquisition unit, a character string attribute determination unit, a content item acquisition unit, an item name determination unit, and a content item acquisition unit. The item position identification unit identifies the position of the character string recognized by the character recognition processing unit 33B. The specific item processing unit determines the attribute of the character string whose position has been identified, and acquires the recognized character string based on the attribute as a specific content item written on the form. The cell area recognition unit recognizes a cell area based on the extracted ruled lines when the ruled line extraction processing unit 32 is successful (when the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is equal to or greater than a predetermined threshold). The character string recognition unit recognizes character strings by performing character recognition processing on the form image. The line information acquisition unit acquires line information consisting of the character string recognized by the character string recognition unit. The character string attribute determination unit determines the attribute of the character string recognized by the character string recognition unit. The content item acquisition unit acquires the character string recognized by the character string recognition unit as a specific content item described on the form based on the attribute determined by the character string attribute determination unit. The item name determination unit determines whether the character string recognized by the character string recognition unit includes a predetermined item name. When the row information acquisition unit acquires two or more row information within the cell area recognized by the cell area recognition unit, the content item acquisition unit acquires the character string recognized by the character string recognition unit as a specific content item described on the form based on the attribute determined by the character string attribute determination unit.

[0044] 7 is a flowchart showing a first example of form processing according to this embodiment. The processing in FIG. 7 corresponds to the pre-registration into the definition body 33D.

[0045] In step ST1, the image reading device 2 executes a form reading process under the control of the scanner control unit 31. In step ST2, the ruled line extraction processing unit 32 executes a ruled line extraction process on the read image of the form. In step ST3, it is determined whether the ruled line extraction process by the ruled line extraction processing unit 32 was successful. That is, it is determined whether the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is equal to or greater than a predetermined threshold. If the ruled line extraction process by the ruled line extraction processing unit 32 was successful (step ST3: Yes), the process proceeds to step ST4; if the ruled line extraction process by the ruled line extraction processing unit 32 failed (step ST3: No), the process proceeds to step ST7.

[0046] In step ST4, ruled line information relating to successful ruled line extraction processing is extracted. In step ST5, the characteristics of the form are registered in a definition body using ruled line information (vertical / horizontal attributes, position, length, etc.) as a key. In step ST6, the position and attributes (character type, etc.) of character recognition items are registered as specified by the user. When processing steps ST5 and ST6, a definition body in which ruled line information (vertical / horizontal attributes, position, length, etc.) and the position and attributes (character type, etc.) of character recognition items are registered is referenced. This definition body may be definition body 33D shown in Figure 3, or it may be a definition body provided separately from definition body 33D.

[0047] In step ST7, character information from the entire form is automatically extracted. In step ST8, the extracted character information from the entire form is analyzed and sorted into fixed character string candidates (fixed text) and variable character string candidates (variable text), and the data is registered in the definition body 33D. More specifically, the size, arrangement, and attributes (character type, etc.) of the character information to be analyzed are used as feature key data, and based on this feature key data, the fixed character string candidates (fixed text) and variable character string candidates (variable text) are sorted and registered. Alternatively, based on feature key data (a conditional expression using these as variables), the fact that the character information to be analyzed is located at the top of the form, the high degree of certainty of the character information to be analyzed, and the degree of uniformity of the spacing of the character information to be analyzed are used as feature key data, and based on this feature key data, the fixed character string candidates (fixed text) and variable character string candidates (variable text) are sorted and registered. In step ST9, the position and attributes (character type, etc.) of the character recognition items are registered as specified by the user. In the processing of steps ST7 to ST9, the definition body 33D can be referenced as appropriate to use the results of full character recognition (character strings, coordinates, etc.), feature key data, positions of character recognition items, and attributes (character type, etc.).

[0048] 8 is a flowchart showing a second example of form processing according to this embodiment. The processing in FIG. 8 corresponds to the recognition processing of the scanned form.

[0049] In step ST11, the image reading device 2 executes a form reading process under the control of the scanner control unit 31. In step ST12, the ruled line extraction processing unit 32 executes a ruled line extraction process on the read form image. In step ST13, it is determined whether the ruled line extraction process by the ruled line extraction processing unit 32 was successful. That is, it is determined whether the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is equal to or greater than a predetermined threshold. If the ruled line extraction process by the ruled line extraction processing unit 32 was successful (step ST13: Yes), the process proceeds to step ST14; if the ruled line extraction process by the ruled line extraction processing unit 32 failed (step ST13: No), the process proceeds to step ST17.

[0050] In step ST14, ruled line information relating to the successful ruled line extraction process is extracted. In step ST15, the type of form is identified as the extracted ruled line information key. In step ST16, recognition of the character information written on the form is performed according to a definition body (either the same as definition body 33D or a different definition body). When processing steps ST15 and ST16, the definition body (either the same as definition body 33D or a different definition body) can be referenced as appropriate. Once processing of step ST16 is complete, proceed to step ST19.

[0051] In step ST17, character information from the entire surface of the form is automatically extracted. In step ST18, a recognition item is searched for from the automatically extracted character information using unique character information as a key. When processing step ST18, the definition body 33D can be referenced as appropriate. After processing step ST18 is completed, the process proceeds to step ST19.

[0052] In step ST19, the form discrimination / character recognition result is notified (for example, displayed on the display 3D of the form processing device 3).

[0053] In this way, the form processing device of the present disclosure has a ruled line extraction processing unit that executes a ruled line extraction process on a form, and a form recognition processing unit that executes a form recognition process in different modes when the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is equal to or greater than a predetermined threshold value and when the number is less than the predetermined threshold value (when the ruled line extraction process is successful and when it fails). This makes it possible to execute appropriate form processing regardless of the result of the ruled line extraction process.

[0054] Furthermore, in the form processing device disclosed herein, when the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is less than a predetermined threshold (when the ruled line extraction process fails), the form recognition processing unit pre-registers fixed string candidates and variable string candidates that may be included in the form, and compares the string candidates read from the form with the fixed string candidates and variable string candidates. As a result, even for forms without ruled lines or forms with unclear ruled lines, a definition body based on the fixed string candidates and variable string candidates can be created, and form sorting and form type can be automatically determined before character recognition can be performed.

[0055] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various improvements and modifications can be made without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0056] 1. Report processing system 2. Image reader 3. Document processing equipment 3D display 30 Control Unit 31 Scanner control unit 32 Ruled line extraction processing unit (ruled line reading processing unit) 33 Form recognition processing section 33A Definition field comparison part 33B Character recognition processing section 33C String candidate sorting part 33D definition body 33E Character Recognition Dictionary 34 Recognition result display area 300 CPU(Central Processing Unit) 301 Input Device 302 Output Device 303 ROM (Read Only Memory) 304 RAM (Random Access Memory) 305 Interface (I / F) 306 Bus

Claims

1. a ruled line extraction processing unit that executes a ruled line extraction process on a form; a form recognition processing unit that executes a form recognition process in different modes depending on whether the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is equal to or greater than a predetermined threshold or whether the number is less than the predetermined threshold; and When the number of vertical and horizontal ruled lines extracted by the ruled line extraction process is less than a predetermined threshold, the form recognition processing unit pre-registers fixed character string candidates and variable character string candidates that may be included in the form, and compares the character string candidates read from the form with the fixed character string candidates and the variable character string candidates. A document processing device characterized by:

2. The fixed character string candidate is a character string candidate that is always included with the same content in the same type of form, The variable character string candidates are character string candidates that are included in different contents for each form even if the form is of the same type.

2. The document processing device according to claim 1.

3. the form recognition processing unit classifies the character string candidates read from the form into either the fixed character string candidates or the variable character string candidates using at least one of a size, an arrangement, and an attribute of the character string candidates read from the form as an index; 2. The document processing device according to claim 1.

4. the form recognition processing unit classifies the character string candidates read from the form into either the fixed character string candidates or the variable character string candidates, using at least one of the following as an index: the character string candidates read from the form are arranged at the upper end side of the form; the character string candidates read from the form have a high degree of certainty; and the character string candidates read from the form are spaced at regular intervals.

2. The document processing device according to claim 1.

5. the form recognition processing unit defines A as the distance of a character string candidate read from the form from the top edge of the form, B as the certainty of the character string candidate read from the form, C as the degree of uniformity of the spacing between the character string candidates read from the form, and D as the degree of uniformity when comparing the left end / right end of the character string candidate read from the form with the left end / right end of another character string candidate, and defines N1, N2, N3, and N4 as the same or different coefficient values, and if A×N1+B×N2+C×N3-D×N4 is equal to or greater than a predetermined threshold, assigns the character string candidate read from the form to the fixed character string candidate, and if A×N1+B×N2+C×N3-D×N4 is less than the predetermined threshold, assigns the character string candidate read from the form to the variable character string candidate; 2. The document processing device according to claim 1.

6. N1, N2, N3, and N4 are coefficient values that satisfy N1>N2>N3>N4.

6. The document processing device according to claim 5.

7. a step of executing a ruled line extraction process on the form by a ruled line extraction processing unit; a step of executing form recognition processing in different modes depending on whether the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is equal to or greater than a predetermined threshold or whether the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is less than the predetermined threshold; and In the step of executing the form recognition processing by the form recognition processing unit, when the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is less than a predetermined threshold, fixed character string candidates and variable character string candidates that may be included in the form are pre-registered, and the character string candidates read from the form are compared with the fixed character string candidates and the variable character string candidates. A form processing method characterized in that a processor executes each of the above processing steps.

8. a step of executing a ruled line extraction process on the form by a ruled line extraction processing unit; a step of executing form recognition processing in different modes depending on whether the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is equal to or greater than a predetermined threshold or whether the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is less than the predetermined threshold; and In the step of executing the form recognition processing by the form recognition processing unit, when the number of vertical and horizontal ruled lines extracted by the ruled line extraction processing is less than a predetermined threshold, fixed character string candidates and variable character string candidates that may be included in the form are pre-registered, and the character string candidates read from the form are compared with the fixed character string candidates and the variable character string candidates. A form processing program characterized by causing a processor to execute each of the above processing steps.

Images