Form recognition system and form recognition method

Abstract

To address a problem in which a form recognition system cooperating with both a fixed engine and a non-fixed engine has an improved recognition accuracy but the recognition cost thereof has a tendency to increase.SOLUTION: In a form recognition system 100 cooperating with both a fixed engine 161 and a non-fixed engine 162, a process of selecting, from a template database 131, a template for character recognition from a form image read from a form at the fixed engine 161 is executed, if there is no template corresponding to the form on the basis of the template selection result, a character recognition process for the form image is executed by the non-fixed engine 162, and if there is a template corresponding to the form, a character recognition process for the form image using the selected template is executed by the fixed engine 161.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a form recognition system and a form recognition method. [Background technology]

[0002] There are two types of form recognition engines: standard form recognition engines (hereafter, standard engines) and non-standard form recognition engines (hereafter, non-standard engines). When using a standard engine, a template is prepared in advance that defines which range of the form to read, and characters are read based on the template. When reading a form, the standard engine overlays the template on the form and reads only the characters in the range defined in the template. Since the range in which the characters to be read exist is defined in advance, recognition accuracy is high. However, it is necessary to create templates for the number of layout patterns of the read items on the form (for example, for each billing source when reading invoices). Therefore, when there are many layout patterns of forms, such as reading invoices from many business partners, reading all forms with the standard engine requires huge operational (template creation) costs.

[0003] On the other hand, recognition using a non-standard engine involves analyzing the layout of the form (the arrangement and size of characters and images, etc.), searching for key information, and reading the characters linked to that key information. There is no need to create templates in advance. However, non-standard engines have lower recognition accuracy than standard engines. Also, in order to improve recognition accuracy with non-standard engine recognition, it is necessary to collect sample data from many forms and have the engine learn, which means the cost of improving recognition accuracy is high. Furthermore, recognition costs are high because layout analysis is performed to search for key information and all characters on the form are read.

[0004] Furthermore, there is a method in which reading is performed by both a template engine and a non-template engine. There are cases where a form that cannot be read correctly by a template engine (for which no template has been created) can be read correctly by a non-template engine. If the recognition results of the template engine and the non-template engine do not match, it is necessary to manually determine which recognition result to use. Patent Document 1 describes a technology in which, in order to improve reading accuracy, reading is performed by multiple recognition engines, the respective reading results are compared, and if there is a mismatch, the reading results are corrected by manual judgment, etc. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-204417 A Summary of the Invention [Problem to be solved by the invention]

[0006] In a method of reading with both a formula engine and an atypical engine, reading is always performed with both the formula engine and the atypical engine in one recognition process, so the costs of both are always incurred. In the technology described in Patent Document 1, reading is performed with both the formula engine and the atypical engine, which improves recognition accuracy, but does not take into consideration cost reduction.

[0007] On the other hand, in business where it is necessary to recognize forms with a variety of layouts, the approach of using a non-standard engine that can recognize forms with any layout has become common. However, non-standard engines have high recognition costs and lower recognition accuracy than standard engines. Therefore, there is a problem that using only non-standard engines for form recognition results in high recognition costs and correction costs for the read results.

[0008] Given the above situation, there was a demand for a method to improve recognition accuracy while reducing the costs associated with form recognition in tasks that process forms with a variety of layouts using a form recognition system that works with both fixed form engines and non-fixed form engines as a form recognition engine. [Means for solving the problem]

[0009] In order to solve the above problems, a form recognition system according to one aspect of the present invention cooperates with both a fixed form recognition engine and an unfixed form recognition engine used for recognizing characters written on a form. The form recognition system includes a form recognition processing unit and a template selection processing unit. The document recognition processing unit uses a standard document recognition engine and an unstandardized document recognition engine to perform character recognition processing on a document image, which is a scanned image of a document, and stores the character recognition results of the document image by the unstandardized document recognition engine in a memory unit. The template selection processing section executes a process of selecting, from a template database, a template to be used for character recognition in a fixed form recognition engine from a form image of the form, and outputs the template selection result to the form recognition processing section. Then, based on the template selection result, if a template for the form does not exist, the form recognition processing unit performs character recognition processing on the form image using a non-standard form recognition engine, and if a template for the form exists, performs character recognition processing on the form image using the standard form recognition engine using the selected template. Effect of the Invention

[0010] According to at least one aspect of the present invention, in a task of processing forms of various layouts using a form recognition system that works with both a standard form recognition engine and an unstandard form recognition engine, it is possible to improve recognition accuracy while reducing the cost of form recognition. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an example of a functional configuration of a document recognition system according to an embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of a hardware configuration of a document recognition system according to an embodiment of the present invention. [Diagram 3] 11 is a sequence chart showing an example of character recognition processing (no template match) by the form recognition system according to the embodiment of the present invention. [Figure 4] 11 is a sequence chart showing an example of character recognition processing (template match detected) by the form recognition system according to the embodiment of the present invention. [Diagram 5] 11A and 11B are diagrams illustrating examples of feature amounts extracted from a form image. [Figure 6] FIG. 13 is a diagram showing an example of a configuration of a document recognition result table according to an embodiment of the present invention. [Figure 7] 11 is a flowchart illustrating an example of a template candidate counting process performed by a template creation recommendation processing unit of the document recognition system according to the embodiment of the present invention. [Figure 8] FIG. 11 is a diagram showing an example of a template candidate table according to an embodiment of the present invention. [Figure 9] 11 is a sequence chart showing an example of a template creation recommendation process by a template creation recommendation processing unit of the form recognition system according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, examples of embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and the accompanying drawings, the same components or components having substantially the same functions are designated by the same reference numerals, and redundant description will be omitted.

[0013] [Functional configuration of the document recognition system] First, the functional configuration of a document recognition system according to an embodiment of the present invention will be described with reference to FIG.

[0014] Fig. 1 is a diagram showing an example of the functional configuration of a form recognition system according to an embodiment of the present invention. The form recognition system 100 shown in Fig. 1 is a form recognition system that cooperates with both a standard form recognition engine (standard engine) and an unstandard form recognition engine (unstandard engine) as form recognition engines. The form recognition system 100 communicates with a user terminal 170 via a communication network such as the Internet 180.

[0015] The user terminal 170 is a terminal used by a user who receives services provided by the document recognition system 100. The user terminal 170 may be a desktop PC (Personal Computer), a notebook PC, or the like.

[0016] The form recognition system 100 includes a web server 110, a service control unit 120, a template selection processing unit 130, a form recognition processing unit 140, and a template creation recommendation processing unit 150.

[0017] The web server 110 is a server on a web system that provides information and functions to the user terminal 170 via the Internet 180. The web server 110 is a computer with a communication function, or server software that runs on a computer. The web server 110 may be provided outside the form recognition system 100. Communication between the form recognition system 100 and the user terminal 170 is performed via the web server 110, but in the following explanation, the description of the web server 110 during communication will be omitted.

[0018] The service control unit 120 is a control unit that controls the entire document recognition system 100. As an example, the service control unit 120 controls to issue commands to each processing unit based on the contents of a request received from the user terminal 170, and controls to obtain data corresponding to the request from each processing unit and respond to the user terminal 170.

[0019] The template selection processing unit 130 is a processing unit that executes a process of selecting a template image corresponding to an input image from among a plurality of pre-registered template images (also simply referred to as "templates"). The input image is, for example, digital image data of a form read by an image scanner or the like. The template image is data that defines which range in the input image (form image) is to be read in character recognition processing using OCR (Optical Character Recognition) software or the like. In general, the template image defines the reading position and size (range) within the form. Character recognition processing is often executed when a document is read by an image scanner or the like. The template images are stored in a template database 131.

[0020] The form recognition processing unit 140 is a processing unit that requests the regular engine 161 and the non-regular engine 162 to perform character recognition processing of an input image. The form recognition processing unit 140 also receives character recognition results from the regular engine 161 and the non-regular engine 162 and accumulates the character recognition results. The character recognition results are stored in a form recognition result table 141 as form recognition results.

[0021] The standard engine 161 and the non-standard engine 162 are form recognition engines that perform character recognition processing to recognize characters written on a form image in accordance with a request from the form recognition system 100, and transmit the processing results to the form recognition system 100 as character recognition results.

[0022] The template creation recommendation processing unit 150 is a processing unit that tallyes up template candidates whose creation is recommended based on the accumulated form recognition results. In addition, the template creation recommendation processing unit 150 is a processing unit that transmits a list of templates whose creation is recommended to the user to the user terminal 170. The template candidates are stored in a template candidate table 151.

[0023] [Hardware configuration of document recognition system] Here, the hardware configuration of the document recognition system 100 will be described with reference to FIG. Fig. 2 is a diagram showing an example of a hardware configuration of the form recognition system 100. A calculator 200 shown in Fig. 2 is an example of hardware used as a computer. The form recognition system 10 according to this embodiment realizes form recognition performed by the respective functional blocks shown in Fig. 1 in cooperation with each other by the calculator 200 (computer) executing a program.

[0024] The computer 200 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203, all of which are connected to a system bus. The computer 200 further includes a non-volatile storage 204 and a communication interface 205.

[0025] The CPU 201 reads out program code of software for realizing each function according to the present embodiment from the ROM 202, loads it into the RAM 203, and executes it. Variables, parameters, etc. generated during the calculation processing of the CPU 201 are temporarily written to the RAM 203, and these variables, parameters, etc. are read out by the CPU 201 as appropriate. The CPU 201 executes the program code read out from the ROM 202 to realize each function of the service control unit 120, the template selection processing unit 130, the form recognition processing unit 140, and the template creation recommendation processing unit 150 of the form recognition system 100. However, other processors such as an MPU (Micro Processing Unit) may be used instead of the CPU 201.

[0026] The non-volatile storage 204 can store data used by a program, data obtained by executing a program, and the like. The template database 131, the form recognition result table 141, and the template candidate table 151 of the form recognition system 100 are configured using the non-volatile storage 204. An operating system (OS) and a program executed by the CPU 201 may be recorded in the non-volatile storage 204. As the non-volatile storage 204, a hard disk drive (HDD), a solid state drive (SSD), a disk medium using light or magnetism, a semiconductor memory card, or the like is used. Note that a cloud environment can also be used to store information of the template database 131, the form recognition result table 141, and the template candidate table 151.

[0027] A communication device such as a network interface card (NIC) is used as the communication interface 205. The communication interface 205 is configured to be capable of transmitting and receiving various data to and from an external device via a communication network such as a LAN or the Internet to which a terminal is connected, or a dedicated line, etc. The function of the web server 110 of the document recognition system 100 is realized by using the communication interface 205.

[0028] The hardware of the user terminal 170 can also be configured using the computer 200. A display device and an input device (not shown) are connected to the computer 200 of the user terminal 170. The display device is a monitor such as a liquid crystal display, and displays a GUI screen or the results of arithmetic processing performed by the CPU 201. The input device generates an input signal according to a user's operation and outputs it to the CPU 201. The input device may be, for example, a mouse, a keyboard, or a touch sensor, and the user can operate the input device to input information or instructions. The display device and the input device may be integrated into a touch panel. The display device and the input device may be connected to the computer 200 of the document recognition system 100.

[0029] [Character recognition processing] Next, the character recognition process performed by the form recognition system 100 will be described with reference to FIGS.

[0030] (If no template is available) First, the character recognition process when there is no corresponding template will be described with reference to FIG. 3 is a sequence chart showing an example of character recognition processing (no template match) by the form recognition system 100. Since no templates exist in the initial operation stage 320 (steps S8 to S12), the form recognition system 100 accumulates template candidates while recognizing all forms with the non-standard engine 162. After the processing of steps S1 to S3, the processing of steps S8 to S12 is executed.

[0031] First, the user terminal 170 transmits a scanned image (document image) of a document to be recognized to the document recognition system 100 (service control unit 120) and requests execution of character recognition (S1).

[0032] Next, the service control unit 120 of the form recognition system 100 transmits the form image to the template selection processing unit 130 (S2).

[0033] Next, the template selection processing unit 130 detects the feature amount of the form image, and searches for a template similar to the layout of the form image from among the templates registered in the template database 131. For example, when multiple similar templates are searched for, the template selection processing unit 130 selects the most similar template. The template selection processing unit 130 transmits the template selection result (matched / not matched) to the service control unit 120. Specifically, if a matching template is found, the template selection processing unit 130 transmits information about the template and the feature amount of the form image to the service control unit 120, and if a matching template is not found, the template selection processing unit 130 transmits information about not matching and the feature amount of the form image to the service control unit 120 (S3).

[0034] A feature amount is information that indicates the coordinates of the edges (edges, edges, corners, etc.) of characters, frames, lines, etc., and can also be interpreted as a feature point. For line segments in frames and lines, the coordinate information of the midpoint between the ends can be obtained as a feature amount.

[0035] (Example of features) Here, an example of the feature amount extracted from the form image will be described with reference to FIG. FIG. 5 is a diagram showing an example of feature amounts extracted from a form image. FIG. 5 shows an example of feature amounts extracted from a form image, in which feature amounts of a horizontally long frame and a character string within the frame are extracted. For example, if the character string "name" is placed within the frame, this character string is presumed to be an item for writing the "name" of a user. A circle is marked at the edge portion, which is a feature amount (feature point). In FIG. 5, a circle is marked at four corners of the frame, two centers of the sides of the frame (between the corners), and 26 edges of each character. When a template is used, an image including items such as those shown in FIG. 5, i.e., an item image, can be easily detected from the form image.

[0036] Returning to the description of the sequence chart in Fig. 3, after the process of step S3, when the service control unit 120 receives a template selection result (no template match in the initial operation stage 320) from the template selection processing unit 130, it transmits the form image and feature amount to the form recognition processing unit 140 (S8).

[0037] Next, the form recognition processing unit 140 transmits the form image to the non-standard engine 162 and requests character recognition (S9).

[0038] Next, the non-standard engine 162 transmits the character recognition result for the form image to the form recognition processing unit 140 (S10). The character recognition result includes the item name and the value of each item. An item is information for expressing a category of information used in a business using the form (e.g., name, transfer account, etc.). For example, an example of an item value is a handwritten or printed character string (name) of the user of the form for a name, or a handwritten or printed numeric string for a transfer account.

[0039] Next, the form recognition processing unit 140 records a list of item names included in the character recognition result and the feature amounts of the form image received from the service control unit 120 in the form recognition result table 141 (S11).

[0040] (Document recognition result table) Here, the configuration of the document recognition result table 141 will be described with reference to FIG. 6 is a diagram showing an example of the configuration of the form recognition result table 141. The form recognition result table 141 has fields for ID, item name, and feature amount.

[0041] In the ID field, information of an identifier (such as a number, a code, or a character string) for uniquely identifying a form recognized by the unformatted engine 162 within the form recognition result table 141 is stored. The item name field stores information about the names of items included in one form. When the information required of the user is the same in different forms recognized by the non-standard engine 162, or when the contents of the forms are similar, the item names may be duplicated. The feature amount field stores coordinate information of the feature amount of the form recognized by the non-standard engine 162.

[0042] For example, in Figure 6, the data with ID “1” has the item names “recipient, billing amount, issuer” and the feature amount is “(x1 1 ,y1 1 ), (x1 2 ,y1 2 ), (x1 3 ,y1 3 ),...". The data with ID "2" has the item names "recipient, billing amount, issuer, transfer account" and the feature value is "(x2 1 ,y2 1 ), (x2 2 ,y2 2 ), (x2 3 ,y2 3 ), ···”.

[0043] Returning to the description of the sequence chart in Fig. 3, after the process of step S11, the form recognition processing unit 140 transmits the character recognition result of the form image to the service control unit 120 (S12). The process of step S12 may be performed in parallel with the process of step S11, or the order of these processes may be reversed.

[0044] Next, the service control unit 120 transmits the character recognition result for the form image to the user terminal 170 (S13). The user terminal 170 receives the character recognition result for the form image by the non-standard engine 162 from the form recognition system 100, and displays it on the display device.

[0045] (If a template is available) Next, the character recognition process when a corresponding template is available will be described with reference to FIG. 4 is a sequence chart showing an example of character recognition processing (template match found) by the form recognition system 100. In the operation improvement stage 310 (steps S4 to S7), forms for which templates have been created are recognized by the formula engine 161, resulting in higher accuracy and lower costs. After the processing of steps S1 to S3, the processing of steps S4 to S7 is executed.

[0046] First, user terminal 170, service control unit 120, and template selection processing unit 130 execute the processes of steps S1 to S3.

[0047] Next, when the service control unit 120 receives the template selection result (a template match was found in the operation improvement stage 310) from the template selection processing unit 130, it sends the form image and a template based on the template selection result to the form recognition processing unit 140 (S4).

[0048] Next, the form recognition processing unit 140 extracts an item image from the form image using the received template, and transmits the item image to the formula engine 161 to request character recognition (S5). An item image is a part of the form image, and is an image of an item that has a blank or other area where information is entered by handwriting or printing. It can be considered that a plurality of different item images are extracted from the form image.

[0049] Next, the formula engine 161 transmits the character recognition result for the item image to the form recognition processing unit 140 (S6).

[0050] Next, the form recognition processing unit 140 transmits the character recognition result for the item image to the service control unit 120 as the character recognition result for the form image (S7).

[0051] Next, the service control unit 120 transmits the character recognition result for the form image to the user terminal 170 (S13). The user terminal 170 receives the character recognition result for the form image (item image) by the formula engine 161 from the form recognition system 100, and displays it on the display device.

[0052] In the initial operation stage 320, when the form recognition system 100 receives a recognition execution request from a user, it performs character recognition using the non-standard engine 162, and calculates and accumulates a list of recognized items and feature amounts of the form image. The form recognition system 100 records the form image that has undergone character recognition and the character recognition results. Then, the form recognition system 100 periodically tallys up template candidates from the accumulated form images, their list of items, and feature amounts, determines whether there is a template that can be created to reduce recognition costs from the accumulated template candidates (see FIG. 8), and transmits the determination result to the user. The user creates a template based on the received determination result.

[0053] In the operation improvement stage 310, when the form recognition system 100 receives a recognition execution request from a user, it calculates the feature amount of the transmitted form image, searches for a template with similar feature amount, and if a corresponding template is found, it uses the template to recognize the form image with the fixed form engine 161. If a corresponding template is not found, it recognizes the form image with the non-fixed form engine 162, and calculates and accumulates a list of recognized items and the feature amount of the form image.

[0054] As described above, templates are accumulated in the operation improvement stage 310, but no templates exist in the initial operation stage 320. As shown in Fig. 7, template candidates are accumulated based on the form recognition results from the initial operation stage 320 to the operation improvement stage 310. Then, as will be described later, the template creation recommendation processing unit 150 creates a list of recommended templates to be created from the tabulation results of the template candidates (forms), the recognition cost of the formula engine 161, the recognition cost of the unformulated engine 162, and the creation cost of the template, and prompts the user to create a template.

[0055] [Template candidate aggregation process] Next, the template candidate counting process performed by the template creation recommendation processing unit 150 of the document recognition system 100 will be described with reference to FIG. 7 is a flowchart showing an example of a template candidate counting process by the template creation recommendation processing unit 150 of the form recognition system 100. The template creation recommendation processing unit 150 repeatedly executes the processes of steps S21 to S30 (process A) the number of times corresponding to the number of form recognition results.

[0056] First, the template creation recommendation processing unit 150 sets the "item name" of the form recognition result [i] as items [i] and the "feature amount" of the form recognition result [i] as feature [i] (S21). The form recognition result [i] is data linked to information (i = 1,...,N) in the ID field of the form recognition result table 141 shown in Fig. 6. "N" corresponds to the number of forms recognized by the non-standard engine 162 in the character recognition processing sequence in Figs. 3 and 4.

[0057] Here, the template creation recommendation processing unit 150 repeatedly executes the processes of steps S22 to S26 (process B) the number of times corresponding to the number of template candidates.

[0058] First, the template creation recommendation processing unit 150 sets the “item name” of the template candidate [j] to items[j] and the “feature amount” of the template candidate [j] to feature[j] (S22). The template candidate [j] is data linked to information (j=1, , M) in the ID field of the template candidate table 151 shown in FIG.

[0059] (Template candidate table) Here, the configuration of template candidate table 151 will be described with reference to FIG. 8 is a diagram showing an example of the configuration of the template candidate table 151. The template candidate table 151 has fields for ID, item name, feature amount, and number of cases.

[0060] The ID field stores identifier information (such as a number, code, or character string) for uniquely distinguishing records in the template candidate table 151. Records in the template candidate table 151 are created for each group of highly similar forms (form recognition results). In highly similar forms, the information in the ID field and the information in the records linked to them (item name, feature amount) are the same.

[0061] The item name field stores the same information as the item name field shown in Figure 6 for each group of highly similar forms. In the feature field, for each group of highly similar forms, the same information as the feature field shown in Fig. 6 is stored. When the number of matching coordinate information (number of corresponding points) in step S24 includes the number of cases where the coordinate information is the same as described above and also the number of cases where the coordinate information is within a predetermined distance, the coordinate information stored in the feature field may be the most common coordinate (for example, the matching coordinate) or the average of coordinates that can be considered to be the same (the matching coordinates and the coordinates within the predetermined distance). The number of items field stores information on the number of highly similar forms.

[0062] For example, in Figure 8, the data with ID "1" has the item names "recipient, billing amount, issuer" and the feature value "(x1 1 ,y1 1 ), (x1 2 ,y1 2 ), (x1 3 ,y1 3 ),...", and the number of items is "1000". The data with ID "2" has the item names "recipient, billing amount, issuer, transfer account", and the feature amount is "(x2 1 ,y2 1 ), (x2 2 ,y2 2 ), (x2 3 ,y2 3 ),...", and the number of items is "100". The data with ID "3" has the item names "Address, Name, Payment Amount,...", and the feature value is "(x3 1 ,y3 1 ), (x3 2 ,y3 2 ), (x3 3 ,y3 3 ), ···”, and the number of items is “3”.

[0063] Returning to the explanation of the flowchart in Fig. 7, after the processing of step S22, the template creation recommendation processing unit 150 judges whether the item name "items[i]" of the form recognition result [i] and the item name "items[j]" of the template candidate [j] are the same (S23). The item name "items[i]" of the form recognition result [i] matching the item name "items[j]" of the template candidate [j] means that the items included in the "item name" of the form recognition result [i] match the items included in the "item name" of the template candidate [j].

[0064] When items[i] and items[j] match (YES judgment in S23), the template creation recommendation processing unit 150 calculates the number of correspondence points "point" between the feature amount "feature[i]" of the form recognition result [i] and the feature amount "feature[j]" of the template candidate [j] (S24). That is, the template creation recommendation processing unit 150 compares the coordinate information included in the feature amount of the form recognition result [i] with the coordinate information included in the feature amount of the template candidate [j], and calculates the number of matching coordinate information. Note that the match of coordinate information is not limited to the case where the compared coordinate information is the same, and may be configured to include the case where the compared coordinate information is within a predetermined distance.

[0065] Next, the template creation recommendation processing unit 150 j ≧θ, and point j >point max It is determined whether the condition "is satisfied" (S25). This determination process is a process of searching for the template candidate (j=1, , M) that has the most corresponding points (the most similarity) between the form recognition result [i] and the feature quantity. j , θ, and point max The definition is as follows:

[0066] point j : The number of corresponding points between the feature values ​​of the template candidate [j] and the feature values ​​of the form recognition result [i] θ: Threshold of the response score determined based on business requirements. jis equal to or greater than the threshold θ, the features of the template candidate [j] and the form recognition result [i] can be considered to be similar. point max : The value of the number of corresponding points when the number of corresponding points is the highest from the start of process B to the present (the most corresponding points). j The most compatible points are max Check if you have updated the.

[0067] Here, the correspondence score between the feature amount of the form recognition result [i] and the feature amount of the template candidate [j] is calculated, but it is also possible to calculate the similarity of the entire feature amount from the correspondence score between the feature amount of the form recognition result [i] and the feature amount of the template candidate [j] and use it for the tabulation process of the template candidates based on the similarity. Furthermore, it is also possible to calculate the similarity of the entire item names between the item names of the form recognition result [i] in step 23 and the item names of the template candidate [j] and use it for the tabulation process of the template candidates.

[0068] Next, if the determination condition is met in step S25 (YES in S25), the template creation recommendation processing unit 150 calculates the most common correspondence score point max The corresponding point at this time is j At this time, the value j of the ID field of the template candidate table 151 is updated by index max (S26).

[0069] After a NO determination in step S23, a NO determination in step S25, or the processing of step S26, the template creation recommendation processing unit 150 executes processing B (S22 to S26) on the next template candidate [j+1]. Then, when processing B has been completed for all template candidates, the template creation recommendation processing unit 150 proceeds to step S27.

[0070] After the process B is completed, the template creation recommendation processing unit 150 checks whether there is a template candidate similar to the form recognition result [i] (index maxIf there is a template candidate similar to the form recognition result [i] (YES in S27), the template creation recommendation processing unit 150 searches the template candidate table 151 for the corresponding template candidate [index max The “number of items” in the “Items” column is increased by 1 (S28). The higher the number, the more forms a single template candidate can handle, so creating a template is recommended.

[0071] Next, the template creation recommendation processing unit 150 prepares for processing the next form recognition result [i+1] by max and the most corresponding points max That is, the template creation recommendation processing unit 150 resets the index max and set it to "null" and set the maximum number of corresponding points max is set to “0” (S29).

[0072] If there is no template candidate similar to the form recognition result [i] in step S27 (NO judgment in S27), the template creation recommendation processing unit 150 newly registers the form recognition result [i] in the template candidate table 151 as a new template candidate (S30).

[0073] After the process of step S29 or step S30, the template creation recommendation processing unit 150 executes process A (S21 to S30) for the next form recognition result [i+1]. Then, when the template creation recommendation processing unit 150 finishes process A for all form recognition results, it ends this template candidate counting process.

[0074] In the example of FIG. 7, in steps S23 and S25, the template creation recommendation processing unit 150 compares the item names of the form contained in the character recognition results of the form images stored in the storage unit (form recognition result table 141) and the template candidate storage unit (template candidate table 151) (S23), and if the item names match, compares the feature amounts contained in the character recognition results of both form images to determine the similarity of the forms (character recognition results) (S24, S25). However, this is not limiting. For example, a configuration may be used in which the item names of the form contained in the character recognition results of both form images are compared to determine the similarity of the forms (character recognition results). Also, the feature amounts of the form contained in the character recognition results of both form images may be compared to determine the similarity of the forms (character recognition results).

[0075] When determining the similarity of forms by comparing the form item names included in the character recognition results, the processing load is smaller than when feature amounts are used, so the form similarity determination can be performed at low cost. Also, when determining the similarity of forms by comparing the form feature amounts included in the character recognition results, the form similarity determination can be performed with high accuracy.

[0076] [Template creation recommendation process] Next, the template creation recommendation process by the template creation recommendation processing unit 150 of the form recognition system 100 will be described with reference to Fig. 9. In this embodiment, whether or not to recommend template creation is determined based on the cost (expense) required for form recognition in the cases where a new template is created and where not.

[0077] FIG. 9 is a sequence chart showing an example of a template creation recommendation process by the template creation recommendation processing unit 150 of the form recognition system 100.

[0078] First, the user terminal 170 requests the form recognition system 100 (the service control unit 120) to execute a template creation recommendation (S41). It is desirable for the user to request the execution of the template creation recommendation in accordance with a predetermined condition such as a fixed cycle. For example, one example of the predetermined condition is every predetermined period (e.g., one month), or when the form recognition system 100 notifies that a set number of form recognition results have been accumulated. Note that the predetermined period is not limited to one month, and various periods such as several months or one week can be set.

[0079] Next, the service control unit 120 requests a list of creation recommended templates from the template creation recommendation processing unit 150 (S42).

[0080] Next, the template creation recommendation processing unit 150 executes a process of acquiring a list of template candidates from the template candidate table 151 (S43).

[0081] Next, the template creation recommendation processing unit 150 extracts template candidates that satisfy the cost reduction conditions from the template candidate list, and creates a creation recommended template list that recommends template creation to the user (S44). Equation (1) is shown below as an example of the cost reduction conditions. The creation recommended template list includes, for example, identifiers, item names, and feature amount information of one or more template candidates.

[0082] Cost fixed × N + Cost template < Cost unfixed × N (1) Cost fixed : The cost of recognizing template candidate forms using a template engine (cost per sheet) Cost unfixed : The cost of recognizing template candidate forms using a non-standard engine (cost per sheet) Cost template :Template creation cost (expenses) N: Number of template candidates (number of form recognition results accumulated in one month)

[0083] Here, a specific example will be given of the determination made by the template creation recommendation processing unit 150 when extracting template candidates that satisfy the formula (1) from the template candidate list. fixed , Cost unfixed , Cost template , and N, the following conditions are set.

[0084] Cost fixed : Recognition cost by template engine = 1 yen / sheet Cost unfixed :Recognition cost by non-standard engine = 50 yen / sheet Cost template :Template creation cost = 50,000 yen N: Number of template candidates (number of form recognition results accumulated in one month) = 10,000 sheets, 10 sheets

[0085] (When the number of items accumulated in one month is 10,000) 1 yen / sheet x 10,000 sheets + 50,000 yen < 50 yen / sheet x 10,000 sheets →60,000 yen < 500,000 yen

[0086] In this way, when the number of documents accumulated in one month is 10,000, the cost including template creation (left side) is smaller than the recognition cost by the non-standard engine (right side), so the form recognition system 100 recommends the user to create a template. After that, the form recognition system 100 performs character recognition on the corresponding form by the standard engine using the template created after the recommendation.

[0087] (When 10 items are accumulated in one month) 1 yen / sheet x 10 sheets + 50,000 yen > 50 yen / sheet x 10 sheets →50,010 yen > 500 yen

[0088] In this way, when the number of documents accumulated in one month is 10, the cost including template creation (left side) exceeds the recognition cost by the non-standard engine (right side), so the form recognition system 100 does not recommend template creation. The form recognition system 100 continues to perform character recognition by the non-standard engine on the corresponding documents.

[0089] In this embodiment, the template creation recommendation processing unit 150 specifies a form (template candidate) for which template creation is recommended based on the number of similar character recognition results accumulated for the template candidate over a predetermined period ( FIG. 8 ), the character recognition cost per form by the formula engine 161, the character recognition cost per form by the non-formula engine 162, and the creation cost of a template used for character recognition by the formula engine 161. By creating templates in this way while taking cost into consideration, it is possible to increase opportunities for character recognition with a formula engine that has high recognition accuracy and is low cost, and it is possible to reduce form recognition costs as operation progresses.

[0090] Next, the template creation recommendation processing unit 150 transmits a list of creation recommended templates to the service control unit 120 (S45).

[0091] Next, the service control unit 120 transmits a list of creation recommended templates to the user terminal 170 (S46). After the process of step S46, this template creation recommendation process ends.

[0092] The user uses a template creation tool (not shown) based on the information of the list of recommended templates received by user terminal 170 (template candidate identifiers, item names, and feature amounts) to create templates for one or more forms that have been recommended for creation.

[0093] As described above, the form recognition system according to the present embodiment cooperates with both a fixed form recognition engine and an unfixed form recognition engine used for recognizing characters written on a form. The form recognition system includes a form recognition processing unit (form recognition processing unit 140) and a template selection processing unit (template selection processing unit 130). The document recognition processing unit uses a standard document recognition engine and an unstandardized document recognition engine to perform character recognition processing on a document image, which is a scanned image of a document, and accumulates the character recognition results of the document image by the unstandardized document recognition engine in a memory unit (document recognition result table 141). The template selection processing unit executes a process of selecting a template to be used for character recognition in a fixed form recognition engine from a form image of a form from a template database (template database 131), and outputs the template selection result to the form recognition processing unit. Then, based on the template selection result, if a template for the form does not exist, the form recognition processing unit performs character recognition processing on the form image using a non-standard form recognition engine, and if a template for the form exists, performs character recognition processing on the form image using the standard form recognition engine using the selected template.

[0094] Furthermore, the form recognition system according to this embodiment includes a template candidate storage unit (template candidate table 151) and a template creation recommendation processing unit (template creation recommendation processing unit 150). The template candidate storage unit stores the character recognition results of the form image as template candidates for each similar character recognition result. The template creation recommendation processing unit compares the character recognition result of the form image read out from the storage unit (form recognition result table 141) with the character recognition result stored in the template candidate storage unit, and if a character recognition result similar to the character recognition result in the template candidate storage unit exists in the storage unit, it adds 1 to the number of character recognition results of the corresponding template candidate, and if no similar character recognition result exists in the storage unit, it registers the character recognition result in the template candidate storage unit as a new template candidate, extracts template candidates that satisfy the cost reduction conditions from the template candidate storage unit, and transmits them to the user terminal.

[0095] The template creation recommendation processing unit extracts template candidates for which the cost of form recognition will be lower when the template is created than before the template was created, based on the cost of character recognition using the fixed form recognition engine, the cost of character recognition using the unfixed form recognition engine, and the cost of creating the template, and transmits information (e.g., item names, feature amounts, etc.) of the extracted template candidates as templates recommended for creation to the user terminal.

[0096] In the embodiment described above, if a template does not exist for the form based on the template selection result of the template selection processing unit, character recognition processing is performed by an unstandardized form recognition engine, and if a template exists for the form, character recognition processing is performed by an standard form recognition engine using the selected template.

[0097] In this way, the fixed form recognition engine and the non-fixed form recognition engine are selected alternatively depending on whether or not a template for the target form is available, and when a template is available, character recognition processing is performed by the fixed form recognition engine, thereby reducing the cost of form recognition. In addition, because the template is used to perform character recognition by the fixed form recognition engine, the recognition accuracy can be improved.

[0098] In the above-described embodiment, a form that can be read with the same template is identified from the recognition result of the form, and the creation of the template is recommended to the user. More specifically, the necessity of template creation is determined based on the accumulation of recognition results by the non-standard form recognition engine and the cost of form recognition (which varies depending on the recognition cost by each engine, the template creation cost, and the number of processed cases), and the creation of the template is recommended. In other words, in the embodiment, among the forms that have been recognized by the non-standard form recognition engine, forms that should be switched to recognition by the standard form recognition engine are identified in consideration of the cost of form recognition. With this mechanism, in a business that processes forms of various layouts using a form recognition system that cooperates with both the standard form recognition engine and the non-standard form recognition engine, it is possible to improve the recognition accuracy while suppressing the form recognition cost.

[0099] In addition, in this embodiment, information on template candidates (forms) that can be created to reduce form recognition costs is presented to the user, so that the user can learn ways to reduce form recognition costs without manual effort.

[0100] The present invention is not limited to the above-described embodiment, and various other applications and modifications are possible without departing from the gist of the present invention described in the claims. For example, the above-described embodiment is a detailed and specific description of the configuration in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to a configuration having all of the components described. In addition, it is also possible to add, replace, or delete other components to a part of the configuration of the above-described embodiment.

[0101] In addition, the above-mentioned configurations, functions, processing units, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. As the hardware, a broad processor device such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) may be used.

[0102] Furthermore, each component of the document recognition system 100 according to the above-described embodiment may be implemented in any hardware as long as the respective hardware can transmit and receive information to each other via a network. Furthermore, the processing performed by a certain processing unit may be realized by a single piece of hardware, or may be realized by distributed processing using multiple pieces of hardware. [Explanation of symbols]

[0103] 100: document recognition system, 110: web server, 120: service control unit 130...template selection processing unit, 131...template database, 140...form recognition processing unit, 141...form recognition result table, 150...template creation recommendation processing unit, 151...template candidate table, 161...type engine, 162...non-type engine, 170...user terminal, 180...Internet, 200...computer, 201...CPU, 202...ROM, 203...RAM, 204...non-volatile storage, 205...communication interface, 310...operation improvement stage, 320...initial operation stage

Claims

1. A form recognition system that cooperates with both a fixed form recognition engine and a non-fixed form recognition engine used to recognize characters written on forms, a form recognition processing unit that executes character recognition processing on a form image, which is a scanned image of a form, using the standard form recognition engine and the non-standard form recognition engine, and stores the character recognition result of the form image by the non-standard form recognition engine in a storage unit; a template selection processing unit that executes a process of selecting a template to be used for character recognition by the standard form recognition engine from a template database based on a form image of the form, and outputs a template selection result to the form recognition processing unit; The form recognition processing unit executes character recognition processing on the form image by the non-standard form recognition engine when a template for the form does not exist based on the template selection result, and executes character recognition processing on the form image by the standard form recognition engine using the selected template when a template for the form exists. Document recognition system.

2. a template candidate storage unit that stores the character recognition results of the form image as template candidates for each similar character recognition result; a template creation recommendation processing unit that compares a character recognition result of the form image read from the storage unit with the character recognition result stored in the template candidate storage unit, and when a character recognition result similar to the character recognition result of the template candidate storage unit exists in the storage unit, adds 1 to the number of character recognition results of the corresponding template candidate, and when no similar character recognition result exists in the storage unit, registers the character recognition result in the template candidate storage unit as a new template candidate, and extracts template candidates that satisfy a condition for cost reduction from the template candidate storage unit and transmits the extracted template candidates to a user terminal.

2. A document recognition system according to claim 1.

3. The template creation recommendation processing unit extracts template candidates that will reduce the cost of form recognition when the template is created compared to before the template was created, from the cost of character recognition by the fixed form recognition engine, the cost of character recognition by the unfixed form recognition engine, and the cost of creating the template, and transmits information on the extracted template candidates to the user terminal as templates recommended for creation.

3. A document recognition system according to claim 2.

4. The template creation recommendation processing unit identifies the template candidate for which creation of the template is recommended based on the number of similar character recognition results accumulated for the template candidate during a predetermined period, the cost of character recognition per form by the fixed form recognition engine, the cost of character recognition per form by the non-fixed form recognition engine, and the cost of creating the template.

4. A document recognition system according to claim 3.

5. The template creation recommendation processing unit compares item names included in the character recognition results of the form images stored in the storage unit and the template candidate storage unit, and if the item names match, compares feature amounts included in the character recognition results of both the form images to determine the similarity of the character recognition results.

3. A document recognition system according to claim 2.

6. The template creation recommendation processing unit compares item names included in the character recognition results of the form image stored in the storage unit and the template candidate storage unit, and determines similarity of the character recognition results.

3. A document recognition system according to claim 2.

7. The template creation recommendation processing unit compares feature amounts included in the character recognition results of the form image stored in the storage unit and the template candidate storage unit to determine similarity of the character recognition results.

3. A document recognition system according to claim 2.

8. A method for recognizing a form using a form recognition system that cooperates with both a fixed form recognition engine and a non-fixed form recognition engine used for recognizing characters written on a form, comprising: A process of selecting a template to be used for character recognition by the standard form recognition engine from a template database based on a form image, which is a scanned image of the form, and outputting a template selection result; a process of executing character recognition processing on the form image by the non-standard form recognition engine when a template for the form does not exist based on the template selection result, and executing character recognition processing on the form image by the standard form recognition engine using the selected template when a template for the form exists; and storing a result of character recognition of the form image by the non-standard form recognition engine in a storage unit. Document recognition method.

Images