Information processing device, information processing method, and information processing program

Abstract

To reduce a burden on a user.SOLUTION: An information processing apparatus has: an information acquisition unit that extracts information indicating an issuer of a document from document image data, refers to a storage unit storing fixed information associated with the issuer of the document, and acquires the fixed information corresponding to the information indicating the issuer of the document; and a display control unit that outputs, to a terminal device, screen data including the information indicating the issuer of the document and the fixed information acquired by the information acquisition unit.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program for recognizing forms. Mu Regarding. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there has been known a form recognition technology that reads character strings from a form image and extracts, from the read character strings, character strings that correspond to each other as item names and item values. Summary of the Invention [Problem to be solved by the invention]

[0003] Form layouts vary widely, and some forms include pre-fixed item names and item values. Conventional technologies require users to confirm the results of form recognition each time form recognition is performed, even for these pre-fixed item names and item values. Furthermore, conventional technologies require users to correct the results each time the pre-fixed item names and item values ​​are not correctly extracted.

[0004] As described above, in the conventional technology, the user must check and correct the pre-fixed item names and item values, which makes the work complicated and places a burden on the user.

[0005] The present invention has been made in view of the above circumstances, and aims to reduce the burden on users. [Means for solving the problem]

[0006] The disclosed technology includes a storage unit that stores information indicating the issuer of a form and information associated with the issuer of the form in advance in association with each other; Accountan extraction unit that extracts information indicating the issuer of the form and information associated with the issuer of the form from the form image data; and an update unit that, if the information indicating the issuer of the form extracted by the extraction unit is not stored in the storage unit, associates the information indicating the issuer of the form extracted by the extraction unit with the information associated with the issuer of the form and registers them in the storage unit; an information acquisition unit that acquires, when the information indicating the issuer of the form extracted by the extraction unit is stored in the storage unit, information associated with the extracted information indicating the issuer of the form from the storage unit; and a display control unit that outputs screen data to a terminal device, the screen data including the information indicating the issuer of the form and information associated with the information indicating the issuer of the form acquired by the information acquisition unit; With When an edit is made to information associated with the information indicating the issuer of the form displayed on the terminal device, the update unit updates the information associated with the information indicating the issuer of the form stored in the storage unit in accordance with the edit. , an information processing device. [Effects of the Invention]

[0007] This reduces the burden on the user. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of a document recognition system. [Figure 2] FIG. 2 illustrates an example of a hardware configuration of a server device. [Figure 3] FIG. 2 is a first diagram showing an example of an extraction definition database; [Figure 4] FIG. 10 is a second diagram showing an example of the extraction definition database. [Figure 5] FIG. 2 is a diagram illustrating an example of a dictionary database. [Figure 6] FIG. 10 is a diagram illustrating an example of a business partner database. [Figure 7] FIG. 2 is a diagram illustrating functions of a server device. [Figure 8] FIG. 2 is a sequence diagram illustrating the operation of the document recognition system. [Figure 9] FIG. 10 is a diagram illustrating an example of a job list. [Figure 10] 10 is a first flowchart illustrating processing by a document recognition unit. [Figure 11] 10 is a second flowchart illustrating the processing of the document recognition unit. [Figure 12] FIG. 1 is a diagram illustrating character recognition. [Figure 13A] FIG. 10 is a diagram showing an example of an extraction result of financial institution information. [Figure 13B] FIG. 10 is a diagram illustrating an example of recognition result data. [Figure 14] 10 is a third flowchart illustrating the processing of the document recognition unit. [Figure 15] 10 is a fourth flowchart illustrating the processing of the document recognition unit. [Figure 16] FIG. 10 is a first diagram showing a display example. [Figure 17] FIG. 10 is a second diagram showing a display example. [Figure 18] FIG. 10 is a third diagram showing a display example. [Figure 19] FIG. 4 is a fourth diagram showing a display example. [Figure 20] FIG. 5 is a fifth diagram showing a display example. [Figure 21] FIG. 6 is a sixth diagram showing a display example. [Figure 22] FIG. 10 is a diagram illustrating an example of output data. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment will be described below with reference to the accompanying drawings. Fig. 1 is a diagram showing an example of the system configuration of a form recognition system.

[0010] The form recognition system 100 of this embodiment includes a server device 200, an image forming device 300, and a terminal device 400. In the form recognition system 100 of this embodiment, the server device 200, the image forming device 300, and the terminal device 400 are connected via a network.

[0011] Furthermore, the terminal device 400 of this embodiment is connected to a core system 500, for example, via a network. The core system 500 is a system that performs specific processing using the results of recognition of a form by the form recognition system 100 of this embodiment, for example.

[0012] In this embodiment, an invoice will be described as an example of a form.

[0013] The document recognition system 100 of this embodiment extracts information (invoice information) indicating the billing source that issued the invoice and the billing amount from the document image obtained by scanning the invoice. The document recognition system 100 then references customer information that associates the billing source with the financial institution to which the invoice amount will be deposited, and identifies the financial institution that corresponds to the extracted billing source. The customer information is information that is stored in advance in the server device 200, and is information in which item names and item values ​​are fixed. In other words, the customer information is fixed information that is associated with the billing source (the issuer of the document). Details of the customer information will be described later.

[0014] In addition, in this embodiment, document recognition may be used to extract information (financial institution information) from the document image regarding the financial institution to which the billing amount will be deposited, and the extracted information may be compared with information indicating the financial institution identified from the customer information.

[0015] Furthermore, the core system 500 may be an accounting system that transfers billed amounts to financial institutions.

[0016] In the document recognition system 100 of this embodiment, the server device 200 extracts the items and item values ​​contained in the document from image data showing the image of the document read by the scanner function of the image forming device 300, and converts the items and item values ​​into text data that corresponds to each other.

[0017] In the following description, an image of a form read by the scanner function of the image forming apparatus 300 is referred to as a form image, and image data representing the form image is referred to as form image data.

[0018] In the following explanation, extracting the names of items (item names) and the values ​​of these items (item values) contained in a form image, converting the items and item values ​​into text data, and associating them is referred to as form recognition.

[0019] The server device 200 of this embodiment has a storage unit 220 and a form recognition processing unit 230. The storage unit 220 is provided with an image database 240, an extraction definition database 250, a dictionary database 260, a customer database 270, and a recognition result database 280.

[0020] Form image data is stored in the image database 240. Extraction definition information is stored in the extraction definition database 250. The extraction definition information is referenced by the form recognition processing unit 230 when extracting character strings contained in financial institution information from a form image.

[0021] Dictionary information for identifying financial institutions is stored in the dictionary database 260. The dictionary information is information acquired in advance and is referenced by the document recognition processing unit 230 when identifying the name of a financial institution, etc.

[0022] The business partner database 270 stores business partner information related to business partners of organizations (tenants) that use the services provided by the document recognition system 100. In other words, a tenant is a billing party that receives an invoice. A business partner of a tenant is an organization that receives payment for the amount stated on the invoice from the tenant that received the invoice. In other words, a business partner of a tenant is the billing party that issued the invoice (the issuer of the document). Business partner information is information that is stored in advance in the business partner database 270 for each tenant, and is information in which the item names and item values ​​are fixed in advance.

[0023] The recognition result database 280 stores information indicating the results of form recognition by the form recognition processing unit 230.

[0024] The form recognition processing unit 230 of this embodiment acquires form image data stored in the image database 240, performs character recognition, and extracts a group of character strings included in the form image. The form recognition processing unit 230 then extracts invoice information from the group of character strings. The form recognition processing unit 230 also references extraction definition information 253 stored in the extraction definition database 250 to extract character strings to be included in financial institution information from the group of character strings.

[0025] The extraction definition information 253 of this embodiment includes definition information 251 that defines the character string to be included in the financial institution information, and condition information 252 that indicates the conditions for extracting the character string to be included in the financial institution information.

[0026] The document recognition processing unit 230 refers to the definition information 251 to identify a character string corresponding to the name of a financial institution from the group of character strings extracted by character recognition, and extracts a character string (first character string) indicating the name of a financial institution based on its positional relationship with the identified character string.

[0027] That is, the first character string is extracted based on a character string identified from the group of character strings extracted by character recognition. In other words, the character string identified from the group of character strings extracted by character recognition becomes a character string that serves as a base point for extracting the first character string.

[0028] In addition, the document recognition processing unit 230 refers to the condition information 252 and extracts other character strings (second character strings) to be included in the financial institution information based on their positional relationship with the character string (first character string) indicating the financial institution name.

[0029] That is, the second character string is extracted using the first character string as a base point. In other words, the first character string is a character string identified based on the previous extraction base point, and serves as a base point for extracting the second character string.

[0030] Then, the document recognition processing unit 230 associates the first character string with the second character string from the document image data to generate financial institution information.

[0031] Furthermore, the form recognition processing unit 230 determines the name of the financial institution, etc., based on the character string extracted based on the extraction definition information 253 and the dictionary information stored in the dictionary database 260. Then, the form recognition processing unit 230 stores the recognition result in which the invoice information and the financial institution information are associated with each other in the recognition result database 280.

[0032] Furthermore, the document recognition processing unit 230 of this embodiment refers to the business partner database 270 to identify business partner information corresponding to the billing source included in the invoice information extracted by document recognition. The document recognition processing unit 230 may then cause the terminal device 400 to display information indicating the financial institution included in the business partner information as remittance destination information indicating the destination of the invoice amount together with the invoice information.

[0033] In this embodiment, by obtaining the payee information from the customer database 270 in this manner, even if there is an error in the financial institution information extracted from the document image data, the payee information, which is a pre-fixed item name and item value, can be obtained.

[0034] Therefore, according to this embodiment, the user (tenant) does not need to check financial institution information every time document recognition is performed, thereby reducing the burden on the user.

[0035] In this embodiment, the payee information may be stored in the recognition result database 280 as part of the recognition result data, in association with the bill information and financial institution information.

[0036] Furthermore, the form recognition processing unit 230 may convert the recognition result data into a format compatible with the core system 500 and output it to the core system 500 .

[0037] In this embodiment, character recognition in form recognition can thus extract multiple character strings associated with a character string corresponding to the name of a financial institution. In other words, this embodiment can extract multiple character strings associated with a character string extracted from a form image, thereby improving the accuracy of form recognition.

[0038] In the form recognition system 100 of this embodiment, the image forming device 300 is a multifunction device with a scanner function. The image forming device 300 is equipped with applications for realizing functions such as a copy function, a fax function, and a scanner function, and these functions are realized by selecting an application corresponding to each function.

[0039] The terminal device 400 of this embodiment is used by a user (tenant) who uses the form recognition system 100. Furthermore, the terminal device 400 may display the result of form recognition by the server device 200.

[0040] In this embodiment, the tenant is, for example, a company, etc. More specifically, the tenant is, for example, an establishment, company, organization, etc. that has concluded a contract to use the services provided by the document recognition system 100.

[0041] 1, the server device 200 has five databases in the storage unit 220, but this is not limiting. Some of the databases may be provided in an external device to the server device 200, or all of the databases may be provided in an external device.

[0042] 1, the server device 200 implements the form recognition processing unit 230, but the present invention is not limited to this. The form recognition processing unit 230 may be implemented by a plurality of information processing devices.

[0043] In addition, in the example of Figure 1, the document recognition system 100 includes one image forming device 300 and one terminal device 400, but the number of image forming devices 300 and one terminal device 400 included in the document recognition system 100 may be any number.

[0044] Next, the hardware configuration of the server device 200 of this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the hardware configuration of the server device.

[0045] As shown in FIG. 2, the server device 200 is constructed by a computer, and as shown in FIG. 2, it is equipped with a CPU 201, a ROM 202, a RAM 203, an HD 204, an HDD (Hard Disk Drive) controller 205, a display 206, an external device connection I / F (Interface) 208, a network I / F 209, a bus line 210, a keyboard 211, a pointing device 212, a DVD-RW (Digital Versatile Disk Rewritable) drive 214, and a media I / F 216.

[0046] Of these, the CPU 201 controls the overall operation of the server device 200. The ROM 202 stores programs used to drive the CPU 201, such as an IPL. The RAM 203 is used as a work area for the CPU 201. The HD 204 stores various data, such as programs. The HDD controller 205 controls the reading and writing of various data from and to the HD 204 under the control of the CPU 201. The display 206 displays various information, such as a cursor, menus, windows, characters, or images. The external device connection I / F 208 is an interface for connecting various external devices. In this case, the external devices are, for example, USB (Universal Serial Bus) memories or printers. The network I / F 209 is an interface for data communication using a communication network. The bus line 210 is an address bus, a data bus, or the like, for electrically connecting the components, such as the CPU 201, shown in FIG. 2.

[0047] The keyboard 211 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 212 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 214 controls reading and writing of various data from and to a DVD-RW 213, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I / F 216 controls reading and writing (storing) of data from and to a recording medium 215, such as a flash memory.

[0048] The hardware configuration of the terminal device 400 of this embodiment is the same as that of the server device 200, and therefore a description thereof will be omitted. Note that the terminal device 400 may be, for example, a tablet-type terminal device or a smartphone, and may have a display operation device realized by a touch panel or the like instead of the keyboard 211, the pointing device 212, and the display 206.

[0049] Next, the extraction definition database 250 of the server device 200 of this embodiment will be described with reference to Figures 3 and 4. Figure 3 is a first diagram showing an example of the extraction definition database. Figure 3 shows an example of definition information 251 included in extraction definition information 253 stored in the extraction definition database 250.

[0050] The definition information 251 has the following information items: item name, extraction base point, extraction direction, extraction range, and dictionary reference. In the extraction definition database 250, the item "item name" is associated with other items, and information including the value of the item "item name" and the values ​​of the other items constitutes the definition information 251.

[0051] The value of the item "item name" indicates the item name associated with the item value. The value of the item "item name" is extracted based on the positional relationship between the item name and the character string indicated by the value of the corresponding item "extraction base point." In other words, the value of the item "item name" is the item name associated with the first character string or the second character string.

[0052] The value of the "item name" item is also an item of information included in the financial institution information. In the example of Figure 3, the values ​​of the "item name" item include the bank name, branch name, account type, account number, and account holder name. These values ​​are items of financial institution information.

[0053] The value of the item "extraction base point" indicates a character string that serves as a base point when extracting a character string that will be the item value of the item name indicated by the value of the item "item name".

[0054] The value of the item “Extraction direction” indicates the positional relationship between the base character string and the character string that is the item value of the item name indicated by the value of the item “Item name.” In other words, the value of the item “Extraction direction” indicates the direction in which the character string (second character string) that is the item value of the item name indicated by the value of the item “Item name” exists relative to the base character string (first character string), which is a character string extracted in the past.

[0055] The value of the item "Extraction range" indicates the number of character strings to extract. The value of the item "Dictionary reference" indicates whether to perform processing to determine character strings by referring to dictionary information stored in the dictionary database 260.

[0056] In Figure 3, for example, the character string that becomes the item value of the item name "Bank Name" is a character string that is located to the right or below the character strings such as "Financial institution" and "Payment destination" among the group of character strings extracted from the form image, and there is one corresponding character string, which is determined based on the dictionary database 260.

[0057] Furthermore, for example, the character string that becomes the item value of the item name "branch name" is the character string located to the right of the character string that becomes the extraction base point, or the character string located below the character string that becomes the extraction base point, with the character string extracted as the item value of the item name "bank name" being the extraction base point, and there is one corresponding character string, which is a character string determined based on the dictionary database 260.

[0058] Therefore, in this case, the character string extracted as the item value corresponding to the item name "Bank Name" becomes the first character string, and the character string extracted as the item value corresponding to the item name "Branch Name" becomes the second character string.

[0059] For example, the character string that becomes the item value of the item name "account type" is the character string located to the right of the character string that becomes the extraction base point, or the character string located below the character string that becomes the extraction base point, or the character string located to the left of the character string that becomes the extraction base point, with the character string extracted as the item value of the item name "branch name" being used as the extraction base point. Furthermore, there are two character strings that become the item value of the item name "account type", and these character strings are determined based on the dictionary database 260.

[0060] In this case, the character string extracted as the item value corresponding to the item name "branch name" becomes the first character string, and the character string extracted as the item value corresponding to the item name "account type" becomes the second character string.

[0061] 4 is a second diagram showing an example of the extraction definition database 253. FIG.

[0062] The condition information 252 has the following information items: branch name extraction condition, account type extraction condition, and account number extraction condition. In the extraction definition database 250, the condition information 252 is information including the value of the item "item name."

[0063] The value of the item "Branch name extraction conditions" indicates the conditions for the character string to be extracted as the field value of the item "Branch name". The value of the item "Account type extraction conditions" indicates the conditions for the character string to be extracted as the field value of the item "Account type". The value of the item "Account number extraction conditions" indicates the conditions for the character string to be extracted as the field value of the item "Account number". In Figure 4, the condition for the character string extracted as the item value for the item name "branch name" is that the character string ends with "branch," "head office," "branch," "sales office," "sales department," etc. Also, the condition for the character string extracted as the item value for the item name "account type" is that the character string contains "ordinary," "transaction," "current," "account," "savings," or "account." Also, the condition for the character string extracted as the item value for the item name "account number" is that it contains a number.

[0064] Next, the dictionary database 260 will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the dictionary database.

[0065] The dictionary information stored in the dictionary database 260 of this embodiment has bank names and branch names as information items, and each item is associated with another. In this embodiment, the dictionary information is information that associates the value of the item "bank name" with the value of the item "branch name."

[0066] In the example of FIG. 5, it can be seen that the bank name "XXX Bank" has "AAA Branch," "CCC Branch," and "EEE Branch."

[0067] Next, the supplier database 270 will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the supplier database.

[0068] In this embodiment, the supplier database 270 may be provided for each tenant. The supplier database 270 includes information items such as tenant name, supplier name, basic information, person in charge of transaction, and payment information.

[0069] The item "tenant name" is associated with other items in the business partner database 270. In this embodiment, information including the value of the item "tenant name" and the values ​​of the other items in the business partner database 270 is business partner information.

[0070] The value of the item "tenant name" indicates the name of the tenant and is information for identifying the tenant. Note that information other than the tenant name may be used as information for identifying the tenant. Specifically, for example, a tenant ID or the like may be used as information for identifying the tenant.

[0071] The value of the item "Customer Name" indicates the name of the tenant's customer, and is information for identifying the customer. Note that a customer is the party that issues an invoice. Therefore, the value of the item "Customer Name" in this embodiment is information for identifying the issuer of the document (invoice). Note that the value of the item "Customer Name" may be any information that identifies the customer, and may be a customer ID or the like that identifies the customer (invoicer).

[0072] The value of the item "Basic Information" is information including the telephone number, address, etc. of the business partner identified by the business partner name. The value of the item "Transaction Manager" is information including the name, contact information, etc. of the transaction manager.

[0073] The item "payment information" is associated with items such as "payment method," "financial institution name," "branch name," "account type," "account number," "account holder name," "fees to be paid," and "payment terms."

[0074] The value of the "Payment Method" item indicates the payment method for the invoice amount to the business partner. The value of the "Financial Institution Name" item indicates the financial institution to which the invoice amount will be deposited. The value of the "Branch Name" item indicates the branch name of the financial institution to which the invoice amount will be deposited, and the values ​​of the "Account Type," "Account Number," and "Account Name" items indicate the account type, account number, and account name, respectively.

[0075] The value of the item "fee payer" indicates the party that pays the fee incurred at the time of payment. The value of the item "payment terms" indicates the conditions for making payment.

[0076] In this embodiment, in the customer database 270, information including the values ​​of the items "Financial institution name," "Branch name," "Account type," "Account number," and "Account holder name" associated with the item "Payment information" becomes the payee information indicating the payee of the billed amount.

[0077] In this way, the customer information of this embodiment is fixed information in which the item names and item values ​​are pre-fixed depending on the issuer of the form. More specifically, the payee information included in the customer information stored in the customer database 270 of this embodiment is fixed information in which the item values ​​and item values ​​are pre-determined depending on the issuer of the form. Fixed information is information included in a form that is pre-determined depending on the issuer of the form. In other words, fixed information is information that does not change if the form is issued by the same issuer.

[0078] Note that Figure 6 shows an example of the customer database 270, and the customer database 270 does not need to include all of the items shown in Figure 6. For example, the customer database 270 only needs to include values ​​for the items "tenant name" and "customer name," and values ​​for the items "financial institution name," "branch name," "account type," "account number," and "account holder name" that indicate deposit destination information.

[0079] Furthermore, the business partner information for each tenant may include, for example, account information of a financial institution held by each tenant. In other words, the business partner information may include payment source information that is the source of payment for the invoice amount.

[0080] Next, the functions of the server device 200 will be described with reference to Fig. 7. Fig. 7 is a diagram for explaining the functions of the server device.

[0081] In the server device 200 of this embodiment, the CPU 201 reads and executes an information processing program stored in the HD 504 or the like, thereby realizing the functions of each unit described below.

[0082] The server device 200 of this embodiment includes a form recognition processing unit 230. The form recognition processing unit 230 includes an input receiving unit 221, an authentication processing unit 222, and a form recognition unit 290.

[0083] The input accepting unit 221 accepts various inputs to the server device 200. Specifically, the input accepting unit 221 accepts input of authentication information for logging in to the form recognition system 100. The authentication information is, for example, a user ID and a password. In addition, the input accepting unit 221 accepts input of form image data transmitted from the image forming device 300, for example.

[0084] The authentication processing unit 222 performs authentication based on the authentication information input and accepted by the input accepting unit 221. The authentication processing unit 222 may, for example, transmit the input authentication information to an authentication server provided outside the server device 200 and obtain the result of authentication by the authentication server. When the user is authenticated, the authentication processing unit 222 may pass the form image data accepted by the input accepting unit 221 to the form recognition unit 290.

[0085] The document recognition unit 290 of this embodiment has an image data storage unit 291, a job list generation unit 292, an image data acquisition unit 293, a character recognition unit 294, a first extraction unit 295, a second extraction unit 296, a recognition result generation unit 297, an update unit 299, a display control unit 284, and an output unit 285.

[0086] When the image data storage unit 291 receives the form image data from the image forming device 300 , it stores the data in the image database 240 .

[0087] The job list generation unit 292 receives input of form image data from the image forming apparatus 300 via the input receiving unit 221, registers the job in the job list, and manages the job list. Details of the job list will be described later.

[0088] The image data acquisition unit 293 acquires the form image data stored in the image database 240 .

[0089] The character recognition unit 294 extracts character strings and information indicating the positions of the character strings from the form image data, and stores the extracted character strings as reading result information. Therefore, the reading result information includes the character strings extracted from the form image data. The reading result information will be described in detail later.

[0090] The first extraction unit 295 extracts information indicating the billing source that issued the invoice and information indicating the billing amount as character strings from the read result information, and sets the extracted character strings as invoice information. Note that the first extraction unit 295 may, for example, hold information indicating the definitions of character strings included in the invoice information in advance, and extract character strings included in the invoice information based on this information.

[0091] The second extraction unit 296 extracts financial institution information by referring to the definition information 251 and the condition information 252 in the extraction definition database 250. The processing of the second extraction unit 296 will be described in detail later.

[0092] Therefore, the first extraction unit 295 is an example of an invoice information extraction unit that extracts invoice information from the form image data, and the second extraction unit 296 is an example of a financial institution information extraction unit that extracts financial institution information from the form image data.

[0093] The recognition result generating unit 297 generates recognition result data that associates the invoice information extracted by the first extracting unit 295 with the financial institution information extracted by the second extracting unit 296 .

[0094] The information acquisition unit 298 acquires payment information corresponding to the biller included in the invoice information from the business partner database 270. Specifically, the information acquisition unit 298 references the business partner database 270 and identifies business partner information whose business partner name matches the biller included in the invoice information extracted by the first extraction unit 295. Then, the information acquisition unit 298 acquires payment information included in the identified business partner information. Furthermore, the information acquisition unit 298 of this embodiment acquires payee information included in the acquired payment information.

[0095] The update unit 299 updates the supplier database 270 when, for example, edits are made to the payee information displayed on the confirmation screen for the recognition result displayed on the terminal device 400 .

[0096] The display control unit 284 controls the display of a screen on the terminal device 400 etc. Specifically, the display control unit 284 may generate screen data showing a recognition result confirmation screen to be displayed on the terminal device 400.

[0097] The output unit 285 outputs screen data showing the recognition result confirmation screen generated by the display control unit 284 to the terminal device 400. In other words, the output unit 285 displays the recognition result confirmation screen generated by the display control unit 284 on a web browser (display unit) of the terminal device 400. In addition, the output unit 285 converts the recognition result data into data in a format compatible with the core system 500 and outputs the data.

[0098] The operation of the form recognition system 100 of this embodiment will be described below with reference to Fig. 8. Fig. 8 is a sequence diagram illustrating the operation of the form recognition system.

[0099] In the form recognition system 100, when the image forming apparatus 300 receives a request from a user to start an application for performing form recognition (step S801), the image forming apparatus 300 starts the application (step S802). The user may be, for example, a tenant administrator.

[0100] Next, when the image forming apparatus 300 receives authentication information input from the user (step S803), it transmits this authentication information to the server apparatus 200 and makes an authentication request (step S804).

[0101] Upon receiving the authentication request, the server device 200 performs authentication using the authentication processing unit 222 and notifies the image forming device 300 of the result (step S805). Note that the operation shown here is for when the user is authenticated.

[0102] Next, the image forming apparatus 300 receives an instruction to scan the form (step S806), scans the form, and acquires form image data (step S807). Next, the image forming apparatus 300 transmits the form image data to the server apparatus 200 (step S808).

[0103] In the server device 200, when the input receiving unit 221 receives input of form image data, the authentication processing unit 222 passes this form image data to the form recognition unit 290 (step S809). Note that, in the example of Fig. 8, the form image data is passed to the form recognition unit 290 via the authentication processing unit 222, but this is not limiting. The form image data may be passed to the form recognition unit 290 without passing through the authentication processing unit 222.

[0104] When the form recognition unit 290 of the server device 200 receives the form image data, the job list generation unit 292 registers the job in the job list (step S810). Next, the form recognition unit 290 stores the form image data in the image database 240 using the image data storage unit 291 (step S811).

[0105] Next, the form recognition unit 290 uses the image data acquisition unit 293 to acquire form image data to be recognized from the image database 240 (step S812), and executes processing to recognize the form image (step S813).

[0106] Specifically, in the document recognition unit 290 of this embodiment, the first extraction unit 295 extracts invoice information from a group of character strings included in the reading result information, the second extraction unit 296 extracts financial institution information from the group of character strings, and the recognition result generation unit 297 generates recognition result data that associates the invoice information with the financial institution information.

[0107] Furthermore, the form recognition unit 290 generates complemented reading result information by using the complementing unit 298 to complement the character strings included in the reading result information based on the dictionary database 260. The process of step S813 will be described in detail later.

[0108] Next, the form recognition unit 290 stores the recognition result data generated by the recognition result generation unit 297 in the recognition result database 280 (step S814).

[0109] In the document recognition system 100, when the terminal device 400 receives a browser display instruction input from the user (step S815), it makes a browser display request to the server device 200 to display the recognition result data (step S816).

[0110] Furthermore, the terminal device 400 accepts input of authentication information from the user (step S817) and transmits this authentication information to the server device 200 (step S818).

[0111] The server device 200 authenticates the user through the authentication processing unit 222 (step S819), and notifies the terminal device 400 of the result (step S820). Note that the process shown here is for when the user is authenticated.

[0112] Next, when the terminal device 400 receives an input of an instruction to view the recognition result of the form image from the user (step S821), it sends a view request to the server device 200 (step S822).

[0113] Upon receiving the request for viewing, the server device 200 acquires the recognition result data stored in the recognition result database 280 (step S823). Subsequently, the server device 200 acquires payment information included in the recognition result data (step S824). The process of step S824 will be described in detail later.

[0114] Next, the server device 200 generates a recognition result confirmation screen and displays it on the browser of the terminal device 400 (step S825), and allows the user to confirm the recognition result on the displayed recognition result confirmation screen, thereby terminating the processing.

[0115] Here, the job list generated by the job list generating unit 292 of this embodiment will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the job list.

[0116] When the server device 200 of this embodiment receives form image data from the image forming device 300, the job list generating unit 292 stores the form image data and the job ID as a job list in association with each other.

[0117] The job list 91 of this embodiment has the following information items: job ID, user ID, form image file path, recognition result file path, and status.

[0118] The value of the item "Job ID" is an identifier for identifying a job. In other words, the value of the item "Job ID" is an identifier for identifying form image data received from the image forming apparatus 300.

[0119] The value of the item “user ID” is an identifier for identifying the user who has logged in to the document recognition system 100 .

[0120] The value of the item "Image file path" is information indicating the location where the form image data is stored. The value of the item "Recognition result file path" is information indicating the location where the recognition result data resulting from recognizing the form image is stored.

[0121] The value of the item "status" indicates the progress of recognition of the form image, in other words, the value of the item "status" indicates the state of the job.

[0122] In this embodiment, there are four values ​​for the item "status": unprocessed state, draft saved state, confirmed state, and externally output state. Each state is as follows.

[0123] Unprocessed state (state 1): This is the state immediately after the recognition result data has been obtained by form recognition, and the recognition results have not yet been confirmed.

[0124] Draft saved state (state 2) This is the state when the recognition result data is still being checked, and the recognition result data has not yet been finalized.

[0125] Confirmed state (state 3): The state in which the work of checking the recognition result data has been completed and the data has been stored in the recognition result database 280 as confirmed information. External output completed state (state 4): A state in which the output of the recognition result data to an accounting system or the like linked with the document recognition system 100 has been completed.

[0126] When the server device 200 of this embodiment receives form image data from the image forming device 300, the job list generation unit 292 assigns a job ID to the form image data, associates it with the user ID acquired as authentication information, and adds a record to the job list 91. Then, when the form image data is stored in the image database 240, the job list generation unit 292 adds information indicating the storage location as the value of the item "image file path."

[0127] Furthermore, when form definition information to be referenced is identified in the recognition of a form image, the job list generation unit 292 adds the definition ID as the value of the item "definition ID." Then, when the job list 91 is completed and the recognition result data is stored in the recognition result database 280, the job list generation unit 292 adds information indicating the storage location as the value of the item "recognition result file path," and sets the value of the item "status" to "completed."

[0128] In this embodiment, for example, a job list generated by the job list generation unit 292 may be displayed. In this case, the user can check the progress of recognition of the form image via the job list screen displayed based on the job list.

[0129] Next, the processing of the document recognition unit 290 of this embodiment will be described with reference to Fig. 10 and Fig. 11. Fig. 10 shows details of the processing of step S813 in Fig. 8.

[0130] Fig. 10 is a first flowchart illustrating the processing of the form recognition unit. More specifically, Fig. 10 shows the processing of extracting a character string that will be the item value of the item name "Bank Name" in the financial institution information.

[0131] The form recognition unit 290 of this embodiment acquires form image data from the image database 240 using the image data acquisition unit 293 (step S1001).

[0132] Next, the form recognition unit 290 cuts out an area where characters are formed (hereinafter, a character field) from the form image using the character recognition unit 294, cuts out the characters in the cut-out character field, and performs character recognition (step S1002). At this time, the character recognition unit 294 also acquires coordinates indicating the position of the cut-out character.

[0133] Next, the form recognition unit 290 generates and stores reading result information based on the positional relationship of each character recognized by the character recognition unit 294 (step S1003). Specifically, the form recognition unit 290 recognizes characters whose mutual distance is within a preset threshold as one character string based on the coordinates of each recognized character, and generates reading result information by associating the character string with information such as coordinates indicating the position of the area where the character string is formed. Details of the reading result information will be described later.

[0134] Next, the form recognition unit 290 causes the second extraction unit 296 to identify, from the group of character strings included in the read result information, a character string that matches the character string set as the extraction base point corresponding to the item name "bank name" in the definition information 251 (step S1004). Note that, if there is no character string that matches the extraction base point corresponding to the item name "bank name", the second extraction unit 296 may identify a character string that partially matches the character string set as the extraction base point. In other words, the second extraction unit 296 identifies a character string that will be the base point for extracting the item value of the item name "bank name".

[0135] Next, the second extraction unit 296 determines whether or not a character string that serves as an extraction base point for the item name "bank name" has been identified in step S1003 (step S1005).

[0136] In step S1005, if a corresponding character string is not identified, the second extraction unit 296 ends the process of extracting the item value of the item name "bank name".

[0137] In step S1005, when a character string that serves as the extraction base point for the item name "bank name" is identified, the second extraction unit 296 extracts, from the group of character strings included in the read result information, a character string that is arranged in the direction indicated by the extraction direction corresponding to the item name "bank name" in the definition information 251 (step S1006). The character string extracted here is a character string that becomes the item value of the item name "bank name".

[0138] Next, the second extraction unit 296 compares the dictionary information stored in the dictionary database 260 with the string extracted in step S1006 to determine the item value of the item name "bank name" (step S1007), and terminates the process of extracting the item value of the item name "bank name".

[0139] In the embodiment, the first extraction unit 295 may perform a process of extracting invoice information from the form image data before the second extraction unit 296 extracts financial institution information. Also, the first extraction unit 295 may perform a process of extracting invoice information after the second extraction unit 296 extracts financial institution information.

[0140] In the example of Figure 10, in step S1007, the second extraction unit 296 determines the item value of the bank name by referring to the dictionary information stored in the dictionary database 260, but the second extraction unit 296 does not have to refer to the dictionary database 260.

[0141] 11 is a second flowchart illustrating the processing of the form recognition unit, which shows the processing of extracting a character string that becomes the item value of the item name that follows the item name "bank name" in the financial institution information.

[0142] In this embodiment, the document recognition unit 290, using the second extraction unit 296, refers to the definition information 251 and obtains the character string that serves as the extraction base point corresponding to the item name (item name to be extracted) from which the item value is to be extracted, and the extraction direction (step S1101).

[0143] Next, the second extraction unit 296 determines whether or not a character string that serves as an extraction base point has been acquired (step S1102). If the corresponding character string has not been acquired in step S1102, the second extraction unit 296 proceeds to step S1105, which will be described later.

[0144] In step S1102, if a corresponding character string is acquired, the second extraction unit 296 refers to the definition information 251 and acquires a character string that is arranged within the extraction range along the extraction direction of the acquired character string and that satisfies the condition information 252 (step S1103).

[0145] Next, the second extraction unit 296 determines whether or not the corresponding character string has been acquired in step S1103 (step S1104). If the corresponding character string has been acquired in step S1104, the second extraction unit 296 proceeds to step S1107, which will be described later.

[0146] In step S1104, if a corresponding character string is not acquired, the second extraction unit 296 determines whether or not there is a character string that can be used as the next extraction base point (step S1105).

[0147] In step S1105, if a corresponding character string exists, the second extraction unit 296 returns to step S1101.

[0148] If no corresponding character string exists in step S1105, the second extraction unit 296 sets the item value of each item name included in the definition information 251 to be blank (step S1106), and proceeds to step S1110, which will be described later. Therefore, in this embodiment, if the item value of the item name "bank name" is not extracted, the item values ​​of all item names in the financial institution information will be blank.

[0149] If the corresponding character string cannot be acquired in step S1104, the second extraction unit 296 refers to the value of the item "dictionary reference" in the definition information 251, and determines whether to refer to the dictionary database 260 (step S1107).

[0150] If it is determined in step S1107 that the dictionary database 260 is not to be referenced, the second extraction unit 296 proceeds to step S1109, which will be described later.

[0151] If it is determined in step S1107 that the dictionary database 260 is to be referenced, the second extraction unit 296 references the dictionary database 260 and compares it with the extracted character string (step S1108).

[0152] Next, the second extraction unit 296 determines a character string that will be the item value of the item name (step S1109).

[0153] Next, the second extraction unit 296 determines whether or not item values ​​have been associated with all item names included in the definition information 251 (step S1110). If it is determined in step S1110 that all item values ​​have not been associated, the second extraction unit 296 returns to step S1101.

[0154] In step S1110, if all item names are associated with item values, the recognition result generating unit 297 of the document recognition unit 290 generates recognition result data in which financial institution information is associated with invoice information (step S1111).

[0155] Next, the recognition result generating unit 297 registers the storage destination of the recognition result data in the job list 91 (step S1112), and ends the process.

[0156] In this way, the second extraction unit 296 of this embodiment repeats the process of treating the character string extracted based on its positional relationship with the character string set as the extraction base point as the character string that will be the next extraction base point.Then, the second extraction unit 296 of this embodiment treats the character string extracted based on its positional relationship with the character string set as the extraction base point as the item value of the item name corresponding to the character string set as the extraction base point.

[0157] Next, the processing of the form recognition unit 290 will be specifically described with reference to Figures 12, 13A, and 13B. Figure 12 is a diagram for explaining character recognition.

[0158] Information 121 shown in FIG. 12 is the result of character recognition performed by the character recognition unit 294 on the form image data.

[0159] The information 121 shows an example in which a character field is cut out, and the results of recognizing characters in the character field and the coordinates indicating the positions of the characters are extracted.

[0160] The character recognition unit 294 of this embodiment recognizes a character string (character string) by connecting adjacent characters from this information 121. Specifically, the character recognition unit 294 may recognize a combination of characters in which the distance between characters is within a predetermined number of pixels as one word.

[0161] 12 is information indicating the character string recognized by the character recognition unit 294 and the area of ​​the character string, and is information indicating the result of reading the form image by the character recognition unit 294.

[0162] Furthermore, the character recognition unit 294 recognizes an area separated by vertical and horizontal lines from the form image data as one cell, and assigns identification information (cell ID) for identifying each cell.

[0163] Information 122 shown in FIG. 12 is information indicating the correspondence between the area of ​​each cell recognized from the form image and the cell ID.

[0164] In FIG. 12, for example, the area in which the character string "Payment destination" is recognized is identified from the height and width in the information 121, with the position indicated by the X coordinate and the Y coordinate as the reference.

[0165] It can also be seen that the area in which the keyword "name" was recognized in the information 121 is included in cell ID "2." Therefore, the keyword "payment destination" is associated with cell ID "37" as a character string included in the read result information.

[0166] In this way, the form recognition unit 290 of this embodiment associates character strings with cells from the form image data.

[0167] 13A is a diagram showing an example of the extraction result of financial institution information. The second extraction unit 296 of this embodiment first extracts the bank name from the items included in the financial institution information by the process of FIG.

[0168] Specifically, the second extraction unit 296 identifies a character string that serves as an extraction base point for the first item name "bank name" from the group of character strings included in the read result information in the definition information 251. Then, the second extraction unit 296 identifies a character string that serves as an item value for the item name "bank name" based on the positional relationship with the identified character string (first character string) defined by the definition information 251.

[0169] In the definition information 251, the item name "Bank Name" is a character string located to the right or below the character string of the extraction base point.

[0170] Here, for example, when the character string "transfer destination" is identified from the group of character strings in the read result information, the second extraction unit 296 uses this "transfer destination" as the extraction base point and sets the character string located to the right or below "transfer destination" as the item value of the item name "bank name."

[0171] If multiple extraction directions are defined in the definition information 251, the presence or absence of a character string at the corresponding position is determined in order from the first defined extraction direction.

[0172] In the example of FIG. 13A, it can be seen that the character string "XXX Bank" has been identified as the item value of the item "Bank Name."

[0173] Furthermore, even if the bank name indicated by the character string identified in the item value of the item "Bank Name" through character recognition is incorrect, the second extraction unit 296 will compare it with dictionary information and determine that the character string that partially matches the identified character string is the character string indicating the bank name.

[0174] Next, the second extraction unit 296 identifies the character string that serves as the extraction base point corresponding to the next item name "branch name" in the definition information 251. In the definition information 251, the character string that serves as the extraction base point for the item name "branch name" is the extracted bank name. Therefore, in this case, "XXX Bank" serves as the character string that serves as the extraction base point.

[0175] Because the extraction direction corresponding to the item name "branch name" is right and bottom, the second extraction unit 296 first determines whether or not there is a character string located to the right of "XXX Bank." If a character string exists in the corresponding position, this character string is set as the item value of the item name "branch name."

[0176] Furthermore, if there is no corresponding character string on the right, the second extraction unit 296 determines whether or not there is a character string arranged to the right of "XXX Bank." If there is a character string in the corresponding position, the second extraction unit 296 sets the character string as the item value of the item name "Branch Name."

[0177] The second extraction unit 296 performs this process for all item names included in the financial institution information, and information associating each item name with its item value is extracted as financial institution information 131.

[0178] Fig. 13B is a diagram showing an example of recognition result data. The recognition result data 141 of this embodiment includes data in which financial institution information 131 extracted by the second extraction unit 296 is associated with invoice information 142 extracted by the first extraction unit 295. Note that the recognition result data of this embodiment may also include payee information. The recognition result data 141 shown in Fig. 13B may be in a state before payee information is included in the recognition result data.

[0179] The invoice information 142 includes, for example, information items such as an invoice number for identifying the invoice, the issuer of the invoice (invoice source), the invoice date, and the invoice amount.

[0180] Next, the processing of step S824 in Fig. 8 will be described in detail with reference to Fig. 14. Fig. 14 is a third flowchart illustrating the processing of the form recognition unit 290. Fig. 14 shows the processing by the information acquisition unit 298 of the form recognition unit 290.

[0181] 8, the information acquisition unit 298 of this embodiment refers to the supplier database 270 (step S1401) after acquiring the recognition result data. Next, the information acquisition unit 298 determines whether supplier information exists in the supplier database 270, in which the supplier name in the supplier information completely matches the billing party in the invoice information included in the recognition result data (step S1402).

[0182] In step S1402, if the corresponding supplier information exists in supplier database 270, information acquisition section 298 proceeds to step S1410, which will be described later.

[0183] If there is no corresponding supplier information in supplier database 270 in step S1402, information acquisition unit 298 determines whether there is supplier information in supplier database 270 in which the standardized edit distance between the supplier name in the supplier information and the biller in the invoice information is equal to or less than a reference value (step S1403). Note that it is sufficient to determine here whether there is supplier information in which the supplier name is similar to the biller, and the method of determining whether they are similar may be a method other than the method of calculating the standardized edit distance.

[0184] In step S1403, if the relevant supplier information exists in supplier database 270, information acquisition section 298 proceeds to step S1409, which will be described later.

[0185] In step S1403, if there is no corresponding customer information in customer database 270, information acquisition unit 298 determines whether financial institution information has been extracted by second extraction unit 296 (step S1404).

[0186] Specifically, the information acquisition unit 298 determines whether the recognition result data 141 includes the financial institution information 131 or not.

[0187] If financial institution information is not extracted in step S1404, the information acquisition unit 298 proceeds to step S1411, which will be described later.

[0188] If financial institution information is extracted in step S1404, the information acquisition unit 298 determines whether or not multiple pieces of financial institution information have been extracted (step S1405).

[0189] In step S1405, if multiple pieces of financial institution information are not extracted, that is, if only one piece of financial institution information is extracted, the information acquisition unit 298 sets this financial institution information as the deposit destination information (step 1406) and proceeds to step S1411, which will be described later.

[0190] In step S1405, if multiple pieces of financial institution information are extracted, the information acquisition unit 298 causes the terminal device 400 to display a financial institution information selection screen (step S1407), sets the selected financial institution information as the deposit destination information (step S1408), and proceeds to step S1411, which will be described later.

[0191] In step S1403, if the relevant supplier information exists in supplier database 270, information acquisition unit 298 refers to the supplier information with the smallest standardized edit distance (step S1409).

[0192] Next, the information acquisition unit 298 acquires the payment information included in the referenced business partner information, and sets the payee information included in the payment information (step S1410).

[0193] Next, the information acquisition unit 298 compares the financial institution information extracted by the second extraction unit 296 with the deposit destination information included in the customer information in the customer database 270, and determines whether the two match (step S1411).

[0194] If the two do not match in step S1411, the information acquisition unit 298 generates an alert to be displayed on a confirmation screen for the recognition result, and ends the process.

[0195] If the two match in step S1411, the information acquisition unit 298 ends the process.

[0196] In this embodiment, business partner information whose business partner name matches or is similar to the billing party extracted from the document image data is obtained from the business partner database 270, and the payee information included in the pre-stored payment information is set in the recognition result data as part of the business partner information.

[0197] Therefore, in this embodiment, when a tenant pays the billing amount to the billing party, there is no need to check the payment destination information displayed on the recognition result confirmation screen every time, thereby reducing the workload of the tenant (user).

[0198] In addition, in this embodiment, if there is no customer information corresponding to the billing source in the customer database 270, the financial institution information extracted from the document image data is set as the payee information, thereby reducing the effort required for the tenant (user) to enter the payee information.

[0199] Furthermore, in this embodiment, the payee information acquired from the customer database 270 is compared with the financial institution information extracted from the form image data, and if they do not match, an alert is displayed on the recognition result confirmation screen. Therefore, according to this embodiment, it is possible to prompt the user to confirm whether the payee information extracted by the server device 200 is correct.

[0200] Next, referring to Fig. 15, the processing of the update unit 299 when the payee information displayed on the recognition confirmation screen is updated will be described. Fig. 15 is a fourth flowchart illustrating the processing of the document recognition unit. The processing shown in Fig. 15 is processing performed by the update unit 299 after an operation is performed on the recognition result confirmation screen to change the state of the recognition result data to a confirmed state.

[0201] When the update unit 299 of this embodiment receives an operation to change the state of the recognition result data to a confirmed state (step S1501), it determines whether the supplier database 270 has been referenced in the processing by the information acquisition unit 298 (step S1502).

[0202] In step S1502, if the customer database 270 is not referenced, the update unit 299 associates the currently set payee information (the character string extracted by the second extraction unit 296) with the billing source and stores (registers) it in the customer database 270 as new customer information (step S1503), and proceeds to step S1506, which will be described later.

[0203] In step S1502, if supplier database 270 is referenced, update unit 299 determines whether or not the payee information has been edited (step S1504). In step S1504, if the payee information has not been edited, update unit 299 proceeds to step S1506, which will be described later.

[0204] If the payee information has been edited in step S1504, update unit 299 updates the payee information associated with the supplier name corresponding to the biller in supplier database 270 (step S1505).

[0205] Specifically, the update unit 299 may determine that the payment destination information has been edited, for example, when the payment information included in the customer information in the customer database 270, which includes a customer name that matches or is similar to the biller, does not match the payment destination information that was set when the operation to set the status to confirmed was accepted.

[0206] Next, the update unit 299 sets the state of the recognition result data to a confirmed state, and the output unit 285 generates output data (step S1506), thereby ending the process.

[0207] In this embodiment, when the payee information is edited in this way, the supplier database 270 can be updated to match the edited content. Therefore, in this embodiment, the user (tenant) can update the supplier database 270, which is the master data, by editing the payee information once. Therefore, in this embodiment, there is no need to check or correct the payee information every time form recognition is performed, thereby reducing the burden on the user.

[0208] Next, display examples of this embodiment will be described with reference to Fig. 16 to Fig. 21. Fig. 16 is a first diagram showing a display example.

[0209] 16 may be displayed on, for example, the terminal device 400. The screen 161 is an example of a confirmation screen for the recognition result that is displayed on the browser of the terminal device 400 in step S826.

[0210] Screen 161 includes display fields 162 and 163, and operation buttons 169 and 170. A form image indicated by form image data is displayed in display field 162. In the example of Fig. 16, the form is an invoice, and the form image includes area 164a for displaying invoice information and area 164b for displaying financial institution information.

[0211] The display field 163 displays the recognition result data obtained by performing form recognition on the form image data displayed in the display field 162 by the form recognition unit 290 .

[0212] Display field 163 includes display fields 165, 166, 167, and 168. Display field 165 displays invoice information 142. Invoice information 142 is part of the recognition result data. Display field 166 displays financial institution information 131. Display field 167 displays remittance destination information. Display field 168 displays the value of the item "status" in job list 91.

[0213] In the example of FIG. 16, the invoice information 142 extracted from the area 164a is displayed in the display field 165. The financial institution information 131 extracted from the area 164b is displayed in the display field 166. In the example of Fig. 16, the financial institution information 131 extracted from the area 164a indicates one financial institution.

[0214] In the example of FIG. 16, the payee information acquired from the business partner information corresponding to the biller "XX Corporation" included in the invoice information 142 is displayed in the display field 167.

[0215] In the example of Figure 16, the billing party is "XX Corporation." Therefore, the information acquisition unit 298 of the document recognition unit 290 refers to the supplier database 270. The information acquisition unit 298 then determines whether or not there is supplier information whose supplier name exactly matches "XX Corporation."

[0216] In this case, since the relevant customer information exists (see Figure 6), the information acquisition unit 298 extracts the financial institution, branch name, account type, account number, and account holder name from the payment information contained in the relevant customer information, and sets them as the deposit destination information in the display field 167.

[0217] In this embodiment, since the financial institution information displayed in display field 166 matches the deposit destination information displayed in display field 167, no alert is generated and no alert is displayed on screen 161.

[0218] The display field 167 also includes a display component 167c for editing the payee information displayed in the display field 167.

[0219] In this embodiment, when an operation to select a display component 167c included in the display column 167 is accepted, it becomes possible to edit the content displayed in the corresponding display column. Editing the content displayed in the display column will be described in detail later.

[0220] Operation button 169 is an operation button for changing the status value of job list 91 from unprocessed to draft saved. When operation button 169 is operated, display field 168 changes from "unprocessed" to "draft saved."

[0221] The operation button 170 is an operation button for changing the status value of the job list 91 to the confirmed state. In this embodiment, when the operation button 170 is operated, the status value of the job list 91 is changed to the confirmed state. In this embodiment, the recognition result data whose status has been changed to the confirmed state can be converted by the output unit 285 into data in a format compatible with the core system 500.

[0222] Fig. 17 is a second diagram showing a display example. Screen 161A shown in Fig. 17 shows a case where area 164b showing financial institution information includes information showing a plurality of financial institutions.

[0223] In this case, a plurality of pieces of financial institution information are displayed in display field 166A included in display field 163A. Display field 163A also includes selection fields 166a and 166b for selecting a plurality of pieces of financial institution information.

[0224] In the example of FIG. 17, financial institution information can be selected by selecting either selection field 166a or 166b.

[0225] 17, when financial institution information is selected, the information acquisition unit 298 may refer to the customer database 270 and identify customer information whose customer name matches "XX Co., Ltd." and whose financial institution name matches the financial institution name included in the selected financial institution information. The information acquisition unit 298 may then extract payee information from the identified customer information and display it in the display field 167.

[0226] 17, the selection field for financial institution information is displayed on screen 161A, which is the confirmation screen for the recognition result, but this is not limiting. The selection screen for financial institution information may be displayed on terminal device 400 or the like before the confirmation screen for the recognition result is displayed.

[0227] Furthermore, in this embodiment, for example, when multiple pieces of financial institution information are extracted, financial institutions where the tenant holds an account may be registered in advance in a tenant DB (not shown) in the storage unit 220, and financial institution information containing the same financial institution name as the registered financial institution corresponding to the tenant may be automatically selected. This can reduce fees for remittances between financial institutions. Furthermore, in this embodiment, remittance fees between different financial institutions are stored in the storage unit 220, and a financial institution with a low transmission fee for the financial institution corresponding to the tenant may be automatically selected from the multiple extracted financial institutions. Furthermore, information indicating the extracted financial institutions and information indicating the financial institution corresponding to the tenant may be transmitted to another system via a network, the transmission fee may be received, and the financial institution with the low transmission fee may be automatically selected based on the received transmission fee.

[0228] Fig. 18 is a third diagram showing a display example. In screen 161B shown in Fig. 18, display field 166, which displays financial institution information, is not displayed in display field 163B. In other words, in the example of Fig. 18, display field 165 and display field 167 are displayed in display field 163B, and bill information and payee information are displayed.

[0229] In this embodiment, for example, if the financial institution information extracted by the second extraction unit 296 and the deposit destination information acquired by the information acquisition unit 298 are completely identical, the display of the financial institution information extracted from the form image data is omitted, and only the deposit destination information, which is a pre-fixed item name and item value, is displayed.

[0230] In this embodiment, by omitting the display of financial institution information in this way, it is possible to present only necessary information to the user viewing screen 161B.

[0231] Fig. 19 is a fourth diagram showing a display example, in which display field 167A automatically displays content other than deposit destination information.

[0232] 19, the billing date is displayed in display field 165a of display field 165. The billing date is information included in the bill information.

[0233] In FIG. 19, the information acquisition unit 298, for example, refers to this billing date and the payment information included in the customer information whose customer name is "XX Co., Ltd.", and sets the payment deadline to be displayed in the display field 167a of the display field 167A.

[0234] In the customer database 270, the value of the item "payment terms" in the payment information included in the customer information for the customer whose name is "XX Corporation" is "payment due date: the 25th of two months from now" (see FIG. 6). Therefore, the information acquisition unit 298 derives September 25, 2018 as the 25th of two months from July 20, 2018, which is the invoice date, and displays this in the display field 167a.

[0235] In this manner, in this embodiment, information other than the payee information displayed in the display field 167 may also be automatically set from the bill information and payment information.

[0236] In this embodiment, by doing so, it is possible to reduce the number of items that the user (tenant) must manually input, thereby reducing the burden on the user.

[0237] Fig. 20 is a fifth diagram showing a display example. Screen 163D shown in Fig. 20 shows a case where the financial institution information extracted from the form image data by second extraction unit 296 does not match the payee information acquired from supplier database 270 by information acquisition unit 298.

[0238] On screen 161D, display field 167B displays alert 167b indicating that the financial institution information and the deposit destination information do not match. In this embodiment, by operating display component 167c, a screen for editing the supplier information stored in supplier database 270 can be displayed.

[0239] Incidentally, cases in which the financial institution information and the payee information do not match may occur, for example, when the referenced business partner information is incorrect, when the name of the financial institution included in the business partner information has been changed, or when a character string that does not indicate the billing source is mistakenly extracted as the billing source.

[0240] 21 is a sixth diagram showing a display example. Screen 161E shown in Fig. 21 shows the screen after display component 167c is operated on screen 161D shown in Fig. 20.

[0241] Screen 161E displays screen 171 for editing the supplier information for the supplier whose name is "XX Corporation."

[0242] Screen 171 displays a list of supplier information stored in supplier database 270, including supplier information with the supplier name "XX Corporation."

[0243] In this embodiment, for example, if there is an error in the supplier information to be referenced, the user can select the supplier information to be referenced from screen 171.

[0244] In addition, in this embodiment, if there is a change in the name of the financial institution, for example, the relevant customer information can be edited on screen 171.

[0245] In this embodiment, after such an operation is performed, when the operation button 170 is operated and the status value of the job list 91 is set to the confirmed state, the recognition result data that has been set to the confirmed state is converted by the output unit 285 into data in a format compatible with the core system 500.

[0246] Next, an example of data output to the core system 500 by the output unit 285 of this embodiment will be described with reference to FIG.

[0247] Fig. 22 is a diagram showing an example of output data. The output data 231 shown in Fig. 22 includes payment source information, payment destination information, billing amount, transfer execution date, and the like.

[0248] In the output data 231 of this embodiment, the payment source information is information included in the business partner information stored in the business partner database 270, and is information about the account held by the tenant.

[0249] The payee information is the payee information included in the customer information stored in the customer database 270.

[0250] The billing amount is included in the bill information extracted from the form image data by the form recognition unit 290. The transfer execution date may be the date on which the output data 231 is output to the core system 500, for example.

[0251] The output data 231 of this embodiment may be in any format that allows the core system 500 to automatically execute a process for transferring (paying) the billed amount from the tenant to the billing source.

[0252] In this manner, in this embodiment, the billing source is identified from the form image data of the bill, and remittance destination information, in which item names and item values ​​are predetermined, is obtained from the supplier database 270.

[0253] Therefore, according to this embodiment, it is possible to eliminate the need to check whether the payee information is correct each time the payee information is displayed on the confirmation screen for the recognition result.

[0254] Furthermore, in this embodiment, the financial institution information extracted from the form image data is compared with the deposit destination information, and if the two do not match, the supplier database 270, which serves as master data, can be updated.

[0255] Therefore, even if the payee information is incorrect, the user only needs to correct the master data once, eliminating the need to repeatedly correct the payee information.

[0256] In the above embodiment, an invoice is described as an example of a form, but the type of form to which this embodiment is applied is not limited to an invoice. Also, in this embodiment, the fixed information is payee information, but this is not limiting.

[0257] The form may be, for example, an order form, and in that case, the fixed information may be the address of the orderer or the address of the delivery destination, which are fixed in advance depending on the orderer that issued the order form. Furthermore, the form may be, for example, an exam answer sheet, and in that case, the fixed information may be, for example, the name of the teacher teaching the subject, which is fixed in advance depending on the subject for which the exam was conducted (i.e., information indicating which subject the exam was for), and which indicates the issuer of the answer sheet.

[0258] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to perform each function described above.

[0259] Additionally, the devices described in the embodiments are merely representative of one of several computing environments for implementing the embodiments disclosed herein.

[0260] In one embodiment, server apparatus 200 includes multiple computing devices, such as a server cluster, configured to communicate with each other over any type of communications link, including a network, shared memory, etc., to perform the processes disclosed herein. Similarly, server apparatus 200 may include multiple computing devices configured to communicate with each other.

[0261] Furthermore, server device 200 can be configured to share the disclosed processing steps in various combinations. For example, processes performed by server device 200 can be performed by other server devices. Similarly, functions of server device 200 can be performed by other server devices. Furthermore, the server device and the elements of the other server devices can be integrated into a single server device or can be separated into multiple devices.

[0262] The correspondence table in the statement may be generated by the learning effect of machine learning. Also, by using machine learning to classify keywords that may be included in the description of transaction details and account items, it is not necessary to use a correspondence table.

[0263] Here, machine learning is a technology that allows a computer to acquire human-like learning capabilities, where the computer autonomously generates the algorithms necessary for making judgments such as data classification from training data that is input in advance, and applies these to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods; the learning method for machine learning is not important.

[0264] Although the present invention has been described above based on the embodiments, the present invention is not limited to the requirements shown in the above embodiments. These requirements can be changed without departing from the spirit of the present invention, and can be appropriately determined depending on the application form. [Explanation of symbols]

[0265] 100 Document Recognition System 200 Server device 220 Storage section 230 Form recognition processing unit 240 image database 250 Extraction Definition Databases 260 Dictionary Database 270 tenant databases 280 Recognition Result Database 284 Display control unit 285 Output Section 290 Document Recognition Unit 291 Image data storage unit 292 Job List Generation Unit 293 Image data acquisition unit 294 Character recognition section 295 First extraction part 296 Second extraction part 297 Recognition result generation unit 298 Information Acquisition Department 299 Update Department 300 Image forming device 400 Terminal Equipment 500 Core Systems [Prior art documents] [Patent documents]

[0266] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-51339

Claims

1. a storage unit that stores information indicating the issuer of a form and information associated with the issuer of the form in advance in association with each other; an extraction unit that extracts information indicating the issuer of the form and information associated with the issuer of the form from the form image data; an update unit that, when the information indicating the issuer of the form extracted by the extraction unit is not stored in the storage unit, associates the information indicating the issuer of the form extracted by the extraction unit with information associated with the issuer of the form and registers them in the storage unit; an information acquisition unit that, when the information indicating the issuer of the document extracted by the extraction unit is stored in the storage unit, acquires information associated with the extracted information indicating the issuer of the document from the storage unit; a display control unit that outputs screen data to a terminal device, the screen data including information indicating the issuer of the form and information associated with the information indicating the issuer of the form acquired by the information acquisition unit; The information processing device wherein, when an edit is made to information associated with information indicating the issuer of the report displayed on the terminal device, the update unit updates the information associated with information indicating the issuer of the report stored in the memory unit to match the content of the edit.

2. An information processing device as described in Claim 1, wherein extraction by the extraction unit is performed based on a definition of the string to be extracted.

3. The information acquisition unit An information processing device as described in claim 1, wherein, when the information indicating the issuer of the document extracted by the extraction unit is not stored in the memory unit, it determines whether information indicating an issuer similar to the extracted information indicating the issuer exists in the memory unit, and obtains information associated with the information indicating the similar issuer.

4. a recognition processing unit that refers to information indicating a definition of the character string to be extracted, identifies a character string that serves as an extraction base point from a plurality of character strings extracted by character recognition of the form image data, and extracts a character string that corresponds to the character string that serves as the extraction base point based on a positional relationship with the character string that serves as the extraction base point; The recognition processing unit 3. The information processing apparatus according to claim 2, wherein a character string extracted based on a positional relationship with the character string serving as the extraction base point is set as an item value of an item name associated with the character string serving as the extraction base point.

5. The display control unit An information processing device as described in claim 4, which outputs screen data to a terminal device, the screen data including information indicating the issuer of the report, information associated with the information indicating the issuer of the report, and item values ​​of item names extracted based on the recognition processing unit.

6. The information processing apparatus according to claim 5 , wherein the document is an invoice.

7. the item names extracted by the recognition processing unit are item names of information related to financial institutions, The information processing apparatus according to claim 6 , wherein the recognition processing unit associates the item names with the item values ​​to generate financial institution information.

8. The information acquisition unit determining whether or not information associated with the information indicating the issuer of the document stored in the storage unit matches information associated with the information indicating the issuer of the extracted document; The display control unit An information processing device as described in claim 7, wherein the screen data includes information for displaying an alert when information associated with information indicating the issuer of the report stored in the memory unit does not match information associated with information indicating the issuer of the extracted report.

9. An information processing device as described in claim 8, wherein when multiple pieces of information that can be associated with information indicating the issuer of the document are extracted, a selection screen is displayed and the selected information is made to be information that can be associated with the extracted information indicating the issuer of the document.

10. An information processing method by an information processing device, the information processing device comprising: storing information indicating the issuer of the form and information associated with the issuer of the form in a storage unit in advance in association with each other; extracting information indicating the issuer of the form and information associated with the issuer of the form from the form image data; If the extracted information indicating the issuer of the form is not stored in the storage unit, the extracted information indicating the issuer of the form and information associated with the issuer of the form are associated with each other and registered in the storage unit; When the extracted information indicating the issuer of the form is stored in the storage unit, information associated with the extracted information indicating the issuer of the form is acquired from the storage unit; outputting screen data including information indicating the issuer of the form and information associated with the acquired information indicating the issuer of the form to a terminal device; An information processing method in which, when editing is performed on information associated with information indicating the issuer of the report displayed on the terminal device during registration, the information associated with information indicating the issuer of the report stored in the memory unit is updated to match the content of the editing.

11. storing information indicating the issuer of the form and information associated with the issuer of the form in a storage unit in advance in association with each other; extracting information indicating the issuer of the form and information associated with the issuer of the form from the form image data; If the extracted information indicating the issuer of the form is not stored in the storage unit, the extracted information indicating the issuer of the form and information associated with the issuer of the form are associated with each other and registered in the storage unit; When the extracted information indicating the issuer of the form is stored in the storage unit, information associated with the extracted information indicating the issuer of the form is acquired from the storage unit; outputting screen data including information indicating the issuer of the form and information associated with the acquired information indicating the issuer of the form to a terminal device; An information processing program that causes a computer to execute a process in which, when editing is performed on information associated with information indicating the issuer of the report displayed on the terminal device during registration, the information associated with information indicating the issuer of the report stored in the memory unit is updated to match the content of the editing.

Images