Information processing device, information processing method, program, and information processing system

Abstract

To enable more efficient business form information management.SOLUTION: An information processing device is provided, comprising a display control unit configured to respond to editing of given information included in business form information by causing an input screen for the edited given information and associated information to be displayed on a terminal device, and a storage unit configured to store management information obtained by associating information entered on the input screen with the edited given information.SELECTED DRAWING: Figure 20

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a program, and an information processing system. [Background technology]

[0002] 2. Description of the Related Art Conventionally, a system has been known that extracts items and item values ​​included in a form image and manages form information in which the items and item values ​​are associated with each other. Summary of the Invention [Problem to be solved by the invention]

[0003] Form information is managed using predetermined information contained in the form information as a key. However, in conventional systems, the predetermined information may be edited in anticipation of use in a downstream system to which the form information is output. Specifically, for example, editing may be performed to add a character string to the predetermined information that is not associated with the predetermined information on the form. Such editing may be performed by each user of the conventional system according to their own convenience, and when such editing is performed on the predetermined information, management of the form information becomes cumbersome.

[0004] The disclosed technology has been developed in consideration of the above circumstances, and aims to more efficiently manage form information. [Means for solving the problem]

[0005] The disclosed technology is Extracted from form image data a display control unit that, when editing is performed on predetermined information included in form information, displays on a terminal device an input screen for information related to the edited predetermined information; information input on the input screen; and the edited predetermined information. , the predetermined information before editing; and a storage unit that stores management information in which the above-mentioned items are associated with each other. [Effects of the Invention]

[0006] According to the disclosed technology, form information can be managed more efficiently. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of a document recognition system. [Figure 2] FIG. 2 illustrates an example of a hardware configuration of a server device. [Figure 3] FIG. 2 illustrates an example of a hardware configuration of a terminal device. [Figure 4] FIG. 10 is a diagram illustrating an example of a type determination database. [Figure 5] FIG. 2 is a diagram showing types of form definition information stored in a form definition database. [Figure 6] FIG. 10 is a diagram illustrating an example of general definition information. [Figure 7] FIG. 10 is a diagram illustrating an example of individual company definition information. [Figure 8] FIG. 10 is a diagram illustrating an example of specific definition information. [Figure 9] FIG. 2 is a diagram illustrating an example of a management information database. [Figure 10] FIG. 2 is a diagram illustrating functions of a server device. [Figure 11] FIG. 2 is a diagram illustrating functions of a terminal device. [Figure 12] FIG. 2 is a sequence diagram illustrating the operation of the document recognition system. [Figure 13] FIG. 10 is a diagram illustrating an example of a job list. [Figure 14] 10 is a first flowchart illustrating processing by a document recognition unit. [Figure 15] 10 is a second flowchart illustrating the processing of the document recognition unit. [Figure 16] 10 is a third flowchart illustrating the processing of the document recognition unit. [Figure 17] 10 is a flowchart illustrating a process of a form management unit. [Figure 18] FIG. 10 is a first diagram showing a display example of a terminal device. [Figure 19]FIG. 10 is a second diagram showing an example of a display on the terminal device. [Figure 20] FIG. 10 is a third diagram showing an example of a display on the terminal device. DETAILED DESCRIPTION OF THE INVENTION

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment will be described below with reference to the accompanying drawings. Fig. 1 is a diagram showing an example of the system configuration of a form recognition system.

[0009] The form recognition system 100 of this embodiment includes a server device 200, an image forming device 300, and a terminal device 400. In the form recognition system 100 of this embodiment, the server device 200, the image forming device 300, and the terminal device 400 are connected via a network. The form recognition system 100 of this embodiment is an example of an information processing system.

[0010] Furthermore, the terminal device 400 of this embodiment is connected to a core system 500, for example, via a network. The core system 500 is a system that performs specific processing using the results of recognition of a form by the form recognition system 100 of this embodiment. Specifically, the core system 500 may be, for example, an accounting system, an estimate creation system, or the like.

[0011] In the document recognition system 100 of this embodiment, the server device 200 extracts the items and item values ​​contained in the document from image data showing the image of the document read by the scanner function of the image forming device 300, and converts the items and item values ​​into text data that corresponds to each other.

[0012] In the following description, the image of the form read by the scanner function of the image forming apparatus 300 is referred to as a form image, and image data representing the form image is referred to as form image data. Note that the image forming apparatus 300 does not necessarily have to be included in the form recognition system 100.

[0013] In the following description, extracting items included in a form image and item values ​​that are the values ​​of these items, and associating the items with the item values, is referred to as form recognition.

[0014] The server device 200 of this embodiment includes a storage unit 210, a form recognition unit 220, and a form management unit 290. The storage unit 210 includes a form image database 230, a type determination database 240, a form definition database 250, a recognition result database 260, and a management information database 265.

[0015] The form image database 230 stores form image data. The type determination database 240 stores information referenced when determining the type of form. The form definition database 250 stores detail definition information referenced when the form recognition unit 220 performs form recognition. The recognition result database 260 stores form information (recognition result data) that is information indicating the results of form recognition by the form recognition unit 220. The management information database 265 stores management information for managing form information.

[0016] The form recognition unit 220 of this embodiment acquires form image data stored in the form image database 230, and determines the type of form by referring to the type determination database 240 and identifies the detail definition information to be referenced. Then, the form recognition unit 220 performs form recognition using the identified detail definition information in the form definition database 250, and stores the form information acquired as a result in the recognition result database 260. The detail definition information includes the item names (detail item names) of the detail items included in the form and information indicating the position where the item names are written.

[0017] Here, the form recognition unit 220 of this embodiment refers to the general definition information, individual company definition information, and specific definition information stored in the form definition database 250 when performing form recognition.

[0018] The general definition information in this embodiment is information that defines words corresponding to item names in general forms. The individual company definition information is information that defines words corresponding to item names in forms specific to a user who uses the form recognition system 100 and information to be extracted from form images. The specific definition information is form definition information that defines a specific form issued by a specific business or the like. A specific form is, for example, a commonly used delivery slip or the like. Details of the type determination database 240 and the form definition database 250 will be described later.

[0019] In this embodiment, by using three types of detail definition information when recognizing a form, it is possible to recognize the detail details of a form in addition to general-purpose forms, thereby improving the accuracy of form recognition.

[0020] In this embodiment, when specified information contained in the form information recognized by the form recognition unit 220 is edited, the form management unit 290 displays an input screen that allows the user to enter information related to the specified information.

[0021] Then, the form management unit 290 stores in the management information database 265 management information that associates information related to the predetermined information with the edited predetermined information.

[0022] The predetermined information in this embodiment is information that serves as a primary key used to manage form information stored in the recognition result database 260. Specifically, the predetermined information is information that indicates the issuer of the form.

[0023] Here, in this embodiment, the issuer of a form refers only to the name of the corporation or individual that issued the form, and does not include the name of a branch office or department.

[0024] Furthermore, the information related to the predetermined information includes the predetermined information and information (identification information) that identifies the edited predetermined information. In other words, the information related to the predetermined information is information related to the issuer of the form, and includes information that indicates the issuer of the form and information that identifies the issuer after editing.

[0025] Therefore, the management information in this embodiment includes predetermined information, information for identifying the predetermined information after editing, and the predetermined information after editing. In other words, the management information includes the issuer of the form, information for identifying the issuer after editing, and the issuer after editing.

[0026] In this manner, in this embodiment, the management information database 265 manages the predetermined information and the edited predetermined information in association with each other as separate items.

[0027] Therefore, in this embodiment, even if specified information contained in the report information is edited, by referring to the management information, the report information can be managed in the same way as if the specified information had not been edited, allowing for efficient management of the report information.

[0028] In the form recognition system 100 of this embodiment, the image forming device 300 is a multifunction device with a scanner function. The image forming device 300 is equipped with applications for realizing functions such as a copy function, a fax function, and a scanner function, and these functions are realized by selecting an application corresponding to each function.

[0029] The terminal device 400 of this embodiment is used by a user who uses the form recognition system 100. The terminal device 400 may also display a screen or the like for allowing the user to check the results of form recognition by the server device 200.

[0030] The user in this embodiment may be, for example, an establishment, a company, an organization, or the like that has concluded a contract to use the services provided by the document recognition system 100.

[0031] 1, the server device 200 has five databases in the storage unit 210, but the configuration is not limited to this. Some of the databases may be provided in an external device to the server device 200, or all of the databases may be provided in an external device.

[0032] 1, the server device 200 is assumed to realize the form recognition unit 220 and the form management unit 290, but this is not limiting. The form recognition unit 220 and the form management unit 290 may be realized by a plurality of information processing devices.

[0033] In addition, in the example of Figure 1, the document recognition system 100 includes one image forming device 300 and one terminal device 400, but the number of image forming devices 300 and one terminal device 400 included in the document recognition system 100 may be any number.

[0034] Next, the hardware configuration of the server device 200 of this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the hardware configuration of the server device.

[0035] The server device 200 of this embodiment is constructed by a computer, and as shown in FIG. 2, includes a CPU 201, a ROM 202, a RAM 203, an HD 204, an HDD (Hard Disk Drive) controller 205, a display 206, an external device connection I / F (Interface) 208, a network I / F 209, a data bus B, a keyboard 211, a pointing device 212, a DVD-RW (Digital Versatile Disk Rewritable) drive 214, and a media I / F 216.

[0036] Of these, the CPU 201 controls the overall operation of the server device 200. The ROM 202 stores programs used to drive the CPU 201, such as an IPL. The RAM 203 is used as a work area for the CPU 201. The HD 204 stores various data such as programs. The HDD controller 205 controls the reading and writing of various data from and to the HD 204 under the control of the CPU 201. The display 206 is a display unit that displays various information such as a cursor, menus, windows, characters, or images.

[0037] The external device connection I / F 208 is an interface for connecting various external devices. In this case, the external devices are, for example, a USB (Universal Serial Bus) memory or a printer. The network I / F 209 is an interface for data communication using the communication network N. The data bus B is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 201 shown in FIG. 2.

[0038] The keyboard 211 is a type of input means having multiple keys for inputting characters, numbers, various instructions, etc. The pointing device 212 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 214 controls reading and writing of various data from and to a DVD-RW 213, which is an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R, etc. The media I / F 216 controls reading and writing (storing) of data from and to a recording medium 215, such as a flash memory.

[0039] The server device 200 of this embodiment may be, for example, a smartphone, a tablet terminal, a PDA (Personal Digital Assistant), a wearable PC, or the like.

[0040] The terminal device 400 of this embodiment is a computer having an arithmetic processing unit and a storage device, and may be, for example, a tablet-type terminal device, a smartphone, etc. Fig. 3 is a diagram showing an example of the hardware configuration of the terminal device.

[0041] The terminal device 400 of this embodiment includes a CPU 401 , a ROM 402 , a RAM 403 , an EEPROM 404 , a CMOS sensor 405 , an image sensor I / F 406 , an acceleration / direction sensor 407 , a media I / F 409 , a data bus 410 , and a GPS receiving unit 411 .

[0042] Of these, the CPU 401 is an arithmetic processing unit that controls the overall operation of the terminal device 400. The ROM 402 stores the CPU 401 and programs used to drive the CPU 401, such as the IPL. The RAM 303 is used as a work area for the CPU 301. The EEPROM 404 reads or writes various data, such as programs for the smartphone (terminal device), under the control of the CPU 401. The ROM 402, RAM 403, and EEPROM 404 are examples of storage devices of the terminal device 400.

[0043] The CMOS (Complementary Metal Oxide Semiconductor) sensor 405 is a type of built-in imaging means that captures an image of a subject (mainly a self-portrait) and obtains image data under the control of the CPU 401. Note that instead of a CMOS sensor, an imaging means such as a CCD (Charge Coupled Device) sensor may also be used.

[0044] The imaging element I / F 406 is a circuit that controls the driving of the CMOS sensor 305. The acceleration / direction sensor 407 is a variety of sensors, such as an electronic magnetic compass that detects geomagnetism, a gyrocompass, and an acceleration sensor. The media I / F 409 controls the reading and writing (storage) of data from and to a recording medium 408, such as a flash memory. The data bus 410 is an address bus, a data bus, or the like, for electrically connecting the various components, such as the CPU 401, shown in FIG. 3. The GPS receiver 411 receives GPS signals from GPS satellites.

[0045] The terminal device 400 also includes a long-distance communication circuit 412, an antenna 412a of the long-distance communication circuit 412, a microphone 415, a speaker 416, an audio input / output I / F 417, a display (display unit) 418, an external device connection I / F (Interface) 419, a short-distance communication circuit 420, an antenna 420a of the short-distance communication circuit 420, and a touch panel 421.

[0046] Of these, the long-distance communication circuit 412 is a circuit that communicates with other devices via a communication network. The microphone 415 is a built-in circuit that converts sound into an electrical signal. The speaker 416 is a built-in circuit that converts the electrical signal into physical vibrations to produce sounds such as music or voice. The sound input / output I / F 417 is a circuit that processes the input and output of sound signals between the microphone 415 and the speaker 416 under the control of the CPU 401.

[0047] The display 418 is a type of display means such as a liquid crystal display or organic electroluminescence (EL) display that displays an image of a subject, various icons, etc. The external device connection I / F 419 is an interface for connecting various external devices. The short-range communication circuit 420 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The touch panel 421 is a type of input means that allows a user to operate the terminal device 400 by pressing the display 418. The display 418 is an example of a display unit that the terminal device 400 has.

[0048] Next, the type determination database 240 and the form definition database 250 included in the server device 200 of this embodiment will be described.

[0049] 4 is a diagram showing an example of a type determination database. The type determination database 240 of this embodiment has the following information items: definition ID, definition type, document type, and identification character string. In the type determination database 240, the item "definition ID" is associated with other items, and information including the value of the item "definition ID" and the values ​​of the other items is called type determination information.

[0050] The value of the item "definition ID" indicates identification information for identifying form definition information.

[0051] The value of the item "Definition Type" indicates the type of form definition information. In this embodiment, the types of form definition information include general definition information, individual company definition information, and specific definition information. Specific definition information is form definition information that defines a specific form issued by a specific business operator, etc. Specifically, a specific form is, for example, a commonly used delivery slip or the like.

[0052] The value of the item "document type" indicates the type of form, such as an invoice, estimate, or order form. The value of the item "identification string" indicates a string for identifying the type of form. Specifically, for example, a form image containing the string "invoice" or "claim" will have the type of form indicated by the item "document type" as "invoice."

[0053] In the example of Figure 4, for example, the type of report definition information for definition ID "H01" is general definition information, and the general definition information for definition ID "H01" is information that defines the layout of an "invoice" and is referenced when the character string "invoice" or "invoice" is extracted from the report image data.

[0054] Next, the form definition database 250 of this embodiment will be described with reference to Figures 5 to 8. Figure 5 is a diagram showing the types of form definition information stored in the form definition database.

[0055] The form definition database 250 of this embodiment stores general definition information 251, individual company definition information 252, and specific definition information 253.

[0056] The general definition information 251 in this embodiment is form definition information for a form with a highly versatile layout that can be used by multiple users. The general definition information 251 is the form definition information with the lowest priority among the three form definition information.

[0057] The individual company definition information 252 in this embodiment is form definition information for a form with a layout uniquely designed by the user, and is form definition information exclusive to the user. Of the three form definition information, the individual company definition information 252 is the definition that should be most respected and is the form definition information with the highest priority.

[0058] The specific definition information 253 of this embodiment is form definition information that indicates the layout of a form that is generally used among forms. The priority of the specific definition information 253 is intermediate among the three form definition information.

[0059] The form recognition unit 220 of this embodiment refers to these three pieces of form definition information to recognize form image data. Therefore, in this embodiment, even if the form image data is for a form whose layout is not defined in the general definition information 251, this form image data can be recognized as long as it is defined in the individual company definition information 252 or the specific definition information 253.

[0060] 6 is a diagram showing an example of general-purpose definition information. General-purpose definition information 251 is generated for each definition ID, and includes information items such as an item ID, an item name, a direction, and a distance. In other words, the general-purpose definition information 251 of this embodiment associates the item "definition ID" with other items, and includes the value of the item "definition ID" and the values ​​of the other items.

[0061] The value of the item "item ID" is identification information for identifying an item included in a form. The value of the item "item name" indicates the name of the item.

[0062] The items "direction," "distance," etc. indicate the range in which to extract the character string. For example, if the value of the "direction" item is "RD" and the value of the "distance" item is "20 mm," this indicates that the character string will be extracted from a range within 20 mm downward from the bottom right coordinate of the rectangle circumscribing the character string that is the item name.

[0063] In other words, the items "direction," "distance," etc. in this embodiment are conditions that indicate the range in which character strings are extracted, and these conditions are associated with the item names. Note that in the example of Fig. 5, the items "direction" and "distance" are given as conditions that indicate the range in which character strings are extracted, but other items may also be included in the conditions.

[0064] Therefore, for example, a string extracted from within a range of 20 mm downward from the bottom right coordinate of the rectangle circumscribing the string with the item name "Total" of item ID "2" is recognized as the value of the item corresponding to the item name "Total."

[0065] The general definition information 251 includes information about all items included in the form.

[0066] 7 is a diagram showing an example of individual company definition information. The individual company definition information 252 of this embodiment is registered for each user, and includes information items such as a user ID, a definition ID, an item name, a direction, and a distance.

[0067] The value of the item "user ID" is identification information for identifying a user. In this embodiment, the definition ID, item name, direction, distance, etc. are the same as those in the general definition information 251, so their explanation will be omitted.

[0068] In this embodiment, the definition ID of the form definition information whose definition type is individual company definition information may include a user ID.

[0069] Furthermore, information about items for which the user has registered the item name and a condition indicating the range in which the item value is described is stored in the individual company definition information 252. Therefore, the item ID included in the individual company definition information 252 may become part of the item ID included in the general definition information 251.

[0070] The example in FIG. 7 shows the individual company definition information 252 registered by the user ID "A", and this individual company definition information 252 has a definition ID of "A01" and a document type of "invoice for company R".

[0071] 8 is a diagram showing an example of specific definition information. The specific definition information 253 of this embodiment is registered for each specific form, and includes information items such as a definition ID, an item name, a direction, and a distance. In this embodiment of FIG. 8, the definition ID, the item name, the direction, the distance, and the like are the same as those of the general definition information 251, and therefore a description thereof will be omitted.

[0072] The specific definition information 253 of this embodiment may be provided for each specific business operator. In this case, the specific definition information 253 may include a business operator ID as an information item.

[0073] In the following description, an invoice is an example of a form, invoice information is an example of form information, and the issuer of the invoice (billing source) is the specified information (primary key) used to manage the invoice information.

[0074] The management information database 265 of this embodiment will be described below with reference to Fig. 9. Fig. 9 is a diagram showing an example of the management information database.

[0075] The management information database 265 of this embodiment includes, as information items, a management information ID, the name of the requester, the name of the requester after editing, a telephone number, and account information.

[0076] The value of the item "management information ID" is information assigned to the management information and is identification information for identifying the management information. The management information ID may be assigned when the management information is stored in the management information database 265, for example.

[0077] The value of the "Name of biller" field indicates the name of the biller of the invoice. More specifically, the value of the "Name of biller" field is only the name of the corporation or individual, and does not include branch or department names, etc. The name of the biller is, in other words, specified information.

[0078] The value of the item "Name of billing party after editing" indicates the billing party included in the edited invoice information. More specifically, "Name of billing party after editing" indicates the edited item value of the item "billing party" which is the primary key of the invoice information. In other words, the billing party of the edited invoice information is the specified information after editing.

[0079] The value of the "telephone number" item is the telephone number corresponding to the edited name of the biller, and the value of the "account number" item is the account number corresponding to the edited name of the biller. In other words, the values ​​of the "telephone number" and "account number" items can be said to be information that identifies the edited name of the biller. In other words, the values ​​of the "telephone number" and "account number" items are information that identifies the issuer of the edited form.

[0080] In this way, the management information stored in the management information database 265 of this embodiment corresponds information indicating the requesting source, information indicating the requesting source after editing, and information identifying the requesting source after editing.

[0081] Next, the functions of the server device 200 of this embodiment will be described with reference to Fig. 10. Fig. 10 is a diagram for explaining the functions of the server device.

[0082] In the server device 200 of this embodiment, the CPU 201 reads and executes an information processing program stored in the RAM 203, the HD 204, or the like, thereby realizing the functions of each unit described below.

[0083] The server device 200 of this embodiment has a form recognition unit 220, a form management unit 290, and a communication unit 298. The form recognition unit 220 has an input reception unit 221, an authentication processing unit 222, and a recognition processing unit 270. The form management unit 290 includes an operation reception unit 291, a display control unit 292, and an information update unit 293.

[0084] In the form recognition unit 220, the input acceptance unit 221 (first input acceptance unit) accepts various inputs to the server device 200. Specifically, the input acceptance unit 221 accepts the input of authentication information for logging in to the form recognition system 100. The authentication information is, for example, a user ID and a password. The input acceptance unit 221 also accepts the input of form image data transmitted from the image forming device 300. The input acceptance unit 221 also accepts various requests and notifications transmitted from the terminal device 400. The input acceptance unit 221 may accept inputs to the server device 200 via the communication unit 298, or may accept inputs made directly to the server device 200.

[0085] The authentication processing unit 222 performs authentication based on the authentication information input and accepted by the input accepting unit 221. Note that the authentication processing unit 222 may, for example, transmit the input authentication information to an authentication server provided outside the server device 200 and obtain the result of authentication by the authentication server. When the user is authenticated, the authentication processing unit 222 may pass the form image data accepted by the input accepting unit 221 to the recognition processing unit 270.

[0086] The recognition processing unit 270 of this embodiment has an image data storage unit 271, a job list generation unit 272, an image data acquisition unit 273, a preprocessing unit 274, a character recognition unit 275, a definition identification unit 276, an item extraction unit 277, a display screen generation unit 278, and an output unit 279.

[0087] When the input receiving unit 221 receives form image data from the image forming apparatus 300 via the communication unit 298 , the image data storage unit 271 stores the data in the form image database 230 .

[0088] When the input receiving unit 221 receives input of form image data from the image forming apparatus 300 via the communication unit 298, the job list generating unit 272 registers the job in the job list and manages the job list. Details of the job list will be described later.

[0089] The image data acquisition unit 273 acquires form image data stored in the form image database 230. The preprocessing unit 274 performs preprocessing on the form image data acquired by the image data acquisition unit 273 in order to perform form recognition.

[0090] The character recognition unit 275 extracts character strings and information indicating the positions where the character strings are arranged from the form image data, and stores the extracted data as read result data. The read result data will be described in detail later.

[0091] The definition specifying unit 276 is an acquisition unit that specifies the detail definition information to be referenced during form recognition based on the character string included in the read result data, and acquires the specified detail definition information from the form definition database 250.

[0092] The item extraction unit 277 extracts and associates items and item values ​​included in the form image data based on the read result data to generate recognition result data, and stores the data in the recognition result database 260 .

[0093] At this time, the item extraction unit 277 of this embodiment refers to the detail definition information identified by the definition identification unit 276, and extracts the detail item names written in the detail section of the form in association with the values ​​of the detail items.

[0094] In the following description, information that associates the detailed item names written in the detailed section of a form with the values ​​of the detailed items may be referred to as detailed information. The detailed information in this embodiment is information included in the recognition result data and is a part of the recognition result data.

[0095] Furthermore, the recognition result data in this embodiment includes information (issuer information) that indicates the issuer of the form, which is written on the form. In other words, the recognition result data includes information that is indicated by the character string written on the form.

[0096] The display screen generation unit 278 uses the recognition result data stored in the recognition result database 260 to generate a confirmation screen for the recognition result.

[0097] The output unit 279 outputs the confirmation screen of the recognition result generated by the display screen generation unit 278 via the communication unit 298. In other words, the output unit 279 causes the confirmation screen of the recognition result generated by the display screen generation unit 278 to be displayed on the web browser of the terminal device 400 via the communication unit 298.

[0098] The form management unit 290 of this embodiment includes an operation reception unit 291 , a display control unit 292 , and an information update unit 293 .

[0099] The operation acceptance unit 291 accepts an operation input to a screen displayed on the web browser of the terminal device 400 via the communication unit 298. More specifically, in this embodiment, the communication unit 298 receives information indicating an operation from the terminal device 400, and the operation acceptance unit 291 accepts input of the information received by the communication unit 298.

[0100] The display control unit 292 (first display control unit) instructs the terminal device 400 to display a screen via the communication unit 298. Specifically, the display control unit 292 causes the terminal device 400 to display an input screen for registering management information and an input screen for generating individual company definition information. The display control unit 292 also generates an XML format file for displaying the screen, and causes the input screen to be displayed on the web browser of the terminal device 400 via the communication unit 298.

[0101] The device on which the display control unit 292 displays the screen may be the server device 200 or a device other than the terminal device 400, and may be any display device having a display function.

[0102] The information update unit 293 generates individual company definition information based on the information input from the terminal device 400 and stores it in the form definition database 250. In addition, the information update unit 293 generates management information based on the information input from the terminal device 400 and stores it in the management information database 265.

[0103] The communication unit 298 of the present embodiment is a function realized by, for example, the network I / F 209 or the like, and transmits and receives data between the server device 200 and an external device. Specifically, for example, the communication unit 298 transmits a notification to the terminal device 400 instructing it to display a screen in accordance with control by the display control unit 292.

[0104] Next, the functions of the terminal device 400 of this embodiment will be described with reference to Fig. 11. Fig. 11 is a diagram for explaining the functions of the terminal device.

[0105] The functions of each unit described below are realized by the CPU 401 of the terminal device 400 reading and executing a program stored in the ROM 402 or the like.

[0106] The terminal device 400 of this embodiment includes an input receiving unit 431 , a display control unit 432 , and a communication unit 433 .

[0107] The input receiving unit 431 (second input receiving unit) receives input for a screen displayed on the web browser of the terminal device 400 by the display control unit 432. Specifically, the input receiving unit 431 receives an operation to edit the recognition result on the confirmation screen of the recognition result displayed on the web browser of the terminal device 400.

[0108] The display control unit 432 (second display control unit) controls the display on the display 418 of the terminal device 400. Specifically, the display control unit 432 causes the display 418 (display unit) to display a screen in accordance with an instruction from the server device 200.

[0109] The communication unit 433 is a function realized by the long-distance communication circuit 412, the antenna 412a of the long-distance communication circuit 412, etc., and transmits and receives data between the terminal device 400 and other devices.

[0110] The operation of the form recognition system 100 of this embodiment will be described below with reference to Fig. 12. Fig. 12 is a sequence diagram illustrating the operation of the form recognition system.

[0111] In the form recognition system 100, when the image forming apparatus 300 receives a request from a user to start an application for performing form recognition (step S1201), the image forming apparatus 300 starts the application (step S1202).

[0112] Next, upon receiving authentication information input from the user (step S1203), the image forming apparatus 300 transmits the authentication information to the server apparatus 200 and makes an authentication request (step S1204).

[0113] Upon receiving the authentication request, the server device 200 performs authentication using the authentication processing unit 222 and notifies the image forming device 300 of the result (step S1205). Note that the operation shown here is when the user is authenticated.

[0114] Next, the image forming apparatus 300 receives an instruction to scan the form (step S1206), scans the form, and acquires form image data (step S1207). Next, the image forming apparatus 300 transmits the form image data to the server apparatus 200 (step S1208).

[0115] In the server device 200, when the input receiving unit 221 receives input of form image data, the authentication processing unit 222 passes this form image data to the recognition processing unit 270 (step S1209). Note that in the example of Fig. 12, the form image data is passed to the recognition processing unit 270 via the authentication processing unit 222, but this is not limiting. The form image data may also be passed to the recognition processing unit 270 without passing through the authentication processing unit 222.

[0116] When the recognition processing unit 270 of the server device 200 receives the form image data, the job list generating unit 272 registers the job in the job list (step S1210). Next, the recognition processing unit 270 stores the form image data in the form image database 230 using the image data storing unit 271 (step S1211).

[0117] Next, the recognition processing unit 270 acquires form image data to be recognized from the form image database 230 using the image data acquisition unit 273 (step S1212), and executes processing to recognize the form image (step S1213). Details of the processing in step S1212 will be described later.

[0118] Next, the recognition processing unit 270 stores the recognition result data generated by the item extraction unit 277 in the recognition result database 260 (step S1214).

[0119] In the document recognition system 100, when the terminal device 400 receives a browser display instruction input from a user via the input receiving unit 431 (step S1215), it sends a browser display request to the server device 200 via the communication unit 433 to display the recognition result data (step S1216).

[0120] Furthermore, the terminal device 400 receives authentication information input from the user through the input receiving unit 431 (step S1217), and transmits this authentication information to the server device 200 through the communication unit 433 (step S1218).

[0121] The server device 200 authenticates the user using the authentication processing unit 222 (step S1219), notifies the terminal device 400 of the authentication result, and causes the terminal device 400 to display a list of jobs (step S1220). Note that the process shown here is for when the user is authenticated.

[0122] Next, when the terminal device 400 receives a job selection from the job list via the input receiving unit 431 (step S1221), the terminal device 400 sends a viewing request to the server device 200 via the communication unit 433, regarding this selection as an instruction to view the recognition results of the form image (step S1222).

[0123] Upon receiving the viewing request, the server device 200 causes the recognition processing unit 270 to obtain the recognition result data stored in the recognition result database 260 (step S1223), generates a confirmation screen for the recognition result and displays it on the browser of the terminal device 400 (step S1224), and allows the user to confirm the recognition result on the displayed confirmation screen for the recognition result.

[0124] Next, when the terminal device 400 receives an input of an instruction to register the individual company definition information and management information through the input receiving unit 431 (step S1225), it transmits a registration request for the individual company definition information and management information to the server device 200 through the communication unit 433 (step S1226). In other words, this registration request is a display request for an input screen for inputting the individual company definition information and management information.

[0125] When the input receiving unit 221 of the server device 200 receives this display request, the display control unit 292 of the form management unit 290 causes the terminal device 400 to display an input screen for the individual company definition information and management information (step S1227).

[0126] Next, when the terminal device 400 accepts input of the individual company definition information and management information (step S1228), it transmits the input information to the server device 200 (step S1229). The form management unit 290 in the server device 200 generates the individual company definition information and management information using the information updated by the information transmitted from the terminal device 400 (step S1230), stores them in the form definition database 250 and the management information database 265 (step S1231), and terminates the process.

[0127] Here, a job list generated by the job list generating unit 272 of this embodiment will be described with reference to Fig. 13. Fig. 13 is a diagram showing an example of a job list.

[0128] When the server device 200 of this embodiment receives form image data from the image forming device 300, the job list generating unit 272 stores the form image data and the job ID as a job list in association with each other.

[0129] The job list 101 of this embodiment has the following information items: job ID, user ID, form image file path, recognition result file path, and status.

[0130] The value of the item "Job ID" is identification information for identifying a job. In other words, the value of the item "Job ID" is identification information for identifying the form image data received from the image forming apparatus 300.

[0131] The value of the item “user ID” is identification information for identifying the user who has logged in to the document recognition system 100 .

[0132] The value of the item "Image file path" is information indicating the location where the form image data is stored. The value of the item "Recognition result file path" is information indicating the location where the recognition result data resulting from recognizing the form image is stored.

[0133] The value of the item "status" indicates the progress of recognition of the form image, in other words, the value of the item "status" indicates the state of the job.

[0134] In this embodiment, there are four values ​​for the item "status": unprocessed state, draft saved state, confirmed state, and externally output state. Each state is as follows.

[0135] Unprocessed state (state 1): This is the state immediately after the recognition result data has been obtained by form recognition, and the recognition results have not yet been confirmed.

[0136] Draft saved state (state 2) This is the state when the recognition result data is still being checked, and the recognition result data has not yet been finalized.

[0137] Confirmed state (state 3): The state in which the work of checking the recognition result data has been completed and the data has been stored in the recognition result database 260 as confirmed information.

[0138] External output completed state (state 4): A state in which the output of the recognition result data to the core system 500 linked to the document recognition system 100 has been completed.

[0139] When the server device 200 of this embodiment receives form image data from the image forming device 300, the job list generation unit 272 assigns a job ID to the form image data, associates it with the user ID acquired as authentication information, and adds a record to the job list 101. Then, when the form image data is stored in the form image database 230, the job list generation unit 272 adds information indicating the storage location as the value of the item "image file path."

[0140] Furthermore, when the recognition result data is stored in the recognition result database 260, the job list generation unit 272 adds information indicating the storage location as the value of the item "recognition result file path" and sets the value of the item "status" to "confirmed."

[0141] In this embodiment, for example, a job list generated by the job list generation unit 272 may be displayed. In this case, the user can check the progress of recognition of the form image via the job list screen displayed based on the job list.

[0142] Next, the processing of the recognition processing unit 270 of this embodiment will be described with reference to Fig. 14. Fig. 14 is a first flowchart illustrating the processing of the form recognition unit. Fig. 14 shows details of the processing of step S1213 in Fig. 12. Fig. 14 also shows the processing when multiple form image data are received from the image forming device 300.

[0143] In this embodiment, the recognition processing unit 270 acquires form image data from the form image database 230 using the image data acquisition unit 273 (step S1401), and then converts the form image data format into png format using the preprocessing unit 274 (step S1402). Next, the preprocessing unit 274 selects one page of form image data (step S1403) and performs preprocessing (step S1404). Specifically, the preprocessing is, for example, correction processing for aligning the orientation of the form image.

[0144] Next, the recognition processing unit 270 cuts out an area where characters are formed (hereinafter, a character field) from the form image using the character recognition unit 275 (step S1405). Next, the recognition processing unit 270 cuts out the characters in the cut-out character field using the character recognition unit 275 and performs character recognition (step S1406). At this time, the character recognition unit 275 also acquires coordinates indicating the position of the cut-out characters.

[0145] Next, the recognition processing unit 270 generates and stores read result data based on the positional relationship of each character recognized by the character recognition unit 275 (step S1407). Specifically, the recognition processing unit 270 recognizes characters whose mutual distance is within a preset threshold as one character string based on the coordinates of each recognized character, and generates read result data by associating the character string with information such as coordinates indicating the position of the area where the character string is formed. Details of the read result data will be described later.

[0146] Next, the recognition processing unit 270 extracts items and item values ​​using the individual company definition information 252 identified based on the read result data, the type determination database 240, and the management information database 265, using the item extraction unit 277 (step S1408).

[0147] Next, the recognition processing unit 270 extracts items and item values ​​using the specific definition information 253 identified based on the read result data and the type determination database 240 by the item extraction unit 277 (step S1409).

[0148] Next, the recognition processing unit 270 extracts items and item values ​​using the general definition information 251 identified based on the read result data and the type determination database 240 by the item extraction unit 277 (step S1410). Details of each process from step S1408 to step S1410 will be described later.

[0149] Next, the recognition processing unit 270 determines whether or not the extraction of the items and their values ​​has been successful (step S1411). Specifically, the recognition processing unit 270 may determine that the extraction of the items and their values ​​has been successful if the items and their values ​​extracted by the item extraction unit 277 exist between steps S1408 and S1410.

[0150] If extraction of the item and its value fails in step S1411, the recognition processing unit 270 notifies the terminal device 400 or the image forming device 300 of an error (step S1412), and the process proceeds to step S1415, which will be described later.

[0151] If the extraction of the items and their values ​​is successful in step S1411, the recognition processing unit 270 generates recognition result data by associating and merging the extracted items and their values ​​with each other using the item extraction unit 277 (step S1413). The merging process will be described in detail later.

[0152] Next, the recognition processing unit 270 causes the job list generating unit 272 to register the definition ID of the form definition information used to generate the recognition result data and the storage destination of the recognition result data in the job list 101 (step S1414).

[0153] Next, the recognition processing unit 270 determines whether or not the process has been executed up to the last page (step S1415). If the process has not been executed up to the last page in step S1415, the recognition processing unit 270 returns to step S1403.

[0154] In step S1415, if processing has been performed up to the last page, the recognition processing unit 270 causes the display screen generation unit 278 to display on the terminal device 400 a recognition result confirmation screen generated using the recognition result data (step S1416), and terminates processing.

[0155] The recognition processing unit 270 may display the recognition result data after receiving a request to view the recognition result data from the terminal device 400.

[0156] Next, the processing of the item extraction unit 277 of this embodiment will be described with reference to Fig. 15. Fig. 15 is a second flowchart illustrating the processing of the form recognition unit. Fig. 15 shows details of the processing for identifying each definition information performed in steps S1408 to S1410 of Fig. 14.

[0157] When the recognition processing unit 270 of this embodiment stores the read result data, the definition specifying unit 276 extracts the definition ID corresponding to the identification character string included in the read result data from the type determination database 240 (step S1501).

[0158] Next, the definition identification unit 276 determines whether or not there is a definition ID whose definition type is individual company definition information among the extracted definition IDs (step S1502). That is, here, it is determined whether or not individual company definition information 252 is registered for the type of form indicated by the identification character string included in the read result data.

[0159] In step S1502, if a corresponding definition ID does not exist, this type of report does not have registered individual company definition information 252 that defines a user-specific layout, so the item extraction unit 277 terminates the process and proceeds to step S1409 in Figure 14.

[0160] In step S1402, if a corresponding definition ID exists, the definition identification unit 276 refers to the form definition database 250 and identifies the individual company definition information 252 including the definition ID (step S1503). If a corresponding definition ID exists, this means that the form has a layout unique to the user. The details of step S1503 will be described later.

[0161] Next, the item extraction unit 277 extracts and associates items and item values ​​from the read result data based on the identified individual company definition information (step S1504).

[0162] Note that the process of extracting items and item values ​​using specific definition information in step S1409 and the process of extracting items and item values ​​using general definition information 251 in step S1410 are similar except that the individual company definition information in step S1408 becomes specific definition information or general definition information, so explanations will be omitted.

[0163] Next, the process of identifying individual company definition information will be described with reference to Fig. 16. Fig. 16 is a third flowchart illustrating the process of the form recognition unit. Fig. 16 shows details of the process of the definition identification unit 276 in step S1503 of Fig. 15. More specifically, Fig. 16 shows the process of identifying individual company definition information.

[0164] The definition specifying unit 276 of this embodiment selects the recognition result data that has been determined from the recognition result data stored in the recognition result database 260 (step S1601).

[0165] Next, the definition specifying unit 276 compares the selected recognition result data with the read result data stored in step S1407 of FIG. 14, and determines whether the character matching rate is equal to or greater than a predetermined threshold value (step S1602).

[0166] In step S1602, if the character matching rate is less than the predetermined threshold, the definition specifying unit 276 proceeds to step S1609, which will be described later.

[0167] In step S1602, if the character matching rate is equal to or greater than a predetermined threshold, definition identification unit 276 determines whether or not the value of the item "billing source" in the read result data is included in the item "name of billing source after editing" in management information database 265 (step S1603). In step S1603, if the value of the item "billing source" is not included in the item "name of billing source" in management information database 265, definition identification unit 276 proceeds to step S1609, which will be described later.

[0168] In step S1603, if the value of the item "billing source" is included in the item "name of billing source" of the management information database 265, the definition identification unit 276 identifies management information in which the value of the item "name of billing source" matches the value of the item "billing source" (step S1604).

[0169] Next, the definition specifying unit 276 determines whether or not there is an item among the items included in the read result data whose value matches the value of the item included in the specified management information (step S1605). Note that in step S1605, the value of the item does not have to match the value of the item included in the specified management information completely, and it is determined that there is a match if the value of the item matches part of the value of the item included in the specified management information. If there is no corresponding item in step S1605, the definition specifying unit 276 proceeds to step S1609, which will be described later.

[0170] If a corresponding item is found in step S1605, it is determined whether the corresponding item identifies a single billing source (step S1606).

[0171] In step S1606, if one request source is identified, the definition identification unit 276 stores the identified management information in a list as management information to be referenced when identifying the request source (step S1607), and proceeds to step S1609. Note that this list may store only the management information ID included in the management information.

[0172] In step S1606, if the billing source is not identified as one, the definition identifying unit 276 selects management information to be registered in the list (step S1608), and the process proceeds to step S1607.

[0173] Specifically, the definition specifying unit 276 selects, from the specified management information, management information that contains character strings that match the most with character strings contained in the read result data.

[0174] After storing the management information in the list, the definition specifying unit 276 determines whether or not the processes up to step S1608 have been performed for all the recognition result data that have been determined in the recognition result database 260 (step S1609).

[0175] In step S1609, if the processing has not been performed for all the recognition result data in the confirmed state, the definition specifying unit 276 returns to step S1601.

[0176] In step S1609, when all the recognition result data in the confirmed state has been processed, the definition specifying unit 276 refers to the management information stored in the list and specifies the request source (step S1610).

[0177] Specifically, the definition identification unit 276 identifies the management information stored in the list whose character strings match the most with the character strings in the read result data, and identifies the name of the requester of the identified management information as the requester.

[0178] Next, the definition specifying unit 276 specifies the individual company definition information corresponding to the specified billing source as the individual company definition information to be used for form recognition (step S1611), and ends the process.

[0179] In this embodiment, the billing source is identified in this manner using the management information stored in the management information database 265. Therefore, according to this embodiment, the billing source can be identified with high accuracy.

[0180] Furthermore, in this embodiment, since the billing source can be identified with high accuracy, appropriate individual company definition information can be identified, and the accuracy of document recognition can be improved.

[0181] Next, the registration of management information by the form management unit 290 will be described with reference to Fig. 17. Fig. 17 is a flowchart illustrating the processing of the form management unit. Fig. 17 shows details of the processing from step S1221 to step S1231 in Fig. 12.

[0182] In this embodiment, when the operation receiving unit 291 receives an instruction to view the recognition result data from the terminal device 400, the form management unit 290 causes the display control unit 292 to display a confirmation screen for the recognition result data on the terminal device 400 (step S1701).

[0183] Next, the form management unit 290 determines whether or not a request to register the recognition result data has been received from the terminal device 400 via the operation receiving unit 291 (step S1702). Specifically, the server device 200 determines whether or not an instruction has been given to store the recognition result data displayed on the confirmation screen in the recognition result database 260.

[0184] If the registration request is not accepted in step S1702, the form management unit 290 proceeds to step S1707, which will be described later.

[0185] If a registration request is accepted in step S1702, the form management unit 290 determines whether or not the recognition result data has been corrected (step S1703).

[0186] If no correction has been made in step S1703, the form management unit 290 ends the process.

[0187] If corrections have been made in step S1703, the form management unit 290 determines whether or not the individual company definition information 252 has been referenced when generating this recognition result data (step S1704). Specifically, the form management unit 290 determines whether or not there is individual company definition information 252 associated with the billing source included in the read result data.

[0188] If the individual company definition information is not referenced in step S1704, the form management unit 290 proceeds to step S1708, which will be described later.

[0189] In step S1704, if the individual company definition information is referenced, the form management unit 290 determines whether the corrected part is the billing source (step S1705).

[0190] In step S1705, if the corrected part is not the billing source, the form management unit 290 causes the information update unit 293 to update the individual company definition information in accordance with the correction content (step S1706), and ends the process. The case where the corrected part is not the billing source is, for example, when the detail part in the recognition result data is corrected.

[0191] In this embodiment, the recognition result data may be registered after the process is completed by the form management unit 290. Specifically, the form management unit 290 may store the recognition result data in the recognition result database 260 after the process of FIG.

[0192] In step S1705, if the corrected (edited) part is the billing source, the form management unit 290 causes the display control unit 292 to display an input screen for management information on the terminal device 400 (step S1707).

[0193] Next, the form management unit 290 causes the operation reception unit 291 to receive information input on the input screen displayed on the terminal device 400 (step S1708).

[0194] Next, the form management unit 290 causes the information update unit 293 to update the management information database 265 (step S1709), and ends the process.

[0195] Specifically, the information update unit 293 generates management information by associating the information entered on the input screen with the billing source corrected on the confirmation screen of the recognition result data, and stores the generated management information in the management information database 265.

[0196] The billing source corrected on the confirmation screen is, in other words, the issuer after editing, and is the predetermined information after editing.

[0197] If a request to register the recognition result is not received in step S1702, the form management unit 290 determines whether an instruction to register a form definition has been received by the operation receiving unit 291 (step S1710).

[0198] If the instruction to register the form definition is not accepted in step S1710, the form management unit 290 returns to step S1701.

[0199] In step S1701, when an instruction to register a form definition is accepted, the form management unit 290 causes the display control unit 292 to display an input screen for the individual company definition information 252 on the terminal device 400 (step S1711).

[0200] Next, the form management unit 290 receives input of the individual company definition information on the input screen through the operation receiving unit 291 (step S1712).

[0201] Next, the form management unit 290 causes the information update unit 293 to store the input individual company definition information 252 in the form definition database 250 (step S1713), and ends the process.

[0202] 18 to 20, examples of display on the terminal device 400 will be described below. Fig. 18 is a first diagram showing an example of display on the terminal device.

[0203] A screen 501 shown in FIG. 18 is an example of a confirmation screen for the recognition result displayed on the terminal device 400 in step S1224 of FIG.

[0204] Screen 501 includes display fields 502 and 503, and display components 508 and 509. A form image indicated by form image data is displayed in display field 502. In the example of Fig. 18, the form is an invoice, and includes area 504 indicating the details section of the form image.

[0205] 20, the detail portion in the form image is expressed in a table format. In other words, the detail portion in the form image is an example of a table format image.

[0206] A display field 503 displays the recognition result data obtained by performing form recognition on the form image data displayed in the display field 502 by the form recognition unit 220 .

[0207] Display column 503 includes display columns 505, 506, and 507. Display column 505 displays the value of the item "status" in job list 101. Display column 506 displays billing source information including the billing source. The billing source information is part of the recognition result data.

[0208] Furthermore, display field 506 includes display field 506a. Display field 506a displays the name of the billing party, "XX Trading Co., Ltd." The name of the billing party displayed in display field 506a shown in Fig. 18 is the name of the billing party extracted from the form image data displayed in display field 502. In other words, display field 506a shown in Fig. 18 displays the name of the billing party before editing.

[0209] Detailed information is displayed in display field 507. Specifically, a list of values ​​of detailed items included in the detailed information is displayed in display field 507. Display field 507 includes display fields 511, 512, 513, 514, 515, and 516. Display field 507 also includes display component 510.

[0210] The display component 510 is a display component for instructing the display of an extraction guide image. In other words, the display component 510 is a display component for notifying the server device 200 of an instruction to set areas for item names and item values ​​to be extracted from a tabular image.

[0211] Display field 511 displays the value of the detail item with the detail item name "item name" displayed in area 504 showing the detail details in the form image. Display field 512 displays the value of the detail item with the detail item name "item code" displayed in area 504. Display field 513 displays the value of the detail item with the detail item name "item price" displayed in area 504.

[0212] Display field 514 displays the value of the detail item with the detail item name "unit price" displayed in area 504. Display field 515 displays the value of the detail item with the detail item name "quantity" displayed in area 504. Display field 516 displays the value of the detail item with the detail item name "order number" displayed in area 504. In this embodiment, item values ​​are extracted from items corresponding to the detailed item names in the detailed section of the form image and displayed in display fields 511, 512, 513, 514, 515, and 516. In other words, the items corresponding to these detailed item names are the items to be extracted (extraction target items). The extraction target items may be fixed items or may be changeable by the user.

[0213] 18 shows a case where the extraction of recognition result data from the form image data is successful. Therefore, the values ​​of all the detail item names in the detail section displayed in area 504 are displayed in display field 507.

[0214] The display component 508 is a display component for changing the status value of the job list 101 from an unprocessed state to a draft saved state. When the display component 508 is operated, the display field 505 changes from "unprocessed" to "draft saved."

[0215] The display component 509 is a display component for setting the status value of the job list 101 to the confirmed state. In this embodiment, when the display component 509 is operated, the status value of the job list 101 is set to the confirmed state, and the individual company definition information 252 is not generated.

[0216] Next, a case where the billing source is edited will be described with reference to Fig. 19. Fig. 19 is a second diagram showing an example of a display on the terminal device.

[0217] 19, in addition to the name of the billing party before editing, "XX Trading Co., Ltd.", "XX Branch" has been entered in display field 506a. In other words, in screen 501A shown in Fig. 19, the name of the billing party has been edited, and the edited name of the billing party is displayed in display field 506a.

[0218] In this embodiment, when display component 509 is operated on screen 501A shown in FIG. 19, an input screen is displayed for entering information related to the name of the billing party before editing, which is displayed in display field 506a of FIG. 18.

[0219] The input screen will be described with reference to Fig. 20. Fig. 20 is a third diagram showing an example of a display on the terminal device.

[0220] 20 displays an input screen 520. The input screen 520 includes input fields 521, 522, and 523, a display field 524, and a display component 525.

[0221] Input field 521 is an input field for entering the name of the biller, input field 522 is an input field for entering the telephone number of the biller, and input field 523 is an input field for entering the account number of the biller.

[0222] In other words, the name of the billing source before editing is input into input field 521, and information for identifying the billing source after editing is input into input fields 522 and 523.

[0223] Display field 524 displays precautions to take when entering information in input fields 521, 522, and 523. Specifically, display field 524 displays a message urging the user to enter only the corporate name or individual name, without entering the branch name, department name, etc., when entering the biller in input field 521. Display field 524 also displays a message urging the user to enter the telephone number and account number included in the form image displayed in display field 502 in input fields 522 and 523.

[0224] The display component 525 is operated after input into the input fields 521 to 523 is completed. In this embodiment, when input into the input fields 521 to 523 is completed and the display component 525 is operated, the terminal device 400 transmits the information input on the input screen 520 to the server device 200.

[0225] When the server device 200 receives the information entered on the input screen, it associates this information with the edited billing source displayed in the display field 506 a and stores it in the management information database 265 as management information.

[0226] In the example of Figure 20, the edited billing source "XX Branch of XX Trading Co., Ltd." displayed in display field 506a, the unedited billing source "XX Trading Co., Ltd." entered in input field 521, the telephone number entered in input field 522, and the account number entered in input field 523 are associated with each other and treated as management information.

[0227] In this way, in this embodiment, when the billing source is edited in the recognition result data, an input screen for inputting information for associating the edited billing source with the billing source before editing is displayed on the terminal device 400. Then, in this embodiment, management information that associates the input information with the edited billing source is generated and stored in the management information database 265, so that the edited billing source and the unedited billing source can be managed as separate items.

[0228] Therefore, according to this embodiment, even if the issuer of a document extracted during document recognition is edited by the user, by referring to the management information database 265, the document information can be managed efficiently in the same way as before the issuer was edited. The following describes a case where the core system 500 is an accounting system and a user of the document recognition system 100 uses the recognition result data in the accounting system, and edits the biller (company name) to add strings such as the biller's branch name, the billee's branch name, and the purpose of the recognition result data. In this case, the character string added by editing is not recorded as a series of characters with the billing source on the invoice (document), and is not associated with the billing source. Therefore, if the edited billing source is registered as the billing source in the individual company definition information, the recognition accuracy of the billing source in the document recognition may decrease. In this embodiment, when editing is performed to set a character string that is not associated with a billing source on the form as the billing source, the billing source on the form and the edited billing source are each treated as separate items and are managed in association with each other. Therefore, in this embodiment, even if such editing is performed, it is possible to prevent the management of the recognition result data in the document recognition system 100 from becoming complicated. In other words, in this embodiment, even if the user of the document recognition system 100 edits the description of the billing source in different ways, it is possible to prevent the management of document information from becoming complicated.

[0229] Furthermore, in this embodiment, when performing form recognition, the individual company definition information to be used is identified by referring to the management information database 265. Therefore, according to this embodiment, it is possible to improve the accuracy of form recognition.

[0230] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to perform each function described above.

[0231] Additionally, the devices described in the embodiments are merely representative of one of several computing environments for implementing the embodiments disclosed herein.

[0232] In one embodiment, server apparatus 200 includes multiple computing devices, such as a server cluster, configured to communicate with each other over any type of communications link, including a network, shared memory, etc., to perform the processes disclosed herein. Similarly, server apparatus 200 may include multiple computing devices configured to communicate with each other.

[0233] Furthermore, server device 200 can be configured to share the disclosed processing steps in various combinations. For example, processes performed by server device 200 can be performed by other server devices. Similarly, functions of server device 200 can be performed by other server devices. Furthermore, the server device and the elements of the other server devices can be integrated into a single server device or can be separated into multiple devices.

[0234] The database in the statement may be generated by the learning effect of machine learning. Also, by using machine learning to classify keywords that may be included in the description of transaction details and account items, it is possible to eliminate the need for a correspondence table.

[0235] Here, machine learning is a technology that allows a computer to acquire human-like learning capabilities, in which the computer autonomously generates algorithms necessary for judgments such as data classification from learning data that is input in advance, and applies these to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods; any learning method for machine learning is acceptable.

[0236] Although the present invention has been described above based on the embodiments, the present invention is not limited to the requirements shown in the above embodiments. These requirements can be changed without departing from the spirit of the present invention, and can be appropriately determined depending on the application form. [Explanation of symbols]

[0237] 100 Document Recognition System 200 Server device 200 210 Storage section 220 Document Recognition Unit 230 Form Image Database 240 Species Identification Database 250 Report Definition Database 260 Recognition Result Database 265 Management Information Database 270 Recognition processing section 290 Document Management Department 292 Display control unit 293 Information Update Department 300 Image forming device 400 Terminal Equipment 500 Core Systems [Prior art documents] [Patent documents]

[0238] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-158030

Claims

1. A display control unit that, when editing is performed on predetermined information included in form information extracted from form image data, displays on a terminal device an input screen for information related to the edited predetermined information; an information processing device having a storage unit that stores management information that associates information input on the input screen, the predetermined information after editing, and the predetermined information before editing.

2. The information related to the predetermined information is The information processing apparatus according to claim 1 , further comprising identification information for identifying the predetermined information after editing, and the predetermined information.

3. The information processing apparatus according to claim 1 , wherein the input screen is displayed together with the form image data.

4. The predetermined information is The information processing apparatus according to claim 1 , wherein the form information is information indicating an issuer of a form shown in a form image including the form information.

5. form definition information that defines the positional relationship between the items and values ​​of the items of a form specific to a user is stored in the storage unit; The information processing apparatus according to claim 1 , further comprising: a definition specifying unit that specifies the form definition information to be referenced when extracting form information from a form image, using management information stored in the storage unit.

6. A document management unit that determines whether the modified part in the document information is the specified information, The display control unit The information processing apparatus according to claim 3 , wherein the input screen is displayed when the corrected portion is predetermined information.

7. A document management unit that determines whether or not the modified part of the document information is the specified information, determining whether or not the form definition information was referenced when the form information was extracted from the form image data; The display control unit 6. The information processing apparatus according to claim 5, wherein the input screen is displayed when the form definition information is referenced and the corrected portion is the predetermined information.

8. An information processing device as described in claim 1, wherein the document image data indicates a document image including the document information.

9. An information processing method by an information processing device, the information processing device comprising: When editing is performed on predetermined information included in the form information extracted from the form image data, an input screen for inputting information related to the edited predetermined information is displayed on the terminal device; An information processing method, comprising: storing management information in a storage unit that associates the information input on the input screen, the predetermined information after editing, and the predetermined information before editing.

10. The information related to the predetermined information is 10. The information processing method according to claim 9, further comprising the step of: providing identification information for identifying the predetermined information after editing; and providing the predetermined information.

11. 11. The information processing method according to claim 9, wherein the input screen is displayed together with form image data showing a form image including the form information.

12. The predetermined information is The information processing method according to claim 11 , wherein the form information is information indicating an issuer of a form indicated by a form image including the form information.

13. form definition information that defines the positional relationship between the items and values ​​of the items of a form specific to a user is stored in the storage unit; 13. The information processing method according to claim 9, further comprising: identifying the form definition information to be referenced when extracting form information from a form image, using management information stored in the storage unit.

14. Determining whether the corrected portion in the form information is the specified information; The information processing method according to claim 11, wherein the input screen is displayed when the corrected portion is predetermined information.

15. A determination is made as to whether or not the form definition information was referenced when the form information was extracted from the form image data; 14. The information processing method according to claim 13, wherein the input screen is displayed when the form definition information is referenced and the corrected portion is the predetermined information.

16. An information processing method as described in claim 9, wherein the document image data indicates a document image including the document information.

17. When editing is performed on predetermined information included in form information extracted from form image data, an input screen for information related to the edited predetermined information is displayed on the terminal device, A program that causes an information processing device to execute a process of storing management information in a storage unit that associates the information input on the input screen, the specified information after editing, and the specified information before editing.

18. An information processing system in which a terminal device and an information processing device are connected via a network, The information processing device includes: a first input receiving unit that receives a display request transmitted from the terminal device when predetermined information included in the form information extracted from the form image data is edited in the terminal device; a first display control unit that receives the display request and causes the terminal device to display an input screen for information related to the edited predetermined information; a storage unit for storing management information that associates the information input on the input screen, the predetermined information after editing, and the predetermined information before editing, The terminal device a second input receiving unit that receives edits to the form information; and a second display control unit that causes the input screen to be displayed on a display unit.

Images