Data output system, information processing system, information processing method, program

Abstract

To provide a data output system that allows a user to associate marks and processing with each other.SOLUTION: The present invention is an information processing system that is communicably connected with one or more devices and executes predetermined processing, and has: a data storage unit that, based on a user's operation in a first device, stores setting information in which identification information for identifying the details of the processing and the predetermined processing are associated with each other; an area specification unit that specifies an area including the identification information from image data received from a second device; and a processing unit that executes the predetermined processing based on the area specified by the area specification unit and the setting information.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a data output system, an information processing system, Information a processing method, and a program.

Background Art

[0002] In the daily operations of organizations such as companies and organizations, there are often cases where documents such as expense settlement statements, invoices, or orders need to be processed. Persons in charge within the organization perform daily tasks such as extracting or aggregating handling dates, project numbers, or amounts from documents.

[0003]

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the conventional technology has a problem that the user cannot associate a mark with a process. For example, if the user cannot register the correspondence between a mark and a process by attaching a mark to a predetermined sheet of paper with a marker pen or the like, the person in charge needs to set the correspondence between the mark and the process in a file or the like.

[0005] In view of the above problems, an object of the present invention is to enable a user to associate identification information added to a predetermined sheet of paper with a process. Technology

Means for Solving the Problems

[0006] In view of the above problems, the present invention is an information processing system that is communicably connected to one or more devices and executes a predetermined process, and based on a user operation in a first device, a data storage unit that stores setting information associating a mark that identifies the content of the process with the predetermined process, A communication unit that receives second image data of a predetermined document including the mark added by the user from the first device, a region detection unit that detects a region including the mark from the second image data, position information of the detected region, a predetermined process corresponding to the mark included in the region, and an application selected by the user in the first device, and a management unit that stores them in association with each other, and based on the position information and the predetermined process associated with the application selected by the second device for the third image data received from the second device and a processing unit that executes a predetermined process.

Advantages of the Invention

[0007] It is possible to provide an information processing system capable of associating identification information added by a user to a predetermined sheet with a process.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37A

Figure 37B

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47

Figure 48

Figure 49

Figure 50

Figure 51

Figure 52

Figure 53

Figure 54

Figure 55

Figure 56

Figure 57

Figure 58

Figure 59A

Figure 59B

Embodiments for Carrying Out the Invention

[0009] Hereinafter, as an example of embodiments for carrying out the present invention, a data output system and a processing method performed by the data output system will be described.

Examples

[0010] <Outline of Processing or Operation> In organizations such as companies and organizations, processing of forms such as expense settlement documents, invoices, or order forms often occurs. Persons in charge within the organization daily perform tasks such as extracting, aggregating, etc. dates, project numbers, or amounts from forms. After the processed forms are filed, they are stored, but the storage space also incurs costs. Although the digitization of forms is gradually progressing, tasks such as the person in charge renaming the file name automatically assigned to the form file also take time.

[0011] There are systems and services for automating claim processing and order placement and reception, but many of them are expensive, so there are not many small and medium-sized enterprises that can invest in them. The forms used vary depending on the organization, and even for the same form, the items used for processing may differ depending on the organization. For this reason, there is currently a situation where the processing of these forms has to be carried out manually.

[0012] In this embodiment, the information processing system receives, from a device, image data in which a predetermined mark (such as the color or shape of a marker) and the process corresponding to the mark are associated. As a result, the user side associates the mark with the process. Further, the information processing system receives, from a device, image data of a form on which the mark is formed. The information processing system associates the mark, the process, and the area information in the form with a setting file. When the form is scanned by the device, the information processing system executes the process associated with the area of the form specified by the area information.

[0013] As a result, the information processing system can perform various processes specified by the user only on a specific area on the document.

[0014] With reference to FIGS. 1 and 2, an outline of the process or operation performed by the data output system will be described. FIG. 1 is a diagram for explaining the process or operation in which the data output system associates a mark with a process. FIG. 2 is a diagram for explaining the process or operation in which the data output system 100 creates a file from the acquired data obtained from the form.

[0015] (1) The data output system 100 can cooperate with the external service system 40. The data output system 100 cooperates with the external service system 40 via an application. The application operating on the data output system 100 may be simply referred to as the "application", and the application operating on the external service system 40 may be referred to as the cooperation destination application. In this case, the application IDs of the application and the cooperation destination application are associated with each other, and the data output system 100 stores the association relationship.

[0016] (2) The administrator 8 of the data output system 100 prints a registration form registered in the information processing system 10 on the device 20 and colors with a marker pen the color corresponding to the process. For example, the administrator 8 can associate an arbitrary color with the process, such as green for OCR, blue for OMR, red for trimming, etc. The device 20 scans the registration form to generate image data (first image data) and transmits it to the information processing system 10.

[0017] (3) The information processing system 10 detects the correspondence between colors and processes from the image data, associates the colors with the processes, and registers them in the configuration file. The information processing system 10 stores the configuration file in the external service system 40.

[0018] Subsequently, a description will be given with reference to FIG. 2. FIG. 2(a) is a diagram for explaining the setting of area information in the form, and FIG. 2(b) is a diagram for explaining the acquisition of data from the form using the area information.

[0019] (4) The administrator 8 uses the form 7 that is routinely used by users within the organization, and uses a marker pen to specify which process (OCR: Optical Character Reader, OMR: Optical Mark Reader, trimming) is to be performed on which area of the form 7. For example, the administrator 8 may fill in or surround the area of the form 7 with a marker pen. The device 20 reads the form 7 with the marked area and transmits the generated image data (second image data) to the information processing system 10. Note that the form 7 to be marked may have characters or the like entered for the items, or may only have the items, or may be blank.

[0020] (5) The information processing system 10 identifies, from the image data of the form, the areas colored with the same color as the color registered in the configuration file acquired from the external service system 40, associates the area information of the area with the color and process in the configuration file, and stores it in the configuration file. The area information is, for example, the coordinates of the diagonal vertices of a rectangular area in the image data. The information processing system 10 stores the configuration file in the external service system 40.

[0021] (6)-1 The administrator 8 connects the terminal device 30 to the external service system 40, downloads the configuration file, inputs the language, date and time, etc. of the form into the configuration file, and complements the configuration file.

[0022] (6)-2 Also, the administrator 8 connects the terminal device 30 to the information processing system 10 and performs an operation of associating the app ID of the app with the setting file. The administrator 8 can associate any number of app IDs with the setting files. As a result, the icon that accepts the selection of the app is associated with the area information where the image processing is performed.

[0023] (7) When a user 6 within the organization submits a form 7 to the organization for expense settlement or the like, the user presses the icon of the app corresponding to this form 7 on the device 20. The user can easily select an app from a plurality of form names or the like. The user causes the device 20 to read a handwritten form or a printed form with the necessary items. The device 20 transmits the app ID of the app and the image data (third image data. For example, a PDF file) generated by reading the form to the information processing system 10.

[0024] (8) The information processing system 10 specifies the setting file by the app ID and acquires it from the external service system 40. The information processing system 10 acquires (for example, cuts out) the area of the area information included in the acquired setting file from the image data for each process. Therefore, the area of the form registered by the administrator 8 is acquired.

[0025] (9) The information processing system 10 performs the process associated with the area on each area. Therefore, for each area, character codes such as characters, presence or absence of checks, or image data, etc. are obtained. Hereinafter, these are simply referred to as acquired data.

[0026] (10) The information processing system 10 outputs the acquired data to an electronic file (for example, a CSV (Comma Separated Value) file) and transmits the file and the form image data to the external service system 40. When the information processing system 10 designates the app ID of the linked app, the file and the image data can be transmitted by designating the linked app.

[0027] (11) The external service system 40 registers image data and files in the folder (or directory) designated by the administrator 8. Since the file contains information necessary for settlement, the person in charge can perform the settlement process.

[0028] As described above, in the data output system of this embodiment, the administrator 8 can register the mark (color) and the content of the process. Since the device 20 reads the form with the area marked and the information processing system 10 acquires the image data thereof, the process can be associated and held for each area of the form. Then, the information processing system 10 can acquire only the area from the image data of the form transmitted from the device 20 and perform the process corresponding to the area. As a result, it becomes possible to associate the image processing area and the process with each of any number of applications, and the user can cause the desired process (for example, OCR process, etc.) to be performed on the desired area of the image data generated by scanning only by selecting an application according to the type of form, for example.

[0029] <Supplement regarding the correspondence between color and process> The series of processes described with reference to FIGS. 1 and 2 will be supplemented by dividing them into operations 1 to 3. FIG. 3 is a diagram for explaining the registration of the correspondence between color and process (operation 1) using a registration form.

[0030] FIG. 3(a) is the registration form before coloring.

[0031] FIG. 3(b) is the registration form after coloring. The device 20 scans the registration form in FIG. 3(b).

[0032] FIG. 3(c) is a part of the setting file in which color and process are associated. Two methods can be considered for associating color and process.

[0033] 1: The information processing system side pre-holds the coordinates (within the registration form) from which the color is obtained and the process obtained from those coordinates associated or the information. Example of pre-parameters · The color extracted at the coordinates (X coordinate: 100, Y coordinate: 200, 1, 50) is OCR · The color extracted at the coordinates (X coordinate: 100, 200; Y coordinate: 51, 100) is masked (OMR in Table 4). · The color extracted at the coordinates (X coordinate: 100, 200; Y coordinate: 101, 150) is trimmed. 2: The administrator associates the color with the process on the management site. Figure 4 is an example of the management site 150 for associating colors with processes. The management site 150 is provided by the information processing system 10 to the terminal device as a web page or a web application. The administrator can register the color of the process with the radio button 151. In this case, the colors of the marker pens used by the user are limited to three.

[0034] In any method, as shown in Fig. 1(c), the numerical value identifying the marker color (green) is associated with OCR. The numerical value identifying the marker color (blue) is associated with OMR. The numerical value identifying the marker color (red) is associated with trimming.

[0035] Figure 5 is a diagram for explaining the correspondence between the area information and the process (operation 2) using the original form of the form.

[0036] Figure 5(a) is the original form of the form.

[0037] Figure 5(b) is the form after coloring. The device 20 scans the form in Fig. 5(b).

[0038] Figure 5(c) is a part of the setting file in which the area information and the process are associated.

[0039] As shown in Fig. 5(d), since the color and the process are associated in the setting file, the information processing system 10 detects the color of the registration form (already registered in the setting file) from the form, and associates the coordinates of the area where this color is detected with the process in the setting file as area information.

[0040] Therefore, in this embodiment, the administrator can change the process for the area with the color applied to the area of the form. Ex1: Colored in blue → OCR, colored in red → Masking Ex2: Colored in blue → Beginning of file name, colored in red → End of file name

[0041] FIG. 6 is a diagram for explaining processing (operation 3) on an area within a form using the form.

[0042] FIG. 6(a) is the original sheet on which the user has entered necessary items.

[0043] FIG. 6(b) shows the processing performed on each area of the form. Since OCR and file name assignment are registered as processing in the title area 140 of the form, the title becomes the file name. Since masking is registered as processing in the prior application area 141 of the form, the prior application area is masked. Since trimming is registered as processing in the remarks area 142 of the form, the remarks area is acquired as image data.

[0044] <Regarding terms> A mark is a form (type) as a means for distinguishing from others. In the present embodiment, color is an example of a mark, but characters, alphabets, symbols, etc. may also be marks.

[0045] The first device is a device or the terminal device of an administrator. The second device is, for example, a device.

[0046] "Based on the user's operation" includes the case where the user sets using a registration form and the case where the user sets using a management screen.

[0047] The setting information is information for associating, for example, colors and the content of processing (see, for example, Table 4).

[0048] The administrator 8 is a system administrator on the customer side of the data output system, etc., and is a person who makes settings regarding the data output system for general users. The user is a general user on the customer side. Since a general user may perform the work of the administrator 8, it is not necessary to distinguish between the two.

[0049] A form is a general term for books, vouchers, etc. It also refers to office paper with blank spaces for users to fill in. In this embodiment, the form will be mainly described, but any name can be used as long as it is a manuscript that can be read by device 20. For example, the form may be a document, a file, a written document, a report, a newspaper, a magazine, etc. Note that the form may not be printed on paper media (there may be cases where device 20 does not need to read the form).

[0050] The type of image data to be processed can be any kind, such as PDF, JPEG, PNG, TIFF, BMP, etc.

[0051] In this embodiment, the "configuration file" is a file in which processing and area information are set, and the "file" is a file in which acquired data is output. A file is a collection of data, which is the basic unit of data managed by a computer.

[0052] To output data means to enable the user to visually confirm the data. In this embodiment, the data is output to a file, for example.

[0053] A web application refers to software or its mechanism that operates on a browser by the cooperation of a program written in a programming language (for example, JavaScript (registered trademark)) that operates on the browser and a program on the web server side and is executed on the browser.

[0054] <System configuration example> First, the system configuration of the data output system 100 according to this embodiment will be described with reference to FIG. 7. FIG. 7 is a diagram showing an example of the system configuration of the data output system 100 according to this embodiment.

[0055] The data output system 100 shown in FIG. 7 includes an information processing system 10 and a device 20, and is communicably connected via a wide-area network N1 such as the Internet. The data output system 100 communicates with an external service system 40 which is an external system. Also, an arbitrary terminal device 30 can communicate with the information processing system 10 via the networks N1 and N2.

[0056] The information processing system 10 is realized by one or more information processing devices, and provides various services realized by a series of processes in cooperation with an external service system 40 such as a cloud service via the network N1. Specific examples of the services provided by the information processing system 10 according to the present embodiment will be described later. The information processing system 10 may be realized by cloud computing or may be realized by a single information processing device. Cloud computing refers to a usage form in which resources on a network are used without awareness of specific hardware resources. The information processing system 10 may exist on the Internet or on-premises. Note that a series of processes are provided by one application, and a series of processes are also referred to as a "processing flow" or a "workflow".

[0057] The device 20 is various electronic devices used by a user. The device 20 is, for example, an image processing device such as an MFP (Multifunction Peripheral), an image forming device, a PC (Personal Computer), a projector, an electronic blackboard, a digital camera, etc. The device 20 is connected to the network N2. The user can use the device 20 to utilize various services provided by at least one of the information processing system 10 or the external service system 40.

[0058] Hereinafter, when distinguishing each of a plurality of devices 20, they will be described with subscripts such as "device 201", "device 202", etc.

[0059] The terminal device 30 is, for example, a desktop PC (personal computer), a notebook PC, a smartphone, a tablet terminal, etc. used by the administrator 8 or the user. In addition, the terminal device 30 may be any device on which a web browser can operate. The terminal device 30 is connected to the network N2. The administrator 8 or the user can operate the terminal device 30 to use various services provided by the information processing system 10 or the external service system 40, and can further set the linked application or the application.

[0060] For the sake of convenience of explanation, the device 20 and the terminal device 30 are described separately, but the device 20 and the terminal device 30 may be collectively referred to as a device.

[0061] In addition, hereinafter, when distinguishing a plurality of terminal devices 30 from each other, they are described using suffixes such as "terminal device 301" and "terminal device 302".

[0062] The external service system 40 is one or more information processing devices that provide services by executing a linked application via the network N1. Data management or processing is performed by executing the linked application, and the data itself or the processing result becomes a service.

[0063] Examples of the external service system 40 include, for example, cloud services, ASPs (Application Service Providers), etc., and may include various external services provided via the network. In this embodiment, a storage service is described as an example of the service. The external service system 40 may exist on the Internet or on-premises.

[0064] In addition, hereinafter, when distinguishing a plurality of external service systems 40 from each other, the illustrated "external service system 401" is described as external service system 40A, "external service system 402" is described as external service system 40B, etc.

[0065] <Hardware Configuration Example> Referring to FIG. 8, the hardware configurations of the information processing system 10 and the terminal device 30 included in the data output system 100 according to this embodiment will be described.

[0066] <<Information Processing System and Terminal Device>> FIG. 8 is a diagram showing an example of the hardware configurations of the information processing system 10 and the terminal device 30 according to this embodiment. As shown in FIG. 8, the information processing system 10 and the terminal device 30 are constructed by a computer, and include a CPU 501, a ROM 502, a RAM 503, an HD 504, an HDD (Hard Disk Drive) controller 505, a display 506, an external device connection I / F (Interface) 508, a network I / F 509, a bus line 510, a keyboard 511, a pointing device 512, a DVD-RW (Digital Versatile Disk Rewritable) drive 514, and a media I / F 516.

[0067] Among these, the CPU 501 controls the operations of the entire information processing system 10 and the terminal device 30. The ROM 502 stores programs used for driving the CPU 501 such as IPL. The RAM 503 is used as a work area for the CPU 501. The HD 504 stores various data such as programs. The HDD controller 505 controls the reading or writing of various data to and from the HD 504 according to the control of the CPU 501. The display 506 displays various information such as a cursor, a menu, a window, characters, or an image. The external device connection I / F 508 is an interface for connecting various external devices. The external devices in this case are, for example, a USB (Universal Serial Bus) memory, a printer, and the like. The network I / F 509 is an interface for performing data communication using the network N2. The bus line 510 is an address bus, a data bus, etc. for electrically connecting the components such as the CPU 501 shown in FIG. 8.

[0068] Also, the keyboard 511 is a type of input means having a plurality of keys for inputting characters, numerical values, various instructions, etc. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 514 controls reading or writing of various data with respect to the DVD-RW 513 as an example of a removable recording medium. Note that the DVD-RW may be a DVD-R or the like. The media I / F 516 controls reading or writing (storage) of data with respect to the recording medium 515 such as a flash memory.

[0069] <<Device>> FIG. 9 is a hardware configuration diagram of an image forming apparatus which is an example of the device 20. As shown in FIG. 9, the image forming apparatus includes a controller 910, a short-range communication circuit 920, an engine control unit 930, an operation panel 940, and a network I / F 950.

[0070] Among these, the controller 910 has a CPU 901 which is a main part of a computer, a system memory (MEM-P) 902, a north bridge (NB) 903, a south bridge (SB) 904, an ASIC (Application Specific Integrated Circuit) 906, a local memory (MEM-C) 907 which is a storage part, an HDD controller 908, and an HD 909 which is a storage part, and is configured to connect between the NB 903 and the ASIC 906 by an AGP (Accelerated Graphics Port) bus 921.

[0071] Among these, the CPU 901 is a control part that performs overall control of the image forming apparatus. The NB 903 is a bridge for connecting the CPU 901, the MEM-P 902, the SB 904, and the AGP bus 921, and has a memory controller that controls reading and writing with respect to the MEM-P 902, and a PCI (Peripheral Component Interconnect) master and an AGP target.

[0072] MEM-P902 consists of a ROM902a which is a memory for storing programs and data for realizing each function of the controller 910, and a RAM902b which is used as a memory for developing programs and data and for drawing during memory printing. Note that the program stored in the RAM902b may be configured to be recorded and provided on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD in an installable or executable file format.

[0073] SB904 is a bridge for connecting the NB903 with PCI devices and peripheral devices. The ASIC906 is an IC (Integrated Circuit) for image processing applications having hardware elements for image processing, and has the role of a bridge for connecting the AGP bus 921, PCI bus 922, HDD controller 908, and MEM-C907 respectively. This ASIC906 consists of a PCI target and an AGP master, an arbiter (ARB) forming the core of the ASIC906, a memory controller for controlling the MEM-C907, a plurality of DMACs (Direct Memory Access Controllers) for performing operations such as rotation of image data by means of hardware logic, etc., and a PCI unit for performing data transfer via the PCI bus 922 between the scanner unit 931, printer unit 932, and facsimile unit. Note that the ASIC906 may have a USB (Universal Serial Bus) interface and an IEEE1394 (Institute of Electrical and Electronics Engineers 1394) interface and be connected to these.

[0074] MEM-C907 is a local memory used as a copy image buffer and a code buffer. HD909 is a storage for accumulating image data, font data used at the time of printing, and forms. HD909 controls the reading or writing of data to HD909 according to the control of CPU901. The AGP bus 921 is a bus interface for a graphics accelerator card proposed to speed up graphic processing. By directly accessing MEM-P902 with high throughput, the graphics accelerator card can be made faster.

[0075] Also, the short-range communication circuit 920 is provided with an antenna 920a of the short-range communication circuit. The short-range communication circuit 920 is a communication circuit such as NFC or Bluetooth (registered trademark).

[0076] Furthermore, the engine control unit 930 has a scanner unit 931, a printer unit 932, and a facsimile unit 933. Also, the operation panel 940 includes a panel display unit 940a such as a touch panel for displaying current setting values, selection screens, etc. and receiving inputs from the operator, and a hard key 940b including a numeric keypad for receiving setting values of conditions related to image formation such as density setting conditions and a start key for receiving a copy start instruction. The controller 910 controls the entire image forming apparatus, for example, controls drawing, communication, inputs from the operation panel 940, etc. The scanner unit 931 or the printer unit 932 includes an image processing part such as error diffusion or gamma conversion.

[0077] Note that the image forming apparatus can sequentially switch and select the document box function, the copy function, the printer function, and the facsimile function by the application switching key of the operation panel 940. When the document box function is selected, the image forming apparatus becomes the document box mode, when the copy function is selected, it becomes the copy mode, when the printer function is selected, it becomes the printer mode, and when the facsimile mode is selected, it becomes the facsimile mode.

[0078] The network I / F 950 is an interface for data communication using the network N2. The short-range communication circuit 920 and the network I / F 950 are electrically connected to the ASIC 906 via the PCI bus 922.

[0079] <Regarding functions> Next, the functional configuration of the data output system 100 according to the present embodiment will be described with reference to FIG. 10. FIG. 10 is a diagram showing an example of the functional configuration of the device 20, the terminal device 30, and the information processing system 10 according to the present embodiment.

[0080] <<Device>> First, the device 20 includes a second communication unit 21, a display control unit 22, an operation reception unit 23, an image data generation unit 24, a facsimile processing unit 25, an e-mail processing unit 26, and an output unit 27. Each of these functional units is a function or means realized by the CPU 901 executing instructions included in one or more programs installed in the device 20. For example, the second communication unit 21, the display control unit 22, and the operation reception unit 23 are realized by a web browser, and the others are realized by individual applications (native apps).

[0081] The second communication unit 21 transmits and receives various types of information to and from the information processing system 10. In the present embodiment, the second communication unit 21 receives screen information such as an application list screen from the information processing system 10 and transmits image data of a form, an application ID, etc. to the information processing system 10.

[0082] The display control unit 22 interprets the screen information of various screens and displays it on the panel display unit 940a. The operation reception unit 23 receives various operations of the administrator 8 or the user on the various screens displayed on the panel display unit 940a.

[0083] When the application selected by the operation reception unit 23 generates image data, the image data generation unit 24 scans the document with the scanner unit 931 to generate image data. The image data generation unit 24 has a scanning function. The image data generation unit 24 converts the image data into a PDF file. The image data (such as JPEG, TIFF, PNG) may remain in the original format without conversion. The facsimile processing unit 25 performs processing related to the reception and transmission of facsimiles by the facsimile unit 933, and requests the information processing system 10 to execute the application associated in advance when a facsimile is received. Note that the image data generation unit 24 may request the application corresponding to the facsimile transmission source (FAX number).

[0084] The e-mail processing unit 26 performs processing related to the sending and receiving of e-mails, and requests the information processing system 10 to execute the application associated in advance when an e-mail is received. Note that the e-mail processing unit 26 may request the application corresponding to the e-mail transmission source (e-mail address).

[0085] The output unit 27 outputs the paper on which the image is formed by performing image forming processing. The image forming method may be an electrophotographic method or an inkjet method.

[0086] <<Terminal Device>> The terminal device 30 includes a first communication unit 31, a display control unit 32, and an operation reception unit 33. These functional units are functions or means realized by the CPU 501 executing instructions included in one or more programs installed in the terminal device 30. Note that this program may be a web browser or dedicated software for communicating with the information processing system 10.

[0087] The first communication unit 31 transmits and receives various types of information to and from the information processing system 10 or the external service system 40. In the present embodiment, it receives various types of screen information and the like from the information processing system 10 or the external service system 40, and transmits the information set by the administrator 8 or the user to the information processing system 10 or the external service system 40.

[0088] The display control unit 32 interprets the screen information of various screens and displays it on the display 506. The operation reception unit 33 receives various operations of the administrator 8 or the user on the various screens displayed on the display 506.

[0089] <<Information Processing System>> The information processing system 10 includes an application data service unit 11, a setting reception unit 12, an authentication service unit 14, a Web service processing unit 15, an input / output service unit 16, a component 18, and a communication unit 19. These functional units are functions or means realized by the CPU 501 executing instructions included in one or more programs installed in the information processing system 10. In FIG. 10, for the convenience of drawing, one information processing system 10 has each functional unit, but each functional unit may be distributed and arranged in a plurality of information processing devices.

[0090] The communication unit 19 transmits and receives various types of information to and from the terminal device 30 and the device 20. The setting reception unit 12 provides a setting site to the terminal device 30 and receives settings for the setting site. The site may refer to a Web page or a Web application in some cases, or may refer to the server-side system that provides the Web page or the Web application to the terminal device 30 in some cases. The setting reception unit 12 has a terminal screen providing unit 42, and the terminal screen providing unit 42 provides an initial screen, an application list screen, an association screen, etc. for allowing the administrator 8 to set information necessary for the execution of the application. Note that the terminal screen providing unit 42 generates screen information described by HTML, XML, CSS (Cascade Style Sheet), JavaScript (registered trademark), etc. in response to an HTTP request from the terminal device 30, and transmits the screen information to the terminal device 30 as an HTTP response.

[0091] The application data service unit 11 has an application data management unit 41. The application data management unit 41 manages the application data 43 of applications purchased by customers such as enterprises for each customer. The application data 43 associates, for example, the application with the above-described setting file. The application data 43 is stored in the application data storage unit 13. Note that an organization such as a customer as viewed from the data output system 100 is called a tenant.

[0092] The Web service processing unit 15 provides screen information to the device 20 and accepts the execution of an application from the device 20. The Web service processing unit 15 has an application execution unit 46 and a device screen providing unit 47. The application execution unit 46 accepts the execution of an application from the device 20 and requests the input / output processing unit 48 to execute the application. The device screen providing unit 47 uses the screen information 54 to provide the screen information of the application list screen and the application screen to the device 20.

[0093] The input / output service unit 16 performs data input / output using the authentication service unit 14, the application processing information storage unit 17, and the component 18. The input / output service unit 16 has an input / output processing unit 48 and an application management unit 49. The input / output processing unit 48 executes the application requested by the application execution unit 46. At the time of execution, the input / output processing unit 48 refers to the application processing information 53 and requests each process included in the application from the component 18.

[0094] The application management unit 49 manages the application processing information 53 and the screen information 54 of the application stored in the application processing information storage unit 17. In the application processing information 53, it is set which process the application performs in which order (for example, after OCR, a process request is made to the external service system 40, etc.). The screen information 54 is information on the screens displayed by the device 20, such as the icon of each application, the application list screen, and each application screen. The application processing information 53 and the screen information 54 are stored in the application processing information storage unit 17 for each application.

[0095] The authentication service unit 14 performs processes related to the authentication of users (including the administrator 8) and the device 20. The authentication service unit 14 has a user management unit 44 and a device management unit 45. The user management unit 44 manages information related to users. The authentication service unit 14 holds authentication information such as, for example, user names and passwords. Also, the authentication information includes that of the information processing system 10 and that of the external service system 40. The device management unit 45 manages the devices 20 associated with the tenant. That is, it manages by associating the identification information of the tenant with the device 20, and when the device 20 is specified, the tenant is also specified.

[0096] The component 18 is a general term for the individual functions that actually perform each process. One block within the component 18 corresponds to one process. The component 18 has, for example, an authentication processing unit 61, an external service processing unit 62, a determination unit 63, a PDF processing unit 64, a region detection unit 65, a data storage unit 66, a region specification unit 67, an OCR processing unit 68, an OCR correction unit 69, an OMR processing unit 71, a trimming processing unit 72, and a color detection unit 73.

[0097] The authentication processing unit 61 is a component 18 that requests the authentication service unit 14 to perform authentication-related processes.

[0098] The external service processing unit 62 is a component 18 that requests a processing request to the external service system 40. That is, the external service processing unit 62 is a function for cooperating with the external service system 40. The external service processing unit 62 is prepared for each type of the external service system 40.

[0099] The determination unit 63 makes various determinations and controls the branch of the processing flow. The PDF processing unit 64 performs processes related to PDF files, such as extracting image data from a PDF file or creating a PDF file with transparent text. This is for the device 20 to transmit image data in the form of a PDF file.

[0100] The area detection unit 65 detects the marks associated with the processing from the image data of the registration form. Also, the area detection unit 65 detects the rectangular area marked with a marker pen or the like from the image data of the form, and creates the coordinates (area information) of the rectangular area.

[0101] Based on the area information, the area specification unit 67 specifies the processing target area from the image data of the form. The area specification unit 67 acquires the specified processing target area (for example, it may be cut out). The OCR processing unit 68 performs OCR processing on only the area and the entire image data. The OCR correction unit 69 corrects the OCR processing result. The OMR processing unit 71 performs OMR processing on the area. The trimming processing unit 72 performs trimming processing on the area. Note that, in addition to what is shown in the figure, the component 18 may have components such as translation.

[0102] The data storage unit 66 associates the color with the processing, and further associates the processing with the area information and registers it in the setting file. Also, the data storage unit 66 outputs a file including the acquired data (data acquired from the area of the form) generated by the OCR processing unit 68, the OMR processing unit 71, and the trimming processing unit 72 through the processing.

[0103] The color detection unit 73 detects the color colored with a marker pen by the administrator 8 on the registration form.

[0104] <External service system> The external service system holds the registration form 5 and the setting file 9. Among these, the registration form 5 is registered in advance. The setting file 9 is registered by the administrator 8.

[0105]

Table 1

[0106] When the administrator 8 or the user logs in, the application management unit 49 acquires a list of applications for which the administrator 8 or the user has usage rights from the application processing information storage unit 17 and provides it to the device 20. Also, when an application is specified from the list of applications, the application management unit 49 identifies the screen information and application processing information associated with the application.

[0107]

Table 2

[0108] Note that "completed the linkage setting" means that the user wishes to link with the external service system 40 and it has been confirmed that the user can log in with the authentication information of the external service system 40. When the user inputs the authentication information of the external service system 40 from the setting site and can log in, the identification information of the external service system is set in the external service system information in Table 2(a). Also, the authentication information of the external service system 40 is registered in Table 2(b).

[0109] Table 2(b) contains information about the external service systems of each user, and has columns for user ID, external service system 40, user ID, and password. The authentication information for the external service system 40 that the user has linked and set up is registered in Table 2(b). The authentication information for the external service system 40 is the information required for the user to log in to the external service system 40. For users like U002, the authentication information may vary for each external service system 40. Therefore, the authentication information for the external service system 40 is associated with each user of the data output system 100. The information processing system 10 identifies the authentication information for the linked external service system 40 according to the authentication information such as the user ID and password of the information processing system 10 obtained from the device 20.

[0110] Also, the authentication information may be the authentication information for the external service system 40 or the authentication information required for linking with the external service system 40. In addition to the user ID and password, it may also be a token or the like obtained from the external service system 40 when setting up the link with the external service system 40.

[0111] Table 2(c) is the authentication information for the shared application. The authentication information for the external service system 40 is associated with the identification information of the shared application. Since the authentication information for the shared application is that of the administrator 8, the identification information of the administrator 8 is registered.

[0112]

Table 3

[0113]

Table 4

[0114] · The ID is the identification information of the area information. It is automatically numbered, and there is no need for the administrator 8 or the user to be aware of it.

[0115] · The marker color is the color of the cell read from the registration form (shown in FIG. 13) (an example of identification information for identifying the content of the process). The color of the cell is registered, for example, in the HSV color space. The "H" (hue) in the HSV color space is called the hue and indicates the "color tone". The range of "H" is 0 - 360 degrees. The "S" (saturation) is the saturation and indicates the "vividness". The range of "S" is 0 - 100%. The "V" (Value) is the brightness and indicates the "brightness". The range of "V" is 0 - 100%. Note that the color space representing the marker color may also be HSB, RGB, Lab, XYZ, etc.

[0116] ·The process is the content of the process associated with the cell on the registration form. The process is, for example, OCR, OMR, and trimming, but there may be various processes such as translation, masking (solid filling), encryption, etc.

[0117] ·The starting point X coordinate, starting point Y coordinate, ending point X coordinate, and ending point Y coordinate are area information. These represent the diagonal vertices of the rectangular area of the marked form. The coordinates of a single point, width, and height may also be specified. The unit of the coordinates may be, for example, cm, pixel, etc.

[0118] The above information becomes the information automatically set by the information processing system 10. The following information is input by the administrator 8.

[0119] ·The format is the data format of the area. For example, there are text, Y / N, image, date (YYYY / MM / DD), numeric, etc. Since the data format is known, the OCR processing unit 68 can perform OCR processing with higher accuracy. The format is a pull-down menu. In this way, the type of data is set for each area.

[0120] ·The CSV column name is the name given to the area. In the case of a form, the title of the rectangular frame, etc. is input, but the administrator 8 can input any value. The CSV column name (when the CSV position is specified) is output to the file together with the data.

[0121] ·The CSV position (column) specifies in which column of the file the information processing system 10 writes the data extracted from the area.

[0122] ·The CSV position (starting row) specifies in which row of the file the information processing system 10 writes the data extracted from the area. Since the row number increases each time the form is read, the "starting row" is set when no data is written in the file. For example, the data obtained from the first form is set in the second row, and the data obtained from the second form is set in the third row. In contrast, the column position may be fixed.

[0123] · The OCR language is set to indicate which language the area is described in. Since the language is known, the OCR processing unit 68 can perform OCR processing with higher accuracy. The OCR language is in a pull-down menu. In this way, the data type is set for each area.

[0124] · The order used for the file name is used for the file name of the file to which the data acquired from the area is output in this embodiment. The order used for the file name is the order of the data used for the file name of the file output by the information processing system 10. For example, when the "document name" is No. 1, the file name is "expense settlement statement.csv" based on the data (expense settlement statement) acquired from the area of the document name.

[0125] <App processing information> Next, with reference to FIG. 11, the details of the app processing information will be described. FIG. 11 is a diagram showing an example of the app processing information 53 stored in the app processing information storage unit 17.

[0126] The app processing information 53 shown in FIG. 11 is information regarding a series of processes for realizing a service that requests processing of the image data generated by scanning with the device 20 to the "external service system B".

[0127] The app processing information 53 includes a flow ID 1201A indicating the identification information of the app processing information 53, a flow name 1202A indicating the name of the app processing information 53, and a flow detail 1203A indicating the processing content of a series of processes. The flow name 1202A also serves as the name of the app.

[0128] In addition, the flow detail 1203A includes a processing content 1223A indicating a request for processing to the "external service system B".

[0129] In processing content 1223A, the component ID "ExService_B" of component 18 that executes processing in cooperation with "External Service System B", and the parameters "AppId" and "Folder" are specified. "ExService_B" corresponds to the external service processing unit 62 for the external service system B. The "AppId" stores the ID of the linked application (the correspondence between the application and the linked application is determined in advance), and the "Folder" stores the folder for saving files (the upload destination folder) (the upload destination folder set by the administrator 8 is set). As a result, the information processing system 10 can save the scanned image data to the "External Service System B".

[0130] In addition, in the case of an application that does not cooperate with the linked application, the "AppId" is omitted. The illustrated parameters are only a part, and the application processing information may include more parameters.

[0131] In FIG. 11, for simplicity of explanation, only one component is specified, but in the application processing information 53, a series of processing content 1223A performed by the application is registered as many as the number of processes (components).

[0132] FIG. 12 is a diagram schematically showing a series of processes performed by the application. FIG. 12(a) shows a series of processes of an application (the application corresponding to the color registration button 112 described later) that registers the correspondence between the marker color and the process in the setting file. One block corresponds to the processing content 1223A. Details of each process will be described with reference to FIG. 15.

[0133] FIG. 12(b) shows a series of processes of an application (the application corresponding to the form registration button 113 described later) that registers area information in the marker color and process of the setting file. Details of each process will be described with reference to FIG. 25.

[0134] FIG. 12(c) shows a series of processes of an application that outputs data from the area of the form (for example, an expense settlement form application). Details of each process will be described with reference to FIG. 37.

[0135] <Registration in the Setting File for the Correspondence between Color and Processing> Next, with reference to FIGS. 13 to 20, the setting of the correspondence between color and processing by the administrator 8 and the creation of the setting file will be described.

[0136] FIG. 13 is a diagram for explaining a method of setting the correspondence between color and processing using a registration form. FIG. 13(a) is the registration form before the administrator 8 colors it with a marker pen. The original paper of the registration form is registered in the external service system 40.

[0137] In the first row of the registration form, the items for each column (first column: marker color, second column: processing) are described. In this embodiment, there are three types of "processing" (OCR·OMR·trimming), and in the registration form, the processing is set in advance in the order of the second row: OCR, the third row: OMR, and the fourth row: trimming. The cell 130 of the marker color in the original state of the registration form is blank.

[0138] FIG. 13(b) is a diagram in which the administrator 8 colors the cell 130 with a marker pen for each processing. The administrator 8 will use, for example, green, blue, and red marker pens among the marker pens he / she owns. The administrator 8 assigns green to the OCR process, blue to the OMR process, and red to the trimming process. A predetermined interval is provided between the lines of the original paper of the registration form so that color mixing does not occur due to the bleeding of each color.

[0139] The administrator 8 colors the cell 130 of the registration form with the marker pen he / she owns according to the assignment of the desired marker color and each processing determined by himself / herself. By scanning the registration form with the device 20, the correspondence between the marker color and the processing is registered in the information processing system 10.

[0140] Also, in FIG. 13, the cell 130 is filled, but any mark may be set in the cell 130. For example, when symbols such as triangles or circles, alphabets, enclosing lines, etc. are formed in the cell 130, the information processing system 10 registers these marks and the processing in the setting file. In this case, the information processing system 10 may or may not consider the color of the mark.

[0141] Note that, instead of covering the cell 130 with a marker pen, the administrator 8 may indicate the marker color with characters or symbols such as "green or G", "blue or B", "red or R", etc. In this case, the information processing system associates the marker color with the content of the process by character recognition.

[0142] Also, the terminal device 30 may be connected to the information processing system 10, and the administrator 8 may input a color with characters or the like for the Web page displayed on the terminal device 30.

[0143] Furthermore, the administrator 8 may be able to register the registration form itself. The terminal device 30 is connected to the information processing system 10, and for the Web page displayed on the terminal device 30, the administrator 8 registers the content of the process and creates a registration form as shown in FIG. 13(a).

[0144] <Example of the device screen> FIG. 14 is an example of an administrator screen 110 displayed when the administrator 8 logs in to the device 20. The administrator screen 110 has an administrator menu 118 and a general user menu 119. The administrator menu 118 is a menu for the administrator 8 to register a configuration file. The general menu is a list of applications that the administrator 8 can use as a user.

[0145] The registration form button 111 in the administrator menu 118 is a button for the administrator 8 to print a registration form. The color registration button 112 is a button for the administrator 8 to cause the device 20 to scan a registration form and register the correspondence between the marker color and the process in the configuration file. The form registration button 113 is a button for the administrator 8 to cause the device 20 to scan the marked form and register the correspondence between the process and the area information in the configuration file.

[0146] Note that the registration form button 111, the color registration button 112, and the form registration button 113 are each a Web application, and the administrator 8 can register the configuration file by working according to the Web application.

[0147] When the registration form button 111 of the administrator menu 118 is pressed, the device 20 displays the registration form. The administrator 8 prints the registration form and colors it with a marker pen. Next, when the administrator 8 presses the color registration button 112 of the administrator menu 118 and performs a scan of the registration form colored with the marker pen, the image data of the registration form is transmitted to the information processing system 10. The information processing system 10 registers the correspondence between the marker color and the process in the configuration file.

[0148] Next, the administrator 8 presses the form registration button 113 of the administrator menu 118, and scans the original form of the form colored with the marker pen with the device 20. The image data of the form is transmitted to the information processing system 10, and the information processing system 10 registers the correspondence between the process and the area information in the configuration file.

[0149] Note that when the form registration button 113 is pressed, the device 20 may display the correspondence between the marker color received from the information processing system 10 and the process. For example, the device 20 displays information as shown in Fig. 13(b) on the operation panel. When the administrator 8 wants to change the association between the color and the process, the administrator 8 presses the up and down buttons 114 on the operation panel. The device 20 shifts the process column one line at a time each time the up and down buttons 114 are pressed. By doing so, the administrator 8 can change the association between the color and the process.

[0150] When the administrator 8 as a user presses the app of the general user menu 119, the app starts. When the user scans the filled form with the device 20, the information processing system executes the process associated with the area specified by the area information in the configuration file for each area.

[0151] Fig. 15 is an example of a sequence diagram showing the procedure for the administrator 8 to register the correspondence between the color and the process in the configuration file using the registration form.

[0152] S1: The administrator 8 operates the device 20 to log in to the information processing system 10, and presses the registration form button 111 from the administrator menu 118.

[0153] S2: The second communication unit 21 of the device 20 transmits a request to execute the application corresponding to the registration form button 111 to the information processing system 10.

[0154] S3: The communication unit 19 of the information processing system 10 receives the request to execute the application. The application execution unit 46 specifies the application ID of the application corresponding to the registration form button 111 and requests the input / output processing unit 48 to execute. The input / output processing unit 48 refers to the application processing information 53 associated with the application ID of the application corresponding to the registration form button 111 and executes the application. First, the input / output processing unit 48 requests the external service processing unit 62 for a list of registration forms.

[0155] S4: The external service processing unit 62 communicates with the external service system 40 using the authentication information of the administrator 8 and requests a list of registration forms that the administrator 8 can use.

[0156] S5: Since the second communication unit 21 of the device 20 receives the list of registration forms, the display control unit 22 displays the list of registration forms. The administrator 8 selects the registration form in which the process to be executed is registered. The operation reception unit 23 receives the selection of the registration form. If there is only one type of registration form, the user may give an instruction to print instead of making a selection.

[0157] S6: The output unit 27 of the device 20 prints the registration form selected by the administrator 8. In the sequence diagram, the process of the device 20 obtaining the registration form from the external service system 40 is omitted. It is preferable that the registration form has the identification information of the registration form or the position information of the cell 130 to be colored formed by a barcode or the like. Details will be described in FIG. 16.

[0158] S7: The administrator 8 colors the cell 130 associated with the process of the printed registration form with a marker pen. Then, the administrator 8 presses the color registration button 112 from the administrator menu 118.

[0159] S8: The operation reception unit 23 of the device 20 receives the execution of the color registration button 112, and the image data generation unit 24 scans the registration form to generate a PDF file.

[0160] S9: The second communication unit 21 of the machine 20 transmits a request to execute the app corresponding to the PDF file and the color registration button 112 to the information processing system 10.

[0161] S10: The communication unit 19 of the information processing system 10 receives the PDF file. The app execution unit 46 requests the input / output processing unit 48 to execute by specifying the app ID of the app corresponding to the color registration button 112. The input / output processing unit 48 refers to the app processing information 53 associated with the app ID corresponding to the color registration button 112 and executes the app. First, the input / output processing unit 48 requests the PDF processing unit 64 to extract image data by specifying the PDF file. The PDF processing unit 64 extracts the image data of the registration form from the PDF file.

[0162] S11: Next, the input / output processing unit 48 transmits the image data to the color detection unit 73 and requests the detection of the correspondence between the color and the process.

[0163] S12: The color detection unit 73 detects the color information from the predetermined cell 130 of the image data.

[0164] Figure 16 schematically shows the detection of the color information. A barcode 101 indicating the orientation is formed at a corner such as the upper left on the registration form, and the color detection unit 73 rotates the image data so that the barcode 101 comes to the upper left. Then, the color detection unit 73 detects the color information from the predetermined cell 130 (with a determined position) for each process.

[0165] As shown in Fig. 16(a), the OCR process is determined by the color within the rectangle of two position coordinates AB (an example of position information) indicating the diagonal, the OMR process is determined by the color within the rectangle of position coordinates CD, and the trimming process is determined by the color within the rectangle of position coordinates EF. The correspondence between such position coordinates AB, CD, and EF and the processes can be stored in the barcode 101 or held by the application data management unit 41 (in this case, the barcode 101 may preferably include at least the identification information of the registration form, but if there is only one type of registration form, the identification information may not be necessary).

[0166] As shown in Fig. 16(b), the barcode 101 or the application data management unit 41 holds the position coordinates of the cells 130 to be processed and colored in association with each other. The color detection unit 73 acquires the H value of HSV from each cell 130 based on the position coordinates in Fig. 16(b). When a color corresponding to red (H = (0 to 60°, 301 to 360°)) is detected from the cell 130 associated with trimming, the color detection unit 73 registers H = (0 to 60°, 301 to 360°) in the setting file in association with trimming. When a color corresponding to green (H = (61 to 180°)) is detected from the cell 130 associated with OCR, the color detection unit 73 registers H = (61 to 180°) in the setting file in association with OCR. When a color corresponding to blue (H = (181 to 300°)) is detected from the cell 130 associated with OMR, the color detection unit 73 registers H = (181 to 300°) in the setting file in association with OMR. Instead of registering the entire range of H values of red, blue, and green in this way, the color detection unit 73 may register the color (H value) with a margin such as ±10% for the detected H value in the setting file.

[0167] Note that there is also a method in which the administrator uses the management site 150 described in Fig. 4 for the method of associating colors with processes. The sequence of settings using the management site 150 is as follows. The communication unit 19 transmits the screen information of the management site 150 to the terminal device 30 and receives the correspondence between the mark received by the terminal device 30 and the process. The data storage unit 66 stores the process and the mark in association with each other in the setting file.

[0168] S13: Return to FIG. 15 for explanation. The application execution unit 46 acquires the correspondence between colors and processes from the color detection unit 73. The application execution unit 46 transmits the correspondence between colors and processes to the data storage unit 66 and requests the generation of a configuration file. The data storage unit 66 assigns numbers to the IDs and creates a configuration file in which the marker colors and processes are associated. The data storage unit 66 transmits the configuration file to the application execution unit 46. At this point, the information is in the state entered in columns A to C of Table 4. Note that the correspondence between the marker colors and processes may be stored in a separate independent file instead of the configuration file.

[0169] S14, S15: The application execution unit 46 transmits the configuration file to the external service system 40 via the external service processing unit 62. The external service processing unit 62 acquires the authentication information of the administrator 8 in the external service system 40 from the user management unit 44. Thereby, the external service processing unit 62 logs in to the external service system 40 and saves the configuration file in the top folder of the administrator 8 (or the folder designated by the administrator 8). When the administrator 8 designates the file name of the configuration file, the external service processing unit 62 assigns this file name to the configuration file. If the file name is not set, the current date and time, etc. become the file name.

[0170] Thus, the configuration file in which the marker colors and processes are associated is registered in the external service system 40.

[0171] <<Specification of file name by processing>> In this embodiment, it is described that the administrator 8 sets the file name of the CSV file in which the acquired data is stored, but the administrator 8 can set the file name of the CSV file by coloring the registration form.

[0172] Figure 17 shows an example of the use of a registration form in which the process is applied to the setting of the file name. In Figure 17, the process is limited to OCR, and the data storage unit 66 can use the acquired data at a plurality of locations for the file name. The administrator 8 colors the registration form with the color of the marker pen specified for the file name. In the registration form of Figure 17(a), OCR is set as the process, and the order "file name 1" and "file name 2" used for the file name are set.

[0173] As shown in Figure 17(b), the administrator 8 designates with a marker pen the color of the marker pen used for file name 1 (for example, green) and the color of the marker pen used for file name 2 (for example, blue).

[0174] As shown in Figure 17(c), the numerical value indicating the color and the process (used for the file name) are associated with the setting file. Two methods can be considered for the association between the color and the process.

[0175] 1: The information processing system side pre-holds the coordinates from which the color is obtained (within the registration form) and the process obtained from those coordinates associated with each other. · The color extracted at the coordinates (X coordinate: 100, 200, Y coordinate: 1, 50) is file name 1 · The color extracted at the coordinates (X coordinate: 100, 200, Y coordinate: 51, 100) is file name 2

[0176] 2: The administrator associates the color and the process on the management site. Figure 18 is an example of the management site 152 for associating the color and the process. The administrator can register the color used for specifying the file name with the radio button 153. In this case, the colors of the marker pens used by the user are limited to three.

[0177] In either method, as shown in Figure 17(c), the numerical value identifying the marker color (green) is associated with file name 1. The numerical value identifying the marker color (blue) is associated with file name 2. Note that file names 1 and 2 are the order in which they are used for the file name.

[0178] FIG. 19 shows an example of a form in which the administrator 8 has designated areas at two locations with a marker pen. FIG. 19(a) shows the original form, and FIG. 19(b) shows the form colored with a marker pen. When the device 20 scans the form in FIG. 19(b), the information processing system 10 identifies the colored areas on the form and registers the area information in the setting file.

[0179] In FIG. 19(b), the area of the "Expense Settlement Statement", which is the title of the form, is designated with a green marker pen, and the area of the application date is designated with a blue marker pen. For example, if the application date is "2020 / 10 / 10", the file name of the CSV file when the device scans the form in FIG. 19(b) will be "Expense Settlement Statement20201010.csv". Note that three or more colors of marker pens can be specified for the file name.

[0180] Also, it is not always necessary to register up to the order of the areas used for the file name (files 1, 2). In this case, the data storage unit 66 may set the file name in the order of color registration in the setting file. Therefore, it is not necessary to change the color of the marker pen for each cell of the registration form.

[0181] <<Modification Example of Registration Form>> As shown in FIG. 20, the CSV position may be registered in the registration form in advance. FIG. 20 is a diagram showing a modification example of the registration form. FIG. 20(a) is the registration form before coloring. Since this registration form has columns for the CSV column name 154 and the CSV position 155, the administrator does not need to manually set the CSV position in the setting file.

[0182] FIG. 20(b) is the registration form after coloring. The device 20 scans the registration form in FIG. 20(b).

[0183] FIG. 20(c) shows a part of the setting file in which the colors, processes, and area information are associated.

[0184] <Registration of Area Information> Next, with reference to FIG. 21 and the like, the registration of area information will be described. FIG. 21 is a diagram for explaining a method of marking areas on a form. First, FIG. 21(a) shows the form (original paper) before designating the area with a marker pen. For the original paper of the form, the administrator 8 designates the area with a marker pen of the color registered in the area where the process to be executed with the registration paper is to be performed. For example, assume that the administrator 8 wants to set the title area 140 for OCR, the pre-application area 141 for OMR, and the remarks area 142 for trimming.

[0185] FIG. 21(b) shows the form paper colored in the area with a marker pen. The area 140 of the "Expense Settlement Form", which is the title of the form, is colored with a green marker pen. The area 141 of "Pre-application" is colored with a blue marker pen. The area 142 of "Remarks" at the bottom of the form is colored with a red marker pen. By scanning this form paper with the device 20, the association between the area and the process for the form is registered in the setting file.

[0186] According to the setting file in Table 4, the OCR processing unit 68 performs OCR processing on the area 140 of the "Expense Settlement Form", which is the title of the form, and generates acquisition data. Also, the OMR processing unit 71 performs OMR processing on the area 141 of "Pre-application" and generates acquisition data. The trimming processing unit 72 performs trimming on the area 142 of the remarks and generates acquisition data (for example, "Sorry for the late application."). The trimmed acquisition data is image data (JPEG, GIF, PNG, TIFF, BMP, etc.).

[0187] FIG. 21(c) shows an example of a form in which the user has filled in the necessary items. From the colored state in FIG. 21(b), "Expense Settlement Form", "Pre-apply", and "Sorry for the late application." become the acquisition data.

[0188] Also, as shown in FIG. 22(a), the area does not have to be filled in. In FIG. 22(a), a frame line 602 surrounding the area is formed with a marker pen. The area detection unit 65 detects pixels other than black and white, and detects the circumscribed rectangle of the pixels other than black and white (frame line 602).

[0189] Note that the method of marking the area is not limited to the method shown in FIG. 21 or FIG. 22. For example, the area may be specified by a double line, or the area may be specified within a range surrounded by a predetermined symbol (%, $, #, △, etc.).

[0190] Also, as shown in FIG. 22(b), there may be a plurality of areas colored in the same color on one form. For example, as shown in FIG. 22(b), when the administrator 8 designates the area 143 of the data on the form with a green marker pen, all the areas 143 become the targets of OCR processing.

[0191] Next, assume that the administrator 8 presses the form registration button 113 in FIG. 14. The administrator 8 logs in to the information processing system 10 from the device 20 and selects the form registration button 113.

[0192] Referring to FIG. 23, the effect of designating an area by the administrator filling in the area will be described. FIG. 23 shows a comparative example of the method of designating an area in a form.

[0193] FIG. 23(a) shows a method in which the administrator colors the area with a marker pen and handwrites a △ mark 161 in the same area. However, this method is time-consuming.

[0194] FIG. 23(b) shows a method in which the administrator handwrites a △ mark 162 so as to surround the area with a marker pen. However, in this method, since the area is not completely determined, after the information processing system 10 recognizes the △ mark 162, it is necessary to recognize the rectangle of the area, etc., and the area recognition process becomes complicated.

[0195] On the other hand, by coloring (filling in) the area with a marking pen, the administrator can easily perform "area designation" and "processing designation" at once. · Area designation → Shape... mainly rectangle · Processing designation → Other than shape... color

[0196] <Area registration screen> FIG. 24 is an example of the area registration screen 610 displayed by the device 20 when registering area information. The area registration screen 610 has a message 611 saying "Please set the form with the area to be processed colored and press 'Start'", a reading setting field 612, and a start button 613. The user sets the original (form) on the contact glass and presses the start button 613. The user may set the reading conditions (color / black and white, single-sided / double-sided, orientation, resolution, etc.) in the reading setting field 612 as necessary. Note that regarding color / black and white, it may be fixed to color so that the user does not accidentally set it to black and white.

[0197] Note that it is preferable that the administrator 8 can specify the file name of the setting file from the area registration screen 610 in FIG. 24. This is to make it easier to search for the setting file from the external service system 40. Also, it is preferable that the administrator 8 can set the folder where the setting file is saved.

[0198] FIG. 25 is an example of a sequence diagram showing the procedure in which the information processing system 10 acquires area information from the form marked by the administrator 8 and creates a setting file.

[0199] S101: The administrator 8 operates the device 20 to log in to the information processing system 10 and presses the form registration button 113 from the administrator menu 118. Thereby, the display control unit 22 of the device 20 displays the area registration screen 610. The administrator 8 places the form marked with the area on the contact glass and presses the start button 613. The image data generation unit 24 generates the image data of the form and converts it into a PDF file.

[0200] S102: The second communication unit 21 of the device 20 transmits the PDF file to the information processing system 10 together with the execution request of the application corresponding to the form registration button 113.

[0201] S103: The communication unit 19 of the information processing system 10 receives a PDF file. The app execution unit 46 requests the input / output processing unit 48 to execute by specifying the app ID of the app corresponding to the form registration button 113. The input / output processing unit 48 refers to the app processing information 53 associated with the app ID of the app corresponding to the form registration button 113 and executes the app. First, the input / output processing unit 48 requests the PDF processing unit 64 to extract image data by specifying the PDF file. The PDF processing unit 64 extracts form image data from the PDF file.

[0202] S104: The app execution unit 46 requests the external service processing unit 62 for a configuration file in order to determine the color to be extracted from the image data.

[0203] S105: The external service processing unit 62 obtains a configuration file that can be obtained with the authentication information of the administrator 8 from the external service system 40. Note that the administrator 8 may select a desired configuration file from the list of configuration files.

[0204] S106: Next, the input / output processing unit 48 requests the area detection unit 65 to obtain an area by specifying the image data and the marker color registered in the configuration file. The area detection unit 65 obtains an area from the image data based on the color of the image data.

[0205] Figure 26 is a diagram for explaining the detection of an area from image data. As shown in Fig. 26(a), the colored portion 601 designated with a marker pen has a color corresponding to the marker pen. For example, in the HSV color space, the range of red is H=(0~60°, 301~360°), the range of green is H=(61~180°), and the range of blue is H=(181~300°).

[0206] The area detection unit 65 searches the image data for a color included in the H range of the marker color in the configuration file, for example. Note that the area detection unit 65 detects only areas where the colored areas are continuous with an area of a certain size or more. This eliminates the need to detect portions that the user has colored incorrectly.

[0207] FIG. 26(b) shows the detected color-attached portion 601. The region detection unit 65 sets the circumscribed rectangle 620 of the detected color-attached portion 601. Then, the region detection unit 65 sets margins above, below, left, and right of the circumscribed rectangle 620, and determines a region 621 including the margins. In FIG. 26(b), a [mm] is set on the left and right, and b [mm] is set above and below. The reason for setting the margins is that when the device 20 reads the form, misalignment may occur. The coordinates of the upper left corner 622 in FIG. 26(b) are the starting point X coordinate and the starting point Y coordinate of the region information, and the coordinates of the lower right corner 623 are the ending point X coordinate and the ending point Y coordinate (region information).

[0208] Also, as shown in FIG. 26(c), the region detection unit 65 may detect the outer frame line 625 of the color-attached portion 601 detected based on H of HSV. The region detection unit 65 searches for black pixels outward in the vertical direction of the color-attached portion 601 (circumscribed rectangle 620). When a black pixel is found, it traces the black pixel, and when it returns to the original position, it determines that the outer frame line 625 has been detected. By doing so, the maximum region where data is entered can be detected, and even if misalignment occurs when the device 20 reads the form, the region identification unit 67 can easily identify the data.

[0209] Note that the administrator 8 can set how much margin the region detection unit 65 provides for the region specified by the marker pen, etc. Details will be described on the association screen 640-2 in FIG. 31. The region detection unit 65 sets region information based on the settings of the association screen 640-2.

[0210] S107: Returning to FIG. 25 for explanation. Next, the input / output processing unit 48 designates the image data in which the region is specified by the marker pen and requests the external service processing unit 62 to upload it to the external service system 40.

[0211] S108: The external service processing unit 62 acquires the authentication information in the external service system 40 of the administrator 8 from the user management unit 44. Thereby, the external service processing unit 62 logs in to the external service system 40 and saves the image data in the top folder of the administrator 8 (or the folder designated by the administrator 8). When the administrator 8 designates the file name of the setting file on the area registration screen 610, the external service processing unit 62 assigns the same file name to the file of the image data. If no file name is set, the current date and time, etc. become the file name.

[0212] S109: Next, the input / output processing unit 48 designates the area information detected by the area detection unit 65 and requests the data storage unit 66 to create a setting file. The data storage unit 66 arranges the area information in the fixed columns of the CSV file and generates a setting file. At this point, the information is in the state of being entered in columns A to G of Table 4.

[0213] S110: Next, the input / output processing unit 48 designates the setting file and requests the external service processing unit 62 to upload it to the external service system 40.

[0214] S111: Since the external service processing unit 62 has already logged in to the external service system 40, it saves the setting file in the top folder of the administrator 8 (or the folder designated by the administrator 8) (overwrite save). Since the file names of the image data and the setting file are the same, the administrator 8 can easily determine which image data the setting file was generated from.

[0215] <Registration of Other Information in the Setting File> With reference to FIGS. 27 and 28, the settings for columns H to M in the setting file will be described. FIG. 27 is an example of a sequence diagram showing the procedure for the administrator 8 to set information in columns H to M of the setting file.

[0216] S31: After registering the configuration file in the external service system 40, the administrator 8 operates the terminal device 30 to connect the terminal device 30 to the external service system 40. The administrator 8 inputs his / her authentication information and logs in to the external service system 40.

[0217] S32: Since the external service system 40 is a storage service, the external service system 40 sends a list of files registered by the administrator 8 to the terminal device 30. For example, a list of files in the top folder is sent. If it is not the top folder, the administrator 8 specifies the hierarchy. The list of files also includes the configuration file and the image data file registered by the administrator 8 from the device 20.

[0218] S33: The administrator 8 selects the configuration file to check the content of the configuration file or set the information in columns H to M. The operation reception unit 33 of the terminal device 30 receives the selection, and the first communication unit 31 requests the external service system 40 for the configuration file by specifying the file ID of the configuration file.

[0219] S34: The communication unit 19 of the external service system 40 receives the request for the configuration file and sends the configuration file to the terminal device 30.

[0220] S35: The administrator 8 checks the content of the configuration file and sets the information in columns H to M. The operation reception unit 33 of the terminal device 30 receives the setting.

[0221] S36: When the administrator 8 performs an operation to upload the configuration file, for example, the first communication unit 31 sends the configuration file with the information set in columns H to M to the information processing system 10.

[0222] In the above manner, the configuration file is registered in the external service system 40.

[0223] FIG. 28 is a diagram for explaining the correspondence between the setting file and the form. As described in Table 4, the administrator 8 sets the format, CSV column name, CSV position (column), CSV position (row), OCR language, and the order to be used for the file name. Note that the administrator 8 does not necessarily have to set all the columns. Also, depending on the processing content of column C, the designations of H to M are ignored.

[0224] In the example of FIG. 28, it is set that the data obtained from the area information of ID=1 corresponds to the "document name" in the CSV column name. "Document name" is used first in the file name. It is set that the data obtained from the area information of ID=2 corresponds to the "pre-application" in the CSV column name. Also, since the CSV position of the pre-application is set to column 3, row 2, the data obtained from the area information of ID=2 is written to the 3rd column and 2nd row of the file. It is set that the data obtained from the area information of ID=3 corresponds to the "remarks" in the CSV column name. Also, since the CSV position of the remarks is set to column 4, row 2, the data obtained from the area information of ID=3 is written to the 4th column and 2nd row of the file.

[0225] <Association between the application and the setting file> Next, with reference to FIGS. 29 and 30, the association between the application and the setting file will be described. FIG. 29 is an example of a screen displayed by the terminal device 30 regarding the association between the application and the setting file. The administrator 8 operates the terminal device 30 to cause the terminal device 30 to communicate with the information processing system 10.

[0226] First, FIG. 29(a) is the initial screen 630 displayed when the administrator 8 logs in to the information processing system 10. The operation menu permitted for the administrator 8 is displayed. To associate the application with the setting file, the administrator 8 selects the icon 631 for displaying the application list.

[0227] As shown in Table 2, since the user management unit 44 registers the apps and external service system information available for each user, the information processing system 10 can provide the terminal device 30 with a list of apps available for the administrator 8 for each external service system 40.

[0228] Figure 29(b) shows an example of the app list screen 632. The app list screen 632 displays a list of apps available for the administrator 8. The administrator 8 selects an app 633 (for example, an expense settlement statement app) from the app list screen 632 to associate it with the configuration file registered in the external service system 40. When the operation reception unit 33 receives the selection of the app 633, the association screen 640 shown in Figure 29(c) is displayed. The association screen 640 has an external service system selection column 641, an upload destination folder column 642, and a configuration file column 643.

[0229] · The external service system selection column 641 is a column for the administrator 8 to set the external service system 40 that is the storage destination of the file for writing out the data acquired from the area. Therefore, the administrator 8 can change the external service system 40 pre-associated with the app or set a new one.

[0230] · The upload destination folder column 642 is a column for the administrator 8 to set the folder that is the storage destination of the file for writing out the data acquired from the area in the external service system 40. When the administrator 8 presses the folder selection button 642a, the information processing system 10 logs in to the external service system 40 with the authentication information of the administrator 8 in the external service system 40, and the top folder accessible by the administrator 8 in the external service system 40 is displayed on the terminal device 30. The administrator 8 can select any folder.

[0231] · The setting file field 643 is a field for the administrator 8 to select a setting file to be associated with the application. When the administrator 8 presses the file selection button 644, the information processing system 10 logs in to the external service system 40 with the authentication information of the administrator 8 in the external service system 40, and the top folder that can be accessed by the administrator 8 in the external service system 40 is displayed on the terminal device 30. The top folder (or the folder set by the administrator 8) contains the setting files registered by the information processing system 10 and updated by the administrator 8. If there is no setting file in the top folder, the administrator 8 can search for the setting file from the sub-folders of the top folder.

[0232] Figure 30 is an example of a sequence diagram for explaining the process of the administrator 8 associating an application with a setting file via the screen of Figure 29. The process of Figure 30 starts from the state where the screen of Figure 29(a) is displayed.

[0233] S41: The administrator 8 logs in to the information processing system 10 and inputs an operation to display the application list from the initial screen 630 to the terminal device 30. The operation reception unit 33 of the terminal device 30 receives the operation, and the first communication unit 31 requests the information processing system 10 for the application list.

[0234] S42: The communication unit 19 of the information processing system 10 receives the request for the application list, and the terminal screen providing unit 42 obtains the applications associated with the user ID of the administrator 8 from the user management unit 44. The terminal screen providing unit 42 creates the screen information of the application list screen including the applications that the administrator 8 can use for each external service system 40. The communication unit 19 transmits the screen information of the application list screen to the terminal device 30.

[0235] S43: The administrator 8 selects the application to which the setting file is to be associated. The operation reception unit 33 of the terminal device 30 receives the selection. After that, as described above, the terminal device 30 displays the association screen 640.

[0236] S44: The administrator 8 presses the file selection button 644 on the association screen 640. The operation reception unit 33 receives the press, and the first communication unit 31 sends a request for the file list stored in the external service system 40 set on the association screen 640 to the information processing system 10.

[0237] S45: The communication unit 19 of the information processing system 10 receives the request for the file list, and the application execution unit 46 designates the external service system 40 to the input / output processing unit 48 to request the file list. The input / output processing unit 48 requests the file list from the external service processing unit 62 corresponding to the external service system 40. The external service processing unit 62 obtains the authentication information in the external service system 40 (associated with the application or set by the user on the association screen) of the administrator 8 from the user management unit 44, and sends a request for the file list to the external service system 40.

[0238] S46: Based on the authentication information of the administrator 8, the external service system 40 permits the administrator 8 to log in and obtains a list of files associated with the administrator 8. The list of files is sent to the terminal device 30. Note that in FIG. 30, it is assumed that the external service system 40 where the configuration file is stored and the external service system 40 where the files of the data obtained from the area are stored are the same, but the two may be different. In this case, after the administrator 8 selects the configuration file, the administrator 8 changes the external service system 40 on the association screen 640.

[0239] S47: The display control unit 32 of the terminal device 30 displays the list of files. The administrator 8 selects the configuration file to be associated with the application. The operation reception unit 33 of the terminal device 30 receives the selection.

[0240] S48: The first communication unit 31 of the terminal device 30 designates the identification information of the configuration file, the identification information of the application, the external service system 40, and the upload destination folder, and sends a request for associating the configuration file with the application to the information processing system 10.

[0241] S49: The communication unit 19 of the information processing system 10 receives a request for associating a configuration file with an app, and the app data management unit 41 registers the association between the configuration file and the app in the app data 43. That is, the app data management unit 41 registers by associating the applicationId, applicationName, fileId, fileName, and upload destination folder (see Table 3). Note that the external service system 40 is registered in the app processing information 53.

[0242] As described above, the app and the configuration file are associated. Thereafter, a general user can output the data in the form area to a file simply by pressing an icon from the device 20.

[0243] Note that in FIG. 30, the method by which the administrator 8 associates an app with area information on the association screen 640 has been described, but the association between the app and the area information may be performed when registering the area information.

[0244] In this case, for example, a column for entering the app name is displayed on the area registration screen 610 in FIG. 24. The administrator 8 enters the app name of the app to be associated with the configuration file. After creating the configuration file, the information processing system 10 can associate the app with the entered app name with the configuration file. That is, if the administrator 8 sets the app name and causes the form to be read, the configuration file is generated, and the source file for linkage and the configuration file are associated.

[0245] Also, on the association screen 640 shown in FIG. 29(c), the administrator 8 can set the expansion width of the area. FIG. 31 shows a modified example of the association screen 640. Note that in the description of FIG. 31, the differences from FIG. 29(c) will be mainly described.

[0246] The association screen 640-2 in FIG. 31 has a setting field 645 for the expansion width of the area. In the setting field 645 for the expansion width of the area, the administrator 8 can set how much to expand the area information (OCR area) from the area specified with the marker pen. As a result, the area information in Table 4 becomes wider than the area where the color of the marker pen adheres, and even if the area specified by the administrator 8 with the marker pen is narrower than the width to be marked, or the user enters data outside the area of the form, the OCR processing unit 68 can perform OCR processing on the characters in the area.

[0247] However, not limited to this example, in the association screen 640-2, the administrator 8 may set to include only the inside surrounded by the marker (for example, if the OCR area is surrounded by a marker, the marker part is not included in the area), or to include the area including the marker part.

[0248] Also, regarding how the administrator 8 designates the area, the association screen 640-2 may accept the color, pen type, shape, etc. For example, the administrator 8 can set to surround it with a yellow marker in a square shape, surround it with a red pen in a round shape, or fill it with a green marker.

[0249] <Duplication of the app> Since the shapes of the forms are various, the locations of the areas where data is entered by the user are also various depending on the form. Therefore, in this embodiment, the administrator 8 can register the area information according to the shape of each form. As one method of registering the area information for each form, a method in which the administrator 8 duplicates the app will be described.

[0250] FIG. 32 shows a selection screen 650 of several applications. As an example, when there are a specification application, a payment slip application, and an OCR designated area application, area information of the specification is associated with the specification application, and area information of the payment slip is associated with the payment slip application in the application data 43, respectively. When the user wants to obtain data from the area of the specification, the user can start the specification application and scan the specification to output data of the desired area. When the user wants to obtain data from the area of the payment slip, the user can start the payment slip application and scan the payment slip to output data of the desired area.

[0251] The processing performed by each application is described in the application processing information 53. Although the processing performed by the specification application and the payment slip application differs in the file storage destination, the upload destination folder, and the setting file, the components to be executed are the same. Therefore, if the administrator 8 duplicates the specification application or the payment slip application, the information processing system 10 can create an application corresponding to an arbitrary form by performing area designation, area acquisition, creation of a setting file, and association between the setting file and the duplicated application.

[0252] The duplication of the application will be described with reference to FIG. 33. FIG. 33 is an example of a screen operated by the administrator 8 when duplicating the application. FIG. 33(a) is an initial screen 630 displayed on the terminal device 30 when the administrator 8 logs in to the information processing system 10. The initial screen 630 displays a setting button 651 for the administrator 8 to perform various settings.

[0253] FIG. 33(b) is an application management screen 652 displayed after the setting button 651 is pressed. The application management screen 652 displays a group name 653 in which several applications are grouped. Note that the screen transition is an example for explanation, and omission of the illustrated screen or addition of a new screen may be performed.

[0254] On the right side of group name 653, a list of apps belonging to this group is displayed. The apps displayed in the app list are those for which administrator 8 has been granted usage rights. There are apps exclusive to the administrator as well as apps used by other users. Administrator 8 presses the settings button 654 of the source app to be duplicated. When the settings button 654 is pressed, a menu is displayed, and within the menu there is a duplicate button 655.

[0255] When administrator 8 presses the duplicate button 655, a duplicate app settings screen (popup screen) 660 shown in Fig. 33(c) is displayed. The duplicate app settings screen 660 has an app name settings field 661, an icon list field 662, and a duplicate request button 663. In the app name settings field 661, administrator 8 inputs an arbitrary app name for the new app created by duplication. The app name can be arbitrary, but duplicate app names are excluded or numbered like "~(2)". The app name may also be called an icon name or a settings name in addition to the app name.

[0256] An icon list is displayed in the icon list field 662. Administrator 8 can select a desired icon. It is also possible to press the reference button 664 to display another icon. When administrator 8 presses the duplicate request button 663, the app ID, app name, and icon of the source app are sent to the information processing system 10. The information processing system 10 receives these, and the app management unit 49 uses the app processing information 53 and screen information 54 of the source app to newly register an app. Among the app processing information 53, non-duplicate app IDs are numbered, and the app name set by administrator 8 is set. The screen information is the same as that of the source app, but among these, the icon is the one selected by administrator 8. The app processing information, shared apps, and items of the external service system 40 are the same as those of the source app. For example, when users can identify apps by app names etc., it is not necessary to set different icons for all apps.

[0257] In addition, in this embodiment, although a new image processing area is added along with the duplication of the application, instead of duplicating the application, the administrator 8 may perform a setting to associate the addition of a new image processing area with identification information that accepts selection on the device in the form of adding settings or adding icons.

[0258] FIG. 34 is an example of a sequence diagram showing the procedure by which the information processing system 10 registers the duplication of an application.

[0259] S51: The administrator 8 inputs the authentication information of the information processing system 10 into the terminal device 30. The operation reception unit 33 of the terminal device 30 receives the input of the authentication information, and the first communication unit 31 transmits the authentication information to the information processing system 10, whereby the administrator 8 logs in to the information processing system 10.

[0260] S52: Upon logging in, the terminal screen providing unit 42 of the information processing system 10 transmits the screen information of the initial screen to the terminal device 30 via the communication unit 19. Thereafter, in response to the operation of the administrator 8, the terminal screen providing unit 42 transmits the screen information of the screen shown in FIG. 33 to the terminal device 30.

[0261] S53: Since the administrator 8 inputs the application name, etc. into the terminal device 30, the operation reception unit 33 receives the input.

[0262] S54: The first communication unit 31 of the terminal device 30 transmits the application ID, application name, and icon of the original application to be duplicated to the information processing system 10.

[0263] S55: The communication unit 19 of the information processing system 10 receives these, and the application management unit 49 uses the application processing information 53 and screen information 54 of the original application to newly register an application.

[0264] In this way, the administrator 8 can duplicate an application. The administrator 8 can associate a setting file with the newly generated application by duplication and can acquire data from a predetermined area of a desired form.

[0265] The administrator 8 can create a plurality of new applications from a single source application, and the information processing system 10 can accept the selection of a different configuration file for each of the plurality of new applications. Since it is only necessary to assign some identification information to the same processing flow, the duplication of applications is just an example, and identification information may be assigned to the same processing flow in the form of adding icons or adding settings.

[0266] <Obtaining Data from Forms Using the Association between Applications and Configuration Files> After the association between the application and the configuration file is completed, the information processing system 10 can obtain data from the form by having the user read the form on the device 20.

[0267] Referring to FIGS. 35 and 36, the acquisition of data from the form will be described. FIG. 35 is a diagram showing an example of screen transition displayed on the device 20.

[0268] (1) As shown in FIG. 35(a), the user causes the device 20 to display the application list screen. For example, when the user wants to settle expenses, the user selects the icon of the expense settlement application. Although the user knows which form corresponds to which application, it is preferable that the application name displayed on the application icon 690 matches the name of the form (manuscript). This application name may also be called the icon name or the setting name. That is, it is preferable that the name of the form is included in the icon 690. Further, it is preferable that the icon 690 includes a thumbnail of the form. It becomes easier to prevent the user from selecting an icon that does not correspond to the form. Alternatively, when the user presses the icon 690, a thumbnail of the form may be displayed. The user can check the icon 690 by enlarging or partially displaying the thumbnail.

[0269] Also, a link may be embedded in the icon 690. The link has, for example, a URL including the application ID, or a URL associated with the application ID on the information processing system side. By pressing the icon 690, the device 20 connects to the URL and notifies the information processing system 10 of the application ID.

[0270] Also, when the user selects a form not supported by the icon 690, the device 20 may warn the user. For example, the device 20 compares the image data of the read form with the image data associated with the icon 690, and when the two are different, it queries the user by displaying a message asking whether to continue the process. The information processing system 10 may perform the comparison of the image data. As a comparison method, for example, there is a method in which the device 20 or the information processing system 10 performs pattern matching on the image data of the form read with the image data associated with the icon 690. Also, when an app ID is printed on the form with a barcode or the like, there is a method in which the device 20 or the information processing system 10 compares this app ID with the app ID associated with the icon 690.

[0271] Also, what the user selects may be an icon, and what the device 20 sends to the information processing system 10 may be a string such as an app ID or a setting number. That is, the thing itself selected does not have to be an app ID (identification information).

[0272] (2) When the device 20 scans the expense settlement document, the user prepares the expense settlement document (Fig. 35(b)). The expense settlement document may be prepared by the user by handwritten, or may be printed out by the device 20. It does not have to be printed out by the user himself / herself (publicly issued documents, documents sent from other companies, etc.).

[0273] (3) The app selected by the user displays the screen of Fig. 35(c) based on the screen information 54. The screen of Fig. 35(c) is the same as Fig. 24. The user sets the printed expense settlement document on the contact glass and presses the start button 613 to let the device 20 read it.

[0274] The image data of the form is sent to the information processing system 10, and a file is output by the following processing.

[0275] FIG. 36 is a diagram for explaining a file stored in the external service system 40. FIG. 36(a) shows an example of the upload destination folder of the external service system 40. That is, the file output by the information processing system 10 is stored in the upload destination folder set by the administrator 8 in FIG. 29. The file hierarchy of "My Drive>Expense Settlement Document Storage" in FIG. 36(a) is an example, and the administrator 8 can specify a folder with access rights.

[0276] FIG. 36(b) shows two files stored in the upload destination folder. Two files are generated by one read operation. A. CSV file 681 in which the data obtained from the form area is stored. B. PDF file 682 containing the image data of the form.

[0277] The CSV files created by the same application will be separate each time a read operation is performed. Since the file name is attached with a serial number and a date, the file will not be overwritten. On the other hand, the data obtained by the same application may be added to the same CSV file even if the timing of the read operation is different. This method will be described later. By doing so, the person in charge of general affairs etc. can perform settlement etc. on the data of each employee by simply accessing one CSV file.

[0278] <<When the device does not scan the form>> In the description of FIG. 35, the expense settlement document was scanned by the device 20, but the information processing system 10 can receive the image data of the form from the terminal device 30 etc. and obtain the data from the area of the image data. In this case, the terminal device 30 transmits the image data of the form to the information processing system 10 as follows.

[0279] · The user uses the word processor operating on the terminal device 30 to display the expense settlement statement and inputs the necessary items. Therefore, the necessary items are in character code. The user connects the terminal device 30 to the information processing system 10, logs in, and displays the application list. Since the screen of the terminal device 30 is the same as the screen in Fig. 35(a), the user selects the desired application. The terminal screen providing unit 42 of the information processing system 10 transmits screen information capable of receiving the expense settlement statement to the terminal device 30. When the user selects a PDF file or the like containing the image data of the expense settlement statement on this screen, the terminal device 30 can transmit the PDF file to the information processing system 10.

[0280] Therefore, the user does not need to print out the form, and the information processing system 10 can obtain data from the form. In this case, the processing of the device 20 in the process shown in Fig. 37 described later is performed by the terminal device 30.

[0281] <Operation procedure> Fig. 37 is an example of a sequence diagram in which the information processing system 10 receives a PDF file containing the image data of the form from the device 20 and obtains the acquisition data from the area.

[0282] S61: The user who has printed out the form operates the device 20 to input his / her authentication information in the information processing system 10. The operation receiving unit 23 of the device 20 receives the input of the authentication information, and the second communication unit 21 transmits the authentication information to the information processing system 10. As a result, the user logs in to the information processing system 10. By logging in, the display control unit 22 of the device 20 displays a list of applications for which the user has permission to use (Fig. 35(a)). The user selects an application with an application name that matches, for example, the name of the printed form. The operation receiving unit 23 receives the selection.

[0283] S62: The second communication unit 21 of the device 20 transmits an application screen acquisition request to the information processing system 10 by specifying the application ID.

[0284] S63: The communication unit 19 of the information processing system 10 receives an app screen acquisition request, and the device screen providing unit 47 acquires the screen information 54 of the app specified by the app ID from the app management unit 49. The screen information of the app screen of the app is transmitted to the device 20. Thereby, the display control unit 22 of the device 20 displays the app screen (Fig. 35(c)).

[0285] S64: The user sets the printed form on the contact glass and presses the start button 613. The operation reception unit 23 of the device 20 receives the press, the image data generation unit 24 generates the image data of the form, and converts this image data into a PDF file. The form may be multiple sheets.

[0286] S65: The second communication unit 21 of the device 20 designates the app ID and the PDF file of the app and requests the information processing system 10 to execute the app.

[0287] S66: The communication unit 19 of the information processing system 10 receives the app execution request, and the app execution unit 46 starts the execution of the app. The app execution unit 46 designates the app ID and requests the input / output service unit 16 to execute the app. The input / output service unit 16 acquires the app processing information 53 via the app management unit 49. Thereby, the processing flow of the app is obtained. Also, the input / output processing unit 48 acquires the app data 43 associated with the app ID from the app data storage unit 13. Thereby, the input / output processing unit 48 can identify the configuration file associated with the app.

[0288] The input / output processing unit 48 requests the external service processing unit 62 to acquire the configuration file based on the app data 43 and the app processing information 53.

[0289] S67: The external service processing unit 62 connects to the URL in the fileName item registered in the app data 43 and acquires a configuration file from the external service system 40. Note that when the app for which execution is requested is not a shared app, the external service processing unit 62 logs in to the external service system 40 using the authentication information of the user in the external service system 40. When the app for which execution is requested is a shared app, the external service processing unit 62 logs in to the external service system 40 using the authentication information of the administrator 8 in the external service system 40.

[0290] S68: Next, the input / output processing unit 48 requests the data storage unit 66 to acquire area information from the configuration file based on the app processing information 53.

[0291] S69: The data storage unit 66 acquires all area information from the predetermined rows and columns of the configuration file and returns it to the input / output processing unit 48. Since the area information is registered for each process, the area information is acquired as many times as the number of processes.

[0292] S70: Next, the input / output processing unit 48 requests the area identification unit 67 to acquire (crop) the area from the image data by specifying the image data and the area information based on the app processing information 53.

[0293] S71: The area identification unit 67 identifies and acquires the processing target area specified by the area information from the image data, and returns the image of the processing target area to the input / output processing unit 48 as many times as the number of processes. Note that for the processing target areas of the OCR process and the OMR process, the area identification unit 67 does not have to acquire (crop) the processing target area from the image data. In this case, the OCR processing unit 68 and the OMR processing unit 71 may perform the OCR process or the OMR process on the processing target area of the image data specified by the area information.

[0294] When there are multiple processing target areas for the OCR process, steps S72 and S73 are repeatedly executed as many times as the number of processing target areas.

[0295] S72: Based on the application processing information 53, the input / output processing unit 48 designates an image of the processing target area and requests the OCR processing unit 68 to perform OCR processing on the processing target area. At this time, the input / output processing unit 48 may designate the format and OCR language acquired from the setting file in association with the processing target area.

[0296] S73: The OCR processing unit 68 performs OCR processing on the processing target area using a recognition engine corresponding to the format and OCR language. The OCR processing unit 68 returns the data (OCR processing result) converted into the character code to the input / output processing unit 48.

[0297] Note that the recognition engine is an identification machine that learns the correspondence between, for example, an image of a character and a character code using learning data in which the two are set, and outputs an OCR processing result (character code) from the image. The simpler the learning data, such as only Japanese, only English, only numerical values, or only dates, the higher the identification accuracy of the identification machine. On the other hand, when a plurality of Japanese, English, numerical values, and dates are mixed in the learning data, various data can be recognized, but the identification accuracy tends to decrease. Therefore, the OCR processing unit 68 can improve the recognition accuracy by using a recognition engine corresponding to the format and OCR language.

[0298] Here, machine learning is a technology for enabling a computer to acquire learning ability like a human. It refers to a technology in which a computer autonomously generates an algorithm necessary for judgment such as data identification from learning data taken in advance, and applies this to new data for prediction. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning methods. Furthermore, a learning method combining these learning methods may also be used, and the learning method for machine learning is not limited. Also, the recognition engine does not have to be generated by the learning effect of machine learning.

[0299] Next, when there are a plurality of processing target areas for OMR processing, steps S74 and S75 are repeatedly executed the number of times corresponding to the number of processing target areas.

[0300] S74: Based on the application processing information 53, the input / output processing unit 48 designates an image of the processing target area and requests the OMR processing unit 71 to perform OMR processing on the processing target area.

[0301] S75: The OMR processing unit 71 performs OMR processing on the processing target area. The OMR processing unit 71 returns the acquired data (OMR processing result) converted into the presence or absence of marks to the input / output processing unit 48.

[0302] Next, when there are multiple processing target areas for trimming processing, steps S76 and S77 are repeatedly executed for the number of processing target areas.

[0303] S76: Based on the application processing information 53, the input / output processing unit 48 designates an image of the processing target area and requests the trimming processing unit 72 to perform trimming processing on the processing target area.

[0304] S77: The trimming processing unit 72 performs trimming processing on the processing target area. The trimming processing unit 72 returns image data (trimming processing result) as the acquired data obtained from the processing target area to the input / output processing unit 48.

[0305] S78: Next, based on the application processing information 53, the input / output processing unit 48 designates the acquired data, CSV column names, CSV positions, and file names (acquired from the setting file) obtained from each processing target area, and requests the data storage unit 66 to output a file having the acquired data.

[0306] S79: The data storage unit 66 writes the CSV column names at the beginning of the column, writes the acquired data into the cells of the CSV file specified by the CSV position, and returns the CSV file to the input / output processing unit 48. The specified file name is given to the CSV file. Note that the format of the output destination file is not limited to the CSV file, and it may be a table format file, a text file, or a file of other applications.

[0307] S80: Next, based on the application processing information 53, the input / output processing unit 48 designates the OCR processing result (CSV file), the format, and the OCR language, and requests the OCR correction unit 69 to correct the OCR processing result. This format is obtained from the setting file shown in Table 4 and is set by the administrator 8.

[0308] Figure 38 is a diagram for explaining the correction of the OCR processing result. When a format is set in the setting file, the OCR correction unit 69 can correct the OCR processing result. The rules for the specified character string and the conversion character are defined in the OCR correction unit 69 as shown in Figure 38. The conversion rule 2201 in Figure 38 is the rule when the format is NUMERIC. For example, since the numerical values do not include I, B, and Z, they are corrected to 1, 8, and 2 respectively. The conversion rule 2202 in Figure 38 is the rule when the language is English. For example, since English does not include 1, 2, and 8, they are corrected to I, B, and Z respectively.

[0309] S81: Returning to Figure 37B, the OCR correction unit 69 returns the corrected OCR processing result (CSV file) to the input / output processing unit 48.

[0310] Steps S82 to S85 are executed for the number of CSV files. The number of CSV files created in one reading operation may be one case regardless of the number of forms, or the same as the number of pages of the form. Which one is adopted is set in the application.

[0311] S82: Next, based on the application processing information 53, the input / output processing unit 48 requests the external service processing unit 62 to upload the OCR processing result (CSV file). The upload destination folder is set in the application data 43.

[0312] S83: The external service processing unit 62 saves the CSV file in the upload destination folder of the external service system 40.

[0313] S84: If metadata can be set, step S84 is executed. When the external service system 40 supports setting metadata (items set by the user on the external service system, such as strings and dates), the user can set the metadata associated with the CSV file to be uploaded. The user can input the metadata from the screen displayed by the device 20. The input / output processing unit 48 requests the external service processing unit 62 to upload the metadata. The upload destination folder is the same as the CSV file.

[0314] S85: The external service processing unit 62 saves the metadata in the upload destination folder of the external service system 40.

[0315] S86: When multiple form data are read, the user may want to save them in individual files. In this case, based on the application processing information 53, the input / output processing unit 48 requests the PDF processing unit 64 to perform page splitting together with the image data of multiple pages. This process may be omitted, and in that case, the input / output processing unit 48 may save the multiple data as one file.

[0316] S87: The PDF processing unit 64 returns the PDF files decomposed page by page to the input / output processing unit 48.

[0317] S88: Based on the application processing information 53, the input / output processing unit 48 specifies the PDF file and requests the OCR processing unit 68 to perform OCR processing.

[0318] S89: The OCR processing unit 68 performs OCR processing on the PDF file and sets the character code generated by the OCR processing as transparent text in the PDF file.

[0319] Steps S90 to S93 are executed for the number of PDF files.

[0320] S90: The input / output processing unit 48 requests the external service processing unit 62 to upload a PDF file with text based on the application processing information 53. The upload destination folder is set in the application data 43.

[0321] S91: The external service processing unit 62 saves the PDF file with text in the upload destination folder of the external service system 40.

[0322] S92: When the external service system 40 supports the above metadata (items set by the user on the external service system, such as character strings, dates, etc.), the user can input the metadata from the screen displayed by the device 20. The input / output processing unit 48 requests the external service processing unit 62 to upload the metadata. The upload destination folder is the same as that of the PDF file.

[0323] S93: The external service processing unit 62 saves the metadata in the upload destination folder of the external service system 40.

[0324] As described above, a CSV file and a PDF file different from the CSV file are registered in the upload destination folder of the external service system 40.

[0325] In FIG. 37, the image data to be processed is a PDF file, but the image data to be processed may be image data in any format such as JPEG, PNG, TIFF, BMP, etc.

[0326] In this embodiment, it is assumed that the device 20 scans and processes a large number of forms at once. On the other hand, the user may use the same format of form only once, or when the processing area changes each time, registering the area information in the setting file only becomes a burden for the administrator. For this reason, the operation of registering the area information may be omitted, and the area of the form filled in by the user may be directly reorganized and read by the device 20. The information processing system 10 performs processing on each area according to the correspondence between the color registered in the setting file and the processing.

[0327] <Display of CSV Files and PDF Files Registered in External Service System> When the person in charge of the company calculates the expenses of employees, etc., the terminal device 30 is operated to connect to the external service system 40. Since the upload destination folder is notified to the person in charge, the CSV file and the PDF file can be acquired by the terminal device 30.

[0328] FIG. 39 is an example of a file list screen 670 of the external service system 40 displayed by the terminal device 30. As shown in FIG. 39(a), a PDF file 671 and a CSV file 672 are stored in the upload destination folder. FIG. 39(b) shows the content of the CSV file 672 displayed by the terminal device 30. That is, FIG. 39(b) is data output from the form. In FIG. 39(b), data acquired from two areas is stored. The CSV column names set in the setting file are also set in each column, and the user can determine what data each column is.

[0329] Also, as shown in FIG. 40, it is preferable that the device 20 also displays the processing result. FIG. 40 is an example of an output data display screen 691 displayed by the device 20. The output data display screen 691 displays the executed application name 692 and a list 693 of files output by the application. These are transmitted from the information processing system 10 to the device 20. The CSV file contains the content of the CSV file, and when the user presses the CSV file, the device 20 can display the data 694 acquired from the form area and stored in the CSV file.

[0330] The user who has confirmed the file name and data can determine whether the form has been processed correctly.

[0331] <When Multiple Forms are Read> Referring to FIG. 41, the case where a plurality of forms are read in a single reading operation will be described. FIG. 41 is a diagram showing a plurality of forms and the output file. As shown in FIG. 41(a), when the application is executed, a plurality of forms are read in a single reading operation. The single reading operation refers to the case where the start button 613 is pressed only once. The device 20 reads the plurality of forms in order, and the information processing system 10 acquires a plurality of data from the same area of the image data of each form.

[0332] FIG. 41(b) is a diagram showing a CSV file when a plurality of forms are read in a single reading operation. The first row is the data acquired from the first form, and the second row is the data acquired from the second form. In this way, since the plurality of data of the plurality of forms are summarized in one CSV file, it becomes easier for the person in charge to perform tabulation and graphing.

[0333] Note that when a plurality of forms are read in a single reading operation, the data storage unit 66 increments the CSV position (start row) of the setting file one by one. That is, the data storage unit 66 may write the data in the next row for each form.

[0334] When a plurality of forms are read in a single reading operation, it is also possible for the information processing system 10 to output CSV files equal in number to the number of forms.

[0335] Also, when a plurality of forms are read in a plurality of reading operations (when read at different timings), it is also possible for the information processing system 10 to output the data to one same CSV file. There are cases where the external service system 40 merges and cases where the information processing system 10 merges.

[0336] · When the external service system 40 merges, the external service processing unit 62 requests the external service system 40 to merge the CSV files. If the applications are the same, the same string (e.g., expense settlement statement) is included in the file names of the CSV files, so the external service processing unit 62 can request the merge of the CSV files by specifying the file names. The external service system 40 adds the data in the same column number of the CSV file for which the merge is requested to the same column number of the already saved CSV file.

[0337] · When the information processing system 10 merges, the external service processing unit 62 downloads the CSV file from the external service system 40 by specifying the file name. The external service processing unit 62 adds the data in the same column number of the newly created CSV file to the same column number of the acquired CSV file. The external service processing unit 62 uploads the merged CSV file to the external service system 40.

[0338] In this way, the information processing system 10 can acquire the same area from the received plurality of image data, the OCR processing unit 68 performs OCR processing on the same area in the plurality of image data, and the data storage unit 66 can output the plurality of data generated from the same area in the plurality of image data to one file.

[0339] <Main effects> As described above, in the data output system 100 of the present embodiment, the administrator 8 can associate the marks and processes on the registration form with a marker pen. Further, the device 20 reads the form with the marked area, and the information processing system 10 stores the process and area information for each form. Then, the area is acquired from the image data of the form transmitted from the device 20, and the information processing system 10 performs the process set in the area. As a result, it becomes possible to easily create a file by extracting only a specific area from a plurality of forms of the same type.

Example

[0340] In this embodiment, an information processing system 10 that recognizes different recognition targets by OCR according to the color of an area will be described.

[0341] With reference to FIGS. 42 and 43, an outline of the processing or operations performed by the data output system 100 will be described. In the description of FIGS. 42 and 43, the differences from FIGS. 1 and 2 will mainly be described.

[0342] In the process of FIG. 42, the process of (2) is different from that of FIG. 1.

[0343] (2) The administrator 8 of the data output system 100 prints the registration form registered in the information processing system 10 by the device 20 and colors the color corresponding to the process with a marker pen. For example, the administrator 8 can associate any color with the process, such as "OCR for Chinese characters" being green, "OCR for dates" being blue, "OCR for alphanumeric characters" being red, etc. The device 20 scans the registration form to generate image data (first image data) and transmits it to the information processing system 10.

[0344] Subsequently, it will be described with reference to FIG. 43. FIG. 43(a) is a diagram for explaining the setting of area information in the form, and FIG. 43(b) is a diagram for explaining the acquisition of data from the form using the area information. Since the process of FIG. 43 is the same as that of FIG. 2, the main differences will be described.

[0345] (4) The administrator 8 uses the form 7 that users within the organization use daily and designates with a marker pen which process ("OCR for Chinese characters", "OCR for dates", "OCR for alphanumeric characters") to be performed on which area of the form 7.

[0346] The subsequent processes (5) to (11) may be the same as those of FIG. 2.

[0347] In addition to the effects of Example 1, the data output system 100 of this embodiment allows the administrator 8 to register more detailed processing content and marks (colors). In OCR, it is known that the recognition rate varies greatly depending on whether or not the recognition target is restricted. For example, when the OCR processing unit 68 performs OCR on the postal code "〒", if it is not known in advance that the recognition target is the postal code, it is often misrecognized as "T". In this embodiment, since the OCR recognition target is set for each area, the OCR recognition accuracy can be improved.

[0348] Also, although the administrator may use a website to set what is entered in the form area, in this embodiment, the OCR recognition target can be set for each area by a simple method in which the administrator marks on the registration form.

[0349] <Supplementary Note on the Correspondence between Color and Processing> The series of processes described in FIGS. 42 and 43 will be supplemented by dividing them into Operations 1 to 3. FIG. 44 is a diagram for explaining the registration of the correspondence between color and processing (Operation 1) using the registration form.

[0350] FIG. 44(a) is the registration form before coloring.

[0351] FIG. 44(b) is the registration form after coloring. The device 20 scans the registration form in FIG. 44(b).

[0352] FIG. 44(c) is a part of the setting file in which color and processing are associated. Two methods can be considered for associating color and processing.

[0353] 1: The information processing system side pre-holds or associates the coordinates from which the color is obtained (within the registration form) and the processing obtained from those coordinates. Example of Pre-parameters · The color extracted at the coordinates (X coordinate: 100, Y coordinate: 200 for X and 1, 50 for Y) performs OCR on Chinese characters · The color extracted at the coordinates (X coordinate: 100, Y coordinate: 200 for X and 51, 100 for Y) performs OCR on the date and time · The color extracted at the coordinates (X coordinate: 100, 200; Y coordinate: 101, 150) is alphanumeric for OCR. 2: The administrator associates the color with the process on the management site. Figure 45 is an example of the management site 150 for associating colors with processes. The management site 150 is provided by the information processing system 10 to the terminal device as a web page or a web application. The administrator can register the color of the process with the radio button 171. In this case, the colors of the marker pens used by the user are limited to three.

[0354] In any method, as shown in Figure 44(c), the numerical value identifying the marker color (green) is associated with the fact of performing OCR on Chinese characters. The numerical value identifying the marker color (blue) is associated with the fact of performing OCR on the date. The numerical value identifying the marker color (red) is associated with the fact of performing OCR on alphanumeric characters.

[0355] Figure 46 is a diagram for explaining the correspondence between area information and processes (operation 2) using the original form of the form. Figure 46(a) is the original form of the form.

[0356] Figure 46(b) is the form after coloring. The device 20 scans the form in Figure 46(b). Figure 46(c) is a part of the setting file in which the area information and the process are associated.

[0357] As shown in Figure 46(d), since the color and the process are associated in the setting file, the information processing system 10 detects the color of the registration paper (already registered in the setting file) from the form, and associates the coordinates of the area where this color is detected with the process in the setting file as area information.

[0358] Therefore, in this embodiment, the administrator can change the content of the OCR process for the area with the color applied to the area of the form. Ex1: Coloring with blue → OCR for Chinese characters, coloring with red → OCR for the date Ex2: Coloring with blue → OCR for postal codes, coloring with red → OCR for URLs FIG. 47 is a diagram for explaining the processing (operation 3) on the area in the form using the form. FIG. 47(a) is the original sheet filled in with necessary items by the user.

[0359] FIG. 47(b) shows the processing performed on each area of the form. Since the processing of performing OCR on Chinese characters as OCR1 is registered in the name area 172 of the form, the name is regarded as Chinese characters (including katakana and hiragana) and OCR is performed. Since the processing of performing OCR on the date and time as OCR2 is registered in the application date area 173 of the form, the application date is regarded as the date and time and OCR is performed. Since the processing of performing OCR on alphanumeric characters as OCR3 is registered in the employee number area 174 of the form, the employee number is regarded as alphanumeric characters and OCR is performed.

[0360] <Regarding the functions> In this embodiment, it will be described assuming that the hardware configuration diagrams of FIGS. 8 and 9 described in the above embodiment and the functional configuration diagram shown in FIG. 10 can be used.

[0361] FIG. 48 is a diagram showing the functional configuration of the OCR processing unit 68 in this embodiment. The OCR processing unit 68 has a first OCR processing unit 68a, a second OCR processing unit 68b, and a third OCR processing unit 68c.

[0362] The first OCR processing unit 68a performs character recognition of Chinese characters. The second OCR processing unit 68b performs character recognition of dates. The third OCR processing unit 68c performs character recognition of alphanumeric characters. The first OCR processing unit 68a to the third OCR processing unit 68c are each an identification device that has learned the correspondence between the images of Chinese characters, dates, or alphanumeric characters and character codes by machine learning.

[0363] Machine learning is a technology for enabling a computer to acquire learning capabilities similar to those of humans. It refers to a technology in which a computer autonomously generates algorithms necessary for judgments such as data discrimination from pre-imported learning data and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning. Furthermore, a learning method that combines these learning methods may also be used, and the learning method for machine learning is not limited.

[0364] Also, as shown in Table 5, in this embodiment, the color of the region and the OCR language are associated in the configuration file.

[0365]

Table 5

[0366] The first OCR processing unit 68a performs character recognition on Chinese characters when the region is green. The second OCR processing unit 68b performs character recognition on the date when the region is blue. The third OCR processing unit 68c performs character recognition on alphanumeric characters when the region is red.

[0367] Therefore, in this embodiment, since the recognition target is restricted according to the Chinese characters, date, and alphanumeric characters described in the region, the recognition accuracy can be improved.

[0368] Note that the types of OCR (recognition targets) are not limited to these, and may also include postal codes, email addresses, alphanumeric characters in half-width, numbers (in the form without hyphens between numbers such as phone numbers and postal codes), numbers (in the form with hyphens between numbers such as phone numbers and postal codes), or URLs, etc. Also, in this embodiment, when referring to Chinese characters, it includes strings of hiragana and katakana. Further, various languages (English, French, German, Chinese, Arabic, etc.) may be recognition targets.

[0369] <App processing information> Next, the details of the app processing information will be described. The same applies to FIG. 11 in this embodiment.

[0370] FIG. 49 is a diagram schematically showing a series of processes performed by the app. In the description of FIG. 49, the differences from FIG. 12 will be mainly described. FIG. 49(c) shows a series of processes of an app (for example, an expense settlement form app) that outputs data from the form area. In this embodiment, OCR of Chinese characters, date and time, and alphanumeric characters is performed according to the OCR language (recognition target) set in the setting file.

[0371] <Registration of correspondence between color and process in the setting file> Next, with reference to FIG. 50, the setting of the correspondence between color and process by the administrator 8 and the creation of the setting file will be described.

[0372] FIG. 50 is a diagram for explaining a method of setting the correspondence between color and process using a registration form. In the description of FIG. 50, the differences from FIG. 13 will be mainly described. FIG. 50(a) is the registration form before the administrator 8 colors it with a marker pen. The original paper of the registration form is registered in the external service system 40.

[0373] The items of each column (first column: marker color, second column: process) are described in the first row of the registration form. In this embodiment, there are three types of "processes" (OCR for Chinese characters, OCR for date and time, OCR for alphanumeric characters), and the processes of "OCR for Chinese characters in the second row, OCR for date and time in the third row, and OCR for alphanumeric characters in the fourth row" are preset in the registration form. The cell 130 of the marker color in the original state of the registration form is blank.

[0374] Figure 50(b) is a diagram in which the administrator 8 colored the cell 130 with a marker pen for each process. The administrator 8 shall use, for example, green, blue, and red marker pens among the marker pens he / she owns. The administrator 8 assigns green to Chinese characters for OCR, blue to date and time for OCR, and red to alphanumeric characters for OCR. A predetermined interval is provided between the lines of the original registration form so that the colors do not bleed and cause color mixing.

[0375] The administrator 8 colors the cell 130 of the registration form with the marker pen he / she owns according to the assignment of the desired marker color determined by himself / herself and each process. When the device 20 scans the registration form, the correspondence between the marker color and the process is registered in the information processing system 10.

[0376] <Example of the device screen> The screen example of the device 20 may be the same as that in FIG. 14. The sequence diagram showing the procedure for the administrator 8 to use the registration form to register the correspondence between the color and the process in the configuration file may be the same as that in FIG. 15. However, the process in step S12 is different.

[0377] S12: The color detection unit 73 detects color information from the predetermined cell 130 of the image data.

[0378] Figure 51 is a diagram schematically showing the detection of color information. In the description of Figure 51, the differences from Figure 16 will be mainly described. As shown in Figure 51(a), the process of OCR for Chinese characters is determined by the color within the rectangle of two position coordinates AB (an example of position information) indicating the diagonal, the process of OCR for date and time is determined by the color within the rectangle of position coordinates CD in the same way, and the process of OCR for alphanumeric characters is determined by the color within the rectangle of position coordinates EF.

[0379] As shown in FIG. 51(b), the barcode 101 or the application data management unit 41 holds the position coordinates of the cells 130 to be processed and colored in association with each other. The color detection unit 73 acquires the H value of HSV from each cell 130 based on the position coordinates in FIG. 51(b). When a color corresponding to red (H = (0 to 60°, 301 to 360°)) is detected from the cell 130 associated with alphanumeric characters for OCR, the color detection unit 73 sets H = (0 to 60°, 301 to 360°) in HSV and registers it in the setting file in association with the alphanumeric characters for OCR. When a color corresponding to green (H = (61 to 180°)) is detected from the cell 130 associated with Chinese characters for OCR, the color detection unit 73 registers H = (61 to 180°) in the setting file in association with the Chinese characters for OCR. When a color corresponding to blue (H = (181 to 300°)) is detected from the cell 130 associated with date and time for OCR, the color detection unit 73 registers H = (181 to 300°) in the setting file in association with the date and time for OCR.

[0380] Note that as a method of associating colors with processes, there is also a method in which the administrator uses the management site 150 described in FIG. 45.

[0381] <<Specification of File Name by Process>> Also in this embodiment, the administrator 8 can set the file name of the CSV file by coloring the registration form.

[0382] FIG. 52 shows an example of use of a registration form in which the process is applied to the setting of the file name. In the description of FIG. 52, the differences from FIG. 17 will mainly be described. The process being limited to OCR is the same as in FIG. 17. In the registration form of FIG. 52(a), OCR1 and OCR2 are set as processes, and the order "file name 1" and "file name 2" to be used for the file name are set.

[0383] As shown in FIG. 52(b), the administrator 8 designates with a marker pen the color of the marker pen (for example, green) to be used for file name 1 and the color of the marker pen (for example, blue) to be used for file name 2.

[0384] As shown in Fig. 52(c), a numerical value indicating color and a process (used for the file name) are associated with the configuration file. The two methods of associating color and process may be the same as those in Fig. 17.

[0385] Fig. 53 is an example of the management site 152 for associating color and process. The administrator 8 can register the color used for specifying the file name with the radio button 153. In this case, the colors of the marker pens used by the user are limited to three.

[0386] As shown in Fig. 52(c), a numerical value identifying the marker color (green) is associated with the file name 1. A numerical value identifying the marker color (blue) is associated with the file name 2. Note that the file names 1 and 2 are the order in which they are used for the file name.

[0387] Fig. 54 is an example of a form in which the administrator 8 has designated areas at two locations with a marker pen. Fig. 54(a) shows the original form, and Fig. 54(b) shows the form colored with a marker pen. When the device 20 scans the form in Fig. 54(b), the information processing system 10 identifies the colored areas on the form and registers the area information in the configuration file.

[0388] In Fig. 54(b), the area 172 for "Name" is designated with a green marker pen, and the area 173 for the application date is designated with a blue marker pen. For example, if the name is "Rikou Tarou" and the application date is "2020 / 10 / 10", when the device 20 scans the form in Fig. 54(b), the file name of the CSV file will be "RikouTarou20201010.csv". Note that the number of marker pen colors specified for the file name may be three or more.

[0389] Also, it is not always necessary to register up to the order of the areas (file 1, 2) used for the file name. In this case, the data storage unit 66 may set the file name in the order of color registration in the configuration file. Therefore, it is not necessary to change the color of the marker pen for each cell of the registration form.

[0390] <<Modification Example of the Registration Form>> As shown in FIG. 55, the CSV position may be registered in advance on the registration form. In the description of FIG. 55, the differences from FIG. 20 will mainly be explained. FIG. 55(a) is the registration form before coloring. Since this registration form has columns for CSV column name 154 and CSV position 155, there is no need for the administrator to manually set the CSV position in the setting file.

[0391] FIG. 55(b) is the registration form after coloring. The device 20 scans the registration form of FIG. 55(b). FIG. 55(c) is a part of the setting file in which colors, processes, and area information are associated.

[0392] <Registration of Area Information> Subsequently, referring to FIG. 56 and the like, the registration of area information will be described. FIG. 56 is a diagram for explaining the method of marking areas on a form. First, FIG. 56(a) is the form (original paper) before designating an area with a marker pen. For the original paper of the form, the administrator 8 designates an area with a marker pen of the color registered for the area where the process registered on the registration form is to be executed. For example, the administrator 8 wishes to set the name area 172 as the area for OCR of Chinese characters, the application date area 173 as the area for OCR of date and time, and the employee number area 174 as the area for OCR of alphanumeric characters.

[0393] FIG. 56(b) is the form paper colored in the area with a marker pen. The employee area 172 is colored with a green marker pen. The application date area 173 is colored with a blue marker pen. The employee number area 174 is colored with a red marker pen. By scanning this form paper with the device 20, the association between the area and the process for the form is registered in the setting file.

[0394] According to the setting file in Table 5, the first OCR processing unit 68a performs OCR processing on the name area 172 with Chinese characters as the recognition target to generate acquisition data. The second OCR processing unit 68b performs OCR processing on the alphanumeric area 173 with date and time as the recognition target to generate acquisition data. The third OCR processing unit 68c performs OCR processing on the employee number area 174 with alphanumeric characters as the recognition target to generate acquisition data.

[0395] Figure 56(c) is an example of a form filled in by the user with necessary information. From the coloring state in Figure 56(b), "Taro Riko", "2020 / 10 / 10", and "50412" become the acquired data.

[0396] <Area Registration Screen> Figure 57 is an example of the area registration screen 610 displayed by the device 20 when registering area information. In the description of Figure 57, the differences from Figure 24 will be mainly explained. In Figure 57, the area marked on the form is different from that in Figure 24.

[0397] Note that the procedure for the information processing system 10 to obtain area information from the form marked by the administrator 8 and create a configuration file may be the same as that in Figure 25. The same applies to Figures 26 and 27 as in Example 1.

[0398] Figure 58 is a diagram for explaining the correspondence between the configuration file and the form. In the example of Figure 58, it is set that the data obtained from the area information of ID=1 corresponds to "Name" in the CSV column name. "Name" is used as the first one in the file name. It is set that the data obtained from the area information of ID=2 corresponds to "Application Date" in the CSV column name. Also, since the CSV position of the application date is set to column 3, row 2, the data obtained from the area information of ID=2 is written to the 3rd column and 2nd row of the file. It is set that the data obtained from the area information of ID=3 corresponds to "Employee Number" in the CSV column name. Also, since the CSV position of the remarks is set to column 4, row 2, the data obtained from the area information of ID=3 is written to the 4th column and 2nd row of the file.

[0399] After this, Figures 29 to 36 may be the same as in Example 1.

[0400] <Operation Procedure> Figure 59 is an example of a sequence diagram in which the information processing system 10 receives a PDF file including the image data of the form from the device 20 and obtains the acquired data from the area. In the description of Figure 59, the differences from Figure 37 will be mainly explained.

[0401] First, the processing of steps S61 to S71 may be the same as that in FIG. 37.

[0402] When there are a plurality of processing target areas for OCR processing, steps S72 and S73 are repeatedly executed for the number of processing target areas.

[0403] S72: The input / output processing unit 48 designates an image of the processing target area based on the application processing information 53 and requests the OCR processing unit 68 to perform OCR processing on the processing target area. In this embodiment, the input / output processing unit 48 designates the format and OCR language (Chinese characters, date and time, alphanumeric characters) obtained from the setting file in association with the processing target area.

[0404] S73: The OCR processing unit 68 performs OCR processing on the processing target area using a recognition engine corresponding to the format and OCR language. That is, the first OCR processing unit 68a recognizes Chinese characters, the second OCR processing unit 68b recognizes dates, and the third OCR processing unit 68c recognizes alphanumeric characters. The OCR processing unit 68 returns the data (OCR processing result) converted into a character code to the input / output processing unit 48.

[0405] S78, S79: Next, the input / output processing unit 48 designates the acquired data, CSV column name, CSV position, and file name (acquired from the setting file) acquired from each processing target area based on the application processing information 53, and requests the data storage unit 66 to output a file having the acquired data.

[0406] Thereafter, the processing of steps S80 to S93 may be the same as that in FIG. 37.

[0407] As described above, it becomes possible to display the CSV file and the PDF file registered in the external service system, and these display methods may be the same as those in the first embodiment.

[0408] <Main effects> As described above, in addition to the effects of Example 1, the data output system 100 of this embodiment enables the administrator 8 to register more detailed processing content and marks (colors). Since the OCR recognition target is set for each area, the OCR recognition accuracy can be improved. Also, in this embodiment, the OCR recognition target can be set for each area by a simple method in which the administrator marks on the registration form.

[0409] <Other application examples> As described above, the best mode for carrying out the present invention has been described using examples. However, the present invention is not limited to such examples, and various modifications and substitutions can be made without departing from the gist of the present invention.

[0410] For example, in this embodiment, the information processing system 10 outputs the data acquired from the area of the form to a file. However, instead of creating a file, it may be sent by email or displayed on the device 20.

[0411] Also, the configuration examples such as in FIG. 10 are divided according to the main functions in order to facilitate the understanding of the processing by the terminal device 30, the device 20, and the information processing system 10. The present invention is not limited by the way of dividing the processing units and their names. The processing of the terminal device 30, the device 20, and the information processing system 10 can be further divided into more processing units according to the processing content. Also, one processing unit can be divided so as to include more processing.

[0412] Also, the device group described in the examples only shows one of a plurality of computing environments for implementing the embodiments disclosed in this specification. In one embodiment, the information processing system 10 includes a plurality of computing devices such as a server cluster. The plurality of computing devices are configured to communicate with each other via an arbitrary type of communication link including a network or a shared memory, and implement the processing disclosed in this specification.

[0413] Furthermore, the information processing system 10 can be configured to share the processing steps disclosed in the present embodiment, for example, FIGS. 37 and the like, in various combinations. For example, the processes executed by a predetermined unit can be executed by a plurality of information processing devices included in the information processing system 10. Also, the information processing system 10 may be integrated into one server device or may be divided into a plurality of devices.

[0414] Each function of the embodiment described above can be realized by one or a plurality of processing circuits. Here, the "processing circuit" in this specification refers to a processor programmed to execute each function by software like a processor implemented by an electronic circuit, an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array) designed to execute each function described above, and devices such as conventional circuit modules.

Explanation of Reference Numerals

[0415] 10 Information processing system 20 Device 30 Terminal device 100 Data output system

Prior Art Documents

Patent Documents

[0416]

Patent Document 1

Claims

1. An information processing system that is communicably connected to one or more devices and executes a predetermined process, comprising: a data storage unit that stores setting information associating a mark for identifying the content of the process with the predetermined process based on a user operation in a first device; a communication unit that receives second image data of a predetermined document including the mark added by the user from the first device; a region detection unit that detects a region including the mark from the second image data; a management unit that stores in association the position information of the detected region, the predetermined process corresponding to the mark included in the region, and the application selected by the user in the first device; a processing unit that executes a predetermined process on third image data received from a second device based on the position information and the predetermined process associated with the application selected by the second device; An information processing system, characterized by comprising the above.

2. The information processing system according to claim 1, wherein the mark is color information.

3. The information processing system according to claim 1 or 2, characterized in that the first device has a communication unit that transmits screen information of a screen for receiving the association between the mark and the process.

4. The data storage unit stores the association between the process and the mark based on the mark included in the first image data received from the first device. The information processing system according to any one of claims 1 to 3.

5. The information processing system according to claim 4, wherein the data storage unit stores the association between the process performed by the information processing system and the mark based on the position information of the mark formed in the first image data.

6. The first image data includes position information of a region corresponding to the content of the process. The information processing system according to claim 4 or 5, wherein the data storage unit stores the association between the process and the mark based on the mark formed in the region of the position information and the content of the process corresponding to the region of the position information.

7. The information processing system holds position information of a region corresponding to the content of the process in association with the identification information of the first image data. The information processing system according to any one of claims 4 to 6, wherein the data storage unit stores the process and the mark in association with each other based on the mark formed in the region of the position information and the content of the process corresponding to the region of the position information.

8. The information processing system according to any one of claims 1 to 7, wherein the process is OCR, OMR, trimming, translation, masking, or encryption.

9. The information processing system according to claim 2, wherein the process is OCR, and the data storage unit stores that different recognition targets are recognized according to the color information.

10. The information processing system according to claim 9, wherein the recognition target is at least one of a character string, a postal code, an email address, alphanumeric characters in half-width, a number (a format without a hyphen between numbers), a number (a format with a hyphen between numbers), or a URL.

11. A communication unit that receives second image data of a predetermined document from the first device, and a region detection unit that detects a region in which the mark associated with the process is formed from the second image data. The information processing system according to any one of claims 1 to 10, wherein the data storage unit stores the position information of the region in association with the process.

12. The communication unit receives the third image data of the predetermined document from the second device, and has a region identification unit that identifies a process target region based on the position information of the region from the third image data. The processing unit executes the process associated with the position information of the region on the process target region, The information processing system according to claim 11, wherein the data storage unit outputs acquisition data acquired by the processing unit processing the process target region.

13. The data storage unit outputs a file including the acquisition data, The information processing system according to claim 12, further comprising an external service processing unit that stores the file in an external service system.

14. The process designates one or more file names, The information processing system according to claim 13, wherein the data storage unit sets the acquisition data acquired by the processing unit processing the process target region as the file name of the file.

15. The information processing system according to any one of claims 1 to 14, characterized in that the first device and the second device are the same device.

16. An information processing method performed by an information processing system capable of communicating with one or more devices and executing a predetermined process, comprising: a step in which a data storage unit stores setting information associating a mark for identifying the content of the process with the predetermined process based on a user operation in a first device; a step in which a communication unit receives second image data of a predetermined document including the mark added by the user from the first device; a step in which a region detection unit detects a region including the mark from the second image data; a step in which a management unit stores in association the position information of the detected region, the predetermined process corresponding to the mark included in the region, and the application selected by the user in the first device; a step in which a processing unit executes a predetermined process on third image data received from a second device based on the position information and the predetermined process associated with the application selected by the second device; An information processing method characterized by comprising the steps.

17. An information processing system capable of communicating with one or more devices and executing a predetermined process, a data storage unit that stores setting information associating a mark for identifying the content of the process with the predetermined process based on a user operation in a first device; a communication unit that receives second image data of a predetermined document including the mark added by the user from the first device; a region detection unit that detects a region including the mark from the second image data; a management unit that stores in association the position information of the detected region, the predetermined process corresponding to the mark included in the region, and the application selected by the user in the first device; a processing unit that executes a predetermined process on third image data received from a second device based on the position information and the predetermined process associated with the application selected by the second device, A program for causing the system to function as such.

Images