Information processing device and information processing program

Abstract

To provide an image processing device, and an image processing program capable of allowing a user to select, as a method for creating definition information, from the followings; i.e. a method where a user newly creates, a method to divert the definition information created in other papers prepared in advance, and a method create using the result of classifying an accepted paper by artificial intelligence.SOLUTION: An image processing device 20 includes a CPU21. The CPU 21 receives an image of a form with an entry field for information to be entered, and presents three kinds of methods in a mode to allow a user to select therefrom; i.e. a method where a user newly creates, a method to divert the definition information created in other papers prepared in advance, and a method to create using the result of classifying the received paper by artificial intelligence.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] Patent Document 1 discloses a form definition creation device that creates a format definition of a form for character recognition and includes at least a storage unit and a control unit. The storage unit is characterized by including a format definition storage means for storing the format definition of the original form to be diverted, and an image storage means for storing an image of the destination form read by an image reading device. The control unit is characterized by including an item position search means for searching for individual read items based on the format definition of the original form on the image of the destination form, and an item matching means for moving the read items of the original form based on the search results by the item position search means.

[0003] Patent Document 2 discloses a form OCR program that causes a computer to execute a form OCR process that recognizes characters in entry columns arranged on a form from a form image obtained by reading the form. In this form, entry columns and preprinted item names are arranged for a plurality of items, and the entry columns and item names are each separated by ruled lines. This form OCR program is characterized by comprising a full-page OCR processing step and a partial OCR processing step. In the full-page OCR processing step, OCR processing is executed on the entire surface of the form image, and the positions of entry boxes that separate the entry columns, the positions of item name boxes that surround the item names, and character strings within these boxes are recognized. In addition, in the partial OCR processing step, a column identification processing step is executed to identify entry columns to be OCR-processed again based on the result information of the full-page OCR processing step, and OCR processing is executed partially on the identified entry columns.

[0004] Patent Document 3 discloses an information processing device having an extraction means for extracting a rectangular portion as a writing frame corresponding to a writing field in which information is to be written from an image of a paper sheet on which the writing field is formed, and a display control means for displaying the extraction result by the extraction means on a display means. This information processing device has an image editing means for editing the image to extract the rectangular portion as the writing frame according to a user's instruction after displaying the extraction result, and a re-extraction means for re-extracting the rectangular portion as the writing frame from the image reflecting the editing. This information processing device also has an output means for outputting definition information used to extract information to be written in the writing field, and an output means for outputting definition information indicating the correspondence between the writing frame extracted by the re-extraction means and the attributes of the information to be written in the writing field. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2015-184815 A [Patent Document 2] JP 2005-173730 A [Patent Document 3] JP 2019-159932 A Summary of the Invention [Problem to be solved by the invention]

[0006] In order to extract information to be written in a writing field from an image of a paper on which the writing field is formed, it is necessary to create definition information in advance that indicates the attributes of the information to be written in the writing field. There are three methods for creating definition information: a method in which the user creates the definition information anew; a method in which definition information created in another paper prepared in advance is reused; and a method in which the user creates the definition information using the results of classifying the received paper by artificial intelligence. However, in the existing system, only one of the above three types of creation methods is applied, and there is a problem in that the user cannot select among these creation methods.

[0007] An object of the present invention is to enable a user to select a method for creating definition information. [Means for solving the problem]

[0008] In order to achieve the above object, an information processing device according to a first aspect includes a processor, and the processor receives an image of a paper sheet having a writing field for writing information, and generates definition information indicating attributes of information to be written in the writing field, the definition information being generated anew by a user, the definition information being generated in another paper sheet prepared in advance being reused, and the definition information being generated by the received paper sheet. Categories The present invention presents three types of methods for creating a map using the results of classification by artificial intelligence, in a manner selectable by the user.

[0009] In addition, in the information processing device of the second aspect, in the information processing device of the first aspect, the processor further presents at least one of the most recommended creation method among the three methods and reference information defined for each of the three methods.

[0010] In addition, an information processing device according to a third aspect is an information processing device according to the second aspect, in which the most recommended creation method is derived using the similarity between the characteristics of the accepted paper and the characteristics of the other paper.

[0011] In addition, in an information processing device according to a fourth aspect, in the information processing device according to the third aspect, the processor presents the reusing method as the most recommended creation method when the similarity is equal to or greater than a predetermined threshold.

[0012] In addition, in an information processing device according to a fifth aspect, in the information processing device according to the third or fourth aspect, when the similarity is less than a predetermined threshold, the processor presents, as the most recommended creation method, the method which takes the shortest time to create the definition information, between the method of newly creating and the method of creating using the results classified by the artificial intelligence.

[0013] In addition, an information processing device according to a sixth aspect is an information processing device according to any one of the second to fifth aspects, in which the reference information in the newly creating method is information indicating the creation time when the definition information is created using that method.

[0014] In addition, in an information processing device according to a seventh aspect, in an information processing device according to any one of the second to sixth aspects, the reference information in the reusing method is information indicating the other paper that has the highest similarity to the characteristics of the received paper.

[0015] In addition, in an information processing device according to an eighth aspect, in an information processing device according to any one of the second to seventh aspects, the reference information in the method for creating the reference information using the results of classification by artificial intelligence is information indicating the results of classification of the received paper by artificial intelligence.

[0016] In addition, in an information processing device according to a 9th aspect, in the information processing device according to any one of the 1st to 8th aspects, the processor also presents a method of creating the definition information by combining the three types of methods for each entry field.

[0017] Further, in order to achieve the above object, an information processing program according to a tenth aspect of the present invention receives an image of a paper sheet having a writing field for writing information, and provides a method for creating definition information indicating attributes of information to be written in the writing field, the method being a method in which the user creates the definition information anew, the method being a method in which the definition information created in another paper sheet prepared in advance is reused, and the method being a method in which the definition information is generated by the user. Categories The method presents three types of methods, one of which is to create a new image by using the results of classification by artificial intelligence, in a manner selectable by the user, and causes a computer to execute the process. Effect of the Invention

[0018] According to the first and tenth aspects, the user can select a method for creating the definition information.

[0019] According to the second aspect, at least one of the most recommended creation method and the reference information determined for each of the three presented creation methods can be visually indicated.

[0020] According to the third aspect, the most recommended creation method can be derived using the similarity with other paper sheets prepared in advance.

[0021] According to the fourth aspect, when the degree of similarity is equal to or greater than a predetermined threshold, the method of utilizing definition information of another paper that has been prepared in advance can be visually indicated as the most recommended creation method.

[0022] According to the fifth aspect, the method that takes the shortest time to create definition information, between a new creation method and a creation method using the results classified by artificial intelligence, can be visually displayed as the most recommended creation method.

[0023] According to the sixth aspect, when definition information is newly created, the creation time can be visually indicated.

[0024] According to the seventh aspect, it is possible to visually display information indicating the paper sheet having the highest similarity among other paper sheets prepared in advance.

[0025] According to the eighth aspect, the result of classification of the received paper by artificial intelligence can be visually displayed.

[0026] According to the ninth aspect, it is also possible to visually indicate how the definition information is created by combining the three presented creation methods for each entry field. [Brief description of the drawings]

[0027] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a configuration of a form system according to an embodiment. [Diagram 2] 1 is a block diagram showing an example of a configuration of an information processing device according to an embodiment. [Diagram 3] 10 is a flowchart illustrating an example of information processing according to the embodiment. [Figure 4]FIG. 13 is a schematic diagram showing an example of a creation method display screen according to the embodiment. [Diagram 5] FIG. 13 is a schematic diagram showing an example of a recommendation column reference information display screen according to the embodiment; [Figure 6] 11 is a schematic diagram showing an example of a creation method display screen on which a recommendation column creation icon is displayed according to the embodiment; FIG. [Figure 7] FIG. 13 is a schematic diagram showing an example of a new reference information display screen according to the embodiment. [Figure 8] FIG. 13 is a schematic diagram showing an example of a reuse reference information display screen according to the embodiment; [Figure 9] FIG. 13 is a schematic diagram showing an example of an AI reference information display screen according to the embodiment. [Figure 10] 11 is a flowchart illustrating an example of a recommended method derivation process according to the embodiment. [Figure 11] 10 is a flowchart illustrating an example of a received image reading process according to the embodiment. [Figure 12] FIG. 13 is a schematic diagram showing an example of a new creation screen according to the embodiment; [Figure 13] FIG. 13 is a schematic diagram showing an example of a new creation screen when a reading and writing field according to the embodiment is extracted; [Figure 14] FIG. 13 is a schematic diagram showing an example of a recommendation column creation screen according to the embodiment. [Figure 15] FIG. 13 is a schematic diagram showing an example of a reuse screen according to the embodiment; [Figure 16] FIG. 13 is a schematic diagram showing an example of an AI-utilizing screen according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, an example of an embodiment of the present disclosure will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are given the same reference numerals. In addition, the dimensional ratios of the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0029] FIG. 1 is a diagram showing a schematic configuration of a form system 10 according to this embodiment.

[0030] 1, a form system 10 includes an information processing device 20, a client terminal 40, and an input device 60. These devices are connected to a network (not shown) and can communicate with each other via the network. As an example of this network, the Internet, a LAN (Local Area Network), a WAN (Wide Area Network), etc. are applied.

[0031] The information processing device 20 performs OCR (Optical Character Recognition) processing on image data of a single or multiple sheets of paper, including a form, input through the input device 60, and manages a series of processing flows for outputting the results of the OCR processing to a predetermined output destination. In this embodiment, a server computer is applied as the information processing device 20. However, a personal computer (PC) or a device such as a smartphone may also be applied as the information processing device 20. The specific configuration and operation of the information processing device 20 will be described later.

[0032] The client terminal 40 transmits various instructions related to the OCR process to the information processing device 20. The various instructions include, for example, an instruction to start reading information of image data, an instruction to display the result of reading information of image data, and the like. The client terminal 40 also displays various information such as the result of the OCR process performed by the information processing device 20 in response to the various instructions received, and notifications related to the OCR process. For example, a general-purpose computer device such as a server computer or a personal computer is applied to the client terminal 40. Although only one client terminal 40 is illustrated in FIG. 1, this is not limiting, and multiple client terminals 40 may be prepared and, for example, the client terminals 40 may be used for different processes.

[0033] The input device 60 inputs image data to be subjected to OCR processing to the information processing device 20. Examples of the input device 60 include a general-purpose computer device such as a server computer or a PC, and an image forming device having a scanning function, a printer function, a FAX function, etc. In addition to the input device 60, image data may also be input to the information processing device 20 from the client terminal 40.

[0034] Next, an overview of the form system 10 will be described.

[0035] The form system 10 is a system in which an information processing device 20 performs OCR processing on image data input via an input device 60, and outputs the results of the OCR processing to a predetermined output destination.

[0036] In the OCR process, the information processing device 20 manages various processes, including (1) business design and operation verification, (2) data input, (3) data reading, (4) form discrimination, confirmation and correction, (5) reading result confirmation and correction, (6) business check, (7) data output, and (8) return. In this embodiment, the OCR process refers not only to the process of simply reading characters, symbols, etc. from image data, but also to post-processing such as character correction.

[0037] As an example of the management of various processes, (1) business design and operation verification, (2) data input, (3) data reading, (6) business check, and (7) data output processes are each automatically executed by the information processing device 20. Also, as an example of the management of various processes, (4) document discrimination, confirmation, and correction, and (5) reading result confirmation and correction processes are each accepted by input by the user via the client terminal 40. Also, as an example of the management of various processes, (8) return process may be automatically executed by the information processing device 20 or may be accepted by input by the user via the client terminal 40.

[0038] In the (1) business design and operation verification process, job rules are created that include read definition settings, output settings, and business check settings. In the read definition settings, as an example, a read range is set, which is the range in which image data information is read in the process of "(3) data reading". More specifically, as an example, a definition may be set such that the item value, which is the value, is read from near the right of the item extracted as the key. In the output settings, as an example, a file format and output destination of the output data output in the process of "(7) data output". In the business check settings, as an example, a format, such as the required input items and the number of characters that can be entered, in the form to be detected in the process of "(6) business check" are set.

[0039] In the process of (2) data input, image data is accepted as input from the input device 60. The accepted image data is registered as a job, which is an execution unit of the process of "(3) data reading."

[0040] (3) In the data reading process, the information of the image data in the job is read using a job rule for the job to be executed, selected by the user from the job rules created in the process of "(1) Business Design and Operation Verification." As an example, in this process, the form included in the image data in the job is identified (hereinafter referred to as "form identification") and characters and symbols within the reading range are read.

[0041] In the (4) form identification, confirmation, and correction process, the image data in the job is divided into records that indicate the forms included in the job based on the results of the form identification performed in the "(3) Data Reading" process. After that, in this process, the divided records are displayed, and the user can confirm and correct the form identification.

[0042] In the process of (5) checking and correcting the read result, the read result of the characters and symbols within the read range performed in the process of "(3) data reading" is displayed, and the user can check and correct the read result.

[0043] In the (6) task check process, errors in each preceding process are detected by the task check setting included in the job rule for the job selected by the user from among the job rules created in the process of "(1) Task design and operation verification." The detection results may be presented to the user.

[0044] (7) In the data output process, output data is created using the output settings included in the job rules for the job selected by the user from the job rules created in the process of "(1) Business design and operational verification," and the created output data is output to a predetermined output destination.

[0045] In the (8) return process, the process executed in the OCR process is returned to the process one or more steps before. As an example, the return is instructed by the user from the client terminal 40 during the execution of various processes such as "(4) Document discrimination, confirmation, and correction" and "(5) Read result confirmation and correction." Also, as an example, the return is instructed from the client terminal 40 of the administrator depending on the result of the check by the administrator performed between the processes of "(6) Business check" and "(7) Data output."

[0046] In the above OCR processing, the process of "(1) business design and operation verification" is executed before the processes after "(3) data reading" are performed, i.e., before the form system 10 is put into operation. Furthermore, the process of "(1) business design and operation verification" may be executed during the operation of the form system 10, when the processes after "(3) data reading" are being performed. As an example, the job rules created in the process of "(1) business design and operation verification" before the operation of the form system 10 can be modified according to the results of the process of "(5) checking and correcting the reading results" during the operation of the form system 10.

[0047] Next, a hardware configuration of the information processing device 20 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of a hardware configuration of the information processing device 20 according to this embodiment.

[0048] The information processing device 20 according to this embodiment includes a CPU 21, a memory 22 as a temporary storage area, a non-volatile storage unit 23, and a communication I / F unit 28. Each component is connected to each other via a bus B2.

[0049] The storage unit 23 is realized by a storage device such as an HDD, an SSD, or a flash memory. The storage unit 23 as a storage medium stores an information processing program 23A. The CPU 21 reads the information processing program 23A from the storage unit 23, expands it in the memory 22, and sequentially executes the processes of the information processing program 23A.

[0050] The communication I / F unit 28 is an interface for the information processing device 20 to communicate with the client terminal 40 and the input device 60. For this communication, for example, a wired communication standard such as Ethernet (registered trademark) or FDDI is used.

[0051] Next, a hardware configuration of the client terminal 40 according to this embodiment will be described. The client terminal 40 according to this embodiment includes an input unit 44 and a display unit 45.

[0052] The input unit 44 includes a pointing device such as a mouse, and a keyboard, and is used to receive various types of input information.

[0053] The display unit 45 is, for example, a liquid crystal display, and displays various information. The display unit 45 may function as the input unit 44 by adopting a touch panel system.

[0054] Next, the operation of the information processing device 20 according to the present embodiment will be described with reference to Fig. 3. When a user instructs the information processing device 20 to execute information processing from the client terminal 40, the CPU 21 executes the information processing program 23A, thereby executing the information processing shown in Fig. 3.

[0055] In step 200, the CPU 21 waits until it receives image data of a form on which a field for entering information is formed from the input device 60 via the communication I / F unit 28. Hereinafter, the image data received by the CPU 21 is referred to as a "reception image," the form corresponding to the reception image is referred to as a "reception form," and the field formed on the reception form is referred to as a "reception field." Note that in this embodiment, a single or multiple sheets of paper including a slip are used as the reception form, and characters and symbols are used as the information to be entered in the reception field.

[0056] In step 202, the CPU 21 executes a recommended method derivation process for deriving a recommended method, which is the most recommended creation method, among the creation methods of the definition information indicating the attributes of the information to be entered in the acceptance entry field. The recommended method derivation process according to this embodiment will be described later with reference to FIG.

[0057] In step 204, the CPU 21 displays on the display unit 45 a creation method display screen in a predetermined format.

[0058] As shown in FIG. 4, the creation method display screen according to this embodiment displays a reception image and a message prompting the user to select a creation method for definition information. In this embodiment, the following methods for creating definition information are displayed in a selectable manner: a method in which the user creates the definition information anew; a method in which definition information is reused that is created on a sheet prepared in advance and different from the reception sheet (hereinafter simply referred to as "another sheet"); and a method in which the definition information is created using a result of classification of the reception image by artificial intelligence (AI). In the following, the method in which the user creates the definition information anew is referred to as a "new creation method," and the method in which definition information is reused that is created on another sheet is referred to as a "reuse method." In addition, in the following, the method in which the definition information is created using a result of classification of the reception image by AI is referred to as an "AI-using method."

[0059] In addition, on the creation method display screen according to this embodiment, a method for creating definition information may be displayed by combining the above three types of methods for each reception entry field.

[0060] In addition, the example shown in Fig. 4 illustrates a case where there is only one reception image. However, there may be multiple reception images. In this case, multiple reception images may be displayed simultaneously on the creation method display screen, or only the reception image of the page specified by the user may be displayed on the creation method display screen.

[0061] Furthermore, on the creation method display screen according to this embodiment, an icon that displays reference information when specified (hereinafter referred to as a "reference information icon") and a recommended method derived by the recommended method derivation process ("new creation method" in the example shown in FIG. 4) are displayed. Note that the creation method display screen according to this embodiment does not display the reference information itself. However, the reference information itself may be displayed on the creation method display screen.

[0062] The creation method display screen according to this embodiment also displays a button for displaying an explanation of each creation method, a button for temporarily saving the settings of the creation method selected by the user, and a button for returning to the screen that was displayed before the accepted image was accepted, etc. However, in order to avoid confusion, these are not shown in the illustration here.

[0063] In step 206, the CPU 21 determines whether or not the selection of a reference information icon has been accepted on the creation method display screen via the input unit 44. If the determination in the processing of step 206 is affirmative, the CPU 21 proceeds to step 208, and if the determination is negative, the CPU 21 proceeds to step 218.

[0064] In step 208, the CPU 21 determines whether the recommended method is a new creation method. If the determination in the process of step 208 is affirmative, the CPU 21 proceeds to step 210.

[0065] In step 210, the CPU 21 determines whether there is a field that the AI ​​recommends reading from among the accepted fields (hereinafter, referred to as a "recommended field"). If the determination in step 210 is affirmative, the CPU 21 proceeds to step 212.

[0066] In step 212, the CPU 21 displays on the display unit 45 a recommendation column reference information display screen in a predetermined format, superimposed on the grayed-out creation method display screen.

[0067] As shown in Fig. 5, the recommended column reference information display screen according to this embodiment presents a recommended method (in the example shown in Fig. 5, "New Creation Method") and reference information. In the example shown in Fig. 5, although there is no category of a form showing the result of classifying received images by AI as reference information, a message is presented indicating that there is a recommended column and a message prompting the user to select whether or not to create a recommended column and create definition information. If the user wants to create a recommended column and create definition information, the user selects Yes button 45A via the input unit 44, or if the user does not want to create a recommended column, the user selects No button 45B.

[0068] In step 214, the CPU 21 determines whether or not an instruction to create a recommendation column has been received. In other words, the CPU 21 determines whether or not the Yes button 45A has been selected via the input unit 44 on the recommendation column reference information display screen. If the determination in the process of step 214 is affirmative, the CPU 21 proceeds to step 216. On the other hand, if the determination in the process of step 214 is negative, in other words, if the No button 45B has been selected via the input unit 44 on the recommendation column reference information display screen, the CPU 21 returns to step 204.

[0069] In step 216, when the command is received, the CPU 21 displays on the display unit 45 a creation method display screen to which an icon for displaying a recommendation column creation screen (to be described later) (hereinafter, referred to as a "recommendation column creation icon") has been added.

[0070] As shown in FIG. 6, the creation method display screen according to this embodiment is different from the creation method display screen shown in FIG. 5 in that a recommendation column creation icon is displayed.

[0071] In step 218, the CPU 21 determines whether or not a selection of any definition information creation method or a recommendation column creation icon has been accepted on the creation method display screen within a predetermined time (e.g., 5 minutes) via the input unit 44. If the determination in the processing of step 218 is affirmative, the CPU 21 proceeds to step 220, and if the determination is negative, the CPU 21 returns to step 206.

[0072] In step 220, the CPU 21 executes a received image reading process for reading the received image, and ends this information processing. The received image reading process according to this embodiment will be described later with reference to FIG.

[0073] Returning to step 210, if the determination is negative, the CPU 21 proceeds to step 222. In step 222, the CPU 21 displays on the display unit 45 a new reference information display screen in a predetermined format, superimposed on the grayed-out creation method display screen.

[0074] As shown in FIG. 7, the new reference information display screen according to this embodiment displays a recommended method ("New Creation Method" in the example shown in FIG. 7) and reference information. In the example shown in FIG. 7, a message is displayed as reference information to the effect that a new creation method is recommended because there is no reusable definition information and no category of forms that can be classified by AI. In addition, the creation time (5 minutes in the example shown in FIG. 7) when creating definition information using the new creation method is also displayed as reference information. When the user checks the recommended method and reference information and wishes to return to the creation method display screen, the user selects the back button 45C via the input unit 44.

[0075] In step 224, the CPU 21 determines whether or not an instruction to display a creation method display screen has been received. In other words, the CPU 21 determines whether or not the back button 45C has been selected via the input unit 44 on the new reference information display screen, the reuse reference information display screen, or the AI ​​reference information display screen. If the determination in the processing of step 224 is positive, the CPU 21 returns to step 204, and if the determination is negative, the CPU 21 waits until the determination is positive. The reuse reference information display screen and the AI ​​reference information display screen will be described later.

[0076] Returning to step 208, if the determination is negative, the CPU 21 proceeds to step 226. In step 226, the CPU 21 determines whether or not the recommended method is a diversion method. If the determination is positive in the processing of step 226, the CPU 21 proceeds to step 228. On the other hand, if the determination is negative in the processing of step 226, in other words, if the recommended method is an AI-utilizing method, the CPU 21 proceeds to step 230.

[0077] In step 228 , the CPU 21 displays the reuse reference information display screen in a predetermined format on the display unit 45 , superimposed on the grayed-out creation method display screen, and then proceeds to step 224 .

[0078] As shown in FIG. 8, the reuse reference information display screen according to this embodiment displays a recommended method (in the example shown in FIG. 8, "Reuse Method") and reference information. In the example shown in FIG. 8, as reference information, a message is displayed indicating that the similarity of another paper (in the example shown in FIG. 8, "Order Form (2020)") is equal to or greater than a predetermined threshold D1 (in the example shown in FIG. 8, "70%"), and therefore it is recommended to reuse the definition information of the other paper. If the user checks the recommended method and reference information and wishes to return to the creation method display screen, the user selects the back button 45C via the input unit 44.

[0079] In step 230 , the CPU 21 displays an AI reference information display screen in a predetermined format on the display unit 45 , superimposed on the grayed-out creation method display screen, and then proceeds to step 224 .

[0080] As shown in FIG. 9, the AI ​​reference information display screen according to this embodiment displays a recommended method (in the example shown in FIG. 9, "AI usage method") and reference information. In the example shown in FIG. 9, although there is no definition information that can be reused as reference information, since the received image can be classified by AI (an example of the form category shown in FIG. 9 is "order form"), a message is displayed recommending the use of the classification results. If the user checks the recommended method and reference information and wishes to return to the creation method display screen, the user selects the back button 45C via the input unit 44.

[0081] In this embodiment, the categories of forms are purchase orders, invoices, and receipts. However, in addition to these categories, application forms and the like may be applied as form categories, or a single or multiple combinations of these may be applied.

[0082] In addition, in this embodiment, the recommended reference screen, the new reference screen, the reuse reference screen, and the AI ​​reference screen are displayed on the display unit 45 superimposed on the grayed-out creation method display screen. However, these reference screens may be displayed on the display unit 45 without being superimposed on the creation method display screen.

[0083] In the present embodiment, the reference information is information that is determined for each of the three types of creation methods and indicates the reason why the recommended method displayed on the creation method display screen is derived as the most recommended creation method. However, the reference information may simply be information determined for each of the three types of methods.

[0084] Specifically, as reference information for the new creation method, the creation time when the definition information is created by this method, other forms with the highest similarity and information indicating that similarity, and the number of recommended fields, etc. may be applied. Furthermore, as reference information for the reuse method, other forms with the highest similarity and their similarity, and the application rate of the reusable entry fields in the other forms, etc. may be applied. Note that, if there are multiple pieces of information, a rank according to the degree of similarity or the degree of application rate may also be applied as reference information for the reuse method. Furthermore, as reference information for the AI ​​utilization method, the category of the form, the number of entry fields in that category, and the comparison result of each creation time when the definition information is created by the new creation method and the AI ​​utilization method, etc. may be applied.

[0085] Next, details of the recommended method derivation process according to this embodiment will be described with reference to FIG.

[0086] In step 400 of Fig. 10, the CPU 21 derives a feature of the received image (hereinafter referred to as a "received feature"). Specifically, the CPU 21 derives at least one of features such as the position of the reception entry field, the size of the reception entry field, and the proportion of the characters or the like written in the reception entry field as the received feature. Here, each received feature is normalized to a value between 0 and 1.

[0087] In step 402, the CPU 21 derives the similarity between the received image and all other papers. Specifically, the CPU 21 first reads out the feature amounts of the other papers from the storage unit 23, and derives the distance difference between the received feature amounts and the feature amounts of the other papers. For example, the CPU 21 derives the sum of squares of the distance difference between the feature amounts or the sum of the absolute values ​​of the distance as the distance difference between the received feature amounts and the feature amounts of the other papers. The CPU 21 then adds or weights the distance difference to derive the distance difference between the received image and the other papers as the similarity.

[0088] In step 404, the CPU 21 determines whether or not there is another sheet of paper whose similarity with the received image is equal to or greater than a predetermined threshold value D1, among the similarities derived in the process of step 402. If the determination in the process of step 404 is affirmative, the CPU 21 proceeds to step 406, and if the determination is negative, the CPU 21 proceeds to step 408.

[0089] In step 406, the CPU 21 sets the recommended method as the reuse method and ends the recommended method derivation process. When the recommended method derivation process ends, the CPU 21 proceeds to step 204 of the information processing (see FIG. 3).

[0090] In step 408, the CPU 21 judges whether the received image can be classified by the AI. Specifically, the CPU 21 judges whether the received image can be classified into a form category by the AI ​​from the received feature. If the CPU 21 judges that the received image is positive in the process of step 408, the CPU 21 proceeds to step 410, and if the CPU 21 judges that the received image is negative, the CPU 21 proceeds to step 414.

[0091] In step 410, the CPU 21 derives the creation time for creating definition information using the AI ​​utilization method and for creating definition information using the new creation method, based on the type of reception entry field and the number of reception entry fields, etc.

[0092] In step 412, the CPU 21 determines, as the recommended method, either the AI ​​utilization method or the new creation method, whichever has the shorter creation time, and ends this recommended method derivation process.

[0093] In step 414, the CPU 21 sets the recommended method as the new creation method, and ends the recommended method derivation process. In this step, the CPU 21 also derives the creation time in the case where definition information is created using the new creation method.

[0094] In the recommended method derivation process according to this embodiment, the CPU 21 determines whether or not there is another paper whose similarity is equal to or greater than a predetermined threshold value D1 (step 402), and then determines whether or not the received image can be classified by AI (step 408). However, the CPU 21 may determine whether or not there is another paper whose similarity is equal to or greater than a predetermined threshold value D1, after determining whether or not the received image can be classified by AI. Also, when the user creates definition information for the first time, the CPU 21 may first execute the process of step 408 without executing the process of step 402, etc.

[0095] Next, details of the received image reading process according to this embodiment will be described with reference to FIG.

[0096] 11, the CPU 21 determines whether or not an instruction to newly create definition information has been received. In other words, the CPU 21 determines whether or not "New Creation" has been selected on the creation method display screen via the input unit 44. If the determination in the process of step 600 is affirmative, the CPU 21 proceeds to step 602.

[0097] In step 602, the CPU 21 displays on the display unit 45 a new creation screen in a predetermined format.

[0098] 12, the new creation screen according to the present embodiment displays a reception image and a message encouraging the user to select a rectangular area by dragging the reception image and an entry field for reading information from the reception image (hereinafter, referred to as a "read entry field"). The user extracts the read entry field by dragging a rectangular area, which is the area for reading information from the reception image, via the input unit 44.

[0099] In step 604, CPU 21 waits until a read-and-write field is extracted on the creation method display screen.

[0100] When the user extracts the read and write fields, as shown in Fig. 13, the new creation screen according to this embodiment displays a column for inputting definition information corresponding to the read and write fields, the read result, and a read button 45D in an associated manner. In other words, the new creation screen according to this embodiment displays a column for inputting attributes of information to be entered in the read and write fields, the read result of the information entered in the read and write fields, and a read button 45D for reading the information entered in the read and write fields in an associated manner. Furthermore, the new creation screen according to this embodiment displays a one-page read button 45E for reading all information entered in all the extracted read and write fields, and an end button 45F for ending this received image reading process.

[0101] In step 606, the CPU 21 determines whether or not an instruction to read information entered in the read / write field has been accepted. In other words, the CPU 21 determines whether or not the read button 45D has been selected on the new creation method screen via the input unit 44. If the determination in the process of step 606 is affirmative, the CPU 21 proceeds to step 608, and if the determination is negative, the CPU 21 proceeds to step 610.

[0102] In step 608, the CPU 21 reads the information entered in the read / write field corresponding to the selected read button 45D, and displays the read result in the read result field on the new creation screen.

[0103] In step 610, the CPU 21 determines whether or not an instruction to read information entered in all the read / write fields has been accepted. In other words, the CPU 21 determines whether or not the read one page button 45E has been selected on the new creation method screen via the input unit 44. If the determination in the process of step 610 is affirmative, the CPU 21 proceeds to step 612, and if the determination is negative, the CPU 21 returns to step 606.

[0104] In step 612, the CPU 21 reads all the information entered in all the extracted read / write fields, and displays the read results in the read results on the new creation screen.

[0105] In step 614, CPU 21 waits until definition information corresponding to all the extracted read / write fields is input via input unit 44.

[0106] In step 616, the CPU 21 waits until it receives an instruction to end the received image reading process. Specifically, if the End button 45F is selected via the input unit 44 on the new creation method screen, the CPU 21 ends this received image reading process, and if the End button 45F is not selected, the CPU 21 returns to step 606. When the received image reading process ends, the CPU 21 also ends the information processing (see FIG. 3).

[0107] Returning to step 600, if the determination is negative, the CPU 21 proceeds to step 618. In step 618, the CPU 21 determines whether or not an instruction to create a recommendation column and create definition information has been accepted. In other words, the CPU 21 determines whether or not a recommendation column creation icon has been selected via the input unit 44 on the creation method display screen. If the determination is positive in the processing of step 618, the CPU 21 proceeds to step 620.

[0108] In step 620, the CPU 21 displays on the display unit 45 a recommendation column creation screen in a predetermined format.

[0109] As shown in Fig. 14, the recommendation column creation screen according to this embodiment displays the received image, definition information corresponding to the recommended columns created by the AI, the read result, and the read button 45D in association with each other. In other words, the recommendation column creation screen according to this embodiment displays the attributes of the information to be entered in the recommended columns, the read result of the information entered in the recommended columns, and the read button 45D for reading the information entered in the recommended columns in association with each other. Furthermore, the recommendation column creation screen according to this embodiment displays a one-page read button 45E for reading all the information entered in all the recommended columns, and an end button 45F for ending this received image reading process.

[0110] After executing the process of step 620, CPU 21 may extract a field and accept input of definition information corresponding to the field. This allows the user to create definition information by combining the recommended field created by the AI ​​and the field extracted by the user.

[0111] In addition, the new creation screen and recommendation column creation screen according to this embodiment also display a button for temporarily saving the read result, a button for canceling the read result, and a button for returning to the screen (creation method display screen) that was displayed before these screens were displayed, etc. However, to avoid confusion, illustrations of these are omitted here.

[0112] In step 622, the CPU 21 determines whether or not an instruction to read information entered in the recommendation column has been received. In other words, the CPU 21 determines whether or not the read button 45D has been selected on the recommendation column creation screen via the input unit 44. If the determination in the process of step 622 is affirmative, the CPU 21 proceeds to step 624, and if the determination is negative, the CPU 21 proceeds to step 628.

[0113] In step 624, the CPU 21 reads the information entered in the recommendation column corresponding to the read button 45D selected in step 622, and displays the read result in the read result on the recommendation column creation screen.

[0114] In step 626, the CPU 21 determines whether or not an instruction to end the received image reading process has been received. Specifically, when the end button 45F is selected via the input unit 44 on the recommendation column creation screen, the CPU 21 ends the received image reading process, and when the end button 45F is not selected, the CPU 21 returns to step 622.

[0115] In step 628, the CPU 21 determines whether or not an instruction to read information entered in all recommendation columns has been received. In other words, the CPU 21 determines whether or not the one-page read button 45E has been selected on the recommendation column creation screen via the input unit 44. If the determination in the process of step 628 is affirmative, the CPU 21 proceeds to step 630, and if the determination is negative, the CPU 21 returns to step 622.

[0116] In step 630, the CPU 21 reads the information entered in all the recommendation columns, and displays the read results in the read results section of the recommendation column creation screen.

[0117] The CPU 21 may display the result of reading information previously entered in the recommendation column as the reading result on the recommendation column creation screen, without displaying the read button 45D and the one-page read button 45E on the recommendation column creation screen.

[0118] In step 632, the CPU 21 waits until it receives an instruction to end the received image reading process. Specifically, when the end button 45F is selected via the input unit 44 on the recommendation column creation screen, the CPU 21 ends the received image reading process, and when the end button 45F is not selected, the CPU 21 waits until it is selected.

[0119] Returning to step 618, if the determination is negative, the CPU 21 proceeds to step 634. In step 634, the CPU 21 determines whether or not an instruction to create definition information by reusing definition information created on another sheet has been accepted. In other words, the CPU 21 determines whether or not the "reuse method" has been selected via the input unit 44 on the creation method display screen. If the determination is positive in the processing of step 634, the CPU 21 proceeds to step 636.

[0120] In step 636, the CPU 21 displays on the display unit 45 the reuse screen in a predetermined format.

[0121] As shown in FIG. 15, the diversion screen according to this embodiment displays the received image and images of other sheets whose similarity is equal to or greater than a predetermined threshold D1. The user selects one of the images of the other sheets for which the user wishes to divert definition information, and then selects the read button 45G. The diversion screen according to this embodiment may further display the received image and the similarity of each of the displayed other sheets. In addition, in the diversion screen according to this embodiment, when there are multiple sheets of other sheets corresponding to the displayed image, images of all pages may be displayed, or only images of pages designated by the user may be displayed.

[0122] In step 638, the CPU 21 waits until it receives a selection of any image from among the images of other sheets displayed on the diversion screen and an input of the read button 45G.

[0123] In step 640, the CPU 21 reads the received image by using the definition information of the other paper selected in step 638, and ends this received image reading process.

[0124] Returning to step 634, if the determination is negative, in other words, if an instruction to create definition information using the result of classifying the received image by AI has been received, the CPU 21 proceeds to step 642. In step 642, the CPU 21 displays an AI-utilizing screen in a predetermined format on the display unit 45.

[0125] As shown in FIG. 16, the AI-use screen according to this embodiment displays an image of a reception image, a category of a form classified by AI ("Order Form" in the example shown in FIG. 16), and an image of a pre-stored form category other than the above-mentioned form category ("Invoice" and "Receipt" in the example shown in FIG. 16). When creating definition information using a form category classified by AI, the user selects only the read button 45G. On the other hand, when creating definition information using a category other than the form category classified by AI, the user selects the displayed form image and the read button 45G.

[0126] In addition, the diversion screen and the AI ​​use screen according to this embodiment also display a button for returning to the creation method display screen, and a button for displaying other papers than the other papers or report categories other than the report categories displayed on these screens, etc. However, in order to avoid confusion, illustrations thereof are omitted here.

[0127] In step 644, CPU 21 waits until it accepts the selection of read button 45G displayed on the AI-use screen.

[0128] In step 646, the CPU 21 reads the received image by utilizing the definition information of the category of the form classified by AI or the category corresponding to the selected image, and ends this received image reading process.

[0129] Although the embodiments have been described above, the technical scope of the present invention is not limited to the scope described in the above embodiments. Various modifications or improvements can be made to the above embodiments without departing from the gist of the invention, and such modifications or improvements are also included in the technical scope of the present invention.

[0130] Furthermore, the above-mentioned embodiments do not limit the inventions described in the claims, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. The above-mentioned embodiments include inventions at various stages, and various inventions are extracted by combining the multiple constituent elements disclosed. Even if some constituent elements are deleted from all the constituent elements shown in the embodiments, as long as the effect is obtained, the configuration from which these several constituent elements are deleted can be extracted as an invention.

[0131] For example, in the above embodiment, the definition information creation method, the recommended creation method, and the reference information are presented by display on the display unit 45 of the client terminal 40. However, the above presentation may be presented by audio via a speaker or the like of the client terminal 40.

[0132] In addition, in each of the above embodiments, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU: Central Processing Unit, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0133] In addition, the operations of the processor in each of the above embodiments may be performed not only by one processor, but also by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate.

[0134] In the present embodiment, the information processing program 23A is installed in the storage unit 23, but the present invention is not limited to this. The information processing program 23A according to the present embodiment may be provided in a form recorded in a computer-readable storage medium. For example, the information processing program 23A according to the present embodiment may be provided in a form recorded in an optical disc such as a CD (Compact Disc)-ROM and a DVD (Digital Versatile Disc)-ROM, or in a semiconductor memory such as a USB (Universal Serial Bus) memory and a memory card. The information processing program 23A according to the present embodiment may be obtained from an external device via the communication I / F unit 28.

[0135] In the above embodiment, the information processing is realized by a software configuration using a computer by executing a program, but the present invention is not limited to this. For example, the information processing may be realized by a hardware configuration or a combination of a hardware configuration and a software configuration.

[0136] In addition, the configuration of the information processing device 20 described in the above embodiment is just one example, and it goes without saying that unnecessary parts may be deleted and new parts may be added without departing from the spirit of the present invention.

[0137] Furthermore, the processing flow of the information processing program described in the above embodiment (see FIG. 3) is also one example, and it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the present invention. [Explanation of symbols]

[0138] 10 Reporting System 20 Information processing device 21 CPU 22 Memory 23 Memory section 28 Communication I / F section 40 Client Terminals 60 Input Devices

Claims

1. A processor is provided. The processor, Accepting an image of a form having fields for information to be entered; As a method for creating definition information indicating attributes of information to be entered in the entry field, three methods are presented in a manner selectable by the user: a method in which the user creates the definition information anew; a method in which the definition information created in another form prepared in advance is reused; and a method in which the definition information is created using the results of classification of the category of the accepted form by artificial intelligence. Information processing device.

2. The processor further presents at least one of a most recommended creation method among the three types of methods and reference information determined for each of the three types of methods. The information processing device according to claim 1 .

3. the most recommended creation method is a creation method derived using a similarity between the feature amount of the accepted paper and the feature amount of the other paper. The information processing device according to claim 2 .

4. and when the degree of similarity is equal to or greater than a predetermined threshold, the processor presents the reuse method as the most recommended creation method. The information processing device according to claim 3 .

5. When the degree of similarity is less than a predetermined threshold, the processor presents, as the most recommended creation method, one of the new creation method and the creation method using the result of classification by the artificial intelligence, whichever takes the shortest time to create the definition information.

5. The information processing device according to claim 3.

6. The reference information in the new creation method is information indicating a creation time when the definition information is created by the new creation method. The information processing device according to any one of claims 2 to 5.

7. The reference information in the method of reusing is information indicating the other paper having the highest similarity to the feature amount of the received paper. The information processing device according to any one of claims 2 to 6.

8. In the method for creating the reference information using the result of classification by the artificial intelligence, the reference information is information indicating the result of classification of the received paper by the artificial intelligence. The information processing device according to any one of claims 2 to 7.

9. The processor also presents a method for creating the definition information by combining the three types of methods for each entry field. The information processing device according to any one of claims 1 to 8.

10. Accepting an image of a form having fields for information to be entered; As a method for creating definition information indicating attributes of information to be entered in the entry field, three methods are presented in a manner selectable by the user: a method in which the user creates the definition information anew; a method in which the definition information created in another form prepared in advance is reused; and a method in which the definition information is created using the results of classification of the category of the accepted form by artificial intelligence. An information processing program for causing a computer to execute processing.

Images