Image processing device, image processing method, and program

Abstract

To record any character to be recorded in any document according to the type of format even when there are many types of document formats. [Solution] An image processing device acquires image data of a document form and a recorded character string recorded by an operator from among the character strings included in the image data, extracts feature values ​​of character strings recognized by character recognition processing on the image data that match the recorded character string, performs machine learning on the feature values ​​of character strings extracted from the image data of multiple document forms of the same format to generate feature values ​​of a target to be read, and records the character string recognized from the image data of the new document form if the feature values ​​extracted from the character string recognized by character recognition processing on the acquired image data of the new document form match the feature values ​​of the target to be read. In addition, the feature values ​​are information indicating the attributes of the character string and the range of the character string in the image of the document form.

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] There are image processing devices that read documents by subjecting images of documents acquired using a scanner or the like to OCR (Optical Character Recognition) processing. When documents are in various formats, such image processing devices store document format definition information in advance and specify the area to be read, thereby improving the accuracy of OCR processing. A related technology is disclosed in Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2016-48444 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, when there are many different document formats as described above, it takes a lot of work for the operator to store all of the format definition information in the image processing device.

[0005] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide an image processing device, an image processing method, and a program that solve the above-mentioned problems. [Means for solving the problem]

[0006] According to a first aspect of the present invention, an image processing apparatus acquires image data of a document form, and a recorded character string that is recorded in a database by an operator as a character string to be recorded, in association with an image of the document form, from among character strings included in the image data. And obtain image data of new documents and formsthe read target feature generation unit generates, as a feature of a read target, a feature common to the document form of the same format by machine learning using the feature of a string that matches the recorded string among strings that are extracted from the image data of each of a plurality of documents and forms of the same format; and a recording unit records, for each string recognized by character recognition processing on the image data of the new document and form acquired by the acquisition unit, a string recognized from the image data of the new document and form when the feature extracted by the feature extraction unit matches the feature of the read target generated by the read target feature generation unit, wherein the feature is information indicating an attribute of the string and a range of the string in the image of the document and form.

[0007] According to a second aspect of the present invention, an image processing method includes acquiring image data of a document form and a recorded character string that is recorded in a database by an operator in association with an image of the document form as a character string to be recorded from among character strings included in the image data, extracting feature amounts that are information indicating characteristics of a character string that matches the recorded character string from among character strings recognized by character recognition processing on the image data, and generating feature amounts common to document forms of the same format as feature amounts to be read by machine learning using the feature amounts of character strings that match the recorded character string from among character strings extracted from image data of each of a plurality of document forms of the same format; Acquire image data of new documents and forms, For each character string recognized by character recognition processing on image data of a new document form, if a feature extracted as information indicating the characteristics of the character string matches a feature of the target to be read generated by the method of generating feature amounts of the target to be read by the machine learning, the character string recognized from the image data of the new document form is recorded, and the feature amount is information indicating the attributes of the character string and the range of the character string in the image of the document form.

[0008] According to a third aspect of the present invention, a program includes the steps of: acquiring image data of a document form; and, from among character strings included in the image data, a recorded character string that an operator has recorded in a database in association with the image of the document form as a character string to be recorded; extracting features that are information indicating characteristics of a character string that matches the recorded character string from among character strings recognized by character recognition processing on the image data; and generating features common to document forms of the same format by machine learning using features of character strings that match the recorded character string from among character strings extracted from image data of multiple document forms of the same format, as features to be read. acquiring image data of a new document; The above New document image data The program executes a step of recording the character string recognized from the image data of the new document form when, for each character string recognized by character recognition processing on the image data of the new document form acquired in the acquiring step, a feature extracted as information indicating the characteristics of the character string matches the feature of the target to be read generated in the step of generating feature of the target to be read by the machine learning, and the feature is information indicating the attribute of the character string and the range of the character string in the image of the document form. Effect of the Invention

[0009] According to the present invention, even if there are many different document formats, any characters to be recorded in any document according to the types of the formats can be recorded. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an overview of an image processing system including an image processing device. [Diagram 2] FIG. 1 is a diagram illustrating a hardware configuration of an image processing device. [Diagram 3] FIG. 2 is a functional block diagram of the image processing device. [Figure 4] FIG. 2 is a diagram showing an example of a document form. [Diagram 5] FIG. 2 is a diagram showing an overview of a data table stored in a database. [Figure 6] FIG. 2 is a first diagram showing a processing flow of the image processing device according to the first embodiment. [Figure 7] FIG. 11 is a second diagram showing the processing flow of the image processing device according to the first embodiment. [Figure 8] FIG. 11 is a functional block diagram of an image processing device according to a second embodiment. [Figure 9] FIG. 11 is a first diagram showing a processing flow of an image processing apparatus according to a second embodiment. [Figure 10] FIG. 11 is a second diagram showing the processing flow of the image processing device according to the second embodiment. [Figure 11] FIG. 13 is a diagram showing a processing flow of an image processing device according to a sixth embodiment. [Figure 12] FIG. 13 is a diagram illustrating an example of connection between an image processing system according to a seventh embodiment and an existing OCR system. [Figure 13] FIG. 13 is a diagram showing a processing flow of an image processing device according to a sixth embodiment. [Figure 14] FIG. 13 is a diagram showing an overview of an image processing system 200 according to an eighth embodiment. [Figure 15] FIG. 23 is a diagram showing a processing flow of a terminal device according to the eighth embodiment. [Figure 16] FIG. 23 is a diagram showing a processing flow of a server device according to the eighth embodiment. [Figure 17] FIG. 1 is a diagram illustrating a minimum configuration of an image processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] An image processing apparatus according to an embodiment of the present invention will now be described with reference to the drawings. 1 is a diagram showing an outline of an image processing system including an image processing device according to this embodiment. As shown in FIG. 1, the image processing system 100 comprises an image processing device 1, an image reading device 2, a recording device 3, and a database 4. The image processing device 1 is connected to the image reading device 2 by a communication cable. The image reading device 2 optically acquires image data of documents and forms, etc., and outputs the image data to the image processing device 1. The image processing device 1 performs OCR processing on the image data of the documents and forms, and performs character recognition. The image processing device 1 outputs the character recognition result to the recording device 3, and the recording device 3 records the character recognition result in a database. The database 4 is connected to the image processing device 1 and the recording device 3. The database 4 stores the correspondence between the image data of multiple documents and forms registered in the past from the recording device 3 and the recorded character strings indicating the character strings to be recorded among the character strings contained in the image data. The character strings indicated by the recorded character strings are important character strings to be recorded and stored in the database 4 among the character strings described in the documents and forms. An operator who uses the image processing system 100 uses the recording device 3 to register in advance in the database 4 the image data of multiple documents and forms registered in the past and the recorded character strings among the character strings contained in the image data.

[0012] It is assumed that the correspondence between the image data of the document form and the information on the recorded character strings indicating the character strings to be recorded among the information on the character strings contained in the image data is sufficiently recorded for many document forms in the recording device 3. In this state, the image processing device 1 performs processing.

[0013] FIG. 2 is a diagram illustrating a hardware configuration of the image processing device. 2, the image processing device 1 is a computer including a central processing unit (CPU) 11, an interface (IF) 12, a communication module 13, a read only memory (ROM) 14, a random access memory (RAM) 15, and a hard disk drive (HDD) 16. The communication module 13 may perform wireless communication with the image reading device 2, the recording device 3, and the database 4, or may perform wired communication, or may have both of these functions.

[0014] FIG. 3 is a functional block diagram of the image processing device. The CPU 11 of the image processing device 1 executes the stored programs to provide the functions of a control unit 101, an acquisition unit 102, a feature extraction unit 103, a read object feature generation unit 104, and a recording unit 105.

[0015] The control unit 101 controls the other functional units. The acquisition unit 102 acquires image data of a document form. The feature extraction unit 103 extracts, for each image data of a document form, a feature indicating a characteristic of a recorded character string included in the image data of the document form, based on the character recognition processing results of the image data of the multiple document forms. The read target feature generating unit 104 generates a first feature of the recorded character string by using a feature corresponding to image data of the document form. The recording unit 105 uses the first feature amount to extract and record a record character string from the information of the character string read from the image data of the new document form.

[0016] By carrying out such processing, the image processing apparatus 1 reduces the effort required for recording character string information that is to be recorded and is included in image data of a new document form.

[0017] FIG. 4 is a diagram showing an example of a document form. As shown in this figure, the mark of the company that created the document, the creation date, the person in charge of creating the document, and the document contents are described in a format specific to the document form. For example, if the document form is a purchase order form, the document contents indicate one or more sets of information such as the name of the ordered product and the number of items ordered. Based on a certain document form, the worker uses the recording device 3 to record a specific record character string to be recorded among the character strings described in the document form in the database 4. Specifically, while looking at the document form, the worker inputs the specific record character string to be recorded by the recording device 3 in the database 4. The worker also causes the image reading device 2 to read the image data of the document form. The image reading device 2 reads the document form based on the operator's operation and outputs it to the image processing device 1. Then, based on the operator's operation and the control of the image processing device 1, the recording device 3 records the image data of one document form and the specific record character string to be recorded among the character strings described in the document form in association with each other in the database 4. In the example of Fig. 4, date 51, supplier 52, product name 53, quantity 54, and price 55 are specific record character strings to be recorded. Other information such as non-record character strings that are not recorded by the worker is also printed on the document form 5. For example, the information is the name 501 of the purchaser who issued the document form, the purchaser's emblem image 502, the document form title 503, a greeting 504, etc.

[0018] FIG. 5 is a diagram showing an outline of a data table stored in the database. As shown in FIG. 5, the database 4 stores image data of a document form and specific record character strings to be recorded among character strings described in the document form in a record table in association with each other.

[0019] First Embodiment FIG. 6 is a first diagram showing the processing flow of the image processing apparatus according to the first embodiment. Next, the processing flow of the image processing device 1 will be explained step by step. First, a combination of image data for a certain document form and a specific record character string described in the document form is recorded for multiple pages of the same document form in the database 4. For example, assume that specific record character string information for the document form 5 shown in Fig. 4 is recorded for multiple pages of the same document form. In this state, the worker starts up the image processing device 1 and instructs the image processing device 1 to start processing.

[0020] The acquisition unit 102 of the image processing device 1 reads image data of the document form 5 from the database 4 and information on the recorded character string corresponding to the image data (step S601). The acquisition unit 102 outputs the image data and the recorded character string to the feature extraction unit 103. The feature extraction unit 103 performs OCR processing on the image data to detect all character strings in the image data and coordinates within the image data indicating the range of the character strings (step S602). Note that a character string is a group of characters consisting of multiple characters. The feature extraction unit 103 analyzes the range of one group based on the spacing between other characters, etc., extracts one or more characters included in the range as a character string, and detects coordinates indicating the range of the character string in the image data. The characters included as a character string may include symbols such as ideograms and phonetic characters, marks, icon images, etc.

[0021] The feature extraction unit 103 compares the character string extracted from the image data by OCR processing with the recorded character string read from the database 4 together with the image data. Among the character strings extracted from the image data by OCR processing, the feature extraction unit 103 identifies the character string in the image data that matches the character information of the recorded character string, the attributes of the characters included in the character string, and the coordinates of the range (step S603). The character attributes are information represented by numbers, alphabets, hiragana, kanji, the number of characters, character height, font, etc. The coordinates of the range of the character string are information indicating the coordinates of the first character included in the character string, the coordinates of the last character, etc. The feature extraction unit 103 generates features including the identified information for multiple sheets of the same document form 5 (step S604).

[0022] A feature amount is generated by the feature amount extraction unit 103 for each character string recorded in the document form 5. The read target feature amount generation unit 104 acquires the features in the same multiple documents forms 5, analyzes the character attributes included in each feature amount corresponding to the recorded character string, and the coordinates indicating the range of the character string, and generates one feature amount for each. This feature amount analysis is performed by a process such as machine learning. Machine learning is also simply called learning. The feature generated by the read target feature generating unit 104 through analysis such as machine learning is called a first feature. That is, the read target feature generating unit 104 uses multiple sheets of the same document form 5 to generate a first feature for each recorded character string in the document form 5 (step S605). The first feature is a feature for recognizing the recorded character string, and includes character attributes and coordinates indicating the range of the character string. The read target feature generating unit 104 records the first feature for each of one or more recorded character strings in the document form 5 in the database 4 by linking it to the identifier of the document form 5 (step S606).

[0023] For example, the read target feature generation unit 104 records in the database 4 each of the first features indicating character attributes, coordinates indicating the range of the character strings, etc., for each of the recorded character strings included in the document form 5, such as date 51, supplier 52, product name 53, quantity 54, and amount 55, by linking them to the identifier of the document form 5.

[0024] Through the above processing, the image processing device 1 can generate information (first feature amount) used to reduce the labor of the worker in recording the record character string by machine learning or the like, and store the information in the database 4. This enables the image processing device 1 to automatically record the record character string in the database 4 based on the image data of a new document form. The processing will be described below.

[0025] FIG. 7 is a second diagram showing the processing flow of the image processing device according to the first embodiment. The operator performs an operation to make the image reading device 2 read a new document form. As a result, the image reading device 2 generates image data of the document form and outputs it to the image processing device 1. The acquisition unit 102 of the image processing device 1 acquires image data (step S701). The acquisition unit 102 outputs the image data to the feature amount extraction unit 103. The feature amount extraction unit 103 performs OCR processing on the image data to detect a character string, a character feature included in the character string, and a coordinate in the image data of the range of the character string (step S702). The feature amount extraction unit 103 generates a third feature amount including the detected information for each character string in the image data (step S703). In other words, the third feature amount is information indicating the feature of a character string included in the document form of the newly read image data. After that, the group identification unit 107 reads out one or more first feature amounts for the document form of the image data to be processed from the database 4 (step S704). The group identification unit 107 outputs the third feature amount and the first feature amount to the recording unit 105.

[0026] The recording unit 105 acquires a third feature amount for one or more character strings in the image data and one or more first feature amounts. The recording unit 105 uses coordinates indicating the range of character strings included in each first feature amount to determine whether all third feature amounts having coordinates corresponding to the coordinates indicated by each first feature amount exist (step S705). When all third feature amounts having coordinates corresponding to the coordinates of each first feature amount exist, character descriptions exist in all items in the document form corresponding to the recorded character strings. On the other hand, when all third feature amounts having coordinates corresponding to the coordinates of each first feature amount do not exist, character descriptions do not exist in any items in the document form.

[0027] If the answer is YES in step S705, the recording unit 105 determines whether the character attribute included in the first feature matches the character attribute included in the corresponding third feature identified based on the coordinates (step S706).

[0028] When the determination result in step S706 is YES and the character attributes match, the recording unit 105 generates a confirmation screen displaying a rectangular frame in the range of the recording character string based on the coordinates indicated by one or more third feature amounts in the image data currently being processed. The recording unit 105 outputs the confirmation screen to the monitor (step S707). The worker can check the rectangular area displayed on this confirmation screen to confirm the recording character string that the image processing device 1 is going to record. This allows the worker to check whether there is anything missing in the recording character string. An icon image of either an OK or NG button is displayed on the confirmation screen. By selecting the OK button among these button icon images, the worker can indicate that there is nothing missing in the selection as the recording character string. On the other hand, by selecting the NG button among the button icon images, the worker can indicate that there is something missing in the selection as the recording character string.

[0029] The reason for outputting the confirmation screen to the monitor will be explained with reference to FIG. 4. In FIG. 4, six product names 53 are entered in the recorded character strings. If the maximum number of product names 53 entered in the past document form was six, product names 53 within the range of 1 to 6 are automatically determined as recorded character strings for the new document form. Therefore, for example, if seven product names 53 are entered in the new form, the first to sixth items are both YES in steps S705 and S706, and the image processing device 1 ends without recording the seventh character string. To improve such an event, the image processing device 1 displays a confirmation screen before recording the recorded character strings in step S707, and asks the operator whether it is OK to record and end the process.

[0030] In response to the operator's pressing of the icon image of the button, the recording unit 105 determines whether or not there is a missing selection of the recorded character string (step S708). If there is no missing selection, the recording unit 105 records the character string included in the third feature in the recording table in association with the identification information of the document form (step S709).

[0031] For example, assume that the third feature amount a3, the third feature amount b3, the third feature amount c3, and the third feature amount d3 are acquired from the image data of the document form. Assume that the third feature amount a3 matches the first feature amount a1 recorded in advance in the database, the third feature amount b3 matches the first feature amount b1, the third feature amount c3 matches the first feature amount c1, and the third feature amount d3 matches the first feature amount d1. In this case, the recording unit 105 records the character strings included in the third feature amount a3, the third feature amount b3, the third feature amount c3, and the third feature amount d3 in the recording table of the document form.

[0032] If the answer is NO in step S705, NO in step S706, or NO in step S708, the recording unit 105 performs processing in the case where a third feature having coordinates corresponding to the coordinates indicated by the first feature does not exist. Specifically, the recording unit 105 generates input image data of a form image in which an input field is provided in the range of coordinates of the first feature where the third feature of the corresponding coordinates in the image data did not exist, and outputs the generated data to the monitor (step S710). The input image data may be data described in a markup language such as HTML or XML. While viewing the input image data, the worker operates an input device such as a keyboard of the image processing device 1 to input a recording character string into the input field in the input image data displayed on the monitor. A save button is displayed in the input image data, and when the save button is pressed, the recording unit 105 generates a third feature including the character string newly input into the input field of the input image data in addition to the third feature already acquired for the document form (step S711). The recording unit 105 records the identifier of the form image data and the character string entered in the input field in the database 4 in association with each other. The image processing device 1 updates the first feature by performing the processing flow shown in Fig. 6 again, and can expand the range of character strings that can be automatically recorded. As a result, the next time the same document form is processed, the character string can be automatically recorded, eliminating the need for the worker to input the character string. The recording unit 105 records the character strings contained in all the third feature values ​​in the recording table of the document form (step S712).

[0033] According to this process, the image processing device 1 can automatically record the record character string in the image data of the newly input document form by using the image data of the document form and the record character string recorded in advance by the worker. Therefore, the image processing device 1 can reduce the labor of the worker in recording the record character string in the document form. Even if the record character string is not entered in the document form, if the description item corresponding to the record character string that should be entered is not entered, the image processing device 1 outputs the input image data. This makes it possible to find an error in which an entry item that should be entered in the document form has not been entered, and to easily record the record character string indicated by that entry item.

[0034] Second Embodiment FIG. 8 is a functional block diagram of an image processing apparatus according to the second embodiment. As shown in this figure, the image processing device 1 according to the second embodiment has the functions of a group classification unit 106 and a group identification unit 107 in addition to the respective functional units shown in Fig. 3. The hardware configuration of the image processing device 1 is similar to the configuration shown in Fig. 2.

[0035] FIG. 9 is a first diagram showing the processing flow of the image processing apparatus according to the second embodiment. Next, the processing flow of the image processing device 1 according to the second embodiment will be explained step by step. In the database 4, a large number of combinations of image data for a plurality of different document forms and specific record character strings described in each document form are recorded for each document form. For example, assume that specific record character string information for each of the different document forms 5 shown in Fig. 4 is recorded for multiple pages. In this state, the operator starts up the image processing device 1 and instructs the image processing device 1 to start processing.

[0036] The acquisition unit 102 of the image processing device 1 judges whether all the information on the image data of the document form 5 and the recorded character string corresponding to the image data has been read from the database 4 (step S901). If NO, the acquisition unit 102 reads the image data of the document form 5 and the information on the recorded character string corresponding to the image data from the database 4 (step S902). The acquisition unit 102 outputs the image data and the recorded character string to the feature extraction unit 103. The feature extraction unit 103 performs OCR processing on the image data to detect all the character strings in the image data and their coordinates within the image data (step S903). Note that a character string is a group of characters consisting of multiple characters. The feature extraction unit 103 analyzes the range of one group based on the spacing with other characters, etc., extracts one or more characters included in the range as a character string, and detects the coordinates indicating the range of the character string in the image data. The characters included as a character string may include symbols such as ideograms and phonetic characters, marks, icon images, etc.

[0037] The feature extraction unit 103 compares the character string extracted from the image data by OCR processing with the recorded character string read from the database 4 together with the image data. Among the character strings extracted from the image data by OCR processing, the feature extraction unit 103 identifies the character string in the image data that matches the character information of the recorded character string, the attributes of the characters included in the character string, and the coordinates of the range (step S904). The character attributes are information represented by numbers, alphabets, hiragana, kanji, the number of characters, character height, font, etc. The coordinates of the range of the character string are information indicating the coordinates of the first character included in the character string, the coordinates of the last character, etc. The feature extraction unit 103 generates features including the identified information for one document form 5 (step S905).

[0038] The feature amount is generated by the feature amount extraction unit 103 for each recorded character string in the document form 5. That is, the feature amount extraction unit 103 generates a feature amount for each document form and for each recorded character string. This feature amount for each document form and for each recorded character string is referred to as an individual first feature amount. The individual first feature amount may include either or both of a character attribute and a coordinate indicating the range of the character string. The feature amount extraction unit 103 records the individual first feature amount of each of one or more recorded character strings in each document form 5 in the database 4 by linking it to the identifier of the document form 5 and the identifier of the recorded character string (step S906). As the identifier of the recorded character string, for example, a coordinate value indicating the position of the recorded character string can be used.

[0039] For example, the feature extraction unit 103 records in the database 4 each of the individual first features indicating character attributes, coordinates indicating the range of the character string, etc., for each of the recorded character strings included in the document form 5, namely, date 51, supplier 52, product name 53, quantity 54, and amount 55, in association with the identifier of the document form 5 and the identifier of the recorded character string.

[0040] The feature extraction unit 103 also identifies non-recorded character strings in the image data that do not match character information included in the recorded character strings, the attributes of the characters included in the non-recorded character strings, and the coordinates of the range (step S907). The feature extraction unit 103 generates features including the identified information for the document form 5 (step S908).

[0041] The feature amount is generated by the feature amount extraction unit 103 for each non-recorded character string in the document form 5. That is, the feature amount extraction unit 103 generates a feature amount for each document form and for each non-recorded character string. This feature amount for each document form and for each non-recorded character string is referred to as an individual second feature amount. The individual second feature amount may include either or both of a character attribute and a coordinate indicating the range of the character string. The feature amount extraction unit 103 records the individual second feature amount of each of one or more non-recorded character strings in each document form 5 in the database 4 by linking it to the identifier of the document form 5 and the identifier of the non-recorded character string (step S909). As the identifier of the non-recorded character string, for example, a coordinate value indicating the position of the recorded character string can be used.

[0042] For example, the feature extraction unit 103 records in the database 4 each individual second feature indicating the non-recorded character strings contained in the document form 5, such as the name of the client 501, the client's emblem image, the title of the document form 503, and the greeting 504, by linking it to the identifier of the document form 5 and the identifier of the recorded character string.

[0043] The database 4 stores image data of a plurality of different document forms 5 and information on recorded character strings corresponding to that image data. The acquisition unit 102 of the image processing device 1 repeats the processes of steps S901 to S911 until it has read the image data and recorded character string information for all document forms 5. Then, it is determined in step S901 that all have been read (step S901: YES).

[0044] In this case, the group classification unit 106 groups the document form 5 based on the individual second feature amount included in the image data of the document form 5 (step S912). For example, the group classification unit 106 groups each document form 5 based on the degree of match of the non-recorded character string indicated by the individual second feature amount, the degree of match of the emblem image, the degree of match of the coordinate range of the non-recorded character string, etc. In this grouping process, the group classification unit 106 determines a group identifier of the document form 5. The group classification unit 106 determines whether grouping has been completed for all document forms 5 (step S913).

[0045] The group classification unit 106 repeats the process of step S912 if grouping of all document forms 5 has not been completed. If grouping of all document forms 5 has been completed, the group classification unit 106 records the identifier of the document form 5 and the group identifier assigned to the document form 5 in association with each other in the group table of the database 4 (step S914).

[0046] Then, the read target feature generating unit 104 reads each individual first feature and each individual second feature of one or more documents / forms 5 belonging to a certain group from the database 4, and generates each group first feature and each group second feature corresponding to each individual first feature and each individual second feature of the documents / forms 5 belonging to the group (step S915). Each group first feature may be a value such as an average of each individual first feature of the documents / forms 5 belonging to the group. Similarly, each group second feature may be a value such as an average of each individual second feature of the documents / forms 5 belonging to the group. Each group first feature and each group second feature may be generated by any method as long as it is a feature calculated by using a predetermined statistical processing method, machine learning method, or the like so as to specify a recorded character string or a non-recorded character string of one or more documents / forms 5 belonging to the group. The read object feature amount generating unit 104 calculates a first group feature amount and a second group feature amount for each group, and records them in the database 4 in association with the group identifier (step S916).

[0047] Through the above processing, the image processing device 1 can generate information required for each group of document forms to reduce the labor of the worker in recording the record character strings, and store the information in the database 4. This enables the image processing device 1 to automatically record the record character strings in the database 4 based on the image data of a new document form. The processing will be described below.

[0048] FIG. 10 is a second diagram showing the processing flow of the image processing apparatus according to the second embodiment. The worker performs an operation to have the image reading device 2 read a new document form. As a result, the image reading device 2 generates image data of the document form and outputs it to the image processing device 1. The acquisition unit 102 of the image processing device 1 acquires image data (step S1001). The acquisition unit 102 outputs the image data to the feature extraction unit 103. The feature extraction unit 103 performs OCR processing on the image data to detect character strings, the characteristics of characters contained in the character strings, and the coordinates in the image data of the range of the character strings (step S1002). The feature extraction unit 103 generates a third feature including the detected information for each character string in the image data (step S1003). The third feature is information indicating the characteristics of character strings included in the document form of the newly read image data.

[0049] Next, the group identification unit 107 reads out from the database 4 a group second feature to be used for group identification of a new document form from among the group second features. The group second feature may be, for example, a feature corresponding to the emblem image 502 of the client displayed in the image data of the document form. The group identification unit 107 judges whether information indicated by a certain group second feature can be identified from the image data of the document form acquired in step S1001. The group identification unit 107 performs the same process using the group second feature for all groups. If information matching the group second feature read out from the database 4 can be identified from the image data of the newly read document form, the group having the group second feature is identified as the group of the image data of the newly read document form (step S1004). After that, the group identification unit 107 reads out one or more group first features for the group from the database 4 (step S1005). The group identification unit 107 outputs the third feature and one or more group first features to the recording unit 105. The group first feature is a feature for identifying one or more recorded character strings in the document form that belongs to the group.

[0050] The recording unit 105 acquires a third feature amount for one or more character strings in the image data and one or more group first feature amounts. The recording unit 105 uses coordinates indicating the range of character strings included in each group first feature amount to determine whether all third feature amounts having coordinates corresponding to the coordinates indicated by each group first feature amount exist (step S1006). When all third feature amounts having coordinates corresponding to the coordinates of each group first feature amount exist, character descriptions exist in all items in the document form corresponding to the recorded character strings. On the other hand, when all third feature amounts having coordinates corresponding to the coordinates of each group first feature amount do not exist, character descriptions do not exist in any items in the document form.

[0051] If the answer is YES in step S1006, the recording unit 105 determines whether the character attributes included in the group first feature match the character attributes included in the corresponding third feature identified based on the coordinates (step S1007).

[0052] When the determination result in step S1007 is YES and the character attributes match, the recording unit 105 generates a confirmation screen displaying a rectangular frame within the range of the recording character string based on the coordinates indicated by one or more third feature amounts in the image data currently being processed. The recording unit 105 outputs the confirmation screen to a monitor (step S1008). The operator can confirm the rectangular area displayed on this confirmation screen to confirm the recording character string that the image processing device 1 is going to record. This allows the operator to confirm whether there is anything missing in the recording character string. An icon image of either an OK or NG button is displayed on the confirmation screen. By selecting the OK button among these button icon images, the operator can indicate that there is nothing missing in the selection as the recording character string. On the other hand, by selecting the NG button among the button icon images, the operator can indicate that there is something missing in the selection as the recording character string.

[0053] In response to the operator's pressing of the icon image of the button, the recording unit 105 determines whether or not there is a missing selection of the recorded character string (step S1009). If there is no missing selection, the recording unit 105 records the character string included in the third feature in the recording table in association with the identification information of the document form (step S1010).

[0054] For example, assume that the third feature amount a3, the third feature amount b3, the third feature amount c3, and the third feature amount d3 are acquired from the image data of the document form. Assume that the third feature amount a3 matches the group first feature amount g11 recorded in advance in the database, the third feature amount b3 matches the group first feature amount g12, the third feature amount c3 matches the group first feature amount g13, and the third feature amount d3 matches the group first feature amount g14. In this case, the recording unit 105 records the character strings included in the third feature amount a3, the third feature amount b3, the third feature amount c3, and the third feature amount d3 in the recording table of the document form.

[0055] If the answer is NO in step S1006, NO in step S1007, or NO in step S1009, the recording unit 105 performs processing in the case where there is no third feature having coordinates corresponding to the coordinates indicated by the group first feature. Specifically, the recording unit 105 generates input image data of a form image in which an input field is provided in the range of coordinates of the group first feature where no third feature of the corresponding coordinates in the image data does not exist, and outputs the generated data to the monitor (step S1011). The input image data may be data described in a markup language such as HTML or XML. While viewing the input image data, the worker operates an input device such as a keyboard of the image processing device 1 to input a recording character string into the input field in the input image data displayed on the monitor. A save button is displayed in the input image data, and when the save button is pressed, the recording unit 105 generates a third feature including the character string newly input into the input field of the input image data in addition to the third feature already acquired for the document form (step S1012). The recording unit 105 records the identifier of the form image data and the character string entered in the input field in the database 4 in association with each other. The image processing device 1 performs the processing flow shown in Fig. 9 again to update the group first feature and the group second feature, thereby expanding the range of character strings that can be automatically recorded. As a result, the next time the same document form is processed, the character string can be automatically recorded, eliminating the need for the worker to input the character string. The recording unit 105 records the character strings contained in all the third features in the record table of the document form (step S1013).

[0056] According to this process, the image processing device 1 can automatically record the record character string in the image data of a document form, regardless of the type of the newly input document form, by using the image data and record character strings of a plurality of different document forms recorded in advance by the worker. Therefore, the image processing device 1 can reduce the labor of the worker in recording the record character string in the document form. Even if the record character string is not entered in the document form, if the description item corresponding to the record character string that should be entered is not entered, the image processing device 1 outputs the input image data. This makes it possible to find an error in which an entry item that should be entered in the document form has not been entered, and to easily record the record character string indicated by that entry item.

[0057] <Third embodiment> As another example of the processing of the image processing device 1, an operator may register a group of document forms in advance in the image processing device 1. For example, when an operator previously registered image data of a document form, the operator inputs a group identifier according to the type of the document form, links it with the image data of the document form, and registers it in the database 4. This prevents different types of forms from being mixed in the same group due to processing errors of the image processing device 1, and makes it possible to extract the first group feature and the second group feature with high accuracy. In this case, the operator inputs the group of the document form at the time of registration, but for a new form, the group is identified using the second group feature as in step S1004.

[0058] <Fourth embodiment> As another example of the processing of the image processing device 1, in step S912, the image processing device 1 may group documents and forms using the individual first feature amount, or may group documents and forms using the individual second feature amount together with the individual first feature amount. The individual first feature amount is a feature amount of a recorded character string, but if the documents and forms are of the same type, the coordinates of the recorded character string and its character attributes are considered to be the same, and it becomes possible to group forms using the individual first feature amount. In this case, the acquisition unit 102 acquires a plurality of form image data and recorded character strings that are to be recorded among the character strings included in the form image data. The group classification unit 106 then groups the form image data based on the individual first feature. The read target feature generation unit 104 then generates a group first feature of the recorded character strings for each group using the individual first feature corresponding to the form image data included in the group. Alternatively, the initial grouping may be performed by an operator as described in the third embodiment, and new documents and forms may be grouped using the individual first feature values ​​in the process of step S1004. This makes it possible to read the recorded character strings with high accuracy in the OCR process.

[0059] <Fifth embodiment> In the second embodiment, a new group of forms is identified based on the second feature in step S1004. However, in another processing mode, the image processing device 1 does not perform the process of identifying the group, but identifies all groups set by the operator in order for each group, reads out the group first feature, and counts the number of matches with the third feature. In the case of a correct group, the group first feature and the third feature should match most frequently, so the image processing device 1 records the character strings included in each of the third feature of the identified group when the number of matches is greatest in step S1008. This makes it possible to record the recorded character strings without identifying the group. In this case, the image data of the document form stored in the database 4 for generating the group first feature may be grouped in advance by an operator. The acquisition unit 102 acquires a plurality of form image data and recorded character strings that are to be recorded among character strings included in the form image data. The feature extraction unit 103 then extracts individual first feature values ​​that indicate characteristics of the recorded character strings based on the result of character recognition processing of the form image data acquired by the acquisition unit 102. The read target feature generation unit 104 generates a group first feature value of the recorded character strings for each group using the individual first feature values ​​corresponding to the form image data included in a predetermined group that has been set in advance.

[0060] Sixth Embodiment 7 in the first embodiment, the image processing device 1 may perform machine learning of features in both the case where character recognition is successful and the case where character recognition is unsuccessful. This point will be described in the sixth embodiment. The sixth embodiment is similar to the first embodiment except for the process of FIG. 7 of the first embodiment, and FIGS. 1 to 6 are cited.

[0061] Fig. 11 is a diagram showing a processing flow of an image processing apparatus according to a sixth embodiment. Steps S1101 to S1108 in Fig. 11 are similar to steps S701 to S708 in Fig. 7. Steps S1110 to S1112 in Fig. 11 are similar to steps S709 to S711 in Fig. 7. Step S1114 in Fig. 11 is similar to step S712 in Fig. 7. 7. Therefore, in the processing of FIG. 11, step S1109 is inserted between steps S708 and S709 in FIG. 7, and step S1113 is inserted between steps S711 and S712 in FIG. 7, and otherwise it is the same as FIG. In either step S1109 or S1113, the read target feature generating unit 104 of the image processing device 1 performs the process of Fig. 6 to update the first feature. The read target feature generating unit 104 does not need to repeat the process of Fig. 6 from an unlearned state, and only needs to perform additional learning for one sheet of image data.

[0062] Here, in order to perform learning only when the OCR process fails (when the recorded character string cannot be properly obtained by the OCR process), it is conceivable to not perform machine learning in step S1109 in Fig. 11 and to perform machine learning only in step S1113. However, in this case, if the image processing device 1 succeeds in the OCR process, machine learning is not performed, and as a result, statistical information on the format of the document form may not be reflected in the machine learning. For example, consider a case where 99 out of 100 documents have the recorded character string in the bottom left, and only one has the recorded character string in the top right. In this case, if the image processing device 1 succeeds in OCR processing for documents with the recorded character string in the same position (step S1108: YES), it performs learning once for each of the cases where the recorded character string is in the bottom left and the top right. Although the actual ratio is 99:1, the image processing device 1 performs learning at a 1:1 ratio, which may result in over-learning for the case where the recorded character string is in the top right. In response to this, as shown in FIG. 11, the image processing device 1 can perform machine learning (updating the first feature) whether the OCR processing is successful or unsuccessful, thereby allowing the statistical information of the document form format to be reflected in the machine learning. Similarly, in the second to fifth embodiments, the image processing device 1 may perform the process of FIG. 9 either between steps S1009 and S1010 or between steps S1012 and S1013 in FIG.

[0063] Seventh Embodiment The image processing device 1 may acquire data from a system that scans or photographs documents, such as an existing OCR system, and perform machine learning. This point will be described in the seventh embodiment. In the following, an example will be described in which the image processing device 1 acquires data from an existing OCR system. However, the source of data provided to the image processing device 1 is not limited to an existing OCR system, and may be various systems or devices. For example, the image processing device 1 may acquire data from a system (for example, a sales information system or an accounting information system) that stores the date, destination, amount, and image data serving as evidence of sales in a database. In particular, the system or device providing the information does not need to have an OCR function.

[0064] Fig. 12 is a diagram showing an example of connection between an image processing system according to the seventh embodiment and an existing OCR system. In the configuration shown in Fig. 12, in addition to the configuration in Fig. 1, an existing OCR system 9 is connected to an image processing device 1. In this configuration, the existing OCR system 9 uses an image data generation function (scanning function) and a user interface, and does not use a character recognition function. Character recognition is performed by the image processing device 1. The other points are the same as those in the first embodiment. Figs. 1 to 7 are referred to and replaced with the acquisition of image data and the user interface as using the existing OCR system 9.

[0065] Fig. 13 is a diagram showing a processing flow of the image processing device according to the seventh embodiment. Fig. 13 shows the relationship between the processing performed by the image processing device 1 and the processing performed by the existing OCR system 9. In the processing of Fig. 13, the existing OCR system performs scanning according to the operation of the operator and generates image data (sequence S1301). The existing OCR system 9 transmits the obtained image data to the image processing device 1 (sequence S1302). Upon receiving the image data, the image processing device 1 performs OCR processing (sequence S1303). For example, the image processing device 1 performs the processes from step S702 in Fig. 7 onwards to obtain a recorded character string. In particular, the recording unit 105 identifies the recorded character string from among the character strings resulting from the OCR processing. The image processing apparatus 1 transmits the character string (recorded character string) resulting from the character recognition to the existing OCR system 9 (sequence S1304). The existing OCR system 9 presents the character recognition result to the operator, and receives a confirmation operation after confirmation and correction of the character recognition result (sequence S1305). The existing OCR system 9 transmits the confirmed character string (recorded character string of the correction result) to the image processing device 1 (sequence S1306). Upon receiving the correction result, the image processing device 1 performs machine learning (sequence S1307). The image processing device 1 acquires image data in sequence S1302, and acquires a character string as a result of correction confirmed by an operator in sequence S1306. Using this image data and character string, machine learning can be performed, for example, according to the processing procedure in Fig. 6. The image processing device 1 may perform machine learning collectively after a certain amount of data has been accumulated, or may perform additional learning each time data is obtained. In this way, according to the seventh embodiment, the image processing device 1 can perform machine learning by utilizing the general use by an operator of the existing OCR system 9. In particular, the read object feature amount generation unit 104 can update the first feature amount. Similarly, in the second to sixth embodiments, the image processing device may perform machine learning using an existing OCR system.

[0066] Eighth embodiment The learning function of the image processing device may be implemented as a server to accelerate the learning process, which will be described in the eighth embodiment. 14 is a diagram showing an outline of an image processing system 200 according to an eighth embodiment. In the configuration of FIG. 14, the image reading device 2 is the same as that in FIG. The functions of the image processing device 1 in FIG. 1 are divided into a learning function unit 1b that performs machine learning and an OCR function unit 1a that performs other functions, and are configured as a server-client. In particular, the learning function unit 1b generates and updates the first feature amount. The learning function unit 1b also performs the function of the recording device 3 to manage the learning result database 4b. The OCR function unit 1a performs functions such as OCR processing, extraction of the third feature amount, identification of recorded character strings, and user interface for operators. The OCR function unit 1a detects character strings, detects the characteristics of characters contained in the character strings, and detects the coordinates of the range of the character strings in the OCR processing. The OCR function unit 1a is stored in a terminal device 6. The learning function unit 1b is stored in a server device 7. The database 4 in Fig. 1 is divided into an OCR result database 4a and a learning result database 4b. The OCR result database 4a stores the results of OCR processing such as a record table. The learning result database 4b stores data obtained during machine learning such as the first feature amount. The learning result database 4b is stored in the server device 7. On the other hand, the OCR result database is configured as a single device.

[0067] FIG. 15 is a diagram showing a processing flow of a terminal device according to the eighth embodiment. 15, the terminal device 6 reads image data of a document form using the image processing device 2, and downloads the learning result corresponding to the image data from the server device 7 (step S1501). In particular, the terminal device 6 acquires a first feature amount as the learning result. The terminal device 6 performs OCR processing on all character strings included in the image data, and displays on the screen those that match the learning result (step S1502). Specifically, the terminal device 6 uses the first feature amount that is the learning result to identify recorded character strings that match the first feature amount.

[0068] The terminal device 6 determines whether all the OCR target character strings in the image data have been correctly read (step S1503). If it is determined that there are character strings that have not been read (step S1503: NO), the terminal device 6 displays the OCR results on a screen to the worker and accepts correction operations (step S1504). The terminal device 6 transmits the read character string (or the corrected character string if the character string is corrected by the operator) to the OCR result database 4a and records the character string (step S1505). Specifically, the terminal device 6 stores the recorded character string in the OCR result database 4a. If the answer is YES in step S1503, the process also proceeds to step S1505. The terminal device 6 transmits the read character string, position information, and other learning data to the server device 7 (step S1506). Through the process of FIG. 15, the terminal device 6 records the recorded character string to be recorded as the OCR result in the OCR result database, and also provides the server device 7 with learning data.

[0069] FIG. 16 is a diagram showing a processing flow of the server device according to the eighth embodiment. In the process of FIG. 16, the server device 7 receives and stores learning data from the terminal device 6 (step S1601). The server device 7 determines whether or not N pieces of unlearned learning data (N is a positive integer) have been accumulated (step S1602). If it is determined that no unlearned learning data has been accumulated (step S1602: NO), the process returns to step S1601. On the other hand, if it is determined that N pieces of unlearned learning data have been accumulated (step S1602: YES), the server device 7 reads reception result data of the form received as learning data from the learning result database (step S1603). Next, the server device 7 adds new learning data to the read learning result data, and performs re-learning (step S1604). The server device 7 registers the re-learning result in the learning result database 4b (step S1605). Then, the server device 7 waits until it receives the learning data from the terminal device 6 (step S1606). After step S1606, the process returns to step S1601. 16, the server device 7 updates the first feature amount as a learning result. The updated first feature amount is registered in the learning result database 4b and made available for use by the terminal device 6. According to the eighth embodiment, the learning function of the image processing device can be implemented as a server, and learning data can be consolidated to accelerate learning.

[0070] The timing when the server device 7 performs machine learning such as re-learning is not limited to the timing when a certain amount of learning data is accumulated as shown in step S1602 in Fig. 16. For example, the server device 7 may perform learning at regular intervals. In particular, the server device 7 may start learning according to a time condition in addition to or instead of the condition of the number of data.

[0071] An example of the server device 7 performing learning at regular intervals is when the server device 7 performs learning at night or on holidays when the terminal device 6 is not in operation. If the server device 7 performs learning while the terminal device 6 is in operation, the learning function unit 1b of the server device 7 will access the learning result database. This may cause inconveniences such as a delay in access from the terminal device 6 to the learning result database 4b, or inability to access at the required timing. Therefore, by having the server device 7 perform learning when the terminal device 6 is not in operation, it is possible to avoid any impact on the access from the terminal device 6 to the learning result database 4b.

[0072] In addition to or instead of the server device 7, the terminal device 6 may perform machine learning. Here, if the server device 7 performs machine learning and the terminal device 6 does not perform learning, the processing accuracy of the terminal device 6 will not improve until the server device 7 performs machine learning, updates the learning result (first feature amount), and shares the learning result with the terminal device 6. For example, if the server device 7 performs machine learning only at night every day, the terminal device 6 will not be able to use the updated learning result until the next morning. For example, even if the terminal device 6 generates learning data in the morning and transmits it to the server device 7, the learning result will not be reflected in the processing of the terminal device 6 until the next morning, about one day later.

[0073] Therefore, each of the terminal devices 6 performs machine learning every time it processes one form. The terminal device 6 reflects the learning result in its own processing, and transmits the learning result or learning data to the server device 7. The terminal device 6 generates a feature (third feature) by processing the form. Then, the terminal device 6 updates the first feature acquired from the learning result database using the generated feature. For example, in step S1506 of FIG. 15, the terminal device 6 transmits the learning data to the server device 7, and performs machine learning by itself using the learning data, thereby updating the first feature stored in the terminal device 6. Alternatively, in step S1506 of FIG. 15, the terminal device 6 may transmit the learning result (obtained feature) to the server device 7 in addition to the learning data. By learning by itself, the terminal device 6 can reflect the results of its own learning in its own processing. In this respect, it is expected that the processing accuracy of the terminal device 6 will improve faster than when the terminal device 6 does not perform machine learning by itself and waits for the learning results to be updated by the server device 7. The machine learning performed by each of the terminal devices 6 is called provisional learning.

[0074] When the terminal device 6 transmits learning data to the server device 7, the process performed by the server device 7 is the same as that described with reference to FIG. On the other hand, when the terminal device 6 transmits the learning result to the server device 7, the server device 7 accumulates the learning result from each terminal device 6. Then, the server device 7 reflects the learning result obtained from the terminal device 6, for example, at night, in the learning result stored in the learning result database 4b. Specifically, the learning function unit 1b of the server device 7 updates the first feature stored in the learning database 4b by using the feature acquired by the terminal device 6 in learning. The process of updating the first feature amount stored in the learning result database is referred to as main learning.

[0075] FIG. 17 is a diagram showing the minimum configuration of an image processing device. As shown in this figure, the image processing device 1 is required to include at least a feature extraction unit 103 and a read object feature generation unit 104. The feature extraction unit 103 extracts, for each piece of form image data, a feature that indicates the characteristics of a character string contained in the form image data, based on the result of character recognition processing of a plurality of previously registered form image data. The read target feature generating unit 104 generates a first feature of a recorded character string in the form image data by using a feature corresponding to the form image data.

[0076] Each of the above-mentioned devices has a computer system inside. The programs for causing each device to perform the above-mentioned processes are stored in computer-readable recording media of the devices, and the above processes are performed by the computer of each device reading and executing the programs. Here, computer-readable recording media refers to magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memories, etc. Also, the computer programs may be distributed to a computer via a communication line, and the computer that receives the programs may execute the programs.

[0077] The above program may be for implementing some of the functions of the above-mentioned processing units, or may be a so-called differential file (differential program) that can implement the above-mentioned functions in combination with a program already recorded in the computer system. [Explanation of symbols]

[0078] 1. Image processing device 1a OCR function section 1b Learning function section 2. Image reading device 3. Recording device 4. Database 4a Learning results database 4b OCR result database 5 Document Form 6 Terminal equipment 7. Server equipment 101...Control section 102...Acquisition part 103 Feature extraction unit 104: Reading object feature quantity generation unit 105 Recording section 106 Group Classification Section 107 Group Specific Department 108... Work object determination unit 109... Work data generation unit

Claims

1. an acquisition unit that acquires image data of a document form and a recorded character string that is recorded in a database by an operator as a character string to be recorded in association with the image of the document form among character strings included in the image data, and also acquires image data of a new document form; a feature extraction unit that extracts a feature that is information indicating a feature of a character string that matches the recorded character string from a character string recognized by a character recognition process on the image data; a read object feature quantity generating unit that generates, as a read object feature quantity, a feature quantity common to the document form of the same format by machine learning using a feature quantity of a character string that matches the recorded character string among character strings extracted from image data of each of a plurality of documents and forms of the same format; a recording unit which records, for each character string recognized by a character recognition process on the image data of the new document form acquired by the acquisition unit, the character string recognized from the image data of the new document form when the character string extracted by the character string extraction unit matches the character string of the read target generated by the read target character string generation unit; Equipped with The feature amount is information indicating an attribute of the character string and a range of the character string in the image of the document form. Image processing device.

2. The read target feature amount generation unit performs machine learning on the feature amount of the character string extracted from the image data for each group in which the image data of each of the plurality of documents / forms is grouped in advance according to the format of the document / form, and generates the feature amount of the read target for each group. The image processing device according to claim 1 .

3. a group classification unit that classifies the image data of each of the plurality of document forms into groups based on a degree of matching between the plurality of document forms and features of character strings that are indicated by feature amounts of character strings that match the recorded character strings and are extracted from the image data of each of the plurality of document forms; The image processing device according to claim 2 .

4. The feature extraction unit further extracts features of character strings other than the character string that matches the recorded character string from among character strings extracted from the image data of each of the plurality of documents and forms; a group classification unit that classifies the image data of each of the plurality of document forms into groups based on a degree of matching indicating a degree of matching between the plurality of document forms of features of character strings, which are indicated by feature amounts of character strings other than the character string that matches the recorded character string, extracted from the image data of each of the plurality of document forms; The image processing device according to claim 2 .

5. the read target feature generating unit generates a feature common to the documents / forms included in the same group from feature amounts of character strings other than the character string matching the recorded character string, the feature amounts being extracted from the image data of the documents / forms included in the same group, thereby generating feature amounts for each of the groups; a group identification unit that, when a feature indicated by the feature amount for each group coincides with a feature indicated by the feature amount extracted from the image data of the new document form, identifies the group as the group to which the image data of the new document form belongs; the recording unit records a character string recognized from the image data of the new document form when the feature amount extracted from the image data of the new document form matches a feature amount of the read target of the specified group. The image processing device according to claim 4.

6. Acquiring image data of a document form and, among character strings included in the image data, a recorded character string that an operator has recorded in a database in association with the image of the document form as a character string to be recorded; extracting a feature amount, which is information indicating a feature of a character string that matches the recorded character string, from a character string recognized by a character recognition process on the image data; generating, by machine learning using features of character strings that match the recorded character string among character strings extracted from image data of a plurality of documents having the same format, a feature common to the documents having the format as a feature to be read; Acquire image data of new documents and forms, For each character string recognized by character recognition processing on image data of a new document form, if a feature amount extracted as information indicating the characteristics of the character string matches a feature amount of the read target generated by the method for generating feature amounts of the read target by the machine learning, the character string recognized from the image data of the new document form is recorded; The feature amount is information indicating an attribute of the character string and a range of the character string in the image of the document form. Image processing methods.

7. On the computer, acquiring image data of a document form and, among character strings included in the image data, a recorded character string that an operator has recorded in a database in association with the image of the document form as a character string to be recorded; extracting a feature amount, which is information indicating a feature of a character string that matches the recorded character string, from a character string recognized by a character recognition process on the image data; generating, by machine learning using features of character strings that match the recorded character string among character strings extracted from image data of a plurality of documents having the same format, a feature common to the documents having the same format as a feature to be read; acquiring image data of a new document; a step of recording the character string recognized from the image data of the new document form when a feature amount extracted as information indicating a feature of each character string recognized by a character recognition process on the image data of the new document form acquired in the step of acquiring the image data of the new document form matches the feature amount of the read target generated in the step of generating feature amounts of the read target by the machine learning; Run the command, The feature amount is information indicating an attribute of the character string and a range of the character string in the image of the document form. program.

Images