Information Processing System and Information Processing Method

Abstract

To generate a template file used to recognize an object from image data of a fixed form document without requiring manual operation of a user.SOLUTION: An information processing apparatus includes: a marker extracting unit that extracts a marker from image data, and determines a marker position indicating a position of the marker; a character data extracting unit that extracts character data from the image data, and determines a character position indicating a position of the character data; an attribute data setting unit that sets character data, the character position of which has a specific relationship with the marker position, as an attribute of the marker; and a template generating unit that sets an area to be recognized, which is an area as a target of object recognition, on the basis of the marker position, and generates a template file including the marker position, the character data set as the attribute of the marker, and the area to be recognized.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing program, and an information processing method for generating a template file used to recognize an object from image data of a fixed-form document. The present disclosure relates to an information processing system having this information processing apparatus.

Background Art

[0002] As a method for extracting variable objects (for example, characters indicating an amount in a form, or a check mark in a check box in a check sheet) from image data of fixed-form documents such as forms and check sheets, an optical character recognition (OCR) process using a template file is known.

[0003] When an information processing apparatus executes this template file, it performs OCR processing on a designated area and extracts variable objects (for example, characters indicating an amount in a form, or a check mark in a check box in a check sheet) entered in this area. Thus, the OCR process using a template file is effective for extracting variable objects (for example, characters, check marks) from fixed-form documents.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] In OCR processing using a template file, a template file including an area manually specified by the user in advance is created. For example, Patent Document 1 automatically sets an area for performing recognition processing such as OCR processing, but it is necessary to input area designation document data prepared by the user in advance. Patent Document 2 sets whether to apply recognition processing such as OCR processing for each object when creating form data.

[0006] In OCR processing using a template file, there are also cases where an object (check mark) entered in a large number of check boxes in a check sheet in which a large number of check items are described is extracted. It is considered that the check sheet is more likely to have its document content and layout frequently changed compared to a form, such as when the number of check items increases or decreases. In the case of such a document, each time the document content and layout are changed, it can be a great deal of trouble for the user to manually create a template file.

[0007] In view of the above circumstances, an object of the present disclosure is to generate a template file used for recognizing an object from image data of a regular document without requiring manual operation by the user.

Means for Solving the Problem

[0008] An information processing apparatus according to one embodiment of the present disclosure a marker extraction unit that extracts a marker from the image data and determines a marker position indicating the position of the marker; a character data extraction unit that extracts character data from the image data and determines a character position indicating the position of the character data; an attribute data setting unit that sets character data having a predetermined relationship with the marker position and the character position as an attribute of the marker; sets a recognition target area that is an area to be recognized for object recognition based on the marker position, generates a template file including the marker position, the character data set as an attribute of the marker, and the recognition target area a template generation unit, and includes.

[0009] According to this embodiment, the information processing apparatus extracts marker and character data from image data, sets the character data as an attribute of the marker based on the relative positional relationship between the marker and the character data, and generates a template file including the marker position, the character data, and the recognition target area. Thereby, it is possible to automatically generate a template file from the image data without the user manually specifying the recognition target area.

[0010] The template file may be used to recognize an object included in the recognition target area from another image data having the same layout as the image data.

[0011] Thereby, variable objects can be automatically extracted from the standard document.

[0012] The marker is a checkbox, and the recognition target area may include the checkbox.

[0013] In the OCR process using the template file, there is also a case of extracting an object (check mark) entered in a large number of checkboxes in a check sheet in which a large number of check items are described. It is considered that the check sheet is likely to have its document content and layout frequently changed, such as the addition or deletion of check items. In the case of such a document, every time the document content and layout are changed, there is no need for the user to manually create a template file, and a template file can be automatically generated from the image data.

[0014] The attribute data setting unit, when there are a plurality of markers having marker positions in a predetermined relationship with respect to the character position of the character data, may determine a marker to which the character data should be set as an attribute based on the relationship between the marker positions of the plurality of markers.

[0015] It is conceivable that there are a plurality of markers having marker positions in a predetermined relative positional relationship with respect to the text position. For example, a marker is arranged in the first row, a character string is arranged in the second row, a marker is arranged in the third row, and a character string is arranged in the fourth row, and the intervals between the rows are equal. In this case, with respect to the character string in the second row, the markers in the first row and the third row have the same positional relationship. In this case, based on the relationship between the marker positions of the marker in the first row and the marker in the third row, a marker to which the character data of the character string in the second row should be set as an attribute is determined. For example, it can be determined that the marker in the preceding row (the marker in the first row) is set as an attribute for the character data of the character string in the second row.

[0016] The image data may be image data of a fixed-form document.

[0017] Using a template file, variable objects can be extracted from a fixed-form document.

[0018] The template file may be described in XML.

[0019] An information processing system according to an aspect of the present disclosure a marker extraction unit that extracts a marker from image data and determines a marker position indicating the position of the marker; a character data extraction unit that extracts character data from the image data and determines a character position indicating the position of the character data; an attribute data setting unit that sets character data having a predetermined relationship between the marker position and the character position as an attribute of the marker; sets a recognition target area, which is an area to be the target of object recognition, based on the marker position, generates a template file including the marker position, the character data set as an attribute of the marker, and the recognition target area a template generation unit; and an information processing apparatus having the same; A template file execution device having a template file execution unit that executes the template file to recognize an object included in a recognition target region from another image data having the same layout as the image data. It comprises.

[0020] An information processing program according to an aspect of the present disclosure causes a processor of an information processing apparatus to extract a marker from image data and determine a marker position indicating the position of the marker, a marker extraction unit; extract character data from the image data and determine a character position indicating the position of the character data, a character data extraction unit; set, as attribute data of the marker, character data in which the marker position and the character position are in a predetermined relationship, an attribute data setting unit; set a recognition target region, which is a region to be the target of object recognition, based on the marker position, and generate a template file including the marker position, the character data set as the attribute of the marker, and the recognition target region. Template generation unit to operate as.

[0021] An information processing method according to an aspect of the present disclosure extracts a marker from image data, determines a marker position indicating the position of the marker, extracts character data from the image data, determines a character position indicating the position of the character data, sets, as the attribute of the marker, character data in which the marker position and the character position are in a predetermined relationship, sets a recognition target region, which is a region to be the target of object recognition, based on the marker position, and generates a template file including the marker position, the character data set as the attribute of the marker, and the recognition target region.

Advantages of the Invention

[0022] According to the present disclosure, it is possible to generate a template file used for recognizing an object from image data of a standard document without requiring a manual operation by the user.

[0023] Note that the effects described herein are not necessarily limited, and any of the effects described in the present disclosure may be applicable.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0026] 1. Information Processing System

[0027] FIG. 1 shows an information processing system according to an embodiment of the present disclosure.

[0028] The information processing system 10 includes an information processing apparatus 100, a template file execution apparatus 200, an image forming apparatus 300, and a file server apparatus 400. The information processing apparatus 100, the template file execution apparatus 200, the image forming apparatus 300, and the file server apparatus 400 are communicably connected via a network N. The network N includes the Internet, a LAN (Local Area Network), and the like.

[0029] The information processing apparatus 100 and the template file execution apparatus 200 are computers in which a processor such as a CPU or GPU loads an information processing program recorded in a ROM into a RAM and executes it. The information processing apparatus 100 generates a template file. The template file execution apparatus 200 executes the template file generated by the information processing apparatus 100.

[0030] The image forming apparatus 300 is, for example, an MFP (Multifunction Peripheral). The image forming apparatus 300 scans a fixed-form document such as a form or a check sheet to generate image data and transmits it to the information processing apparatus 100.

[0031] The file server apparatus 400 has a large-capacity non-volatile storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The file server apparatus 400 stores image data of fixed-form documents such as forms and check sheets and the template file generated by the information processing apparatus 100.

[0032] 2. Configuration of Information Processing Apparatus

[0033] FIG. 2 shows the configuration of the information processing apparatus.

[0034] The information processing apparatus 100 operates as an image input unit 101, a marker extraction unit 102, a character data extraction unit 104, an attribute data setting unit 106, and a template generation unit 107 by a processor such as a CPU or GPU loading an information processing program recorded in a ROM into a RAM and executing it. The marker data storage unit 103 and the character data storage unit 105 are set in the storage area of the storage device.

[0035] 3. Operation Flow of Information Processing Apparatus

[0036] FIG. 3 shows the operation flow of the information processing apparatus.

[0037] The image input unit 101 acquires the image data generated by the image forming apparatus 300 or the image data stored in the file server apparatus 400 (step S101). The image data is the image data of a fixed-form document such as a form or a check sheet. The image data is scan data generated by optically reading a physical fixed-form document or PDF data storing a print image of an electronic fixed-form document.

[0038] FIG. 4 shows an example of the image data of a fixed-form document.

[0039] As an example of the image data of a fixed-form document, the image data 500 of a check sheet is shown. The image data 500 of the check sheet includes a plurality of check boxes B1, B2, B3 and a plurality of character strings T1, T2, T3.

[0040] The marker extraction unit 102 extracts markers from the image data 500 (step S102). The marker extraction unit 102 has learned the images of the markers to be extracted. A marker is a reference for setting a region to be an object of object recognition (recognition target region). In this example, the marker is a check box. That is, the marker extraction unit 102 extracts a plurality of check boxes B1, B2, B3 as markers from the image data 500. The marker extraction unit 102 determines the positions of the markers (marker positions). The marker positions are indicated by coordinates with respect to the entire region of the image data 500. That is, the marker extraction unit 102 determines the coordinates of each of the plurality of check boxes B1, B2, B3 with respect to the entire region of the image data 500 as the marker positions. The marker extraction unit 102 stores the marker positions of each of the plurality of check boxes B1, B2, B3 in the marker data storage unit 103 (step S103).

[0041] The character data extraction unit 104 extracts character data from the image data 500 by OCR processing (step S104). The character data extraction unit 104 performs OCR processing on all the character strings included in the image data 500 to extract character data. In this example, the character data extraction unit 104 extracts all the character data T1, T2, and T3 from the image data 500. The character data extraction unit 104 also extracts all the character strings located at positions other than near markers (checkboxes), such as titles and text bodies. The character data extraction unit 104 determines the positions (character positions) of the extracted character data. The character position is indicated by coordinates with respect to the entire area of the image data 500. That is, the character data extraction unit 104 determines the coordinates of each of the plurality of character data T1, T2, and T3 with respect to the entire area of the image data 500 as the character position. The character data extraction unit 104 stores, in the character data storage unit 105, the character positions of each of the plurality of character data T1, T2, and T3 and the character data T1, T2, and T3 themselves extracted by OCR processing in association with each other (step S105).

[0042] The attribute data setting unit 106 sets, as the attributes of the marker, the character data in which the marker position and the character position have a predetermined relative positional relationship (step S106). The attribute data setting unit 106 sets attributes for each of the plurality of markers one by one. The specific method is as follows.

[0043] FIG. 5 shows the operation flow of the attribute data setting unit.

[0044] The attribute data setting unit 106 reads the marker positions (coordinates) of the markers (checkboxes B1, B2, B3) from the marker data storage unit 103 (step S201). The attribute data setting unit 106 reads a plurality of character data T1, T2, T3 and their character positions from the character data storage unit 105 (step S202). The attribute data setting unit 106 determines, for each of the plurality of markers, character data in which the marker position and the character position have a predetermined relative positional relationship based on the marker positions of the plurality of markers (checkboxes B1, B2, B3) (step S203). In the case of the example in FIG. 4, for example, the attribute data setting unit 106 determines the character data T1 in which the character position is in the closest relationship to the marker position for the checkbox B1.

[0045] Incidentally, as another example (not shown), it is conceivable that there are a plurality of markers having marker positions in a predetermined relative positional relationship with respect to the character positions of the character data. For example, markers are arranged in the first row, character strings in the second row, markers in the third row, and character strings in the fourth row, and the intervals between the rows are equal. In this case, for the character string in the second row, the markers in the first row and the third row have the same positional relationship. In this case, the attribute data setting unit 106 determines the marker for which the character data should be set as an attribute based on the relationship of the marker positions of the plurality of markers. That is, the attribute data setting unit 106 determines the marker for which the character data of the character string in the second row should be set as an attribute based on the relationship between the marker position of the marker in the preceding row (the marker in the first row) and the marker position of the marker in the third row. For example, the attribute data setting unit 106 determines that the marker in the preceding row (the marker in the first row) should be set as the attribute for the character data of the character string in the second row.

[0046] The attribute data setting unit 106 determines character data (character data T1, T2, T3) in which the marker position and the character position have a predetermined relative positional relationship for each of all markers (checkboxes B1, B2, B3) (step S204). In this example, the attribute data setting unit 106 determines character data T1 for checkbox B1, determines character data T2 for checkbox B2, and determines character data T3 for checkbox B3. The attribute data setting unit 106 does not set, as the attribute of any marker, character data in which the character position has no predetermined relationship with any marker position (step S205, no). For example, the attribute data setting unit 106 does not set, as the attribute of any marker, character data located at a position other than near the marker (checkbox), such as a title or a text body.

[0047] The attribute data setting unit 106 sets the determined character data as the attribute of the marker (step S206). In this example, the attribute data setting unit 106 sets character data T1 as the attribute of marker B1, sets character data T2 as the attribute of marker B2, and sets character data T3 as the attribute of marker B3.

[0048] Based on the marker positions, the template generation unit 107 sets an area (recognition target area) to be the target of object recognition by OCR processing. Specifically, the recognition target area is indicated by coordinates with respect to the entire area of the image data 500. The recognition target area is an area where variable objects should be filled in. When the marker is a checkbox, the area where a variable object (i.e., a check mark) should be filled in, that is, the area including the checkbox (i.e., the area inside the checkbox) is the recognition target area. In this example, based on the marker position (coordinates) of the checkbox B1, the template generation unit 107 sets the coordinates of the area including the checkbox B1 as the recognition target area. The template generation unit 107 associates the marker position of the checkbox B1, the character data T1 set as an attribute of the checkbox B1 (marker), and the recognition target area (inside the checkbox B1). The template generation unit 107 associates the marker position of the checkbox B2, the character data T2 set as an attribute of the checkbox B2 (marker), and the recognition target area (inside the checkbox B2). The template generation unit 107 associates the marker position of the checkbox B3, the character data T3 set as an attribute of the checkbox B3 (marker), and the recognition target area (inside the checkbox B3).

[0049] The template generation unit 107 generates a template file including these marker positions (coordinates), the character data set as an attribute of the marker, and the recognition target area (coordinates) (step S107). The template file is described in, for example, XML (Extensible Markup Language). The template generation unit 107 stores the generated template file in the file server device 400. The template file is used to recognize variable objects (i.e., check marks) included in the recognition target area (inside the checkbox) from another image data with the same layout as the image data 500 of the check sheet by OCR processing.

[0050] After that, the template file execution device 200 acquires and executes a template file from the file server device 400. By executing the template file, the template file execution device 200 recognizes a variable object (i.e., a check mark) included in the recognition target area (inside the check box) from another image data having the same layout as the image data 500 of the check sheet by OCR processing.

[0051] 4. Conclusion

[0052] As a method for extracting variable objects from image data of regular documents, OCR processing using a template file is known. In OCR processing using a template file, a template file including an area manually specified by the user in advance is created.

[0053] In OCR processing using a template file, there are cases where objects (check marks) entered in a large number of check boxes in a check sheet with a large number of check items are extracted. It is considered that the check sheet is more likely to have its document content and layout frequently changed compared to forms, such as when the number of check items increases or decreases. For example, at sites where check sheets are frequently used, such as in the education field, check sheets with different contents are created each time. In a check sheet for collecting real estate information, it is necessary to check a large number of check items according to various properties, from bath facilities to antennas and lines. In the case of such documents, each time the document content and layout are changed, it can be a great deal of trouble for the user to manually create a template file.

[0054] In contrast, according to the present embodiment, the information processing apparatus 100 extracts marker and character data from the image data, sets the character data as an attribute of the marker based on the relative positional relationship between the marker and the character data, and generates a template file including the marker position, the character data, and the recognition target area. Thereby, it is possible to automatically generate a template file from the image data without the user having to manually specify the recognition target area.

[0055] Although each embodiment and each modification example of the present technology have been described above, the present technology is not limited to only the above-described embodiments, and it goes without saying that various changes can be made without departing from the gist of the present technology.

Explanation of Reference Numerals

[0056] 10 Information processing system 100 Information processing apparatus 101 Image input unit 102 Marker extraction unit 103 Marker data storage unit 104 Character data extraction unit 105 Character data storage unit 106 Attribute data setting unit 107 Template generation unit 200 Template file execution apparatus 300 Image forming apparatus 400 File server apparatus 500 Image data

Claims

A marker extraction unit that extracts a marker from image data that is image data of a fixed-form document and determines a marker position indicating the position of the marker, A character data extraction unit that extracts character data from the image data and determines a character position indicating the position of the character data, An attribute data setting unit that sets character data in which the marker position and the character position are in a predetermined relationship as an attribute of the marker, A recognition target area setting unit that sets a recognition target area that is an area to be recognized for object recognition based on the marker position, A template generation unit that generates a template file including the marker position, the character data set as an attribute of the marker, and the recognition target area, An information processing apparatus having a template generation unit, And an information processing system having a template file execution unit that executes the template file and recognizes an object included in the recognition target area from another image data having the same layout as the image data. A template file execution apparatus having a template file execution unit that executes the template file and recognizes an object included in the recognition target area from another image data having the same layout as the image data, An information processing system comprising:

2. The information processing system according to claim 1, Wherein the template file is used to recognize an object included in the recognition target area from another image data having the same layout as the image data. An information processing system.

3. The information processing system according to claim 1 or 2, Wherein the marker is a checkbox, And the recognition target area includes the checkbox. An information processing system.

4. The information processing system according to any one of claims 1 to 3, Wherein the attribute data setting unit, When there are a plurality of markers having marker positions in a predetermined relationship with respect to the character positions of the character data, a marker to be set with the character data as an attribute is determined based on the relationship of the marker positions of the plurality of markers. Information processing system.

5. The information processing system according to any one of claims 1 to 4, The template file is described in XML. Information processing system.

6. Extract markers from image data that is image data of a standard document, determine marker positions indicating the positions of the markers, Extract character data from the image data, determine character positions indicating the positions of the character data, Set character data in which the marker position and the character position are in a predetermined relationship as an attribute of the marker, Set a recognition target area, which is an area to be the target of object recognition, based on the marker position, Generate a template file including the marker position, the character data set as an attribute of the marker, and the recognition target area. A template file execution device executes the template file to recognize an object included in the recognition target area from another image data having the same layout as the image data. Information processing method.

Images