Information processing device and information processing program

Abstract

To allow for associating attribute values of sub-attributes with specific attributes later on even when the attribute values of the sub-attributes for supplementing content of attributes from an image cannot be acquired together in association with attribute values of the specific attributes.SOLUTION: An information processing device 10 provided herein is configured to extract attribute values of multiple attributes from an OCR result of an image 2 according to an extraction rule 18, generate an extraction result in which the extracted attribute values are associated with respective attributes, and associate attribute values of sub-attributes for supplementing content of the attributes included in the extraction result with attributes with content to be supplemented according to an association rule 19 for associating the attribute values of the sub-attributes with the attributes with content to be complemented.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device and an information processing program. [Background technology]

[0002] Patent Document 1 discloses an image processing system having an input means for inputting a document image having a plurality of regions each containing a character string, a division means for dividing the document image input by the input means into regions, a recognition means for recognizing a character string from the document image input by the input means, a determination means for determining whether the character string recognized by the recognition means for each region divided by the division means is included in a character string pre-stored for each region, and a conversion means for converting the character string recognized by the recognition means into a specific character string corresponding to the character string pre-stored for each region when the determination means determines that the character string recognized by the recognition means is included in the character string pre-stored for each region. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-84186 Summary of the Invention [Problem to be solved by the invention]

[0004] 2. Description of the Related Art Known extraction techniques for extracting attribute values ​​relating to specific attributes from an image that has undergone OCR (Optical Character Recognition) processing include a key-value extraction method and a character pattern extraction method.

[0005] However, with these extraction methods, if a character string that supplements the content of an attribute is present in a position other than the pre-specified position, it is impossible to obtain supplementary information about the content of the attribute value extracted by these extraction methods. Therefore, compared to when the attribute value of a specified attribute is extracted from an image together with the supplementary information, it may be difficult to use the extracted result of the attribute value in subsequent processing.

[0006] The technology disclosed herein aims to provide an information processing device and an information processing program that can later associate an attribute value of a secondary attribute with a specific attribute, even if it is not possible to associate and acquire the attribute value of a secondary attribute that supplements the content of the attribute from an image with the attribute value of the specific attribute. [Means for solving the problem]

[0007] An information processing device according to a first aspect includes a processor that extracts attribute values ​​of a plurality of attributes from an OCR result of an image according to predetermined extraction rules, generates an extraction result in which the extracted attribute values ​​correspond to each attribute, and controls associating the attribute values ​​of secondary attributes, which are attributes whose attribute values ​​are extracted from the image and which supplement the content of other attributes, with other attributes included in the extraction result according to association rules that associate the attribute values ​​of the secondary attributes with the other attributes.

[0008] In the information processing device of the second aspect, in the information processing device of the first aspect, the association rule includes conditions for making the association and association information that specifies how to associate the other attribute with the secondary attribute, and when the conditions are met, the processor controls associating the attribute value of the secondary attribute with the other attribute in accordance with the association information.

[0009] An information processing device according to a third aspect is an information processing device according to the second aspect, wherein the conditions include content that specifies at least one of the position of the attribute value of the secondary attribute in the image and the content of the attribute value of the secondary attribute.

[0010] An information processing device according to a fourth aspect is an information processing device according to any one of the first to third aspects, wherein when the processor associates the secondary attribute with the other attribute, the processor also controls to associate with the other attribute an attribute value of a related secondary attribute, which, like the secondary attribute, supplements the content of the other attribute, and for which an attribute value is not associated by the extraction rule, and whose attribute value is uniquely determined by using the attribute value of the secondary attribute.

[0011] An information processing device according to a fifth aspect is the information processing device according to the fourth aspect, wherein the associated sub-attribute is an attribute whose attribute value is not included in the image.

[0012] An information processing device according to a sixth aspect is the information processing device according to any one of the first to fifth aspects, wherein the extraction results are represented by a data structure that hierarchically manages related attributes.

[0013] An information processing device according to a seventh aspect is an information processing device according to any one of the first to sixth aspects, wherein the attribute value of the secondary attribute is an attribute value that is located in a position that will not be extracted as an attribute value of the other attribute even when extracted from the image according to the extraction rule.

[0014] An information processing program according to an eighth aspect is a program for causing a computer to execute control to extract attribute values ​​of a plurality of attributes from an OCR result of an image in accordance with predetermined extraction rules, generate extraction results in which the extracted attribute values ​​are associated with each attribute, and associate attribute values ​​of secondary attributes, which are attributes whose attribute values ​​are extracted from the image and which supplement the content of other attributes, with other attributes included in the extraction results in accordance with association rules that associate the attribute values ​​of the secondary attributes with the other attributes. [Effects of the Invention]

[0015] According to the first and eighth aspects, even if it is not possible to associate the attribute value of a secondary attribute that supplements the content of an attribute from an image with the attribute value of a specific attribute and obtain them together, it is possible to later associate the attribute value of the secondary attribute with the specific attribute.

[0016] According to the second aspect, there is an effect that the form of association between another attribute and a sub-attribute can be changed by changing the definition of the association rule.

[0017] According to the third aspect, there is an effect that the form of association between another attribute and a secondary attribute can be changed using at least one of the position of the attribute value of the secondary attribute in the image and the content of the attribute value of the secondary attribute.

[0018] According to the fourth aspect, there is an effect that an attribute value can be associated with an attribute for which an attribute value could not be set using the extraction rule and the association rule.

[0019] According to the fifth aspect, there is an effect that an attribute value can be associated with an attribute that is not included in an image.

[0020] According to the sixth aspect, there is an effect that it is possible to grasp the relevance between attributes from the extraction result of the attribute values.

[0021] According to the seventh aspect, it is possible to associate the attribute value of a secondary attribute that is located at a position not assumed in the extraction rule with another attribute. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 2 is a block diagram illustrating an example of a functional configuration of an information processing device. [Figure 2] FIG. 10 is a diagram illustrating an example of a known extraction method for extracting attribute values ​​for attributes from OCR results. [Figure 3] FIG. 10 is a diagram showing an example in which the attribute value of a secondary attribute cannot be extracted from the OCR result. [Figure 4] FIG. 2 is a diagram illustrating an example of the configuration of a main part of an electrical system in an information processing device. [Figure 5] FIG. 10 is a diagram illustrating an example image of an estimate. [Figure 6] 10 is a flowchart showing an example of the flow of an association process. [Figure 7] FIG. 10 is a diagram showing an example of an extraction result of attribute values ​​extracted from an OCR result of an estimate. [Figure 8] FIG. 10 is a diagram illustrating an example of an association rule applied to a quotation. [Figure 9] FIG. 10 is a diagram showing an example of an extraction result of attribute values ​​in an estimate in which attributes are associated with each other. [Figure 10] FIG. 10 is a diagram showing an example of an extraction result of attribute values ​​in an estimate in which attribute values ​​of related sub-attributes are set. [Figure 11] FIG. 10 is a diagram showing an example image of an invoice. [Figure 12] This is an example of the extraction results of attribute values ​​extracted from the OCR results of an invoice. [Figure 13] FIG. 10 illustrates an example of association rules applied to invoices. [Figure 14] FIG. 10 is a diagram showing an example of the extraction results of attribute values ​​in an invoice in which attributes are associated with each other. [Figure 15] FIG. 10 is a diagram showing an example of the extraction result of attribute values ​​in an invoice in which attribute values ​​of related sub-attributes are set. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, embodiments of the technology of the present disclosure will be described with reference to the drawings. Note that the same components and processes are denoted by the same reference numerals throughout the drawings, and redundant description will be omitted.

[0024] FIG. 1 is a block diagram showing an example of the functional configuration of an information processing device 10 that extracts attribute values ​​for predetermined attributes from an image 2 generated by optically reading a document formed on a recording medium such as paper.

[0025] There are no restrictions on the type of imaged document from which attribute values ​​can be extracted by information processing device 10, and any type of document containing text may be used, such as an invoice, estimate, form, contract, or blueprint. In other words, images 2 from which attribute values ​​can be extracted by information processing device 10 include images 2 of various types of documents containing some information written in text. Naturally, documents may also contain expressions other than text, such as figures and photographs.

[0026] An attribute is an item used to identify the information you want to obtain from Image 2. An attribute value for an attribute is the content of each attribute represented by characters included in Image 2.

[0027] As an example, considering quotation image 2, the user wants to obtain information from quotation image 2 such as what kind of product the quotation is for, how many products are included in the quotation, and how much the quotation costs, so for example, "product name," "quantity," and "quote amount" would be the attributes. Also, if quotation image 2 contains information representing the contents of each attribute of "product name," "quantity," and "quote amount," such as "ballpoint pen," "2 pens," and "200 yen," then "ballpoint pen," "2 pens," and "200 yen" would be the attribute values ​​for "product name," "quantity," and "quote amount," respectively.

[0028] An information processing device 10 that extracts attribute values ​​for predetermined attributes from an image 2 includes functional units, such as an image receiving unit 11, a user interface (UI) unit 12, an image processing unit 13, a control unit 14, and an output unit 15, memory areas for an OCR (Optical Character Recognition) result DB (Database) 16 and an extraction result DB 17, and extraction rules 18 and association rules 19.

[0029] The image receiving unit 11 receives an image 2 as the target for extracting attribute values ​​from an optical device such as a scanner that optically reads the contents of a document and generates an image 2 of the document, and passes the received image 2 to the image processing unit 13.

[0030] The UI unit 12 accepts instructions from a worker (hereinafter referred to as "user") who is attempting to extract attribute values ​​of attributes from image 2 using the information processing device 10, such as an instruction to start accepting image 2 by the image accepting unit 11, and notifies the user of various information such as the operation and status of the information processing device 10.

[0031] The image processing unit 13 extracts character information from the image 2 accepted by the image accepting unit 11, and also extracts attribute values ​​of a plurality of predetermined attributes from the extracted character information. For this purpose, the image processing unit 13 includes an OCR processing unit 13A and an extraction unit 13B.

[0032] The OCR processing unit 13A performs known image recognition on the received image 2 and converts portions of the image 2 that correspond to characters into character codes. That is, the OCR processing unit 13A treats the portions of the image 2 that correspond to characters as character information, allowing copying and character search. Hereinafter, the character information obtained from the image 2 by the OCR processing unit 13A will be referred to as the "OCR result." The OCR processing unit 13A stores the OCR result in the OCR result DB 16.

[0033] The extraction unit 13B extracts an attribute value for at least one predetermined attribute from the OCR results stored in the OCR result DB 16 in accordance with a predetermined extraction rule 18, and performs processing to associate the extracted attribute value with the attribute.

[0034] As an extraction method for extracting attribute values ​​for attributes from the OCR results, known extraction methods such as a key-value extraction method and a character pattern extraction method are used.

[0035] Figure 2 shows an example of a known extraction method for extracting attribute values ​​for attributes from OCR results, where Figure 2(A) shows an example of extraction using the key-value extraction method, and Figure 2(B) shows an example of extraction using the character pattern extraction method.

[0036] In the key-value extraction method, a keyword (corresponding to a "key") that is predefined for each attribute and indicates what character string the attribute is represented by in image 2, and information on the relative position based on the keyword, are used to extract an attribute value (corresponding to a "value") for the attribute from image 2. For example, if the attribute is the total amount including tax, extraction unit 13B extracts the attribute value for the attribute using extraction rule 18 that prescribes that the total amount including tax in image 2 is expressed as "total amount (tax included)" and that the total amount including tax is written to the right of "total amount (tax included)."

[0037] As shown in Figure 2(A), if there is a place in image 2 where "Total amount (tax included)" is written (the place represented by box 3A), the extraction unit 13B follows extraction rule 18 and extracts the character string written to the right of "Total amount (tax included)" (the character string "￥10,000" represented by box 3B) as an attribute value for the attribute "Total amount (tax included)" and associates it with the attribute.

[0038] In the character pattern extraction method, attribute values ​​for attributes are extracted from image 2 using extraction rules 18 that predefine the characteristics of character strings that represent attribute values ​​in image 2 for each attribute. Character string characteristics used to display attribute values ​​include, for example, whether a character string representing a company name begins or ends with "(Kaisha)" or "Kabushiki Kaisha," or whether a character string representing a monetary amount contains numbers. Note that regular expressions, for example, are used to express character string patterns that define the characteristics of such character strings.

[0039] For example, if extraction rule 18 specifies a string pattern such as "Total ¥((tax included|tax included|consumption tax included|consumption tax included)¥).*(¥d{1,3}(,¥d{3})*)(yen|-|¥.-|)", extraction unit 13B extracts the string "Total (tax included): ¥10,000" represented in box 3 from image 2 shown in Figure 2 (B), and associates the extracted string with an attribute value of an attribute such as "total amount including tax."

[0040] When associating an attribute value with an attribute, the extraction unit 13B may associate not only a character string representing the attribute value but also the coordinate values ​​in image 2 of the character string extracted as the attribute value with the attribute. The coordinate values ​​of a character string in image 2 are coordinate values ​​in a two-dimensional coordinate system with an origin P at any position in image 2. For example, the coordinate values ​​are represented by the coordinate values ​​of the upper left vertices of the rectangular frames 3 and 3B surrounding the character string representing the attribute value, such as point Q shown in FIGS. 2(A) and 2(B). The coordinate values ​​of the attribute value are not limited to the upper left vertices of the frames 3 and 3B. As long as the position of the character string representing the attribute value in image 2 can be identified, any point in the frames 3 and 3B may be used as the coordinate value of the attribute value. Furthermore, instead of a position in the frames 3 and 3B, for example, any point in the character string representing the attribute value, such as the center coordinates of the first character, may be used as the coordinate value of the attribute value. A "character string" according to the technology disclosed herein refers to a sequence of one or more characters.

[0041] The extraction unit 13B stores the extraction result in which the extracted attribute value is associated with each attribute, that is, the attribute value extraction result 30, in the extraction result DB 17. The attribute value extraction result 30 will be described in detail later with reference to FIG.

[0042] When the image processing unit 13 extracts attribute values ​​for each predetermined attribute from the image 2, the control unit 14 controls the association of the attribute value of the secondary attribute with other attributes. Here, the "secondary attribute" refers to an attribute whose attribute value is extracted from the image 2 and which supplements the content of the other attributes.

[0043] 3 is a diagram showing an example of an attribute value for the "estimated amount" attribute extracted from image 2 by extraction unit 13B using the character pattern extraction method. In the example of image 2 shown in FIG. 3, the character string "excluding consumption tax" is displayed in a line different from the line showing the total amount. Therefore, for example, extraction rule 18, which extracts attribute values ​​only from the same line showing the total amount, cannot extract the character string "excluding consumption tax" as an attribute value of "estimated amount."

[0044] Thus, for example, if extraction unit 13B extracts only the value "10,000 yen" from image 2 as the attribute value for the "estimated amount" attribute, it is unclear whether the attribute value for "estimated amount" includes or excludes tax. On the other hand, if an attribute value "excluding consumption tax" is associated with an attribute called "consumption tax information" by a different extraction rule 18 from the extraction rule 18 that extracts the attribute value for "estimated amount," then by associating the attribute value of "consumption tax information" with the attribute of "estimated amount," it is possible to determine that the attribute value of "estimated amount," "10,000 yen," is an amount excluding tax. In this case, "consumption tax information" is an attribute that supplements the content of the estimated amount, and is therefore a secondary attribute of the attribute called "estimated amount."

[0045] The attribute value of the secondary attribute is associated with other attributes in accordance with a predefined association rule 19. The association rule 19 will be described in detail later.

[0046] In this way, the control unit 14 controls the association of the attribute value of the secondary attribute with other attributes included in the attribute value extraction result 30 in accordance with the association rule 19 that associates the attribute value of the secondary attribute with other attributes. The control unit 14 reflects the association result obtained by associating the secondary attribute with the attribute in the attribute value extraction result 30 stored in the extraction result DB 17. Hereinafter, the attribute value extraction result 30 reflecting the association result of the secondary attribute with the attribute will be referred to as the "attribute association result 32." The attribute association result 32 will be described in detail later with reference to FIG. 9.

[0047] The output unit 15 outputs the attribute association result 32 stored in the extraction result DB 17 in accordance with an instruction from the control unit 14. Outputting the attribute association result 32 means making the attribute association result 32 available for a user to check. Therefore, the following are all examples of outputting the attribute association result 32: transmitting the attribute association result 32 to an external device via a communication line, displaying the attribute association result 32 on a display, printing the attribute association result 32 on a recording medium such as paper by an image forming device, and storing the attribute association result 32 in a storage device to which the user has permission to read.

[0048] 1 is configured using, for example, a computer 20. FIG. 4 is a diagram showing an example of the configuration of the main parts of an electrical system in the information processing device 10 configured using the computer 20.

[0049] 1, a ROM (Read Only Memory) 22 that stores an information processing program executed by the CPU 21, a RAM (Random Access Memory) 23 that is used as a temporary work area for the CPU 21, a nonvolatile memory 24, and an input / output interface (I / O) 25. The CPU 21, the ROM 22, the RAM 23, the nonvolatile memory 24, and the I / O 25 are connected to each other via a bus 26.

[0050] The nonvolatile memory 24 is an example of a storage device that maintains stored information even if the power supplied to the nonvolatile memory 24 is cut off, and is, for example, a semiconductor memory or a hard disk. The nonvolatile memory 24 does not necessarily have to be built into the computer 20, and may be a storage device that is detachable from the computer 20, such as a memory card. The OCR result DB 16 and the extraction result DB 17 are constructed in the nonvolatile memory 24, for example.

[0051] To the I / O 25, for example, a communication unit 27, an input unit 28, and a display unit 29 are connected.

[0052] The communication unit 27 is connected to a communication line and is provided with a communication protocol for communicating with external devices such as storage devices and computers connected to the same connection line.

[0053] The input unit 28 is a device that receives instructions from a user and notifies the CPU 21, and may be, for example, a button, a touch panel, a keyboard, a mouse, etc. The information processing device 10 executes a function instructed by the user via the input unit 28.

[0054] The display unit 29 is a device that displays information processed by the CPU 21 as an image, and may be, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, or a projector that projects an image onto a screen.

[0055] The input unit 28 and the display unit 29 cooperate with the UI unit 12 shown in FIG. 1 to accept various instructions from the user and notify the user of various information relating to the operation and state of the information processing device 10.

[0056] Note that the units connected to the I / O 25 are not limited to the units illustrated in Fig. 4. For example, a scanner unit that optically reads the contents of a document placed on the platen glass and converts the document contents into an image may be connected to the I / O 25. In this case, the CPU 21 receives the document image 2 from the scanner unit via the I / O 25.

[0057] If the scanner unit is not connected to the I / O 25, the information processing device 10 may receive the image 2 from an external device, for example, via the communication unit 27. The information processing device 10 may also receive the image 2 from a storage device that is detachably attached to the computer 20, such as a memory card.

[0058] Next, the operation of the information processing device 10 for associating attribute values ​​of related attributes, such as an attribute related to a price and an attribute related to consumption tax, even if the attribute values ​​of the related attributes are extracted separately from the image 2 will be described.

[0059] Here, the association process for associating attribute values ​​of sub-attributes with associated attributes will be described using an example of associating attribute values ​​of each associated attribute from an estimate image 2A as shown in FIG.

[0060] Fig. 6 is a flowchart showing an example of the flow of the association process executed by the CPU 21 when the estimate image 2A shown in Fig. 5 is received from an external device via the communication unit 27. An information processing program that defines the association process is stored in advance in, for example, the ROM 22 of the information processing device 10. The CPU 21 of the information processing device 10 reads the information processing program stored in the ROM 22 and executes the association process.

[0061] First, in step S10, the CPU 21 performs known image recognition on the image 2A, generates OCR results by converting the parts of the image 2A corresponding to characters into character codes, and stores the results in the OCR result DB 16 constructed in the non-volatile memory 24.

[0062] In step S20, the CPU 21 selects one extraction rule 18 from at least one extraction rule 18 pre-stored in the non-volatile memory 24. For ease of explanation, the extraction rule 18 selected in step S20 will be referred to as the "selected extraction rule 18."

[0063] In step S30, the CPU 21 extracts attribute values ​​of attributes specified in the selective extraction rules 18 from the OCR results of the image 2A stored in the OCR result DB 16 in accordance with the selective extraction rules 18.

[0064] In step S40, the CPU 21 stores the attribute values ​​extracted from the OCR results of the image 2A in accordance with the selective extraction rule 18 in step S30 in the extraction result DB 17 constructed in the non-volatile memory 24 in association with the attributes from which the attribute values ​​were extracted.

[0065] In step S50, the CPU 21 determines whether or not there is an unselected extraction rule 18 that has not yet been selected in step S20 among the extraction rules 18 pre-stored in the non-volatile memory 24. If there is an unselected extraction rule 18, the process proceeds to step S20, and one of the unselected extraction rules 18 is selected as a new selected extraction rule 18.

[0066] By repeatedly executing steps S20 to S50 until it is determined in the determination process of step S50 that there are no unselected extraction rules 18, the attribute values ​​extracted from image 2A are associated with the attributes for which attribute values ​​are to be extracted in each extraction rule 18, and the attribute value extraction results 30 are stored in extraction result DB 17.

[0067] If it is determined in the determination process of step S50 that there is no unselected extraction rule 18, the process proceeds to step S60.

[0068] Fig. 7 is a diagram showing an example of the extraction result 30 of the attribute values ​​stored in the extraction result DB 17. The extraction result 30 of the attribute values ​​shown in Fig. 7 includes, for example, the attributes of "amount," "total," "tax included," "tax excluded," "consumption tax rate," "multiply rate," "before discount," and "consumption tax information," corresponding to the contents of the estimate image 2A shown in Fig. 5.

[0069] In image 2A of the quotation shown in Figure 5, the total amount column contains "¥400,000," the markup column contains "80%," the unit price column contains "500,000," the remarks column contains "1)" which reads "The estimated amount does not include consumption tax," and the remarks column contains "2)" which reads "The consumption tax rate is 10%." Therefore, according to extraction rule 18, the attribute values ​​of "¥400,000," "10%, "80%, "500,000," and "Consumption tax not included" are associated with the attributes of "Total," "Consumption tax rate," "Markup," "Before discount," and "Consumption tax information," respectively.

[0070] As already explained, each attribute may be associated with a coordinate value of the attribute value as well as the attribute value. In the attribute value extraction result 30 shown in Fig. 7, the attribute value and the coordinate value of the attribute value are associated with each attribute.

[0071] Note that if there is an attribute included in the attribute value extraction result 30 for which no attribute value could be extracted using the predefined extraction rules 18, the attribute value of that attribute will be left blank. Therefore, in the attribute value extraction result 30 shown in Fig. 7, no attribute values ​​for the attributes "tax included" and "tax excluded" could be extracted using any of the predefined extraction rules 18, and so the attribute values ​​for the attributes "tax included" and "tax excluded" are left blank.

[0072] This occurs when the character string indicating whether the total amount in quotation image 2A shown in Figure 5 is tax-exclusive or tax-inclusive is not in the position specified by extraction rule 18, is expressed by a character string different from the character string pattern specified by extraction rule 18, or when quotation image 2A does not contain information indicating whether the amount is tax-inclusive or tax-exclusive in the first place.

[0073] 7, the attribute value extraction result 30 is represented by a data structure that manages related attributes hierarchically. For example, the attributes "total," "consumption tax rate," "tax rate," "price before discount," and "consumption tax information" are associated as sub-attributes under "amount," which is located at the top of the data structure, i.e., the root. Furthermore, two attributes, "tax included" and "tax excluded," are associated as sub-attributes under "total."

[0074] The CPU 21 can obtain detailed information about higher-level attributes by following the lines, or "links," that connect related attributes in the attribute value extraction result 30. For example, in the attribute value extraction result 30 in Figure 7, the attribute value "400,000 yen" obtained by following the link from "Amount" to "Total" represents the total amount of the estimate. Also, the attribute value "10%" obtained by following the link from "Amount" to "Consumption Tax Rate" represents the consumption tax rate applied to the amount.

[0075] However, since the attribute values ​​for the "tax included" and "tax excluded" attributes in the attribute value extraction result 30 shown in Figure 7 are blank, it is unclear whether the total amount "400,000 yen" obtained by following the link from "Amount" to "Total" is the total amount including tax or the total amount excluding tax.

[0076] 6, the CPU 21 selects one association rule 19 from at least one association rule 19 pre-stored in the non-volatile memory 24. For ease of explanation, the association rule 19 selected in step S60 will be referred to as the "selected association rule 19."

[0077] 8 is a diagram showing an example of the association rule 19. The association rule 19 includes a condition for determining whether or not to associate attributes with each other, and association information that specifies which attributes should be associated with each other and how the attributes should be associated with each other when the condition is met.

[0078] In association rule 19 in Figure 8, the part that indicates the condition is "If the coordinates of [amount-consumption tax information] are included in the bottom third of the image, and the attribute values ​​of [amount-consumption tax information] include 'consumption tax' and 'not included'," and the part that indicates the association information is "The attribute value of [amount-total] is set to the attribute value of [amount-total-excluding tax]."

[0079] The "Amount-Consumption Tax Information" attribute is a secondary attribute of the "Amount-Total" attribute, as it supplements the content of the attribute, such as whether the amount associated with the "Amount-Total" attribute is tax-exclusive or tax-inclusive.

[0080] By specifying in the conditions of association rule 19 the position of the secondary attribute (in this case, "amount-consumption tax information") in image 2A of the quotation, that is, the position condition that specifies the position of the attribute value of the secondary attribute, it may be possible to identify the secondary attribute corresponding to the attribute whose content is to be supplemented from image 2A, even if the attribute value of the attribute (in this case, [amount-total]) whose content is supplemented by the secondary attribute and the attribute value of the secondary attribute are in a positional relationship that was not anticipated by extraction rule 18, or are written in a distant location that was not anticipated by extraction rule 18.

[0081] In other words, the association rule 19 can associate the attribute value of a secondary attribute that is located in a position that would not be extracted from the estimate image 2A by the extraction rule 18 with an attribute whose content is supplemented by the secondary attribute. The attribute whose content is supplemented by the secondary attribute is an example of the "other attribute" according to the technology of the present disclosure.

[0082] The reason why the condition "The coordinates of [Amount-Consumption Tax Information] are included in the bottom 1 / 3 of the image" is set in association rule 19 shown in Figure 8 is because information about consumption tax is often written in the remarks section, and the remarks section is often written at the end of an estimate.Whether the coordinates of [Amount-Consumption Tax Information] are included in the bottom 1 / 3 of the image can be determined from the coordinate values ​​associated with the attribute values ​​of [Amount-Consumption Tax Information].

[0083] Similarly, by specifying in the conditions of association rule 19 what character strings are included as attribute values ​​of the secondary attribute, i.e., by specifying character string conditions that define the content of the attribute value of the secondary attribute, it may be possible to identify from image 2A the secondary attribute that corresponds to the attribute whose content is to be supplemented, even if the attribute value of the secondary attribute is expressed in an expression that was not anticipated by extraction rule 18.

[0084] It is not necessary to specify both a position condition and a character string condition as the conditions of the association rule 19; it is sufficient to specify at least one of a position condition and a character string condition. Also, a condition different from the position condition and the character string condition may be specified as the condition of the association rule 19. If the character string representing the attribute value of the secondary attribute has visual characteristics related to the characters, such as size, font, and color, that is, character characteristics, it may be possible to identify the secondary attribute corresponding to the attribute whose content is to be supplemented from the image 2A by specifying a character condition that specifies the character characteristics as the condition of the association rule 19.

[0085] The conditions of such association rule 19 are specified by the user, but the CPU 21 may also modify the conditions of association rule 19. For example, the CPU 21 may identify the position of the notes column in image 2A from the position of a character string including "notes," and modify the conditions of association rule 19 so that the coordinate range of [amount-consumption tax information] specified in the conditions of association rule 19 includes coordinates corresponding to the identified position of the notes column. In this case, the attribute value of the attribute specified in the conditions of association rule 19 (e.g., [amount-consumption tax information]) is written in image 2A, but even if the attribute value of the specified attribute is written in a location other than the position specified in the conditions (e.g., "bottom 1 / 3 of the image"), the condition of association rule 19 will be satisfied.

[0086] The association information of association rule 19 is information that specifies how to associate the attribute to be supplemented with the content and the secondary attribute when the conditions of association rule 19 are met and a secondary attribute corresponding to the attribute to be supplemented with the content is identified from image 2A.

[0087] In the example of association information in association rule 19 shown in Figure 8, the attribute value of [Amount-Consumption Tax Information], which is identified as a secondary attribute, is associated with the attribute of "Amount-Total", and by specifying an instruction to set the attribute value of [Amount-Total] to the attribute value of [Amount-Total-Excluding Tax], it is intended to make it clear that the amount associated with the attribute of [Amount-Total] is exclusive of tax.

[0088] 8 shows an example in which the association rule 19 is defined in a natural language such as Japanese or English, but the method for defining the association rule 19 is not limited to this. For example, the association rule 19 may be defined using a known programming language or an artificial language such as a regular expression. Using an artificial language rather than a natural language can reduce ambiguity in the defined content, so the association rule 19 can be expressed more clearly. Naturally, the association rule 19 may be defined using a unique artificial language that is used exclusively for the information processing device 10.

[0089] 6, the CPU 21 determines whether or not the condition of the selection association rule 19 is met. If the condition of the selection association rule 19 is met, the process proceeds to step S80.

[0090] In step S80, since the condition of the selection association rule 19 is met, the CPU 21 associates the attribute value of the secondary attribute with another attribute in accordance with the processing defined in the association information of the selection association rule 19, and proceeds to step S90.

[0091] 8 is selected as the selected association rule 19, the CPU 21 sets the attribute value of [Amount-Total] in the attribute value extraction result 30 as the attribute value of [Amount-Total-Excluding Tax]. Therefore, if the attribute value extraction result 30 before execution of step S80 is the attribute value extraction result 30 shown in Fig. 7, the attribute value of [Amount-Total], "¥400,000", is set as the attribute value of [Amount-Total-Excluding Tax] as shown in Fig. 9.

[0092] Therefore, even if it is unclear from the attribute value of the total amount of the estimate extracted from the OCR results of image 2A by extraction rule 18 alone whether this total amount is inclusive of tax or exclusive of tax, association rule 19 associates the total amount with the total amount exclusive of tax, so CPU 21 can recognize that the total amount is exclusive of tax by following the link of attribute value extraction result 30 as [Amount-Total-Excluding Tax].

[0093] When the coordinate values ​​of the attribute values ​​are associated with the attributes, the CPU 21 may associate the coordinate values ​​of the attribute values ​​with other attributes together with the attribute values, even if this is not explicitly stated in the association information of the selected association rule 19.

[0094] On the other hand, if it is determined in the determination process of step S70 that the condition of the selection association rule 19 is not satisfied, the CPU 21 proceeds to step S90 without executing the process of step S80. In other words, if the condition of the selection association rule 19 is not satisfied, the CPU 21 does not associate attributes with each other.

[0095] In step S90, the CPU 21 determines whether or not there is an unselected association rule 19 that has not yet been selected in step S60 among the association rules 19 pre-stored in the non-volatile memory 24. If there is an unselected association rule 19, the process proceeds to step S60, and one of the unselected association rules 19 is selected as a new selected association rule 19.

[0096] By repeatedly executing steps S60 to S90 until it is determined in the determination process of step S90 that there are no unselected association rules 19, attributes are associated with each other in accordance with the association information included in each association rule 19 for which the condition is met. As a result, attribute association results 32 using the association rules 19 are stored in the extraction result DB 17.

[0097] If it is determined in the determination process of step S90 that there is no unselected association rule 19, the association process shown in FIG. 6 ends.

[0098] As described above, even if the predefined extraction rule 18 does not allow the attribute value of the secondary attribute to be associated with the attribute value of the attribute to be supplemented and acquired together, the information processing device 10 can use the association rule 19 to associate the attribute value of the secondary attribute with the attribute to be supplemented later.

[0099] After the association process is completed, if the CPU 21 receives an instruction to output the attribute association result 32 from the user via the input unit 28, the CPU 21 outputs the attribute association result 32 in the format specified by the output instruction.

[0100] 6 has been described as an example in which the CPU 21 selects all of the association rules 19 pre-stored in the non-volatile memory 24. However, when attribute values ​​are associated with all of the attributes included in the attribute value extraction result 30 and the association between the attributes is completed, the CPU 21 may end the association process shown in FIG. 6 even if all of the association rules 19 have not been selected.

[0101] 6 has been described as an example in which attributes are associated with each other in accordance with the association information of the association rules 19. However, if there is an attribute that is not specified in the association information of the association rules 19 but that has become newly associateable by associating attributes with each other in accordance with the association information, the CPU 21 may associate the attribute value of the sub-attribute with such an attribute.

[0102] For example, as shown in FIG. 9, suppose there is an attribute association result 32 in which the coordinate value of the attribute value "¥400,000" is associated with the attribute "excluding tax." In this case, the CPU 21 recognizes from the attribute association result 32 shown in FIG. 9 that the total amount "¥400,000" is the total amount excluding tax. The CPU 21 also recognizes from the attribute association result 32 shown in FIG. 9 that the consumption tax rate for the total amount is "10%." Therefore, the CPU 21 can calculate the total amount including tax as "¥440,000" from the total amount excluding tax and the consumption tax rate for the total amount. Therefore, as shown in FIG. 10, the CPU 21 can set "¥440,000" to [Amount-Total-Tax Included].

[0103] 9, an attribute that supplements the content of the total amount and to which no attribute value is associated in any of the extraction rules 18 can be used to uniquely determine the total amount including tax by using a secondary attribute for the amount, such as "consumption tax rate." In this way, an attribute whose attribute value is not set in the extraction rules 18 and the association rules 19 but whose attribute value can be uniquely determined by using the attribute value of an attribute that has already been set is called a "related secondary attribute."

[0104] As can be seen from the fact that the attribute value of "tax included" in the attribute association result 32 shown in Fig. 9 is not included in the estimate image 2A, the attribute may be an attribute whose attribute value is not originally included in the document image 2. Naturally, the related sub-attribute may also be an attribute whose attribute value is included in the document image 2 but whose attribute value could not be matched or associated by the extraction rule 18 and the association rule 19.

[0105] The introduction of an invoice (qualified invoice) system as a method for deduction of consumption tax on purchases is underway, and the association process in the information processing device 10 shown in FIG. 6 is also used to associate attributes on invoices.

[0106] An invoice is a bill sent by a seller to a buyer showing the applicable tax rate for each product and the total amount for each tax rate. Figure 11 shows an example of image 2B of an invoice.

[0107] Image 2B of the invoice shown in Figure 11 lists the applicable tax rate category and tax-inclusive price for each product listed in the product name column, as well as the total tax-inclusive price for each product. Image 2B of the invoice shown in Figure 11 also lists the total tax-inclusive price for products subject to the reduced tax rate (8%) and the total tax-inclusive price for products subject to the normal tax rate (10%) separately, and the last line of Image 2B of the invoice includes a note stating that "products marked with an asterisk are subject to the reduced tax rate (8%)."

[0108] Figure 12 shows an example of an extraction result 30 of attribute values ​​extracted from the OCR result of image 2B using predefined extraction rules 18 when invoice image 2B is received by information processing device 10. In the extraction result 30 of attribute values ​​shown in Figure 12, subtotal (1), subtotal (2), ..., subtotal (N) are attributes corresponding to the prices of each product listed in the product name column in invoice image 2B of Figure 11. Note that "N" represents a natural number.

[0109] If extraction rule 18 is specified as a rule for extracting attribute values ​​in the same row direction, then the line listing each product's price does not contain a numerical value representing the price excluding tax or the consumption tax rate. Therefore, in the attribute value extraction result 30 shown in FIG. 12, attribute values ​​are not associated with the attributes "excluding tax" and "consumption tax rate." On the other hand, the line listing each product's price contains a symbol representing the price including tax and the consumption tax rate category. Therefore, in the attribute value extraction result 30 shown in FIG. 12, attribute values ​​are associated with the attributes "including tax" and "category." Note that, for example, in the attribute value extraction result 30 shown in FIG. 12, an attribute value for the attribute [Amount-Subtotal (2)-Category] appears to be unset. This is because the category field for stationery in image 2B of the invoice in FIG. 11 is blank. Therefore, the attribute [Amount-Subtotal (2)-Category] is set to an attribute value consistent with the content of the category field in image 2B of the invoice in FIG. 11.

[0110] Furthermore, the last line of image 2B of the invoice contains a note stating that "* denotes a product subject to the reduced tax rate (8%)," so extraction rule 18 associates "* denotes a product subject to the reduced tax rate (8%)" with the attribute of "consumption tax information."

[0111] As already explained, each attribute may be associated with a coordinate value of the attribute value as well as the attribute value. In the attribute value extraction result 30 shown in Fig. 12, the attribute value and the coordinate value of the attribute value are associated with each attribute.

[0112] On the other hand, suppose that two association rules 19 as shown in FIG. 13 are prepared for the attribute value extraction result 30 as shown in FIG. 12 extracted from the invoice image 2B of FIG. 11 according to the extraction rule 18.

[0113] For convenience of explanation, one association rule 19 is referred to as a "first association rule 19A" and the other association rule 19 is referred to as a "second association rule 19B."

[0114] In the first association rule 19A, the part that indicates the condition is, "If the coordinates of [Amount-Consumption Tax Information] are included in the bottom 1 / 3 of the image, and the attribute values ​​of [Amount-Consumption Tax Information] include "*" and "Reduced Tax Rate", and [Amount-Subtotal (N)-Category] includes "*", then "Set [Amount-Subtotal (N)-Consumption Tax Rate] to "8%". " is the part that indicates the association information.

[0115] In addition, in the second association rule 19B, the condition is stated as follows: "If the coordinates of [Amount-Consumption Tax Information] are included in the bottom 1 / 3 of the image, and the attribute values ​​of [Amount-Consumption Tax Information] include "*" and "Reduced Tax Rate", and [Amount-Subtotal (N)-Category] does not include "*", and the association information is stated as follows: "Set [Amount-Subtotal (N)-Consumption Tax Rate] to '10%'".

[0116] In invoice image 2B shown in Figure 11, the attribute value for the "Amount-Consumption Tax Rate" attribute is entered in the last line of invoice image 2B, and the attribute value for the "Amount-Consumption Tax Rate" attribute contains the character strings "*" and "Reduced Tax Rate." In addition, because the attribute value for [Amount-Subtotal (1)-Category] is "*," the condition of first association rule 19A is met for the [Amount-Subtotal (1)] attribute. Therefore, in accordance with the association information of first association rule 19A, "8%" is set as the attribute value for the [Amount-Subtotal (1)-Consumption Tax Rate] attribute.

[0117] On the other hand, since the attribute value of [Amount-Subtotal (2)-Category] is not "*", the condition of the second association rule 19B is met for the attribute of [Amount-Subtotal (2)]. Therefore, in accordance with the association information of the second association rule 19B, "10%" is set as the attribute value for the attribute of [Amount-Subtotal (2)-Consumption Tax Rate].

[0118] Fig. 14 is a diagram showing an example of an attribute association result 32 obtained when the first association rule 19A and the second association rule 19B shown in Fig. 13 are applied to the attribute value extraction result 30 shown in Fig. 12 and the association process shown in Fig. 6 is executed. As shown in Fig. 14, by applying the association rule 19, the attributes "category" and "consumption tax information" are associated with each other, and as a result, the applicable consumption tax rate is set for each product even for the attribute "consumption tax rate", which could not be associated with an attribute value using only the extraction rule 18.

[0119] As already explained, in invoice image 2B, CPU 21 may also set attribute values ​​of related sub-attributes from attribute association results 32. Fig. 15 is a diagram showing an example of setting an attribute value for the "tax excluded" attribute from attribute association results 32 shown in Fig. 14. CPU 21 can set an attribute value for the "tax excluded" attribute from "tax included price" and "consumption tax rate".

[0120] While one aspect of the information processing device 10 has been described above using the embodiment, the disclosed form of the information processing device 10 is merely an example, and the form of the information processing device 10 is not limited to the scope described in the embodiment. Various modifications or improvements can be made to the embodiment without departing from the gist of the present disclosure, and forms incorporating such modifications or improvements are also included in the technical scope of the disclosure. For example, the order of the association process shown in FIG. 6 may be changed without departing from the gist of the present disclosure.

[0121] In the above embodiment, the association process is implemented by software. However, the same process as the association process shown in Fig. 6 may be implemented by hardware. In this case, the processing speed can be increased compared to when the association process is implemented by software.

[0122] In the above embodiment, the term "processor" refers to a processor in a broad sense, including a general-purpose processor (e.g., CPU 21) and a dedicated processor (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0123] Furthermore, the operations of the processors in the above embodiments may not only be performed by a single processor, but may also be performed by multiple processors located at physically separate locations working together. Furthermore, the order of the operations of the processors is not limited to the order described in the above embodiments, and may be changed as appropriate.

[0124] In the above embodiment, an example in which the information processing program is stored in the ROM 22 has been described, but the storage destination of the information processing program is not limited to the ROM 22. The information processing program of the present disclosure can also be provided in a form recorded on a storage medium readable by the computer 20. For example, the information processing program may be provided in a form recorded on an optical disc such as a CD-ROM (Compact Disk Read Only Memory) or a DVD-ROM (Digital Versatile Disk Read Only Memory). Furthermore, the information processing program may be provided in a form recorded on a portable semiconductor memory such as a USB (Universal Serial Bus) memory or a memory card.

[0125] ROM 22, non-volatile memory 24, CD-ROM, DVD-ROM, USB, and memory cards are examples of non-transitory storage media.

[0126] Furthermore, the information processing device 10 may download an information processing program from an external device connected to the communication unit 27 via a communication line and store the downloaded information processing program in a non-transitory storage medium. In this case, the CPU 21 of the information processing device 10 reads the information processing program downloaded from the external device from the non-transitory storage medium and executes the association process. [Explanation of symbols]

[0127] 2(2A, 2B) Images 3 (3A, 3B) slots 10. Information processing equipment 11 Image Reception Section 12 UI section 13 Image processing section 13A OCR processing section 13B Extraction part 14 Control Unit 15 Output section 16 OCR result DB 17 Extraction result DB 18 (Selection) Extraction Rules 19 (Selection) Association Rules 19A First Association Rule 19B Second Association Rule 20 Computer 21 CPU 22 ROM 23 RAM 24 Non-volatile memory 25 I / O 26 Bus 27 Communication Unit 28 Input Units 29 Display Unit 30 Attribute value extraction results P origin Q point

Claims

1. a processor; the processor extracts attribute values ​​of a plurality of attributes from the OCR result of the image in accordance with a predetermined extraction rule, and generates an extraction result in which the extracted attribute values ​​are associated with each attribute; When controlling the association of attribute values ​​of secondary attributes with other attributes included in the extraction result in accordance with an association rule for associating attribute values ​​of secondary attributes, which are attributes whose attribute values ​​are extracted from the image and which supplement the contents of other attributes, with the other attributes, the control also controls the association of attribute values ​​of related secondary attributes, which are attributes that supplement the contents of the other attributes like the secondary attributes, whose attribute values ​​are not associated with the extraction rule, and whose attribute value is uniquely determined by using the attribute value of the secondary attribute, with the other attributes. Information processing device.

2. the association rule includes a condition for performing the association and association information that specifies how the other attribute and the sub-attribute are associated with each other, When the condition is met, the processor performs control to associate the attribute value of the secondary attribute with the other attribute in accordance with the association information. The information processing device according to claim 1 .

3. The conditions include content that defines at least one of the position of the attribute value of the secondary attribute in the image and the content of the attribute value of the secondary attribute. The information processing device according to claim 2 .

4. The related sub-attribute is an attribute whose value is not included in the image.

4. The information processing device according to claim 1.

5. The extraction results are represented by a data structure that hierarchically manages related attributes.

5. The information processing device according to claim 1.

6. The attribute value of the secondary attribute is an attribute value that is located at a position that is not extracted as an attribute value of the other attribute even when extracted from the image according to the extraction rule.

6. The information processing device according to claim 1.

7. a processor; the processor extracts attribute values ​​of a plurality of attributes from the OCR result of the image in accordance with a predetermined extraction rule, and generates an extraction result in which the extracted attribute values ​​are associated with each attribute; The attribute value of a secondary attribute is an attribute whose attribute value is extracted from the image and which supplements the content of another attribute, and is located at a position where it is not extracted as an attribute value of the other attribute even when extracted from the image according to the extraction rule, and according to an association rule, the attribute value of the secondary attribute is controlled to be associated with the other attribute included in the extraction result. Information processing device.

8. On the computer, extracting attribute values ​​of a plurality of attributes from the OCR result of the image in accordance with a predetermined extraction rule, and generating an extraction result in which the extracted attribute values ​​are associated with each attribute; When controlling the association of attribute values ​​of secondary attributes with other attributes included in the extraction result in accordance with an association rule for associating attribute values ​​of secondary attributes, which are attributes whose attribute values ​​are extracted from the image and which supplement the contents of other attributes, with the other attributes, a control is executed to also associate attribute values ​​of related secondary attributes, which are attributes that supplement the contents of the other attributes like the secondary attributes, whose attribute values ​​are not associated with the extraction rule, and whose attribute value is uniquely determined by using the attribute value of the secondary attribute, with the other attributes. Information processing program.

9. A computer, extracting attribute values ​​of a plurality of attributes from the OCR result of the image in accordance with a predetermined extraction rule, and generating an extraction result in which the extracted attribute values ​​are associated with each attribute; a control for associating the attribute value of a secondary attribute with another attribute included in the extraction result in accordance with an association rule for associating the attribute value of the secondary attribute, which is an attribute whose attribute value is extracted from the image and which supplements the content of another attribute, and which is located at a position where it is not extracted as the attribute value of the other attribute even when extracted from the image in accordance with the extraction rule, with the other attribute; Information processing program.

Images