Information processing apparatus, information processing method, and program

Abstract

To detect an error in application of a label to a unique expression.SOLUTION: An information processing apparatus 100 acquires a character string of a unique expression extracted from input document data, acquires a first label and a second label different from the first label to be applied to the character string, determines compatibility between the first label and the second label, and outputs a result of the determination.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a technology for assigning labels to named entities extracted from document data. [Background technology]

[0002] There is a technology that automatically assigns characters extracted from document data such as document images (hereinafter referred to as "document data") as the file name of an electronic file containing the document data, or the name of a folder in which the electronic file is saved (hereinafter simply referred to as "file name"). Characters that are automatically assigned to a file name include characters indicating the creation date and time of the electronic file, as well as the name of an organization or person that issued the document, a number for identifying the document, and characters indicating the publication date and the like of the document. Named entity extraction technology, which is an application technology of natural language processing, can be used to extract characters from document data. In order to create teacher data corresponding to learning data used for learning a learning model that realizes named entity extraction, the following label assignment (hereinafter also referred to as "annotation") work is required. The annotation work is, for example, the work of associating a character area (hereinafter referred to as "character area") in the document data to be worked on, or characters extracted from the character area (hereinafter referred to as "named entity"), with a label indicating the attribute of the named entity.

[0003] Patent Document 1 discloses a technology that supports annotation work by a worker who performs annotation work (hereinafter referred to as an "annotator"). The technology disclosed in Patent Document 1 is such that, when an operation input by the annotator to associate a character area in document data with a label is received, the label is assigned to a character obtained by character recognition of the character area. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-248669 A Summary of the Invention [Problem to be solved by the invention]

[0005] The meaning of a label corresponding to a named entity may change depending on the document attribute such as the document type. For example, when a label of "document issuer" or "document issuer" is assigned to a named entity extracted from document data whose document attribute is "invoice", the label of "document issuer" is assigned to a named entity corresponding to a purchaser or delivery destination. On the other hand, in document data whose document attribute is "purchase order", the label of "document issuer" is assigned to a named entity corresponding to a purchaser or delivery destination, not to a "document issuer". The label of "document issuer" is assigned to a named entity corresponding to a contractor or purchaser. When annotation is performed manually as in the technology disclosed in Patent Document 1, the annotator determines the presence or absence of a named entity corresponding to a purchaser or delivery destination while considering the document attributes, and assigns a label such as "document issuer" or "document issuer" to the named entity. However, due to misrecognition or misoperation of the annotator, an incorrect label may be assigned to a named entity. Therefore, an object of the present disclosure is to provide a technology for detecting errors in labeling named entities by annotators. [Means for solving the problem]

[0006] An information processing device according to the present disclosure includes: a character string acquisition means for acquiring a character string of a named entity extracted from input document data; a first label acquisition means for acquiring a first label to be assigned to the character string acquired by the character string acquisition means; a second label acquisition means for acquiring a second label different from the first label to be assigned to the character string acquired by the character string acquisition means; a determination means for determining consistency between the first label acquired by the first label acquisition means and the second label acquired by the second label acquisition means; and an output means for outputting a result of the determination by the determination means. Effect of the Invention

[0007] According to the present disclosure, it is possible to detect errors in labeling named entities. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device. [Diagram 2] FIG. 2 is a block diagram showing an example of a functional configuration of an information processing device. [Diagram 3] 11 is a flowchart illustrating an example of a processing flow in an information processing device. [Figure 4] FIG. 4 is a diagram illustrating an example of an input image. [Diagram 5] FIG. 13 illustrates an example of a GUI. [Figure 6] FIG. 13 is a diagram showing an example of a first label list. [Figure 7] FIG. 13 illustrates an example of a GUI. [Figure 8] FIG. 13 illustrates an example of a GUI. [Figure 9] FIG. 13 illustrates an example of a GUI. [Figure 10] FIG. 13 is a diagram illustrating an example of a correspondence map. [Figure 11] FIG. 13 illustrates an example of a GUI. [Figure 12] FIG. 13 illustrates an example of a GUI. [Figure 13] FIG. 13 illustrates an example of a GUI. [Figure 14] FIG. 11 is a diagram for explaining an example of a consistency determination process. [Figure 15] FIG. 13 illustrates an example of a GUI. [Figure 16] FIG. 13 is a diagram showing an example of an input image and a label list. [Figure 17] FIG. 11 is a diagram showing an example of label information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the embodiments do not limit the present disclosure, and all configurations described in the embodiments are not necessarily essential for solving the problems of the present disclosure.

[0010] <First embodiment> [Hardware configuration] 1 is a block diagram showing an example of a hardware configuration of an information processing device 100 according to the first embodiment. The information processing device 100 includes a control unit 101, a ROM 102, a RAM 103, a HDD 104, a display unit 105, an input unit 106, and a scanner 107. The control unit 101 is configured with a CPU (Central Processing Unit) and the like, and executes various processes by reading out various programs stored in the ROM 102 or the HDD 104 and the like. At least a part of the processes performed by the control unit 101 may be executed by dedicated hardware. Examples of the dedicated hardware include an ASIC (application specific integrated circuit), an FPGA (field-programmable gate array), and a DSP (digital signal processor).

[0011] ROM 102 stores programs and data that do not require modification. RAM 103 temporarily stores programs and data supplied from ROM 102 or HDD 104, or from the outside via a network not shown in FIG. 1. RAM 103 is used as a main memory and work area for control unit 101. HDD 104 is configured with a hard disk drive or the like, and stores various data and various programs. Note that functions or processes of information processing device 100 described below are realized by control unit 101 reading out a program stored in ROM 102 or HDD 104 and executing this program.

[0012] The display unit 105 is configured with a liquid crystal display or the like, and displays a GUI (Graphical User Interface) or the like for the user to operate the information processing device 100 or to view the state of processing in the information processing device 100. The input unit 106 is configured with a keyboard, a mouse, or the like, and receives various input operations by the user and inputs various instructions corresponding to the input operations to the control unit 101. The display unit 105 and the input unit 106 may be provided integrally, such as a touch panel. The scanner 107 reads a paper surface and generates a scanned image. The scanner 107 is not limited to a contact type, and may be a non-contact type such as an imaging device such as a document camera or a digital still camera.

[0013] The information processing device 100 recognizes characters included in a scanned image (hereinafter referred to as a "document image") obtained by reading a document such as a form formed on a paper surface with the scanner 107, acquires text data, and stores the text data in a storage device such as the HDD 104. The information processing device 100 may acquire data in a format such as a portable document format (PDF) in which image and text data are integrated via a network (not shown), and store the data in a storage device such as the HDD 104. The control unit 101 also operates as a display control unit that controls the display unit 105, an operation control unit that controls the input unit 106, and a reading control unit that controls the scanner 107. Hereinafter, the display unit 105, the input unit 106, and the scanner 107 will be described as being present inside the information processing device 100, but at least one of them may be present as a separate device outside the information processing device 100.

[0014] [System Configuration] 2 is a block diagram showing an example of a functional configuration of the information processing device 100 according to the first embodiment. The information processing device 100 has an image acquisition unit 201, a text acquisition unit 202, a type acquisition unit 203, a first label acquisition unit 204, an identification unit 205, a second label acquisition unit 208, a determination unit 209, an output unit 210, and a display control unit 211. Each unit included in the information processing device 100 as a functional configuration is realized by the control unit 101 executing a program stored in the ROM 102, the HDD 104, or the like, using the RAM 103 as a work area.

[0015] The image acquisition unit 201 reads out data of a document image stored in a storage device such as the HDD 104 from the storage device and acquires it as data of an input image. The image acquisition unit 201 may directly acquire data of an image obtained by scanning with the scanner 107 from the scanner 107 and use the data as data of the input image. The text acquisition unit 202 acquires data (e.g., character code strings) of a text area (hereinafter referred to as an "input text area") including text in the input image and text (hereinafter referred to as an "input text") included in the input text area. A method for detecting an input text area and a method for extracting input text included in an input text area are well known, and therefore a description thereof will be omitted. The text acquisition unit 202 stores the acquired data of the input text in the HDD 104. For example, the data of the input text is stored in association with the data of the input image. The text acquisition unit 202 may store input text area information indicating the position of the input text area in the input image, etc., in the HDD 104 in addition to the data of the input text. The input text area information is stored in association with the data of the input text, for example.

[0016] The type acquisition unit 203 acquires information indicating the type of document (hereinafter referred to as "document type") corresponding to the input image. For example, the type acquisition unit 203 first identifies an input text area having a predetermined size or more from one or more input text areas acquired by the text acquisition unit 202. Next, the type acquisition unit 203 identifies an input text corresponding to the predetermined text from the input texts included in each of the identified one or more input text areas, thereby identifying the document type corresponding to the input image and acquiring information on the document type. Specifically, for example, if there is an input text corresponding to "invoice" among the input texts included in each of the one or more input text areas having a predetermined size or more, information indicating "invoice" is acquired as information on the document type corresponding to the input image. In the following description, the type acquisition unit 203 is assumed to acquire information on the document type of either "purchase order" or "invoice". The type acquisition unit 203 stores the acquired information on the document type in the HDD 104. The information on the document type is stored, for example, in association with the data of the input image.

[0017] The first label acquiring unit 204 acquires information on labels (hereinafter referred to as "first labels") corresponding to each of the data of one or more input texts acquired by the text acquiring unit 202, based on the operation of the input unit 106 by the annotator. Specifically, the annotator associates the input text with the first label by operating the input unit 106 while referring to a GUI for assigning the first label to the input text, which is displayed on the display unit 105. The first label acquiring unit 204 acquires information on the first label associated with the input text by the annotator, and stores this in the HDD 104 in association with the data of the input text. Details of the first label will be described later with reference to FIG. 6.

[0018] The identification unit 205 identifies candidates for a label (hereinafter referred to as a "second label") that corresponds to the data of the input text and is different from the first label, based on a predetermined correspondence map and the information on the first label acquired by the first label acquisition unit 204. The correspondence map defines the correspondence relationship between the first label and the second label, and is, for example, data in a table format such as a lookup table. Details of the correspondence map will be described later with reference to FIG. 10. The second label candidates (hereinafter referred to as "second label candidates") identified by the identification unit 205 are displayed on the display unit 105.

[0019] The second label acquisition unit 208 acquires information on a second label corresponding to data of the input text based on an operation of the input unit 106 by the annotator. Specifically, the annotator associates the input text with the second label by operating the input unit 106 while referring to a GUI for assigning a second label to the input text, which is displayed on the display unit 105. The second label acquisition unit 208 acquires information on the second label associated with the input text by the annotator, and stores this in the HDD 104 in association with the data of the input text. The annotator assigns a second label by an operation of correcting or confirming a second label candidate.

[0020] The determination unit 209 determines the consistency of the first label and the second label assigned to the input text, that is, the consistency of the information of the first label acquired by the first label acquisition unit 204 and the information of the second label acquired by the second label acquisition unit 208, based on the correspondence map. The determination result by the determination unit 209 is displayed on the display unit 105. The annotator can recognize that an error has occurred in the assignment of the label to the named entity by referring to the determination result displayed on the display unit 105. The annotator may also correct the first label or the second label based on the determination result displayed on the display unit 105. By correcting the label in this way, the annotator can assign an appropriate label to the input text. The output unit 210 outputs, for each input image, information on the first label and the second label assigned to the data of the input text included in the input image, and stores this in the HDD 104 in association with the data of the input text. The information on the first label and the second label output by the output unit 210 is used, for example, for learning a learning model that realizes named entity extraction. The learning method of the learning model is well known and therefore will not be described here.

[0021] [Processing flow] Next, the operation of the information processing device 100 will be described with reference to FIG. 3. FIG. 3 is a flowchart showing an example of a processing flow in the information processing device 100 according to the first embodiment. In the following description, the symbol "S" means a step. First, in S301, the image acquisition unit 201 acquires data of an input image to be processed. One piece of data of the input image is selected from a data set of the input image (not shown). Next, in S302, the text acquisition unit 202 extracts an input text area and an input text from the input image acquired in S301, and acquires data of the extracted input text area and input text. A specific example of the input image will be described later with reference to FIG. 4. Next, in S303, the type acquisition unit 203 acquires information on the document type of the input image acquired in S301. Next, in S304, the first label acquisition unit 204 acquires information on the first label corresponding to the input text acquired in S302 based on the operation of the input unit 106 by the annotator. A specific example of the GUI operated by the annotator will be described later with reference to FIGS. 5 and 7 to 9.

[0022] Next, in S305, the identification unit 205 identifies a second label candidate corresponding to the data of the input text based on the correspondence map and the information of the first label acquired in S304. The second label candidate identified in S305 is displayed on the display unit 105 by the display control unit 211. Next, in S306, the second label acquisition unit 208 acquires information of the second label corresponding to the input text based on the operation of the input unit 106 by the annotator. The second label candidates corresponding to each input text are displayed on the display unit 105, and the annotator inputs the second label by correcting or confirming the second label candidate. Next, in S307, the determination unit 209 determines the consistency of the first label and the second label assigned to the input image. Next, in S308, the display control unit 211 displays the determination result in S307 on the display unit 105.

[0023] The annotator can recognize an error in the first label or the second label input by the operation by checking the determination result displayed on the display unit 105. If there is an error in the first label or the second label, the annotator may, for example, re-input the first label or the second label. If the first label is re-input, the information processing device 100 returns to the process of S304, and if the second label is re-input, the information processing device 100 returns to the process of S306.

[0024] After S308, in S309, the output unit 210 determines whether or not an operation to end input has been performed by the annotator. If it is determined in S309 that an operation to end input has not been performed, the information processing device 100 returns to S304 and repeatedly executes the processes from S304 to S309 until it is determined in S309 that an operation to end input has been performed. If it is determined in S309 that an operation to end input has been performed, in S310, the output unit 210 outputs information on the first label and the second label, and stores the information in the HDD 104 in association with the data of the input text to which each label has been assigned. Note that, when outputting the information on the first label and the second label, the output unit 210 may also output information indicating the determination result of the determination process in S307, and store the information in the HDD 104. After S310, the information processing device 100 ends the process of the flowchart shown in FIG. 3.

[0025] [Input image] Fig. 4 is a diagram showing an example of input images 401, 402 acquired by the information processing device 100 according to the first embodiment. In the following description, the input images 401 and 402 are treated as one set of input images, and the information processing device 100 applies the processing flow shown in Fig. 3 to at least one of the input images 401 and 402. The input image 401 is an image corresponding to a document whose document type is "purchase order", and the input image 402 is an image corresponding to a document whose document type is "invoice".

[0026] [GUI] FIG. 5 is a diagram showing an example of a GUI 501 displayed and output by the information processing device 100 according to the first embodiment. The GUI 501 is a display screen displayed on the display unit 105 at the end of the process of S303. The GUI 510 includes an edit pane 511, a thumbnail pane 512, and a search pane 513. The thumbnail pane 512 is an area for selecting an input image to be an annotation target. The thumbnail pane 512 displays thumbnails of each input image (e.g., input images 401 and 402) included in the set of input images, the document type of each input image, and the name (file name) of a file including data of each input image. The annotator selects a thumbnail to be an annotation target from among the thumbnails displayed in the thumbnail pane 512. FIG. 5 shows, as an example, a state in which a thumbnail with a file name of 1_0001.jpg is selected, and a rectangular frame 507 indicating that the thumbnail is selected is superimposed on the selected thumbnail.

[0027] The editing pane 511 is an area for assigning a first label and a second label to an input image selected in the thumbnail pane 512. An input image area 501 of the editing pane 511 displays an input image selected as a target for annotation work, and a file name area 502 displays a file name and a document type corresponding to the input image. The annotator operates a switch button 503 for switching an input mode by operating the input unit 106 to switch between inputting a first label and inputting a second label. Information on the assigned first label and second label, and information on the input text to which the first label or the second label has been assigned, are displayed in the label display area 504 as label information. At the end of the process of S303, the first label and the second label have not been assigned to the input image, so that the input text and the first label and the second label are not displayed in the label display area 504.

[0028] The judge button 505 is a button for instructing the judge unit 209 to execute a process of judging the consistency of the first label and the second label assigned to the input image. When the annotator presses the judge button 505, the process of S307 shown in FIG. 3 is executed. The save button 506 is a button for instructing the output unit 210 to output the label assignment result of the selected input image. When the annotator presses the save button 506, the processes of S309 and S310 shown in FIG. 3 are executed, and the output unit 210 outputs information on the first label and the second label to the HDD 104. When the save button 506 is pressed, the output unit 210 may also output the consistency judgment result and store it in the HDD 104. By storing the consistency judgment result in the HDD 104, the annotator can skip the consistency judgment process when later correcting the label assigned to the input image, and can perform a filter process using a consistency filter described later.

[0029] The search pane 513 is an area for searching for a set of input images to be displayed as thumbnails in the thumbnail pane 512 from among a plurality of sets of input images. A search function based on text input includes a search by file name, a search by text included in the input image, and the like. A search function based on the processing status of a file includes a search that narrows down the search results based on whether or not the input image has been labeled, or a search that narrows down the search results based on whether or not the consistency of the label attached to the input image indicates the presence of an inappropriate label, and the like. Another search function includes a search that narrows down the search results based on whether or not the input image has been attached with a specified first label. In addition to the above-mentioned search function, the search pane 513 may provide an input for a search that narrows down the search results based on whether or not the input image has been attached with a specified second label, although this is not shown in FIG. 5. For example, an input for a search that narrows down the search results based on a filter, such as an ID representing a template of a document image, may also be provided.

[0030] [First label] FIG. 6 is a diagram showing an example of a first label list 601 which is a list of first labels that can be acquired by the first label acquisition unit 204 according to the first embodiment. The first label list 601 includes, as an example, 11 types of first labels. "DATEorder" is a first label that is assigned to an input text indicating a date that represents an order date. Specifically, "DATEorder" is a first label that is assigned to an input text that is clearly indicated as being the date by nearby text in the input image, or that can be determined to be the date. "DATEinvoice" is a first label that is assigned to an input text indicating a date that represents an invoice date. Specifically, "DATEinvoice" is a first label that is assigned to an input text that is clearly indicated as being the date by nearby text in the input image, or that can be determined to be the date. "IDquote" is a first label that is assigned to an input text indicating a number that represents an estimate number. Specifically, "IDquote" is a first label that is assigned to an input text that is clearly indicated as being the number by nearby text in the input image, or that can be determined to be the number.

[0031] "IDorder" is a first label assigned to input text indicating a number representing an order number. Specifically, "IDorder" is a first label assigned to input text that is clearly indicated as being the number by nearby text in the input image, or that can be determined to be the number. "IDinvoice" is a first label assigned to input text indicating a number representing an invoice number. Specifically, "IDinvoice" is a first label assigned to input text that is clearly indicated as being the number by nearby text in the input image, or that can be determined to be the number. "TITLE" is a first label assigned to input text that represents a document title. "ORGbuyer" is a first label assigned to input text indicating an organization name representing a person who orders or purchases a product. Specifically, "ORGbuyer" is a first label assigned to input text that is clearly indicated as being the organization name by nearby text in the input image, or that can be determined to be the organization name.

[0032] "ORGreceiver" is a first label that is assigned to an input text indicating an organization name that represents a delivery destination or a recipient of a product. Specifically, "ORGreceiver" is a first label that is assigned to an input text that is clearly indicated as the organization name by nearby text in the input image, or that can be determined to be the organization name. "ORGseller" is a first label that is assigned to an input text indicating an organization name that represents an order recipient or seller of a product. Specifically, "ORGseller" is a first label that is assigned to an input text that is clearly indicated as the organization name by nearby text in the input image, or that can be determined to be the organization name. "ORGform" is a first label that is assigned to an input text indicating the organization name that created the template that is the basis of the document image, and is a first label that is assigned to an input text that does not have text that clearly indicates its meaning in the vicinity in the input image. "ORGnoform" is a first label that is assigned to an input text indicating the organization name that is not the organization that created the template that is the basis of the document image, and is a first label that is assigned to an input text that does not have text that clearly indicates its meaning in the vicinity in the input image.

[0033] [First label assignment operation] FIG. 7 is a diagram showing an example of a GUI displayed by the information processing device 100 according to the first embodiment, and is an example of a GUI for assigning a first label to an input text in the editing pane 511 shown in FIG. 5. First, the annotator selects "first label" by operating the switching button 503, and switches the input mode to a mode for inputting a first label. Next, the annotator operates the cursor 703 to select an input text area 701 in the input image area 501. The display control unit 211, triggered by the selection of the input text area 701, pops up a label list 702 near the cursor 703 or the input text area 701. The annotator selects a first label to be assigned to the input text corresponding to the input text area 701 from the label list 702. In the following, a description will be given assuming that ORGnoform is selected by the annotator. When ORGnoform is selected, the first label acquisition unit 204 acquires information indicating ORGnoform as information of the first label. As a result, ORGnoform is assigned as the first label to the input text "XXX Co., Ltd." that corresponds to the input text field 701.

[0034] In the label list 702, a first label predicted to be appropriate as a first label to be assigned to the input text corresponding to the selected input text area 701 is displayed at the top. For example, the input text corresponding to the input text area 701 is "XXX Co., Ltd.", and the input text includes the text "Co., Ltd.", which indicates the legal entity. Therefore, the input text corresponding to the input text area 701 is predicted to be an organization name. In addition, a known method such as text search is used to detect that there is no input text that clearly indicates the meaning near the input text area 701. This makes it possible to predict that ORGform or ORGnoform is appropriate as a first label to be assigned to the input text corresponding to the input text area 701. Due to this prediction, ORGform and ORGnoform are displayed at a higher position in the label list 702 than other labels.

[0035] The method of determining the priority order of displaying the first labels in the label list 702 is not limited to the above-mentioned method. For example, the information processing device 100 may use a trained model obtained by learning such as machine learning to infer the first labels to be assigned to the input text corresponding to the input text area 701, and sort the first labels in descending order according to the inference scores and display them in the label list 702.

[0036] [Results of first label assignment] FIG. 8 is a diagram showing an example of a GUI displayed by the information processing device 100 according to the first embodiment, and the GUI is a diagram showing an example of a GUI of an editing pane 810 among the GUIs displayed on the display unit 105 at the end of the process of S304. Similarly, FIG. 9 is a diagram showing an example of a GUI displayed by the information processing device 100 according to the first embodiment, and the GUI is a diagram showing an example of a GUI of an editing pane 910 among the GUIs displayed on the display unit 105 at the end of the process of S304. Note that FIG. 8 shows, as an example, the editing pane 810 when a thumbnail corresponding to a document whose document type is "purchase order" is selected in the thumbnail pane 512. Also, FIG. 9 shows, as an example, the editing pane 910 when a thumbnail corresponding to a document whose document type is "invoice" is selected in the thumbnail pane 512.

[0037] The input image areas 801 and 901 display an input image corresponding to a thumbnail selected in the thumbnail pane 512, and the label display areas 802 and 902 display label information. In the input image displayed in the input image area 801 in FIG. 8, input text areas A to J corresponding to input texts to which a first label has already been added are highlighted by superimposing a rectangular frame or the like. In the input image displayed in the input image area 901 in FIG. 9, input text areas A to H corresponding to input texts to which a first label has already been added are highlighted by superimposing a rectangular frame or the like. The label information in the label display area 802 displays the correspondence between the first label added at the end of the process of S304 and the input text corresponding to the first label. In FIG. 8, the label information displayed in the label display area 802 includes information on the input texts corresponding to the input text areas A to J and the first labels added to each input text. In addition, in FIG. 9, the label information displayed in the label display area 902 includes the input texts corresponding to the input text areas A to H, and information on the first labels assigned to the respective input texts.

[0038] For example, the input text area A in FIG. 8 is an area located near the upper right corner of the input image displayed in the input image area 801, and the input text corresponding to the input text area A is "XXX IS Research". The first label assigned to the input text is "ORGform". Similarly, the input text area A in FIG. 9 is an area located near the center of the upper side of the input image displayed in the input image area 901, and the input text corresponding to the input text area A is "XXX IS Research". The first label assigned to the input text is "ORGform". Note that the second label information in the label information displayed in the label display areas 802 and 902 is all blank, but this is because the second label has not been assigned at the end of the processing of S304.

[0039] [Compatible maps] FIG. 10 is a diagram showing an example of a correspondence map 1001 according to the first embodiment. Specifically, the correspondence map 1001 shown in FIG. 10 is shown in a table format as an example. The correspondence map 1001 includes a first label column and a second label column, and the second label column has a type column that differs for each document type, and the type column has a priority column and an identification column. This means that the second label is identified by different criteria for each document type. In the following, the correspondence map 1001 will be described as including the following five types of second labels. "DATEpublish" is a second label that is assigned to an input text indicating a date that indicates the publication date of a document. "IDdoc" is a second label that is assigned to an input text indicating a document number that can uniquely identify a document. "TITLE" is a second label that has the same meaning as the first label "TITLE". "ORGfrom" is a second label that is assigned to an input text indicating the name of an organization that issues a document. "ORGto" is a second label that is assigned to the input text indicating the name of the organization to which the document was issued.

[0040] According to the correspondence map 1001, if the document type of the document corresponding to the input image is a purchase order, "DATEpublish" is identified as the second label corresponding to the first label "DATEorder". Also, since there is no second label corresponding to the first label "DATEinvoice", no second label is assigned to the input text to which "DATEinvoice" is assigned as the first label. On the other hand, if the document type of the document corresponding to the input image is an invoice, there is no second label corresponding to the first label "DATEorder", so no second label is assigned to the input text to which "DATEorder" is assigned as the first label. Also, "DATEpublish" is identified as the second label corresponding to the first label "DATEinvoice".

[0041] The priority order sequence of the correspondence map 1001 will be described. In a case where a plurality of different first labels corresponding to the same second label are assigned to an input image, it is considered that the second label is assigned to one of the input texts corresponding to each of these first labels. In such a case, if different priorities are set for the second labels in the correspondence map 1001, the second label is assigned only to the input text to which the first label with the highest priority is assigned. Also, the second label is not assigned to the input text to which the first label with a low priority is assigned. For example, in an input image corresponding to a document whose document type is a purchase order, when there is an input text to which ORGbuyer is assigned as the first label and an input text to which ORGreceiver is assigned as the first label, the second label is specified for the input text as follows. Specifically, in this case, ORGfrom is specified as the second label corresponding to the input text to which ORGbuyer is assigned. On the other hand, nothing is specified as the second label for the input text to which ORGreceiver is assigned. When there are a plurality of input texts to which the first label having the highest priority has been assigned, the same second label may be assigned to any number of input texts equal to or greater than one.

[0042] For input text with ORGform or ORGnoform as the first label in a purchase order, you can choose either ORGto or ORGfrom as the second label, but ORGto and ORGfrom are mutually exclusive. In other words, if you specify ORGto as the second label for input text with ORGform as the first label, you can only specify ORGfrom as the second label for input text with ORGnoform as the first label.

[0043] 10, the correspondence map 1001 includes information that "an input text to which ORGto is assigned and an input text to which ORGfrom is assigned are different from each other" as a label constraint condition. ORGto is a second label that is assigned to an input text indicating an organization name that represents the publisher of a document, and ORGfrom is a second label that is assigned to an input text that indicates an organization name that represents the publisher of a document. Therefore, when an input text to which ORGto is assigned and an input text to which ORGfrom is assigned are identical to each other, it can be defined as a state in which an error has occurred in the annotation work.

[0044] The label constraint conditions are not limited to those described above. For example, the label constraint conditions may be defined as follows: "DATEpublish is a label to be assigned to an input text expressed in a predetermined date format." Also, for example, the label constraint conditions may be defined as follows: "When a first label TITLE is assigned to a plurality of input texts, TITLE as the second label is identified only for the input text having the largest input text area." In other words, various label constraint conditions may be defined to detect errors in the assigned labels based on either the first label or the second label, or on the input text to which the first label and the second label are assigned, or on the input text area corresponding to the input text.

[0045] [Second label candidate] 11 is a diagram showing an example of a GUI displayed by the information processing device 100 according to the first embodiment, which is a diagram showing an example of a GUI of an editing pane 810 among the GUIs displayed on the display unit 105 at the end of the process of S305. Label information is displayed in a label display area 1102, and an input image is displayed in an input image area 1101. Specifically, in the second label column of the label information in the label display area 1102, a second label identified using the correspondence map 1001 and information on the first label of the label information displayed in the label display area 802 shown in FIG. 8 is displayed as a second label candidate.

[0046] Furthermore, in the input image displayed in input image area 1101, rectangular frames indicated by solid lines are superimposed on the positions of input text areas D to F and H to I. These input text areas are input text areas corresponding to input text for which the second label has been identified as a second label candidate, as shown in the label information displayed in label display area 1102. Meanwhile, rectangular frames indicated by dashed lines are superimposed on the positions of input text areas A to C. This means that these input text areas are input text areas corresponding to input text for which the second label could not be uniquely identified as a second label candidate using correspondence map 1001.

[0047] In each of the input text areas A to C, the first label given to the input text is ORGform or ORGnoform. According to the correspondence map 1001, the second label corresponding to the first label ORGform or ORGnoform is either ORGto or ORGform, so the annotator needs to decide which one to select. Therefore, in the label information displayed in the label display area 1102, the second label candidates of the input text corresponding to the input text areas A to C are left blank.

[0048] Furthermore, the input text regions G and J are not superimposed or highlighted with a frame or the like in the input image displayed in the input image area 1101. This means that these input text regions are identified as having no second label candidates to display in the label information displayed in the label display area 1102 because there is no second label corresponding to the first label according to the correspondence map 1001.

[0049] [Second label assignment operation] FIG. 12 is a diagram showing an example of a GUI displayed by the information processing device 100 according to the first embodiment, and is an example of a GUI for assigning a second label to an input text in the editing pane 511 shown in FIG. 5. First, the annotator selects "second label" by operating the switching button 503, and switches the input mode to a mode for inputting a second label. When the input mode is switched to a mode for inputting a second label, the identification unit 205 identifies a second label candidate based on the information of the assigned first label and the correspondence map 1001. Furthermore, the display control unit 211 displays a GUI including an editing pane 1110 shown as an example in FIG. 11. Next, the annotator operates the cursor 703 to select an input text area in which to input a second label within the input image area 1101. Here, it is assumed that an input text area 1201 is selected as shown in FIG. 12(a). The input text area 1201 is an area corresponding to the input text area B in FIG. 11, and a rectangular frame drawn by a dashed line is superimposed on the input text area 1201 because a second label candidate could not be uniquely identified.

[0050] Triggered by the selection of the input text area, the display control unit 211 pops up a label list 1202 as shown in FIG. 12(b) in the same manner as the GUI for selecting the first label. The annotator selects a second label to be assigned to the input text corresponding to the input text area 1201 from the label list 1202. The input text corresponding to the input text area 1201 is an input text for which either ORGto or ORGfrom needs to be selected as the second label. Therefore, in the label list 1202, ORGto and ORGfrom are displayed higher than other second labels as candidates for the second label with a higher priority. On the other hand, second labels such as DATEpublish and IDdoc that are not identified by the identification process by the identification unit 205 are displayed lower than other second labels as candidates for the second label with a lower priority. The label list 1202 may include an item for deleting a second label that has already been assigned by the annotation work by the identification unit 205 or the annotator. In the following description, it is assumed that ORGto is selected by the annotator.

[0051] When ORGto is selected, the second label acquiring unit 208 acquires information indicating ORGto as information of the second label. As a result, ORGto is assigned as the second label to "XXX Co., Ltd.", which is the input text corresponding to the input text area 1201. By assigning ORGto as the second label to the input text corresponding to the input text area 1201, the dashed frame that has been superimposed on the input text area 1201 is changed to a solid frame, as shown in FIG. 12(c).

[0052] Also, the dashed frame that was superimposed on input text areas 1203 and 1204 (corresponding to input text areas A and C shown in FIG. 11) is changed to a double-line frame. This means that the second label of the input text corresponding to input text area 1201, which was previously indefinite, has been uniquely identified, as ORGto has been assigned as the second label of the input text corresponding to input text area 1201. In other words, since the first label of input text area 1201 is ORGnoform and the second label is ORGto, the second labels of input text areas 1203 and 1204, whose first labels are ORGform, are determined to be ORGfrom.

[0053] When the second labels of the input text corresponding to the input text areas 1203 and 1204 are both determined to be ORGfrom and have the same priority in the correspondence map 1001, the annotator may delete either of the second labels. For example, when the annotator deletes the second label of the input text corresponding to the input text area 1203, the frame that was superimposed on the input text area 1203 is no longer displayed, as shown in FIG. 12(d).

[0054] [GUI after adding the second label] 13 is a diagram showing an example of a GUI displayed by the information processing device 100 according to the first embodiment, which is a diagram showing an example of an editing pane 1310 of the GUI displayed on the display unit 105 at the end of the process of S306. In the editing pane 1310, label information is displayed in a label display area 1304, and an input image 1302 is displayed in an input image area 1301. Since a mode for inputting a second label is selected by operating a switching button 1303, a frame emphasizing the input text area is superimposed on each input text area in the input image 1302 to which a second label has been assigned.

[0055] The label information displayed in the label display area 1304 displays the input text, the first label, and the second label of each input text area. As described with reference to FIG. 12, the second label 1305 is estimated by the annotator assigning the second label to another input text, and a solid underline is superimposed on the second label 1305 to indicate that it is an estimation result. Similarly, a double-line frame is superimposed on the input text area 1306 corresponding to the input text in question to highlight it and make it distinguishable from other input text areas. When input of the first label and the second label is completed, the annotator presses a judgment button 1307. The judgment unit 209 judges the consistency of the assigned first label and second label using the pressing of the judgment button 1307 as a trigger.

[0056] [Consistency check process] Fig. 14 is a diagram for explaining an example of consistency determination processing by the determination unit 209 according to the first embodiment. Label information 1410a shown in Fig. 14(a) is a result of consistency determination processing performed by the determination unit 209 on the label information displayed in the label display area 1304. When the determination button 1307 is pressed by the annotator, the label information displayed in the label display area 1304 is updated to label information 1410a as a result of the consistency determination processing by the determination unit 209. In the label information 1410a, the second label assigned to the input text 1401a is ORGto, and the second label assigned to the input text 1402a is ORGfrom.

[0057] However, both of the input texts 1401a and 1402a are "XXX Co., Ltd.", which does not satisfy the above-mentioned label constraint condition, and therefore the second label that has been added may be incorrect. Hereinafter, such a state of the second label is referred to as a "conflict" of the second label. When the second labels are in conflict, the second labels that are in conflict are highlighted in the label information 1410a as shown in FIG. 14(a). Note that the highlighting in the label information 1410a is not limited to only the second labels that are in conflict, and may be performed on the first labels or the input texts that correspond to the second labels that are in conflict. By checking the display as shown in FIG. 14(a), the annotator can recognize that there may be an error in the second labels that have been added. In addition, upon receiving such a display, the annotator can correct the second labels that are in conflict.

[0058] 14B shows an example of a determination result when the second label of the input text 1401a is corrected to ORGfrom by the annotator and then the consistency determination process is executed again by the determination unit 209. Since the second labels assigned to the input texts 1401a and 1402a, both of which contain the input text "IROHA Co., Ltd.", have both become ORGfrom, the second label corresponding to the input text 1402b is no longer highlighted.

[0059] However, the first label given to the input text 1401b is ORGform, and the first label given to the input text 1403a is ORGnoform, but the second labels of these input texts are both ORGfrom. In such a case, the state is different from the correspondence relationship in the correspondence map 1001. Therefore, the second labels given to the input texts 1401b and 1403a are highlighted in the label information 1410b as shown in FIG. 14(b). Note that the highlighting in the label information 1410b is not limited to only the second label, and may be performed on the first label or the input text. By checking the display as shown in FIG. 14(b), the annotator can recognize that there is a possibility that an error has occurred in the given second label. In addition, the annotator can correct the second label upon receiving such a display.

[0060] Label information 1410c shown in Fig. 14(c) is an example of a judgment result when the judgment unit 209 executes the consistency judgment process again after the second label of the input text 1403a is corrected to ORGto by the annotator. The consistency judgment result by the judgment unit 209 for the label information 1410c is Pass. Therefore, the second labels assigned to the input text 1401c and the input text 1403b in the label information 1410c are no longer highlighted. In this way, the annotator can recognize that the correspondence relationship in the correspondence map 1001 is satisfied for the labels of all the input texts.

[0061] 15 is a diagram showing an example of a GUI displayed by the information processing device 100 according to the first embodiment, and the GUI is a diagram showing an example of an editing pane 1510 of the GUI displayed on the display unit 105 at the end of the process of S306. Specifically, a label display area 1504 shown in Fig. 15 displays label information including information of a second label candidate identified by the identification unit 205 based on the correspondence map 1001 and the label information displayed in the label display area 902 shown in Fig. 9.

[0062] For the label information displayed in the label display area 1504, the identification unit 205 can uniquely identify second label candidates for all input texts to which the first labels have been assigned. Therefore, in the input image displayed in the input image area 1501, a rectangular frame drawn with a solid line is superimposed on the input text areas A to D and F to G corresponding to the input texts for which the second label candidates have been identified. On the other hand, the input text areas E and H corresponding to the input texts for which the identification unit 205 has determined that there is no second label candidate are not superimposed or highlighted with a frame or the like.

[0063] According to the information processing device 100 configured as above, by determining the consistency of the correspondence between multiple types of labels assigned by an annotator, it is possible to detect an error in the assignment of a label to an input text. Furthermore, by presenting the result of the determination of the consistency to the annotator, the annotator can easily recognize an error in the assignment of a label and correct the error. As a result, according to the information processing device 100, it is possible to reduce errors in the annotation work by the annotator.

[0064] <First Modification of the First Embodiment> In the first embodiment, the identification unit 205 identifies second label candidates from the first label based on the correspondence relationship defined in the correspondence map 1001, and the identified second label candidates are displayed at the top of the label list 1202. In a first modified example of the first embodiment (hereinafter simply referred to as the "first modified example"), a description will be given of a form in which the identification unit 205 estimates and identifies second label candidates corresponding to the input text or the first label using a trained model generated by machine learning or the like. In the label list according to the first modified example, the order of the second label candidates is determined based on, for example, an estimated score included in an estimation result by the trained model.

[0065] The estimation result by the trained model includes, for each second label estimated by the trained model, an estimated score expressed by, for example, a value not less than 0 and not more than 1. The identification unit 205 outputs, as a second label candidate, a second label whose estimated score is equal to or greater than a predetermined threshold value such as 0.8 (hereinafter referred to as the "first threshold value"). In addition, a second label whose estimated score is equal to or greater than a predetermined threshold value such as 0.95 (hereinafter referred to as the "second threshold value") is highlighted in the label list as a second label candidate with high accuracy.

[0066] FIG. 16 is a diagram showing an example of an input image 1601 displayed in an input image area according to the first modified example, and a label list 1602 displayed in a pop-up display. The input image 1601 is an image displayed in the input image area of ​​the GUI at the end of the process of S305. In the following, it is assumed that the identification unit 205 estimates that there is no second label candidate for the input text corresponding to the input text area A. In addition, it is assumed that the identification unit 205 estimates "ORGfrom" as the first second label candidate for the input text corresponding to the input text area B, and estimates "ORGto" as the second second label candidate. In addition, it is assumed that the identification unit 205 estimates "ORGto" as the second label candidate for the input text corresponding to the input text area C. It is assumed that the estimation score of "ORGfrom", which is the first second label candidate, is higher than the estimation score of "ORGto", which is the second second label candidate.

[0067] This means that the input text corresponding to the input text region A does not have a second label with an estimated score equal to or greater than the first threshold. Therefore, a rectangular frame drawn with a thin dashed line, which indicates that there is no second label candidate, is superimposed on the input text region A of the input image 1601, and the input text region A is visually emphasized. Moreover, the above means that the input text corresponding to the input text region B has "ORGfrom" and "ORGto" as second labels with an estimated score equal to or greater than the first threshold. Furthermore, the input text corresponding to the input text region C has "ORGto" as a second label with an estimated score equal to or greater than the first threshold. Therefore, a rectangular frame drawn with a thick dashed line, which indicates that a second label candidate has been specified, is superimposed on the input text regions B and C of the input image 1601, and the input text regions B and C are visually emphasized. Note that the input text region with a solid line frame superimposed on the input image 1601 is the input text region corresponding to the input text in which a second label candidate has been uniquely specified by the specification process using the correspondence map 1001 by the specification unit 205.

[0068] In the label list 1602 that is displayed when the annotator performs an operation to assign a second label to the input text corresponding to the input text region B, the second-ranked second label candidate is displayed at the top following the first-ranked second label candidate. That is, in the label list 1602, ORGfrom is displayed in the upper row, and ORGto in the next row. Also, for example, if the estimated score of ORGfrom, which is the first-ranked second label candidate, is equal to or greater than the second threshold, ORGfrom is displayed in a highlighted manner so as to be distinguishable from the other second label candidates, such as by superimposing an underline or the like, as shown in the label list 1602.

[0069] As described above, by configuring the identification unit 205 to estimate and identify the second label candidate using the trained model, it is possible to prioritize the second label candidates that cannot be prioritized by the identification using the correspondence map 1001. Note that, for the estimation result according to this modification, the identification result based on the correspondence map 1001 and the estimation result by the trained model are displayed in a distinguishable manner in the input image 1601. With such a configuration, it is possible to prompt the annotator to confirm the estimation result of the second label by the trained model. Note that the display method of the estimation result by the trained model is not limited to this. For example, when the estimation result of the second label candidate for the input text is only the first second label candidate, the first second label candidate may be determined as the second label candidate and displayed in the same manner as the identification result based on the correspondence map 1001.

[0070] <Second Modification of the First Embodiment> In the first embodiment, a form has been described in which an annotator assigns a first label, a second label candidate is acquired by the identification unit 205, and the annotator confirms and corrects the second label to assign a final second label. In a second modified example of the first embodiment, a form will be described in which an annotator inputs and assigns all of the first and second labels, and the determination unit 209 determines the consistency of the first and second labels assigned by the annotator.

[0071] 17 is a diagram showing an example of label information 1701, which is a result of the determination unit 209 determining the consistency of the first label and the second label assigned by the annotator to the input image 401 corresponding to a document whose document type is a purchase order. In the label information 1701, ORBbuyer, ORGseller, and ORGreceiver are assigned as the first label to the input text corresponding to the input text areas H, I, and J, respectively. Similarly, ORGfrom and ORGfrom are assigned as the second label to the input text corresponding to the input text areas H and J, respectively, and no second label is assigned to the input text corresponding to the input text area I.

[0072] Here, ORGfrom is assigned as the second label to both the input text corresponding to the input text area H to which ORGbuyer is assigned as the first label, and the input text corresponding to the input text area J to which ORGreceiver is assigned. However, according to the correspondence map 1001, the priority of ORGbuyer is higher than the priority of ORGreceiver. Therefore, the second label corresponding to the input text area J, which has a low priority, is highlighted as ORGfrom assigned as the second label to the input text corresponding to the input text area J, which has a low priority, may be an input error by the annotator. Note that the area to be highlighted is not limited to the area of ​​the second label, and the input text corresponding to the input text area J or the area of ​​the first label may be highlighted.

[0073] Further, the second label is not assigned to the input text corresponding to the input text area I to which ORGseller is assigned as the first label. However, according to the correspondence map 1001, when the first label is ORGseller, ORGto may be assigned as the second label. Therefore, the reason why the second label is not assigned to the input text corresponding to the input text area I is that there is a possibility that the annotator made an input error such as forgetting to assign the label, and the area of ​​the second label corresponding to the input text area I is highlighted. The area to be highlighted is not limited to the area of ​​the second label, and the input text corresponding to the input text area I or the area of ​​the first label may be highlighted.

[0074] As described above, according to the information processing device 100 of this modification, even if an annotator inputs and assigns the first label and the second label, it is possible to detect an error in the assignment of at least one of the first label and the second label. Furthermore, by presenting the consistency determination result to the annotator, the annotator can easily recognize an error in the assignment of the label and correct the error. As a result, according to the information processing device 100, it is possible to reduce errors in the annotation work by the annotator.

[0075] <Third Modification of the First Embodiment> In the first embodiment, a label representing the meaning of the input text included in each input text region in the input image is assigned as the first label. However, the first label is not limited to the label assigned to the input text, and various modifications are possible. For example, a label representing the document type of the input image may be assigned as the first label. Specifically, a label constraint condition such as "when a label "contract" is assigned as the first label representing the document type, a label "ORGto" is not assigned as the second label" may be defined. The determination unit 209 may perform a consistency determination process including the label constraint condition. In addition, for example, a label such as a figure, table, header, or footer may be assigned as the first label to any position or region in the input image. For example, in this case, a label constraint condition such as "when a label "footer" is assigned as the first label, the priority of assigning a label "ORGfrom" as the second label is the lowest" may be defined.

[0076] According to the information processing device 100 configured as above, by determining the consistency of the correspondence between multiple types of labels assigned by an annotator, it is possible to detect an error in the assignment of a label to an input text. Furthermore, by presenting the result of the determination of the consistency to the annotator, the annotator can easily recognize an error in the assignment of a label and correct the error. As a result, according to the information processing device 100, it is possible to reduce errors in the annotation work by the annotator.

[0077] <Other embodiments> The present disclosure can also be realized by a process in which a program for implementing one or more functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) for implementing one or more functions.

[0078] Although the preferred embodiments of the present disclosure have been described, the present disclosure is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present disclosure. Furthermore, the present disclosure allows free combinations of the embodiments, modifications of any of the components of the embodiments, or omission of any of the components of the embodiments, within the scope of the disclosure.

[0079] <Configuration of the present disclosure> The present disclosure includes the following configurations, methods, and programs.

[0080] [Configuration 1] A character string acquisition means for acquiring a character string of a named entity extracted from input document data; a first label acquiring means for acquiring a first label to be assigned to the character string acquired by the character string acquiring means; a second label acquiring means for acquiring a second label different from the first label to be assigned to the character string acquired by the character string acquiring means; a determination means for determining consistency between the first label acquired by the first label acquisition means and the second label acquired by the second label acquisition means; an output means for outputting a result of the determination by the determination means; having An information processing device comprising:

[0081] [Configuration 2] the output means includes a result display means for controlling the display of a result of the determination by the determination means on a display device, a result of the determination by the determining means is displayed on the display device; 2. The information processing device according to configuration 1,

[0082] [Configuration 3] the output means outputs information on the first label acquired by the first label acquisition means and information on the second label acquired by the second label acquisition means in association with information on the character string; 3. The information processing device according to configuration 1 or 2,

[0083] [Configuration 4] said output means outputs the information on the first label acquired by said first label acquisition means and the information on the second label acquired by said second label acquisition means in association with the information on the character string, and outputs information indicating an image area including the character string in the input document data in association with the information on the character string; 4. The information processing device according to any one of configurations 1 to 3,

[0084] [Configuration 5] said output means outputs information on said first label acquired by said first label acquisition means and information on said second label acquired by said second label acquisition means in association with information on said character string, and outputs information indicating a result of the determination by said determination means in association with said input document data; 5. The information processing device according to any one of configurations 1 to 4,

[0085] [Configuration 6] The first label acquisition means acquires, as the first label, a first label candidate selected by a user from a first label list including one or more first label candidates; the second label acquiring means acquires, as the second label, the second label candidate selected by the user from a second label list including one or more second label candidates different from the first label candidate; 6. The information processing device according to any one of configurations 1 to 5,

[0086] [Configuration 7] a specifying means for specifying one or more second label candidates based on a correspondence map indicating a correspondence relationship between the first label and the second label and the first label acquired by the first label acquiring means; and the second label acquisition means acquires, as the second label, the second label candidate selected by the user from among the one or more second label candidates identified by the identification means; 7. The information processing device according to configuration 6,

[0087] [Configuration 8] the output means includes a candidate display means for controlling the display of the one or more second label candidates identified by the identification means on a display device, the second label acquisition means acquires, as the second label, the second label candidate selected by the user from the one or more second label candidates displayed on the display device; 8. The information processing device according to configuration 7,

[0088] [Configuration 9] the output means includes an image display means for controlling the display of a document image corresponding to the input document data on the display device, and when the user selects an image area including the character string in the document image displayed on the display device, the output means displays the one or more second label candidates identified by the identification means as the second label list near the image area; 9. The information processing device according to configuration 8,

[0089] [Configuration 10] when displaying the second label list, the output means displays the second label list in such a manner that the second label candidates having higher priorities are displayed higher on the basis of priorities of the second label candidates; 10. The information processing device according to configuration 9,

[0090] [Configuration 11] The specifying means specifies one or more of the second label candidates, and obtains a score indicating a likelihood of the specified second label candidate in association with the second label candidate; the output means displays the second label candidates in the second label list in a highlighted manner based on the scores when displaying the second label list; 11. The information processing device according to configuration 9 or 10,

[0091] [Configuration 12] the determination means switches a determination criterion based on a document type of the input document data; 12. The information processing device according to any one of configurations 1 to 11,

[0092] [Configuration 13] the determination means determines consistency between the first label acquired by the first label acquisition means and the second label acquired by the second label acquisition means, based on a correspondence map indicating a correspondence relationship between the first label and the second label for each document type and the document type of the input document data; 13. The information processing device according to any one of configurations 1 to 12,

[0093] [method] a character string acquisition step of acquiring a character string of a named entity extracted from the input document data; a first label acquiring step of acquiring a first label to be assigned to the character string acquired by the character string acquiring step; a second label obtaining step of obtaining a second label different from the first label to be assigned to the character string obtained by the character string obtaining step; a determination step of determining consistency between the first label acquired in the first label acquisition step and the second label acquired in the second label acquisition step; an output step of outputting a result of the determination step; containing, An information processing method comprising:

[0094] [program] A program for causing a computer to function as the information processing device according to any one of configurations 1 to 13. [Explanation of symbols]

[0095] 100 Information processing device 202 Text Acquisition Section 204 First label input section 208 Second label input section 209 Judgment section

Claims

1. A character string acquisition means for acquiring a character string of a named entity extracted from input document data; a first label acquiring means for acquiring a first label to be assigned to the character string acquired by the character string acquiring means; a second label acquiring means for acquiring a second label different from the first label to be assigned to the character string acquired by the character string acquiring means; a determination means for determining consistency between the first label acquired by the first label acquisition means and the second label acquired by the second label acquisition means; an output means for outputting a result of the determination by the determination means; having An information processing device comprising:

2. the output means includes a result display means for controlling the display of a result of the determination by the determination means on a display device, a result of the determination by the determining means is displayed on the display device; 2. The information processing apparatus according to claim 1,

3. the output means outputs information on the first label acquired by the first label acquisition means and information on the second label acquired by the second label acquisition means in association with information on the character string; 2. The information processing apparatus according to claim 1,

4. the output means outputs the first label information acquired by the first label acquisition means and the second label information acquired by the second label acquisition means in association with the character string information, and outputs information indicating an image area including the character string in the input document data in association with the character string information; 2. The information processing apparatus according to claim 1,

5. the output means outputs information on the first label acquired by the first label acquisition means and information on the second label acquired by the second label acquisition means in association with information on the character string, and outputs information indicating a result of the determination by the determination means in association with the input document data; 2. The information processing apparatus according to claim 1,

6. The first label acquisition means acquires, as the first label, a first label candidate selected by a user from a first label list including one or more first label candidates; the second label acquiring means acquires, as the second label, the second label candidate selected by the user from a second label list including one or more second label candidates different from the first label candidate; 2. The information processing apparatus according to claim 1,

7. a specifying means for specifying one or more second label candidates based on a correspondence map indicating a correspondence relationship between the first label and the second label and the first label acquired by the first label acquiring means; and the second label acquisition means acquires, as the second label, the second label candidate selected by the user from among the one or more second label candidates identified by the identification means; The information processing device according to claim 6 .

8. the output means includes a candidate display means for controlling the display of the one or more second label candidates identified by the identification means on a display device, the second label acquisition means acquires, as the second label, the second label candidate selected by the user from the one or more second label candidates displayed on the display device; The information processing device according to claim 7 .

9. the output means includes an image display means for controlling the display of a document image corresponding to the input document data on the display device, and when the user selects an image area including the character string in the document image displayed on the display device, the output means displays the one or more second label candidates identified by the identification means as the second label list near the image area; The information processing device according to claim 8 .

10. when displaying the second label list, the output means displays the second label list in such a manner that the second label candidates having higher priorities are displayed higher on the basis of priorities of the second label candidates; The information processing device according to claim 9 .

11. The specifying means specifies one or more of the second label candidates, and obtains a score indicating a likelihood of the specified second label candidate in association with the second label candidate; the output means displays the second label candidates in the second label list in a highlighted manner based on the scores when displaying the second label list; The information processing device according to claim 9 .

12. the determination means switches a determination criterion based on a document type of the input document data; 2. The information processing apparatus according to claim 1,

13. the determination means determines consistency between the first label acquired by the first label acquisition means and the second label acquired by the second label acquisition means, based on a correspondence map indicating a correspondence relationship between the first label and the second label for each document type and the document type of the input document data; 2. The information processing apparatus according to claim 1,

14. a character string acquisition step of acquiring a character string of a named entity extracted from the input document data; a first label acquiring step of acquiring a first label to be assigned to the character string acquired by the character string acquiring step; a second label obtaining step of obtaining a second label different from the first label to be assigned to the character string obtained by the character string obtaining step; a determination step of determining consistency between the first label acquired in the first label acquisition step and the second label acquired in the second label acquisition step; an output step of outputting a result of the determination step; containing, An information processing method comprising:

15. A program for causing a computer to function as the information processing device according to any one of claims 1 to 13.

Images