Information processing device, information processing method, and program

Abstract

To enable learning of a unique expression extraction model corresponding to a plurality of languages at low cost without deteriorating inference accuracy.SOLUTION: A document analysis part acquires an input character string corresponding to each text block and management information of a teacher label corresponding to the input character string from input document data acquired by a document data acquisition part. A machine translation part performs machine translation processing to a plurality of input character strings included in the input document data, and acquires translation character strings translated into an inference object language. A teacher label mapping part 214 generates management information of a teacher label re-assigning the teacher label to the translation character strings in accordance with an order of arranging tokens in the inference object language on the basis of the management information of the teacher label stored in association with the input document data.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an information processing technique for extracting named entities from document data. [Background technology]

[0002] In recent years, methods using machine learning models have been adopted to extract named entities such as proper nouns, such as company names and personal names, and date and time expressions from document data. Examples of machine learning models include CRF (Conditional Random Field) and BERT (Bidirectional Encoder Representations from Transformers). In these methods using machine learning models, when generating an inference model for named entity extraction that supports multiple languages, it is generally divided into two major methods. For example, if the multiple languages ​​are Japanese and English, the first method is to generate training data by collecting domain documents to be inferred in Japanese and assigning teacher labels, and to generate an inference model for Japanese by training a machine learning model using the generated teacher data. Furthermore, an inference model for English is generated in the same procedure as for Japanese. The second method is to generate an inference model that supports multiple languages ​​by using a multilingual machine learning model that has dictionaries for both Japanese and English and training the machine learning model using the generated teacher data. In either case, in order to generate training data, it is necessary to collect domain document data in each language and assign teacher labels, which incurs processing costs.

[0003] In Patent Document 1, a named entity extraction model for a target language is generated at low cost by finding optimal weight parameters for another language (target language) from the weight parameters of a named entity extraction model trained in one language (source language). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] WO 17 / 130434 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology of Patent Document 1 only involves transferring the weight parameters of the named entity extraction model between languages, and it is not possible to capture structures that are unique to the target language and that are not present in the source language, and the accuracy of inference in the target language is lower than that of inference in the source language.

[0006] Therefore, an object of the present invention is to enable learning of a named entity extraction model that supports multiple languages ​​at low cost without reducing inference accuracy. [Means for solving the problem]

[0007] The present invention is an information processing device comprising: an extraction means for extracting a first string corresponding to a truth label from input document data to which a truth label has been assigned; an acquisition means for acquiring a second string obtained by translating the first string extracted by the extraction means into a target language different from the main language of the input document data; and an assignment means for assigning, to the second string acquired by the acquisition means, a truth label corresponding to the truth label assigned to the first string corresponding to the second string, wherein the truth label includes a tag assigned to each token which is a predetermined constituent unit of the first string and the second string, and the assignment means assigns, to at least one of the tokens included in the second string, a tag different from the tag assigned to the token of the first string corresponding to the token when the second string includes a plurality of tokens and the order of corresponding tokens in the first string and the second string is different. Effect of the Invention

[0008] According to the present invention, it is possible to train a named entity extraction model that supports multiple languages ​​at low cost without reducing inference accuracy. [Brief description of the drawings]

[0009] [Figure 1] Hardware configuration diagram according to an embodiment [Diagram 2] System configuration diagram according to an embodiment [Diagram 3] FIG. 1 is a diagram showing an example of association of truth labels included in input document data with translated text according to an embodiment; [Figure 4] FIG. 1 is a flow diagram for explaining a process for generating translation document data to which truth labels have been added from domain document data to which truth labels have been added according to an embodiment; [Diagram 5] FIG. 1 is a diagram for explaining replacement of named entities in translated document data according to an embodiment; [Figure 6] FIG. 1 is a flow diagram illustrating a process for improving teacher data according to an embodiment; [Figure 7] FIG. 1 is a flow diagram illustrating a process for improving teacher data according to an embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0011] <Embodiment 1> [term] The terms used in this embodiment are defined as follows. A string is one or a series of characters, a token is a predetermined structural unit of linguistic information that can be input to a natural language processing model, and a token string is one or a series of tokens. Named entities are collectively referred to as proper nouns such as personal names and organization names, date expressions, and the like that exist in a document. Named entity extraction is a technique for recognizing predefined named entities that exist in a document as character strings. A named entity tag is an entity that assigns a tag in the IOB2 format, which is generally used in named entity extraction, to each token. Document data that is classified as a target for named entity extraction is called domain document data. For example, domain document data is document data of a document type such as an invoice or a medical chart. The language in which the domain document is written is called the source language.

[0012] [Hardware configuration] FIG. 1 is a diagram showing a hardware configuration of an information processing device 100 according to the first embodiment. In the diagram, a CPU 101 executes programs stored in a ROM 102 and a RAM 103 to control various devices connected to a system bus 109. The ROM 102 is realized by a non-volatile memory and is a storage unit for storing a BIOS (Basic Input / Output System) program, a boot program, and the like. The RAM 103 is realized by a volatile memory and is a storage unit used as a work memory when the CPU 201 executes a control program. The input unit 105 is a touch panel, a keyboard, a mouse, a robot controller, and the like, and receives user input. The display device 106 outputs the calculation results of the CPU 101 and various information to the display device according to instructions from the CPU 101. The display device may be a liquid crystal display device, a projector, an LED indicator, and the like, and the type of the display device is not limited. The communication I / F 107 is a communication interface that communicates with an external device via a network, and the communication interface may be Ethernet (registered trademark), USB, serial communication, wireless communication, and the like, and the type of the communication interface is not limited. The I / O 108 is an input / output unit that directly inputs and outputs data to and from an external device such as a scanner.

[0013] [System Configuration] The named entity extraction model learning unit 210 includes a document data acquisition unit 211, a document analysis unit 212, a machine translation unit 213, a teacher label mapping unit 214, a named entity replacement unit 215, a teacher data score calculation unit 216, a teacher data improvement unit 217, and a model learning unit 218. The storage unit 220 stores a document dataset 221 and a teacher dataset 222.

[0014] The document data acquisition unit 211 acquires, as input document data, document data to be processed and to which teacher labels have been added, which is stored in the document data set 221 in the storage unit 220. The input document data is not limited to a particular format, and may be a PDF file or an image file.

[0015] The document analysis unit 212 detects character string areas described in the input document data, and acquires character strings described in the detected character string areas. Furthermore, based on the coordinates of the detected character string areas in the coordinate system defined in each document data, the document analysis unit 212 stores the acquired multiple character strings as a single combined character string. The character strings are combined, for example, by scanning the character string areas from left to right and top to bottom. For example, this can be achieved by using a PDF analysis tool for PDF files, or an OCR engine for image files.

[0016] The machine translation unit 213 performs machine translation of a character string written in the main language of the input document data that can be obtained from the character string area into the inference target language to obtain a translated character string. For example, if there is Japanese document data with teacher labels added and the inference target language is English, it is possible to obtain an English translation character string from a Japanese character string by using a machine translation API of an AI translation service.

[0017] The truth label mapping unit 214 assigns truth labels to translation strings based on the truth label management information stored in association with the input document data. As an example of a named entity tag for a truth label, it is assigned in the IOB2 format, which is generally used in Named Entity Recognition. In the IOB2 format, the tag "B-<arbitrary string for identifying the label>" is assigned to the first token of a string representing a named entity, and the tag "I-<arbitrary string for identifying the label>" is assigned to the second and subsequent tokens. The tag "O" is assigned to tokens that are not included in the previously defined named entities.

[0018] FIG. 3 shows an example of teacher label association for a translation string based on the management information of teacher labels stored in association with input document data. Consider a case where the source string is "Publication date: June 21, 2022" and the named entity label "Date" is assigned to the token string "Publication | Date |: | 2022 | Year | June | Month | 21 | Day" with each token separated by "|". In this case, the tag "B-Date" is assigned to the token "2022", and the tags "I-Date" are assigned to the tokens "Year", "6", "Month", "21", and "Day". Assume that the translation string obtained by machine translation of the source string is "Publication date: June 21, 2022", and the token string is "Publication | date |: | June | 21 |, | 2022". In this case, the tag "B-Date" is assigned to the token "June", and the tag "I-Date" is assigned to the tokens "21", ",", and "2022". In this way, the teacher-labeled translated document data consisting of translation character strings to which teacher labels are associated is obtained.

[0019] The named entity replacement unit 215 replaces named entities belonging to a predetermined classification, such as names, whose translation character strings occur frequently in documents in the translation source language but occur less frequently in documents in the inference target language, with named entities in the inference target language that are predetermined according to the classification. Among the named entities contained in the domain document data, named entities of proper nouns, such as place names and personal names, that are translated into katakana, are greatly influenced by cultural differences with the named entities in the inference target language, and are often not suitable as training data for the inference target language. For example, "John Smith" is often used as an example of how to write a person's name in an English form, but is not often written as a person's name in a Japanese form. Therefore, by replacing the named entities with a person's name that has characteristics that are likely to be written in a Japanese form, translated document data that is more suitable as training data can be obtained.

[0020] The teacher data score calculation unit 216 calculates a teacher data score for quantitatively evaluating the reliability of the translated document data to which the teacher labels have been added as teacher data for the named entity extraction model. The teacher data score is a value calculated from one or more of the content rate of the predetermined important words in the domain document data, the proportion of n-grams of the feature words statistically extracted from the set of domain document data, and the classification score of whether the data is domain document data or not. For example, a case is taken in which the classification of the domain document data is invoice, and the inference target language is Japanese. The content rate of the important words in the domain document data can be realized by calculating the number of important words contained in the translated document data by predefining three words, "invoice", "amount", and "transfer", as important words in the domain document data. The proportion of n-grams of the feature words statistically extracted from the set of domain document data can be realized by calculating the proportion of n-grams containing the feature words present in the translated document data based on information on n-grams containing the feature words present in the set of domain document data. The extraction of feature words in the domain document data can be performed using, for example, TF-IDF (Term Frequency - Inverse Document Frequency). For example, the inference score of a trained classification model capable of classifying the document type of the domain document data can be used as the classification score for whether the document is the domain document data. Even if there is no trained classification model, if a large amount of domain document data is collected, it is also possible to train a classification model for whether the document is the domain document data using the collected domain document data and obtain the inference score of the trained classification model.

[0021] The teacher data improvement unit 217 improves the teacher data by replacing the translation string constituting the teacher-labeled translated document data with a string including n-grams of characteristic words of the domain document data based on the teacher data score. The string including n-grams of characteristic words of the domain document data is obtained by the teacher data score calculation unit 216. After the teacher data improvement process is completed, if the teacher data score is greater than a predetermined threshold, the improved teacher-labeled translated document data is stored in the teacher data set 222 in the storage unit 220.

[0022] The model learning unit 218 uses the teacher data stored in the teacher data set 222 in the storage unit 220 to perform supervised learning of the language model to generate a named entity extraction model.

[0023] The document dataset 221 is a collection of document data with teacher labels added thereto for input to the document data acquisition unit 201. The teacher dataset 222 is a collection of translated document data with improved teacher labels added thereto that is stored by the teacher data improvement unit 217, and is used in supervised learning by the model learning unit 218.

[0024] [Processing flow] Next, a process flow of named entity extraction model learning according to this embodiment, which is realized by software executed by the CPU 101 in the information processing device 100, will be described with reference to FIG.

[0025] In S401, the document data acquisition unit 211 acquires, as input document data, document data to be processed that has been assigned truth labels and is included in the document data set 221 stored in the storage unit 220.

[0026] In S402, the document analysis unit 212 acquires character strings represented by character codes corresponding to each text block detected from the input document data acquired by the document data acquisition unit 211 from the input document data or acquires them by performing OCR processing.

[0027] In S403, the machine translation unit 213 performs machine translation processing on a plurality of input character strings included in the input document data, and acquires translated character strings translated into the inference target language.

[0028] In S404, the truth label mapping unit 214 generates truth label management information by reassigning truth labels to translation strings according to the order of tokens in the inference target language, based on truth label management information stored in association with the input document data. The truth label mapping unit 214 then associates the generated truth label management information with the translation string.

[0029] In S405, the named entity replacement unit 215 determines whether or not the named entity is a predetermined named entity to be replaced. If it is a named entity to be replaced, the process proceeds to S406. If it is not a named entity to be replaced, the process returns to S403 if there is a next translation character string, and if there is no next translation character string, the process proceeds to S407.

[0030] In S406, the named entity replacement unit 215 replaces the named entities included in the translation character string with named entities set according to the inference target language, and generates teacher-labeled translated document data.

[0031] In S407, the teacher data score calculation unit 216 first calculates a teacher score for quantitatively evaluating the reliability of the translated document data to which the teacher labels have been assigned as teacher data. The teacher data improvement unit 217 improves the translated document data to which the teacher labels have been assigned based on the calculated teacher score to make it more suitable as teacher data for the inference target language, and stores the improved data in the teacher dataset 222 in the storage unit 220. A specific improvement method will be described later with reference to Figs. 6 and 7.

[0032] In S408, the model learning unit 218 performs supervised learning of a machine learning model using the teacher data 222 stored in the storage unit 220 to generate a named entity extraction model.

[0033] [Example of how to generate labeled translation data] A specific example will be used to explain steps S402 to S405 in the process flow shown in Fig. 4. Fig. 5 shows input document data 501, a json-format text file 503 that manages character strings and teacher labels for each text block of the input document data 501, and translated document data 504 machine-translated by the machine translation unit 213 from the input document data. Fig. 5 also shows a json-format text file 506 that manages translated character strings and teacher labels for each text block of the translated document data 504.

[0034] Assume that the input document data is, for example, scanned image data of an invoice, and character strings corresponding to four text blocks 502a, 502b, 502c, and 502d are obtained by character recognition processing in S402. In S403, a translated character string is obtained for each text block by machine translation using an AI translation API. Assume that the character string "Invoice" in text block 502a is translated as the translated character string "invoice" in text block 505a. Furthermore, if the named entity tag "O" is assigned as a teacher label to the character string "Invoice" in text block 502a, the named entity tag "O" is also assigned to the translated character string "invoice" in text block 505a.

[0035] Assume that the character string "Bill To:Canon Inc." in text block 502b is translated as the translated character string "Bill To:Canon Inc." 505b in text block 505b. Also assume that for each token of the character string "Canon Inc.", the named entity tag "B-ORG" is assigned to "Canon", the named entity tag "I-ORG" is assigned to "Inc", and the named entity tag "I-ORG" is assigned to ".". In this case, for each token of the translated character string "Canon Inc.", the named entity tag "B-ORG" is assigned to "Canon", the named entity tag "I-ORG" is assigned to "stock", and the named entity tag "I-ORG" is assigned to "company".

[0036] Assume that the character string "John Smith" in text block 502d is translated as the translated character string "John Smith" in text block 505d. The proper noun "John Smith" is likely to be included in documents used in regions where English is the native language, but is unlikely to be included in documents used in regions where Japanese is the native language. Therefore, the translated character string in text block 505e is replaced with, for example, "Shimomaruko Taro," which has the characteristics of a name that appears more frequently in Japanese documents. In this way, by replacing named entities whose frequency of appearance differs greatly depending on the language with named entities in the inference target language to which the same teacher label is assigned, a translated character string that is more suitable as teacher data in the inference target language can be generated.

[0037] [Use case 1: Improving translated document data with teacher labels] In FIG. 6, a use case is assumed in which almost no domain document data to be inferred has been collected, and a specific example is used to explain the teacher data improvement process performed in S406 of the process flow shown in FIG.

[0038] In S601, the teacher data improvement unit 217 calculates a teacher data score for the teacher-labeled translated document data obtained by executing the processes of S401 to S405. First, words that are likely to appear frequently in the domain document data to be inferred are defined as key words. For example, if the domain document data to be inferred is invoice document data, then "claim", "amount", "transfer", etc. are defined as key words. The frequency of occurrence of key words is obtained for translated character strings obtained for each text block included in the teacher-labeled translated document data, and the content rate of key words is calculated. The content rate of key words is calculated by using the total number of words included in the teacher-labeled translated document data as the denominator and the frequency of occurrence of key words as the numerator. The teacher data score based on the content rate of key words is called score word Then, we can define it as follows. Here, we define the set of important words contained in the translated document data with teacher labels as W ILet W be the set of all tokens contained in the translated document data with teacher labels. Let hist be the number of occurrences of elements contained in set w among tokens contained in the translated document data with teacher labels. w Let us assume that.

[0039]

number

[0040] In S602, the teacher data improvement unit 217 determines whether the teacher data score of the teacher-labeled translated document data is equal to or greater than a predetermined threshold. If the teacher data score is equal to or greater than the threshold, the process proceeds to S603. If the teacher data score is less than the threshold, the process ends.

[0041] In S603, the teacher data improvement unit 217 adds teacher-labeled translated document data whose teacher data score is equal to or greater than the threshold to the teacher dataset 222. For example, if the threshold for the teacher data score is set to 0.1, the content rate of heavy words in the above translated character string has a score higher than the threshold, so the above translated character string is added to the teacher dataset as teacher-labeled translated document data.

[0042] If the teacher-labeled translated document data contains many noise characters, such as symbols caused by misrecognition by the OCR engine, characters that match the noise characters may be removed using regular expressions before calculating the content rate of important words. The frequency of appearance of important words may be defined as the teacher data score as it is.

[0043] [Use case 2: Improving translated document data with teacher labels] In FIG. 7, assuming a use case where a large amount of domain document data to which teacher labels have been assigned as the inference target has been collected, the teacher data improvement process performed in S406 of the processing flow shown in FIG. 4 is explained using a concrete example.

[0044] In S701, the teacher data improvement unit 217 calculates a teacher data score for the translated document data. First, to calculate the teacher data score, a statistic determined by the word distribution of the collected set of domain document data to be inferred and the set of general document data is calculated. When the amount of data is small, as in use case 1, statistical processing is not suitable because the accuracy is low, but when the amount of data is large, as in this use case, it is desirable to perform statistical processing. Specifically, TF-IDF is used to extract feature words, which are token strings with high importance in the domain document data.

[0045] First, we generate a TF-IDF feature vector for each document data in the set of domain document data and the set of general document data. The general document data can be obtained from publicly available corpora or databases of document data collected by web crawling such as Common Crawl.

[0046] Next, those token strings for which the TF-IDF value of the TF-IDF feature vector of the set of domain document data is equal to or greater than a predetermined threshold value are extracted as feature words in the domain document data.

[0047] Next, each character string that constitutes the domain document data to be inferred is divided into n-gram units, and n-grams containing the extracted feature words are extracted. Each translated character string that constitutes the translated document data is then similarly divided into n-grams, and the frequency of occurrence of n-grams containing the feature words extracted from the domain document data to be inferred is calculated. The ratio of the frequency of occurrence of n-grams containing feature words to the frequency of occurrence of all n-grams calculated in this way is used as the teacher data score. This makes it possible to quantitatively evaluate the reliability of translated document data to which teacher labels have been added. The teacher data score based on the ratio of n-grams containing feature words is given as score ngram Then, it can be defined as follows. Here, the set of n-grams containing feature words included in the translated document data is G I Let G be the set of all tokens contained in the translated document data. Let hist be the number of occurrences of elements contained in set g among the tokens contained in the translated document data. g Let us assume that.

[0048]

number

[0049]

number

[0050]

number

[0051] In S703, the teacher data improvement unit 217 calculates the teacher data score again for the new translated document data generated in S702. S702 and S703 are repeated until the calculated teacher data score is equal to or greater than a predetermined threshold or the number of trials of S702 and S703 exceeds a predetermined number. After completion, if the teacher data score is equal to or greater than the predetermined threshold, the translated document data with teacher labels added and improved by string replacement is added to the teacher dataset.

[0052] A specific example of S407 in the process flow shown in FIG. 4 will be described. A teacher dataset is loaded from the teacher dataset 222 in the storage unit 220. At this time, the teacher-labeled document data stored in the document dataset 221 in the storage unit 220 may be included in the teacher dataset 222. The machine learning model uses a language model that has been pre-trained with a large amount of document data, such as mBERT. By using a language model trained with a multilingual dictionary, such as mBERT, it is possible to train the language in which the domain document data is written, in addition to the inference target language. When training with document data in multiple languages, teacher-labeled document data is randomly sampled and used for training. Alternatively, after training with teacher-labeled document data, fine-tuning may be performed with teacher-labeled translated document data of the inference target language. Each character string constituting the teacher-labeled translated document data included in the teacher dataset is divided into token units that can be input to BERT by a Tokenizer, and converted into a feature vector. The named entity tags in the IOB2 format managed in the training data are associated with token-unit features to be input to BERT, and are used as a teacher signal for each token. By using a machine learning framework such as Tensorflow to optimize the parameters of BERT using backpropagation and stochastic gradient descent, a named entity extraction model for the target language can be learned.

[0053] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0054] The present disclosure includes the following configurations and methods. [Configuration 1] An extraction means for extracting a first character string corresponding to the truth label from the truth-labeled input document data; an acquiring means for acquiring a second character string obtained by translating the first character string extracted by the extracting means into a target language different from the main language of the input document data; an assignment means for assigning, to a second character string acquired by the acquisition means, a truth label corresponding to the truth label assigned to the first character string corresponding to the second character string; Equipped with the truth label includes a tag assigned to each token, which is a predetermined constituent unit of the first character string and the second character string, when the second string includes a plurality of tokens and the order of corresponding tokens in the first string is different from that of the second string, the assigning means assigns, to at least one of the tokens included in the second string, a tag different from the tag assigned to the token in the first string corresponding to the token; 23. An information processing apparatus comprising: [Configuration 2] a named entity replacement means for replacing a named entity belonging to a predetermined classification in the second character string with a named entity in the target language that is predetermined according to the classification; 2. The information processing device according to configuration 1, further comprising: [Configuration 3] A calculation means for calculating a score for quantitatively evaluating the reliability of the second character string as training data; an improvement means for setting, among the second character strings, a second character string having a score equal to or greater than a predetermined threshold as training data for training a learning model that extracts named entities in the target language; 3. The information processing device according to configuration 1 or 2, further comprising: [Configuration 4] the improving means acquires a feature word of the input document data, and replaces a token string included in the second character string that is similar to a token string including the feature word with a token string including the feature word; 4. The information processing device according to configuration 3. [Configuration 5] The feature word is a predetermined token string. 5. The information processing device according to configuration 4. [Configuration 6] The feature word is a token sequence in which a statistic representing the importance of each token sequence included in the input document data, the statistic being calculated based on the input document data and general document data different from the input document data, is equal to or greater than a predetermined threshold. 5. The information processing device according to configuration 4. [Configuration 7] The statistic is a TF-IDF value. 7. The information processing device according to configuration 6. [Configuration 8] the calculation means includes in the score at least one of a frequency of appearance of the feature word in the second character string, a ratio of token strings including the feature word in the second character string that are included in the input document data, and a score indicating a probability that the input document data is a predetermined document type; 8. The information processing device according to any one of configurations 4 to 7. [Configuration 9] a learning means for learning a learning model for extracting named entities using the second character string as training data; 9. The information processing device according to any one of configurations 1 to 8, further comprising: [Configuration 10] Extracting a first character string corresponding to the truth label from the truth-labeled input document data; obtaining a second character string by translating the first character string extracted in the extracting step into a target language different from the main language of the input document data; an assignment means for assigning a truth label corresponding to the truth label assigned to the first character string corresponding to the second character string to the second character string acquired by the acquiring step; Equipped with the truth label includes a tag assigned to each token, which is a predetermined constituent unit of the first character string and the second character string, the step of assigning includes, when the second string includes a plurality of tokens and the order of corresponding tokens in the first string is different from that of the second string, assigning, to at least one of the tokens included in the second string, a tag different from the tag assigned to the token in the first string corresponding to the token; 23. An information processing method comprising: [Configuration 11] A program for causing a computer to function as the information processing device according to any one of configurations 1 to 9.

Claims

1. An extraction means for extracting a first character string corresponding to the truth label from the truth-labeled input document data; an acquiring means for acquiring a second character string obtained by translating the first character string extracted by the extracting means into a target language different from the main language of the input document data; an assignment means for assigning, to a second character string acquired by the acquisition means, a truth label corresponding to the truth label assigned to the first character string corresponding to the second character string; Equipped with the truth label includes a tag assigned to each token, which is a predetermined constituent unit of the first character string and the second character string, the assigning means, when the second string includes a plurality of tokens and the order of corresponding tokens in the first string and the second string is different, assigns, to at least one of the tokens included in the second string, a tag different from the tag assigned to the token in the first string corresponding to the token; 23. An information processing apparatus comprising:

2. a named entity replacement means for replacing a named entity belonging to a predetermined classification in the second character string with a named entity in the target language that is predetermined according to the classification; 2. The information processing apparatus according to claim 1, further comprising:

3. a calculation means for calculating a score for quantitatively evaluating a reliability of the second character string as training data; an improvement means for setting, among the second character strings, a second character string having a score equal to or greater than a predetermined threshold as training data for training a learning model that extracts named entities in the target language; 2. The information processing apparatus according to claim 1, further comprising:

4. the improving means acquires a feature word of the input document data, and replaces a token string included in the second character string that is similar to a token string including the feature word with a token string including the feature word; 4. The information processing apparatus according to claim 3.

5. The feature word is a predetermined token string.

5. The information processing apparatus according to claim 4.

6. The feature word is a token sequence in which a statistic representing the importance of each token sequence included in the input document data, the statistic being calculated based on the input document data and general document data different from the input document data, is equal to or greater than a predetermined threshold.

5. The information processing apparatus according to claim 4.

7. The statistic is a TF-IDF value.

7. The information processing apparatus according to claim 6,

8. the calculation means includes in the score at least one of a frequency of appearance of the feature word in the second character string, a ratio of token strings in the second character string that include the feature word and are included in the input document data, and a score indicating a probability that the input document data is a predetermined document type.

5. The information processing apparatus according to claim 4.

9. a learning means for learning a learning model for extracting named entities using the second character string as training data; 9. The information processing apparatus according to claim 1, further comprising:

10. Extracting a first character string corresponding to the truth label from the truth-labeled input document data; obtaining a second character string by translating the first character string extracted by the extracting step into a target language different from the main language of the input document data; an assignment means for assigning a truth label corresponding to the truth label assigned to the first character string corresponding to the second character string acquired by the acquiring step; Equipped with the truth label includes a tag assigned to each token, which is a predetermined constituent unit of the first character string and the second character string, the step of assigning includes, when the second string includes a plurality of tokens and the first string and the second string include corresponding tokens in a different order, assigning, to at least one of the tokens included in the second string, a tag different from the tag assigned to the token in the first string corresponding to the token; 23. An information processing method comprising:

11. A program for causing a computer to function as the information processing device according to any one of claims 1 to 8.

Images