Information processing device, information processing method, and program

Abstract

To provide an information processing device capable of achieving both improvement in accuracy of unique expression extraction and suppression of an increase in a processing time, a control method, and a program.SOLUTION: An information processing device comprises an OCR unit which acquires text data from a document image into which a document has been read, a text conversion unit which converts the text data into a token string by decomposing the data in token units, a processing count calculation unit which calculates a processing count necessary for processing by a natural language processing model on the basis of the token string, and a block generation unit which divides the token string into blocks of units in which the token string can be processed by the natural language processing model, inputs a token to the natural language processing model per block, and estimates a unique expression. The block generation unit divides the token string into blocks so that at least a part overlaps between adjacent blocks on the basis of the calculated processing count and selects one of estimation results obtainable from each block about each token belonging to the part overlapping between the adjacent blocks.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] It relates to a technique for extracting specific expressions from documents.

Background Art

[0002] As a technique for extracting a character string (specific expression) corresponding to a predefined item from a document, NER (Named Entity Recognition) is generally known. According to this NER, for example, by predefined items "corporate name" and "expiration date", the character strings "ABC Company" and "2022 / 03 / 07" corresponding to "corporate name" and "expiration date" respectively can be extracted from the text in the document. In recent years, natural language processing models represented by Seq2Seq and Transformer, which are mainstream in natural language processing, can obtain processing results by taking as input a token sequence obtained by splitting the text in a document into units called tokens. When such a model is used for specific expression extraction, specific expressions can be efficiently estimated from the input token sequence. On the other hand, since there is an upper limit to the number of tokens that can be input to the model at one time, when inputting a long text, it was necessary to split it into multiple token sequences for input. In this regard, Patent Document 1 discloses a method of splitting a document into sections such as chapters, sections, and paragraphs and performing specific expression extraction for each section.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When obtaining a token sequence by dividing the text within a document into predetermined sections using the method described in Patent Document 1, the likelihood of the number of tokens constituting the token sequence staying within the input limit increases. However, because each section is processed independently, keywords necessary for identifying named entities may be lost, or the overall context of the text may be lost, potentially resulting in an inability to successfully extract named entities. Furthermore, the more finely the section units are defined, the more times the model needs to be input and processed, ultimately increasing the processing time.

[0005] This disclosure is made in view of the aforementioned issues and aims to achieve both improved accuracy in named entity recognition and suppression of increased processing time. [Means for solving the problem]

[0006] The information processing device relating to this disclosure is an information processing device that extracts named entities from a document using a natural language processing model, and comprises: acquisition means for acquiring text data from a document image obtained by reading the document; conversion means for processing the text data into token units and converting it into a token sequence; calculation means for calculating the number of processing steps required to process the token sequence using the natural language processing model; division means for dividing the token sequence into blocks that can be processed by the natural language processing model; and processing means for inputting each block into the natural language processing model and performing processing to estimate named entities, wherein the division means divides the token sequence into blocks such that at least a portion overlaps between adjacent blocks based on the calculated number of processing steps, and the processing means selects one of the estimation results obtained from each block for each token belonging to the overlapping portion between adjacent blocks. [Effects of the Invention]

[0007] According to this disclosure, it is possible to improve the accuracy of named entity recognition while suppressing an increase in processing time. [Brief explanation of the drawing]

[0008] [Figure 1] A diagram showing the hardware configuration of an information processing device. [Figure 2] A diagram showing an example of the functional configuration of an information processing device. [Figure 3] A diagram illustrating a specific example of extracting named entities from document images. [Figure 4] An example of a table showing the result of appending GT to the string corresponding to the token. [Figure 5] A flowchart illustrating the process of splitting a token sequence to generate input blocks. [Figure 6] A diagram illustrating how a temporary block is determined by overlapping token sequences of length T. [Figure 7] This figure shows an example of the result when a delimiter condition is applied to a temporary block. [Figure 8] A flowchart illustrating the process of extracting named entities from a set of input blocks. [Figure 9] Diagram illustrating the third modified example. [Figure 10] A flowchart showing the processing procedure for determining the estimation result related to Modification 4. [Figure 11] A diagram illustrating the effect of modified example 4. [Modes for carrying out the invention]

[0009] The embodiments for carrying out the present invention will be described below with reference to the drawings. Note that the following embodiments do not limit the invention as defined in the claims, and not all combinations of features described in the embodiments are necessarily essential to the solution of the invention. First, the hardware configuration of the information processing device shown in each embodiment will be described with reference to Figure 1.

[0010] Figure 1 is a hardware configuration diagram of the information processing device 100. In Figure 1, the CPU 101 controls various devices connected to the system bus 109. The ROM 102 stores the BIOS (Basic Input / Output System) program and the boot program. The RAM 103 is used as the main memory of the CPU 101. The external memory 104 stores the programs processed by the information processing device. The input unit 105 consists of various devices used for inputting information, such as a touch panel, keyboard, mouse, and robot controller. The display device 106 consists of an LCD monitor, projector, LED indicator, etc., and displays a user interface screen (UI screen) and calculation results according to instructions from the CPU 101. The communication interface 107 is an interface that communicates information with external devices via a network such as a LAN or the Internet, according to communication standards such as Ethernet (registered trademark), USB, and WiFi. The I / O 108 is an input / output unit that, for example, connects to a scanner (not shown) to receive scanned document images (hereinafter referred to as "document images").

[0011] [Embodiment 1] In this embodiment, if the number of tokens in the token sequence obtained from the input text exceeds the upper limit that can be input to the natural language processing model, the number of processing steps required to process all of them is determined, and the input text is divided so that the token sequences overlap within a range that does not increase the number of processing steps. In this way, the accuracy of named entity recognition is improved while suppressing the increase in processing time. Examples of pre-trained natural language processing models include BERT (Bidirectional Encoder Representations from Transformers) and XLNet. Alternatively, instead of using publicly available models as described above, a model that has been pre-trained from scratch may be used. Furthermore, the model does not necessarily have to have a transformer-based structure; any pre-trained, high-accuracy natural language processing model will suffice. For example, a model with a uniquely designed structure or a model with a structure automatically designed by AUTOML, etc., is acceptable. Hereafter, the explanation will be given using BERT as the pre-trained natural language processing model as an example.

[0012] <System Configuration> Figure 2 shows an example of the functional configuration of an information processing device 100 for extracting named entities from document images according to this embodiment. The information processing device 100 includes an OCR unit 201, a text conversion unit 202, a processing count calculation unit 203, a delimiter condition acquisition unit 204, a block generation unit 205, and a named entity processing unit 206. Each functional unit will be described below.

[0013] The OCR unit 201 obtains information (text data) of characters existing in a document by performing OCR (Optical Character Recognition) on an image area (character area) corresponding to the characters in the document image. This text data is generated by scanning, for example, from left to right and from top to bottom based on the coordinates of each character area, and sequentially combining the recognized corresponding character strings. However, the method for obtaining the text data is not limited to the method of performing OCR on the document image. For example, when text data is included separately from the image data in the electronic file of the document, it may be obtained. Alternatively, text data obtained by speech recognition from voice information may be obtained. The obtained text data is passed to the text conversion unit 202.

[0014] The text conversion unit 202 performs morphological analysis on the input text data, performs word segmentation for each obtained morpheme, decomposes it into tokens, and converts it into a token sequence which is a set of tokens. Here, a token is the minimum unit of language information input to the neural network model. Here, the unit of the token is a morpheme, but it is not limited to this. For example, as the unit of the token, a clause unit, a word unit, or a character unit may be adopted. The natural language processing model used for estimating named entities may be decomposed into tokens according to the unit of the token set during learning. The obtained token sequence is passed to the processing times calculation unit 203 and the block generation unit 205.

[0015] The processing times calculation unit 203 calculates the number of processing times based on the number of tokens constituting the token sequence received from the text conversion unit 202 and the maximum number of tokens that can be processed (input at one time) by the natural language processing model used. Specifically, assuming that the number of tokens constituting the token sequence (the length of the token sequence) is T and the maximum number of tokens that can be input at one time from the token sequence to the natural language processing model (the upper limit token number) is L, the number of processing times P is represented by the following formula (1). P = RoundUp(T / L) ··· Formula (1)

[0016] In the above formula (1), "RoundUp" is a function for rounding up the digits after the decimal point. This number of processing times P is the minimum number of times that must be processed in order to input all the token numbers T into BERT. When generating input blocks while overlapping token sequences in the block generation unit 205 described later, in order to overlap at least M token numbers, the above formula (1) may be replaced with the following formula (2). P = RoundUp((T - M) / (L - M)) ··· Formula (2)

[0017] The information on the number of processing times P calculated using the above formula (1) or the above formula (2) is passed to the block generation unit 205. Here, a specific example will be shown. Now, the number of tokens that can be input into BERT at one time is 512. When using "CLS" and "SEP", which are special tokens representing the start and end of a token sequence, the upper limit number of tokens L = 512 - 2 = 510. If the number of tokens in the input token sequence is 1000, from the above formula (1), 1000 / 510 = 1.96··, and by rounding up, the number of processing times P = 2. Also, when the number of tokens to be overlapped is M = 32, from the above formula (2), (1000 - 32) / (510 - 32) = 2.02, and by rounding up, the number of processing times P = 3.

[0018] The delimiter condition acquisition unit 204 acquires conditions (hereinafter referred to as "delimiter conditions") for dividing the token sequence into input blocks at appropriate points for understanding the context, when the block generation unit 205, described later, divides the token sequence into input blocks. Examples of delimiter conditions include "divide by paragraph," "divide by line break," "divide by period," "divide by punctuation," and "divide by token." The delimiter condition acquisition unit 204 acquires at least two or more delimiter conditions and categorizes them according to the fineness of the division. The token sequence will be divided most finely with "divide by token," which means that the token sequence can be divided anywhere. For example, suppose there is the sentence "This is an apple. That is an orange." If the hypothetical input block described later is "It is. That is an orange.", and the delimiter condition is "divide by period," then it can be divided into "It is." and "That is an orange.", and the input block can be "That is an orange.". The tiered delimiter conditions can mitigate the impact of, for example, the sudden appearance of the string "desu." at the beginning, which would make it difficult to understand the context, thereby improving the accuracy of estimation and learning. These tiered delimiter conditions (hereinafter referred to as the "condition group") are passed to the block generation unit 205.

[0019] The block generation unit 205 divides the token sequence received from the text conversion unit 202 into chunks of tokens (hereinafter referred to as "input blocks") that can be input to BERT. When estimating named entities from text data, it is difficult to know in advance where the named entities are located within the token sequence. Unless special knowledge allows us to know in advance that named entities do not exist or do not need to be estimated in a specific range within the token sequence, the entire token sequence is usually input to BERT. The block generation unit 205 first determines P provisional input blocks, each consisting of an upper limit of L tokens, based on the number of processing iterations P, allowing for overlap between blocks so that the token sequences overlap each other. These P provisional input blocks determined here will be referred to as "provisional blocks" below. Furthermore, the block generation unit 205 removes tokens from both ends of each provisional block based on the delimiter conditions included in the condition set. The input blocks obtained by removing tokens from the ends of the provisional blocks in this way are input to BERT. Details of the process of dividing the token sequence into input blocks will be described later. The data from the divided input blocks is passed to the named entity processing unit 206.

[0020] The named entity processing unit 206 consists of an estimation unit 207, a determination unit 208, and a learning unit 209, and performs the necessary processing to extract named entities from the token sequence. The following describes each component of the named entity processing unit 206.

[0021] The estimation unit 207 inputs the input block received from the block generation unit 205 into BERT and performs the process of estimating named entities from the input block. Specifically, the BERT used here is a neural network model in which a fully connected layer (proprietary layer) for multi-class classification is added to the BERT layer. Each token contained in the input block is encoded and converted into a vector in a format that can be input into BERT, and then input into the BERT layer. By further performing multi-class classification of the output of the BERT layer by a fully connected layer, a named entity tag corresponding to the input token is obtained. For example, suppose the input block is "ABC Corporation" and contains two tokens, "ABC" and "Corporation". For each token, "B-ORG", which refers to the first token of the company name, and "I-ORG", which follows, are estimated as named entity tags in IOB (Inside-Outside-Beginning) format. As a result, the company name ABC Corporation has been estimated as a named entity. Note that tags starting with "B" represent the first token of the named entity, and tags starting with "I" represent the second and subsequent tokens of the named entity. To enable solving these multi-class classification problems, the learning unit 209, described later, trains the BERT layer and the fully connected layer. This allows the estimation unit 207 to perform named entity estimation. When estimating named entity tags, a score representing likelihood is calculated for each token and each candidate named entity tag, and the named entity tag with the highest score is selected as the estimation result. The estimated IOB-format named entity tag and score information are then passed to the decision unit 208.

[0022] The determination unit 208 determines the named entity to be output based on the named entity tags and scores received from the estimation unit 207. First, for the non-overlapping range of the input blocks, the estimation unit 207 estimates only one named entity tag per token, so the result can be determined as the named entity. On the other hand, for the overlapping range of the input blocks, the estimation unit 207 estimates multiple named entity tags for each token. Therefore, it is necessary to determine one named entity tag from among the multiple named entity tags. When selecting one named entity tag from among multiple, the named entity tag with the larger score should be selected. Alternatively, the sum of the multiple scores obtained for each named entity tag can be calculated, and the named entity tag with the largest sum of scores can be selected. In this way, in the overlapping portion of the input blocks, the named entity tag with the largest score is selected to determine the value of the named entity. The determined value of the named entity is displayed, for example, in the display unit 105.

[0023] The learning unit 209 uses tokens that make up the input block to train a BERT layer and a fully connected layer as a multi-class classifier for estimation in the estimation unit 207. For this training, the input and output datasets, specifically training data in which various document images are assigned ground truth data, usually called "GT (Ground Truth)," are used. In this embodiment, each token included in the token sequence is assigned a named entity tag such as "B-ORG" as a GT. For example, if we want to estimate three named entities from the input block—a company name, a personal name, and a date—we would assign one of the seven named entity tags to each token: "B-ORG" representing the company name, followed by "I-ORG," "B-PERSON" representing the personal name, followed by "I-PERSON," "B-DATE" representing the date, followed by "I-DATE," and "O" indicating that it is not a named entity. Here, a specific example of extracting named entities from a scanned image of a quotation will be explained with reference to Figure 3. Now, considering the estimation of three types of named entities from document image 300—company name, personal name, and date—we assign the company name GT to the strings "ABC Corporation" in character area 301 and "DEF Vacuum Corporation" in character area 302. Then, we assign the personal name GT to the string "Taro Yamada" in character area 303, and the date GT to the string "October 31st" in character area 304. Figure 4 is a table showing the results of assigning GTs to the strings corresponding to tokens contained in document image 300 shown in Figure 3. Table 400 shown in Figure 4 consists of three columns: "Identifier," "Token String," and "GT," with each row representing a token. For example, the token "Estimate" in the first row and the token "Document" in the second row are tagged with "O" to indicate that they are not named entities. Also, the token "ABC" in the third row is tagged with "B-ORG," and the token "Corporation" in the fourth row is tagged with "I-ORG," indicating that "ABC Corporation" is the company name "ORG." In this way, named entity tags (GTs) are assigned to the token sequences corresponding to the text within the document. The learning unit 209 refers to the input block-based table 400, encodes each token in the "token string," vectorizes it, and inputs it into the BERT layer.Then, the multi-class classification results output from the fully connected layer are compared with the corresponding GT for each token to calculate the loss and gradient, and the weights of the BERT layer and the fully connected layer are modified. This process is repeated to update the weights of the BERT layer and the fully connected layer, thereby performing learning. The weights obtained through learning are used in the estimation process in the estimation unit 207.

[0024] In this embodiment, the named entity processing unit 206 of a single information processing device 100 contains three functions: an estimation unit 207, a determination unit 208, and a learning unit 209. However, these three functions do not necessarily have to be contained within a single device. For example, the system could be divided into two parts: a first information processing device with a relatively low-spec hardware configuration that includes both the estimation unit 207 and the determination unit 208, and a second information processing device with a relatively high-spec hardware configuration that includes the learning unit 209. In this case, the second information processing device could perform large-scale learning using a large amount of training data, while the first information processing device could perform less computationally intensive estimation, thereby providing a server-edge system at a reduced cost.

[0025] <Splitting the token sequence> Next, the process of splitting the token sequence to generate input blocks will be explained with reference to the flowchart in Figure 5. In the following explanation, the symbol "S" represents a step.

[0026] In S501, initialization processing is performed. Specifically, the following processes are carried out:

[0027] • Load the weights of the natural language processing model (hereinafter simply referred to as "model") and obtain and set the upper limit L for the input token count. • Get and set tokenization method for word segmentation. • Obtain the maximum number of tokens L that can be entered into the model at once, and the minimum number of overlapping tokens M. • Set The set of delimiter conditions is obtained, divided into stages, and output to the block generation unit 205. In S502, the OCR unit 101 acquires text data from the document image. The acquired text data is then passed to the text conversion unit 202.

[0028] In S503, the text conversion unit 202 processes the input text data, breaking it down into token units and converting it into a token sequence. The converted token sequence is then passed to the processing count calculation unit 203 and the block generation unit 205.

[0029] In S504, the processing count calculation unit 203 calculates the number of processing steps for the token sequence obtained in S503, based on the upper limit number of tokens L set in S501. The calculated number of processing steps is passed to the block generation unit 205.

[0030] In S505, the block generation unit 205 receives a token sequence of length T and the number of processing steps P, and determines P temporary blocks according to the upper limit number of tokens L. Figure 6 shows how temporary blocks are determined by overlapping the token sequence of length T when the number of processing steps P=3. Three temporary blocks 601 to 603 with partial overlap have been determined. In this case, Mg represents the range in which temporary block 601 and temporary block 602 overlap. Here, if the overlapping range is maximized under the number of processing steps P, the length of the range Mg can be calculated using the following equation (3). Mg=(P×LT) / (P-1) Equation (3)

[0031] If the number of processing steps P=3, the upper limit of tokens L=512, and the length of the token sequence T=1200, then Mg=(3×510-1200) / (3-1)=165, meaning that 165 tokens will overlap in each temporary block.

[0032] Furthermore, the start position Ps and end position Pe of the k-th temporary block relative to the first (1st) temporary block are expressed by equations (4) and (5) below, respectively. Ps=(k-1)×(L-Mg)...Equation (4) Pe=(k-1)×(L-Mg)+L...Equation (5)

[0033] Therefore, in the example in Figure 6, the first temporary block 601 extends from the starting position Ps_1, which is the position of the first token, to the ending position Pe_2. Similarly, the second temporary block 602 extends from the starting position Ps_2 to the ending position Pe_2, where the starting position Ps_2 is equal to L-Mg. Likewise, the third temporary block 603 extends from the starting position Ps_3 to the ending position Pe_3, where the starting position Ps_3 is equal to 2L-2Mg. The length of all three temporary blocks 601 to 603 is the upper limit number of tokens L. The last token in the last temporary block is placed at the end of the token sequence of length T, which allows for efficient use of the data. In this way, the block generation unit 205 first determines P temporary blocks from the length T of the token sequence and the number of processing steps P.

[0034] In the next step, S506, the block generation unit 205 generates an input block by removing tokens from both ends (or one end) of the temporary block determined in S505, according to each delimiter condition included in the set of conditions received from the delimiter condition acquisition unit 204. Let's explain this using a specific example. Figure 7 shows the results when three types of delimiter conditions are applied to two temporary blocks determined for the token sequence 700, "Today is Sunday. Is the next day Monday or Tuesday?". The first temporary block 701 is in the range "Today is Sunday. The next day is Monday", and the next temporary block 702 is in the range up to ". Is the next day Monday or Tuesday?". Here, we will explain the case where the delimiter conditions included in the received set of conditions are in three stages, becoming more detailed in the order of "delimit with a period", "delimit with a comma", and "delimit with a token".

[0035] ≪When separating with a period≫ Token sequence 710 shows the string obtained when token sequence 700 is split by a period, and " / " indicates the delimiter. As a result of trimming the end of the temporary block according to the delimiter condition "split by a period", an input block is obtained in which the range enclosed by " / " and " / " becomes the longest block. From token sequence 710 when split by a period, first, "The next day is Monday" is trimmed from the end of the first temporary block 701, and the range up to "Today is Sunday." is generated as the first input block. Furthermore, "." is trimmed from the beginning of the second temporary block 702, and the range up to "Is the next day Monday? Or is it Tuesday?" is generated as the second input block.

[0036] ≪When using punctuation marks to separate sections≫ Token sequence 720 shows the string obtained when token sequence 700 is divided by punctuation marks, and " / " indicates the delimiter. By trimming the ends of the temporary blocks according to the delimiter condition "divide by punctuation marks", an input block is obtained in which the range enclosed by " / " becomes the longest block. From token sequence 720 when divided by punctuation marks, first, "Monday" is trimmed from the end of the first temporary block 701, and the range up to "Today is Sunday. The next day is," is generated as the first input block. Furthermore, "." is trimmed from the beginning of the second temporary block 702, and the range up to "The next day is Monday. Or is it Tuesday." is generated as the second input block.

[0037] ≪When using tokens as separators≫ Token sequence 730 shows the string obtained when token sequence 700 is divided into token units, and " / " indicates the delimiter position. As a result of trimming the ends of the temporary blocks according to the delimiter condition "divide by token", an input block is obtained in which the range enclosed by " / " and " / " becomes the longest block. From token sequence 730 when divided by token, first, "de" is trimmed from the end of the first temporary block 701, and the range up to "Today is Sunday. The next day is Monday" is generated as the first input block. Furthermore, nothing is trimmed from the beginning of the second temporary block 702, and the range up to ".The next day is Monday. Or is it Tuesday." is generated as the second input block.

[0038] The block generation unit 205 starts with the coarsest delimiter condition among the multiple delimiter conditions included in the condition set, and generates input blocks by modifying the delimiter conditions and removing temporary blocks. If, as a result of removing temporary blocks (or from the beginning), the number of tokens in the input block is shorter than the upper limit number of tokens L, the tokens are packed to the beginning, and the remaining tokens that do not meet the upper limit number of tokens L are filled to the end with special tokens called padding tokens. This ensures that the number of tokens constituting the input block is equal to the upper limit number of tokens L that can be input into the model.

[0039] In S507, the block generation unit 205 checks whether there are any overlapping portions between adjacent input blocks in all the generated input blocks. If overlapping portions are found between all input blocks, it determines that the optimal input block has been generated and terminates the process. On the other hand, if there is even one place where there is no overlapping portion, the process proceeds to S508.

[0040] In S508, the block generation unit 205 changes the delimiter condition to be applied to one level finer. For example, if the set of conditions includes the three delimiter conditions mentioned above, and the currently applied delimiter condition is "separate with a period", and there are places where no overlap can be confirmed, it will be changed to "separate with a comma". After changing the delimiter condition by one level, the process returns to S506 and continues. This process is repeated until there is overlap between all input blocks. In the example in Figure 7 above, when "separate with a comma" is applied as the delimiter condition, the part "the next day," overlaps, so the process ends at this stage.

[0041] The above describes the process for generating input blocks from a token sequence according to this embodiment. This makes it possible to obtain input blocks in a format that is easy to understand in context from a token sequence of length T, while ensuring a minimum number of processing steps, and as a result contributes to improving the accuracy of named entity estimation and learning.

[0042] <Process for extracting named entities> Next, the process of extracting named entities from the input block group generated from the token sequence as described above by the named entity processing unit 206 will be explained with reference to the flowchart in Figure 8. In the following explanation, the symbol "S" represents a step. In S801, the estimation unit 207 determines the input block of interest from the received group of input blocks and inputs it into the model, estimating a named entity tag for each token. The estimation results (one or more estimated named entity tags and a score for each named entity tag) are passed to the determination unit 208.

[0043] In S802, it is determined whether estimation has been completed for all input blocks included in the received set of input blocks. If estimation processing for all input blocks is complete, the process proceeds to S803. On the other hand, if there are still unprocessed input blocks, the process returns to S801 to determine the next input block to focus on and continues the named entity estimation process. In S803, the determination unit 208 determines one named entity tag for each token based on the estimation results (estimated named entity tags and their scores) for all input blocks included in the input block group.

[0044] The above describes the named entity extraction process in the named entity processing unit 206. The named entity tag determined for each token is then displayed to the user, for example, on the display unit 106. For example, suppose the following processing result is obtained, represented as a pair of {"token string", "named entity tag"}. ·{"Estimate", "O"} ·{"Writing", "O"} ·{ABC, 'B-ORG'} I-ORG Corporation · {Yamada, 'B-PERSON'} · {Mr. / Ms., 'O'} ·{2021, 'B-DATE'} · {Year, 'I-DATE'} ·{10, 'I-DATE'} • {Month, 'I-DATE'} ·{31, 'I-DATE'} · {Japan, 'I-DATE'}

[0045] In this case, for example, the display unit 106 will display "ABC Corporation" as "ORG" representing the company name, "Yamada" as "PERSON" representing the individual name, and "October 31, 2021" as "DATE" representing the date. Note that displaying the processing result on a screen is just one example of how the output can be generated, and is not limited to this.

[0046] <Example 1> When generating input blocks from a sequence of tokens, the flowchart in Figure 5 terminated the process after removing tokens from the ends of the temporary block according to the delimiter conditions, provided there were no areas without overlap (S507). However, the process may be terminated even if there are areas without overlap if the following conditions are met.

[0047] ≪Conditions≫ Regarding the two input blocks corresponding to areas without overlap, they overlap each other at the temporary block stage. - The token immediately following the last token of the previous input block, obtained by removing tokens from the end according to the delimiter conditions, becomes the first token of the following input block, and no tokens are left unprocessed.

[0048] This allows for the generation of input blocks with a coarser level of division, making it easier to grasp the contextual coherence.

[0049] <Modification 2> In the flowchart of Figure 5, when generating input blocks from a token sequence, the number of processing steps P was fixed, and the delimiter condition was changed step by step to generate the input blocks. However, it is also possible to fix the delimiter condition and increase the number of processing steps P. Specifically, first, one delimiter condition to be fixed is decided. Next, when each temporary block is trimmed according to the delimiter condition for the number of processing steps P, if there are tokens that do not overlap between adjacent input blocks, the number of processing steps P is incremented (P+1). Then, Mg is recalculated from equation (3) above, and the calculated Mg is applied to equations (4) and (5) above to determine the range of the temporary block. Each temporary block thus determined is trimmed according to the fixed delimiter condition, and it is checked whether there are any places in all the resulting input blocks where there is no overlap. If there is even one place where there is no overlap, the number of processing steps P is incremented again and the same process is repeated. If the condition is not met, the generation of input blocks can be terminated. In this modified example, although the number of processing steps increases, it becomes easier to grasp the contextual coherence, thereby improving the accuracy of named entity estimation.

[0050] <Variation 3> In the flowchart in Figure 5, when generating input blocks from a token sequence, a temporary block was determined first, and then the tokens at the ends of the temporary block were removed according to the delimiter condition to generate the input block. However, it is also possible to first identify the positions that can be delimited by the delimiter condition, and then generate the input block under a processing count P. The specific method is as follows. First, it is assumed that there are N positions from D_1 to D_N that can be delimited by the delimiter condition. In this case, the beginning and end of the token sequence are also included among the N positions. If two positions, D_I and D_J, are selected from these N positions, then if the number of tokens between D_I and D_J is less than the upper limit number of tokens L that can be input into the model, it can be generated as an input block. All combinations of selecting two from N positions are checked to generate a group of candidate input blocks. Then, P input blocks are selected from this group of candidate input blocks, and if a combination can be selected that can process all tokens, then these are used as input blocks. On the other hand, if it is not possible to process all tokens by selecting P input blocks from the candidate group of input blocks, the same process can be repeated by changing the delimiter condition to one that allows for finer division. The effect of this modified method will be explained in comparison with the case where a temporary block is determined first, and then the delimiter condition is applied to generate the input block. In Figure 9, the token sequence 900 has a length T. D_1 to D_6 represent positions that can be divided by a certain delimiter condition. Now, the number of processing steps P = 3.

[0051] First, let's explain the comparative example. In Figure 9, the three blocks 920, 921, and 922 shown in the center represent input blocks generated by removing some of the tokens from the three temporary blocks 910, 911, and 912 shown on the left, according to the current delimiter conditions. In this case, it can be seen that the token sequence between delimiter position D_3 and delimiter position D_4 cannot be processed. As a result, processing will either have to be continued by changing the delimiter conditions to allow for finer division, or the number of processing steps P will be increased from 3 to 4.

[0052] On the other hand, in this modified example, since input blocks are generated based on the predetermined delimiter positions, the three input blocks 930, 931, and 932 shown on the right in Figure 9 can be obtained. In other words, in this modified example, input blocks can be generated without changing the delimiter conditions or the number of processing steps, and as a result, highly accurate named entity estimation with reduced processing time becomes possible.

[0053] <Modification 4> When the decision unit 208 selects one named entity estimation result for each token in the overlap portion from multiple results, it may make the decision based on whether the token belongs to the first half or the second half of the overlap portion. Specifically, for tokens belonging to the first half, the estimation result from the input block near the beginning of the token sequence is used, and for tokens belonging to the second half, the estimation result from the input block near the end is adopted. The processing flow in the decision unit 208 related to this modified example will be explained below with reference to the flowchart in Figure 10.

[0054] In S1001, overlapping portions are identified from the estimated results of named entity tags corresponding to multiple input blocks. This will be explained with reference to a specific example in Figure 11. Currently, for token sequence 1101, the estimated result of named entity tag 1102 for the first input block and the estimated result of named entity tag 1103 for the second input block have been obtained. Between the first input block and the second input block, tokens 1 to 9 overlap, and multiple estimation results have been obtained. In this case, the token sequence from token 1 to token 9 is identified as the overlapping portion.

[0055] In S1002, the token of interest in the identified overlapping portion is determined, and two estimation results are obtained for that token of interest. In this process, the tokens of interest are determined in order from the token closest to the beginning among the tokens included in the identified overlapping portion.

[0056] In S1003, the processing is allocated according to whether the two obtained estimation results satisfy a predetermined condition. Here, the predetermined condition is as follows. ≪Condition 1≫ · The two estimation results are the same. ≪Condition 2≫ · The two estimation results are tags other than the specific expression tags to be estimated (in this embodiment, "B-ORG", "I-ORG", "B-PERSON", "I-PERSON", "B-DATE", "I-DATE") (in this embodiment, "O"). ≪Condition 3≫ · There are two consecutive tokens that satisfy Condition 1 and Condition 2.

[0057] If all of the above three conditions are satisfied, the process proceeds to S1004. If not, the process returns to S1002 to determine the next token to be noted and continues the processing. For example, in the example of FIG. 11, when paying attention to token 1 of the character string "31", in the estimation result 1102 of the first input block, it is "I-DATE", and in the estimation result 1103 of the second input block, it is "O", which do not match, so Condition 1 is not satisfied. Also, since it is "I-DATE" in the estimation result 1102 of the first input block, Condition 2 is not satisfied either. Therefore, in this case, the process returns to S1002, and token 2 of the character string "day" is determined as the next token to be noted, and the processing continues. In this way, the tokens are sequentially checked until tokens that satisfy the above three conditions are found. In the example of FIG. 11, Condition 1 and Condition 2 are satisfied for token 6 of the character string "person in charge", and further, Condition 1 and Condition 2 are satisfied for token 7 of the character string "person", so Condition 3 is also satisfied. Therefore, when the processing has advanced to token 7, the process proceeds to S1004.

[0058] In S1004, the boundary is determined as follows: the first half is defined as the part up to the token that satisfies the predetermined conditions, and the second half is defined as the part after the token that satisfies the predetermined conditions. In the example in Figure 11, of the tokens 1 to 9 included in the overlapping portion, the boundary is determined to be between token 6 and token 7, with tokens 1 to 6 being the first half and tokens 7 to 9 being the second half.

[0059] In S1005, according to the determined boundary, the estimation result of the first input block is adopted for tokens belonging to the first half, and the estimation result of the second input block is adopted for tokens belonging to the second half.

[0060] The above describes the method for determining the estimation results of each token in the overlapping portion according to this modified example. Note that even after checking all the tokens included in the overlapping portion, it may not be possible to find a token that satisfies all of the above conditions 1 to 3. In that case, in S1003, the processing may be divided based on whether the remaining two conditions are met when either condition 1 or condition 2 is ignored. If no token that satisfies the conditions is found, for example, in the example in Figure 11 above, the overlapping portion may be divided into two halves, with the first half consisting of tokens up to "Co., Ltd." (total company) 5 and the second half consisting of tokens from "Person in Charge" (total person in charge) 6 onwards. Alternatively, when the entire overlapping portion is considered as 1, a ratio for determining the boundary position may be predetermined, for example, the first half may be up to 0.7 from the beginning, and the portion may be divided into two halves according to that ratio. Furthermore, condition 3 is not limited to two consecutive tokens; it may also be three consecutive tokens, or just one (i.e., only conditions 1 and 2 need to be met).

[0061] In this way, the decision of which estimation result to adopt may be made based on whether the token belongs to the first half or the second half. This allows for the adoption of consecutive estimation results within a single input block, improving estimation accuracy.

[0062] <Modification 5> When the decision unit 208 selects one of multiple named entity estimation results for each token in the overlapping portion, it may change the selection method depending on whether the system type prioritizes recall or precision in named entity estimation. A concrete example will be explained below.

[0063] ≪For systems that prioritize recall≫ This type of system provides a user interface (UI) that presents the user with multiple estimated results for named entities and allows them to select the desired entity from among them. In this type of system, if there are undetected named entities, the user will not be able to select them. Therefore, it is appropriate to return as many estimated results as possible. Accordingly, in this type of system, if the estimated results for the named entity tags of the first input block and the second input block are different in the overlapping portion, and one of them is "O", the system should choose the estimated result that is not "O". Furthermore, if both estimation results are not "O" (i.e., both estimation results are named entity tags that should be estimated), the system will select the estimation result that does not violate the IOB format based on the estimation results of the preceding and succeeding tokens. For example, consider a token of interest for which two named entity tags, "I-ORG" and "I-PERSON," have been estimated. In this case, if the estimation result of the preceding token is "B-ORG," then following with "I-PERSON" would be a violation, so "I-ORG" will be selected. If neither estimation result results in a violation, the estimation result with the higher score will be selected. By doing so, user value can be increased.

[0064] ≪For systems that prioritize accuracy≫ This type of system is configured to automatically process other tasks using the results of named entity estimation. In this type of system, if false positives occur, other tasks will also be processed, so a high precision is desirable. An example of such a system is one in which, if the corporate name "XX Limited Company" is extracted as a result of named entity estimation, the system deletes the email corresponding to the string acquired by the OCR unit 101. In this precision-prioritizing type of system, if the estimation results of two named entity tags differ in the overlapping portion, or if the score of the estimated named entity tag is lower than a threshold, the tag is determined to be "O". In this way, in a precision-prioritizing type of system, the determination unit 208 can increase user value by determining the estimation results in a way that suppresses false positives.

[0065] Furthermore, the user may be able to choose whether to prioritize recall or precision, and the selection method in the decision unit 208 may be switched according to the selection result.

[0066] <<Differentiation Example 6>> The above embodiment describes an example where the OCR unit 101 processes all tokens acquired, but it is not limited to this. For example, in the flowchart of Figure 5, after converting to a token sequence (S503) and before calculating the number of processing steps (S504), an additional process may be performed to delete tokens that are not necessary for named entity estimation. In this modified example, the token sequence from which unnecessary tokens have been deleted will be passed to the processing step calculation unit 203. The method for detecting unnecessary tokens from the token sequence is not limited. For example, if the target is text data containing both Japanese and English, and named entities are to be estimated only from the Japanese text, then English strings are unnecessary. In this case, English tokens can be detected and deleted. Also, if named entities are not to be estimated from strings in the header or footer of a document, the OCR unit 101 may delete strings belonging to the header or footer based on information about the position and font size of the strings. Specifically, when acquiring text data from a document image, information about the position and font size within the document is also acquired for each string. Based on the acquired position and size information, for example, tokens whose position is above a predetermined location in the document (e.g., 10% from the top) and whose font size is less than or equal to a predetermined size are determined to belong to the header and are deleted. Similarly, tokens whose position is below a predetermined location in the document (e.g., 10% from the bottom) and whose font size is less than or equal to a predetermined size are determined to belong to the footer and are deleted. This reduces the number of tokens that make up the token sequence to be processed, thereby reducing the number of processing steps and shortening processing time, or increasing the overlap portion with the same number of processing steps, thereby improving the accuracy of named entity recognition estimation.

[0067] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0068] Furthermore, the disclosure of this embodiment includes the following configurations and methods.

[0069] (Composition 1) An information processing device that extracts named entities from documents using a natural language processing model, An acquisition means for obtaining text data from a document image obtained by reading the aforementioned document, A conversion means that processes the text data into token units and converts it into a token sequence, A calculation means for calculating the number of processing steps required to process the token sequence using the natural language processing model, A partitioning means for dividing the token sequence into blocks that can be processed by the natural language processing model, Processing means for inputting each of the aforementioned blocks into the natural language processing model and performing a process to estimate named entities, It has, The division means divides the token sequence into blocks based on the calculated number of processing steps, such that at least a portion of adjacent blocks overlap. The processing means selects one of the estimation results obtained from each block for each token belonging to the overlapping portion between adjacent blocks. An information processing device characterized by the following:

[0070] (Configuration 2) The information processing apparatus according to configuration 1, characterized in that the calculation means calculates the minimum number of processing steps required to process all tokens included in the token sequence.

[0071] (Composition 3) The information processing apparatus according to configuration 2, wherein the division means divides the token sequence into the same number of blocks as the calculated minimum number of processing steps.

[0072] (Composition 4) The system further includes a condition acquisition means for acquiring a delimiter condition for dividing the token sequence into predetermined units, The division means divides the token sequence into blocks based on the delimiter condition. An information processing device according to any one of configurations 1 to 3.

[0073] (Composition 5) The condition acquisition means acquires a plurality of delimiter conditions with different levels of granularity when delimiting the token sequence, The division means divides the token sequence into blocks by sequentially changing the delimiting condition applied from among the plurality of delimiting conditions so as not to increase the calculated number of processing steps. The information processing apparatus according to configuration 4, characterized by the features described above.

[0074] (Composition 6) The aforementioned dividing means is For the aforementioned token sequence, a number of temporary blocks equal to the number of processing iterations is determined according to the upper limit number of tokens in the block. The block is generated by removing the tokens at the end of the temporary block according to the delimiter conditions. The information processing apparatus according to configuration 5, characterized by the features described herein.

[0075] (Composition 7) The information processing apparatus according to configuration 6, characterized in that the division means applies the division conditions in order from the coarsest of the plurality of division conditions, and repeats the process until there are no overlapping areas between all adjacent blocks after removing the tokens at the edges of the temporary blocks.

[0076] (Composition 8) The information processing apparatus according to configuration 7, characterized in that the division means applies the division conditions in order from the coarsest to the coarsest, and even if there is a portion of no overlap between any adjacent blocks after the token at the end of the temporary block has been removed, with respect to the two blocks corresponding to the portion of no overlap, if they overlap with each other at the temporary block stage, and the token following the last token of the front block obtained by removing the end token is the first token of the back block, the processing is terminated at that point.

[0077] (Composition 9) The aforementioned delimiter conditions include two or more of the following: delimiting by paragraph, delimiting by line break, delimiting by period, delimiting by punctuation, delimiting by token. The aforementioned sequence of tokens can be roughly divided in the following order: by paragraph, by line break, by period, by punctuation, and by token. An information processing apparatus according to any one of configurations 5 to 8, characterized by the above.

[0078] (Composition 10) The processing means divides the overlapping portion into a first half and a second half, determines the estimation result of the block near the beginning as the named entity for tokens belonging to the first half, and determines the estimation result of the block near the end as the named entity for tokens belonging to the second half. This is the information processing device according to any one of configurations 1 to 9.

[0079] (Composition 11) The information processing apparatus according to configuration 10, characterized in that the processing means divides the overlapping portion into the first half and the second half according to predetermined conditions for determining the boundary between the first half and the second half.

[0080] (Composition 12) The processing means, when prioritizing recall in named entity estimation, selects the estimation result that indicates a token is a named entity if one of the estimation results for a token included in the overlapping portion indicates that it is not a named entity. This is the information processing apparatus according to any one of configurations 1 to 9.

[0081] (Composition 13) The processing means, when prioritizing precision in named entity estimation, determines an estimation result indicating that the tokens included in the overlapping portion are not named entities if the estimation results differ, as described in any one of configurations 1 to 9.

[0082] (Composition 14) The converted token sequence further includes a deletion means for deleting unnecessary tokens, The information processing apparatus according to any one of configurations 1 to 13, characterized in that the calculation means calculates the number of processing steps for the token sequence from which the unnecessary tokens have been removed.

[0083] (Composition 15) The estimation results include a score representing the likelihood of a named entity tag corresponding to a token. The processing means is characterized in that it selects one of the estimation results based on the score for tokens belonging to the overlapping portion of the block, as described in any one of configurations 1 to 9.

[0084] (Method 1) A method for controlling an information processing device that extracts named entities from documents using a natural language processing model, The acquisition step involves obtaining text data from a document image obtained by reading the aforementioned document, A transformation step involves processing the aforementioned text data into token units and converting it into a token sequence, A calculation step to calculate the number of processing steps required to process the token sequence using the natural language processing model, A partitioning step of dividing the token sequence into blocks that can be processed by the natural language processing model, A processing step which involves inputting each of the aforementioned blocks into the natural language processing model and performing a process to estimate named entities, Includes, In the division step, based on the calculated number of processing steps, the token sequence is divided into blocks such that at least a portion of adjacent blocks overlap. In the processing step, for each token belonging to the overlapping portion between adjacent blocks, one of the estimation results obtained from each block is selected. A control method characterized by the following:

[0085] (Composition 16) A program for causing a computer to function as an information processing device as described in any one of items 1 to 15 of the configuration.

Claims

1. An information processing device that extracts named entities from documents using a natural language processing model, An acquisition means for obtaining text data from a document image obtained by reading the aforementioned document, A conversion means that processes the text data into token units and converts it into a token sequence, A calculation means for calculating the number of processing steps required to process the token sequence using the natural language processing model, A partitioning means for dividing the token sequence into blocks that can be processed by the natural language processing model, Processing means for inputting each of the aforementioned blocks into the natural language processing model and performing a process to estimate named entities, It has, The division means divides the token sequence into blocks based on the calculated number of processing steps, such that at least a portion of adjacent blocks overlap. The processing means selects one of the estimation results obtained from each block for each token belonging to the overlapping portion between adjacent blocks. An information processing device characterized by the following:

2. The information processing apparatus according to claim 1, characterized in that the calculation means calculates the minimum number of processing steps required to process all tokens included in the token sequence.

3. The information processing apparatus according to claim 2, characterized in that the division means divides the token sequence into the same number of blocks as the calculated minimum number of processing steps.

4. The system further includes a condition acquisition means for acquiring a delimiter condition for dividing the token sequence into predetermined units, The division means divides the token sequence into blocks based on the delimiter condition. The information processing apparatus according to feature 1.

5. The condition acquisition means acquires a plurality of delimiter conditions with different levels of granularity when delimiting the token sequence, The division means divides the token sequence into blocks by sequentially changing the delimiting condition applied from among the plurality of delimiting conditions so as not to increase the calculated number of processing steps. The information processing apparatus according to feature 4.

6. The aforementioned dividing means is For the aforementioned token sequence, a number of temporary blocks equal to the number of processing iterations is determined according to the upper limit number of tokens in the block. The block is generated by removing the tokens at the end of the temporary block according to the delimiter conditions. The information processing apparatus according to feature 5.

7. The information processing apparatus according to claim 6, characterized in that the division means applies the division conditions in order from the coarsest of the plurality of division conditions, and repeats the process until there are no overlapping areas between all adjacent blocks after removing the tokens at the edges of the temporary blocks.

8. The information processing apparatus according to claim 7, characterized in that the division means applies the division conditions in order from the coarsest to the coarsest, and even if there is a portion of no overlap between any adjacent blocks after the token at the end of the temporary block has been removed, with respect to the two blocks corresponding to the portion of no overlap, if they overlap with each other at the temporary block stage, and the token following the last token of the front block obtained by removing the end token is the first token of the back block, the processing is terminated at that point.

9. The aforementioned delimiter conditions include two or more of the following: delimiting by paragraph, delimiting by line break, delimiting by period, delimiting by punctuation, delimiting by token. The aforementioned sequence of tokens can be roughly divided in the following order: by paragraph, by line break, by period, by punctuation, and by token. The information processing apparatus according to feature 5.

10. The information processing apparatus according to claim 1, characterized in that the processing means divides the overlapping portion into a first half and a second half, determines the estimation result of the block near the beginning as the named entity for tokens belonging to the first half, and determines the estimation result of the block near the end as the named entity for tokens belonging to the second half.

11. The information processing apparatus according to claim 10, characterized in that the processing means divides the overlapping portion into the first half and the second half according to predetermined conditions for determining the boundary between the first half and the second half.

12. The information processing apparatus according to claim 1, characterized in that, when prioritizing recall in named entity estimation, the processing means selects the estimation result that indicates a token is a named entity if one of the estimation results for a token included in the overlapping portion indicates that it is not a named entity.

13. The information processing apparatus according to claim 1, characterized in that, when prioritizing precision in named entity estimation, if the estimation results differ for tokens included in the overlapping portion, the processing means determines an estimation result indicating that it is not a named entity.

14. The converted token sequence further includes a deletion means for deleting unnecessary tokens, The information processing apparatus according to claim 1, characterized in that the calculation means calculates the number of processing steps for the token sequence from which the unnecessary tokens have been removed.

15. The estimation results include a score representing the likelihood of a named entity tag corresponding to a token. The information processing apparatus according to claim 1, characterized in that the processing means selects one of the estimation results based on the score for tokens belonging to the overlapping portion of the block.

16. A method for controlling an information processing device that extracts named entities from documents using a natural language processing model, The acquisition step involves obtaining text data from a document image obtained by reading the aforementioned document, A transformation step involves processing the aforementioned text data into token units and converting it into a token sequence, A calculation step to calculate the number of processing steps required to process the token sequence using the natural language processing model, A partitioning step of dividing the token sequence into blocks that can be processed by the natural language processing model, A processing step which involves inputting each of the aforementioned blocks into the natural language processing model and performing a process to estimate named entities, Includes, In the division step, based on the calculated number of processing steps, the token sequence is divided into blocks such that at least a portion of adjacent blocks overlap. In the processing step, for each token belonging to the overlapping portion between adjacent blocks, one of the estimation results obtained from each block is selected. A control method characterized by the following:

17. A program for causing a computer to execute the control method described in claim 16.

Images