Sentence extraction device and sentence extraction method

Abstract

To provide a sentence extraction device that extracts a sentence related to a figure or a table in a wider range.SOLUTION: A sentence extraction device comprises a processor and extracts sentences related to a figure or a table from a text from a digital document including the figure or the table and the text. The figure or the table has a label, a caption and a body. The processor has extraction units 202 to 205 that extract from the text a sentence including a second word associated with a first word included in the body of the figure or the table.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a sentence extraction device and a sentence extraction method. [Background technology]

[0002] With the explosive growth of unstructured data, the number of use cases for data analysis to extract useful knowledge from huge amounts of unstructured data is increasing. In particular, digital documents, which are a typical source of unstructured data, contain numerous charts and graphs, and valuable information can be extracted by extracting and analyzing data from these charts and graphs.

[0003] One example of such a use case is supporting the analysis of academic documents. When investigating academic documents such as patents and papers, it is necessary to efficiently understand the figures and tables contained in a large number of documents, but there are limits to human information processing capabilities. Therefore, by automatically extracting figures and tables from documents and presenting them to the investigator, the efficiency of the investigation can be improved.

[0004] However, charts and graphs alone have limitations in their ability to express information and can only analyze limited information, so in addition to the charts and graphs themselves, it is necessary to extract explanatory text for the charts and graphs from the document text and present it together with the charts and graphs.

[0005] As a technology related to this technology, for example, there is a diagram / table caption extraction device disclosed in Patent Document 1. The diagram / table caption extraction device disclosed in Patent Document 1 breaks down the document text into sentences, compares each sentence with the labels and captions of the diagrams, and selects sentences with similar word structures as the diagram / table caption. For example, a sentence that contains many words that are found in the labels and captions of diagrams, such as "Figure 1" or "Table 1," is likely to be suitable as a diagram / table caption. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2003-346161 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, the device disclosed in Patent Document 1 has two problems.

[0008] The first problem is that it is not possible to extract sentences that refer to information inside a diagram or table. For example, sentences that refer to part names in a design drawing or performance values ​​in a product specification sheet cannot be extracted by comparing them with the labels and captions of the diagram or table.

[0009] The second problem is that it is difficult to extract sentences that are important as explanatory texts, even if they do not contain words directly related to the figures and tables. For example, sentences that refer to the contents of figures and tables using pronouns such as "that" and "this" cannot be extracted by focusing only on the origins of words.

[0010] Due to these issues, the device disclosed in Patent Document 1 can only extract a limited range of explanatory text, which ultimately has a negative impact on data analysis.

[0011] The present invention has been made in view of the above problems, and an object of the present invention is to provide a sentence extraction device and a sentence extraction method capable of extracting sentences related to figures or tables from a wider range. [Means for solving the problem]

[0012] In order to solve the above problem, a sentence extraction device according to one aspect of the present invention is a sentence extraction device that has a processor and extracts sentences related to a figure or table from a digital document including the figure or table and a main text, wherein the figure or table has a label, a caption and a main text, and the processor has an extraction unit that extracts a sentence from the main text including a second word related to a first word included in the main text of the figure or table. Effect of the Invention

[0013] According to the present invention, it is possible to realize a sentence extraction device and a sentence extraction method capable of extracting sentences related to figures or tables from a wider range. [Brief description of the drawings]

[0014] [Figure 1] FIG. 2 is a diagram illustrating a hardware configuration of the sentence extraction device according to the embodiment. [Diagram 2] FIG. 2 is a diagram showing a logical configuration of a data extraction program in a memory of the sentence extraction device according to the embodiment. [Diagram 3] 4 is a flowchart showing the overall process of the sentence extraction device according to the embodiment. [Figure 4] 4 is a flowchart showing a diagram data extraction process of the sentence extraction device according to the embodiment. [Diagram 5] FIG. 11 is a diagram illustrating a diagram data extraction process of the sentence extraction device according to the embodiment. [Figure 6] 11 is a flowchart showing a table data extraction process of the sentence extraction device according to the embodiment. [Figure 7] FIG. 11 is a diagram illustrating a table data extraction process of the sentence extraction device according to the embodiment. [Figure 8] 10 is a flowchart showing a diagram label / caption line determination process of the sentence extraction device according to the embodiment. [Figure 9] FIG. 13 is a diagram showing a diagram label / caption line determination process of the sentence extraction device according to the embodiment. [Figure 10] FIG. 13 is a diagram showing a diagram label / caption line determination process of the sentence extraction device according to the embodiment. [Figure 11] 11 is a flowchart showing a main text paragraph division process of the sentence extraction device according to the embodiment. [Figure 12] 4 is a flowchart showing a word table (A) creation process of the sentence extraction device according to the embodiment. [Figure 13] FIG. 2 is a diagram showing a word table (A) creation process of the sentence extraction device according to the embodiment. [Figure 14] 13 is a flowchart showing a word table (B) creation process of the sentence extraction device according to the embodiment. [Figure 15] FIG. 11 is a diagram showing a word table (B) creation process of the sentence extraction device according to the embodiment. [Figure 16] 11 is a flowchart showing a score calculation process of the sentence extraction device according to the embodiment. [Figure 17] FIG. 11 is a diagram illustrating a score calculation process of the sentence extraction device according to the embodiment. [Figure 18] FIG. 11 is a diagram illustrating a score calculation process of the sentence extraction device according to the embodiment. [Figure 19] 11 is a flowchart showing an extraction target sentence determination process of the sentence extraction device according to the embodiment. [Figure 20] FIG. 4 is a diagram illustrating an extraction target sentence determination process of the sentence extraction device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the embodiments described below do not limit the invention according to the claims, and all of the elements and combinations thereof described in the embodiments are not necessarily essential to the solution of the invention.

[0016] In the drawings explaining the embodiments, the same reference numerals are used to designate parts having the same functions, and repeated explanations thereof will be omitted.

[0017] In the following description, an expression such as "xxx data" may be used as an example of information, but the information may have any data structure. In other words, to show that the information does not depend on the data structure, "xxx data" may be referred to as "xxx table." Furthermore, "xxx data" may simply be referred to as "xxx." In the following description, the structure of each piece of information is an example, and the information may be divided and stored, or combined and stored.

[0018] In the following explanation, the process may be described with a "program" as the subject, but since a program is executed by a processor (e.g., a CPU (Central Processing Unit)) to perform a defined process using storage resources (e.g., a memory) and / or a communication interface device (e.g., a port) as appropriate, the subject of the process may also be the program. Processes explained with a program as the subject may also be processes performed by a processor or a computer having that processor.

[0019] The sentence extraction device of this embodiment may have the following configuration, for example.

[0020] That is, when extracting explanatory text for figures and tables from the document body, the sentence extraction device of this embodiment solves this problem by comparing each sentence not only with the label and caption of the figure and table, but also with the words contained in the figure and table body. In addition, for example, when extracting explanatory text for a table, a sentence in which words from multiple cells of the table appear in combination, such as a combination of units in the table header row (such as "℃" or "km") and values ​​in the table data row, is particularly likely to be suitable as an explanatory text, so such sentences are given priority in being selected as explanatory text.

[0021] In addition, the sentence extraction device of this embodiment solves this problem by extracting not only sentences that contain many words related to the figure / table body, label, and caption, but also sentences in the same paragraph as these sentences as explanatory sentences. This is because sentences that do not contain many words related to figures / tables but are important as explanatory sentences for figures / tables are often included in the same paragraph as sentences that contain many words related to figures / tables. However, if the paragraph is long, extracting the entire paragraph may contain unnecessary information that is not explanatory. Therefore, if the paragraph length is equal to or longer than a certain level, only sentences between sentences that contain many words related to figures / tables in the paragraph are extracted as explanatory sentences.

[0022] In this specification, "figure" means a figure and / or table. A figure label is the figure or table number of a figure, for example, "Figure 1." A figure caption is a sentence that explains the contents of a figure or table, and is often written in the area following the figure label. The body of a figure or table is the area where the contents of the figure or table are written, excluding the figure label and caption. The text is the area of ​​a digital document excluding figures and tables, and this text contains the sentences and paragraphs of the digital document.

[0023] An overview of the sentence extraction device according to the embodiment will be described below. Fig. 1 shows the hardware configuration of the sentence extraction device according to the embodiment.

[0024] The sentence extraction device 100 executes a data extraction program (see FIG. 2) and extracts figure / table legend sentences from document files.

[0025] The sentence extraction device 100 is a device capable of various information processing, for example, an information processing device such as a computer. The information processing device has a calculation element (CPU 102 in FIG. 1) constituting a control unit, a storage medium (memory 103 and disk device 104 in FIG. 1) constituting a data storage unit, and further has an input unit such as a communication interface, a mouse, a keyboard, and a screen unit such as a display, as necessary.

[0026] The arithmetic element is, for example, a central processing unit (CPU) or a field-programmable gate array (FPGA). The storage medium is, for example, a magnetic storage medium such as a hard disk drive (HDD), a random access memory (RAM), a read only memory (ROM), or a solid state drive (SSD). In addition, a combination of an optical disk such as a digital versatile disk (DVD) and an optical disk drive is also used as a storage medium. Other known storage media such as magnetic tape media are also used as storage media.

[0027] The storage medium stores programs such as firmware. When the sentence extraction device 100 starts operating (for example, when the power is turned on), the programs such as firmware are read from the storage medium and executed to control the entire sentence extraction device 100. In addition to the programs, the storage medium also stores data necessary for each process of the sentence extraction device 100.

[0028] The sentence extraction device 100 of this embodiment may be configured as a so-called cloud in which a plurality of information processing devices are configured to be able to communicate with each other via a communication network.

[0029] In FIG. 1, a CPU 102 is connected to a memory 103 serving as a main storage device via a bus 101 , and executes a data extraction program on the memory 103 .

[0030] The CPU 102 and memory 103 are connected to a disk device 104, which is a secondary storage device, via a bus 101, and a data extraction program executed on the CPU 102 can read document files stored in the disk device 104 onto the memory 103 and write data to the disk device 104.

[0031] Fig. 2 shows the configuration of a data extraction program 200 stored in the memory 103 of the sentence extraction device 100 according to the embodiment, and Fig. 3 shows an overall processing flow 300 of the data extraction program 200. The overall processing flow 300 consists of the following six steps.

[0032] First, the document file reading unit 201 reads a document file stored in the disk device 104, and develops it as document data in the memory 103 (process 301).

[0033] Secondly, the graphic data extraction unit 202 analyzes the document data read by the document file reading unit 201 and detects graphic elements 421. In addition, the graphic data extraction unit 202 extracts graphic labels 422, graphic captions 423, and text within the graphic elements 421 corresponding to the detected graphic elements 421 as graphic data 431 (process 400).

[0034] Third, the table data extraction unit 203 analyzes the document data read by the document file reading unit 201, and detects the table element 521. In addition, the table data extraction unit 203 extracts the table label 522, the table caption 523, and the text within the table corresponding to the detected table element 521 as table data 531 (process 500).

[0035] Fourth, the main text extraction unit 204 analyzes the document data read by the document file reading unit 201, and extracts the main text 611. In addition, the main text 611 is divided into paragraphs by analyzing the coordinate information of the main text 611 (process 600).

[0036] Fifth, the figure / table legend extraction unit 205 extracts the legend of the figure element 421 extracted by the figure data extraction unit 202 or the legend of the table element 521 extracted by the table data extraction unit 203 from the main text 611 extracted by the main text extraction unit 204 (process 302).

[0037] Sixth, the data storage unit 206 stores the diagram data 431 extracted by the diagram data extraction unit 202, the table data 531 extracted by the table data extraction unit 203, and the diagram legends extracted by the diagram legend extraction unit 205 in the disk device 104 (process 303).

[0038] Fig. 5 shows process 400 in which the figure data extraction unit 202 in the data extraction program 200 according to the embodiment extracts figure labels 422, figure captions 423 and text within figures from document data as figure data 431. Fig. 6 shows an example of the operation of extracting figure data 431 in process 400. The steps of process 400 will be explained below in order.

[0039] First, the graphic data extraction unit 202 reads the document data expanded on the memory 103, and draws a layout image 420 of the document on the memory 103 (process 401). The layout image 420 may be drawn in a raster format or a vector format depending on the implementation of the subsequent process 402.

[0040] Secondly, the graphic data extraction unit 202 detects graphic elements 421 from the document layout image 420 drawn on the memory 103 (process 402). Here, a graphic element refers to a rectangular area surrounding the main body of a graphic, excluding labels and captions. Process 402 can be implemented, for example, by inputting the layout image 420 drawn in raster format to a convolutional neural network, or by clustering drawing instructions in the layout image 420 drawn in vector format.

[0041] Thirdly, the graphic data extraction unit 202 uses the coordinate information of the graphic element 421 detected in process 402 to extract the graphic label 422 and graphic caption 423 from the document data expanded in the memory 103 (process 600).

[0042] Fourth, the graphic data extraction unit 202 extracts text within the graphic from the document data expanded on the memory 103 or the document layout image 420 drawn on the memory 103 (processes 403 to 407). Here, the text within the graphic refers to text contained within the area of ​​the graphic element 421. If the graphic element 421 is embedded in the document in raster format, the text within the graphic is extracted by applying OCR (optical character recognition) to the graphic element 421 extracted as a raster image (processes 404 to 405). On the other hand, if the graphic element 421 is embedded in the document in vector format, the text within the graphic is extracted by extracting all drawing instructions related to the text from the graphic element 421 extracted as a vector image (processes 406 to 407).

[0043] Fifth, the figure data extraction unit 202 stores the extracted figure label 422, figure caption 423 and in-figure text in the memory 103 as figure data 431 (process 408). The figure data 431 is accessed as a record of a table consisting of three columns: a first column 432 for storing figure labels, a second column 433 for storing figure captions, and a third column 434 for storing in-figure text. Columns 432 to 434 are all in plain text format.

[0044] Fig. 6 shows processing 500 in which the table data extraction unit 203 in the data extraction program 200 according to the embodiment extracts table label 522, table caption 523, and text within the table from document data as table data 531. Fig. 7 shows an example of the operation of extracting table data 531 in processing 500. The steps of processing 500 will be described below in order.

[0045] First, the table data extraction unit 203 reads the document data expanded on the memory 103, and draws the layout image 420 of the document on the memory 103 (process 501).

[0046] Secondly, the table data extraction unit 203 detects table elements 521 from the document layout image 420 drawn on the memory 103 (process 502). Here, the table element refers to a rectangular area of ​​the table body excluding the label and caption. Process 502 can be implemented, for example, by inputting the layout image 420 drawn in raster format to a convolutional neural network, or by clustering drawing instructions in the layout image 420 drawn in vector format.

[0047] Thirdly, the table data extraction unit 203 extracts the table label 522 and the table caption 523 from the document data expanded in the memory 103 based on the coordinate information of the table element 521 detected in the process 502 (process 600). The process 600 is common to the figure data extraction unit 202.

[0048] Fourth, the table data extraction unit 203 extracts text within the table from the document data expanded on the memory 103 or the document layout image 420 drawn on the memory 103 (processes 503 to 507). If the table element 521 is embedded in the document in raster format, the text within the table is extracted by applying OCR (optical character recognition) to the table element 521 extracted as a raster image (processes 504 to 505).

[0049] On the other hand, if table element 521 is embedded in the document in vector format, the text within the table is extracted by extracting all drawing instructions related to the text from table element 521 extracted as a vector image (steps 506 to 507). In either case, not only is the text within the table obtained as plain text, but also coordinate information indicating the position within the page of each word contained in the text is obtained.

[0050] Fifth, table data extraction unit 203 divides table element 521 in the row and column directions, and determines in which row and column of table element 521 each word or phrase contained in the table text extracted by processes 503 to 507 is contained. This process is performed by comparing the coordinate range of each row and column obtained by dividing table element 521 with the coordinates of each word or phrase in the table text (process 508).

[0051] Sixth, the table data extraction unit 203 stores the extracted table label 522, table caption 523, and table text in the memory 103 as table data 531 (process 509). The table data 531 is accessed as a record of a table consisting of three columns: a first column 532 for storing the table label, a second column 533 for storing the table caption, and a third column 534 for storing the table text.

[0052] The first column 532 and the second column 533 are in plain text format, but the data in the third column 534 is stored in a table format in which an index indicating the row and column position in the table is assigned to each word in the text in the table. The index is not limited to a single integer value, and for example, for a word that spans multiple rows or columns, it is also possible to express all row and column positions containing the word as an integer range value.

[0053] 8 shows a common process 600 when the figure data extraction unit 202 extracts a figure label 422 and a figure caption 423, or when the table data extraction unit 203 extracts a table label 522 and a table caption 523. The process 600 is executed for each text line 614 of the main text 611 extracted by the main text extraction unit 204.

[0054] First, the graphic data extraction units 202 and 203 obtain the coordinate information of the target text line 614 (process 601).

[0055] An example of the coordinate information acquisition operation in process 601 is shown in Fig. 9. The coordinate information consists of, for example, a set of upper left corner coordinate 612 and lower right corner coordinate 613 of a text line 614 on document layout image 420. The coordinate information of each text line 614 is accessed as a record in a table consisting of three columns: a first column 621 that stores text, a second column 622 that stores the upper left corner coordinate, and a third column 623 that stores the lower right corner coordinate.

[0056] Secondly, the graphic data extraction units 202 and 203 acquire coordinate information of the graphic elements 421 and table elements 521 from which labels and captions are to be extracted (step 602). This coordinate information is, for example, a set of coordinates of the upper left corner and the lower right corner of the graphic elements 421 and table elements 521 on the document layout image 420.

[0057] Thirdly, the graphic data extraction units 202 and 203 calculate the distance (process 603) between the coordinate information of the text line 614 obtained in process 602 and the coordinate information of the graphic element 421 and table element 521 obtained in process 603. This distance is calculated, for example, by the method shown in FIG.

[0058] In FIG. 10, rectangle A 631 and rectangle B 632 correspond to the coordinate information of a graphic element and a text line, respectively. Rectangles A and B are both rectangular with four sides parallel to the coordinate axes, and do not overlap each other. L1 (633) is the length of overlap when rectangle A 631 and rectangle B 632 are projected onto one of the coordinate axes. L2 (634) is the distance between rectangle A 631 and rectangle B 632 in the direction perpendicular to L1 (633). In this case, the distance D between rectangles A and B is defined as

number

[0059] Fourth, the distance between the coordinate information calculated in process 603 is compared with a predefined threshold (process 604). If the distance is closer than the threshold, it is determined that the text line 614 is close to the graphic element. If the distance is farther than the threshold, it is determined that the text line 614 is unrelated to the graphic element, and the process proceeds to the next text line.

[0060] The text lines determined to be adjacent to the graphic element in process 604 are compared with a predefined text pattern (process 605). The text pattern can be implemented, for example, using a regular expression. A regular expression is a finite automaton that determines whether a symbol string matches a specific pattern, expressed as text containing special symbols.

[0061] An example of a text pattern using regular expressions is one that matches a text line beginning with a figure / table label, such as "^Table [0-90-9]+\..+$" or "^Table [0-9]+\..+$". A text line that matches the text pattern is determined to contain a figure / table label / caption, and is output as the figure / table label / caption (process 606).

[0062] 11 shows a process 700 for dividing the main text 611 extracted from the document data into paragraphs by the main text extraction unit 204. The process 700 is performed for each text line 614 in the main text 611.

[0063] First, the main text extraction unit 204 obtains coordinate information of the current text line and the immediately preceding text line in the main text (process 701).

[0064] Second, the body text extractor 204 calculates the distance between the current text line and the previous text line, for example, using the method shown in FIG. 10 (step 702).

[0065] If the distance between the current text line and the previous text line exceeds the threshold, the current text line and the previous text line are determined to belong to different paragraphs, and a new paragraph is started from the current text line (step 708). Otherwise, the current text line and the previous text line are determined to be adjacent, and it is then determined whether the two lines belong to the same paragraph (steps 704 to 707, 709 to 711).

[0066] In this determination process, first, the main text extraction unit 204 compares the coordinates of the left and right ends of the current text line and the immediately preceding text line. In order to allow for document layout errors during the comparison, the coordinates may be determined to be equal if the difference in coordinates is equal to or less than a certain small value. If the coordinates of the left and right edges are equal, it is determined that the current text line and the previous text line belong to the same paragraph, and the current text line is added to the end of the same paragraph as the previous text line (process 712).

[0067] If the coordinates of the left and right edges are not equal, it is determined that the current text line and the previous text line do not belong to the same paragraph, and a new paragraph is started from the current text line (process 708).

[0068] If only the right coordinates are equal but the left coordinates are different, the current text line is added to the end of the same paragraph as the previous text line only if the previous text line was the first line of a paragraph. This is to ensure that the determination is correct even if the first line of a paragraph is indented.

[0069] If only the left edge coordinates are equal and the right edge coordinates are different, the current text line is added to the end of the same paragraph as the previous text line only if the right edge coordinates of the text line before the previous one are equal to those of the previous text line. This is a measure to ensure that the judgment is correct even if the last line of the paragraph does not reach the right edge of the column.

[0070] The process 700 is applied from the first text line to the last text line in the document to divide the body text of the document into paragraphs.

[0071] FIG. 12 shows the process performed by the figure / table explanation extraction unit 205 when creating a word list (A) 812 for each sentence in the main text 611.

[0072] First, the diagram / table explanation extraction unit 205 acquires the text of the sentence for which the word table is to be created (process 801), and divides it into words (process 802). For example, in the case of Japanese, division into words is performed using morphological analysis software (e.g., mecab) or the like.

[0073] Secondly, the figure / table caption extraction unit 205 judges whether each word divided in process 802 is an important word, and extracts only the important words (process 803). Regarding the judgment of important words, for example, implementation that regards nouns, verbs, adjectives, and adverbs as important words, or implementation that regards words in a predefined dictionary as important words can be considered.

[0074] Thirdly, the diagram / table explanation extraction unit 205 creates a word table (A) 812 from the important words extracted in the process 803 (process 804).

[0075] An example of creating a word table (A) 812 from a sentence 811 in the main text is shown in Fig. 13. The word table (A) 812 consists of three columns: a first column 813 that stores representative forms, a second column 814 that stores parts of speech, and a third column 815 that stores types.

[0076] The first column 813 stores a character string that can uniquely identify a word. For example, for Japanese verbs and adjectives, a specific conjugated form or stem may be stored in the first column. The second column 814 stores the part of speech classification of the word output by, for example, morphological analysis software. The third column 815 stores an ID for identifying words that have special meanings, such as numbers or units. Whether a word has a special meaning is determined by matching the word with a predefined dictionary or text pattern.

[0077] Fourth, the diagram / table explanation extraction unit 205 removes duplicate rows from the word table (A) 812 so that the same row does not appear more than once (process 805). Fifth, the diagram / explanation extracting unit 205 stores the word list (A) 812 in the memory 103 (process 806).

[0078] FIG. 14 shows a process 900 in which the figure / table caption extractor 205 creates a word list (B) 912 corresponding to text in a figure / table.

[0079] First, the figure / table legend extraction unit 205 acquires text within a figure / table from the figure data 431 extracted by the figure data extraction unit 202 or the table data 531 extracted by the table data extraction unit 203 (step 901).

[0080] Secondly, the figure / table caption extraction unit 205 divides the text within the figure / table extracted in process 901 into words (process 902).

[0081] Thirdly, the figure / table caption extraction unit 205 extracts important words from among the words in the text within the figure / table obtained in process 902 (process 903).

[0082] Fourth, the diagram / table explanation extraction unit 205 creates a word list (B) 912 from the important words extracted in process 903 (process 904).

[0083] An example of word table (B) 912 is shown in Fig. 15. Like word table (A) 812, word table (B) 912 consists of three columns, namely, first column 913 for storing the representative form of each important word, second column 914 for storing the part of speech, and third column 915 for storing the type, as well as a fourth column 916 for storing the row number indicating the position of each word in the table, and a fifth column 917 for storing the column number, for a total of five columns.

[0084] For text extracted from table data 531, the line number of each word is stored in the fourth column, and the column number of each word is stored in the fifth column. For text extracted from figure data 431, the fourth and fifth columns are set to NaN. NaN is a value that is not equal when compared with any value that may be stored in the fourth and fifth columns.

[0085] Fifth, the diagram / table explanation extraction unit 205 removes duplicate rows from the word table (B) 912 created in process 904 (process 905).

[0086] Sixth, the diagram / caption extraction unit 205 stores the word table (B) 912 from which the duplicated words have been removed in process 905 in the memory 103 (process 906).

[0087] 16 shows process 1000 when the figure / table caption extraction unit 205 calculates the score for each sentence in the main text 611. Process 1000 is executed as many times as the number of figure / table elements in the document for each sentence in the main text 611. Each sentence can be obtained by, for example, dividing the main text 611 at a period or a punctuation mark, but division into sentences with higher accuracy may also be performed using natural language processing software.

[0088] First, the figure / table explanation extraction unit 205 extracts sentences to be used for score calculation from the main text 611 (step 1001).

[0089] Secondly, the diagram / table explanation extraction unit 205 creates a word list (A) 812 of the sentences to be used for score calculation (process 800).

[0090] Thirdly, the diagram / table explanation extraction unit 205 creates a word list (B) 912 of diagram elements that are the subject of score calculation (process 900).

[0091] Fourth, the diagram / explanation extraction unit 205 creates a word table (C) 1031 from the word table (A) 812 and the word table (B) 912 (process 1002).

[0092] An example of word table (C) 1031 is shown in Fig. 17. Word table (C) 1031 includes five columns: a first column 1032 for storing representative forms of words, a second column 1033 for storing parts of speech, a third column 1034 for storing types, a fourth column 1035 for storing row numbers, and a fifth column 1036 for storing column numbers.

[0093] Word table (C) 1031 is obtained by extracting, from among the rows of word table (B) 912, those rows which match at least one row of word table (A) 812 in the first columns 813·913.

[0094] Fifth, the figure / table explanation extraction unit 205 calculates the score of each word in the word table (C) 1031 (process 1003). This process is performed by referring to a score multiplication table 1041 defined in advance.

[0095] The score of each word in the word table (C) 1031 is initialized to 1 point, and for words that match the score multiplier table 1041, the score is multiplied by the multiplier determined in the score multiplier table 1041.

[0096] FIG. 18 shows an example of score calculation. The word table (C) in FIG. 18 contains the words "137" and "m." These match the combination of the first row (header row) of the table and the second and subsequent rows (data rows) of the table, and also the combination of types "number" and "unit." When these conditions are referenced in score multiplier table 1041, a value of 2.5 times is listed as the score multiplier, so the scores for both "137" and "m" are set to 2.5 points.

[0097] Sixth, the figure / table explanation extraction unit 205 outputs the sum of the scores of each word calculated in process 1003 as the score of the sentence (process 1004).

[0098] 19 shows a process 1100 when the graphic caption extractor 205 selects a caption for a particular graphic element in the body text 611. The process 1100 is performed for each graphic element in the document as many times as there are paragraphs in the body text 611.

[0099] First, the figure / table caption extraction unit 205 compares the length of the target paragraph with a predefined threshold (step 1101). The length of the paragraph is measured, for example, in terms of the number of characters or words.

[0100] If the length of the paragraph is equal to or greater than the threshold, all sentences in the paragraph whose scores are equal to or greater than the threshold are listed (process 1106). All sentences listed in process 1106 are selected as explanatory sentences (process 1107). Furthermore, sentences located between sentences listed in process 1106 are also selected as explanatory sentences (process 1108). If no sentences are selected in process 1106, the entire paragraph is deemed not to be explanatory sentences. Finally, the selected sentences are rearranged in the order in which they appear in the document and output (process 1105).

[0101] An example of processes 1106 to 1108 when the length of the paragraph is equal to or greater than the threshold is shown in Fig. 20. In Fig. 20, process 1100 is applied to paragraph 1122 in main text 611. Of sentences 1123 to 1126 in the paragraph, sentences 1123 and 1125 have scores equal to or greater than the predefined threshold of 70.0. In this case, in addition to sentences 1123 and 1125, sentence 1124 sandwiched between them is also selected as an explanatory sentence.

[0102] If the length of the paragraph is less than the threshold, the maximum score of the sentences in the paragraph is calculated (process 1102), and the maximum score is compared with a predefined threshold (process 1103). If the maximum score exceeds the threshold, the entire paragraph is selected as explanatory text (process 1104). If all sentences in the paragraph do not exceed the threshold, the entire paragraph is considered not to be explanatory text. Finally, the selected sentences are rearranged in the order in which they appear in the document and output (process 1105).

[0103] Therefore, according to this embodiment, it is possible to realize a sentence extraction device and a sentence extraction method capable of extracting sentences related to figures or tables from a wider range. Furthermore, the extraction of a wider range of explanatory texts facilitates data analysis using diagrams in digital documents, resulting in more valuable information being extracted through analysis.

[0104] In addition, the above-mentioned embodiments are described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Also, some of the configurations of each embodiment can be added to, deleted from, or replaced with other configurations.

[0105] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor included in the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium storing it constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, ROMs, etc.

[0106] Furthermore, the program code for realizing the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C / C++, perl, Shell, PHP, Java (registered trademark), and the like.

[0107] Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read out and execute the program code stored in the storage means or storage medium.

[0108] In the above-mentioned embodiment, the control lines and information lines are shown as those considered necessary for the explanation, and not all the control lines and information lines are shown in the product. All the components may be connected to each other. [Explanation of symbols]

[0109] 100... sentence extraction device 200... data extraction program 201... document file reading unit 202... figure data extraction unit 203... table data extraction unit 204... main text extraction unit 205... figure / table description extraction unit 206... data storage unit

Claims

1. A sentence extraction device comprising a processor, the device extracting sentences related to a figure or a table from a digital document including the figure or the table and a body text, the device comprising: The figure or table has a label, a caption and a body; The processor has an extraction unit that calculates a degree of relevance between the first word and a sentence containing the second word based on a score table that defines a numerical value corresponding to a combination of a position where a first word included in the body of the figure or table is written in the body of the figure or table and a position where a second word included in the text is written in the body of the figure or table, and extracts from the text the sentence whose degree of relevance exceeds a predetermined first threshold.

2. 2. The sentence extraction device according to claim 1, wherein the extraction unit extracts the sentence including the second word that matches the first word from the main text.

3. The sentence extraction device according to claim 1 , wherein the extraction unit sets, as the second word, a word that is a specific part of speech or a word that is registered in advance.

4. 2. The sentence extraction device according to claim 1, wherein the extraction unit calculates the relevance based on the type of the first word and the score table.

5. A sentence extraction device for extracting sentences related to a figure or a table from a digital document including the figure or the table and a body text, the device comprising: a score table that defines a numerical value according to a combination of a position in the figure or the body of the table where a third word included in the label, caption or body of the figure or the table is written and a position in the figure or the body of the table where a second word included in the body of the text is written, the numerical value being used to select a paragraph including a second word related to the third word, and to extract the sentence from the body of the text based on whether the number of words included in the paragraph including the second word exceeds a predetermined second threshold.

6. A sentence extraction device for extracting sentences related to a figure or a table from a digital document including the figure or the table and a body text, the device comprising: a score table that defines a numerical value according to a combination of a position where a third word included in a label, caption, or body of the figure or table is written in the body of the figure or table, and a position where a second word included in the body of the figure or table is written in the body of the figure or table, and an extraction unit that uses the numerical value in the score table to calculate a relevance between the third word and the sentence included in a paragraph including the second word, and extracts from the body of the text the sentence whose relevance exceeds a predetermined first threshold.

7. A sentence extraction device for extracting sentences related to a figure or a table from a digital document including the figure or the table and a body text, the device comprising: an extraction unit that calculates a degree of association between the third word and the sentence included in a paragraph including the second word based on a score table that defines a numerical value corresponding to a combination of a position where a third word included in a label, caption, or body of the figure or table is written in the body of the figure or table and a position where a second word included in the body of the figure or table is written in the body of the figure or table, and extracts the sentence whose degree of association exceeds a predetermined first threshold value from the body of the text, The sentence extraction device is characterized in that, when the number of words contained in the paragraph exceeds a predetermined second threshold, the extraction unit extracts only the sentence that is sandwiched between multiple sentences whose relevance exceeds the first threshold.

8. 8. The sentence extraction device according to claim 6, wherein the extraction unit extracts the sentence including the second word that matches the third word from the main text.

9. 8. The sentence extraction device according to claim 6, wherein the extraction unit sets a word of a specific part of speech or a word registered in advance as the second word.

10. 8. The sentence extraction device according to claim 6, wherein the extraction unit calculates the degree of association based on the type of the third word and the score table.

11. A sentence extraction method for extracting sentences related to a figure or a table from a digital document including the figure or the table and a body text by a sentence extraction device, comprising: The processor of the sentence extraction device A sentence extraction method characterized by: calculating a degree of relevance between a sentence containing the first word and the second word, using a score table that defines a numerical value corresponding to a combination of a position where a first word included in the body of the figure or table is written in the body of the figure or table and a position where a second word included in the main text is written in the body of the figure or table, based on the numerical value in the score table; and extracting from the main text the sentence whose degree of relevance exceeds a predetermined first threshold.

Images