Information Processing Apparatus, Information Processing Method, and Program

Abstract

This information processing device generates headings in a structured document. An acquisition means acquires a structured document that includes headings and text. A characteristic word extraction means extracts characteristic words from the subelements of the headings included in the structured document. On the basis of the extracted characteristic words, a heading generation means generates new headings associated with subelements.

Description

Technical Field

[0001] The present invention relates to a technique for assigning headings to structured documents.

Background Art

[0002] In a website, a system that outputs search results in response to an input such as a user's keyword in a search engine or the like, or a system that answers a user's inquiry sentence (query) such as a so-called chatbot is known. Such a system refers to structured documents on the web related to the input keyword or query and generates search results or answers. Patent Document 1 describes a method of structuring documents according to their uses. Patent Document 2 describes a method of determining the implicative relationship between headings and text included in structured documents using machine learning.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In order to generate appropriate search results or answers for a user's input, it is required that appropriate headings be assigned to the structured document. However, for example, when headings are assigned by referring to tag information from a structured document such as HTML, the headings may become mere numbers or symbols indicating order, or may have the same content as other headings, resulting in insufficient heading information.

[0005] One object of the present invention is to provide an information processing apparatus capable of generating an appropriate heading based on subheadings and text in a structured document. **Means for Solving the Problems**

[0006] In one aspect of the present invention, an information processing apparatus includes an acquisition means for acquiring a structured document including headings and text, an inappropriate heading detection means for detecting inappropriate headings from the headings included in the structured document, a characteristic word extraction means for extracting characteristic words from the subordinate elements of each of the inappropriate headings included in the structured document, among the extracted characteristic words, characteristic words that have no matching or implicative relationship as , corresponding to the characteristic term a new heading for the inappropriate heading and determine that heading generation means, and is provided with 、 the inappropriate heading is a heading with the same character string as another heading in a parallel relationship in the structured document, or a heading that is in a parallel relationship in the structured document and is composed of numbers or symbols and has no meaning or content .

[0007] In another aspect of the present invention, an information processing method executed by a computer acquires a structured document including headings and text, detects inappropriate headings from the headings included in the structured document, extracts characteristic words from the subordinate elements of each of the inappropriate headings included in the structured document, among the extracted characteristic words, characteristic words that have no matching or implicative relationship as , corresponding to the characteristic term a new heading for the inappropriate heading and determine that the inappropriate heading is a heading with the same character string as another heading in a parallel relationship in the structured document, or a heading that is in a parallel relationship in the structured document and is composed of numbers or symbols and has no meaning or content .

[0008] In still another aspect of the present invention, a program acquires a structured document including headings and text, detects inappropriate headings from the headings included in the structured document, For each inappropriate heading included in the structured document, extracting characteristic words from subordinate elements of the inappropriate heading; Among the extracted feature words, those that are not in a matching or entailing relationship as , corresponding to the characteristic term A new headline for the bad one and determine The computer executes the process 、 the inappropriate heading is a heading with the same character string as another heading in a parallel relationship in the structured document, or a heading that is in a parallel relationship in the structured document and is composed of numbers or symbols and has no meaning or content . [Effects of the Invention]

[0009] According to the present invention, it is possible to generate an appropriate heading based on the sub-headings and text in a structured document. [Brief explanation of the drawings]

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

[0011] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. First Embodiment [Overall configuration] FIG. 1 shows the overall configuration of a heading generation device according to the first embodiment. The heading generation device 100 outputs a heading-complemented document in which an appropriate heading is assigned to an input document. If the input document has already been structured, the heading generation device 100 determines whether the headings included in the structured document are appropriate, and outputs a heading-complemented document in which headings determined to be inappropriate have been corrected. On the other hand, if the input document is not structured, the heading generation device 100 first structures the input document, then corrects inappropriate headings and outputs a heading-complemented document.

[0012] [Structured Document] A structured document is a document in which the structure of the document is marked up, and typical examples include XML (eXtensible Markup Language) and HTML (Hyper Text Markup Language). In XML and HTML documents, the structure of the document is expressed using character strings called tags.

[0013] Figure 2 shows an example of the hierarchical structure of a structured document. This document explains the term "vacation" and includes headings 2, 2a, and 2b, as well as text 3a and 3c. Heading 2 is the top-level (first-level) heading, and headings 2a and 2b are lower-level (second-level) headings. Text 3a and 3b correspond to headings 2a and 2b, respectively. In this structured document, both headings 2a and 2b are "annual vacation" and contain the same character string. Therefore, when this structured document is used for searching or browsing, it is possible that the correct search results or answers will not be output in response to a user's input regarding "annual vacation." Thus, if the character string of a heading in a structured document is identical to another heading in a parallel relationship, the heading cannot be distinguished, and the heading is considered inappropriate. Furthermore, even if the character strings of the headings are not identical, they can also be considered inappropriate if the character strings of the headings are similar or have an implication relationship.

[0014] Additionally, if the character strings of parallel headings in a structured document do not have sufficient semantic content, the headings are also inappropriate. For example, if the character strings of headings are only numbers or symbols, such as "1.", "2.", "(a)", "(b)", or if they simply indicate the order of sections, such as "Chapter 1" and "Chapter 2", the headings are also inappropriate if they do not have any specific semantic content.

[0015] In this way, if the heading of a structured document is inappropriate, the output for user search or browsing may be inappropriate. Therefore, the heading generation device 100 detects inappropriate headings in structured documents and corrects them to appropriate headings.

[0016] [Summary of how headings are generated] Figure 3 shows another example of a structured document. This example is also a structured document related to the term "vacation" and is composed of a hierarchical structure of multiple headings 2 and text 3. Note that for the sake of simplicity, some headings and text are omitted from Figure 3.

[0017] If a heading included in a structured document is inappropriate, the heading generation device 100 generates a new heading in place of the inappropriate heading (hereinafter also referred to as an "inappropriate heading"). Specifically, the heading generation device 100 generates a new heading to replace the inappropriate heading based on the subelements of the inappropriate heading. Here, the "subelements" include at least one of a heading (subheading) 2 and text 3 that exist in a lower hierarchy than the inappropriate heading.

[0018] FIG. 4 shows a case where two parallel headings Xa and Xb in the structured document shown in FIG. 3 are inappropriate. In this example, the two headings Xa and Xb are both "Vacation Types," and the two headings are inappropriate because the character strings are the same. In this case, the heading generation device 100 generates new headings in place of the inappropriate headings Xa and Xb. Specifically, the heading generation device 100 generates a new heading to replace the inappropriate heading Xa based on a subordinate element 4a (i.e., document A) of the inappropriate heading Xa. The heading generation device 100 also generates a new heading to replace the inappropriate heading Xb based on a subordinate element 4b (i.e., document B) of the inappropriate heading Xb.

[0019] Specifically, the headline generation device 100 generates new headlines to replace the inappropriate headings Xa and Xb by using the subelements 4a and 4b of the inappropriate headings Xa and Xb so that the new headlines are different from each other. In other words, the headline generation device 100 generates new headlines so that the new headlines to replace the inappropriate headings Xa and Xb do not fall under the category of inappropriate headings.

[0020] [Hardware configuration] 5 is a block diagram showing the hardware configuration of the headline generating device 100. As shown in the figure, the headline generating device 100 includes an interface (IF) 11, a processor 12, a memory 13, a recording medium 14, and a database (DB) 15.

[0021] IF11 performs data input / output with an external device. Specifically, the document to be subjected to the heading generation process is input through IF11. Further, the heading-complemented document with inappropriate headings corrected by the heading generation device 100 is output to the external device through IF11.

[0022] The processor 12 is a computer such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and controls the entire heading generation device 100 by executing a pre-prepared program. Specifically, the processor 12 executes the heading generation process described later.

[0023] The memory 13 is composed of a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 13 is also used as a working memory during the execution of various processes by the processor 12.

[0024] The recording medium 14 is a non-volatile and non-temporary recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the heading generation device 100. The recording medium 14 stores various programs executed by the processor 12. When the heading generation device 100 executes various processes, the programs recorded on the recording medium 14 are loaded into the memory 13 and executed by the processor 12.

[0025] The database 15 temporarily stores the document input through IF11. Note that the heading generation device 100 may include an input unit such as a keyboard and a mouse for the user to give instructions and input, and a display unit such as a liquid crystal display.

[0026] [Functional Configuration] FIG. 6 is a block diagram showing the functional configuration of the heading generation device 100. The heading generation device 100 includes a document input unit 21, a structuring unit 22, an inappropriate heading detection unit 23, a characteristic word extraction unit 24, a heading generation unit 25, and a document output unit 26.

[0027] A document (hereinafter also referred to as the "target document") for which headings are to be generated is input to the document input unit 21. When the target document input to the document input unit 21 is a structured document, that is, a document that has already been structured, the document input unit 21 outputs the document to the inappropriate heading detection unit 23. On the other hand, when the target document is an unstructured document (non-structured document), the document input unit 21 outputs the input document to the structuring unit 22 and receives the structured target document from the structuring unit 22. Then, the document input unit 21 outputs the structured target document to the inappropriate heading detection unit 23 and the document output unit 26.

[0028] The structuring unit 22 structures the input unstructured document and outputs it to the document input unit 21 as a structured document. The structuring unit 22 performs, for example, a process of extracting and tagging a character string corresponding to a heading in the input unstructured document, generates a structured document, and outputs it to the document input unit 21.

[0029] The inappropriate heading detection unit 23 identifies portions in the input target document where heading generation is necessary. Specifically, the heading generation unit 25 extracts headings among the headings included in the target document that correspond to the aforementioned inappropriate headings. Then, the inappropriate heading detection unit 23 outputs the subordinate elements of the inappropriate headings to the keyword extraction unit 24. In the example of FIG. 4, the inappropriate heading detection unit 23 detects inappropriate headings Xa and Xb and outputs their subordinate elements 4a and 4b, that is, documents A and B, to the keyword extraction unit 24. Also, the inappropriate heading detection unit 23 outputs information indicating the positions of the inappropriate headings in the target document to the document output unit 26. In the example of FIG. 4, the inappropriate heading detection unit 23 outputs information indicating the positions of the inappropriate headings Xa and Xb to the document output unit 26.

[0030] The feature word extraction unit 24 extracts feature words from the sub-elements of the inappropriate heading. A "feature word" is a word that characterizes the content of a document corresponding to the sub-element. In the example of FIG. 4, the feature word extraction unit 24 extracts feature words from document A corresponding to sub-element 4a, and extracts feature words from document B corresponding to sub-element 4b. The feature word extraction unit 24 outputs multiple feature words for each document along with their feature scores. The feature score indicates the degree to which the feature word is distinctive, and the larger the feature score, the more distinctive the feature word is in that document. For example, the feature word extraction unit 24 outputs a feature word list ranked by feature score for each of documents A and B to the heading generation unit 25.

[0031] The headline generation unit 25 generates new headlines corresponding to the inappropriate headlines using the characteristic words input from the characteristic word extraction unit 24. At this time, the headline generation unit 25 generates the new headlines so that the new headlines are different from each other. In the example of FIG. 4, the headline generation unit 25 generates new headlines corresponding to the inappropriate headlines Xa and Xb so that they are differentiated from each other. Then, the headline generation unit 25 outputs the generated new headlines to the document output unit 26.

[0032] The document output unit 26 obtains information indicating the position of the inappropriate heading from the inappropriate heading detection unit 23, and also obtains the new heading generated by the heading generation unit 25. Then, the document output unit 26 corrects the inappropriate heading in the target document using the new heading, and outputs the document as a heading-complemented document.

[0033] Specifically, in the example of Figure 4, assume that a new heading "Annual Leave" is generated for the inappropriate heading Xa, and a new heading "Other Leave" is generated for the inappropriate heading Xb. In this case, as a first method for correcting the inappropriate headings, the document output unit 26 replaces the inappropriate headings with the new headings. That is, the document output unit 26 replaces the inappropriate heading Xa "Type of Leave" with the new heading "Annual Leave" and replaces the inappropriate heading Xb "Type of Leave" with the new heading "Other Leave."

[0034] Also, as a second method for correcting an inappropriate heading, the document output unit 26 adds a new heading to the inappropriate heading. In the above case, the document output unit 26 corrects the inappropriate heading Xa "Types of Leave" to a new heading "Types of Leave (Annual Leave)", and corrects the inappropriate heading Xb "Types of Leave" to a new heading "Types of Leave (Other Leave)".

[0035] In this way, the heading generation device 100 can correct the inappropriate headings included in the target document and output it as a document with headings completed. Also, according to the heading generation device 100, even when the target document is not structured, after structuring the target document by the structuring unit 22, appropriate headings can be assigned.

[0036] [Configuration Example of Feature Word Extraction Unit] Next, a configuration example of the feature word extraction unit 24 will be described. (First Example) The feature word extraction unit 24x according to the first example extracts feature words based on the frequency of appearance of words in a document corresponding to a lower-level element. FIG. 7 shows a configuration when extracting feature words using the TF-IDF (Term Frequency-Inverse Document Frequency) method as a representative method. The feature word extraction unit 24x includes a word segmentation unit 31 and a TF-IDF calculation unit 32. The word segmentation unit 31 performs morphological analysis on a document corresponding to a lower-level element of an inappropriate heading and divides it into units of words.

[0037] The TF-IDF calculation unit 32 calculates a TF-IDF value for each word. The TF-IDF value is an index obtained by multiplying the TF value and the IDF value. The TF value indicates "the number of times a certain word appears in a document", and the IDF value is "the reciprocal of the number of documents containing a certain word divided by the total number of documents". The TF-IDF value is given by the following formula. TF-IDF( w,d ) = TF w,d × log(Nd / DF w、d )

[0038] In the above formula, "TF w,d" is the number of times the word w appears in a document d, and "DF w、d " is the number of documents containing the word w, and "Nd" indicates the total number of documents. A large TF-IDF value means that a particular word appears frequently in a document and that word does not appear very often in other documents. The larger the TF-IDF value, the more characteristic the word is in the document. Therefore, the feature word extraction unit 24x calculates the TF-IDF value of each word contained in the document corresponding to the lower-level element, and extracts a predetermined number of words as feature words from the ones with larger values. In this case, the TF-IDF value is used as the feature score of the feature word.

[0039] (Second example) The feature word extraction unit 24y according to the second example reduces the dimension of the words appearing in the document using principal component analysis (hereinafter referred to as "PCA") and extracts feature words. FIG. 8(A) shows the configuration of the feature word extraction unit 24y using PCA. The feature word extraction unit 24y includes a word segmentation unit 41, a matrix creation unit 42, and a PCA unit 43. The word segmentation unit 41 morphologically analyzes the document input as the lower-level element of the input inappropriate heading and divides it into units of words.

[0040] The matrix creation unit 42 creates a document × appearance word matrix. The document × appearance word matrix shows the appearance frequency of each word in that document for each document. FIG. 8(B) shows an example of the matrix. In this example, the matrix shows the appearance frequency of each word in the documents A and B corresponding to the aforementioned lower-level elements 4a and 4b in those documents. The matrix creation unit 42 outputs the created matrix to the PCA unit 43.

[0041] Based on the input matrix, the PCA unit 43 performs principal component analysis on the words appearing in each document and extracts a predetermined number of principal components. For example, when reducing the matrix of document × appearing words to n dimensions, the PCA unit 43 extracts the first to n principal components of the words appearing in the document and outputs n words corresponding thereto as feature words. At this time, the contribution rate of each principal component is used as the feature score of each feature word. In this way, the PCA unit 43 outputs a predetermined number of feature words for each document in descending order of the feature scores.

[0042] (The third example) The feature word extraction unit 24z according to the third example reduces the dimension of the words appearing in the document using auto encoding and extracts feature words. FIG. 9 shows the configuration of the feature word extraction unit 24z using auto encoding. The feature word extraction unit 24z includes a word segmentation unit 51, an encoder 52, an auto encoder 53, and a decoder 54. The word segmentation unit 51 morphologically analyzes the document input as a subordinate element of the input inappropriate heading and divides it into units of words.

[0043] The encoder 52 represents the words appearing in each document as vectors of a predetermined dimension using a word distribution representation or a word embedding method. As an example of the word distribution representation or the word embedding, for example, Word2vec, Doc2vec, BERT (Bidirectional Endocer Representation from Transformers), fastText, etc. can be used. Note that instead of the method using the pre-trained model as described above, each document may be vectorized using a simple model such as Bag of Words.

[0044] The auto encoder 53 reduces the dimension of the vectorized words for each document and generates a vector after dimension reduction. The decoder 54 returns the vector after dimension reduction to words using the above-described word distribution representation or word embedding method. In this way, the feature word extraction unit 24z outputs a predetermined number of feature words together with their feature scores.

[0045] In the above configuration, the document input unit 21 is an example of acquisition means, the structuring unit 22 is an example of structuring means, the inappropriate heading detection unit 23 is an example of inappropriate heading detection means, the characteristic word extraction unit 24 is an example of characteristic word extraction means, the heading generation unit 25 is an example of heading generation means, and the document output unit 26 is an example of document correction means. Further, the matrix creation unit 42 is an example of matrix creation means, and the PCA unit 43 is an example of an analysis unit.

[0046] [Heading Generation Process] FIG. 10 is a flowchart of the heading generation process by the heading generation device 100. This process is realized by the processor 12 shown in FIG. 5 executing a program prepared in advance and operating as each element shown in FIG. 6.

[0047] First, the document input unit 21 acquires the target document (step S11) and determines whether the target document is structured (step S12). If the input target document is structured (step S12: Yes), the document input unit 21 outputs the target document to the inappropriate heading detection unit 23. On the other hand, if the input target document is not structured (step S12: No), the document input unit 21 outputs the target document to the structuring unit 22, and the structuring unit 22 structures the target document (step S13). Then, the structuring unit 22 outputs the structured target document to the document input unit 21, and the document input unit 21 outputs the input target document to the inappropriate heading detection unit 23.

[0048] The inappropriate heading detection unit 23 determines whether the input target document contains an inappropriate heading (step S14). Specifically, the inappropriate heading detection unit 23 determines whether a plurality of headings in a parallel relationship in the target document are inappropriate. If the target document does not contain an inappropriate heading (step S14: No), the process ends.

[0049] On the other hand, when the target document contains inappropriate headings (Step S14: Yes), the feature word extraction unit 24 extracts feature words for each document corresponding to the subordinate elements of the inappropriate heading and creates a feature word list (Step S15). The feature word extraction unit 24 extracts feature words using any of the above-described first to third methods. The feature word list is a list in which a plurality of extracted feature words for each document are sorted in descending order of the feature score. In the example of FIG. 4, the feature word extraction unit 24 creates feature word lists La and Lb for documents A and B corresponding to the subordinate elements 4a and 4b of the inappropriate headings Xa and Xb, respectively, and outputs them to the heading generation unit 25.

[0050] The heading generation unit 25 generates a new heading corresponding to the inappropriate heading using the feature word list input from the feature word extraction unit 24. At this time, the heading generation unit 25 generates a new heading so that the new headings are differentiated from each other. Specifically, the heading generation unit 25 acquires the first elements KYa and KYb of the feature word lists La and Lb (Step S16) and determines whether there are remaining elements in each of the feature word lists La and Lb (Step S17). If there are no remaining elements in either of the feature word lists La and Lb (Step S17: No), the process proceeds to Step S20.

[0051] On the other hand, if there are remaining elements in either of the feature word lists La and Lb (Step S17: Yes), the heading generation unit 25 determines whether the element KYa and the element KYb are identical or in an implicative relationship (Step S18). If the element KYa and the element KYb are not identical or in an implicative relationship (Step S18: No), the feature words corresponding to the element KYa and the element KYb are different and do not correspond to the inappropriate heading, so the process proceeds to Step S20.

[0052] On the one hand, when element KYa and element KYb are identical or in an implicative relationship (step S18: Yes), the heading generation unit 25 changes elements KYa and KYb to the next elements in the characteristic word lists La and Lb (step S19). Here, the next element is the element with the next highest characteristic score in the characteristic word list. At this time, instead of changing both elements KYa and KYb to the next elements simultaneously, the heading generation unit 25 may change only one of elements KYa and KYb to the next element. Then, the process returns to step S17, and the heading generation unit 25 performs the processes of steps S17 to S18 for the changed elements KYa and KYb. And when there are no remaining elements in the characteristic word lists La and Lb (step S17: No), or when element KYa and element KYb are no longer in an identical or implicative relationship (step S18: No), the process proceeds to step S20.

[0053] In step S20, the heading generation unit 25 determines elements KYa and KYb at that time as new headings. In this way, characteristic words that are not in an identical or implicative relationship are determined as new headings. Next, the document output unit 26 corrects the inappropriate headings in the target document using the new headings and outputs a document with headings complemented (step S21). Note that, as described above, as a method for correcting inappropriate headings, the inappropriate headings may be replaced with new headings, or new headings may be appended to the inappropriate headings. Then, the heading generation process ends.

[0054] As described above, the heading generation unit 25 acquires elements KYa and KYb from the characteristic word lists La and Lb in descending order of characteristic scores, and determines elements KYa and KYb that do not correspond to inappropriate headings as new headings. Thereby, the characteristics of documents A and B are fully represented, and characteristic words that are not in inappropriate relationships such as identity or implicative relationships are obtained as new headings.

[0055] (Modification example) In the above flowchart, in steps S17 to S19, when the first element of the keyword list is in a matching or implicative relationship, a process of changing to the next element is performed. However, when the keyword extraction unit 24 uses PCA as in the second example and when it uses an autoencoder as in the third example, since the keywords extracted for basically different documents are differentiated, steps S17 to S19 may be omitted. In this case, the first elements KYa and KYb of each keyword list obtained in step S16 may be adopted as new headings respectively.

[0056] <Second Embodiment> Next, a second embodiment of the present invention will be described. FIG. 11 is a block diagram showing the functional configuration of an information processing apparatus according to the second embodiment. The information processing apparatus 70 includes an acquisition unit 71, a keyword extraction unit 72, and a heading generation unit 73. The acquisition unit 71 acquires a structured document including a heading and text. The keyword extraction unit 72 extracts keywords from the subordinate elements of the heading included in the structured document. The heading generation unit 73 generates a new heading corresponding to the subordinate element based on the extracted keywords.

[0057] FIG. 12 is a flowchart of the heading generation process in the second embodiment. First, the acquisition unit 71 acquires a structured document including a heading and text (step S31). The keyword extraction unit 72 extracts keywords from the subordinate elements of the heading included in the structured document (step S32). The heading generation unit 73 generates a new heading corresponding to the subordinate element based on the extracted keywords (step S33).

[0058] According to the information processing apparatus 70 of the second embodiment, since keywords are extracted from the subordinate elements of the heading included in the structured document and a new heading is generated, an appropriate heading can be generated for the target document.

[0059] Some or all of the above embodiments may be described as follows in the following supplementary notes, but are not limited thereto.

[0060] (Appendix 1) An acquisition means for acquiring a structured document including a heading and text, A keyword extraction means for extracting keywords from subordinate elements of the heading included in the structured document, A heading generation means for generating a new heading corresponding to the subordinate element based on the extracted keywords, An information processing apparatus comprising the above.

[0061] (Appendix 2) The information processing apparatus according to Appendix 1, wherein the subordinate element includes at least one of a sub-heading existing below the heading in the structured document and text existing below the heading.

[0062] (Appendix 3) The information processing apparatus according to Appendix 1 or 2, wherein the keyword extraction means extracts keywords using TF-IDF values.

[0063] (Appendix 4) The keyword extraction means includes A matrix creation means for generating a matrix of the appearance frequencies of words appearing in each document included in the subordinate element, An analysis unit for reducing the dimension of the words that appear using principal component analysis and extracting the keywords, The information processing apparatus according to Appendix 1 or 2, comprising the above.

[0064] (Appendix 5) The keyword extraction means includes An encoder for converting words appearing in each document included in the subordinate element into vectors, An autoencoder for reducing the dimension of the vectors using autoencoding, A decoder for converting the vectors with reduced dimensions into words and outputting the keywords, The information processing apparatus according to Appendix 1 or 2, comprising the above.

[0065] (Appendix 6) It is provided with inappropriate heading detection means for detecting inappropriate headings from the headings included in the structured document. The feature word extraction means extracts feature words from the subordinate elements of the inappropriate heading. The heading generation means generates a new heading based on the extracted feature words. The information processing apparatus according to any one of Appendices 1 to 5.

[0066] (Appendix 7) The information processing apparatus according to Appendix 6, which is provided with document correction means for replacing the inappropriate heading in the document with the new heading to generate a corrected document.

[0067] (Appendix 8) The information processing apparatus according to Appendix 6, which is provided with document correction means for adding at least a part of the new heading to the inappropriate heading in the document to generate a corrected document.

[0068] (Appendix 9) The inappropriate heading is a heading with the same character string as another heading in a parallel relationship in the document. The information processing apparatus according to any one of Appendices 6 to 8.

[0069] (Appendix 10) The inappropriate heading is a heading that is in a parallel relationship in the document, is composed of numbers or symbols, and has no meaning or content. The information processing apparatus according to any one of Appendices 6 to 9.

[0070] (Appendix 11) The information processing apparatus according to any one of Appendices 1 to 9, which is provided with structuring means for converting the input document into the structured document.

[0071] (Appendix 12) Obtain a structured document including headings and text. For the headings included in the structured document, extract feature words from the subordinate elements of the heading. An information processing method for generating a new heading corresponding to the subordinate element based on the extracted feature words.

[0072] (Appendix 13) Obtain a structured document including a heading and text, For the headings included in the structured document, extract characteristic words from the subordinate elements of the heading, A recording medium storing a program that causes a computer to execute a process of generating a new heading corresponding to the subordinate element based on the extracted characteristic words.

[0073] The present invention has been described with reference to the embodiments and examples above, but the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

Explanation of Reference Numerals

[0074] 2 Heading 3 Text 12 Processor 21 Document Input Unit 22 Structuring Unit 23 Inappropriate Heading Detection Unit 24 Characteristic Word Extraction Unit 25 Heading Generation Unit 26 Document Output Unit 31, 41, 51 Word Splitting Unit 32 TF-IDF Calculation Unit 42 Matrix Creation Unit 43 PCA Unit 52 Encoder 53 Autoencoder 54 Decoder

Claims

1. An acquisition means for acquiring a structured document including a heading and text, An inappropriate heading detection means for detecting an inappropriate heading from the headings included in the structured document, For each of the inappropriate headings included in the structured document, a characteristic word extraction means for extracting characteristic words from the subordinate elements of the inappropriate heading, Among the extracted characteristic words, a heading generation means for determining characteristic words that have no matching or implicative relationship as new headings for the inappropriate headings corresponding to the characteristic words, Comprising, The inappropriate heading is a heading with the same character string as another heading in a parallel relationship in the structured document, or a heading that is in a parallel relationship in the structured document, composed of numbers or symbols, and has no meaning or content. An information processing apparatus.

2. The information processing apparatus according to claim 1, wherein the subordinate elements include at least one of a subordinate heading existing below the heading in the structured document and text existing below the heading.

3. The information processing apparatus according to claim 1 or 2, wherein the characteristic word extraction means extracts characteristic words using TF-IDF values.

4. The characteristic word extraction means, A matrix creation means for generating a matrix of the appearance frequencies of words appearing in each document included in the subordinate elements, An analysis means for reducing the dimension of the words that appear using principal component analysis and extracting the characteristic words, The information processing apparatus according to claim 1 or 2, comprising.

5. The characteristic word extraction means, For each document included in the subordinate elements, an encoder for converting the words appearing in the document into vectors, An autoencoder for reducing the dimension of the vectors using autoencoding, A decoder for converting the vectors with reduced dimensions into words and outputting the characteristic words, The information processing apparatus according to claim 1 or 2, comprising.

6. The information processing apparatus according to claim 1, comprising a document correction means for replacing the inappropriate heading in the structured document with the new heading to generate a corrected document.

7. The information processing apparatus according to claim 1, comprising a document correction means for adding the new heading to the inappropriate heading in the structured document to generate a corrected document.

8. An information processing method executed by a computer, Acquiring a structured document including a heading and text, Detecting an inappropriate heading from the headings included in the structured document, For each inappropriate heading included in the structured document, extract characteristic words from the subordinate elements of the inappropriate heading, Among the extracted characteristic words, determine, as new headings for the inappropriate headings corresponding to the characteristic words, the characteristic words that have no matching or implicative relationship, The inappropriate heading is an information processing method that is a heading with the same character string as another heading in a parallel relationship in the structured document, or a heading that is in a parallel relationship in the structured document and is composed of numbers or symbols and has no meaning or content.

9. Obtain a structured document including headings and text, Detect inappropriate headings from the headings included in the structured document, For each inappropriate heading included in the structured document, extract characteristic words from the subordinate elements of the inappropriate heading, Cause a computer to execute a process of determining, as new headings for the inappropriate headings corresponding to the characteristic words, the characteristic words that have no matching or implicative relationship among the extracted characteristic words, The inappropriate heading is a program that is a heading with the same character string as another heading in a parallel relationship in the structured document, or a heading that is in a parallel relationship in the structured document and is composed of numbers or symbols and has no meaning or content.

Images