Information processing device, related word detection method, and related word detection program

Abstract

To detect, without using a trained model, a related word which is not contained in a document to be processed, but which is related to the document to be processed, an information processing device (1) comprises: a related document retrieval unit (11) that uses an extracted word extracted from the document to be processed to retrieve a related document related to the extracted word; and a related word detection unit (12) that detects a related word that is related to the document to be processed from among candidate words extracted from the related document detected by the related document retrieval means.

Description

[Technical Field]

[0001] The present invention relates to a technique for detecting keywords associated with a document. [Background technology]

[0002] Techniques for detecting keywords from documents have been proposed. For example, Non-Patent Document 1 describes a technique for extracting important keywords using a document summarization model. The technique described in Non-Patent Document 1 can extract words that capture the meaning of the sentence by extracting a group of word vectors that are close to the embedding vector of the entire document.

[0003] Furthermore, Non-Patent Document 2 describes a text-to-text model that is trained using documents and desirable keywords as training data. The text-to-text model described in Non-Patent Document 2 can output words that do not appear in documents as keywords. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Xinnian Liang et. al.,"Unsupervised Keyphrase Extraction by Jointly Modeling Local and Global Context",15 Sep, 2021 [Non-patent document 2] Colin Raffel et. al.,"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer", 28 Jul, 2020 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Non-Patent Document 1 has a problem in that it cannot output words that do not appear in the text. Also, the Text-to-Text model described in Non-Patent Document 2 can output words that do not appear in the document, but has a problem in that it requires retraining the model to handle words in fields not included in the training data or newly added words.

[0006] One aspect of the present invention has been made in consideration of the above-mentioned problems, and one example of its purpose is to provide a technology that makes it possible to detect related words that are not contained in a target document but are related to the target document without using a trained model. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present invention includes a related document search means for searching for related documents related to extracted words extracted from a target document, and a related word detection means for detecting related words related to the target document from among candidate words extracted from the related documents detected by the related document search means.

[0008] A related word detection method according to one aspect of the present invention includes at least one processor using extracted words extracted from a target document to search for related documents related to the extracted words, and detecting related words related to the target document from among candidate words extracted from the related documents.

[0009] A related word detection program according to one aspect of the present invention is a program for causing a computer to function as a related document search means that uses extracted words extracted from a target document to search for related documents related to the extracted words, and a related word detection means that detects related words related to the target document from among candidate words extracted from the related documents detected by the related document search means. [Effects of the Invention]

[0010] According to one aspect of the present invention, related words that are not contained in a target document but are related to the target document can be detected without using a trained model. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment. [Figure 2] 1 is a flow chart showing the flow of a related word detection method according to the first exemplary embodiment. [Figure 3] FIG. 10 is a diagram illustrating an overview of a related word detection method according to the second exemplary embodiment. [Figure 4] FIG. 10 is a block diagram showing the configuration of an information processing device according to a second exemplary embodiment. [Figure 5] FIG. 10 is a diagram showing an example of a screen for a user to specify granularity according to the second exemplary embodiment. [Figure 6] 10 is a flowchart showing the flow of processing executed by an information processing device according to the second exemplary embodiment. [Figure 7] FIG. 10 is a diagram showing an example of a display screen of related words output by an information processing device according to the second exemplary embodiment. [Figure 8] FIG. 1 is a diagram illustrating an example of a computer that executes instructions of a program, which is software that realizes the functions of each device according to each exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0013] (Configuration of information processing device) The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes a related document search unit 11 (related document search means) and a related word detection unit 12 (related word detection means).

[0014] The related document search unit 11 uses extracted words extracted from a target document to search for related documents related to the extracted words. The related word detection unit 12 detects related words related to the target document from among the candidate words extracted from the related documents detected by the related document search unit 11.

[0015] As described above, the information processing device 1 according to this exemplary embodiment is configured to include a related document search unit 11 that uses extracted words extracted from a target document to search for related documents related to the extracted words, and a related word detection unit 12 that detects related words related to the target document from among the candidate words extracted from the related documents detected by the related document search unit 11. Therefore, the information processing device 1 according to this exemplary embodiment can detect related words that are not contained in the target document but are related to the target document without using a trained model.

[0016] (Related Word Detection Program) The functions of the information processing device 1 described above can also be realized by a program. The related word detection program according to this exemplary embodiment causes a computer to function as a related document search means that uses extracted words extracted from a target document to search for related documents related to the extracted words, and as a related word detection means that detects related words related to the target document from among candidate words extracted from the related documents detected by the related document search means. This related word detection program makes it possible to detect related words that are not contained in the target document but are related to the target document without using a trained model.

[0017] (Flow of related word detection method) The flow of the related word detection method according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the related word detection method. Note that the execution entity of each step in this related word detection method may be a processor provided in the information processing device 1, or a processor provided in another device, or each step may be executed by a processor provided in a different device.

[0018] In S11, at least one processor uses an extracted term extracted from a target document to search for related documents that are related to the extracted term.

[0019] In S12, at least one processor detects related words related to the target document from among the candidate words extracted from the related documents.

[0020] As described above, the related term detection method according to this exemplary embodiment includes, by at least one processor, using extracted terms extracted from a target document to search for related documents related to the extracted terms, and detecting related terms related to the target document from among candidate terms extracted from the related documents. This related term detection method can detect related terms that are not contained in the target document but are related to the target document without using a trained model.

[0021] Exemplary Embodiment 2 (Overview of related word detection method) A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. FIG. 3 is a diagram showing an overview of a related term detection method (hereinafter referred to as the present method) according to this exemplary embodiment. The present method is a method for detecting related terms related to a target document. Here, the target document is a document containing one or more sentences. The target document may be represented by unstructured data such as text data, image data, or audio data, or may be represented by semi-structured data in XML (eXtensible Markup Language) format, for example. Related terms are words that are not contained in the target document but are related to the target document. In the example of FIG. 3, the related terms are "energy industry" and "commodity crops."

[0022] In this method, first, the extraction unit 203 extracts extracted terms from the target document. Here, extracted terms are words contained in the target document. Extracted terms can also be said to be important keywords contained in the target document. In the example of FIG. 3, the extracted terms are "Country A," "Country B," and "economic cooperation." As an example, the extracted terms are extracted from the target document using the document summarization model described in Non-Patent Document 1 mentioned above.

[0023] Next, in this method, the search query generation unit 204 generates a search query using the extracted word. As an example, the search query is a combination of the extracted word and a sentence containing the extracted word (a sentence extracted from the target document). However, the search query is not limited to the above example, and other queries may be used. For example, the search query may be the extracted word itself.

[0024] Next, in this method, the related document search unit 205 searches for related documents from a corpus containing multiple documents using the search query. Here, the corpus includes, for example, an external corpus such as an online dictionary, newspaper articles, or a social networking service (SNS). The corpus may also include reconstructed documents generated by reconstructing documents included in the corpus. The related documents are documents related to the extracted word, and for example, are documents included in the corpus or parts of documents included in the corpus. The related documents may also be the reconstructed documents.

[0025] Next, in this method, the candidate word extraction unit 206 extracts candidate words from the related documents. Candidate words are words contained in the related documents that are candidates for related words. Candidate words are extracted from the related documents using, for example, the document summarization model described in Non-Patent Document 1 mentioned above. In the example of Figure 3, the candidate words are "warm," "President of Country A," "energy industry," and "cash crop."

[0026] Next, in this method, the related word detection unit 208 detects related words from the extracted candidate words. The candidate words may include words that have low relevance to the target document. For example, in the example of Figure 3, the candidate words "warm" and "President of Country A" have low relevance to the content of the target document. Therefore, in this method, those candidate words that are related to the target document are detected as related words. As described above, related words are words that are not contained in the target document but are related to the target document. Related words can also be said to be keywords associated with the target document. For example, the related words are output by being displayed on a display or the like and presented to the user.

[0027] (Configuration of information processing device) FIG. 4 is a block diagram showing the configuration of an information processing device 2 according to a second exemplary embodiment. The information processing device 2 is a device that detects related terms related to a target document. As shown in the figure, the information processing device 2 includes a control unit 20 that controls each unit of the information processing device 2, and a storage unit 21 that is a storage device that stores various data used by the information processing device 2. The information processing device 2 also includes an input unit 22 that accepts user input operations to the information processing device 2, and an output unit 23 that causes the information processing device 2 to output data. The information processing device 2 also includes a communication unit 24 that communicates with other devices via a communication line. The information processing device 2 may be a device dedicated to extracting related terms, or may be a general-purpose device that can be used for other purposes.

[0028] The control unit 20 also includes a reception unit 201 (reception means), a target document acquisition unit 202, an extraction unit 203 (extraction means), a search query generation unit 204 (search query generation means), a related document search unit 205 (related document search means), a candidate word extraction unit 206 (candidate word extraction means), a score calculation unit 207 (score calculation means), a related word detection unit 208 (related word detection means), and an output control unit 209. The storage unit 21 stores a specified granularity 211, target documents 212, extracted words 213, a search query 214, related documents 215, candidate words 216, scores 217, and related words 218.

[0029] The receiving unit 201 receives a specification of the granularity of the hierarchy. In this exemplary embodiment, each word constituting a target document is classified in a hierarchical structure. As an example of a classification method, for example, each word is assigned a major category, a medium category, and a minor category. For example, for the word "mandarin orange," the major category is "food," the medium category is "fruit," and the minor category is "citrus." The granularity of the hierarchy refers to the depth of the hierarchical hierarchy of words, and in the above example, the minor category is the deepest level (finest classification). Note that granularity may also be expressed in terms such as "depth," "degree," "level," "position," "layer," or "layer."

[0030] The receiving unit 201 may acquire data indicating the above-mentioned designation input via the input unit 22, or may acquire data indicating the above-mentioned designation from a storage destination (which may be in the storage unit 21 of the information processing device 2 or a storage device outside the information processing device 2) designated by the user of the information processing device 2. The receiving unit 201 stores the received designation of the granularity of the hierarchy in the storage unit 21 as a designated granularity 211. The designated granularity 211 is used when the extraction unit 203 extracts extracted words.

[0031] The target document acquisition unit 202 acquires a target document for which related words are to be detected, and stores the acquired target document in the storage unit 21 as a target document 212. The target document acquisition unit 202 may acquire a target document input via the input unit 22, or may acquire the target document from a storage location specified by a user of the information processing device 2 (which may be in the storage unit 21 of the information processing device 2, or may be a storage device outside the information processing device 2). The target document is typically text data, but as described above, data in other formats may also be the target document. In other words, the "target document" may be any data format as long as it contains at least one sentence.

[0032] The extraction unit 203 extracts extracted terms from the target document 212 and stores them in the storage unit 21 as extracted terms 213. As an example, the extraction unit 203 extracts extracted terms 213 from among the words that make up the target document 212 based on the specified granularity 211. However, the extraction unit 203 may extract extracted terms 213 without referring to the specified granularity. A method for extracting extracted terms 213 by the extraction unit 203 will be described later. Note that if the target document acquisition unit 202 acquires a target document in a data format other than text data, the extraction unit 203 may convert the acquired target document into text data and extract extracted terms from the text data.

[0033] The search query generation unit 204 uses the extracted terms 213 extracted by the extraction unit 203 to generate a search query to be used for searching for related documents, and stores the search query in the storage unit 21 as the search query 214. As an example, the search query generation unit 204 generates the search query 214 including the extracted terms 213 extracted from the target document 212 and sentences in the target document 212 that include the extracted terms 213. A method for generating the search query 214 by the search query generation unit 204 will be described later.

[0034] The related document search unit 205 uses the search query 214 to search for related documents from a corpus containing multiple documents. The corpus that is the search target for related documents includes the external corpus 4 connected via the communication unit 24. An internal corpus may be provided in the storage unit 21, and in this case, the internal corpus may be used as the search target instead of or in addition to the external corpus 4. The related document search unit 205 stores the searched related documents as related documents 215 in the storage unit 21. A method for searching for related documents 215 by the related document search unit 205 will be described later.

[0035] The candidate word extraction unit 206 extracts candidate words from the related documents 215 and stores them in the storage unit 21 as candidate words 216. The method of extracting the candidate words 216 by the candidate word extraction unit 206 will be described later.

[0036] The score calculation unit 207 calculates a score, which is an index value indicating the relevance between the target document 212 and the candidate word 216, and stores the score in the storage unit 21 as the score 217. As an example, the score calculation unit 207 calculates the score indicating the relevance between the target document 212 and the candidate word 216 using a scoring device that is used in a search engine to calculate a score indicating the relevance between a search term and a website.

[0037] The related word detection unit 208 detects related words from among the candidate words 216 and stores them in the storage unit 21 as related words 218. As an example, the related word detection unit 208 detects the related words 218 from among the candidate words 216 based on the scores 217 calculated by the score calculation unit 207. However, the method by which the related word detection unit 208 detects the related words 218 is not limited to the above example, and the related word detection unit 208 may detect the related words 218 from the candidate words 216 using other methods.

[0038] The output control unit 209 causes an output device to output the related words 218. The output device that outputs the related words 218 is, for example, connected to the output unit 23 or the communication unit 24. For example, the output device includes a display device such as a liquid crystal display or a touch panel, a speaker that outputs sound, or a projector. However, the output device is not limited to the above examples and may be another output device.

[0039] (Extraction of extracted words) The following describes a method for extracting the extracted terms 213 by the extraction unit 203. The extraction unit 203 may extract the extracted terms 213 from the target document 212 using, for example, the document summary model described in Non-Patent Document 1. Alternatively, the extraction unit 203 may extract the extracted terms 213 from the target document 212 using a named entity recognition technique.

[0040] When using a named entity extraction technique, the extraction unit 203, for example, estimates the type of each word constituting the target document 212 using the named entity extraction technique, and extracts words of a specific type (e.g., a person's name, a country name, etc.) as the extracted terms 213. Here, the type of each word indicates the classification result by named entity classification. In other words, the extraction unit 203 may extract words of a type that matches a type included in the whitelist as the extracted terms 213. Furthermore, the extraction unit 203 may, for example, extract words of a type other than the specific type from among the words constituting the target document 212 as the extracted terms 213. In other words, the extraction unit 203 may extract words of a type other than a type included in the blacklist as the extracted terms 213.

[0041] Furthermore, when multiple types are classified in a hierarchical structure, the extraction unit 203 may extract, as the extracted word 213, a word of a type corresponding to a specific hierarchical level. The specific hierarchical level may be a predetermined hierarchical level, or may be a hierarchical level corresponding to the specified granularity 211 specified by a user's input operation. For example, when the specified granularity 211 is "medium category," the extraction unit 203 may extract, as the extracted word 213, a word for which a medium category is set but no minor category is set. In this case, for example, the extraction unit 203 may extract the word "apple," which has a major category of "food" and a medium category of "fruit," but may not extract "Jonagold," which has a minor category of "variety" in addition to these categories. In this case, the extraction unit 203 may convert "Jonagold" to the medium category of "fruit" and extract "apple" as the extracted word 213. Alternatively, a granularity may be set for each category in advance. In this case, the extraction unit 203 may extract, as the extracted word 213, a word for which a specified granularity is set.

[0042] Fig. 5 is a diagram showing an example of a screen for a user to specify a specified granularity 211. The screen SC1 in Fig. 5 is displayed on, for example, an output device (display) connected to the output unit 23 or the communication unit 24. The screen SC1 includes a target document 212_1, a slide bar 220, and an extracted word list 213_1.

[0043] The slide bar 220 is an object that allows the user to specify the specified granularity 211. The slide bar 220 on the screen SC1 allows the user to select "Concept Granularity" in three stages: "Low," "Middle," and "High." The user operates the slide bar 220 to specify the specified granularity 211. Of the words contained in the target document 212_1, words that belong to a hierarchy corresponding to the specified granularity 211 specified by the user are extracted and displayed on the screen SC1 as an extracted word list 213_1.

[0044] When multiple types are classified in a hierarchical structure, the whitelist may be prepared in advance for each hierarchy, and the extraction unit 203 may use the whitelist corresponding to the specified granularity 211 to perform the extraction process of the extracted words 213. Alternatively, the blacklist may be prepared in advance for each hierarchy, and the extraction unit 203 may use the blacklist corresponding to the specified granularity 211 to perform the extraction process of the extracted words 213.

[0045] Furthermore, the extraction unit 203 may combine a plurality of techniques to extract the extracted terms 213. For example, the extraction unit 203 may extract both extracted terms extracted using the document summary model and extracted terms extracted by the named entity extraction technique as the extracted terms 213. However, the technique by which the extraction unit 203 extracts the extracted terms 213 from the target document 212 is not limited to the above example, and the extraction unit 203 may extract the extracted terms 213 from the target document 212 by another technique.

[0046] In addition to the extracted words, the extracted word list 213_1 on the screen SC1 displays the classifications set for the extracted words and sentences containing the extracted words in the target document 212_1. By displaying the classifications and sentences containing the extracted words together with the extracted words, the user can recognize what classifications the extracted words belong to and what contexts the extracted words are used in, making it easier for the user to select extracted words that interest them.

[0047] (Generating search queries) Next, a method for generating a search query 214 by the search query generation unit 204 will be described. For example, the search query generation unit 204 may use the extracted word 213 extracted by the extraction unit 203 as the search query 214 as is. Alternatively, the search query generation unit 204 may generate a search query 214 that includes the extracted word 213 and at least a portion of N sentences (N is a natural number) surrounding the extracted word 213 in which the extracted word 213 appears. Here, when N=1, the search query 214 includes the extracted word 213 and at least a portion of a sentence including the extracted word 213.

[0048] In the example of FIG. 3, search query q1 includes the extracted word 213 "Country A" and a sentence containing "Country A." Search query q2 includes the extracted word 213 "Country B" and a sentence containing "Country B." Search query q3 includes the extracted word 213 "economic cooperation" and a sentence containing "economic cooperation."

[0049] (Search for related documents) Next, a method for searching for related documents 215 by the related document search unit 205 will be described. The related document search unit 205 uses the search query 214 to search for related documents 215 from a corpus containing multiple documents. Here, the corpus may include a reconstructed document generated by reconstructing a document using key words that are relatively important among the words contained in the document. Here, the key words may be, for example, words that are linked among the words contained in the document, or words that are hashtagged. Furthermore, the key words may be, for example, words extracted from information accompanying the document, such as the properties of the document file or the name of the author. As an example, the reconstructed document is a document in which key words are listed. Furthermore, the reconstructed document may be a sentence created by supplementing key words with other words. The method for generating sentences from words is not particularly limited, and for example, a known method may be used.

[0050] The corpus used by the related document search unit 205 may be selected by the user. For example, if the search query 214 is related to a newspaper article, the user can select a corpus that includes the newspaper article. Alternatively, if the search query 214 is related to a cooking recipe, the user can select a corpus that includes the cooking recipe. In this way, by the user selecting a corpus that is close to the characteristics of the search query, the related document search unit 205 can easily search for documents that are highly relevant to the search query.

[0051] The reconstructed document may be generated by the information processing device 2 (for example, the related document search unit 205) or by another device. In other words, the reconstructed document may be generated by any entity. Furthermore, the reconstructed document may be generated by any method.

[0052] As an example, the related document searching unit 205 searches for the related documents 215 using a so-called Sparse Retriever method. That is, the related document searching unit 205 determines, as the related documents 215, documents that have a high degree of word overlap between the search query 214 and the document. The related document searching unit 205 may also search for the related documents 215 using a so-called Dense Retriever method. In this case, the related document searching unit 205 converts the search query 214 into an embedding vector, and determines, as the related documents 215, documents whose vector expressions are similar to the embedding vector (the distance between the vectors is short). However, the method by which the related document searching unit 205 searches for the related documents 215 is not limited to the example described above, and the related document searching unit 205 may search for the related documents 215 using other methods.

[0053] (Candidate word extraction) Next, a method for extracting candidate words 216 by the candidate word extraction unit 206 will be described. As an example, the candidate word extraction unit 206 may extract candidate words 216 from related documents 215 using the document summary model described in Non-Patent Document 1 mentioned above. Alternatively, the candidate word extraction unit 206 may extract candidate words 216 using a named entity extraction technique. Extraction of candidate words 216 using the named entity extraction technique is similar to the technique used by the extraction unit 203 to extract extracted words 213 from target documents 212, and therefore, description thereof will not be repeated here.

[0054] The candidate words 216 extracted from the related documents 215 also include those extracted from accompanying information or structural information accompanying the related documents 215. In other words, the candidate word extraction unit 206 extracts, as candidate words 216, relatively important keywords identified based on at least one of structural information indicating the structure of the related documents 215 and accompanying information accompanying the related documents 215. Here, the structural information is, for example, link information attached to words included in the related documents 215. Since more important words are more likely to be linked, the candidate word extraction unit 206 can extract important words as candidate words 216 by designating linked words as candidate words 216. The accompanying information may include, for example, file properties, meta information such as author names, or hashtags. However, the method by which the candidate word extraction unit 206 extracts the candidate words 216 is not limited to the above example; the candidate word extraction unit 206 may extract the candidate words 216 from the related documents 215 using other methods. The candidate words 216 may include some or all of the extracted words 213 .

[0055] (Related word detection) A method for detecting related words by the related word detection unit 208 will be described. As an example, the related word detection unit 208 detects related words 218 from among the candidate words 216 based on a score 217 indicating the relevance between the target document 212 and the candidate words 216. As an example, the score 217 is a real number ranging from 0 to 1, and the closer to 0 the lower the relevance, and the closer to 1 the higher the relevance, but this is not limiting. For example, the related word detection unit 208 detects the candidate words 216 whose calculated score is equal to or greater than a threshold value. ofIt may also be detected as a related word.

[0056] The score 217 is, for example, a score that represents the distance between an embedding vector calculated from the candidate word 216 and an embedding vector calculated from the target document 212. Here, the embedding vector calculated from the candidate word 216 may be a vector of the candidate word 216 itself, or may be a vector of a sentence including the candidate word 216, or of the sentence and its surrounding sentences.

[0057] Furthermore, the embedding vector calculated from the target document 212 may be a vectorization of the target document 212 itself, or may be a vectorization of a sentence containing the extracted word 213 extracted from the target document 212.

[0058] The embedding vector is a value calculated by an embedding model that represents arbitrary data in a vector space. The embedding model is a model in which the similarity of data is expressed as a distance in space.

[0059] The method for training the embedding model is not limited to a specific one, and general machine learning techniques may be used. For example, the related word detection unit 208 may use a model trained by a training algorithm using a multilayer neural network as the embedding model.

[0060] The related word detection unit 208 can calculate a score 217 that takes into account the semantic similarity between the candidate word 216 and the target document 212 by using the embedding vector.

[0061] Furthermore, the method by which the related word detection unit 208 calculates the score 217 is not limited to the embedding vector, and any method may be used. For example, the related word detection unit 208 may use an existing natural language processing technique such as syntactic analysis to vectorize the candidate words 216 and the target document 212 and calculate the score 217.

[0062] As another example of how the score calculation unit 207 calculates the score 217, the score calculation unit 207 may use a scorer used in a search engine to calculate a score indicating the relevance between a search term and a website. The scorer is, for example, a trained model generated by machine learning the relevance between a search term and a website.

[0063] (Processing flow) The flow of the process (related word detection method) executed by the information processing device 2 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of the process executed by the information processing device 2.

[0064] In S21, the receiving unit 201 receives a specification of the granularity of the hierarchy and stores it in the memory unit 21 as specified granularity 211. In S22, the target document acquisition unit 202 acquires the target document and stores it in the memory unit 21 as target document 212. In S23, the extraction unit 203 extracts extracted terms from the target document 212 and stores it in the memory unit 21 as extracted terms 213. In S24, the search query generation unit 204 generates a search query using the extracted terms 213 and stores it in the memory unit 21 as search query 214.

[0065] In S25, the related document search unit 205 searches the corpus for related documents using the search query 214, and stores them in the memory unit 21 as related documents 215. In S26, the candidate word extraction unit 206 extracts candidate words from the related documents 215, and stores them in the memory unit 21 as candidate words 216.

[0066] In S27, the score calculation unit 207 calculates the score 217 for each of the candidate words 216. In S28, the related word detection unit 208 detects related words 218 from among the candidate words 216 based on the scores 217 calculated by the score calculation unit 207.

[0067] In S29, the output control unit 209 outputs the related words 218 detected by the related word detection unit 208. The output control unit 209 may output the related words 218 to an output device connected via the output unit 23 or the communication unit 24. Note that output of the related words 218 is not essential, and for example, the related word detection unit 208 may store the related words 218 in a storage destination specified by the user of the information processing device 2 (which may be in the storage unit 21 of the information processing device 2 or a storage device external to the information processing device 2), and then end the processing of FIG.

[0068] When the related words 218 include the extracted words 213, the output control unit 209 may output the related words 218 other than the extracted words 213, or may also output the extracted words 213 included in the related words 218. When outputting both the extracted words 213 and the related words 218, it is preferable that the output control unit 209 presents the extracted words 213 and the related words 218 (those not included in the extracted words 213) in a distinguishable manner, for example by changing the display mode of the extracted words 213 and the related words 218.

[0069] FIG. 7 is a diagram showing an example of a display screen of related words 218 output by the output control unit 209. In FIG. 7, related words 218_2 are related words detected by the information processing device 2 for the extracted word "XXX" included in the target document. The extracted word is the name of a character in the story, and this name is also used as a company name. 212_2 is a sentence containing the extracted word in the target document. Meanwhile, related words 218_3 are related words detected by the information processing device 2 for the extracted word "XXX" included in another target document. 212_3 is a sentence containing the extracted word in the target document. In the example screen of FIG. 7, related words are shown as "associated keywords."

[0070] 7, even for the same extracted word, the related word 218_2 is different from the related word 218_3. Specifically, the related word 218_2 is a word that corresponds to the fact that the extracted word is used as a company name, while the related word 218_3 is a word that corresponds to the fact that the extracted word is used as a character name in a story. In this way, the information processing device 2 can extract, for each of the target documents 212_2 and 212_3, the related words 218_2 and 218_3 that effectively capture the context of the target documents, including words that are not included in the target documents.

[0071] (Effects of information processing devices) As described above, according to this exemplary embodiment, related documents 215 related to extracted terms 213 extracted from a target document 212 are searched for, and related terms 218 are detected from the searched related documents 215, thereby making it possible to detect related terms 218 that are not included in the target document 212. Furthermore, according to this exemplary embodiment, new domains, words, and concepts can be handled simply by replacing (or adding) the corpus. In other words, according to this exemplary embodiment, detection processing that can handle new topics can be performed simply by replacing or adding the corpus, without retraining the model.

[0072] Furthermore, in the information processing device 2 according to the present exemplary embodiment, the words constituting the target document 212 are classified in a hierarchical structure, and the information processing device 2 according to the present exemplary embodiment is configured to include a receiving unit 201 that receives a designation of the granularity of the hierarchy, and an extracting unit 203 that extracts extracted terms 213 from the words constituting the target document 212 based on the designated granularity. This configuration provides the effect of being able to detect related terms 218 corresponding to the extracted terms 213 extracted at the granularity desired by the user, in addition to the effects provided by the information processing device 1 according to the first exemplary embodiment.

[0073] Furthermore, the information processing device 2 according to this exemplary embodiment includes a search query generation unit 204 that generates a search query 214 that includes extracted terms 213 extracted from a target document 212 and a sentence in the target document 212 that includes the extracted terms 213, and the related document search unit 205 is configured to search for related documents 215 using the search query 214. This configuration makes it possible to perform a search that takes into account the context of the sentence that includes the extracted terms 213, thereby making it possible to detect related documents 215 that are highly relevant.

[0074] Furthermore, in the information processing device 2 according to this exemplary embodiment, the related document search unit 205 searches for related documents 215 from a corpus containing multiple documents, and the corpus contains reconstructed documents generated by reconstructing the documents using key words that are relatively important among the words contained in the documents.

[0075] In a reconstructed document reconstructed using key words extracted from a document, the main points are condensed compared to the original document, so the relevance between the extracted words 213 extracted from the target document 212 and the reconstructed document can be determined relatively accurately. Therefore, with the above configuration, it is possible to detect relevant related documents 215.

[0076] Furthermore, the information processing device 2 according to this exemplary embodiment employs a configuration including a candidate word extraction unit 206 that extracts, as candidate words 216, keyword words that are relatively important and that are identified based on at least one of structural information indicating the structure of the related documents 215 and associated information associated with the related documents 215. With this configuration, keyword words that are considered to be highly important can be extracted as candidate words 216.

[0077] Furthermore, the information processing device 2 according to this exemplary embodiment is provided with a score calculation unit 207 that calculates a score 217 indicating the relevance between the target document 212 and the candidate words 216 using a scoring device used in a search engine to calculate a score indicating the relevance between a search term and a website, and a related word detection unit 208 that is configured to detect related words 218 from among the candidate words 216 based on the score 217.

[0078] Search engines calculate a score indicating the relevance between a search term entered by a user and each website searched, and present the results to the user in descending order of score. The scorer used to calculate the score is one that can accurately calculate a score that represents the relevance between a search term and a website, and is continually being improved to further increase the accuracy of score calculation.

[0079] According to the above configuration, a scorer used for calculating such a score is used to calculate a score 217 indicating the relevance between the target document 212 and the candidate word 216, and related words 218 are detected based on the score 217. Therefore, it becomes possible to detect appropriate related words 218 based on the score 217 that accurately represents the relevance between the target document 212 and the candidate word 216.

[0080] [Modification] The execution entity of each process described in the above embodiment may be any entity and is not limited to the above example. In other words, a related word detection system having the same functions as the information processing device 2 can be constructed using multiple devices that can communicate with each other. For example, by distributing each block shown in FIG. 4 among multiple devices, a related word detection system having the same functions as the information processing device 2 can be constructed. For example, the search for related documents 215 and the detection of related words 218 may be executed by different devices. Furthermore, each process included in the flow of FIG. 6 may be shared and executed by multiple devices (processors).

[0081] [Software implementation example] Some or all of the functions of the information processing devices 1 and 2 may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0082] In the latter case, the information processing devices 1 and 2 are realized, for example, by a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 8. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program (related word detection program) P for operating the computer C as the information processing devices 1 and 2. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing devices 1 and 2.

[0083] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0084] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0085] The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communications network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0086] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0087] [Appendix 2] Some or all of the above-described embodiments can also be described as follows, but the present invention is not limited to the following described aspects. (Appendix 1) An information processing device comprising: a related document search means for searching for related documents related to extracted words using extracted words extracted from a target document; and a related word detection means for detecting related words related to the target document from among candidate words extracted from the related documents detected by the related document search means.

[0088] (Appendix 2) The information processing device described in Appendix 1, wherein each word constituting the target document is classified in a hierarchical structure, and the information processing device is equipped with a receiving means for receiving a specification of the granularity of the hierarchy, and an extraction means for extracting the extracted words from the words constituting the target document based on the specified granularity.

[0089] (Appendix 3) 3. An information processing device according to claim 1, further comprising a search query generation means for generating a search query including the extracted words extracted from the target documents and sentences in the target documents that include the extracted words, wherein the related document search means searches for the related documents using the search query.

[0090] (Appendix 4) The information processing device of any one of Appendices 1 to 3, wherein the related document search means searches for the related documents from a corpus containing a plurality of documents, and the corpus includes reconstructed documents generated by reconstructing the documents using key words that are relatively important among the words contained in the documents.

[0091] (Appendix 5) An information processing device described in any one of appendices 1 to 4, comprising a candidate word extraction means for extracting, as the candidate words, key words of relatively high importance identified based on at least one of structural information indicating the structure of the related documents and accompanying information accompanying the related documents.

[0092] (Appendix 6) An information processing device as described in any one of appendices 1 to 5, comprising a score calculation means for calculating a score indicating the relevance between the target document and the candidate words using a scoring device used to calculate a score indicating the relevance between a search term and a website in a search engine, and the related word detection means for detecting the related words from among the candidate words based on the score.

[0093] (Appendix 7) A related word detection method, comprising: at least one processor using extracted words extracted from a target document to search for related documents related to the extracted words; and detecting related words related to the target document from among candidate words extracted from the related documents.

[0094] (Appendix 8) A related word detection program that causes a computer to function as a related document search means that uses extracted words extracted from a target document to search for related documents related to the extracted words, and a related word detection means that detects related words related to the target document from among candidate words extracted from the related documents detected by the related document search means.

[0095] [Appendix 3] Some or all of the above-described embodiments can also be expressed as follows. An information processing device comprising at least one processor that executes a related document search process that uses extracted words extracted from a target document to search for related documents related to the extracted words, and a related word detection process that detects related words related to the target document from among candidate words extracted from the related documents detected in the related document search process.

[0096] The information processing device may further include a memory that stores a program for causing the processor to execute the related document search process and the related word detection process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0097] 1, 2 Information processing device 11, 205 Related Document Search Section 12, 208 Related Words Detection Unit 203 Extraction part 204 Search query generation unit 206 Candidate word extraction part 207 Score Calculation Unit

Claims

1. Each word constituting the target document is classified in a hierarchical structure, a receiving means for receiving a specification of the granularity of the hierarchy; an extraction means for extracting extracted words from among words constituting the target document based on a specified granularity; a related document search means for searching for related documents related to the extracted words by using the extracted words extracted from the target documents; a candidate word extraction means for extracting, from the related documents detected by the related document search means, key words of relatively high importance identified based on at least one of structural information indicating the structure of the related documents and accompanying information accompanying the related documents, as candidate words; and related word detection means for detecting related words related to the target document from among the candidate words extracted from the related documents detected by the related document search means.

2. a search query generation means for generating a search query including the extracted words extracted from the target documents and sentences in the target documents that include the extracted words; The information processing apparatus according to claim 1 , wherein the related document search means searches for the related documents using the search query.

3. the related document retrieval means retrieves the related documents from a corpus including a plurality of documents; The information processing apparatus according to claim 1 , wherein the corpus includes a reconstructed document generated by reconstructing the document using key words that are relatively important among words contained in the document.

4. score calculation means for calculating a score indicating the relevance between the target document and the candidate word using a scorer used in a search engine to calculate a score indicating the relevance between a search word and a website; The information processing apparatus according to claim 1 , wherein the related word detection means detects the related words from among the candidate words based on the scores.

5. Each word constituting the target document is classified in a hierarchical structure, At least one processor Accepting the specification of the granularity of the hierarchy; extracting extracted words from among the words constituting the target document based on a specified granularity; using the extracted words extracted from the target document to search for related documents related to the extracted words; extracting, from the related documents, key words of relatively high importance identified based on at least one of structural information indicating the structure of the related documents and accompanying information accompanying the related documents, as candidate words; and detecting related words related to the target document from the candidate words extracted from the related documents.

6. Each word constituting the target document is classified in a hierarchical structure, Computer, A receiving means for receiving a specification of the granularity of the hierarchy; an extraction means for extracting extracted words from among words constituting the target document based on a specified granularity; a related document search means for searching for related documents related to the extracted words by using the extracted words extracted from the target documents; a candidate word extraction means for extracting, from the related documents detected by the related document search means, key words of relatively high importance specified based on at least one of structural information indicating the structure of the related documents and accompanying information accompanying the related documents, as candidate words; and A related word detection program for functioning as a related word detection means for detecting related words related to the target document from among the candidate words extracted from the related documents detected by the related document search means.

Images