Information processing program, information processing method, and information processing device

Abstract

Provided is an information processing device which acquires a plurality of sentences including a plurality of words. This information processing device applies processing to the plurality of sentences, the processing identifying a set of feature words from the plurality of words on the basis of sentence vectors of the sentences including the plurality of words and word vectors of the plurality of words. The information processing device classifies the plurality of sentences such that sentences including the same set of feature words are included in the same cluster.

Description

[Technical Field]

[0001] The present invention relates to an information processing program and the like. [Background technology]

[0002] A huge amount of data, such as text, is stored in a database (DB), and it is necessary to appropriately search for data similar to a search query specified by a user from the DB. In the following explanation, text will be described as a sentence containing multiple words.

[0003] In conventional technology, an inverted index is set when text is registered in a DB, and a data search is performed when a search query is received. For example, in conventional technology, when text is registered in a DB, the vector of each sentence (hereinafter referred to as sentence vector) is calculated in advance, and similar sentence vectors are classified into the same cluster. In conventional technology, the positions of multiple sentences included in the same cluster are associated with their representative vectors and set in the inverted index, thereby improving the efficiency of the search process.

[0004] In the prior art, when calculating a sentence vector, the word vectors of each of the multiple words that make up the sentence are calculated, and the word vectors of all the words are summed to calculate the sentence vector. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2020 / 095357 [Patent Document 2] Japanese Patent Application Publication No. 2019-101993 Summary of the Invention [Problem to be solved by the invention]

[0006] A sentence contains various words such as nouns, verbs, adjectives, and particles, and if a sentence vector is calculated by simply adding up the word vectors of all the words contained in the sentence, as in the conventional technology, the sentence vector may not clearly indicate the characteristics of the sentence. When clustering is performed using such a sentence vector, multiple sentences that should be classified into different clusters may end up being classified into the same cluster.

[0007] In one aspect, the present invention aims to provide an information processing program, an information processing method, and an information processing device that are capable of appropriately clustering text. [Means for solving the problem]

[0008] In the first proposal, a computer is caused to perform the following process: The computer acquires a plurality of sentences each having a plurality of words; The computer performs a process for the plurality of sentences to identify a set of characteristic words from the plurality of words based on the sentence vector of the sentence each having a plurality of words and the word vectors of the plurality of words; The computer classifies the plurality of sentences so that sentences having the same set of characteristic words are included in the same cluster. [Effects of the Invention]

[0009] The text can be clustered appropriately. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram for explaining the process of identifying characteristic words of a sentence. [Figure 2] FIG. 2 is a diagram for explaining the clustering process. [Figure 3] FIG. 3 is a diagram (1) for explaining the processing in the search phase. [Figure 4] FIG. 4 is a diagram (2) for explaining the processing in the search phase. [Figure 5] FIG. 5 is a functional block diagram showing the configuration of an information processing device according to this embodiment. [Figure 6] FIG. 6 is a diagram illustrating an example of the data structure of the word vector dictionary. [Figure 7] FIG. 7 is a diagram illustrating the data structure of an inverted index. [Figure 8] FIG. 8 is a flowchart illustrating a processing procedure in the preparation phase of the information processing device according to the present embodiment. [Figure 9] FIG. 9 is a flowchart illustrating a processing procedure of the search phase of the information processing device according to the present embodiment. [Figure 10] FIG. 10 is a flowchart showing the procedure of a search process based on multiple sentences. [Figure 11] FIG. 11 is a diagram for explaining other processes of the information processing device. [Figure 12] FIG. 12 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same functions as the information processing apparatus of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an information processing program, an information processing method, and an information processing device disclosed in the present application will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to these embodiments. [Example]

[0012] The information processing device according to this embodiment performs a preparatory phase process and a search phase process. First, the preparatory phase process executed by the information processing device will be described. For example, the preparatory phase includes a process for identifying feature words in sentences and a process for clustering sentences.

[0013] 1 is a diagram for explaining the process of identifying characteristic words in a sentence. In FIG. 1, a case will be explained in which characteristic words are identified from the sentence “Umasu wa amai ninjin ga suki da.” (Horses like sweet carrots), which is registered in the text DB 50.

[0014] The information processing device identifies the vector of each word included in the sentence "Horses like sweet carrots." based on a word vector dictionary that defines the relationship between words and word vectors. In the following explanation, word vectors are referred to as "word vectors." For example, the word vector of "horse" is denoted as wv-a. The word vector of "sweet" is denoted as wv-b. The word vector of "carrot" is denoted as wv-c. The word vector of "like" is denoted as wv-d. The word vectors of the words "is," "ga," and "da" are not shown in the figures. The information processing device calculates the sentence vector sv1 of the sentence "Horses like sweet carrots." by accumulating the word vectors of each word in the sentence "Horses like sweet carrots."

[0015] The information processing device calculates the cosine similarity between the sentence vector sv1 and each of the word vectors wv-a to wv-d, and identifies, based on the cosine similarity, words of the word vectors that deviate from the sentence vector sv1 as “characteristic words.” For example, the information processing device identifies, as characteristic words, words of word vectors whose cosine similarity with the sentence vector sv1 is equal to or greater than a threshold.

[0016] 1, the cosine similarity between the sentence vector sv1 and the word vector wv-a, the cosine similarity between the sentence vector sv1 and the word vector wv1-c, and the cosine similarity between the sentence vector sv1 and the word vector wv-d are each set to a threshold or greater. Then, the information processing device identifies the word "horse" in the word vector wv-a, the word "carrot" in the word vector wv1-c, and the word "like" in the word vector wv-d as characteristic words.

[0017] Next, the information processing device clusters the sentences based on the characteristic words identified in the process of Fig. 1. Fig. 2 is a diagram for explaining the clustering process. The information processing device identifies a word cluster ID for each characteristic word based on the word cluster dictionary 60. In the word cluster dictionary 60, multiple words are classified into multiple clusters, and each cluster is assigned a word cluster ID. The information processing device identifies the word cluster ID set for the cluster to which the characteristic word belongs, and sets the identified word cluster ID as the word cluster ID of the characteristic word.

[0018] For example, the information processing device, based on the word cluster dictionary 60, identifies the word cluster ID "l" set for a cluster including the feature word "horse" and sets it as the word cluster ID of the feature word "horse." The information processing device, based on the word cluster dictionary 60, identifies the word cluster ID "m" set for a cluster including the feature word "carrot" and sets it as the word cluster ID of the feature word "carrot." The information processing device, based on the word cluster dictionary 60, identifies the word cluster ID "n" set for a cluster including the feature word "like" and sets it as the word ID of the feature word "like."

[0019] By executing the above process, the information processing device identifies the word cluster IDs "l", "m", and "n" corresponding to the characteristic words "horse", "carrot", and "like". The information processing device sets the set of word cluster IDs "l", "m", and "l" as the set of word cluster IDs corresponding to the sentence "Horses like sweet carrots."

[0020] Next, the information processing device identifies the sentence cluster to which the sentence belongs based on the set of word cluster IDs set for the sentence and the sentence cluster dictionary 70. Here, the sentence cluster dictionary 70 associates the sentence cluster ID that identifies the cluster of the sentence with the set of word cluster IDs. For example, the sentence cluster ID corresponding to the word cluster IDs "l," "m," and "n" is "Cr1." Therefore, the information processing device identifies the sentence cluster ID of the sentence cluster to which the sentence "Horses like sweet carrots" belongs as "Cr1."

[0021] The information processing device associates the sentence cluster ID of the identified sentence "Horses like sweet carrots" with the position of the sentence "Horses like sweet carrots" in the text DB 50 and registers them in the transposed index 80.

[0022] The information processing device repeatedly executes the above process for each sentence registered in the text DB 50, and registers the relationship between the sentence cluster ID of each sentence and the position of each sentence in the transposed index 80.

[0023] As described above, the information processing device according to this embodiment identifies a set of characteristic words from multiple words included in a sentence, and identifies the sentence cluster ID of the cluster to which the sentence belongs based on the set of characteristic words and the sentence cluster dictionary 70. This allows the sentences to be clustered appropriately.

[0024] Next, the processing of the search phase executed by the information processing device will be described. Fig. 3 is a diagram (1) for explaining the processing of the search phase. In the explanation of Fig. 3, a case will be described in which a single sentence, "Horses like sweet carrots," is specified as a search query q1.

[0025] When the information processing device receives the search query q1, it identifies the characteristic words "horse," "carrot," and "favorite" from the sentence in the search query q1, "Horses like sweet carrots." The process by which the information processing device identifies characteristic words from multiple words included in the sentence is the same as the process described in FIG. 1.

[0026] The information processing device identifies the word cluster ID of each characteristic word based on each characteristic word and the word cluster dictionary 60. For example, the information processing device identifies the word cluster ID "l" set for a cluster including the characteristic word "horse" based on the word cluster dictionary 60, and sets it as the word cluster ID of the characteristic word "horse". The information processing device identifies the word cluster ID "m" set for a cluster including the characteristic word "carrot" based on the word cluster dictionary 60, and sets it as the word cluster ID of the characteristic word "carrot". The information processing device identifies the word cluster ID "n" set for a cluster including the characteristic word "favorite food" based on the word cluster dictionary 60, and sets it as the word ID of the characteristic word "favorite food".

[0027] By performing the above process, the information processing device identifies the word cluster IDs "l," "m," and "n" that correspond to the characteristic words "horse," "carrot," and "favorite food." The information processing device sets the set of word cluster IDs "l," "m," and "l" as the set of word cluster IDs that corresponds to the sentence "Horses like sweet carrots" in the search query q1.

[0028] The information processing device identifies the sentence cluster to which the sentence of search query q1 belongs, based on the set of word cluster IDs set for the sentence of search query q1 and the sentence cluster dictionary 70. For example, the sentence cluster ID corresponding to the word cluster IDs "l," "m," and "n" is "Cr1." Therefore, the information processing device identifies the sentence cluster ID of the sentence cluster to which the sentence of search query q1, "Horses like sweet carrots," belongs, as "Cr1."

[0029] The information processing device identifies the position of the sentence in the text DB 50 that belongs to the sentence cluster ID based on the sentence cluster ID (e.g., "Cr1") corresponding to the sentence of the search query q1 and the transposed index 80. The information processing device extracts the sentence from the identified position and outputs the extracted sentence as a search result.

[0030] As described above, the information processing device identifies a set of characteristic words from multiple words included in the search query q1, and identifies a sentence cluster ID corresponding to the search query q1 based on the set of characteristic words and the sentence cluster dictionary 70. Then, the information processing device performs a search based on the inverted index 80 created in advance and the sentence cluster ID corresponding to the search query q1. This allows the information processing device to appropriately search for sentences corresponding to the search query q1.

[0031] Next, we will explain Figure 4. Figure 4 is a diagram (2) for explaining the processing of the search phase. In the explanation of Figure 4, we will explain the case where multiple sentences such as paragraphs or items are specified as a search query. For example, search query q2 contains multiple sentences such as "This program has the following features: ...It is composed of multiple sub-programs...Its functions are its distinctive features...It has achieved the effect of high speed...".

[0032] The information processing device calculates a sentence vector for each of multiple sentences included in the search query q2. For example, the information processing device calculates the sentence vector for one sentence by integrating the word vectors of multiple words included in one sentence. Alternatively, the information processing device may execute the processes described with reference to FIGS. 1 and 2 to identify a set of word cluster IDs for each characteristic word in the sentence, and use the set of word cluster IDs as the sentence vector.

[0033] In the example shown in Figure 4, the sentence vector of the sentence "Its features are as follows" is sentence vector sv2-1. The sentence vector of the sentence "This program is characterized by the following contents" is sentence vector sv2-2. The sentence vector of the sentence "It is made up of multiple sub-programs" is sentence vector sv2-3. The sentence vector of the sentence "It has achieved the effect of increased speed" is sentence vector sv2-4.

[0034] The information processing device calculates a sentence vector dv1 by integrating the sentence vectors of multiple sentences included in the search query q2.

[0035] The information processing device calculates the cosine similarity between the sentence vector dv1 and each of the sentence vectors sv2-1 to wv2-4, and identifies sentences whose sentence vectors deviate from the sentence vector dv1 based on the cosine similarity as “characteristic sentences.” For example, the information processing device identifies as characteristic sentences sentences whose sentence vectors have a cosine similarity with the sentence vector dv1 that is equal to or greater than a threshold.

[0036] 4, the cosine similarity between the sentence vector dv1 and the sentence vector sv1-1, the cosine similarity between the sentence vector dv1 and the sentence vector sv1-3, and the cosine similarity between the sentence vector dv1 and the sentence vector sv1-4 are each set to a threshold or greater. Then, the information processing device identifies the sentence of the sentence vector sv1-1, the sentence of the sentence vector sv1-3, and the word of the sentence vector sv1-4 as characteristic sentences.

[0037] The information processing device searches the text DB 50 for a sentence corresponding to each characteristic sentence by executing the process described in FIG. 3 for each characteristic sentence. In the example shown in FIG. 4, sentences X1, X2, X3, and X4 are searched for as search candidates corresponding to the characteristic sentence "Its characteristic function is that." Sentences X2, X3, X6, and X10 are searched for as search candidates corresponding to the characteristic sentence "It is made up of multiple sub-projects." Sentences X2, X3, X7, and X22 are searched for as search candidates corresponding to the characteristic sentence "It has achieved high speed."

[0038] The information processing device identifies sentences that are common to the search candidates of each characteristic sentence as the final search result. In the example shown in Figure 4, sentences X2 and X3 that are common to each search candidate are output as the final search result.

[0039] As described above, when the search query q2 contains multiple sentences, the information processing device identifies characteristic sentences and identifies common sentences among the search results corresponding to the characteristic sentences as the final search results. This makes it possible to efficiently search for sentences corresponding to the search query q2 even if the search query q2 contains multiple sentences.

[0040] Next, a configuration example of an information processing device that executes the above-mentioned processing will be described. Fig. 5 is a functional block diagram showing the configuration of an information processing device according to this embodiment. As shown in Fig. 5, the information processing device 100 has a communication unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150.

[0041] The communication unit 110 is connected to an external device or the like via a wired or wireless connection, and transmits and receives information to and from the external device or the like. For example, the communication unit 110 is realized by a NIC (Network Interface Card) or the like. The communication unit 110 may be connected to a network (not shown).

[0042] The input unit 120 is an input device that inputs various types of information to the information processing device 100. The input unit 120 corresponds to a keyboard, a mouse, a touch panel, etc. For example, a user may operate the input unit 120 to input data such as a sentence or a search query.

[0043] The display unit 130 is a display device that displays information output from the control unit 150. The display unit 130 corresponds to a liquid crystal display, an organic EL (Electro Luminescence) display, a touch panel, etc. For example, search results of a search query are displayed on the display unit 130.

[0044] The storage unit 140 includes a word vector dictionary 40, a text DB 50, a word cluster dictionary 60, a sentence cluster dictionary 70, and an inverted index 80. The storage unit 140 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk.

[0045] The word vector dictionary 40 is a table that defines codes and word vectors assigned to words. FIG. 6 is a diagram showing an example of the data structure of the word vector dictionary. As shown in FIG. 6, the word vector dictionary 40 has codes, words, and word vectors (1) to (7). A code is a code assigned to a word. A word is a word included in a character string. Word vectors (1) to (7) are vectors assigned to words. The n-th component of a word vector is represented as a word vector (n) (n=1 to 7).

[0046] The text DB 50 is a database that stores multiple sentences. For example, the text DB 50 includes multiple records. Each record includes multiple sentences.

[0047] In the word cluster dictionary 60, multiple words are classified into multiple clusters, and each cluster is assigned a word cluster ID. Multiple words classified into the same cluster have a cosine similarity between the word vectors of each word that is equal to or greater than a threshold. Other explanations regarding the word cluster dictionary 60 are the same as those given in FIG. 2.

[0048] The sentence cluster dictionary 70 associates sentence cluster IDs, which identify sentence clusters, with sets of word cluster IDs. The same sentence cluster ID is set for multiple sentences that belong to the same sentence cluster. Other explanations regarding the sentence cluster dictionary 70 are the same as those given in FIG. 2.

[0049] The inverted index 80 associates a sentence cluster ID with the position of a sentence belonging to the sentence cluster ID (position in the text DB 50). FIG. 7 is a diagram showing the data structure of the inverted index. As shown in FIG. 7, this inverted index 80 has a plurality of pairs of record pointers and position pointers associated with sentence cluster IDs. The record pointer indicates the position of the corresponding record. The position indicated by the record pointer is defined by the number of words (offset) from the first word of the text DB 50 to the first word of the record. The position pointer indicates the position of the corresponding sentence. The position pointer is defined by the offset from the first word of the record including the corresponding sentence to the first word of the corresponding sentence.

[0050] For example, if the sentence cluster ID of the sentence "Horses like sweet carrots" is "Cr1" and the sentence "Horses like sweet carrots" is included in record R1, the settings are as follows: That is, the record pointer (1) corresponding to the sentence cluster ID "Cr1" is set to the offset of record R1. The position pointer (1) corresponding to the sentence cluster ID "Cr1" is set to the offset of the sentence "Horses like sweet carrots."

[0051] The data structure of the transposed index 80 is not limited to that shown in FIG. 7, but may simply associate a sentence cluster ID with the offset of each sentence belonging to the sentence cluster ID.

[0052] Returning to the description of Fig. 5, the control unit 150 includes an acquisition unit 151, a preprocessing unit 152, and a search unit 153. The control unit 150 is realized by, for example, a central processing unit (CPU) or a micro processing unit (MPU). The control unit 150 may also be implemented by an integrated circuit such as, for example, an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0053] Acquisition unit 151 acquires various types of information via communication unit 110 or input unit 120. For example, when acquisition unit 151 acquires record information, acquisition unit 151 registers the acquired record information in text DB 50.

[0054] The preprocessing unit 152 executes the preparatory phase process described above. The preprocessing unit 152 acquires sentences from the text DB and identifies characteristic words contained in the sentences by executing the process described in FIG. 1. After identifying characteristic words, the preprocessing unit 152 executes sentence clustering as described in FIG. 2. For example, the preprocessing unit 152 identifies word cluster IDs set for clusters to which each characteristic word belongs based on the word cluster dictionary 60. The preprocessing unit 152 identifies the sentence cluster ID to which the sentence belongs based on the pair of identified word cluster IDs and the sentence cluster dictionary 70. The preprocessing unit 152 associates the cluster ID of the sentence with a record pointer and a position pointer that can identify the position of the sentence, and sets them in the transposed index 80.

[0055] The preprocessing unit 152 repeatedly executes the above process for each sentence registered in the text DB 50.

[0056] The search unit 153 executes the process of the search phase described above. The search unit 153 acquires a search query via the communication unit 110 or the input unit 120. The search unit 153 determines whether the search query contains one sentence or multiple sentences.

[0057] A case where a search query includes a single sentence will be described. When the search query includes one sentence, the search unit 153 executes the process described with reference to FIG. 3. For example, the search unit 153 identifies characteristic words from the sentence included in the search query. The search unit 153 identifies a set of word cluster IDs for each characteristic word based on each characteristic word and the word cluster dictionary 60. The search unit 153 identifies a sentence cluster ID corresponding to the search query based on the set of word cluster IDs and the sentence cluster dictionary 70.

[0058] The search unit 153 identifies a set of a record pointer and a position pointer corresponding to the sentence cluster ID based on the sentence cluster ID corresponding to the search query and the transposed index 80. The search unit 153 acquires a sentence (a plurality of sentences) corresponding to the identified set of record pointer and position pointer from the text DB 50, and displays the sentences as search results on the display unit 130. The search unit 153 may notify an external device of the search results.

[0059] Next, a case where a search query includes multiple sentences such as paragraphs or items will be described. When a search query includes multiple sentences, the search unit 153 executes the process described with reference to FIG. 4. The search unit 153 identifies a characteristic sentence from the multiple sentences included in the search query. The search unit 153 identifies a sentence cluster ID of each characteristic sentence based on a set of word cluster IDs corresponding to each characteristic sentence and the sentence cluster dictionary 70. The search unit 153 acquires multiple sentences (search results) corresponding to each characteristic sentence from the text DB 50 based on the sentence cluster ID of each characteristic sentence and the transposed index 80. The search unit 153 searches for sentences common to the search results corresponding to each characteristic sentence as the final search result. The search unit 153 displays the search results on the display unit 130. The search unit 153 may notify an external device of the search results.

[0060] Next, an example of a processing procedure of the information processing device according to this embodiment will be described. Fig. 8 is a flowchart showing the processing procedure of the preparation phase of the information processing device according to this embodiment. As shown in Fig. 8, the preprocessing unit 152 of the information processing device 100 acquires an unprocessed sentence from the text DB 50 (step S101). The preprocessing unit 152 integrates word vectors of multiple words included in the sentence based on the word vector dictionary 40 to calculate a sentence vector (step S102).

[0061] The preprocessing unit 152 identifies characteristic words based on the cosine similarity between the sentence vector and the word vector of each word (step S103). The preprocessing unit 152 identifies word cluster IDs for the characteristic words based on the word cluster dictionary 60 (step S104).

[0062] The preprocessing unit 152 identifies the sentence cluster ID of the cluster to which the sentence belongs based on the pair of word cluster IDs for the characteristic words of the sentence and the sentence cluster dictionary 70 (step S105). The preprocessing unit 152 associates the position information of the sentence (the pair of record pointer and position pointer) with the sentence cluster ID and registers them in the inverted index 80 (step S106).

[0063] If an unprocessed sentence exists in the text DB 50 (step S107, Yes), the preprocessing unit 152 proceeds to step S101. On the other hand, if an unprocessed sentence does not exist in the text DB 50 (step S107, No), the preprocessing unit 152 ends the processing of the preparation phase.

[0064] 9 is a flowchart showing the processing procedure of the search phase of the information processing device according to this embodiment. As shown in FIG. 9, the search unit 153 of the information processing device 100 receives a search query (step S201). The search unit 153 determines whether the search query includes multiple sentences (step S202).

[0065] If the search query does not include multiple sentences (No at Step S202), the search unit 153 proceeds to Step S203. Based on the word vector dictionary 40, the search unit 153 adds up the word vectors of multiple words included in the sentences of the search query to calculate a sentence vector (Step S203).

[0066] The search unit 153 identifies characteristic words included in the sentence of the search query based on the cosine similarity between the sentence vector and the word vector of each word (step S204).The search unit 153 identifies word cluster IDs for the characteristic words based on the word cluster dictionary 60 (step S205).

[0067] The search unit 153 identifies the sentence cluster ID of the cluster to which the search query sentence belongs based on the set of word cluster IDs for the characteristic words of the search query sentence and the sentence cluster dictionary 70 (step S206).The search unit 153 identifies the position information of the sentence corresponding to the sentence cluster ID based on the sentence cluster ID of the search query sentence and the transposed index 80 (step S207).

[0068] The search unit 153 acquires the sentence at the position corresponding to the position information from the text DB 50 (step S208), and outputs the search result (step S209).

[0069] On the other hand, if the search query contains multiple sentences in step S202 (step S202, Yes), the search unit 153 proceeds to step S210. The search unit 153 executes a search process based on the multiple sentences (step S210), and proceeds to step S209.

[0070] Here, an example of the processing procedure for the search process based on multiple sentences shown in step S210 of Fig. 9 will be described. Fig. 10 is a flowchart showing the processing procedure for the search process based on multiple sentences. As shown in Fig. 10, the search unit 153 of the information processing device 100 selects an unselected sentence from the multiple sentences included in the search query (step S301).

[0071] The search unit 153 calculates a sentence vector by accumulating word vectors of multiple words included in the selected sentence based on the word vector dictionary 40 (step S302). The search unit 153 identifies characteristic words included in the sentence based on the cosine similarity between the sentence vector and the word vector of each word (step S303). The search unit 153 identifies word cluster IDs for characteristic words based on the word cluster dictionary 60 (step S304).

[0072] The search unit 153 identifies the sentence cluster ID of the cluster to which the sentence belongs based on the set of word cluster IDs for the characteristic words of the sentence and the sentence cluster dictionary 70 (step S305).The search unit 153 identifies the position information of the sentence corresponding to the sentence cluster ID based on the sentence cluster ID of the sentence and the transposed index 80 (step S306).

[0073] The search unit 153 acquires the sentence (search result) at the position corresponding to the position information from the text DB 50 (step S307).

[0074] If an unprocessed sentence exists in the search query (step S308, Yes), the search unit 153 proceeds to step S301. If an unprocessed sentence does not exist in the search query (step S308, No), the search unit 153 sets a sentence that is common to the search results of each sentence included in the search query as the final search result (step S309), and ends the search process based on multiple sentences.

[0075] Next, the effects of the information processing device 100 according to this embodiment will be described. The information processing device 100 identifies a set of characteristic words from a plurality of words included in a sentence, and identifies the sentence cluster ID of the cluster to which the sentence belongs based on the set of characteristic words and the sentence cluster dictionary 70. This allows the sentences to be clustered appropriately.

[0076] The information processing device 100 calculates the cosine similarity between the sentence vector of a sentence and the word vectors of multiple words, and identifies, as characteristic words, words in the word vector whose cosine similarity with the sentence vector is equal to or greater than a threshold value. This makes it possible to identify characteristic words that deviate from the sentence vector.

[0077] The information processing device 100 generates an inverted index 80 by associating the sentence cluster ID of the cluster to which the sentence belongs with the position information of the sentence. By using the inverted index 80, it becomes possible to easily identify the position information of multiple sentences that belong to the same sentence cluster ID.

[0078] The information processing device 100 identifies a set of characteristic words from multiple words included in a search query q1 having one sentence, and identifies a sentence cluster ID corresponding to the search query q1 based on the set of characteristic words and the sentence cluster dictionary 70. Then, the information processing device 100 performs a search based on an inverted index 80 created in advance and the sentence cluster ID corresponding to the search query q1. This makes it possible to appropriately search for a sentence corresponding to the search query q1.

[0079] When the search query q2 contains multiple sentences, the information processing device 100 identifies characteristic sentences and identifies, as the final search result, sentences that are common among the search results corresponding to the characteristic sentences. This makes it possible to efficiently search for sentences that correspond to the search query q2, even if the search query q2 contains multiple sentences.

[0080] The above-described processing of the information processing device 100 is an example, and the information processing device 100 may execute other processing. The other processing of the information processing device 100 will be described below.

[0081] When the search unit 153 of the information processing device 100 receives a search query having multiple sentences, it identifies multiple characteristic sentences whose cosine similarity is equal to or greater than a threshold, and detects, as the final search result, a sentence that is common to search results using each characteristic sentence. Here, the search unit 153 may further execute a process of increasing or decreasing the number of characteristic sentences by receiving a change in the threshold used for comparison with the cosine similarity.

[0082] For example, the search unit 153 receives, from the input unit 120, a change to the threshold value used when identifying a characteristic sentence, and repeatedly executes a process of displaying the relationship between the changed threshold value and the characteristic sentence on the display unit 130. The larger the threshold value, the fewer the number of characteristic sentences, and the smaller the threshold value, the more the number of characteristic sentences. The search unit 153 confirms the characteristic sentence when it receives a confirmation instruction from the input unit 120. The process after the search unit 153 confirms the characteristic sentence is the same as that of the conventional technology described above.

[0083] Furthermore, when the above process is executed by the search unit 153 of the information processing device 100, a zoom-in / out function can be realized that increases or decreases the number of search candidates by increasing or decreasing the number of characteristic sentences included in a search query such as a paragraph or item.

[0084] Furthermore, while the information processing device 100 described above has been described as clustering multiple sentences in a text string, similar processing can also be performed on information other than sentences, such as the protein primary structure of a genome base sequence or the functional group primary structure of an organic compound's chemical structural formula. For example, the primary structure of a protein contains multiple consecutive Kmer sequences that repeatedly appear. In the following description, a Kmer sequence will be referred to as the "basic structure" of a protein. Note that the "basic structure" of a protein may also be represented by a consecutive amino acid sequence, such as an oligopeptide.

[0085] 11 is a diagram for explaining other processes of the information processing device. As shown in FIG. 11, the primary structure Pro1 of the protein includes multiple basic structures "α-Kmer", "β-Kmer", "γ-Kmer", and "δ-Kmer".

[0086] The information processing device 100 identifies the vector of each basic structure included in the primary structure Pro1 of the protein based on a basic structure vector dictionary that defines basic structures and vectors of the basic structures. For example, the vector of the basic structure "α-Kmer" is defined as v1. The vector of the basic structure "β-Kmer" is defined as v2. The vector of the basic structure "γ-Kmer" is defined as v3. The vector of the basic structure "δ-Kmer" is defined as v4. The information processing device 100 calculates the vector tv1 of the primary structure Pro1 by accumulating the vectors of each basic structure included in the primary structure Pro1 of the protein.

[0087] The information processing device 100 calculates the cosine similarity between the vector tv1 and each of the vectors v1 to v4, and identifies the basic structure of the vector deviating from the vector tv1 based on the cosine similarity as a “characteristic basic structure.” For example, the information processing device identifies the basic structure of the vector whose cosine similarity with the vector tv1 is equal to or greater than a threshold as a characteristic basic structure.

[0088] 11, the cosine similarity between vector tv1 and vector v1, the cosine similarity between vector tv1 and vector v3, and the cosine similarity between vector tv1 and vector v4 are each set to a threshold or greater. Then, the information processing device identifies the basic structure "α-Kmer" of vector v1, the basic structure "γ-Kmer" of vector v3, and the basic structure "δ-Kmer" of vector v4 as characteristic basic structures.

[0089] The information processing device 100 clusters the primary structures based on the characteristic basic structures identified in the above process. Specifically, the process is similar to that shown in FIG. 2, in which a basic structure cluster ID is assigned to each characteristic basic structure, and the cluster ID of the primary structure is identified based on the set of basic structure cluster IDs. Other processes are similar to those described in FIG. 2, except that the characteristic words are replaced with characteristic basic structures and the sentences are replaced with primary structures. Similarly, the search process is similar to those described in FIGS. 3 and 4, in which the characteristic words are replaced with characteristic basic structures and the sentences are replaced with primary structures. Using an inverted index associated with the cluster IDs of protein primary structures, similar receptors can be searched for in response to a search query for a receptor composed of multiple primary structures. This can be applied to search for receptors similar to the receptor that is the target of a biopharmaceutical ligand, thereby enabling the prediction of side effects of the biopharmaceutical.

[0090] Next, an example of the hardware configuration of a computer that realizes the same functions as the information processing device 100 described in the above embodiment will be described. Fig. 12 is a diagram showing an example of the hardware configuration of a computer that realizes the same functions as the information processing device of the embodiment.

[0091] 12, computer 200 has CPU 201 that executes various types of arithmetic processing, input device 202 that accepts data input from a user, and display 203. Computer 200 also has communication device 204 that exchanges data with external devices via a wired or wireless network, and interface device 205. Computer 200 also has RAM 206 that temporarily stores various types of information, and hard disk drive 207. Each of devices 201 to 207 is connected to bus 208.

[0092] The hard disk drive 207 stores an acquisition program 307a, a preprocessing program 207b, and a search program 207c. The CPU 201 reads out each of the programs 207a to 207c and loads them into the RAM 206.

[0093] The acquisition program 207a functions as the acquisition process 206a. The pre-processing program 207b functions as the pre-processing process 206b. The search program 207c functions as the search process 206c.

[0094] The processing of the acquisition process 206a corresponds to the processing of the acquisition unit 151. The processing of the pre-processing process 206b corresponds to the processing of the pre-processing unit 152. The processing of the search process 206c corresponds to the processing of the search unit 153.

[0095] It should be noted that each of the programs 207a to 207c does not necessarily have to be stored in the hard disk device 207 from the beginning. For example, each of the programs may be stored in a "portable physical medium" such as a flexible disk (FD), CD-ROM, DVD, magneto-optical disk, or IC card that is inserted into the computer 200. Then, the computer 200 may read and execute each of the programs 207a to 207c. [Explanation of symbols]

[0096] 40-word vector dictionary 50 Text DB 60 word cluster dictionary 70-sentence cluster dictionary 80 Inverted Index 100 Information processing device 110 Communications Department 120 Input section 130 Display section 140 Storage section 150 control section 151 Acquisition Department 152 Preprocessing section 153 Search Department

Claims

1. Obtaining a plurality of sentences having a plurality of words; performing a process for identifying a set of characteristic words from a plurality of words based on a sentence vector of the sentence having a plurality of words and word vectors of the plurality of words, on the plurality of sentences; The plurality of sentences are classified so that sentences having the same set of feature words are included in the same cluster. An information processing program that causes a computer to execute a process.

2. The information processing program described in claim 1, characterized in that the process of identifying the set of characteristic words calculates the cosine similarity between the sentence vector of the sentence and the word vectors of the multiple words, and identifies as characteristic words words of word vectors whose cosine similarity with the sentence vector is greater than or equal to a threshold.

3. the plurality of sentences are registered in a storage device; 2. The information processing program according to claim 1, further causing a computer to execute a process of generating inverted index information that associates identification information that identifies a cluster to which the sentence belongs with position information of the sentence in the storage device, based on the classification result of the classification process.

4. The information processing program of claim 3, further causing a computer to execute the following process: when a search statement having multiple words is received, a set of characteristic words is identified from the multiple words of the search statement based on a sentence vector of the search statement and a word vector of the words of the search statement, and a search for a sentence corresponding to the search statement from the storage device based on cluster identification information for the identified set of characteristic words and the transposed index information.

5. 5. The information processing program according to claim 4, further causing a computer to execute the following process when a plurality of search sentences are received: identifying a plurality of characteristic sentences based on sentence vectors of the plurality of search sentences; identifying sets of characteristic words from the plurality of characteristic sentences; identifying cluster identification information for each characteristic sentence based on the sets of characteristic words corresponding to the characteristic sentences; and searching the storage device for common sentences that correspond to each characteristic sentence based on the cluster identification information for each characteristic sentence and the transposed index information.

6. 6. The information processing program according to claim 5, further causing a computer to execute a process of increasing or decreasing the number of characteristic sentences identified from the plurality of search sentences.

7. Obtaining a plurality of sentences having a plurality of words; performing a process for identifying a set of characteristic words from a plurality of words based on a sentence vector of the sentence having a plurality of words and word vectors of the plurality of words, on the plurality of sentences; The plurality of sentences are classified so that sentences having the same set of feature words are included in the same cluster. An information processing method characterized in that the processing is executed by a computer.

8. Obtaining a plurality of sentences having a plurality of words; performing a process for identifying a set of characteristic words from a plurality of words based on a sentence vector of the sentence having a plurality of words and word vectors of the plurality of words, on the plurality of sentences; The plurality of sentences are classified so that sentences having the same set of feature words are included in the same cluster. An information processing device comprising a control unit that executes processing.

Images