Vector expression processing device, vector expression processing method and program

Abstract

To process vector expression of a word phrase so as to be able to judge high-order / low-order relation with other words.SOLUTION: A control section that a vector expression processing apparatus considers vector expression showing characteristics of a word phrase as an input, and makes inference based on dictionary data including a combination of a high-order word and a low-order word, and maps an arbitrary word phrase to a distribution space indicating vector expression of a plurality of words, where the distribution space is arranged so as to have a component related to degree of mutual relevance and a component related to mutual high-order / low-order relation.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a vector expression processing device, a vector expression processing method, and a program. [Background technology]

[0002] Thesauruses are used that express synonymous and entailed relationships between words in a hierarchical structure. To facilitate the construction and updating of thesauruses, research is being conducted into techniques for estimating synonymous or entailed relationships between two words. For example, Non-Patent Document 1 discloses a technique for processing vector representations of words into vector representations that emphasize hierarchical relationships based on the hierarchical structure of a thesaurus. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Ivan Vulic, Nikola Mrksic, "Specializing Word Vectors for Lexical Entailment," Proceedings of NAACL-HLT 2018, pages 1134-1145, 2018. Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology has a problem in that it is not possible to determine the hierarchical relationship of words that are not included in the thesaurus.

[0005] The present disclosure processes the vector representation of a phrase so that its hierarchical relationship with other words can be determined. [Means for solving the problem]

[0006] A vector expression processing device according to a first aspect of the present disclosure is a vector expression processing device having a control unit, which takes as input a vector expression representing the characteristics of a phrase, performs inference based on dictionary data including combinations of hypernyms and hyponyms, and maps any phrase into a distribution space showing the vector expressions of multiple words, and the distribution space is arranged so as to have components related to the degree of relevance between the words and components related to their hypernym-hypernym relationships.

[0007] According to the first aspect of the present disclosure, the vector representation of a phrase is processed so that its hierarchical relationship with other words can be determined.

[0008] A second aspect of the present disclosure is a vector expression processing device according to the first aspect, wherein the component relating to the mutual relevance is the smallness of the angle formed by the words, and the component relating to the superordinate-subordinate relationship is the norm length.

[0009] A third aspect of the present disclosure is the vector representation processing device according to the first and second aspects, wherein the distribution space is a Euclidean space.

[0010] A fourth aspect of the present disclosure is a vector expression processing device relating to the first to third aspects, wherein the control unit maps words into the distribution space using a machine learning model trained using training data that associates multiple combinations of the hypernyms and hyponyms with the hypernym-hypernym relationships represented by the combinations.

[0011] A fifth aspect of the present disclosure is a vector expression processing device according to the first to fourth aspects, wherein the control unit structurally visualizes the distribution of the words mapped into the distribution space.

[0012] A sixth aspect of the present disclosure is the vector expression processing device according to the fifth aspect, wherein the control unit expresses the distribution of the words and phrases in a list format.

[0013] A seventh aspect of the present disclosure is the vector expression processing device according to the fifth aspect, wherein the control unit expresses the distribution of the words and phrases in a tree structure.

[0014] An eighth aspect of the present disclosure is the vector expression processing device according to the seventh aspect, wherein the tree structure is hierarchical according to the hierarchical relationship of the words and phrases.

[0015] A ninth aspect of the present disclosure is the vector expression processing device according to the fifth aspect, wherein the control unit expresses the distribution of the words and phrases in a scatter diagram with a predetermined axis representing the degree of ranking.

[0016] A tenth aspect of the present disclosure is a vector expression processing device according to the first to ninth aspects, wherein the control unit accepts input of two terms and determines the hierarchical relationship between the two terms mapped to the distribution space based on an index calculated from the angle between the two terms and a norm.

[0017] An eleventh aspect of the present disclosure is a vector expression processing device according to the first to ninth aspects, wherein the control unit accepts input of two terms, and determines whether or not there is a superordinate / subordinate relationship between the two terms mapped to the distribution space based on an index calculated from the angle between the two terms and the norm, and if there is a superordinate / subordinate relationship and the magnitude relationship between the norms of the two terms is within a predetermined range, determines that the two terms are synonymous.

[0018] A twelfth aspect of the present disclosure is a vector expression processing device according to the first to ninth aspects, wherein the control unit accepts input of two terms, and determines whether or not there is a superordinate-subordinate relationship between the two terms mapped to the distribution space based on an index calculated from the angle between the two terms and the norm, and if there is a superordinate-subordinate relationship and the magnitude relationship between the norms of the two terms is outside a predetermined range, the control unit determines that the two terms are in a superordinate-subordinate relationship.

[0019] A thirteenth aspect of the present disclosure is a vector expression processing device relating to the first to twelfth aspects, wherein the control unit accepts input of a plurality of terms, generates a second distribution space in which terms that are highly related to each other among the plurality of terms are arranged close to each other, and maps the plurality of terms included in the second distribution space into the distribution space.

[0020] A vector expression processing method according to a fourteenth aspect of the present disclosure includes a control unit of a vector expression processing device that takes as input a vector expression representing the characteristics of a phrase and performs inference based on dictionary data including combinations of hypernyms and hyponyms, and maps an arbitrary phrase into a distribution space showing the vector expressions of multiple words, and the distribution space is arranged so as to have components related to the degree of mutual association and components related to the mutual hypernym-hypernym relationship.

[0021] A program according to a fifteenth aspect of the present disclosure causes a control unit of a vector expression processing device to execute the following steps: input a vector expression representing the characteristics of a phrase, and perform inference based on dictionary data including combinations of hypernyms and hyponyms; and map an arbitrary phrase into a distribution space showing the vector expressions of multiple words, the distribution space being arranged to have components related to the degree of relevance to each other and components related to the hypernym-hypernym relationship to each other. [Brief explanation of the drawings]

[0022] [Figure 1] FIG. 2 is a block diagram showing an example of a hardware configuration of a vector representation processing device. [Figure 2] FIG. 2 is a block diagram showing an example of a functional configuration of a vector representation processing device. [Figure 3] 10 is a flowchart showing the procedure of a learning process. [Figure 4] FIG. 2 is a conceptual diagram illustrating an example of dictionary data. [Figure 5] FIG. 1 is a conceptual diagram illustrating an example of a distribution space of word embeddings. [Figure 6] FIG. 10 is a conceptual diagram illustrating an example of vector representation processing. [Figure 7]FIG. 1 is a conceptual diagram illustrating an example of a distribution space of a hierarchical embedded representation. [Figure 8] 10 is a flowchart showing a procedure of processing. [Figure 9] FIG. 1 is a conceptual diagram illustrating an example of a distribution space of a hierarchical embedded representation. [Figure 10] 10 is a flowchart showing the procedure of a determination process. [Figure 11] 10 is a flowchart showing the procedure of an extraction process. [Figure 12] 10 is a flowchart showing the procedure of a visualization process. [Figure 13] FIG. 10 is a conceptual diagram illustrating an example of visualization processing. [Figure 14] FIG. 10 is a conceptual diagram illustrating an example of visualization processing. [Figure 15] FIG. 10 is a conceptual diagram illustrating an example of visualization processing. [Figure 16] FIG. 10 is a conceptual diagram showing a modified example of the distribution space of the hierarchical embedded representation. DETAILED DESCRIPTION OF THE INVENTION

[0023] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0024] [Embodiment] <System configuration> The vector representation processing device in this embodiment is a language processing device that processes vector representations of words into vector representations that can determine the superordinate-subordinate relationships between words. Hereinafter, the vector representations before processing will also be referred to as "word embedded representations," and the vector representations after processing will also be referred to as "hierarchical structure embedded representations."

[0025] The vector representation processing device executes a learning process, a processing process, a determination process, an extraction process, and a visualization process. In the learning process, the vector representation processing device uses dictionary data to learn a processing model that maps the distribution space of word embeddings to the distribution space of hierarchical embedded representations.

[0026] In the processing process, the vector representation processing device uses the processing model learned in the learning process to map the distribution space of word embedded representations in which one or more input words are arranged into the distribution space of hierarchical embedded representations.

[0027] In the determination process, the vector expression processing device determines the synonymous relationship or the superordinate / subordinate relationship between the two input words and phrases based on the distribution space of the hierarchical embedded expression in which the two input words and phrases are arranged. Note that in the determination process, it may be determined only whether or not the two input words and phrases are in a superordinate / subordinate relationship, or it may be determined whether or not the two input words and phrases are in a superordinate / subordinate relationship, and if so, determine the superordinate / subordinate relationship between the words and phrases.

[0028] In the extraction process, the vector expression processing device extracts words that are in a hierarchical or subordinate relationship with an input phrase or words that are highly related to that phrase from the distribution space of the hierarchical embedded expression in which the input phrase is placed.

[0029] In the visualization process, the vector representation processing device structurally visualizes and outputs the distribution space of the hierarchically structured embedded representation.

[0030] <Hardware configuration> Fig. 1 is a block diagram showing an example of the hardware configuration of a vector representation processing device 10 according to this embodiment. As shown in Fig. 1, the vector representation processing device 10 includes a processor 101, a memory 102, an auxiliary storage device 103, an operation device 104, a display device 105, a communication device 106, and a drive device 107. The hardware components of the vector representation processing device 10 are interconnected via a bus 108.

[0031] The processor 101 has various arithmetic devices such as a CPU (Central Processing Unit), etc. The processor 101 reads out various programs installed in the auxiliary storage device 103 onto the memory 102 and executes them.

[0032] The memory 102 has a main storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The processor 101 and the memory 102 form a so-called computer (hereinafter also referred to as a "controller"), and the processor 101 executes various programs read onto the memory 102, thereby enabling the computer to realize various functions.

[0033] The auxiliary storage device 103 stores various programs and various data used when the processor 101 executes the various programs.

[0034] The operation device 104 is an operation device for allowing a user of the vector representation processing device 10 to perform various operations. The display device 105 is a display device for displaying the processing results of various processes executed by the vector representation processing device 10.

[0035] The communication device 106 is a communication device for communicating with external devices via a network (not shown).

[0036] Drive device 107 is a device for loading storage medium 109. The storage medium 109 here includes media that store information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. Storage medium 109 may also include semiconductor memories that store information electrically, such as ROMs and flash memories.

[0037] The various programs to be installed in the auxiliary storage device 103 are installed, for example, by setting the distributed storage medium 109 in the drive device 107 and reading out the various programs stored in the storage medium 109 by the drive device 107. Alternatively, the various programs to be installed in the auxiliary storage device 103 may be installed by being downloaded from a network via the communication device 106.

[0038] <Functional configuration> 2 is a block diagram showing an example of the functional configuration of a vector representation processing device according to this embodiment. As shown in FIG. 2, a vector representation processing device 10 according to this embodiment includes an input unit 11, a conversion model storage unit 12, a conversion unit 13, a learning unit 14, a processing model storage unit 15, a processing unit 16, a distribution space storage unit 17, a determination unit 18, and an output unit 19.

[0039] The conversion model storage unit 12, the processing model storage unit 15, and the distribution space storage unit 17 are realized by the memory 102 or the auxiliary storage device 103 shown in Fig. 1. The input unit 11, the conversion unit 13, the learning unit 14, the processing unit 16, the determination unit 18, and the output unit 19 are realized by the processor 101 shown in Fig. 1 executing various programs read onto the memory 102.

[0040] The input unit 11 accepts input of dictionary data in the learning process. The input unit 11 accepts input of one or more words and phrases in the processing process. The input unit 11 accepts input of two words and phrases in the determination process. The input unit 11 accepts input of one word and phrase in the extraction process.

[0041] The conversion model storage unit 12 stores trained conversion models. The conversion model is a machine learning model that converts words into vector representations by arranging them in a multidimensional vector space. Examples of the conversion model include Bag of Words, Word2Vec, and fastText.

[0042] The conversion model in this embodiment is FastText. Details of FastText are disclosed in Reference 1 below.

[0043] [Reference 1] Bojanowski, P., Grave, E., Joulin, A., and Mikolov, T, "Enriching Word Vectors with Subword Information," TACL, Vol. 5, pp. 135-146, 2017.

[0044] The conversion unit 13 converts a word included in the dictionary data received by the input unit 11 or a phrase received by the input unit 11 into a word embedding representation, using the conversion model stored in the conversion model storage unit 12.

[0045] The learning unit 14 learns a processed model based on the dictionary data received by the input unit 11. The processed model is a machine learning model that maps the distribution space of word embeddings to the distribution space of hierarchical embedded representations. An example of the distribution space is Euclidean space. An example of the processed model is a fully-connected neural network (FNN).

[0046] The machining model storage unit 15 stores the machining model learned by the learning unit 14.

[0047] The processing unit 16 processes the word embedding representations generated by the conversion unit 13 into hierarchical embedded representations by inference using the processing model stored in the processing model storage unit 15.

[0048] Inference using a processed model is a method of arranging or rearranging distributed representations in a distribution space according to a specified algorithm, using information representing the relationships between words contained in dictionary data, or phrase or sentence information describing words. The algorithm used for inference includes methods such as specified rules, formalized calculations, or machine learning.

[0049] The distribution space storage unit 17 stores information representing a distribution space in which the hierarchical structure embedded representation generated by the processing unit 16 is arranged.

[0050] In the determination process, the determination unit 18 determines the synonymous relationship or the superordinate / subordinate relationship between two words and phrases received by the input unit 11 based on the distribution space of the hierarchical embedded expressions stored in the distribution space storage unit 17. In the extraction process, the determination unit 18 extracts words that are in a superordinate / subordinate relationship with one word and phrase received by the input unit 11 or words that are highly related to one another from the distribution space of the hierarchical embedded expressions stored in the distribution space storage unit 17.

[0051] In the determination process, the output unit 19 outputs the determination result by the determination unit 18. In the extraction process, the output unit 19 outputs the words extracted by the determination unit 18. In the visualization process, the output unit 19 structurally visualizes and outputs the distribution space of the hierarchical embedded representation.

[0052] <Learning process flow> FIG. 3 is a flowchart showing an example of the flow of the learning process in this embodiment.

[0053] In step S1, the input unit 11 receives input of dictionary data. Next, the input unit 11 sends the received dictionary data to the conversion unit 13.

[0054] Dictionary data is data that includes a plurality of words and represents synonymous and hierarchical relationships between the words. An example of dictionary data is a thesaurus. Another example of dictionary data is an ontology. Dictionary data is not limited to these, and may be any data that represents synonymous and hierarchical relationships between words.

[0055] FIG. 4 is a conceptual diagram showing an example of a thesaurus. As shown in FIG. 4, the thesaurus is a data structure that represents synonymous and sub-synonymous relationships between words. The thesaurus may be a data structure in which synonymous and sub-synonymous relationships are expressed in a hierarchical structure, or may be a database that accumulates pairs of words that are synonymous or sub-synonymous. Based on the thesaurus, it is possible to extract combinations of words that are synonymous (synonymous pairs) and combinations of words that are sub-synonymous (sub-synonymous pairs). Of the sub-synonymous pairs, the one that is higher in the hierarchy of the thesaurus (the word on the right in the example of FIG. 4) is the hypernym, and the one that is lower in the hierarchy (the word on the left in the example of FIG. 4) is the hyponym.

[0056] Returning to FIG. 3, the explanation will be given. In step S2, the conversion unit 13 receives dictionary data from the input unit 11. Next, the conversion unit 13 reads out a conversion model from the conversion model storage unit 12. Subsequently, the conversion unit 13 converts each word included in the dictionary data into a word embedding by inference using the read out conversion model. Then, the conversion unit 13 sends the dictionary data and the word embeddings of each word to the learning unit 14. Furthermore, the conversion unit 13 stores information representing a distribution space of the word embeddings of each word in the distribution space storage unit 17. The information stored in the distribution space storage unit 17 includes, for example, information representing each axis of the distribution space of the word embeddings and the word embeddings of each word.

[0057] Fig. 5 is a conceptual diagram showing an example of a distribution space of word embeddings. As shown in Fig. 5, word embeddings of words that are highly related to each other are arranged close to each other in a multidimensional distribution space.

[0058] Returning to Figure 3, in step S3, the learning unit 14 receives the dictionary data and word embeddings for each word from the conversion unit 13. Next, the learning unit 14 extracts training data from the dictionary data. The training data is data that associates multiple combinations of hypernyms and hyponyms with the hypernym-hypernym relationships they represent.

[0059] The combination of a hypernym and a hyponym corresponds to the hypernym-hyponym pair shown in Figure 4. The hypernym-hyponym pair corresponds to the positional relationship of each word in the hypernym-hyponym pair (in the example of Figure 4, the hypernym is on the right and the hyponym is on the left).

[0060] Next, the learning unit 14 learns a processing model based on the extracted teacher data. Then, the learning unit 14 stores the learned processing model in the processing model storage unit 15.

[0061] The distribution space of the hierarchical embedded representation has components related to the relevance of words and components related to the hierarchical relationships of words. The component related to the relevance of words is, for example, the angle between two words. The component related to the hierarchical relationships of words is, for example, the magnitude of the word norm (the vector length of the distributed representation).

[0062] The processing model in this embodiment is a fully connected neural network. The input of the fully connected neural network in this embodiment is a word distributed representation, and the output is a hierarchical embedded representation. The activation function uses an identity function in the final layer and a ReLU function in the other layers.

[0063] The loss function consists of two terms, L, relating to the angle and norm, respectively. angle , L norm and the regularization term L reg Specifically, the loss function is A Let be a synonym pair, and B L is expressed as a superordinate-subordinate relation pair in Equation (1).

[0064]

number

[0065] Angular term L angle uses deep metric learning (RLL; Ranked List Loss). Details of deep metric learning are disclosed in Reference 2 below.

[0066] [Reference 2] Wang, X., Hua, Y., Kodirov, E., Hu, G., Garnier, R., and Robertson, NM, "Ranked List Loss for Deep Metric Learning," in CVPR, 2019.

[0067] Angular term L angle is expressed by equations (2) to (4).

[0068]

number

[0069] The term L related to the norm norm is the relation between all superordinate and subordinate pairs B L The distance D between semantic hierarchies is maximized with respect to the norm L norm is expressed by equations (5) and (6).

[0070]

number

[0071] Normalization term L reg is used together with the normalization factor λ(<<1) to prevent the norm of the hierarchical embedded representation from diverging. reg is expressed by equation (7).

[0072]

number

[0073] The learning unit 14 calculates each term L of the loss function. angle , L norm , L reg The parameters of a fully connected neural network are learned by using stochastic gradient descent on the

[0074] 6 is a conceptual diagram showing an example of vector representation processing. As shown in FIG. 6, the learning unit 14 performs angle learning 301, 302 and norm learning 303, 304. In angle learning, the processing model is trained so that the angle between words in a synonymous or hierarchical relationship becomes smaller (301) and the angle between words that do not have such a relationship becomes larger (302). In norm learning, the processing model is trained so that the norm of the hypernym becomes smaller (303) and the norm of the hyponym becomes larger (304). In norm learning, the processing model may be trained so that the norm of the hypernym becomes larger and the norm of the hyponym becomes smaller.

[0075] FIG. 7 is a conceptual diagram showing an example of a distribution space of hierarchical embedded representations. As shown in FIG. 7, the hierarchical embedded representations are arranged in a multidimensional distribution space so that words that are highly related to each other have a small angle (311), and words that are in a superordinate-subordinate relationship have norms (312, 313) that correspond to the superordinate-subordinate relationship. In the example of FIG. 7, the relationship is assumed to be a synonymous relationship (including a synonymous relationship), and the distribution space is arranged so that words that are similar in meaning form a small angle. Note that the relationship is not limited to a synonymous relationship, and may be any relationship between words that is represented in dictionary data.

[0076] The distribution space of the hierarchical embedded representation may be a distribution space generated by LEAR (Lexical Entailment Attract-Repel) disclosed in Non-Patent Document 1. LEAR processes word vectors so that the cosine similarity between hypernyms and hyponyms is large, and the norm of the hypernym is large and the norm of the hyponym is small. However, LEAR takes words and a thesaurus as input and can infer the hypernym-hyponym relationship only for words included in the thesaurus.

[0077] <Processing flow> FIG. 8 is a flowchart showing an example of the flow of processing in this embodiment.

[0078] In step S11, the input unit 11 accepts input of one or more phrases. A phrase is a word or a phrase. The input phrase is, for example, a phrase not included in the dictionary data. Next, the input unit 11 sends the accepted one or more input phrases to the conversion unit 13.

[0079] In step S12, the conversion unit 13 receives an input phrase from the input unit 11. Next, the conversion unit 13 reads a conversion model from the conversion model storage unit 12. Subsequently, the conversion unit 13 converts the input phrase into word embeddings by inference using the read conversion model. Then, the conversion unit 13 sends the word embeddings of the input phrase to the processing unit 16.

[0080] In step S13, the processing unit 16 receives the word embeddings of the input phrase from the conversion unit 13. Next, the processing unit 16 reads out the word embeddings of each word from the distribution space storage unit 17. Subsequently, the processing unit 16 arranges the input phrase and the word embeddings of each word in the distribution space.

[0081] Next, the processing unit 16 reads out the processed model from the processed model storage unit 15. Subsequently, the processing unit 16 maps the distribution space of the word embedding representations into the distribution space of the hierarchical embedded representations by inference using the read out processed model.

[0082] In step S14, the processing unit 16 stores information representing the distribution space of the hierarchically embedded representation in which the input phrase and each word are arranged, in the distribution space storage unit 17. The information stored in the distribution space storage unit 17 includes, for example, information representing each axis of the distribution space of the hierarchically embedded representation, and the hierarchically embedded representation of the input phrase and each word.

[0083] Fig. 9 is a conceptual diagram showing an example of a distribution space of a hierarchical embedded expression. Fig. 9 is an example of a distribution space in which words and phrases not included in the dictionary data are arranged. In Fig. 9, words and phrases not included in the dictionary data are shown shaded. As shown in Fig. 9, according to the vector expression processing device of this embodiment, words and phrases not included in the dictionary data are arranged in appropriate positions.

[0084] For example, the word "mouse" is positioned at a position where it has a small angle with "mammal" and a larger norm than "mammal." Also, for example, the word "warm-blooded animal" is positioned at a position where it has a small angle with "mammal" and a smaller norm than "mammal."

[0085] In the example of Figure 9, the words are arranged in the distribution space so that the norm of the hypernym is small and the norm of the hyponym is large, but the words may be arranged in the distribution space so that the norm of the hypernym is large and the norm of the hyponym is small. In this case, for example, the word "mouse" is arranged at a position where the angle with "mammal" is small and the norm is smaller than that of "mammal."

[0086] <Determination process flow> FIG. 10 is a flowchart showing an example of the flow of the determination process in this embodiment.

[0087] In step S21, the input unit 11 accepts the input of two words and phrases. Next, the input unit 11 sends the accepted two input words and phrases to the determination unit .

[0088] In step S22, the determination unit 18 receives input words and phrases from the input unit 11. Next, the determination unit 18 reads out information representing the distribution space stored in the distribution space storage unit 17. Subsequently, the determination unit 18 determines the relationship between the input words and phrases.

[0089] The determination unit 18 first determines whether the input words and phrases are in a synonymous or hyponym relationship. If the input words and phrases are in a hyponym relationship, the determination unit 18 determines whether each word and phrase is a hypernym or a hyponym. Therefore, the determination result by the determination unit 18 is either a synonymous relationship, a hypernym, a hyponym, or no relationship. The determination unit 18 then sends the determination result to the output unit 19.

[0090] Specifically, the determination unit 18 determines the relationship between input phrases as follows. First, the determination unit 18 calculates a predetermined index using the hierarchical embedded representation of the input phrase. The predetermined index may be, for example, a hyperscore. The hyperscore in this embodiment is calculated by l ,u r is expressed as a hierarchical embedded representation of the input phrase, as shown in Equation (8).

[0091]

number

[0092] The hyperscore in this embodiment is a modified version of the hyperscore disclosed in the following reference 3. Specifically, while the conventional hyperscore is multiplied by the norm ratio, this hyperscore is modified to be multiplied by the logarithm of the norm ratio. This modification makes it possible to prevent the norm ratio from diverging and also allows the threshold to be set more precisely.

[0093] [Reference 3] Nguyen, KA, Koper, M., Walde, im SS, and Vu, NT, "Hierarchical Embeddings for Hypernymy Detection and Directionality," EMNLP, pp. 233-243, 2017.

[0094] Next, the determination unit 18 compares the hyperscore with a predetermined threshold, and determines that there is a hierarchical / subordinate relationship if the hyperscore is equal to or greater than the threshold. On the other hand, if the hyperscore is less than the threshold, it determines that there is no hierarchical / subordinate relationship. The threshold may be set appropriately based on the hyperscores between words included in the dictionary data, etc.

[0095] Next, if the determination unit 18 determines that the input words and phrases are in a hierarchical relationship, it calculates the norm of each of the input words and phrases. Next, the determination unit 18 determines whether the magnitude relationship of the norms of each word and phrase (for example, the absolute value of the difference between the norms) is within a predetermined range. The thresholds representing the predetermined range (for example, the upper and lower limits of the difference between the norms) may be set appropriately based on the magnitude relationship of the norms between words included in the dictionary data.

[0096] The determination unit 18 determines that the input words are in a synonymous relationship when the magnitude relationship of the norms is within a predetermined range, whereas the determination unit 18 determines that the input words are in a hierarchical relationship when the magnitude relationship of the norms is outside the predetermined range.

[0097] If the determination unit 18 determines that the input words are in a hierarchical relationship, it compares the magnitudes of the norms of the input words. The determination unit 18 determines the word with the smaller norm as a hypernym and the word with the larger norm as a hyponym. If the processing model is trained so that hypernyms have larger norms and hyponyms have smaller norms, the determination unit 18 determines the word with the larger norm as a hypernym and the word with the smaller norm as a hyponym.

[0098] The output unit 19 receives the determination result from the determination unit 18. Next, the output unit 19 outputs the determination result to the display device 105 or the like.

[0099] <Extraction process flow> FIG. 11 is a flowchart showing an example of the flow of the extraction process in this embodiment.

[0100] In step S31, the input unit 11 accepts the input of one word or phrase. Next, the input unit 11 sends the accepted input word or phrase to the determination unit .

[0101] In step S32, the determination unit 18 receives the input phrase from the input unit 11. Next, the determination unit 18 reads information representing the distribution space stored in the distribution space storage unit 17. Next, the determination unit 18 extracts words that have a hierarchical relationship with the input phrase from the distribution space of the hierarchical embedded representation. Specifically, the determination unit 18 extracts words that have a hyperscore with the input phrase that is equal to or greater than a predetermined threshold from among the words included in the distribution space of the hierarchical embedded representation. Then, the determination unit 18 sends the extracted words to the output unit 19.

[0102] The determination unit 18 may extract words that are highly relevant to the input phrase from the distribution space of the hierarchical embedded expression. Specifically, the determination unit 18 extracts words that have angles with the input phrase and norm magnitude relationships with the input phrase that fall within a predetermined range from among the words included in the distribution space of the hierarchical embedded expression.

[0103] The determination unit 18 first extracts words whose cosine similarity with the input phrase is equal to or less than a predetermined threshold. Next, the determination unit 18 extracts words whose norm difference between each extracted word and the input phrase is greater than a predetermined threshold. The determination unit 18 then sends the extracted words to the output unit 19.

[0104] The output unit 19 receives the extracted words from the determination unit 18. Next, the output unit 19 outputs the extracted words to the display device 105 or the like.

[0105] <Visualization process flow> FIG. 12 is a flowchart showing an example of the flow of the visualization process in this embodiment.

[0106] In step S41, the output unit 19 reads out information representing the distribution space stored in the distribution space storage unit 17.

[0107] In step S42, the output unit 19 structurally visualizes the distribution space of the hierarchical embedded expressions. For example, the output unit 19 expresses the distribution space of the hierarchical embedded expressions in a list format that lists words in a hierarchical-subordinate relationship.

[0108] Figure 13 is a conceptual diagram showing an example of visualization processing. The example in Figure 13 shows an example of expressing the distribution space of a hierarchically embedded expression in the form of a list enumerating words in a hierarchical hierarchy. As shown in Figure 13, when expressing the distribution space in list form, a list consisting of combinations of hypernyms and hyponyms is constructed. In this case, by sorting the words in descending order of their degree of hierarchy, the hierarchical structure can be easily understood from the list.

[0109] The degree of hierarchy indicates the degree to which two words are ranked higher or lower. For example, the degree of hierarchy is the difference between the norms of the hierarchical embedded representation. The smaller the difference in the norm, the higher the degree of hierarchy, and the larger the difference, the lower the degree of hierarchy.

[0110] The output unit 19 expresses the distribution space of the hierarchical embedded expression, for example, in a tree structure (dendrogram) in which the superordinate-subordinate relationships between words are hierarchized.

[0111] Fig. 14 is a conceptual diagram showing an example of visualization processing. The example in Fig. 14 is an example of expressing the distribution space of a hierarchically embedded expression in a tree structure in which the superordinate-subordinate relationships are hierarchically organized. When expressing the distribution space in a tree structure, as shown in Fig. 14, words in a superordinate-subordinate relationship are connected by lines, and the tree structure is constructed so that words with a higher degree of superordinateity are placed in a higher hierarchy and words with a lower degree of superordinateity are placed in a lower hierarchy.

[0112] The output unit 19 expresses the distribution space of the hierarchical structure embedded expression, for example, in the form of a scatter diagram with a predetermined axis representing the degree of superiority.

[0113] Figure 15 is a conceptual diagram showing an example of visualization processing. The example in Figure 15 shows an example of expressing the distribution space of a hierarchical embedded representation in a scatter plot with a predetermined axis representing the degree of hierarchy. Since the distribution space of a hierarchical embedded representation is a multidimensional space, it is necessary to perform dimensionality reduction using two measures to express it in a two-dimensional scatter plot. In this case, one measure is determined as the degree of hierarchy. The other measure can be any measure.

[0114] The visualization process can be combined with the determination process or extraction process. For example, along with the determination results of the determination process, a diagram showing words in a hierarchical or subordinate relationship with each phrase in a tree structure may be output. Also, for example, words extracted by the extraction process may be output as a tree structure or a scatter diagram.

[0115] [Variations] In the above embodiment, an example was described in which the distribution space of hierarchically embedded expressions has the components related to the degree of relevance as the angle between words, and the components related to the relative superiority / subordinateness as the magnitude relationship of norms. In the modified example, an example will be described in which the distribution space of hierarchically embedded expressions has the components related to the degree of relevance as the x-axis, and the components related to the relative superiority / subordinateness as the y-axis.

[0116] FIG. 16 is a conceptual diagram showing a modified example of the distribution space of hierarchically embedded expressions. The example of FIG. 16 shows the distribution space of hierarchically embedded expressions in a scatter plot with the x-axis representing components related to the degree of relevance (synonymous relationships, etc.) and the y-axis representing components related to the superior-subordinate relationships (superiority). As shown in FIG. 16, the distribution space in the modified example indicates that the closer the distance along the x-axis, the closer the meaning, and the farther the distance along the x-axis, the more distant the meaning. Furthermore, the distribution space in the modified example indicates that the higher the position along the y-axis, the higher the superiority, and the lower the position along the y-axis, the lower the superiority.

[0117] Note that the x-axis and y-axis in this modified example are merely examples, and as long as the distribution space of the hierarchical embedded representation has an axis corresponding to components related to the degree of relevance to each other and an axis corresponding to components related to the superior-subordinate relationship to each other, either axis in the distribution space may correspond to each component.

[0118] <Summary> As described above, according to each embodiment of the present disclosure, it is possible to process a vector representation of a word so that its hierarchical relationship with other words can be determined. The vector representation processing device in this embodiment generates a distribution space based on dictionary data including combinations of hypernyms and hyponyms, where the distribution space has components related to the degree of relevance and components related to the hierarchical relationship with other words, and maps any word into the distribution space. Therefore, the vector representation processing device in this embodiment can process a vector representation of a word so that its hierarchical relationship with other words can be determined.

[0119] In particular, the vector expression processing device in this embodiment structurally visualizes the distribution of words mapped to a distribution space. For example, the vector expression processing device visualizes the distribution of words in a list format that lists words in a hierarchical relationship. Furthermore, for example, the vector expression processing device visualizes the distribution of words in a hierarchical tree structure according to the hierarchical relationship of the words. Furthermore, for example, the vector expression processing device visualizes the distribution of words in a scatter plot with a predetermined axis representing the degree of hierarchical relationship. Therefore, the vector expression processing device in this embodiment allows users to intuitively grasp the hierarchical relationship between any word and words included in dictionary data at a high level.

[0120] The vector expression processing device of this embodiment determines a hierarchical relationship by mapping two words into a distribution space. The vector expression processing device of this embodiment also extracts words that have a hierarchical relationship or are highly related to one another by mapping one word into a distribution space. Therefore, the vector expression processing device of this embodiment can extract words that have a hierarchical relationship or are highly related to one word with high accuracy.

[0121] Although the embodiments have been described above, it will be understood that various changes in form and details can be made without departing from the spirit and scope of the claims. [Explanation of symbols]

[0122] 10 Vector expression processing device 11 Input section 12 Conversion model storage unit 13 Conversion unit 14 Learning Department 15 Machining model memory section 16 Processing Department 17 Distribution space storage unit 18 Judgment section 19 Output section

Claims

1. A vector representation processing device having a control unit, The control unit Using vector representations representing the features of words as input, inference is performed based on dictionary data including combinations of hypernyms and hyponyms. Map any phrase into a distribution space that shows vector representations of multiple words, The distribution space is arranged to have components related to the degree of relevance and components related to the superior-subordinate relationship, The component relating to the mutual relevance is the smallness of the angle formed by the words, and the component relating to the hierarchical relationship is the norm length. Vector representation processing device.

2. A vector representation processing device having a control unit, The control unit Using vector representations representing the features of words as input, inference is performed based on dictionary data including combinations of hypernyms and hyponyms. Map any phrase into a distribution space that shows vector representations of multiple words, The distribution of the words mapped in the distribution space is visualized in a list format, a tree structure, or a scatter diagram with a predetermined axis representing the degree of hierarchy; The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing device.

3. A vector representation processing device having a control unit, The control unit Using vector representations representing the features of words as input, inference is performed based on dictionary data including combinations of hypernyms and hyponyms. Map any phrase into a distribution space that shows vector representations of multiple words, It accepts two phrases, For the two terms mapped onto the distribution space, determining a hierarchical relationship between the two terms based on an index calculated from an angle between the two terms and a norm; The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing device.

4. A vector representation processing device having a control unit, The control unit Using vector representations representing the features of words as input, inference is performed based on dictionary data including combinations of hypernyms and hyponyms. Map any phrase into a distribution space that shows vector representations of multiple words, It accepts two phrases, For the two terms mapped onto the distribution space, determining whether or not there is a hierarchical relationship between the two terms based on an index calculated from the angle and norm between the two terms; If the relation of hierarchical and subordinate terms exists and the magnitude relation of the norms of the two terms is within a predetermined range, the two terms are determined to be in a synonymous relationship; The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing device.

5. A vector representation processing device having a control unit, The control unit Using vector representations representing the features of words as input, inference is performed based on dictionary data including combinations of hypernyms and hyponyms. Map any phrase into a distribution space that shows vector representations of multiple words, It accepts two phrases, For the two terms mapped onto the distribution space, determining whether or not there is a hierarchical relationship between the two terms based on an index calculated from the angle and norm between the two terms; If the relation of hierarchical and subordinate terms exists and the magnitude relation of the norms of the two terms is outside a predetermined range, the two terms are determined to be in a relation of hierarchical and subordinate terms; The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing device.

6. A vector representation processing device having a control unit, The control unit Using vector representations representing the features of words as input, inference is performed based on dictionary data including combinations of hypernyms and hyponyms. Map any phrase into a distribution space that shows vector representations of multiple words, Accepts multiple phrases, generating a second distribution space in which highly related terms among the plurality of terms are arranged in proximity to each other; Mapping the plurality of terms included in the second distribution space into the distribution space; The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing device.

7. The distribution space is a Euclidean space.

7. The vector representation processing device according to claim 1.

8. The control unit Mapping words and phrases into the distribution space using a machine learning model trained using training data that associates a plurality of combinations of the hypernyms and hyponyms with the hypernym-hypernym relationships expressed by the combinations.

7. The vector representation processing device according to claim 1.

9. The control unit visualizes the distribution of the words mapped in the distribution space in the tree structure, The tree structure is hierarchical according to the hierarchy of the words and phrases. The vector representation processing device according to claim 2 .

10. A control unit of the vector expression processing device, A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; Run The distribution space is arranged to have components related to the degree of relevance and components related to the superior-subordinate relationship, The component relating to the mutual relevance is the smallness of the angle formed by the words, and the component relating to the hierarchical relationship is the norm length. Vector representation processing method.

11. A control unit of the vector expression processing device, A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; a step of visualizing the distribution of the words mapped onto the distribution space in a list format, a tree structure, or a scatter diagram with a predetermined axis representing the degree of hierarchy; Run The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing method.

12. A control unit of the vector expression processing device, A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; accepting input of two phrases; For the two terms mapped onto the distribution space, determining a hierarchical relationship between the two terms based on an index calculated from an angle between the two terms and a norm; Run The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing method.

13. A control unit of the vector expression processing device, A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; accepting input of two phrases; For the two terms mapped onto the distribution space, a step of determining whether or not there is a hierarchical relationship between the two terms based on an index calculated from an angle between the two terms and a norm; a step of determining that the two terms are in a synonymous relationship when the relation between the terms exists and the magnitude relation between the norms of the two terms is within a predetermined range; Run The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing method.

14. A control unit of the vector expression processing device, A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; accepting input of two phrases; For the two terms mapped onto the distribution space, a step of determining whether or not there is a hierarchical relationship between the two terms based on an index calculated from an angle between the two terms and a norm; a step of determining that the two terms have a hierarchical relationship when the hierarchical relationship exists and the magnitude relationship between the norms of the two terms is outside a predetermined range; Run The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing method.

15. A control unit of the vector expression processing device, A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; accepting input of multiple words; generating a second distribution space in which highly related terms among the plurality of terms are arranged adjacent to each other; mapping the plurality of terms included in the second distribution space into the distribution space; Run The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. Vector representation processing method.

16. A control unit of the vector expression processing device includes: A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; Execute The distribution space is arranged to have components related to the degree of relevance and components related to the superior-subordinate relationship, The component relating to the mutual relevance is the smallness of the angle formed by the words, and the component relating to the hierarchical relationship is the norm length. program.

17. A control unit of the vector expression processing device includes: A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; a step of visualizing the distribution of the words mapped onto the distribution space in a list format, a tree structure, or a scatter diagram with a predetermined axis representing the degree of hierarchy; Execute The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. program.

18. A control unit of the vector expression processing device includes: A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; accepting input of two phrases; For the two terms mapped onto the distribution space, determining a hierarchical relationship between the two terms based on an index calculated from an angle between the two terms and a norm; Execute The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. program.

19. A control unit of the vector expression processing device includes: A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; accepting input of two phrases; For the two terms mapped onto the distribution space, a step of determining whether or not there is a hierarchical relationship between the two terms based on an index calculated from an angle between the two terms and a norm; a step of determining that the two terms are in a synonymous relationship when the relation between the terms exists and the magnitude relation between the norms of the two terms is within a predetermined range; Execute The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. program.

20. A control unit of the vector expression processing device includes: A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; accepting input of two phrases; For the two terms mapped onto the distribution space, a step of determining whether or not there is a hierarchical relationship between the two terms based on an index calculated from an angle between the two terms and a norm; a step of determining that the two terms have a hierarchical relationship when the hierarchical relationship exists and the magnitude relationship between the norms of the two terms is outside a predetermined range; Execute The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. program.

21. A control unit of the vector expression processing device includes: A procedure for performing inference based on dictionary data including combinations of hypernyms and hyponyms, using vector representations representing features of words as input; A procedure for mapping any phrase into a distribution space that represents vector representations of multiple words; accepting input of multiple words; generating a second distribution space in which highly related terms among the plurality of terms are arranged adjacent to each other; mapping the plurality of terms included in the second distribution space into the distribution space; Execute The distribution space is arranged so as to have components relating to the degree of relevance to each other and components relating to the higher-lower relationship to each other. program.

Images