Meaning relationship learning device, meaning relationship learning method, and recording medium recording meaning relationship learning program

Abstract

The semantic relationship learning device (1) comprises: a negative example data generating unit (12), which generates a plurality of negative example data pairs by combining language data constituting a plurality of positive example data pairs; a feature value extracting unit (13), which extracts feature values ​​from the language data constituting each of the negative example data pairs; a similarity calculating unit (14), which calculates the similarity between the feature values ​​of each negative example data pair; a learning negative example data generating unit (15), which classifies a plurality of negative example data pairs into a plurality of predetermined similarity ranges based on similarity, thereby generating a plurality of learning negative example data sets corresponding to the plurality of similarity ranges; a learning data set selecting unit (17), which selects any learning negative example data set from a plurality of learning negative example data sets in an order according to a selection plan predetermined based on the plurality of similarity ranges; and a learning processing unit (18), which uses the selected learning negative example data set and a plurality of positive example data pairs to perform machine learning processing.

Description

Technical Field

[0001] The present invention relates to a semantic relationship learning device for performing machine learning, and a semantic relationship learning method and a semantic relationship learning program used for performing machine learning. Background Art

[0002] Due to the popularity of personal computers and the Internet, the amount of electronic data that users can access has increased. Therefore, a technology for effectively finding desired data from large-scale electronic data is sought. In the field of natural language processing, in order to retrieve replacement sentences (i.e., sentences that are equivalent in meaning), the following learning device is used: a text pair that is in a synonymous relationship, i.e., a positive example, and a text pair that is not in a synonymous relationship, i.e., a negative example, are manually generated in advance as teaching data, and the semantic relationship between texts is statistically machine-learned using the generated teaching data. If the result of this machine learning is used, it is possible to retrieve and estimate text that is in a synonymous relationship with the input text.

[0003] For example, Patent Document 1 proposes a device for performing synonym determination processing on predicates contained in a text. The device uses pairs of predicates having the same meaning (i.e., text examples in a synonymous relationship with each other) as positive examples, and pairs of predicates having different meanings (i.e., text examples in a non-synonymous relationship with each other) as negative examples, and performs machine learning processing using the positive examples and negative examples as teaching data, and as a result, obtains a determination model for synonym determination processing.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2014-119988 Summary of the invention

[0007] Problem that the invention aims to solve

[0008] However, in the negative examples used in the machine learning process performed by the above-mentioned device, negative examples composed of language data with very close meanings and negative examples composed of language data with very far meanings are not distinguished, and it is considered that they exist mixedly. Therefore, by using such negative examples as machine learning processing for teaching data, it is difficult to obtain a discrimination model that can achieve good discrimination of semantic relationships.

[0009] The present invention is completed to solve the problems of the above-mentioned prior art, and its purpose is to provide a meaning relationship learning device that can perform machine learning processing that can achieve good discrimination of meaning relationships, as well as a meaning relationship learning method and a meaning relationship learning program used for performing machine learning processing that can achieve good discrimination of meaning relationships.

[0010] Means used to solve problems

[0011] The semantic relationship learning device of one scheme of the present invention is characterized in that it comprises: a negative example data generating unit, which obtains a plurality of positive example data pairs from a positive example data storing unit storing a plurality of positive example data pairs, wherein the plurality of positive example data pairs are respectively composed of language data that are in a predetermined semantic relationship with each other, and the negative example data generating unit generates a plurality of negative example data pairs by combining the language data, wherein the plurality of negative example data pairs are respectively composed of language data that are not in the predetermined semantic relationship with each other; a feature value extraction unit, which extracts feature values ​​from the language data that respectively constitute the plurality of negative example data pairs; a similarity calculation unit, which calculates the similarity between the feature values ​​of the language data of each of the plurality of negative example data pairs; and a learning negative example data generating unit, which classifies the plurality of negative example data pairs into predetermined categories based on the similarity. A method for generating a plurality of learning negative data sets corresponding to the plurality of similarity ranges, wherein the plurality of learning negative data sets respectively contain at least one negative data pair among the plurality of negative data pairs; a learning data set selection unit, which selects any learning negative data set among the plurality of learning negative data sets in an order according to a selection plan predetermined based on the plurality of similarity ranges; and a learning processing unit, which uses the learning negative data set and the plurality of positive data pairs selected by the learning data set selection unit to perform machine learning processing.

[0012] The semantic relationship learning method of one scheme of the present invention is characterized in that it has: a negative example data generation step, obtaining the multiple positive example data pairs from a positive example data storage unit storing the multiple positive example data pairs, the multiple positive example data pairs are respectively composed of language data that are in a predetermined semantic relationship with each other, and generating multiple negative example data pairs by combining the language data, the multiple negative example data pairs are respectively composed of language data that are not in the predetermined semantic relationship with each other; an extraction step, extracting feature quantities from the language data that respectively constitute the multiple negative example data pairs; a calculation step, calculating the similarity between the feature quantities of the language data of each of the multiple negative example data pairs; a learning negative example data generation step, classifying the multiple negative example data pairs into predetermined categories based on the similarity. A plurality of similarity ranges are determined in advance, or a plurality of learning negative data pairs generated according to the language data constituting the plurality of negative data pairs and the language data constituting the plurality of positive data pairs are classified into a plurality of predetermined similarity ranges based on the similarities, thereby generating a plurality of learning negative data sets corresponding to the plurality of similarity ranges, wherein the plurality of learning negative data sets respectively contain one or more negative data pairs among the plurality of negative data pairs; a selection step of selecting any learning negative data set among the plurality of learning negative data sets in the order of a selection plan determined in advance based on the plurality of similarity ranges; and a learning step of performing machine learning processing using the selected learning negative data set and the plurality of positive data pairs.

[0013] Effects of the Invention

[0014] According to the present invention, it is possible to perform machine learning processing that can realize good discrimination of semantic relationships. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a functional block diagram schematically showing the configuration of the semantic relationship learning device according to the first embodiment of the present invention.

[0016] Figure 2 This is a diagram showing an example of the hardware configuration of the semantic relationship learning device according to the first embodiment.

[0017] Figure 3 This is a flowchart showing the operation of the semantic relationship learning device according to the first embodiment.

[0018] Figure 4 This is a flowchart showing the operation of the negative example data generating unit in the first embodiment.

[0019] Figure 5 This is a flowchart showing the operation of the feature quantity extraction unit in the first embodiment.

[0020] Figure 6This is a flowchart showing the operation of the similarity calculation unit in the first embodiment.

[0021] Figure 7 This is a flowchart showing the operation of the learning negative example data generating unit in the first embodiment.

[0022] Figure 8 This is a flowchart showing the operation of the learning data selection unit in the first embodiment.

[0023] Fig. 9 This is a flowchart showing the operation of the learning processing unit in the first embodiment.

[0024] Fig.10 This is a functional block diagram schematically showing the structure of a semantic relationship learning device according to Embodiment 2 of the present invention.

[0025] Fig.11 This is a flowchart showing the operation of the semantic relationship learning device according to the second embodiment.

[0026] Fig.12 This is a flowchart showing the operation of the negative example data generating unit in the second embodiment.

[0027] Fig.13 This is a flowchart showing the operation of the feature extraction unit in the second embodiment.

[0028] Fig.14 This is a flowchart showing the operation of the similarity calculation unit in the second embodiment.

[0029] Fig.15 This is a flowchart showing the operation of the learning negative example data generating unit in the second embodiment. DETAILED DESCRIPTION

[0030] Hereinafter, a semantic relationship learning device, a semantic relationship learning method, and a semantic relationship learning program according to an embodiment of the present invention will be described with reference to the drawings. Note that the following embodiments are merely examples, and various modifications are possible within the scope of the present invention.

[0031] The semantic relationship learning device of the embodiment is a learning device that can be set in a semantic relationship discrimination device, and the semantic relationship discrimination device has a semantic relationship discrimination unit that performs discrimination processing of semantic relationships. The semantic relationship learning method of the embodiment is a learning method for obtaining a semantic relationship discrimination model used in the discrimination processing of semantic relationships. The semantic relationship learning program of the embodiment is a software program for obtaining a semantic relationship discrimination model used in the discrimination processing of semantic relationships.

[0032] 《1》Implementation method 1.

[0033] 《1-1》Structure of Implementation Method 1.

[0034] Figure 1 1 is a functional block diagram schematically showing the structure of a semantic relationship learning device 1 according to Embodiment 1 of the present invention. The semantic relationship learning device 1 is a device capable of implementing the semantic relationship learning method according to Embodiment 1. Figure 1 As shown, the semantic relationship learning device 1 includes a negative data generation unit 12, a feature extraction unit 13, a similarity calculation unit 14, a learning negative data generation unit 15, a learning data set selection unit 17, and a learning processing unit 18. In addition, the semantic relationship learning device 1 may also include a positive data storage unit 11 and a learning negative data storage unit 16.

[0035] The positive example data storage unit 11 stores a plurality of positive example data pairs (hereinafter also referred to as "positive examples") A11. The plurality of positive example data pairs are respectively composed of data related to languages ​​that are in a predetermined semantic relationship with each other (hereinafter also referred to as "language data"). In implementation mode 1, the predetermined semantic relationship is a synonymous relationship. Language data pairs that are in a synonymous relationship with each other are, for example, text pairs that are in a synonymous relationship with each other. Here, the text includes words and sentences. In addition, the predetermined semantic relationship is not limited to a synonymous relationship.

[0036] The negative data generating unit 12 obtains a plurality of positive data pairs A11 stored in the positive data storing unit 11. The negative data generating unit 12 generates a plurality of negative data pairs (hereinafter also referred to as "negative examples") A12 by combining the language data that respectively constitute the plurality of positive data pairs A11. The plurality of negative data pairs A12 are respectively constituted by language data that are not in a predetermined semantic relationship with each other. In implementation mode 1, the plurality of negative data pairs A12 are respectively constituted by language data that are not in a synonymous relationship with each other. For example, language data pairs that are not in a synonymous relationship with each other are text pairs that are not in a synonymous relationship with each other. Here, the text includes words and sentences.

[0037] The feature extraction unit 13 extracts feature A13 from the language data constituting the plurality of negative data pairs A12 generated by the negative data generation unit 12, that is, calculates feature A13. As feature, feature suitable for the similarity used and the nature of the language data can be used. For example, in the case where the language data constituting the negative data pairs are text data, as feature for calculating the similarity between texts, one or more of the sentence length of the text, the character type of the text, the result of morpheme analysis of the text, the result of dependency analysis of the text, and a multidimensional vector converted from the text can be used. The method of converting a text into a multidimensional vector is well known, and for example, the method described in non-patent document 1 can be used.

[0038] Non-patent literature 1: Tomas Mikolov, 3 others, "Efficient Estimation of Word Representations in Vector Space", ICLR (International Conference on Learning Representations) 2013

[0039] The similarity calculation unit 14 calculates the similarity between the feature amounts A13 of the plurality of negative example data pairs A12. The similarity calculation unit 14 assigns the calculated similarity to the negative example data pairs A12 corresponding to the similarity, and generates negative example data pairs A14 to which the similarity is assigned.

[0040] As the similarity, a similarity defined in advance according to the nature of the language data can be used. For example, when the language data constituting the negative data pair is text data, one or more of the difference in sentence length between texts, the edit distance of the text (i.e., Levenshtein distance), the degree of repetition of word strings obtained by performing morphological analysis on the text, the degree of repetition of phrase units obtained as a result of performing dependency analysis on the text, the degree of repetition of dependency relations obtained as a result of performing dependency analysis on the text, the Euclidean distance of multidimensional vectors converted from the text, or the cosine similarity can be used.

[0041] For example, the smaller the difference in sentence length between texts, the higher the similarity. In addition, the shorter the edit distance of the text, the higher the similarity. In addition, the greater the degree of repetition of the word string obtained by performing morphological analysis on the text, the higher the similarity. In addition, the greater the degree of repetition of the phrase unit obtained as a result of performing dependency analysis on the text, the higher the similarity. In addition, the greater the degree of repetition of the dependency relationship obtained as a result of performing dependency analysis on the text, the higher the similarity. In addition, the shorter the Euclidean distance of the multidimensional vector converted from the text, the higher the similarity. The method of converting text into a multidimensional vector is well known, for example, the method described in non-patent document 1 can be used. In addition, cosine similarity is used when comparing files to each other in a vector space model, and its calculation method is well known. In addition, the discrimination results such as likelihood output by a learning device prepared in advance can also be used as similarity.

[0042] The learning negative data generation unit 15 classifies the plurality of negative data pairs A14 into a plurality of predetermined similarity ranges based on the negative data pairs A14 assigned similarities calculated by the similarity calculation unit 14, thereby generating a plurality of learning negative data sets A15 corresponding to the plurality of similarity ranges. The plurality of learning negative data sets A15 each include one or more negative data pairs among the plurality of negative data pairs A14 assigned similarities.

[0043] For example, when the negative data pair is a text pair and the similarity is the Euclidean distance between the vectorized texts, the learning negative data generation unit 15 classifies the texts generated by the negative data generation unit 12 based on the Euclidean distance calculated by the similarity calculation unit 14, thereby generating a learning negative data set A15 classified according to the range of each Euclidean distance calculated by the similarity calculation unit 14. Here, the range of the Euclidean distance d1 as the similarity range is, for example,

[0044] The range of 0.1≦d1<0.3 is the first similarity range.

[0045] The range of 0.3≦d1<0.5 is the second similarity range, and

[0046] The range of 0.5≦d1≦0.7 is the third similarity range.

[0047] This example is an example in which the negative example data pairs are classified into three data sets according to the similarity range. In addition, the width of the similarity range of the generated learning negative example data set and the number of similarity ranges are not limited to the above example.

[0048] The learning negative example data storage unit 16 receives a plurality of learning negative example data sets A15 corresponding to a plurality of similarity ranges generated by the learning negative example data generation unit 15 , and stores the data sets as a plurality of learning negative example data sets A16 .

[0049] The learning data set selection unit 17 selects any learning negative example data set A17 from among the plurality of learning negative example data sets A16 stored in the learning negative example data storage unit 16 in an order according to a selection plan determined in advance based on a plurality of similarity ranges.

[0050] Specifically, the learning data set selection unit 17 outputs the learning negative example data set with the lowest similarity in the similarity range to the learning processing unit 18 in sequence, and performs the learning processing in the learning processing unit 18. After one learning processing is completed, the learning negative example data set with the second highest similarity is output to the learning processing unit 18, and the processing is repeated in the same order until the selection plan of the learning negative example data set is completed.

[0051] In other words, the learning data set selection unit 17 determines the selection plan so that the learning negative example data set corresponding to the similarity range with the first lowest (i.e., lowest) value among the multiple similarity ranges is first selected as the first one, and then the learning negative example data set corresponding to the similarity range with the second lowest value among the multiple similarity ranges is selected as the second one. That is, the learning data set selection unit 17 determines the selection plan so that when n is an integer greater than 1, the learning negative example data set corresponding to the similarity range with the nth lowest value among the multiple similarity ranges is selected as the nth one.

[0052] In this way, by starting with the learning negative dataset with low similarity and gradually learning the learning negative dataset with high similarity, the semantic relationship learning device can converge to a better solution earlier. In addition, the generalization performance of the semantic relationship discrimination model obtained by the semantic relationship learning device can be improved. In addition, in the selection plan of the learning negative dataset, an arbitrary number of learning times can also be set. In addition, in order to maximize the accuracy of the semantic relationship learning device, the optimization results based on Bayesian optimization and the like can also be used to determine the selection plan of the learning negative dataset.

[0053] That is, in the learning for the semantic relationship discrimination of electronic data, negative data pairs considering similarity are prepared in advance, and the learning of simple problem settings with low difficulty (for example, problems using negative data pairs composed of language data with low similarity) is first performed, and then the learning of high-difficulty problem settings (for example, problems using negative data pairs composed of language data with high similarity) is performed. In this way, the semantic relationship learning device 1 can not only converge to a better solution earlier, but also improve the generalization performance of the semantic relationship discrimination device equipped with the semantic relationship learning device 1.

[0054] The learning processing unit 18 performs machine learning processing using the learning negative example data set A17 selected by the learning data set selection unit 17 and the plurality of positive example data pairs A11 stored in the positive example data storage unit 11. In the machine learning processing, a known learning algorithm can be used. As algorithms that can be used, there are SVM (Support Vector Machine), neural network or decision tree, etc.

[0055] Figure 21 is a diagram showing an example of the hardware configuration of the semantic relationship learning device 1 of the first embodiment. The semantic relationship learning device 1 includes, for example, a memory 102 as a storage device storing a program as software, that is, the semantic relationship learning program of the first embodiment; and a processor 101 as an information processing unit executing the program stored in the memory 102. The semantic relationship learning device 1 is, for example, a computer. The semantic relationship learning program of the first embodiment is stored in the memory 102 from a storage medium storing information via a reading device (not shown) or via a communication interface (not shown) that can be connected to the Internet or the like. In addition, the semantic relationship learning device 1 may also include an input device such as a mouse 103 and a keyboard 104 as a user operation unit, and an output device such as a display device 105 that displays an image and a sound output unit (not shown) that outputs sound. In addition, the semantic relationship learning device 1 may also include an auxiliary storage device 106 that stores various information such as a database. The auxiliary storage device 106 does not necessarily need to exist inside the semantic relationship learning device 1, and may also be a storage device that exists on the cloud via a communication interface.

[0056] Figure 1 The negative data generator 12, feature extraction unit 13, similarity calculation unit 14, learning negative data generator 15, learning data set selection unit 17 and learning processing unit 18 shown can be implemented by the processor 101 executing the program stored in the memory 102. Figure 1 The positive example data storage unit 11 and the learning negative example data storage unit 16 shown may be part of the auxiliary storage device 106 .

[0057] 《1-2》Actions of implementation 1.

[0058] Figure 3 This is a flowchart showing the operation of the semantic relationship learning device 1 according to the first embodiment. Figure 4 1 is a flowchart showing the operation of the negative example data generating unit 12 in Embodiment 1. First, the negative example data generating unit 12 obtains the language data constituting the plurality of positive example data pairs A11 from the positive example data storing unit 11 (steps S11 and S111).

[0059] Next, the negative data generating unit 12 generates a plurality of negative data pairs A12 by combining the language data obtained from the positive data storing unit 11 (step S12, step S112). The plurality of negative data pairs A12 are respectively composed of language data that are not in a synonymous relationship with each other. That is, the negative data generating unit 12 generates a plurality of negative data pairs A12 based on a combination of language data other than the positive data pair A11. Preferably, the negative data generating unit 12 generates a plurality of negative data pairs A12 based on all combinations of language data other than the positive data pair A11. The negative data generating unit 12 outputs the negative data pairs A12 to the feature extraction unit 13 and the similarity calculation unit 14 (step S113).

[0060] Figure 5 1 is a flowchart showing the operation of the feature extraction unit 13 in Embodiment 1. The feature extraction unit 13 obtains a plurality of negative example data pairs A12, extracts feature values ​​A13 from the language data constituting the plurality of negative example data pairs A12, that is, calculates feature values ​​A13 (steps S13, S131, S132). The feature extraction unit 13 outputs the feature values ​​A13 to the similarity calculation unit 14 (step S133).

[0061] Figure 6 1 is a flowchart showing the operation of the similarity calculation unit 14 in Embodiment 1. The similarity calculation unit 14 obtains a plurality of negative data pairs A12 and a plurality of feature quantities A13, and calculates the similarity between the feature quantities A13 of each of the plurality of negative data pairs A12 (step S14, step S141, step S142). The similarity calculation unit 14 assigns similarity to the negative data pairs, and outputs the negative data pairs A14 assigned with similarity to the learning negative data generation unit 15 (step S143).

[0062] Figure 7 Flowchart showing the operation of the learning negative data generation unit 15 in Embodiment 1. The learning negative data generation unit 15 obtains a plurality of negative data pairs A14 to which similarities are assigned, and classifies the plurality of negative data pairs A14 into a plurality of predetermined similarity ranges, thereby generating a plurality of learning negative data sets A15 corresponding to the plurality of similarity ranges (step S15, step S151, S152). That is, the plurality of learning negative data sets A15 are groups of data including one or more negative data pairs among the plurality of negative data pairs A12. The plurality of learning negative data sets A15 corresponding to the plurality of similarity ranges are stored in the learning negative data storage unit 16 (step S153).

[0063] Figure 81 is a flowchart showing the operation of the learning data set selection unit 17 in Embodiment 1. The learning data set selection unit 17 selects any learning negative example data set A17 from among the plurality of learning negative example data sets A16 stored in the learning negative example data storage unit 16 in the order of the selection plan predetermined based on the plurality of similarity ranges (step S16, step S161). The selected learning negative example data set A17 is output to the learning processing unit 18 (step S162).

[0064] Fig. 9 1 is a flowchart showing the operation of the learning processing unit 18 in the first embodiment. The learning processing unit 18 performs machine learning processing (step S17, step S171) using the selected learning negative example data set A17 and the plurality of positive example data pairs A11 stored in the positive example data storage unit 11. Figure 3 The processing of steps S16 and S17 in is continued until the selection plan of the negative example data set for learning is completed (step S18).

[0065] 《1-3》Effects of implementation method 1.

[0066] As described above, by using the semantic relationship learning device 1, the semantic relationship learning method, or the semantic relationship learning program according to the first embodiment, it is possible to perform a machine learning process that enables good discrimination of semantic relationships.

[0067] Specifically, negative data pairs A14 considering similarity are prepared in advance, and learning is first performed using negative data pairs consisting of language data with low similarity, and then learning is performed using negative data pairs consisting of language data with the second lowest similarity. That is, learning is gradually promoted to a problem setting with a high degree of difficulty (that is, a problem setting using negative data pairs consisting of language data with high similarity). In this way, not only can a better solution be converged earlier, but also the generalization performance can be improved.

[0068] 《2》Implementation method 2.

[0069] 《2-1》Structure of Implementation Method 2.

[0070] Fig.10 1 is a functional block diagram schematically showing the structure of a semantic relationship learning device 2 according to a second embodiment of the present invention. The semantic relationship learning device 2 is a device capable of implementing a semantic relationship learning method according to the second embodiment. Fig.10As shown, the semantic relationship learning device 2 includes a negative data generation unit 22, a feature extraction unit 23, a similarity calculation unit 24, a learning negative data generation unit 25, a learning data set selection unit 27, and a learning processing unit 28. In addition, the semantic relationship learning device 2 may also include a positive data storage unit 21 and a learning negative data storage unit 26. Hereinafter, the second embodiment will be described with the focus on the differences from the first embodiment.

[0071] The positive example data storage unit 21 stores a plurality of positive example data pairs A21. The plurality of positive example data pairs are respectively composed of language data that are in a predetermined semantic relationship with each other. In the second embodiment, the predetermined semantic relationship is the relationship between a question sentence and an answer sentence. The language data that are in a predetermined semantic relationship with each other are, for example, question sentence data and answer sentence data. In addition, the predetermined semantic relationship is not limited to the relationship between a question sentence and an answer sentence.

[0072] The negative data generation unit 22 obtains a plurality of positive data pairs A21 stored in the positive data storage unit 21. The negative data generation unit 22 generates a plurality of negative data pairs A22 by combining the language data that respectively constitute the plurality of positive data pairs A21. The plurality of negative data pairs A22 are respectively constituted by language data that are not in a predetermined semantic relationship with each other. In the second embodiment, the plurality of negative data pairs A22 are respectively constituted by language data that are not in a question sentence and answer sentence relationship with each other. For example, the negative data pairs A22 are data pairs consisting of a question sentence and a question sentence, and data pairs consisting of an answer sentence and an answer sentence.

[0073] The feature extraction unit 23 extracts feature A23 from the language data constituting the plurality of negative data pairs A22 generated by the negative data generation unit 22, that is, calculates feature A23. As in the case of the first embodiment, a feature suitable for the similarity used and the nature of the language data can be used as the feature.

[0074] The similarity calculation unit 24 calculates the similarity between the feature amounts A23 of each of the plurality of negative data pairs A22. The similarity calculation unit 24 assigns the calculated similarity to the negative data pair A22 corresponding to the similarity, and generates a negative data pair A24 to which the similarity is assigned. As the similarity, the same similarity as in the case of the first embodiment can be used.

[0075] The learning negative data generation unit 25 replaces the question data constituting the positive data pair A21 stored in the positive data storage unit 21 with the question data included in the negative data pair A24 to which the similarity generated by the similarity calculation unit 24 is assigned, or replaces the answer data constituting the positive data pair A21 stored in the positive data storage unit 21 with the answer data included in the negative data pair A24 to which the similarity generated by the similarity calculation unit 24 is assigned, thereby generating a plurality of learning negative data pairs. The learning negative data generation unit 25 classifies the language data constituting the plurality of learning negative data pairs into a plurality of predetermined similarity ranges based on the similarities calculated by the similarity calculation unit 24, thereby generating a plurality of learning negative data sets A25 corresponding to the plurality of similarity ranges. The plurality of learning negative data sets A25 each include one or more negative data pairs among the plurality of learning negative data pairs A24 to which the similarity is assigned.

[0076] For example, the following case is described: the learning negative data generating unit 25 generates a learning negative data pair consisting of "question data D11" and "answer data D20" using a negative data pair consisting of "question data D10" and "question data D11" based on a positive data pair consisting of "question data D10" and "answer data D20". In this case, the learning negative data generating unit 25 replaces "question data D10" of the positive data pair with "question data D11" of the negative data pair, thereby generating a learning negative data pair consisting of "question data D11" and "answer data D20". At this time, the generated learning negative data pair is given the similarity given to the negative data pair consisting of "question data D10" and "question data D11".

[0077] In addition, for example, the following case is described: the learning negative data generation unit 25 generates a learning negative data pair consisting of "question data D10" and "answer data D21" using a negative data pair consisting of "answer data D20" and "answer data D21" based on a positive data pair consisting of "question data D10" and "answer data D20". In this case, the learning negative data generation unit 25 replaces "answer data D20" of the positive data pair with "answer data D21" of the negative data pair, thereby generating a learning negative data pair consisting of "question data D10" and "answer data D21". At this time, the generated learning negative data pair is given the similarity given to the negative data pair consisting of "answer data D20" and "answer data D21".

[0078] For example, the similarity between "question sentence A" and "question sentence B" constituting the negative data pair generated by the negative data generating unit 22 is described. The similarity calculating unit 24 converts the text data of "question sentence A" and the text data of "question sentence B" into multidimensional vectors, calculates the Euclidean distance d2 between the texts based on the multidimensional vectors, and sets the Euclidean distance d2 obtained by the calculation as the similarity. By performing such processing, a negative data pair for learning consisting of a "question sentence" and an "answer sentence" to which similarity is assigned is generated.

[0079] The learning negative data generation unit 25 generates a learning negative data set A25 by classifying the learning negative data pairs consisting of "question sentences" and "answer sentences" according to the range of each Euclidean distance d2 based on the calculated Euclidean distance d2. Here, the range of the Euclidean distance d2 as the similarity range is, for example,

[0080] The range of 0.1≦d2<0.3 is the first similarity range.

[0081] The range of 0.3≦d2<0.5 is the second similarity range, and

[0082] The range of 0.5≦d2≦0.7 is the third similarity range.

[0083] This example is an example of classifying the negative example data pairs for learning into three data sets according to different similarity ranges. In addition, the size of the similarity range of the generated negative example data set for learning and the number of similarity ranges are not limited to the above example.

[0084] The learning negative example data storage unit 26 receives the plurality of learning negative example data sets A25 corresponding to the plurality of similarity ranges generated by the learning negative example data generation unit 25 and stores the data sets as a plurality of learning negative example data sets A26 .

[0085] The learning data set selection unit 27 selects any one of the plurality of learning negative example data sets A26 stored in the learning negative example data storage unit 26 in an order according to a selection plan determined in advance based on a plurality of similarity ranges.

[0086] Specifically, the learning data set selection unit 27 outputs the learning negative example data set with the lowest similarity in the similarity range to the learning processing unit 28 in sequence, and performs the learning processing in the learning processing unit 28. After one learning process is completed, the learning negative example data set with the second highest similarity is output to the learning processing unit 28, and the processing is repeated in the same order until the selection plan of the learning negative example data set is completed.

[0087] In other words, the learning data set selection unit 27 determines the selection plan so that the learning negative example data set corresponding to the similarity range with the first lowest (i.e., lowest) value among the multiple similarity ranges is selected as the first one, and then the learning negative example data set corresponding to the similarity range with the second lowest value among the multiple similarity ranges is selected as the second one. That is, the learning data set selection unit 27 determines the selection plan so that when n is an integer greater than 1, the learning negative example data set corresponding to the similarity range with the nth lowest value among the multiple similarity ranges is selected as the nth one.

[0088] In this way, by starting with the learning negative example data set with low similarity and gradually learning the learning negative example data set with high similarity, the semantic relationship learning device 2 can converge to a better solution earlier. In addition, the generalization performance of the semantic relationship discrimination model obtained by the semantic relationship learning device 2 can be improved. In addition, in the selection plan of the learning negative example data set, any number of learning times can also be set. In addition, in order to maximize the accuracy of the semantic relationship learning device 2, the optimization results based on Bayesian optimization and the like can also be used to determine the selection plan of the learning negative example data set.

[0089] That is, in the learning for discriminating the meaning relationship of electronic data, negative example data pairs for learning that take similarity into consideration are prepared in advance, and learning is first performed on simple problem settings with low difficulty (for example, problems using negative example data pairs for learning composed of language data with low similarity), and then learning is performed on problem settings with high difficulty (for example, problems using negative example data pairs for learning composed of language data with high similarity). In this way, the meaning relationship learning device 2 can not only converge to a better solution earlier, but also improve the generalization performance of the meaning relationship discriminating device equipped with the meaning relationship learning device 2.

[0090] The learning processing unit 28 performs machine learning processing using the learning negative example dataset A27 selected by the learning dataset selection unit 27 and the plurality of positive example data pairs A21 stored in the positive example data storage unit 21. The function of the learning processing unit 28 is the same as that of the learning processing unit 18 in the first embodiment.

[0091] In addition, the hardware structure of the semantic relationship learning device 2 of the second embodiment is the same as Figure 2 The hardware structure shown is the same.

[0092] 《2-2》Actions of implementation method 2.

[0093] Fig.11 This is a flowchart showing the operation of the semantic relationship learning device 2 according to the second embodiment. Fig.121 is a flowchart showing the operation of the negative example data generating unit 22 in Embodiment 2. First, the negative example data generating unit 22 obtains a plurality of positive example data pairs A21 from the positive example data storing unit 21, thereby obtaining language data constituting the plurality of positive example data pairs A21, namely, question sentence data and answer sentence data (step S21, step S211).

[0094] Next, the negative data generation unit 22 generates a plurality of negative data pairs A22 by combining the question sentence data obtained from the positive data storage unit 21 and combining the answer sentences (steps S22, S212, S213). The plurality of negative data pairs A22 are each composed of pairs of language data that are not in a predetermined semantic relationship. That is, the negative data generation unit 22 generates a plurality of negative data pairs A22 based on a combination of language data other than the positive data pair A21. The negative data generation unit 22 outputs the negative data pairs A22 to the feature extraction unit 23 and the similarity calculation unit 24 (step S214).

[0095] Fig.13 2 is a flowchart showing the operation of the feature extraction unit 23 in Embodiment 2. The feature extraction unit 23 obtains a plurality of negative example data pairs A22, extracts feature quantities A23 from the language data constituting the plurality of negative example data pairs A22, that is, calculates feature quantities A23 (steps S23, S231, S232). The feature extraction unit 23 outputs the feature quantities A23 to the similarity calculation unit 24 (step S233).

[0096] Fig.14 2 is a flowchart showing the operation of the similarity calculation unit 24 in Embodiment 2. The similarity calculation unit 24 obtains a plurality of negative data pairs A22 and a plurality of feature quantities A23, and calculates the similarity between the feature quantities A23 of each of the plurality of negative data pairs A22 (steps S24, S241, S242). The similarity calculation unit 24 assigns similarity to the negative data pairs, and outputs the negative data pairs A24 assigned with similarity to the learning negative data generation unit 25 (steps S24, S243, S244).

[0097] Fig.15: is a flowchart showing the operation of the learning negative data generation unit 25 in the second embodiment. The learning negative data generation unit 25 classifies the multiple learning negative data pairs generated based on the language data constituting the multiple negative data pairs A24 and the language data constituting the multiple positive data pairs A21 into a plurality of predetermined similarity ranges based on similarity, thereby generating a plurality of learning negative data sets A25 (step S25, step S251, S252). That is, the multiple learning negative data sets A25 are groups of data including one or more negative data pairs in the multiple negative data pairs A24. The multiple learning negative data sets A25 corresponding to the multiple similarity ranges are stored in the learning negative data storage unit 26 (step S253).

[0098] The operations of the learning data set selection unit 27 and the learning processing unit 28 are the same as those of the learning data set selection unit 17 and the learning processing unit 18 in the first embodiment.

[0099] 《2-3》Effects of Implementation Method 2.

[0100] As described above, by using the semantic relationship learning device 2, the semantic relationship learning method, or the semantic relationship learning program of the second embodiment, it is possible to perform machine learning processing that can achieve good discrimination of semantic relationships.

[0101] Specifically, negative data pairs A24 considering similarity are prepared in advance, and learning is first performed using a set of learning negative data pairs consisting of language data with low similarity, and then learning is performed using a set of learning negative data pairs consisting of language data with the second lowest similarity. That is, learning is gradually advanced to a problem setting with a high degree of difficulty (i.e., a problem setting using a set of learning negative data pairs consisting of language data with high similarity). In this way, not only can a better solution be converged earlier, but also generalization performance can be improved.

[0102] Description of symbols

[0103] 1, 2 semantic relationship learning device, 11, 21 positive example data storage unit, 12, 22 negative example data generation unit, 13, 23 feature quantity extraction unit, 14, 24 similarity calculation unit, 15, 25 learning negative example data generation unit, 16, 26 learning negative example data storage unit, 17, 27 learning data set selection unit, 18, 28 learning processing unit.

Claims

1. A semantic relationship learning device, characterized in that: The meaning relationship learning device comprises: a negative example data generating unit, which acquires a plurality of positive example data pairs from a positive example data storing unit storing a plurality of positive example data pairs, wherein the plurality of positive example data pairs are respectively composed of language data that are in a predetermined semantic relationship with each other, and generates a plurality of negative example data pairs by combining the language data, wherein the plurality of negative example data pairs are respectively composed of language data that are not in the predetermined semantic relationship with each other; a feature extraction unit that extracts feature values ​​from the language data that respectively constitute the plurality of negative example data pairs; a similarity calculation unit that calculates the similarity between the feature quantities of the language data of each of the plurality of negative example data pairs, and assigns the similarity to the negative example data pair corresponding to the similarity; a learning negative data generating unit, which generates a plurality of learning negative data pairs by replacing the language data constituting the plurality of positive data pairs with the language data constituting the plurality of negative data pairs assigned the similarity, and assigns the similarity to the plurality of learning negative data pairs, classifies the plurality of learning negative data pairs into a plurality of predetermined similarity ranges, thereby generating a plurality of learning negative data sets corresponding to the plurality of similarity ranges, wherein the plurality of learning negative data sets respectively include one or more negative data pairs among the plurality of negative data pairs; a learning data set selection unit configured to select any one of the plurality of learning negative example data sets in an order according to a selection plan predetermined based on the plurality of similarity ranges; as well as a learning processing unit that performs machine learning processing using the negative example data set for learning selected by the learning data set selection unit and the plurality of positive example data pairs, The learning dataset selection unit determines the selection plan such that, when n is an integer greater than 1, the learning negative example dataset corresponding to the similarity range with the nth lowest value among the plurality of similarity ranges is selected as the nth one.

2. The semantic relationship learning device according to claim 1, characterized in that: The language data in the predetermined semantic relationship are language data in a synonymous relationship with each other, The language data not in the predetermined semantic relationship are language data not in a synonymous relationship with each other.

3. The semantic relationship learning device according to claim 1, characterized in that: The language data in the predetermined semantic relationship are language data representing a question sentence and language data representing an answer sentence, The language data not in the predetermined semantic relationship are the first language data representing a question sentence and the second language data representing a question sentence, or the third language data representing an answer sentence and the fourth language data representing an answer sentence, The learning negative example data generating unit generates the plurality of learning negative example data sets by classifying the plurality of learning negative example data pairs generated from the plurality of negative example data pairs and the plurality of positive example data pairs into a plurality of predetermined similarity ranges based on the similarities.

4. The semantic relationship learning device according to claim 1, characterized in that: The language data in the predetermined semantic relationship are language data representing a question sentence and language data representing an answer sentence, The language data not in the predetermined semantic relationship are the first language data representing a question sentence and the second language data representing a question sentence, or the third language data representing an answer sentence and the fourth language data representing an answer sentence, The learning negative data generation unit replaces the question sentence data of each of the multiple positive data pairs with the first language data or the second language data of any negative data pair among the multiple negative data pairs, or replaces the answer sentence data of each of the multiple positive data pairs with the third language data or the fourth language data of any negative data pair among the multiple negative data pairs, thereby classifying the generated multiple learning negative data pairs into a plurality of predetermined similarity ranges, thereby generating the multiple learning negative data sets.

5. The semantic relationship learning device according to any one of claims 1 to 4, characterized in that: The semantic relationship learning device further includes the positive example data storage unit storing the plurality of positive example data pairs.

6. The semantic relationship learning device according to any one of claims 1 to 4, characterized in that: The semantic relationship learning device further includes a learning negative example data storage unit that stores the plurality of learning negative example data sets generated by the learning negative example data generation unit.

7. The semantic relationship learning device according to any one of claims 1 to 4, characterized in that: The semantic relationship learning device further includes a storage unit that stores a result of the machine learning process performed by the learning processing unit.

8. A method for learning meaning relations, characterized in that: The meaning relationship learning method has the following features: A negative example data generating step, obtaining a plurality of positive example data pairs from a positive example data storing unit storing a plurality of positive example data pairs, wherein the plurality of positive example data pairs are respectively composed of language data having a predetermined semantic relationship with each other, and generating a plurality of negative example data pairs by combining the language data, wherein the plurality of negative example data pairs are respectively composed of language data not having the predetermined semantic relationship with each other; An extraction step of extracting feature quantities from the language data respectively constituting the plurality of negative example data pairs; a calculation step of calculating the similarity between the feature quantities of the language data of each of the plurality of negative example data pairs, and assigning the similarity to the negative example data pairs corresponding to the similarity; a step of generating negative data for learning, using the language data constituting the multiple negative data pairs assigned with the similarity to replace the language data constituting the multiple positive data pairs to generate multiple negative data pairs for learning, and assigning the similarity to the multiple negative data pairs for learning, classifying the multiple negative data pairs for learning into a plurality of predetermined similarity ranges, thereby generating a plurality of negative data sets for learning corresponding to the plurality of similarity ranges, wherein the plurality of negative data sets for learning respectively include one or more negative data pairs among the multiple negative data pairs; A selection step of selecting any one of the plurality of negative example data sets for learning in accordance with an order of a selection plan predetermined based on the plurality of similarity ranges; as well as A learning step, using the selected learning negative example data set and the plurality of positive example data pairs to perform machine learning processing, Among them, in the selection step, the selection plan is determined so that, when n is an integer greater than 1, the learning negative example data set corresponding to the similarity range with the nth lowest value among the multiple similarity ranges is selected as the nth one.

9. A recording medium recording a meaning relationship learning program, characterized in that: The meaning relationship learning program causes the computer to execute the following processing: Negative example data generation processing, obtaining a plurality of positive example data pairs from a positive example data storage unit storing a plurality of positive example data pairs, the plurality of positive example data pairs respectively consisting of language data having a predetermined semantic relationship with each other, and generating a plurality of negative example data pairs by combining the language data, the plurality of negative example data pairs respectively consisting of language data not having the predetermined semantic relationship with each other; Extraction processing, extracting feature quantities from the language data respectively constituting the plurality of negative example data pairs; Calculating the similarity between the feature quantities of the language data of the plurality of negative example data pairs, and assigning the similarity to the negative example data pairs corresponding to the similarity; A process for generating negative data for learning, wherein the language data constituting the plurality of negative data pairs assigned with the similarity are used to replace the language data constituting the plurality of positive data pairs to generate a plurality of negative data pairs for learning, and the similarity is assigned to the plurality of negative data pairs for learning, and the plurality of negative data pairs for learning are classified into a plurality of predetermined similarity ranges, thereby generating a plurality of negative data sets for learning corresponding to the plurality of similarity ranges, wherein the plurality of negative data sets for learning respectively include one or more negative data pairs among the plurality of negative data pairs; A selection process for selecting any one of the plurality of learning negative example data sets in accordance with an order of a selection plan predetermined based on the plurality of similarity ranges; as well as Learning processing, using the selected learning negative example data set and the plurality of positive example data pairs to perform machine learning processing, In the selection process, the selection plan is determined such that, when n is an integer greater than 1, the negative example dataset for learning corresponding to the similarity range with the nth lowest value among the plurality of similarity ranges is selected as the nth one.

Images