Information processing device, generation method, and generation program
The information processing device uses supervised learning to enhance the estimation accuracy of trained models by associating learning data with documents and categories, extracting predicate words, and calculating self-mutual information, effectively addressing the low accuracy issues in unsupervised learning for word meaning estimation in AI technologies.
Patent Information
- Application Number
- PCT/JP2023/018077
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-15
- Publication Date
- 2024-11-21
AI Technical Summary
Unsupervised learning methods result in trained models with low estimation accuracy for word meaning estimation in AI technologies.
An information processing device employing supervised learning to generate trained models by acquiring learning data associated with documents and categories, performing morphological analysis, extracting predicate words, and calculating self-mutual information to improve estimation accuracy.
The solution enables the generation of trained models with high estimation accuracy, reducing the amount of training data required and improving performance compared to unsupervised learning methods.
Smart Images

Figure JP2023018077_21112024_PF_FP_ABST
Abstract
Description
Information processing device, generation method, and generation program
[0001] The present disclosure relates to an information processing device, a generation method, and a generation program.
[0002] Artificial intelligence (AI) technology is used in the field of language. For example, a trained model that estimates the meaning of words contained in a string of characters has been proposed (see Patent Literature 1). The trained model in Patent Literature 1 is generated by unsupervised learning.
[0003] International Publication No. 2022 / 049668
[0004] When unsupervised learning is used as in the above technology, there is a problem in that the estimation accuracy of the trained model generated by unsupervised learning is low.
[0005] The objective of the present disclosure is to generate a trained model with high estimation accuracy.
[0006] According to one aspect of the present disclosure, there is provided an information processing device, the information processing device including: an acquisition unit that acquires a plurality of training data sets in which documents and categories are associated with each other; a morphological analysis execution unit that performs a morphological analysis on each of the plurality of training data sets; an extraction unit that extracts a word that is a predicate from a plurality of words obtained by the morphological analysis; and a calculation / generation unit that calculates mutual information based on the plurality of words obtained by the morphological analysis, the extracted plurality of words, and a plurality of the categories, to generate a trained model that outputs a category corresponding to the input data when the data is input.
[0007] According to the present disclosure, it is possible to generate a trained model with high estimation accuracy.
[0008] 1 is a diagram illustrating a hardware configuration of an information processing device of embodiment 1. FIG. 2 is a block diagram illustrating functions of an information processing device in the learning phase of embodiment 1. FIG. 3 is a diagram illustrating a specific example of processing executed by an information processing device of embodiment 1. (A) and (B) are diagrams illustrating an example of an image of a trained model of embodiment 1. FIG. 4 is a block diagram illustrating functions of an information processing device in the utilization phase of embodiment 1. FIG. 5 is a diagram illustrating an example of an image of a trained model of embodiment 2. FIG. 6 is a block diagram illustrating functions of an information processing device in the learning phase of embodiment 3. FIG. 7 is a diagram illustrating a specific example of processing executed by an information processing device of embodiment 3. FIG. 8 is a diagram illustrating an example of an image of a first trained model of embodiment 3. FIG. 9 is a block diagram illustrating functions of an information processing device in the utilization phase of embodiment 3. FIG. 10 is a diagram illustrating a specific example of processing performed by a second trained model of embodiment 3.
[0009] Hereinafter, embodiments will be described with reference to the drawings. The following embodiments are merely examples, and various modifications are possible within the scope of the present disclosure.
[0010] Embodiment 1. <Learning Phase> Fig. 1 is a diagram showing the hardware configuration of an information processing device according to embodiment 1. The information processing device 100 is a device that executes a generation method. The information processing device 100 may also be called a learning device. The information processing device 100 may also be called a computer. The information processing device 100 has a processor 101, a volatile storage device 102, and a non-volatile storage device 103.
[0011] The processor 101 controls the entire information processing device 100. For example, the processor 101 is a central processing unit (CPU) or a field programmable gate array (FPGA). The processor 101 may be a multiprocessor. The information processing device 100 may also include a processing circuit.
[0012] The volatile storage device 102 is a main storage device of the information processing device 100. For example, the volatile storage device 102 is a random access memory (RAM). The nonvolatile storage device 103 is an auxiliary storage device of the information processing device 100. For example, the nonvolatile storage device 103 is a hard disk drive (HDD) or a solid state drive (SSD).
[0013] Next, a description will be given of the functions of the information processing device 100. Fig. 2 is a block diagram showing the functions of the information processing device in the learning phase of embodiment 1. The information processing device 100 has a storage unit 110, an acquisition unit 120, a morphological analysis execution unit 130, an extraction unit 140, and a calculation / generation unit 150.
[0014] The storage unit 110 may be realized as a storage area secured in the volatile storage device 102 or the non-volatile storage device 103. Some or all of the acquisition unit 120, the morphological analysis execution unit 130, the extraction unit 140, and the calculation and generation unit 150 may be realized by a processing circuit. Some or all of the acquisition unit 120, the morphological analysis execution unit 130, the extraction unit 140, and the calculation and generation unit 150 may be realized as program modules executed by the processor 101. For example, the program executed by the processor 101 is also referred to as a generation program. For example, the generation program is recorded on a recording medium.
[0015] The acquiring unit 120 acquires a plurality of pieces of training data. For example, the acquiring unit 120 acquires the plurality of pieces of training data from the storage unit 110. Alternatively, for example, the acquiring unit 120 acquires the plurality of pieces of training data from an external device. For example, the external device is a cloud server. Note that the external device is not illustrated. Each of the plurality of pieces of training data is associated with a document and a category. Also, a document may be expressed as a character string. Note that a category may be considered as a label in supervised learning.
[0016] The morphological analysis execution unit 130 executes morphological analysis on each of the plurality of training data.
[0017] The extraction unit 140 extracts predicate words from among the multiple words obtained by the morphological analysis. For example, the extraction unit 140 extracts predicate words for each result of the morphological analysis of the training data. In detail, the extraction unit 140 performs the following process for each document. The extraction unit 140 extracts predicate words from among the multiple words obtained by performing morphological analysis on the document. Note that the predicate words are verbs, adjectives, adjectival verbs, or sa-hen nouns.
[0018] Here, the processes performed by the acquisition unit 120, the morphological analysis execution unit 130, and the extraction unit 140 will be described with reference to the drawings.
[0019] FIG. 3 is a diagram showing a specific example of processing executed by the information processing apparatus according to the first embodiment. FIG. 3 shows a plurality of learning data. For example, the acquiring unit 120 acquires “Document 1” and category “c 1 Here, the set C of categories is expressed by equation (1).
[0020]
[0021] The morphological analysis execution unit 130 performs morphological analysis on each of the multiple training data. Note that "W" in Fig. 3 indicates a set of words. Also, "w" in Fig. 3 indicates a word obtained by the morphological analysis.
[0022] The extraction unit 140 extracts predicate words for each result of morphological analysis of the training data. For example, the extraction unit 140 extracts predicate words from "W" in "Document 1." Note that "V" in FIG. 3 indicates a set of predicate words. Also, "v" in FIG. 3 indicates a predicate word.
[0023] The calculation and generation unit 150 generates a trained model by calculating pointwise mutual information (PMI) based on the multiple words obtained by the morphological analysis, the multiple extracted words (i.e., multiple words that are predicates), and the multiple categories. The calculation and generation process will be described in detail. i , w j , cp Specifically, the calculation / generation unit 150 calculates the PMI using formula (2). P indicates the occurrence probability in the document that is the learning data. For example, P(v i ) is the predicate word v in a document. i In addition, i, j, and p are arbitrary values.
[0024]
[0025] If the PMI is negative, the PMI is set to 0. The trained model is i , w j , c p ) In this way, the calculation and generation unit 150 generates a trained model. When data is input, the trained model can output a category corresponding to the data. The trained model can also output a likelihood. Here, the trained model can be expressed as follows:
[0026] 4(A) and (B) are diagrams showing an example of an image of the trained model of embodiment 1. For example, the trained model is expressed as shown in FIG. 4(A). FIG. 4(B) shows the trained model of FIG. 4(A) expressed in a table. Information indicating the correspondence between word w and predicate word v is two-dimensional information. Information indicating the correspondence between category c and the two-dimensional information is three-dimensional information. Therefore, the trained model is expressed as three-dimensional information. The trained model may also be expressed as a third-order tensor.
[0027] Furthermore, the calculation generating unit 150 calculates v i , w j If the number of occurrences of is equal to or less than a predetermined threshold, the PMI (v i , w j , c p ) may be corrected. In other words, the calculation and generation unit 150 may correct the trained model. Specifically, the calculation and generation unit 150 performs the correction using equation (3).
[0028]
[0029] In this way, vi , w j If the number of occurrences of is equal to or less than the threshold, it can be said that the amount of learning required to generate a trained model is small. Therefore, the calculation and generation unit 150 corrects the trained model. This allows the information processing device 100 to improve the estimation accuracy of the trained model. The calculation and generation unit 150 stores the trained model in the storage unit 110. The calculation and generation unit 150 may store the trained model in an external device.
[0030] Here, when unsupervised learning is used, there is a problem in that the estimation accuracy of the trained model generated by unsupervised learning is low.
[0031] According to the first embodiment, the information processing device 100 generates a trained model using supervised learning. The trained model generated by supervised learning has high estimation accuracy. Therefore, the information processing device 100 can generate a trained model with high estimation accuracy.
[0032] Furthermore, when unsupervised learning is used, a large amount of training data is used. On the other hand, in supervised learning, a trained model can be generated with a small amount of training data. Therefore, the information processing device 100 can generate a trained model with a small amount of training data.
[0033] 5 is a block diagram showing functions of an information processing device in the utilization phase of embodiment 1. The information processing device 100a includes a storage unit 110a, an acquisition unit 120a, a morphological analysis execution unit 130a, an extraction unit 140a, an estimation unit 150a, and an output unit 160a. The information processing device 100a may also be referred to as an estimation device.
[0034] Here, the information processing device 100 and the information processing device 100a may be the same device or different devices. For example, if the information processing device 100 and the information processing device 100a are the same device, the information processing device 100a further includes an estimation unit 150a and an output unit 160a. Also, if the information processing device 100 and the information processing device 100a are the same device, the storage unit 110 and the storage unit 110a may be considered to be the same device. Furthermore, if the information processing device 100 and the information processing device 100a are the same device, the functions of the acquisition unit 120a, the morphological analysis execution unit 130a, and the extraction unit 140a may be considered to be the same as the functions of the acquisition unit 120, the morphological analysis execution unit 130, and the extraction unit 140.
[0035] The storage unit 110a may be realized as a storage area reserved in a volatile storage device or a non-volatile storage device included in the information processing device 100a. Some or all of the acquisition unit 120a, the morphological analysis execution unit 130a, the extraction unit 140a, the estimation unit 150a, and the output unit 160a may be realized by processing circuits included in the information processing device 100a. Some or all of the acquisition unit 120a, the morphological analysis execution unit 130a, the extraction unit 140a, the estimation unit 150a, and the output unit 160a may be realized as program modules executed by a processor included in the information processing device 100a.
[0036] The acquiring unit 120a acquires data including characters. For example, the acquiring unit 120a acquires the data from the storage unit 110a. Alternatively, for example, the acquiring unit 120a acquires the data from an external device. Also, the acquiring unit 120a acquires a trained model. For example, the acquiring unit 120a acquires the trained model from the storage unit 110a. Alternatively, for example, the acquiring unit 120a acquires the trained model from an external device.
[0037] The morphological analysis execution unit 130a executes morphological analysis on the data. For example, a set W of words obtained by the morphological analysis is expressed by equation (4).
[0038]
[0039] The extraction unit 140a extracts words that are predicates from the results of the morphological analysis. For example, a set V of extracted words is expressed by equation (5).
[0040]
[0041] The estimation unit 150a estimates a category corresponding to the data acquired by the acquisition unit 120a using multiple words obtained by morphological analysis, extracted words (i.e., words that are predicates), and a trained model.
[0042] The trained model is, as expressed by equation (6), a value L(c p ) is calculated.
[0043]
[0044] When the trained model is corrected, equation (6) is expressed as equation (7).
[0045]
[0046] For each category, the value L(c p After calculating , the trained model outputs the category C corresponding to the maximum value, as shown in equation (8).
[0047]
[0048] In this way, it is estimated that the category corresponding to the data acquired by the acquisition unit 120a is the category CM. In addition, the trained model outputs a likelihood.
[0049] The output unit 160a outputs the category CM and the likelihood. For example, the output unit 160a outputs the category CM and the likelihood to a display of the information processing device 100a.
[0050] Second Embodiment Next, a second embodiment will be described. In the second embodiment, differences from the first embodiment will be mainly described. Furthermore, in the second embodiment, descriptions of the commonalities between the first embodiment and the second embodiment will be omitted.
[0051] <Learning Phase> The processes of the acquisition unit 120, morphological analysis execution unit 130, and extraction unit 140 in the second embodiment are the same as the processes of the acquisition unit 120, morphological analysis execution unit 130, and extraction unit 140 in the first embodiment.
[0052] The calculation and generation unit 150 generates a trained model by calculating self-mutual information based on a plurality of words obtained by morphological analysis, a plurality of extracted words (i.e., a plurality of words that are predicates), and a plurality of categories. However, when calculating the self-mutual information, the calculation and generation unit 150 selects two words from the plurality of words obtained by morphological analysis and calculates the self-mutual information using the two selected words. The calculation and generation process will be described in detail. When calculating the self-mutual information, the calculation and generation unit 150 selects two words from the plurality of words obtained by morphological analysis and calculates the self-mutual information using the two selected words. The calculation and generation process will be described in detail. 1 ~w n From among these, the word w j and the word w k The calculation and generation unit 150 selects v i , w j , w k , c p co-occur. Specifically, the calculation and generation unit 150 calculates the PMI using equation (9). Note that P indicates the occurrence probability in the document that is the training data. Also, i, j, k, and p are arbitrary values.
[0053]
[0054] If the PMI is negative, the PMI is set to 0. The trained model is i , w j , w k , c p ) In this way, the calculation and generation unit 150 generates a trained model. The trained model can output a category and a likelihood. Here, the trained model can be expressed as follows:
[0055] Fig. 6 is a diagram illustrating an example of an image of a trained model according to embodiment 2. As shown in Fig. 6, the trained model is represented as four-dimensional information. The trained model may also be represented as a fourth-order tensor.
[0056] The trained model shown in Fig. 4 is three-dimensional information. On the other hand, the trained model shown in Fig. 6 is four-dimensional information. Therefore, the trained model shown in Fig. 6 has higher estimation accuracy than the trained model shown in Fig. 4. Therefore, according to the second embodiment, the information processing device 100 can generate a trained model with higher estimation accuracy.
[0057] Furthermore, the calculation generating unit 150 calculates v i , w j , w k If the number of occurrences of is equal to or less than a predetermined threshold, the PMI (v i , w j , w k , c p ) may be corrected. In other words, the calculation and generation unit 150 may correct the trained model. Specifically, the calculation and generation unit 150 performs the correction using equation (10).
[0058]
[0059] In this way, v i , w j , w k If the number of occurrences of is equal to or less than the threshold, it can be said that the amount of learning required to generate a trained model is small. Therefore, the calculation and generation unit 150 corrects the trained model. This allows the information processing device 100 to improve the estimation accuracy of the trained model. The calculation and generation unit 150 stores the trained model in the storage unit 110. The calculation and generation unit 150 may store the trained model in an external device.
[0060] <Utilization Phase> The processes of the acquisition unit 120a, morphological analysis execution unit 130a, and extraction unit 140a in the second embodiment are the same as those of the acquisition unit 120a, morphological analysis execution unit 130a, and extraction unit 140a in the first embodiment.
[0061] The estimation unit 150a estimates a category corresponding to the data acquired by the acquisition unit 120a, using multiple words obtained by morphological analysis, extracted words (i.e., words that are predicates), and a trained model. The trained model estimates a value L(c p ) is calculated.
[0062]
[0063] When the trained model is corrected, equation (11) is expressed as equation (12).
[0064]
[0065] For each category, the value L(c p ) is calculated, the trained model outputs the category C M corresponding to the maximum value, as expressed by equation (8). In this way, it is estimated that the category corresponding to the data acquired by the acquisition unit 120a is the category C M. The trained model also outputs the likelihood.
[0066] The output unit 160a outputs the category CM and the likelihood. For example, the output unit 160a outputs the category CM and the likelihood to a display of the information processing device 100a.
[0067] Third Embodiment Next, a third embodiment will be described. In the third embodiment, a method for generating a trained model for estimating categories will be described, which is different from the methods in the first and second embodiments.
[0068] <Learning Phase> Fig. 7 is a block diagram showing functions of an information processing device in the learning phase of embodiment 3. The information processing device 200 is a device that executes the generation method. The information processing device 200 has a storage unit 210, an acquisition unit 220, a morphological analysis execution unit 230, an extraction unit 240, a calculation and generation unit 250, and a generation unit 260. The information processing device 200 may also be called a computer.
[0069] The storage unit 210 may be realized as a storage area secured in a volatile storage device or a non-volatile storage device included in the information processing device 200. Some or all of the acquisition unit 220, the morphological analysis execution unit 230, the extraction unit 240, the calculation / generation unit 250, and the generation unit 260 may be realized by a processing circuit. Some or all of the acquisition unit 220, the morphological analysis execution unit 230, the extraction unit 240, the calculation / generation unit 250, and the generation unit 260 may be realized as program modules executed by a processor included in the information processing device 200. For example, the program executed by the processor is also referred to as a generation program. For example, the generation program is recorded on a recording medium.
[0070] The acquiring unit 220 acquires a plurality of pieces of training data. For example, the acquiring unit 220 acquires the plurality of pieces of training data from the storage unit 210 or an external device. Each of the plurality of pieces of training data is associated with a document and a category. The category may be considered as a label in supervised learning.
[0071] The morphological analysis execution unit 230 executes morphological analysis on each of the plurality of training data.
[0072] The extraction unit 240 extracts predicate words from among the multiple words obtained by the morphological analysis. For example, the extraction unit 240 extracts predicate words for each result of the morphological analysis of the training data. Note that the predicate words are verbs, adjectives, adjectival verbs, or sa-hen nouns. Here, the processes performed by the acquisition unit 220, the morphological analysis execution unit 230, and the extraction unit 240 will be described with reference to the drawings.
[0073] FIG. 8 is a diagram showing a specific example of processing executed by the information processing apparatus according to the third embodiment. FIG. 8 shows a plurality of learning data. For example, the acquiring unit 220 acquires “Document 1” and category “c 1" is acquired. Here, categories are not used in the following processing. Therefore, the category will be omitted. The morphological analysis execution unit 230 performs morphological analysis on each of the multiple pieces of training data. The extraction unit 240 extracts words that are predicates for each result of the morphological analysis of the training data. For example, the extraction unit 240 extracts words that are predicates from "W" in "Document 1".
[0074] The calculation and generation unit 250 calculates the self-mutual information based on the plurality of words obtained by the morphological analysis and the plurality of extracted words (i.e., the plurality of words that are predicates), thereby generating a first trained model. However, when calculating the self-mutual information, the calculation and generation unit 250 selects two words from the plurality of words obtained by the morphological analysis, and calculates the self-mutual information using the two selected words. The calculation and generation process will be described in detail. When calculating the self-mutual information, the calculation and generation unit 250 selects two words from the plurality of words obtained by the morphological analysis, and calculates the self-mutual information using the two selected words. The calculation and generation process will be described in detail. 1 ~w n From among these, the word w j and the word w k The calculation and generation unit 250 selects v i , w j , w k Specifically, the calculation generation unit 250 calculates the PMI using equation (13), where i, j, and k are arbitrary values.
[0075]
[0076] If the PMI is negative, the PMI is set to 0. The first trained model uses the PMI(v i , w j , w k ) In this way, the calculation and generation unit 250 generates the first trained model. Here, the first trained model can be expressed as follows:
[0077] 9 is a diagram illustrating an example of an image of the first trained model according to the third embodiment. As illustrated in FIG. 9, the first trained model is represented as three-dimensional information. The first trained model may also be represented as a third-order tensor.
[0078] Furthermore, the calculation generating unit 250 calculates v i , w j , w k If the number of occurrences of is equal to or less than a predetermined threshold, the PMI (v i , w j , w k ) may be corrected. In other words, the calculation and generation unit 250 may correct the first trained model. Specifically, the calculation and generation unit 250 performs the correction using equation (14). In addition, when j and k are arbitrary values, w j , w k is a permutation. When learning is performed with j<k, the words contained in the sentence may be swapped and learning may be performed. For example, in the sentence "A child watching a cat", w j It's a "cat", lol k becomes a "child". The "child" and the "cat" are interchanged. j But "children" lol k becomes "cat." This allows learning of different sentence meanings, such as "cat watching child" and "child watching cat." Therefore, it is possible to learn word order-dependent meanings that cannot be learned with conventional Bag of Words (BoW).
[0079]
[0080] In this way, v i , w j , w k If the number of occurrences of is equal to or less than the threshold, it can be said that the amount of learning required to generate the first trained model is small. Therefore, the calculation and generation unit 250 corrects the first trained model. This allows the information processing device 200 to improve the estimation accuracy of the first trained model.
[0081] The calculation and generation unit 250 stores the first trained model in the storage unit 210. The calculation and generation unit 250 may store the first trained model in an external device.
[0082] The generation unit 260 generates a second trained model that outputs a category corresponding to input data based on a plurality of training data, a first trained model, and a predetermined method. Here, the predetermined method is a conventional method used in machine learning. Examples of conventional methods include Support Vector Machine and Random Forest. For example, the generation unit 260 generates the second trained model by performing training using a plurality of training data and the first trained model using a predetermined method so as to output a category corresponding to the data. Furthermore, in conventional techniques, words in documents that are training data are converted into numerical vectors such as one-hot vectors and tfidf vectors. Then, the numerical vectors are associated with categories, and training is performed. In this way, in the training phase, learning is performed using vectors.
[0083] Furthermore, the generation unit 260 may generate a second trained model that outputs a category and a likelihood. The generation unit 260 stores the second trained model in the storage unit 210. The generation unit 260 may store the second trained model in an external device.
[0084] Here, when unsupervised learning is used, there is a problem in that the estimation accuracy of the trained model generated by unsupervised learning is low.
[0085] According to the third embodiment, the information processing device 200 generates a second trained model using supervised learning. The trained model generated by supervised learning has high estimation accuracy. Therefore, the information processing device 200 can generate a trained model with high estimation accuracy.
[0086] 10 is a block diagram showing functions of an information processing device in the utilization phase of embodiment 3. The information processing device 200a includes a storage unit 210a, an acquisition unit 220a, a morphological analysis execution unit 230a, an extraction unit 240a, an estimation unit 250a, and an output unit 260a. The information processing device 200a may also be called an estimation device.
[0087] Here, the information processing device 200 and the information processing device 200a may be the same device or different devices. For example, if the information processing device 200 and the information processing device 200a are the same device, the information processing device 200a further includes an estimation unit 250a and an output unit 260a. Also, if the information processing device 200 and the information processing device 200a are the same device, the storage unit 210 and the storage unit 210a may be considered to be the same device. Furthermore, if the information processing device 200 and the information processing device 200a are the same device, the functions of the acquisition unit 220a, the morphological analysis execution unit 230a, and the extraction unit 240a may be considered to be the same as the functions of the acquisition unit 220, the morphological analysis execution unit 230, and the extraction unit 240.
[0088] The storage unit 210a may be realized as a storage area secured in a volatile storage device or a non-volatile storage device included in the information processing device 200a. Some or all of the acquisition unit 220a, the morphological analysis execution unit 230a, the extraction unit 240a, the estimation unit 250a, and the output unit 260a may be realized by processing circuits included in the information processing device 200a. Some or all of the acquisition unit 220a, the morphological analysis execution unit 230a, the extraction unit 240a, the estimation unit 250a, and the output unit 260a may be realized as program modules executed by a processor included in the information processing device 200a.
[0089] The acquiring unit 220a acquires data including characters. For example, the acquiring unit 220a acquires the data from the storage unit 210a. Also, for example, the acquiring unit 220a acquires the data from an external device.
[0090] The acquiring unit 220a also acquires the second trained model. For example, the acquiring unit 220a acquires the second trained model from the storage unit 210a. For example, the acquiring unit 220a also acquires the second trained model from an external device.
[0091] The morphological analysis execution unit 230a performs morphological analysis on the data. The extraction unit 240a extracts predicate words from the results of the morphological analysis. The estimation unit 250a estimates a category corresponding to the data acquired by the acquisition unit 220a using multiple words obtained by the morphological analysis, extracted words (i.e., predicate words), and a second trained model. In detail, the estimation unit 250a vectorizes the multiple words and the extracted words. The estimation unit 250a estimates a category corresponding to the data using sentence vectors of the distributed representation obtained by vectorization and the second trained model. Here, the processing performed by the second trained model will be described using diagrams.
[0092] FIG. 11 is a diagram showing a specific example of processing performed by the second trained model of the third embodiment. FIG. 11 shows the results of morphological analysis. The results of the morphological analysis are "cat," "watching," and "child." j is "cat". i is "watching". k shall be referred to as "children."
[0093] The second trained model detects a vector 11 of the column “cat”. Specifically, the vector 11 is “{PMI(watching, cat, w 1 ), ..., PMI (watching, cat, lol j ), ..., PMI (watching, cat, lol n )}". The second trained model detects a vector 12 in the "child" column. Specifically, the vector 12 is represented as "{PMI(watching, w 1 , children), ..., PMI (watching, lol k , children), ..., PMI (watching, lol n The second trained model detects a vector 13 in the column of "watching". Specifically, the vector 13 is expressed as "{{PMI(watching, w 1 , w 1 ), ..., PMI (I'm watching, lol 1 , w k ), ..., PMI (I'm watching, lol 1 , w n ), ..., {PMI (watching, lolj , w 1 ), ..., PMI (I'm watching, lol j , w k ), ..., PMI (I'm watching, lol j , w n ), ..., {PMI (watching, lol n , w 1 ), ..., PMI (I'm watching, lol n , w k ), ..., PMI (I'm watching, lol n , w n )}}".
[0094] The second trained model concatenates vectors 11, 12, and 13. The second trained model detects the category "child" corresponding to the distributed representation (i.e., sentence vector) represented by the concatenation. For example, in FIG. 11, the distributed representation is represented by plane 14. The second trained model outputs "child". In this way, the category corresponding to the data acquired by the acquisition unit 120a is estimated. The second trained model also outputs a likelihood.
[0095] The output unit 260a outputs the category and the likelihood. For example, the output unit 260a outputs the category and the likelihood to a display of the information processing device 200a.
[0096] The features of the above-described embodiments can be combined with each other as appropriate.
[0097] 11, 12, 13 Vector, 14 Surface, 100 Information processing device, 100a Information processing device, 101 Processor, 102 Volatile storage device, 103 Non-volatile storage device, 110 Storage unit, 110a Storage unit, 120 Acquisition unit, 120a Acquisition unit, 130 Morphological analysis execution unit, 130a Morphological analysis execution unit, 140 Extraction unit, 140a Extraction unit, 150 Calculation / generation unit, 150a Estimation unit, 160a Output unit, 200 Information processing device, 200a Information processing device, 210 Storage unit, 210a Storage unit, 220 Acquisition unit, 220a Acquisition unit, 230 Morphological analysis execution unit, 230a Morphological analysis execution unit, 240 Extraction unit, 240a extraction unit, 250 calculation generation unit, 250a estimation unit, 260 generation unit, 260a output unit.
Claims
1. An information processing device having: an acquisition unit that acquires multiple training data in which documents and categories are associated with each other; a morphological analysis execution unit that performs morphological analysis on each of the multiple training data; an extraction unit that extracts words that are predicates from multiple words obtained by the morphological analysis; and a calculation and generation unit that generates a trained model that outputs a category corresponding to the data when data is input by calculating self-mutual information based on the multiple words obtained by the morphological analysis, the multiple extracted words, and multiple of the categories.
2. The information processing device according to claim 1, wherein the calculation and generation unit corrects the trained model using a constant when the number of occurrences of a word that is a predicate and a word obtained by the morphological analysis is equal to or less than a predetermined threshold.
3. The information processing device of claim 1, wherein, when calculating the self-mutual information, the calculation and generation unit selects two words from the plurality of words obtained by the morphological analysis, calculates the self-mutual information using the two selected words, and generates the trained model which is four-dimensional information.
4. The information processing device according to claim 3, wherein the calculation and generation unit corrects the trained model using a constant when the number of occurrences of a word that is a predicate and two words selected from the multiple words obtained by the morphological analysis is equal to or less than a predetermined threshold.
5. The information processing device according to claim 1 , wherein the calculation and generation unit generates the trained model that outputs the category and the likelihood.
6. An information processing device comprising: an acquisition unit that acquires multiple training data in which documents and categories are associated with each other; a morphological analysis execution unit that performs morphological analysis on each of the multiple training data; an extraction unit that extracts words that are predicates from multiple words obtained by the morphological analysis; a calculation and generation unit that generates a first trained model by calculating self-mutual information based on the multiple words obtained by the morphological analysis and the multiple extracted words; and a generation unit that generates a second trained model that outputs a category corresponding to the data when data is input, based on the multiple training data, the first trained model, and a predetermined method, wherein when calculating the self-mutual information, the calculation and generation unit selects two words from the multiple words obtained by the morphological analysis and calculates the self-mutual information using the two selected words.
7. The information processing device according to claim 6, wherein the calculation and generation unit corrects the first trained model using a constant when the number of occurrences of a word that is a predicate and two words selected from the multiple words obtained by the morphological analysis is equal to or less than a predetermined threshold.
8. The information processing device according to claim 6 or 7, wherein the calculation and generation unit generates the second trained model that outputs the category and the likelihood.
9. A generation method in which an information processing device acquires a plurality of training data in which documents and categories are associated with each other, performs morphological analysis on each of the plurality of training data, extracts a word that is a predicate from a plurality of words obtained by the morphological analysis, and calculates self-mutual information based on the plurality of words obtained by the morphological analysis, the extracted plurality of words, and a plurality of the categories, thereby generating a trained model that, when data is input, outputs a category corresponding to the data.
10. A generation method in which an information processing device acquires a plurality of training data in which documents and categories are associated with each other, performs morphological analysis on each of the plurality of training data, extracts a word that is a predicate from a plurality of words obtained by the morphological analysis, generates a first trained model by calculating self-mutual information based on the plurality of words obtained by the morphological analysis and the extracted plurality of words, generates a second trained model that outputs a category corresponding to the data when data is input, based on the plurality of training data, the first trained model, and a predetermined method, and when calculating the self-mutual information, selects two words from the plurality of words obtained by the morphological analysis, and calculates the self-mutual information using the two selected words.
11. A generation program that causes a computer to execute the following process: acquire multiple training data in which documents and categories are associated with each other; perform morphological analysis on each of the multiple training data; extract a word that is a predicate from multiple words obtained by the morphological analysis; and calculate mutual information based on the multiple words obtained by the morphological analysis, the multiple extracted words, and multiple of the categories, thereby generating a trained model that, when data is input, outputs a category corresponding to the data.
12. A generation program that causes a computer to execute the following processes: acquire multiple training data in which documents and categories are associated with each other; perform morphological analysis on each of the multiple training data; extract words that are predicates from multiple words obtained by the morphological analysis; generate a first trained model by calculating self-mutual information based on the multiple words obtained by the morphological analysis and the extracted multiple words; and generate a second trained model that outputs a category corresponding to the data when data is input, based on the multiple training data, the first trained model, and a predetermined method; wherein when calculating the self-mutual information, the generation program selects two words from the multiple words obtained by the morphological analysis, and calculates the self-mutual information using the two selected words.
Citation Information
Patent Citations
Information processing device, generation method, and generation program
WO2022049668A1
Air conditioner for vehicle and method for controlling the same
JP2008285111A
Document classification method, document classification program and document classification device
JP2014056331A
Abnormality diagnosing model construction method, abnormality diagnosing method, abnormality diagnosing model construction device, and abnormality diagnosing device
WO2022091639A1
