Learning method, estimation method, learning device, estimation device, learning program, and estimation program

Abstract

This training device (10) comprises: a user distribution representation estimation unit (131) that estimates a distribution representation vector of a user, in which features of the user are embedded, on the basis of a user ID by using a first deep learning model and outputs the result of estimation; a text distribution representation estimation unit (132) that estimates a distribution representation vector of first text posted by the user by using a second deep learning model; a text estimation unit (134) that estimates text posted by the user before and / or after the first text on the basis of the distribution representation vector of the user and the distribution representation vector of the text by using a third deep learning model; and a training unit (14) that updates the parameters of the first, second, and third deep learning models on the basis of an error between training data, which consists of the text posted by the user before and / or after the first text, and the text estimated by the text estimation unit (134).

Description

[Technical field]

[0001] The present invention relates to a learning method, an estimation method, a learning device, an estimation device, a learning program, and an estimation program. [Background technology]

[0002] In recent years, in the field of natural language processing, embedding techniques that convert word and text information into distributed representations such as vectors have been attracting attention. Methods for acquiring distributed representations include Word2Vec (Non-Patent Document 1), which acquires word embeddings, BERT (Non-Patent Document 2), which acquires word embeddings for sentences, and Doc2Vec (Non-Patent Document 3), which acquires document embeddings. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Mikolov et al., “Distributed Representations of Words and Phrases and their Compositionality”, 2013. [Non-Patent Document 2] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”, 2019. [Non-Patent Document 3] Quoc V. Le, Tomas Mikolov, “Distributed Representations of Sentences and Document”, 2014. Summary of the Invention [Problem to be solved by the invention]

[0004] With the development of social network services such as microblogging, many people are posting and publishing text. In many cases, a method for obtaining distributed representations of documents, such as Doc2Vec, is used to obtain a distributed representation by treating a series of posts by a user as a single document.

[0005] Text posted on social networks reflects the user's thoughts and values. However, conventional distributed representations focus on the words and sentences themselves, and do not take into account the human who created the sentence. For this reason, it is difficult to obtain distributed representations that take the user into account using conventional methods.

[0006] The present invention has been made in consideration of the above, and aims to provide a learning method, an estimation method, a learning device, an estimation device, a learning program, and an estimation program that are targeted at users who post text and can obtain distributed representations of the users themselves. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, the learning method of the present invention is a learning method executed by a learning device, and includes the steps of: accepting as input a user's identification information and a first text posted by the user; converting the first text into a second text in a format that can be input to a deep learning model; using the first deep learning model, estimating a distributed representation vector of the user in which the user's features are embedded based on the user's identification information, and outputting the user's distributed representation vector; using the second deep learning model, estimating a distributed representation vector of the second text; using a third deep learning model, estimating a third text posted by the user before and / or after the first text based on the user's distributed representation vector and the distributed representation vector of the second text; and using the text posted by the user before and / or after the first text as training data, and updating parameters of the first deep learning model, the second deep learning model, and the third deep learning model based on an error between the training data and the third text.

[0008] Furthermore, the estimation method according to the present invention is an estimation method executed by an estimation device, and is characterized in that it includes a step of receiving, as an input, identification information of a user who posts text, estimating a distributed representation vector of the user, into which the user's features are embedded, based on the user's identification information, using a first deep learning model, and outputting the user's distributed representation vector. Effect of the Invention

[0009] According to the present invention, it is possible to obtain distributed representations of users themselves, targeting users who post text. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of a learning device according to the first embodiment. [Diagram 2] FIG. 2 is a flowchart of a learning process according to the first embodiment. [Diagram 3]FIG. 3 is a diagram illustrating an example of a configuration of an estimation device according to the second embodiment. As shown in FIG. [Figure 4] FIG. 4 is a flowchart illustrating a processing procedure of the estimation process according to the second embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a configuration of an estimation device according to a modification of the second embodiment. In FIG. [Figure 6] FIG. 6 is a flowchart of the estimation process according to the modification of the second embodiment. [Figure 7] FIG. 7 is a diagram illustrating an example of a computer that realizes the learning device and the estimation device by executing a program. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are indicated by the same reference numerals.

[0012] [Embodiment 1] In the first embodiment, learning of a model that acquires a distributed representation of a user who posts text will be described. Here, attention will be paid to acquiring the context of the text posted by the user. It is considered that the type of post or reaction made by a user is greatly influenced by the context before and after the post. In the following description, an example will be described in which a user posts text on a social network.

[0013] Therefore, in embodiment 1, we will explain a learning device that uses a user's ID (identification information) and text posted by the user at a certain time as input as a deep learning model to acquire a distributed representation of a user, and trains a model that estimates the posted text before and after that.

[0014] In this learning process, the model learns what kind of posts a user makes before and after posting a certain piece of text. This allows the model to learn the context of the text posted by the user. Therefore, the distributed representation of the user obtained by the model before inferring the context can be said to strongly reflect the user's own values, thoughts, and background.

[0015] [Learning device] A learning device that performs learning on a model that estimates a distributed representation of a user will be described as the learning device according to the embodiment 1. Fig. 1 is a diagram illustrating an example of the configuration of the learning device according to the embodiment 1.

[0016] The learning device 10 according to the first embodiment is realized, for example, by loading a predetermined program into a computer or the like including a ROM (Read Only Memory), a RAM (Random Access Memory), a CPU (Central Processing Unit), etc., and having the CPU execute the predetermined program. The learning device 10 also has a communication interface for transmitting and receiving various information to and from other devices connected via a network or the like.

[0017] The learning device 10 includes an input unit 11, a text conversion unit 12 (conversion unit), an estimation unit 13, and a learning unit 14. The estimation unit 13 includes a user shared expression estimation unit 131 (first estimation unit), a text shared expression estimation unit 132 (second estimation unit), a vector combination unit 133, and a text estimation unit 134 (third estimation unit).

[0018] The learning device 10 shown in FIG. 1 receives as input a user ID (user identification information) I and an input text T (first text) posted by a user on a social network at time t. Here, I is an integer and T is a character string. Before being input to an estimation unit 13, the input text T is converted by a text conversion unit 12 into a format that can be input to a deep learning model. The user ID I is input to a user distributed expression estimation unit 131 of the estimation unit 13.

[0019] The user distributed expression estimation unit 131 estimates and outputs a user distributed expression V (a distributed expression vector of a user) using a deep learning model. The text estimation unit 134 estimates texts (e.g., output texts T'-2, T'-1, T'+1, T'+2) (third texts) posted by a user before and / or after an input text T using a deep learning model.

[0020] The learning device 10 uses texts (e.g., texts T-2, T-1, T+1, T+2) actually posted by a user before and / or after an input text T as training data (training texts) to train a deep learning model. The output texts T'-2, T'-1, T'+1, T'+2 are outputs used in training the deep learning model. In the first embodiment, the data to be acquired with high accuracy is the user distributed representation V output from the user distributed representation estimation unit 131. The learning device 10 uses the deep learning model to learn model parameters for acquiring the user's distributed representation.

[0021] The input unit 11 receives a user ID I and an input text T as input. The user ID I is identification information of a user. The input text T is text posted on a social network at a certain time t by a user assigned the user ID I. The input unit 11 outputs the user ID I to the estimation unit 13. The input unit 11 outputs the input text T to the text conversion unit 12.

[0022] The input text T for parameter learning is text posted by a user on a blog or Twitter (trademark). When inputting a long piece of text such as a blog, the text is divided into sentences, and inputs such as T-2, T-1, T+1, and T+2 are assigned to each sentence. Any method can be used to divide the text into sentences. When inputting short articles such as those on Twitter (trademark), sentence division is not performed, and each article is assigned an input. Note that in the case of Twitter (trademark), there are posts (replies) that reply to the other person's article, but these are excluded from the learning text.

[0023] The text conversion unit 12 converts the input text T into text (second text) in a format that can be input to the deep learning model. The text conversion unit 12 converts the teacher texts T-2, T-1, T+1, and T+2 into text in a format that can be input to the deep learning model. Here, the text T-2 indicates that the user inputs the text two texts before the Tth input text T. The text T+2 indicates that the user inputs the text two texts after the input text T. It is possible to arbitrarily determine how many texts before and after the input text to refer to as teacher data.

[0024] The text conversion unit 12 first performs morphological analysis or tokenization processing on the text to convert the text into a word string or token string format. Any appropriate method may be applied for the morphological analysis and tokenization processing.

[0025] Next, the text conversion unit 12 converts the words or tokens into corresponding integer IDs, and converts the text into an integer ID string. Here, the text conversion unit 12 may appropriately insert a special token used in the deep learning model. The text conversion unit 12 also converts the teacher text into a format suitable for the deep learning model. For example, if the deep learning model is a model that can only output one sentence, the texts of the teacher texts T-2, T-1, T+1, and T+2 are linked together to form one sentence. At this time, the text conversion unit 12 may use a special token that represents a break in a sentence when linking the teacher texts T-2 and T-1.

[0026] The user distributed representation estimation unit 131 uses a first deep learning model to estimate a user distributed representation V (user distributed representation vector) that embeds the characteristics of the user who posted the text based on the user's ID I, and outputs the user distributed representation V.

[0027] The user distributed representation estimation unit 131 converts the user ID I represented by a one-hot vector into an N-dimensional continuous value vector. The user distributed representation estimation unit 131 may adopt any suitable conversion method. The user distributed representation V is a vector corresponding to the number of elements in the user embedding layer, and becomes the user's distributed representation vector that is the final output. The user distributed representation V is a vector indicating the features derived from the user who posted the text, for example, the user's latent features such as attributes, gender, and inner thoughts.

[0028] The text shared representation estimation unit 132 uses the second deep learning model to estimate a shared representation vector of the input text T whose format has been converted by the text conversion unit 12 (a shared representation vector of the second text). The text shared representation estimation unit 132 acquires a shared representation of a word or token from a word or token ID. The word or token is also input in the form of a one-hot vector together with the user ID. The text shared representation estimation unit 132 expresses the shared representation vector of the input text T as a sequence of shared representations of M-dimensional continuous values. In the case of a word string of length L, the text shared representation estimation unit 132 acquires a shared representation vector of a sequence length L. The text shared representation estimation unit 132 may employ any appropriate method for the conversion to estimate the shared representation vector.

[0029] The vector combining unit 133 combines the user's distributed representation V (user's distributed representation vector) with the distributed representation vector of the input text T, and inputs the combined result to the text estimation unit 134. Here, the vector combining unit 133 may adopt any method appropriate for the subsequent text estimation unit 134. For example, the vector combining unit 133 may combine the user's distributed representation vector with the beginning of the distributed representation vector sequence of the text to make a vector sequence of length L+1. Also, the vector combining unit 133 may combine the user's distributed representation vector with the end of the vectors of all the distributed representations of the text to make a vector sequence of length L and M+N dimensions.

[0030] The text estimation unit 134 estimates output texts T'-2, T'-1, T'+1, and T'+2 posted by users before and / or after the input text T based on the user distributed representations V and the distributed representation vector of the text using a third deep learning model. The text estimation unit 134 estimates output texts T'-2, T'-1, T'+1, and T'+2 based on the user distributed representations V and the distributed representation vector of the text combined by the vector combination unit 133. The text estimation unit 134 employs any model as appropriate as the third deep learning model.

[0031] The learning unit 14 optimizes parameters of a model for acquiring a distributed representation of a user using a deep learning model. The learning unit 14 uses texts T-2, T-1, T+1, and T+2 actually posted by a user before and / or after an input text T as teacher texts, and updates parameters of the first deep learning model, the second deep learning model, and the third deep learning model based on errors between the teacher data and output texts T'-2, T'-1, T'+1, and T'+2. The learning unit 14 may employ any appropriate method as a function for calculating errors and a method for updating parameters.

[0032] The learning unit 14 may also adopt any procedure for learning the entire deep learning model. For example, the learning unit 14 may optimize all parameters of the first deep learning model, the second deep learning model, and the third deep learning model in one learning.

[0033] Furthermore, the learning unit 14 may perform learning of the deep learning model by dividing the learning into a plurality of stages. For example, the learning unit 14 first performs learning on the second deep learning model of the text distributed representation estimation unit 132 and the third deep learning model of the text estimation unit 134. After performing sufficient learning on the second deep learning model and the third deep learning model, the learning unit 14 fixes parameters of the second deep learning model and the third deep learning model and performs learning for optimizing parameters of the first deep learning model of the user distributed representation estimation unit 131. In this way, the learning unit 14 may perform learning on each deep learning model in two stages.

[0034] In the estimation unit 13, the models used by the user distributed expression estimation unit 131, the text distributed expression estimation unit 132, and the text estimation unit 134 may be independent of each other as described above. If the user distributed expression V can be extracted from the user distributed expression estimation unit 131, the user distributed expression estimation unit 131, the text distributed expression estimation unit 132, and the text estimation unit 134 may be configured as one deep learning model.

[0035] [Learning process] Next, a description will be given of the learning process executed by the learning device 10. Fig. 2 is a flowchart showing the processing procedure of the learning process according to the first embodiment.

[0036] 2, the learning device 10 receives a user ID I and an input text T as input (step S1). The user distributed representation estimation unit 131 estimates a user distributed representation V based on the user ID I using the first deep learning model (step S2), and outputs the user distributed representation V (step S3).

[0037] The text conversion unit 12 converts the input text T into text in a format that can be input to the deep learning model (step S4). The text distributed representation estimation unit 132 uses the second deep learning model to estimate distributed representation vectors of the input text T whose format has been converted by the text conversion unit 12 (step S5).

[0038] The vector combining unit 133 combines the user's shared representation V with the shared representation vector of the input text T (step S6) and inputs the combined result to the text estimation unit 134. The text estimation unit 134 uses a third deep learning model to estimate and output output texts T'-2, T'-1, T'+1, and T'+2 posted by users before and / or after the input text T, based on the user's shared representation V and the shared representation vector of the text combined by the vector combining unit 133 (steps S7 and S8).

[0039] The learning unit 14 updates the parameters of the first deep learning model, the second deep learning model, and the third deep learning model based on the errors between the teacher texts T-2, T-1, T+1, and T+2 and the output texts T'-2, T'-1, T'+1, and T'+2 (step S9). The learning device 10 repeats the estimation, error calculation, and parameter update until a predetermined termination condition is satisfied. The termination condition is, for example, when the error becomes equal to or less than a predetermined threshold, when the number of parameter updates reaches a predetermined number, or when the amount of parameter updates becomes equal to or less than a predetermined threshold. The learning device 10 outputs the first deep learning model (including model parameters) for which learning has been completed to the estimation device 20 (described later).

[0040] [Advantages of the First Embodiment] In this way, in the learning device 10 according to the first embodiment, the deep learning model learns what kind of posts a user who posted a certain text makes before and after the posting. This allows the deep learning model to learn the context of the text posted by the user, and the distributed representation vector of the user obtained in the deep learning model before the context is estimated strongly reflects the user's own values, thoughts, and background.

[0041] Therefore, the learning device 10 can provide a deep learning model that targets users who post text on social networks and can obtain distributed representations of the users themselves.

[0042] [Embodiment 2] Next, an estimation device that estimates a distributed representation of a user himself / herself will be described in embodiment 2. Fig. 3 is a diagram illustrating an example of the configuration of the estimation device according to embodiment 2.

[0043] The estimation device 20 shown in Fig. 3 is realized, for example, by loading a predetermined program into a computer or the like including a ROM, a RAM, a CPU, etc., and executing the predetermined program by the CPU. The estimation device 20 also has a communication interface for transmitting and receiving various information to and from other devices connected via a network, etc. The estimation device 20 has a user distributed expression estimation unit 231 (estimation unit).

[0044] The user distributed representation estimation unit 231 uses a first deep learning model. When the user distributed representation estimation unit 231 receives a user ID I of a user who posts text on a social network as an input, the user distributed representation estimation unit 231 estimates a user distributed representation V (a distributed representation vector of the user) in which the user's features are embedded based on the user ID I using the first deep learning model, and outputs the user distributed representation V.

[0045] The first deep learning model used by the user distributed representation estimation unit 231 is trained by the learning device 10. That is, the first deep learning model is a model whose parameters are optimized based on the estimation result of an estimator that estimates text posted by a user before and / or after the text based on a distributed representation vector related to the text posted by the user on a social network and the user's distributed representation vector estimated by the first deep learning model using the user's ID I as input.

[0046] [Estimation process] Fig. 4 is a flowchart showing a processing procedure of the estimation process according to the embodiment 2. As shown in Fig. 4, when the estimation device 20 receives a user ID I of a user as an input (step S11), the user distributed representation estimation unit 231 estimates and outputs a user distributed representation V based on the user ID I using the first deep learning model (steps S12 and S13).

[0047] [Effects of the second embodiment] The user distributed representation V output by the estimation device 20 can obtain a distributed representation of the user himself / herself that strongly reflects the user's values, thoughts, and background by using the first deep learning model trained by the learning device 10.

[0048] Since this user distributed representation V is a basic feature that is expressed as a vector, various applications are possible. For example, by calculating the similarity of the vectors obtained as the user distributed representation V and ranking them, it becomes possible to search for similar users from a more abstract perspective.

[0049] Similarly to other distributed representations, the user distributed representation V can be used as an input to a deep learning model. For example, a dialogue model specialized for an individual can be realized by inputting the user distributed representation V acquired by the estimation device 20 together with a spoken text.

[0050] In addition, it may be possible to realize a personality diagnosis model or a behavior judgment model by using the user distributed representation V. In addition, by creating a recommendation system that uses the user distributed representation V as input, it is expected that product recommendations that are more in line with the user's inner thoughts than conventional collaborative filtering, etc., can be realized.

[0051] [Variations] Next, a modification of the embodiment 2 will be described. Fig. 5 is a diagram illustrating an example of a configuration of an estimation device according to the modification of the embodiment 2. Fig. 6 is a flowchart illustrating a processing procedure of an estimation process according to the modification of the embodiment 2.

[0052] 5 is realized by, for example, loading a predetermined program into a computer or the like including a ROM, a RAM, a CPU, etc., and causing the CPU to execute the predetermined program. The estimation device 20A also has a communication interface for transmitting and receiving various information to and from other devices connected via a network or the like.

[0053] The estimation device 20A receives as input a user ID I and a text F (first text) posted by the user on a social network. The estimation device 20A outputs a user distributed representation V, as well as F'-2, F'-1, F'+1, and F'+2 (third texts) that are estimated to be posted by the user before and / or after the input text F.

[0054] The estimation device 20A is realized, for example, by loading a predetermined program into a computer or the like including a ROM, a RAM, a CPU, etc., and the CPU executing the predetermined program. The estimation device 20A also has a communication interface for transmitting and receiving various information to and from other devices connected via a network, etc.

[0055] The estimation device 20A includes an input unit 21, a text conversion unit 22, and an estimation unit 23. The estimation unit 23 includes a user shared expression estimation unit 231, a text shared expression estimation unit 232, a vector combination unit 233, and a text estimation unit 234.

[0056] The input unit 21 has the same functions as the input unit 11 shown in FIG. 1, and receives as input a user ID I and text F posted by the user on a social network (step S21).

[0057] The user distributed representation estimation unit 231 uses the first deep learning model trained by the learning device 10 to estimate a user distributed representation V based on the user ID I (step S22), and outputs the user distributed representation V (step S23).

[0058] The text conversion unit 22 has the same function as the text conversion unit 12 shown in FIG. 1, and converts the input text F into text (second text) in a format that can be input to the deep learning model (step S24).

[0059] The text distributed representation estimation unit 232 uses the second deep learning model trained by the learning device 10 to estimate a distributed representation vector of the input text F whose format has been converted by the text conversion unit 22, similar to the text distributed representation estimation unit 132 (step S25).

[0060] The vector combining unit 233 has the same function as the vector combining unit 133, and combines the user distributed representation V with the distributed representation vector of the input text F (step S26), and inputs the combined vector to the text estimation unit 234.

[0061] The text estimation unit 234, similar to the text estimation unit 134, uses the third deep learning model trained by the learning device 10 to estimate and output texts F'-2, F'-1, F'+1, and F'+2 posted by the user before and / or after text F based on the user distributed representation V and the distributed representation vector of text F (steps S27, S28).

[0062] Like this estimation device 20A, together with the user distributed representation V, texts F'-2, F'-1, F'+1, and F'+2 posted by the user before and / or after the text F may be estimated.

[0063] [System configuration of the embodiment] Each component of the learning device 10 and the estimation devices 20, 20A is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of the functions of the learning device 10 and the estimation devices 20, 20A is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0064] Furthermore, all or any part of the processes performed in the learning device 10 and the estimation devices 20 and 20A may be realized by a CPU, a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and the GPU. Furthermore, each process performed in the learning device 10 and the estimation devices 20 and 20A may be realized as hardware using wired logic.

[0065] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the process procedures, control procedures, specific names, and information including various data and parameters described above and shown in the drawings can be changed as appropriate unless otherwise specified.

[0066] [program] 7 is a diagram showing an example of a computer in which the learning device 10 and the estimation devices 20 and 20A are realized by executing a program. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0067] The memory 1010 includes a ROM 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0068] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the learning device 10 is implemented as a program module 1093 in which code executable by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for executing a process similar to the functional configuration of the learning device 10 is stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0069] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads out program module 1093 or program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary, and executes the program.

[0070] Note that the program module 1093 and the program data 1094 are not limited to being stored in the hard disk drive 1090, but may be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and the program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or wide area network (WAN)). The program module 1093 and the program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0071] Although the embodiment of the invention made by the present inventor has been described above, the present invention is not limited by the description and drawings that form a part of the disclosure of the present invention according to the present embodiment. In other words, other embodiments, examples, operation techniques, etc. made by those skilled in the art based on the present embodiment are all included in the scope of the present invention. [Explanation of symbols]

[0072] 10 Learning Device 11,21 Input section 12,22 Text conversion section 13,23 Estimation part 14 Learning Department 20,20A Estimation Device 131,231 User Distributed Expression Estimation Unit 132,232 Text Distributed Representation Estimation Unit 133,233 Vector Joint 134,234 Text Estimation Section

Claims

1. A learning method executed by a learning device, comprising: accepting as input an identification of a user and a first text posted by the user; converting the first text into a second text in a format that can be input to a deep learning model; using a first deep learning model, estimating a distributed representation vector of the user in which the user's features are embedded based on the user's identification information, and outputting the distributed representation vector of the user; estimating a distributed representation vector of the second text using a second deep learning model; Using a third deep learning model, estimating a third text posted by the user before and / or after the first text based on the distributed representation vector of the user and the distributed representation vector of the second text; A step of updating parameters of the first deep learning model, the second deep learning model, and the third deep learning model based on an error between the teacher data and the third text, using text posted by the user before and / or after the first text as teacher data; A learning method comprising:

2. An estimation method executed by an estimation device, accepting as input an identification of a user and a first text posted by the user; converting the first text into a second text in a format that can be input to a deep learning model; using a first deep learning model, estimating a distributed representation vector of the user in which the user's features are embedded based on the user's identification information, and outputting the distributed representation vector of the user; estimating a distributed representation vector of the second text using a second deep learning model; Using a third deep learning model, estimating a third text posted by the user before and / or after the first text based on the distributed representation vector of the user and the distributed representation vector of the second text; The estimation method according to claim 1, further comprising:

3. an input unit that receives, as input, identification information of a user and a first text posted by the user; a conversion unit that converts the first text into a second text in a format that can be input to a deep learning model; a first estimation unit that estimates a distributed representation vector of a user, into which features of the user are embedded, based on identification information of the user, using a first deep learning model, and outputs the distributed representation vector of the user; a second estimation unit that estimates a distributed representation vector of the second text using a second deep learning model; a third estimation unit that estimates a third text posted by the user before and / or after the first text based on the distributed representation vector of the user and the distributed representation vector of the second text using a third deep learning model; A learning unit that uses text posted by the user before and / or after the first text as training data, and updates parameters of the first deep learning model, the second deep learning model, and the third deep learning model based on an error between the training data and the third text; A learning device comprising:

4. A method of receiving as input an identification information of a user and a first text posted by the user; a conversion unit that converts the first text into a second text in a format that can be input to a deep learning model; a first estimation unit that estimates a distributed representation vector of a user, into which features of the user are embedded, based on identification information of the user, using a first deep learning model, and outputs the distributed representation vector of the user; a second estimation unit that estimates a distributed representation vector of the second text using a second deep learning model; a third estimation unit that estimates a third text posted by the user before and / or after the first text based on the distributed representation vector of the user and the distributed representation vector of the second text using a third deep learning model; An estimation device comprising:

5. accepting as input an identification of a user and a first text posted by the user; converting the first text into a second text that can be input to a deep learning model; using a first deep learning model, estimating a distributed representation vector of the user in which the user's features are embedded based on the user's identification information, and outputting the distributed representation vector of the user; estimating a distributed representation vector of the second text using a second deep learning model; Using a third deep learning model, estimating a third text posted by the user before and / or after the first text based on the distributed representation vector of the user and the distributed representation vector of the second text; A step of updating parameters of the first deep learning model, the second deep learning model, and the third deep learning model based on an error between the teacher data and the third text, using text posted by the user before and / or after the first text as teacher data; A learning program for a computer to execute the above.

6. A method for providing a service to a user, comprising: receiving as input an identification of a user and a first text posted by the user; converting the first text into a second text in a format that can be input to a deep learning model; using a first deep learning model, estimating a distributed representation vector of the user in which the user's features are embedded based on the user's identification information, and outputting the distributed representation vector of the user; estimating a distributed representation vector of the second text using a second deep learning model; Using a third deep learning model, estimating a third text posted by the user before and / or after the first text based on the distributed representation vector of the user and the distributed representation vector of the second text; An estimation program for causing a computer to execute the above.

Images