Dialogue device and learning method

Abstract

To additionally train an LLM so as to reflect knowledge of a target domain.SOLUTION: A deep learning model includes a TSL on an LLM, for classifying output from the LLM into topics specific to a target domain to reflect the topic to the output from the LLM. A learning unit 10 receives input of a text group relating to the target domain as learning data to learn the deep learning model by using an objective function LTLM for enhancing prediction accuracy of a word based on a context and the topic and an objective function LTDM for making a feature TIDd of a text d and a classification zd of the text d, which are based on the output of the LLM, close to each other.SELECTED DRAWING: Figure 4

Claims

1. A deep learning model comprising a language model and a topic steering layer that takes the output vector of the top layer of the language model as input to infer a semantic category specific to the target domain and infers the next word according to the semantic category, It comprises an input / output unit that inputs a seed word to the deep learning model and outputs a response, The deep learning model was trained by inputting a set of texts related to the target domain as training data, minimizing the difference between the semantic category similarity between texts and the feature similarity between texts based on the output of the language model, and minimizing the prediction error when predicting the next word using the representation obtained by applying the semantic category-based transformation to the output vector of the top layer of the language model. Dialogue device.

2. The dialogue device according to claim 1, The topic steering layer receives the output vector of the top layer of the encoder or decoder of the language model. Dialogue device.

3. A method for training deep learning models, The deep learning model comprises a language model and a topic steering layer that takes the output vector of the top layer of the language model as input to infer a semantic category specific to the target domain and infers the next word according to the semantic category. Computers The deep learning model is trained by inputting a set of texts related to the target domain as training data, minimizing the difference between the semantic category similarity between texts and the feature similarity between texts based on the output of the language model, and minimizing the prediction error when predicting the next word using the representation obtained by applying the semantic category-based transformation to the output vector of the top layer of the language model. Learning methods.

Images