Method and apparatus for tracking dialog state for a patent domain dialog robot

Abstract

The application discloses a dialogue state tracking method and device for a patent field dialogue robot. The method comprises the following steps: based on a public mode guide dialogue data set, a dialogue state tracking model based on a mode guide for the patent field is proposed; according to patent field knowledge, mode information for dialogue state tracking in the patent field is constructed; for each round of human-computer dialogue, the trained dialogue state tracking model is used, the dialogue context and the patent field mode information are used as model inputs, and dialogue state prediction is obtained. The application uses the mode information containing the relevant knowledge of the patent field to guide the prediction of the dialogue state, and can solve the problem of high training cost caused by the lack of large-scale dialogue data in the patent field.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of human-computer dialogue, in particular to a dialogue state tracking method and device for a dialogue robot in the field of patents. BACKGROUND

[0002] Task-oriented dialogue systems are used to help users achieve task goals in specific fields, such as hotel booking, schedule management, etc., and play an increasingly important role in people's daily life. Modular task-oriented dialogue systems are the most common implementation method, which consists of four modules: natural language understanding, dialogue state tracking, dialogue strategy and natural language generation. Among them, dialogue state tracking is a key module, which is responsible for tracking the real-time development of the dialogue, analyzing the user's intent and the related entities mentioned in the dialogue, and converting them into dialogue state information that the system can process. The effect of dialogue state tracking determines the overall quality of the task-oriented dialogue system, which provides a guarantee for the system to respond accurately and smoothly.

[0003] Patent field dialogue robots are used to help patent applicants and related practitioners quickly obtain the required information. Existing dialogue state tracking methods are data-driven and require a certain amount of field dialogue data for model training, which has high time and computational costs. There is currently a lack of large-scale annotated data in the patent field, which limits the performance of existing dialogue state tracking methods for dialogue robots in this field. SUMMARY

[0004] In view of the problem that the lack of dialogue data in the patent field leads to high training costs for dialogue state tracking models, the present application aims to provide a special dialogue state tracking method. This method can train models in other fields where training data is relatively abundant and directly apply them to the patent field where data is relatively scarce. With the help of pre-defined patent field pattern information, dialogue state tracking is guided, solving the problem of high training costs caused by the lack of annotated data in the patent field.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] A dialogue state tracking method for a dialogue robot in the field of patents, comprising the following steps:

[0007] S1, constructing a pattern-guided dialogue state tracking model, training the pattern-guided dialogue state tracking model using a public dataset;

[0008] S2, constructing pattern information in the field of patents according to the contents of the "Patent Examination Guidelines" and the "Patent Law"; the pattern information includes:

[0009] 1) User intent related to the dialogue robot in the field of patents, and natural language description of user intent;

[0010] 2) slots involved in the patent domain dialogue robot, and natural language descriptions of the slots; S3, using the trained dialogue state tracking model for the patent domain dialogue robot to obtain the context of the patent domain dialogue in the dialogue of the dialogue robot, using the dialogue context of the patent domain dialogue and the mode information of the patent domain as input, and using the trained dialogue state tracking model to predict the dialogue state.

[0011] As preferred, the dialogue state tracking model based on the mode guidance in step S1 takes the mode information and the dialogue context at a certain time as input, and sequentially passes through: dialogue context encoding, mode information encoding, intent classification, slot state judgment, and slot value prediction, to output the dialogue state at the time;

[0012] The dialogue context encoding and the mode information encoding use a pre-trained language model TODBERT, and the dialogue context encoding obtains an embedding of the dialogue context, and the mode information encoding obtains an embedding of the user intent and the slot;

[0013] The intent classification is represented by the following formula:

[0014]

[0015] Wherein, t is the embedding of the dialogue context, i j is the embedding of the intent j, W1, W2, b1, b2 are all trainable parameters, h1 is the hidden state of the model, and the intent j corresponds to The probability prediction of the intent is obtained by a softmax activation function;

[0016] All user intents obtained after the mode information encoding are processed according to the above formula to obtain probability predictions of all intents, and the user intent with the maximum probability prediction value is taken as the current user intent;

[0017] The slot state judgment is completed by a three-door classifier, which takes the embedding of the dialogue context and the embedding of the slot as input to judge whether the slot is mentioned in the dialogue context, and if so, the slot value prediction is performed;

[0018] The slot value prediction is realized by predicting the start and end positions of the slot value in the dialogue context, which can be represented by the following formula:

[0019]

[0020] Wherein, s is the embedding of the slot, t i (1≤i≤n) is the vector representation of each word in the dialogue context, n is the length of the dialogue context, h2、 W3, W span , b3, b span are trainable parameters, and the text in the (start_pos s , end_pos s ) interval in the dialogue context is the predicted value of the slot;

[0021] The user intent prediction and slot value prediction obtained by the model are the current dialogue state.

[0022] Preferably, in step S2, the user intents include "patent retrieval" and "application status query", and the natural descriptions of the user intents are "obtaining ten patents most suitable according to one or more fields" and "querying the current application status of a patent according to an application number provided by a user" respectively.

[0023] Preferably, in step S2, the slots include "keyword", "applicant", "inventor", "type", "application date" and "application number", and the natural language descriptions of the slots are "word used for matching with the content of a target patent", "unit submitting a patent application", "actual inventor of a patent", "type of a patent", "submission date of a patent application" and "unique identifier of a patent application document" respectively.

[0024] The application also provides a dialogue state tracking device for a dialogue robot in the patent field, the device having a computer readable storage medium, and a computer program is configured in the computer readable storage medium, and the computer program is executed to realize the steps of any of the methods.

[0025] Compared with the prior art, the application has the beneficial effects that:

[0026] The application considers the problem of lack of data in the patent field, constructs pattern information in the patent field to guide dialogue state tracking, and reduces the dependence on field training data. The natural language descriptions of the intents, slots and other elements provide rich semantic information for state tracking, and when a dialogue state tracking model trained in other fields with relatively sufficient data is used for a dialogue robot in the patent field, the performance of the dialogue state tracking model can still be guaranteed, and the problem of high training cost of the prior art is solved. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 The figure is a flowchart of the dialogue state tracking method for the dialogue robot in the patent field of the application; DETAILED DESCRIPTION

[0028] In order to more clearly illustrate the embodiments of the present application, the specific embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the drawings described below are part of the embodiments of the present application, not all embodiments. Those skilled in the art can also obtain other embodiments without creative labor.

[0029] The embodiment provides a dialogue state tracking method for a dialogue robot in a patent field, and a flowchart of the method is as shown in Figure 1

[0030] Firstly, step S1 is performed, a dialogue state tracking model based on pattern guidance is trained using a public dataset, and the public dataset is composed of two parts of dialogue data and field pattern information. The model sequentially performs the following steps: dialogue context encoding, pattern information encoding, intent classification, slot state judgment, and slot value prediction.

[0031] The dialogue context encoding and the pattern information encoding use a pre-trained language model TODBERT, and embedding of the dialogue context, the user intent and the slot is obtained after encoding;

[0032] The intent classification can be expressed by the following formula:

[0033]

[0034] Wherein, t is embedding of the dialogue context, i j is embedding of the intent j, W1, W2, b1, b2 are all trainable parameters, h1 is a hidden state of the model, and the intent j corresponds to The probability prediction of the intent is obtained through a softmax activation function;

[0035] All user intents in the pattern information are processed according to the above formula to obtain probability prediction of all intents, and the user intent with the maximum probability prediction value is taken as the current user intent;

[0036] The slot state judgment is completed by a three-door classifier, the embedding of the dialogue context and the embedding of the slot are taken as inputs, whether the slot is mentioned in the dialogue context is judged, and if the slot is mentioned, the slot value prediction is performed;

[0037] The slot value prediction is realized by predicting the start and end positions of the slot value in the dialogue context, and can be expressed by the following formula:

[0038]

[0039] Wherein, s is embedding of the slot, t i (1≤i≤n) is a vector representation of each word in the dialogue context, n is the length of the dialogue context, h2、 ​W3, W span , b3, b span are trainable parameters, the text in the (start_pos s , end_pos s ) interval of the dialogue context is the predicted value of the slot;

[0040] The user intent prediction and slot value prediction obtained by the model are the current dialogue state. After the model is trained using the public dataset, it is saved.

[0041] Then, step S2 is performed, in which the pattern information of the patent field is constructed according to the contents of the Patent Examination Guidelines and the Patent Law, including the following contents:

[0042] 1) The user intent involved in the patent field dialogue robot, and the natural language description of the user intent;

[0043] 2) The slot involved in the patent field dialogue robot, and the natural language description of the slot.

[0044] In this embodiment, all the user intents supported by the patent field dialogue robot include "patent retrieval" and "application status query", and the corresponding natural descriptions are "obtaining the ten most suitable patents according to one or more fields" and "querying the current application status of a patent according to the application number provided by the user", respectively.

[0045] In this embodiment, all the slots supported by the patent field dialogue robot include "keyword", "applicant", "inventor", "type", "application date", and "application number", and the corresponding natural language descriptions are "word used for matching with the content of the target patent", "unit submitting the patent application", "actual inventor of the patent", "type of the patent", "submission date of the patent application", and "unique identifier of the patent application document", respectively.

[0046] Finally, step S3 is performed, in which the trained dialogue state tracking model is used for the patent field dialogue robot, taking the dialogue context of the patent field and the pattern information of the patent field as inputs, and using the trained dialogue state tracking model to predict the dialogue state. During the dialogue process, after each round of user input, the dialogue context and the pattern information of the patent field are input into the model, and the model sequentially performs encoding, intent classification, slot state prediction, and slot value prediction to finally obtain the current dialogue state prediction.

[0047] The above method can solve the problem of high training cost caused by the lack of large-scale dialogue data in the patent field by constructing pattern information containing knowledge related to the patent field and guiding the dialogue state tracking model.

[0048] It should be noted that the above only describes the preferred embodiments and principles of the present application, and for those skilled in the art, according to the idea provided by the present application, there are changes in the specific implementation manner, and these changes should also be considered as the protection scope of the present application.

Claims

1. A dialog state tracking method for a patent domain dialog robot, characterized by, The method comprises the following steps: S1, constructing a pattern-guided dialogue state tracking model, and training the pattern-guided dialogue state tracking model using a public dataset; S2, constructing pattern information in the patent field according to the content of the Patent Examination Guidelines and the Patent Law; The pattern information comprises: 1) user intent involved in the patent field dialogue robot, and natural language description of the user intent; 2) slot involved in the patent field dialogue robot, and natural language description of the slot; S3, using the trained dialogue state tracking model for the patent field dialogue robot to obtain the context of the patent field dialogue in the dialogue of the dialogue robot, taking the patent field dialogue context and the pattern information in the patent field as inputs, and using the trained dialogue state tracking model to perform dialogue state prediction; The pattern-guided dialogue state tracking model in step S1 takes the pattern information and the dialogue context at a certain moment as inputs, and sequentially passes through dialogue context encoding, pattern information encoding, intent classification, slot state judgment, and slot value prediction to output the dialogue state at the moment; The dialogue context encoding and the pattern information encoding use a pre-trained language model TODBERT, and the dialogue context encoding obtains an embedding of the dialogue context, and the pattern information encoding obtains embeddings of the user intent and the slot; The intent classification is represented by the following formula: where t is the embedding of the dialogue context, i j is the embedding of the intent j, W1, W2, b1, b2 are all trainable parameters, and h1 is the hidden state of the model, and the intent j corresponds to After the softmax activation function, the probability prediction of the intent is obtained. All user intents obtained after the pattern information encoding are processed according to the above formula to obtain probability predictions of all intents, and the user intent with the maximum probability prediction value is taken as the current user intent; The slot state judgment is completed by a three-door classifier, which takes the embedding of the dialogue context and the embedding of the slot as inputs to judge whether the slot is mentioned in the dialogue context, and if so, the slot value prediction is performed; The slot value prediction is realized by predicting the start and end positions of the slot value in the dialogue context, which can be represented by the following formula: wherein s is the embedding of the slot, t i (1≤i≤n) is the vector representation of each segmented word in the dialogue context, n is the length of the dialogue context, h2、 is the hidden state of the model, W3, W span , b3, b span are all trainable parameters, and the text in the (start_pos s , end_pos s ) interval of the dialogue context is the predicted value of the slot. The user intent prediction and the slot value prediction obtained by the model are the current dialogue state. 2.The dialog state tracking method for a patent domain dialog robot according to claim 1, wherein, In step S2, the user intent includes "patent retrieval" and "application status query", and the natural description of the user intent is "obtaining the ten most suitable patents according to one or more fields" and "querying the current application status of a patent according to the application number provided by the user", respectively. 3.The dialog state tracking method for a patent domain dialog robot according to claim 2, wherein, In step S2, the slot includes "keyword", "applicant", "inventor", "type", "application date", and "application number", and the natural language description of the slot is "word used for matching with the content of the target patent", "unit submitting the patent application", "actual inventor of the patent", "type of the patent", "submission date of the patent application", and "unique identifier of the patent application document", respectively.

4. A dialog state tracking apparatus for a patent domain dialog robot, characterized by, The device has a computer-readable storage medium, and a computer program is configured in the computer-readable storage medium, and the computer program is executed to realize the steps of the method in any one of claims 1 to 3.

Images