Method and apparatus for field division

Abstract

The application discloses a domain division method and device, and relates to the technical field of computers. A specific implementation of the method comprises the following steps: extracting keywords from user dialogues; performing feature calculation on the keywords based on a domain data set to obtain a predicted domain of the user dialogues and a corresponding domain probability; if the domain probability is greater than a preset threshold, dividing the domain of the user dialogues into the predicted domain corresponding to the domain probability; and if the domain probability is less than or equal to the preset threshold, dividing the domain of the user dialogues based on a domain division model. The implementation can solve the problem of tangled dialogues, so that the domain division result is not only dependent on labeled data, and the accuracy of domain division is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and in particular, to a domain division method and device. BACKGROUND

[0002] With the development of artificial intelligence technology, the application of human-computer conversation system has made great progress, and has been applied to smart home, smart sound box, smart customer service and smart assistant and many other fields. According to the different task types, the human-computer conversation system can be divided into task-oriented system and non-task-oriented system. The task-oriented system refers to the conversation service system that helps users complete specific tasks in a specific context. The non-task-oriented system usually generates a natural language-based response to user input without completing a specific service task, such as question and answer and chatting robot.

[0003] As shown in Figure 1 , it is a task type conversation robot architecture, which first divides the user's speech into domains, then selects the robot corresponding to the domain of the speech, and finally responds by the robot in the domain. The purpose of domain division is to determine the topic of the user's current conversation, and to hand over the corresponding domain robot for response. The accuracy of domain division will directly affect the accuracy of response, so how to correctly divide the domain of the user's speech is a key point in the research of task type conversation robot, and has great significance for the development of more intelligent task type robot.

[0004] The division of the domain will directly affect the effect of human-computer interaction. The inventors found that the prior art at least has the following problems:

[0005] 1. The semantic understanding ability of the speech is poor, and the data volume requirement is large;

[0006] 2. The data annotation requirement is high, and the domain is prone to misclassification;

[0007] 3. The cost of later development and maintenance is high. SUMMARY

[0008] Therefore, the embodiments of the present application provide a domain division method and device, which can solve the problem of speech entanglement, make the domain division result not only depend on the annotation data, and improve the accuracy of domain division.

[0009] To achieve the above object, according to one aspect of the embodiments of the present application, a domain division method is provided.

[0010] The domain division method of the embodiments of the present application comprises: extracting keywords from user speech;

[0011] perform feature calculation on the keywords based on a domain dataset to obtain a predicted domain of the user utterance and a corresponding domain probability of the user utterance;

[0012] if the domain probability is greater than a preset threshold, divide the domain of the user utterance into the predicted domain corresponding to the domain probability;

[0013] if the domain probability is less than or equal to the preset threshold, divide the domain of the user utterance based on a domain division model.

[0014] Optionally, before extracting the keywords from the user utterance, the method further comprises:

[0015] matching the user utterance in an utterance template;

[0016] if there is an utterance template matching the user utterance, dividing the domain of the user utterance based on a matching result;

[0017] if there is no utterance template matching the user utterance, extracting the keywords from the user utterance.

[0018] Optionally, extracting the keywords from the user utterance comprises:

[0019] performing word segmentation on the user utterance to obtain at least one utterance word segment;

[0020] converting the utterance word segment into a word vector;

[0021] extracting vector features of the word vector;

[0022] calculating the vector features using a conditional random field algorithm to obtain a state transition graph;

[0023] extracting at least one keyword from the state transition graph.

[0024] Optionally, performing feature calculation on the keywords based on a domain dataset to obtain a predicted domain of the user utterance and a corresponding domain probability of the user utterance comprises:

[0025] obtaining feature data of each keyword corresponding to each domain dataset, and performing uniform processing on the feature data of each keyword; wherein each domain dataset stores keywords and corresponding feature data of the same utterance domain;

[0026] performing feature calculation on the feature data of each keyword based on a gradient decision generation tree to obtain an utterance domain of each keyword and a corresponding utterance probability of each keyword;

[0027] selecting a dialogue field corresponding to a highest dialogue probability as a predicted field of the user dialogue, and taking the highest dialogue probability as a field probability of the predicted field.

[0028] Optionally, the field division model is a deep learning network.

[0029] dividing a field for the user dialogue based on the field division model, comprising:

[0030] inputting the dialogue word segmentation into the deep learning network;

[0031] converting the dialogue word segmentation into a word vector by using an embedding layer of the deep learning network;

[0032] extracting a vector feature of the word vector by using a pooling layer of the deep learning network;

[0033] pools the vector feature by using a full connection layer of the deep learning network;

[0034] normalizing the pooled vector feature by using a mapping layer of the deep learning network to obtain a probability value of a field to which the user dialogue belongs;

[0035] dividing a field for the user dialogue based on the probability value of the field to which the user dialogue belongs.

[0036] To achieve the above object, according to another aspect of the embodiment of the present application, a field division device is provided.

[0037] The field division device of the embodiment of the present application comprises: an extraction module configured to extract a keyword from a user dialogue;

[0038] a calculation module configured to perform feature calculation on the keyword based on a field data set to obtain a predicted field of the user dialogue and a corresponding field probability thereof;

[0039] a first division module configured to divide a field for the user dialogue into the predicted field corresponding to the field probability if the field probability is greater than a preset threshold;

[0040] a second division module configured to divide a field for the user dialogue based on a field division model if the field probability is less than or equal to the preset threshold.

[0041] Optionally, the device further comprises a matching module configured to:

[0042] match the user dialogue in a dialogue template;

[0043] divide a field for the user dialogue based on a matching result if there is a dialogue template matched with the user dialogue;

[0044] If there is no dialogue template matching the user dialogue, the keywords are extracted from the user dialogue.

[0045] Optionally, the extracting module is further configured to:

[0046] Tokenizing the user dialogue to obtain at least one dialogue token;

[0047] Converting the dialogue token into a word vector;

[0048] Extracting vector features of the word vector;

[0049] Calculating the vector features by using a conditional random field algorithm to obtain a state transition graph;

[0050] Extracting at least one keyword from the state transition graph.

[0051] Optionally, the calculating module is further configured to:

[0052] Obtaining feature data of each keyword corresponding to each field data set, and performing uniform processing on the feature data of each keyword; wherein each field data set stores keywords and corresponding feature data of the same dialogue field;

[0053] Performing feature calculation on the feature data of each keyword based on a gradient decision generation tree to obtain a dialogue field of each keyword and a corresponding dialogue probability of each keyword;

[0054] Selecting a dialogue field corresponding to the highest dialogue probability as a predicted field of the user dialogue, and selecting the highest dialogue probability as a field probability of the predicted field.

[0055] Optionally, the field division model is a deep learning network.

[0056] The second division module is further configured to:

[0057] Inputting the dialogue token into the deep learning network;

[0058] Converting the dialogue token into a word vector by using an embedding layer of the deep learning network;

[0059] Extracting vector features of the word vector by using a pooling layer of the deep learning network;

[0060] Pooling the vector features by using a fully connected layer of the deep learning network;

[0061] Performing normalization processing on the pooled vector features by using a mapping layer of the deep learning network to obtain a probability value of a field to which the user dialogue belongs;

[0062] The field of the user utterance is divided based on the probability value of the field to which the user utterance belongs.

[0063] To achieve the above object, according to a further aspect of the embodiments of the present application, an electronic device for field division is provided.

[0064] An electronic device for field division according to an embodiment of the present application comprises one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement a field division method according to an embodiment of the present application.

[0065] To achieve the above object, according to a further aspect of the embodiments of the present application, a computer readable storage medium is provided.

[0066] A computer readable storage medium according to an embodiment of the present application has a computer program stored thereon, when the program is executed by a processor, a field division method according to an embodiment of the present application is implemented.

[0067] An embodiment of the above application has the following advantages or beneficial effects: because the key words are extracted from the user utterance; the feature of the key words is calculated based on the field data set, the predicted field of the user utterance and the corresponding field probability are obtained; if the field probability is greater than the preset threshold, the field of the user utterance is divided into the predicted field corresponding to the field probability; if the field probability is less than or equal to the preset threshold, the field of the user utterance is divided based on the field division model; the technical means overcomes the technical problems that the scale of the utterance rule template increases sharply, the service performance is seriously affected, the cost of later development and maintenance is high, the semantic understanding ability is poor, the data volume requirement is large, the data annotation requirement is high, and the field division is misclassified, thereby solving the utterance entanglement, making the field division result not only dependent on the annotation data, improving the accuracy of the field division, and achieving the technical effect of improving the accuracy of the field division.

[0068] The further effects of the above non-conventional optional mode will be described in the following combined with the specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0069] The accompanying drawings are used to better understand the present application, and do not constitute an improper limitation on the present application. Among them:

[0070] Figure 1 is a task-oriented dialogue robot architecture;

[0071] Figure 2 is a schematic diagram of the main steps of the field division method according to an embodiment of the present application;

[0072] Figure 3 is a schematic diagram of the implementation framework of the field division method according to an embodiment of the present application;

[0073] Figure 4 is a schematic diagram of a bidirectional LSTM combined with CRF model of the method of domain division according to the embodiment of the application;

[0074] Figure 5 is a schematic diagram of a Text-CNN model of the method of domain division according to the embodiment of the application;

[0075] Figure 6 is a schematic diagram of a deep learning training method that can be referred to;

[0076] Figure 7 is a schematic diagram of the main modules of the device for domain division according to the embodiment of the application;

[0077] Figure 8 is an exemplary system architecture diagram to which the embodiment of the application can be applied;

[0078] Figure 9 is a structural schematic diagram of a computer system of a terminal device or a server suitable for being used to implement the embodiment of the application. DETAILED DESCRIPTION

[0079] Exemplary embodiments of the present application are described below with reference to the accompanying drawings, which include various details of the embodiments of the present application to assist in understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, in order to be clear and concise, descriptions of well-known functions and structures are omitted in the following description.

[0080] It should be noted that the embodiments of the present application and the technical features in the embodiments can be combined with each other without conflict.

[0081] In the development process of the task-oriented human-computer dialogue system, the correct division of the domain will directly affect the effect of human-computer interaction.

[0082] The embodiment of the present application proposes a domain division method which integrates domain-related knowledge and can solve the problem of user utterance entanglement according to relevant domain knowledge, which is of great significance for the correct response of task-oriented dialogue robots. Utterance entanglement refers to the same utterance expressing different meanings in different context, for example, saying the next one is switching to the next song in the music scene, and switching to the next news in the news scene.

[0083] The domain division method of the embodiment of the present application is based on the fusion of domain-related knowledge, adopts multiple division methods to cooperate with each other, and realizes domain division by cooperation of domain division based on domain knowledge and domain division based on deep learning classification. Compared with the domain division method based on rules alone, the generalization ability for user utterance is increased. The domain division module based on domain knowledge solves the problem of utterance entanglement that cannot be solved by the domain division method based on deep learning classification alone. Moreover, with the continuous improvement of the domain-related knowledge base, the domain division will be more accurate. In the era of continuous development and application of human-computer dialogue systems, more accurate domain division will bring better user experience to task-oriented dialogue systems.

[0084] Figure 2 FIG. 1 is a schematic diagram of main steps of the domain division method according to the embodiment of the present application.

[0085] As shown in FIG. 2, the domain division method of the embodiment of the present application mainly includes the following steps: Figure 2

[0086] Step S201: Extracting keywords from user utterance.

[0087] The keywords of the user utterance can be used to understand the semantics of the user utterance, so as to calculate the domain division result with higher possibility, i.e., the domain to which the user utterance belongs.

[0088] In the embodiment of the present application, step S201 can be realized by the following ways: performing word segmentation on the user utterance to obtain at least one utterance segmentation; converting the utterance segmentation into a word vector; extracting vector features of the word vector; calculating the vector features by using a conditional random field algorithm to obtain a state transition graph; and extracting at least one keyword from the state transition graph.

[0089] For the extraction of keywords, a deep learning model can be used, such as a model combining bidirectional LSTM and CRF.

[0090] In addition, in order to ensure the support for some basic utterances, some matching rules (i.e., utterance templates) need to be set, which are used to match some fixed utterances. When the user utterance does not match all the utterance templates, step S201 and the like are performed.

[0091] The domain division method of the embodiment of the present application can further include, before step S201: matching the user utterance in the utterance template; if there is an utterance template matched with the user utterance, dividing the domain of the user utterance based on the matching result; and if there is no utterance template matched with the user utterance, extracting keywords from the user utterance.

[0092] ​The field division based on the domain knowledge is essentially collecting the knowledge of a certain field, establishing a corresponding knowledge base, such as a medical field knowledge base, and extracting the user's speech keywords, and dividing the user's speech field according to the possibility of the keywords appearing in the field. The statistical-based field division method obtains the related results by counting the speech related to a certain field. If the knowledge related to a certain field is constantly developing, and the statistical-based field division method is not retrained, the statistical-based field division method may fail. In actual application, the statistical-based field division method can be used for classification, and then the field division based on the domain knowledge is used to verify the result.

[0093] Step S202: performing feature calculation on the keywords based on the domain data set to obtain the predicted field of the user's speech and the corresponding field probability.

[0094] The domain data set stores the keywords in the same field and the corresponding feature data. The feature data is the feature corresponding to the keyword and the value of the feature, such as the forwarding quantity, the comment quantity, the number of related programs, or the playing quantity, etc.

[0095] The keywords are extracted from the user's speech, the features in the domain-related knowledge base are combined, the feature calculation is performed to give the field division result with higher possibility, which can not only solve the problem of speech entanglement, but also make the field division result not only depend on the labeled data, but also improve the accuracy of the field division, thereby solving the problem of speech entanglement.

[0096] In the embodiment of the application, step S202 can be realized by the following way: obtaining the feature data corresponding to each domain data set for each keyword, and uniformly processing the feature data of each keyword; performing feature calculation on the feature data of each keyword based on the gradient decision generation tree to obtain the speech field of each keyword and the corresponding speech probability; selecting the speech field corresponding to the highest speech probability as the predicted field of the user's speech, and taking the highest speech probability as the field probability of the predicted field.

[0097] Each domain data set stores the keywords in the same speech field and the corresponding feature data. The keywords with the same attribute may be labeled in different forms in different fields, for example, “The Romance of the Three Kingdoms” may be labeled as “song” in the music field, and may be labeled as “audio book” in the FM field, and the labels of the keywords need to be unified. The feature calculation is calculated in the way of gradient decision generation tree (GBDT), and the most possible field of each keyword is obtained, wherein the calculation method of GBDT can be realized in the existing way, which is not described here.

[0098] Step S203: If the domain probability is greater than the preset threshold, the domain of the user utterance is divided into the predicted domain corresponding to the domain probability.

[0099] If the domain probability of a certain relevant domain of the keyword is greater than the preset threshold, it indicates that the probability that the relevant domain is the same as the domain of the user utterance is high, and the relevant domain can be used as the domain of the user utterance.

[0100] Step S204: If the domain probability is less than or equal to the preset threshold, the domain of the user utterance is divided based on the domain division model.

[0101] If the domain probability of a certain relevant domain of the keyword is less than or equal to the preset threshold, it indicates that an effective domain division result cannot be obtained by using the domain data set, and the user utterance can be processed by the domain division model based on deep learning. The domain division result based on the deep learning model provides a statistical domain division result. The deep learning model can be a deep learning network (Text-CNN), CNN+Tri+CRF, or Attention-based RNN.

[0102] In the embodiment of the application, step S204 can be implemented in the following manner: inputting the utterance tokenization into a deep learning network; converting the utterance tokenization into a word vector by using the embedding layer of the deep learning network; extracting vector features of the word vector by using the pooling layer of the deep learning network; collecting the vector features by using the fully connected layer of the deep learning network; performing normalization processing on the collected vector features by using the mapping layer of the deep learning network to obtain a probability value of the domain to which the user utterance belongs; and dividing the domain of the user utterance based on the probability value of the domain to which the user utterance belongs.

[0103] If the probability value of the domain to which the user utterance belongs output by the deep learning network is greater than a credible probability value, the domain is used as the domain of the user utterance, otherwise it is considered that the domain is not successfully divided. The credible probability value can be determined according to actual conditions. For example, three predicted domains A, B, and C are preset, and for an utterance, the probability values of being divided into A, B, and C can be calculated. Assuming that 0.6 is the credible probability value, the probability values of the utterance belonging to the three classification results A, B, and C are [0.4, 0.3, 0.3], and then the result given by the current model can be considered as not giving an effective domain division result.

[0104] According to the method for domain division of the embodiment of the present application, the keyword is extracted from the user utterance, the feature of the keyword is calculated based on the domain data set, the predicted domain of the user utterance and the corresponding domain probability are obtained, if the domain probability is greater than the preset threshold, the domain of the user utterance is divided into the predicted domain corresponding to the domain probability, if the domain probability is less than or equal to the preset threshold, the domain division model is used to divide the domain of the user utterance, so that the technical problems of the rapid increase of the size of the utterance rule template, the serious influence on the service performance, the high cost of the later development and maintenance, the poor semantic understanding ability, the large requirement for the data volume, the high requirement for the data labeling and the misclassification of the domain are overcome, the utterance entanglement is solved, the domain division result is not only dependent on the labeled data, and the technical effect of improving the accuracy of the domain division is achieved. Embodiment one

[0106] As shown in Figure 3 The method for domain division of the embodiment of the present application can be divided into three parts, i.e. the domain division based on the utterance template, the domain division based on the keyword and the domain division based on the domain division model. The domain division based on the keyword and the domain division model can be trained by using the neural network method, and the existing deep learning training method is followed.

[0107] Domain division based on utterance template

[0108] The user utterance is matched with some fixed utterance templates, so as to divide the domain of the user utterance. The utterance template can provide support for the domain division of some basic utterances (commonly used utterances of the user), and can match multiple domains such as encyclopedia, music, film and television, radio, conversation, time, stock, weather and control instruction.

[0109] Domain division based on keyword

[0110] If the user utterance is not matched with the utterance template, the keyword in the user utterance is extracted, the feature data corresponding to the keyword is combined for feature calculation, and the domain division result with high possibility is given. This part includes three main steps of keyword extraction, matching feature and feature calculation.

[0111] Among them, the keyword extraction mainly uses the model of bidirectional LSTM combined with CRF (Bi-LSTM+CRF) to extract the keyword. The steps involved in the training method of the Bi-LSTM+CRF model are as follows:

[0112] First, the preparation and unification of the labeled data, the purpose of keyword extraction is to extract keywords from the user's words, and to verify the resources by using the domain knowledge base, so as to solve the problem of words winding. Keywords with the same attribute may be labeled differently in different fields, and need to be unified. If not unified, it may affect the model results, and because there are many types of such words, not unified will also affect the model results. So-called unification is to unify some specific words by using labels. Unification is generally based on labeled data, for example, I want to listen to the Three Kingdoms, "The Three Kingdoms" may be labeled as "song" in the music field, and as "audio book" in the FM field. The adopted labeled data is a collection of labeled data in various fields, and the format of the labeled data is "words domain label". The content surrounded by tags such as " <keyword>< / keyword> " is the keyword that needs to be extracted.

[0113] Then, the Bi-LSTM+CRF model is trained and parameterized. The Bi-LSTM+CRF model follows the existing deep learning training method, and the key parameters that need to be parameterized are:

[0114] embedding_size: User input words will be converted into a word vector as an optional implementation, the vector length is 50;

[0115] batch_size: The main operation of deep learning training is matrix operation, and the size of the matrix will affect the speed of operation and the iteration speed of back propagation. Batch_size refers to the number of words in each training round, that is, the number of word vector matrices converted from user words. As an optional implementation, it is converted to 256;

[0116] batch_per_epoch: Because the training corpus is large, the entire training corpus is usually divided into multiple batches, each batch is called a batch, and training all corpora once is called an epoch;

[0117] drop_out: If the neural network is kept intact, overfitting will occur. Overfitting is a situation where the prediction result of the training corpus is very accurate, but the prediction result of other corpora is poor. Overfitting makes the model lose its practical application significance. To prevent overfitting, some neurons are randomly discarded. As an optional implementation, 0.1 "i.e. ten percent" of the neuron calculation results are randomly discarded during training;

[0118] ​learning_rate: learning_rate refers to the proportion of the size of each change of the gradient to the current gradient value. The learning rate determines the update frequency of the gradient descending to the minimum value. If the learning_rate is too large, the gradient value will miss the optimal gradient value or hover around the optimal gradient value. If the learning_rate is too small, a large amount of time will be wasted in the training before converging to the optimal value. As a preferred embodiment, the learning_rate is 0.01;

[0119] Bidirectional LSTM loss function: as a preferred embodiment, the cross-entropy function is used as the loss function;

[0120] CRF loss function: as a preferred embodiment, CRF takes the maximum likelihood function as the optimization objective function.

[0121] Finally, the Bi-LSTM+CRF model obtained in the previous step is used to extract keywords from the user's language.

[0122] Regarding the matching features, a domain knowledge base (i.e., a domain dataset) can be established in advance, which is used to store the features corresponding to the known keywords. When matching the features, the feature data corresponding to the keywords in the domain knowledge base is obtained. For example, in the domain knowledge base, the following features are mainly stored: encyclopedia entry like amount, encyclopedia entry forwarding amount, music comment amount, music highlight amount, radio program play amount, video program quantity, singer attention quantity, radio artist related program quantity, and video actor related program quantity, etc. Among them, the data format stored in the domain knowledge base is "keyword feature value 1 feature value 2 feature value 3 …… feature value n".

[0123] Regarding feature calculation, the gradient decision-making generation tree (GBDT) is used for calculation. The features use part of the data stored in the domain knowledge base, and each keyword is labeled with the most likely domain. For example, "domain keyword feature value 1 feature value 2 feature value 3 …… feature value n". In addition, the key parameters that need to be tuned for GBDT are:

[0124] n_estimators: the number of regression trees, which is the maximum number of iterations of weak learners. In the embodiment of the present application, the number of regression trees is 300. A learner is a formula that calculates the residual error of a regression tree;

[0125] learning_rate: as a preferred embodiment, the learning_rate is selected as 0.01;

[0126] subsample: the regularization sampling is a non-replacement random sampling for preventing overfitting, in the process of subsampling, for each regression tree hanging chain. As a preferred embodiment, the regularization subsampling is set to 0.7, that is, 70% of the training samples will be used for the training of the current tree, ensuring that each tree has certain differences in training data;

[0127] max_depth: the maximum depth of the decision tree, as a preferred embodiment, the maximum depth of the decision tree is 5;

[0128] loss: define the loss function, provide the objective function for back propagation, as a preferred embodiment, the loss function uses the log-likelihood loss function.

[0129] The keyword related field and its probability (i.e. the predicted field and its corresponding field probability) are obtained by GBDT calculation. It should be noted that if multiple keywords are extracted from the user's speech, the probability of the user's speech belonging to each field will be given as the maximum value of each related field in the keyword.

[0130] Field division based on field division model

[0131] If no effective field division result is given according to the keyword, the field division model is used to divide the field of the user's speech. As a preferred embodiment, the Text-CNN model is used as the field division model to divide the field, the user's speech is input into the Text-CNN model, and the output of the Text-CNN model is the probability value of the field to which the user's speech belongs. Second embodiment

[0133] As shown in Figure 4 , the bidirectional LSTM combined with CRF model used in the field division method of the embodiment of the application. The main process of the bidirectional LSTM combined with CRF model for keyword extraction includes:

[0134] The user's speech is segmented, and at least one speech segmentation is obtained;

[0135] The speech segmentation is input into the embedding layer to convert the segmentation into a word vector

[0136] The word vector is input into the bidirectional LSTM layer to extract the vector feature.

[0137] The vector feature extracted by the bidirectional LSTM is calculated by using the conditional random field algorithm to obtain the most likely state transition graph and extract the possible keyword. Third embodiment

[0139] The GBDT in the embodiment of the application adopts a general GBDT algorithm, and the basic process is as follows:

[0140] The reinforcement learner H(x) can be calculated using the following process:

[0141] 1. Initialize the weak learner:

[0142] Where h0(x) represents the initialization of the weak learner, L(y) i c) is the loss function. In this embodiment of the invention, the logarithmic loss function is used: L(y,h(x))=log(1+exp(-yh(x)))

[0143] 2. For iteration rounds m = 1, 2, ..., M:

[0144] For each sample i = 1, 2, ..., N, calculate the residual, i.e., the negative gradient:

[0145]

[0146] Using the residual as the new true value of the sample, the data (x) i ,r mj The data is used as training data for the next tree to obtain a new regression tree h. m (x) and its corresponding leaf node region R jm ,j=1,2,…,J, where J is the number of tree nodes in the regression tree.

[0147] For the leaf region, j = 1, 2, ..., J, calculate the best-fit value:

[0148]

[0149] Update the learner:

[0150] The final learner is derived from the following formula:

[0151] Specific Implementation Example 4

[0153] like Figure 5 The image shows the Text-CNN model used in the domain segmentation method of this invention. The main process of domain segmentation using the Text-CNN model includes:

[0154] For user statements, the first step is to segment them into [w1,w2,w3…wn];

[0155] In the embedding layer, [w1,w2,w3…wn] will be converted into word vectors;

[0156] In the pooling layer, vector features of word vectors are extracted in a convolutional manner.

[0157] In the full connection layer, vector features extracted by different convolution kernels are collected together.

[0158] In the mapping layer (softmax), the collected vector features are normalized to numbers between 0 and 1, representing the final domain probability of user's dialogue. Specific embodiment five

[0160] The convolutional network is essentially an input-to-output mapping, which can learn a large number of mapping relationships between inputs and outputs without any precise mathematical expression between inputs and outputs. As long as the convolutional network is trained with known patterns, the network has the ability to map between inputs and outputs. The convolutional network performs supervised training, so its sample set is composed of vector pairs such as (word vector, ideal output vector). All these vector pairs should come from the actual "running" results of the system to be simulated by the network. They can be collected from the actual running system. Before starting training, all weights should be initialized with some different small random numbers. "Small random numbers" are used to ensure that the network does not enter a saturated state due to excessive weight values, which can cause training failure; "different" is used to ensure that the network can learn normally. In fact, if the same number is used to initialize the weight matrix, the network has no ability to learn.

[0161] In the embodiments of the present application, the training of the Text-CNNN model and the Bi-LSTM+CRF model can use existing training methods. As shown in Figure 6 The training process of the convolutional neural network is similar to that of the traditional neural network, and the back propagation algorithm is also referred to.

[0162] First stage, forward propagation stage:

[0163] a) Take a sample (X, Yp) from the sample set, and input X into the network;

[0164] b) Calculate the corresponding actual output Op.

[0165] In this stage, information is transmitted from the input layer to the output layer through step-by-step transformation. This process is also performed when the network is normally running after completing the training. In this process, the network performs calculation (actually, it is the point multiplication of the input and the weight matrix of each layer to obtain the final output result):

[0166] Op=Fn(…(F2(F1(XpW(1))W(2))…)W(n))

[0167] Second stage, backward propagation stage:

[0168] a) calculate the difference between the actual output Op and the corresponding ideal output Yp;

[0169] b) adjust the weight matrix by back propagation in the method of minimizing the error.

[0170] For example, the training of the Text-CNNN model

[0171] Data preparation: the format of the adopted labeled data is "dialogue field label";

[0172] Key parameters of the model, the key parameters of the Text-CNNN model are as follows:

[0173] embedding_size: represents the length of the word vector of each word, and is set to 64 as a preferred embodiment;

[0174] learning_rate: determines the parameter update amplitude, and is set to 0.001 as a preferred embodiment;

[0175] batch_size: the number of dialogues in each round of training, and is set to 64 as a preferred embodiment;

[0176] num_epochs: determines the maximum number of iterations of the model on the training set, and is set to 10 as a preferred embodiment;

[0177] drop_out: to prevent overfitting, some neurons are randomly discarded, and is set to 0.2 as a preferred embodiment;

[0178] num_filters: the number of convolution kernels, which determines the number of features after the convolution layer, and is set to 256 as a preferred embodiment;

[0179] kernel_size: the size of the convolution kernel, which affects the influence of the text before and after the current position on the current position. It is set to 3 as a preferred embodiment;

[0180] Loss function: as a preferred embodiment, the cross-entropy loss function is used.

[0181] Figure 7 is a schematic diagram of the main modules of the domain division device according to the embodiments of the application.

[0182] As Figure 7 shown, the domain division device 700 of the embodiments of the application includes an extraction module 701, a calculation module 702, a first division module 703 and a second division module 704.

[0183] Among them,

[0184] The extraction module 701 is configured to extract keywords from the user utterance;

[0185] The calculation module 702 is configured to perform feature calculation on the keywords based on a domain data set, to obtain a predicted domain of the user utterance and a corresponding domain probability thereof;

[0186] The first division module 703 is configured to, when the domain probability is greater than a preset threshold, divide the domain of the user utterance into the predicted domain corresponding to the domain probability.

[0187] The second division module 704 is configured to, when the domain probability is less than or equal to the preset threshold, divide the domain of the user utterance based on a domain division model.

[0188] In the embodiment of the present application, the device 700 for domain division further comprises a matching module (not shown in the figure), which is configured to:

[0189] match the user utterance in an utterance template;

[0190] if there is an utterance template matched with the user utterance, divide the domain of the user utterance based on the matching result;

[0191] if there is no utterance template matched with the user utterance, extract the keywords from the user utterance.

[0192] In the embodiment of the present application, the extraction module 701 is further configured to:

[0193] perform word segmentation on the user utterance to obtain at least one utterance segment;

[0194] convert the utterance segment into a word vector;

[0195] extract vector features of the word vector;

[0196] calculate the vector features by using a conditional random field algorithm to obtain a state transition graph;

[0197] extract at least one keyword from the state transition graph.

[0198] In the embodiment of the present application, the calculation module 702 is further configured to:

[0199] obtain feature data corresponding to each keyword in each domain data set, and perform uniformization processing on the feature data of each keyword; wherein each domain data set stores keywords and corresponding feature data of the same utterance domain;

[0200] The gradient decision generation tree is used for feature calculation on the feature data of each keyword, to obtain a dialogue field of each keyword and a corresponding dialogue probability of the dialogue field;

[0201] The dialogue field corresponding to the highest dialogue probability is selected as the predicted field of the user dialogue, and the highest dialogue probability is taken as a field probability of the predicted field.

[0202] In addition, the field division model is a deep learning network.

[0203] In the embodiment of the application, the second division module 704 is further configured to:

[0204] input the dialogue word segmentation into the deep learning network;

[0205] convert the dialogue word segmentation into a word vector by using an embedding layer of the deep learning network;

[0206] extract a vector feature of the word vector by using a pooling layer of the deep learning network;

[0207] gather the vector feature by using a full connection layer of the deep learning network;

[0208] perform normalization processing on the gathered vector feature by using a mapping layer of the deep learning network, to obtain a probability value of a field to which the user dialogue belongs;

[0209] divide the field of the user dialogue based on the probability value of the field to which the user dialogue belongs.

[0210] The field division device according to the embodiment of the application can be seen as follows: because the technical means of extracting keywords from user dialogue, performing feature calculation on the keywords based on a field data set to obtain a predicted field of the user dialogue and a corresponding field probability of the predicted field, if the field probability is greater than a preset threshold, dividing the field of the user dialogue into the predicted field corresponding to the field probability, and if the field probability is less than or equal to the preset threshold, dividing the field of the user dialogue based on a field division model, the technical problems of a sharp increase in the size of dialogue rule templates, a serious impact on service performance, high cost of later development and maintenance, poor semantic understanding ability, a large requirement for data volume, high requirement for data annotation, and easy misdivision of fields are overcome, and the technical effects of solving dialogue entanglement, making the field division result not only dependent on annotated data, and improving the accuracy of field division are achieved.

[0211] Figure 8 An exemplary system architecture 800 to which the field division method or the field division device according to the embodiment of the application can be applied is shown.

[0212] As Figure 8As shown, the system architecture 800 can include terminal devices 801, 802, 803, a network 804 and a server 805. The network 804 is a medium for providing communication links between the terminal devices 801, 802, 803 and the server 805. The network 804 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0213] The users can use the terminal devices 801, 802, 803 to interact with the server 805 through the network 804 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 801, 802, 803, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0214] The terminal devices 801, 802, 803 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc.

[0215] The server 805 can be a server providing various services, such as a background management server providing support for users to browse shopping websites using the terminal devices 801, 802, 803. The background management server can analyze and process received product information query requests and other data, and feed back the processing results (such as target push information, product information) to the terminal devices.

[0216] It should be noted that the method for dividing fields provided by the embodiments of the present application is generally executed by the server 805, and accordingly, the device for dividing fields is generally arranged in the server 805.

[0217] It should be understood that, Figure 8 The number of terminal devices, networks and servers in the above description is only illustrative. According to the needs of implementation, there can be any number of terminal devices, networks and servers.

[0218] The following refers to Figure 9 which shows a structural schematic diagram of a computer system 900 of a terminal device suitable for implementing the embodiments of the present application. Figure 9 The terminal device shown is only an example and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0219] As Figure 9As shown, the computer system 900 includes a central processing unit (CPU) 901 which can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 902 or loaded into a random access memory (RAM) 903 from a storage section 908. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0220] Connected to the I / O interface 905 are an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a display device such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as necessary. A removable recording medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 910 as necessary, so that a computer program read therefrom is installed into the storage section 908 as necessary.

[0221] In particular, the processes described above with reference to the flow charts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flow charts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable recording medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-described functions defined in the system of the present disclosure are performed.

[0222] It should be noted that the computer-readable medium shown in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component. In the present application, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or component. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0223] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code containing one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order than that shown in the drawings. For example, two blocks that are shown in succession can actually be executed substantially in parallel, and sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams or flowcharts, and the combination of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0224] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. The described modules can also be arranged in a processor, for example, a processor can be described as including an extracting module, a calculating module, a first dividing module and a second dividing module. In some cases, the names of these modules do not constitute a limitation on the modules themselves, for example, the extracting module can also be described as a "module for extracting keywords from user utterances".

[0225] As another aspect, the present application also provides a computer readable medium, which can be included in the device described in the above embodiments, or can exist independently without being assembled into the device. The computer readable medium carries one or more programs, which, when executed by the device, cause the device to include the following steps: S201: extracting keywords from user utterances; S202: performing feature calculation on the keywords based on a domain data set to obtain a predicted domain of the user utterance and a corresponding domain probability; S203: if the domain probability is greater than a preset threshold, dividing the domain of the user utterance into the predicted domain corresponding to the domain probability; S204: if the domain probability is less than or equal to the preset threshold, dividing the domain of the user utterance based on a domain division model.

[0226] According to the technical scheme of the embodiments of the present application, by adopting the technical means of extracting keywords from user utterances; performing feature calculation on the keywords based on a domain data set to obtain a predicted domain of the user utterance and a corresponding domain probability; if the domain probability is greater than a preset threshold, dividing the domain of the user utterance into the predicted domain corresponding to the domain probability; and if the domain probability is less than or equal to the preset threshold, dividing the domain of the user utterance based on a domain division model, the technical problems of the rapid increase in the size of the utterance rule template, the serious impact on service performance, the high cost of later development and maintenance, the poor semantic understanding ability, the large requirement for data volume, the high requirement for data annotation, and the misclassification of domains are overcome, and the technical effects of solving utterance entanglement, making the domain division result not only dependent on annotated data, and improving the accuracy of domain division are achieved.

[0227] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for domain partitioning, characterized in that, include: Extract keywords from user responses; Based on the domain dataset, feature calculation is performed on the keywords to obtain the predicted domain of the user's speech and its corresponding domain probability; If the domain probability is greater than a preset threshold, then the domain of the user's speech is classified as the predicted domain corresponding to the domain probability; If the domain probability is less than or equal to a preset threshold, then the user's speech is divided into domains based on the domain segmentation model; Based on the domain dataset, feature calculations are performed on the keywords to obtain the predicted domain of the user's speech and its corresponding domain probability, including: Based on the gradient decision generation tree, feature calculation is performed on the feature data of each keyword to obtain the discourse domain of each keyword and its corresponding discourse probability. The speech domain corresponding to the highest speech probability is selected as the prediction domain of the user speech, and the highest speech probability is used as the domain probability of the prediction domain.

2. The method according to claim 1, characterized in that, Before extracting keywords from user responses, the process also includes: Match the user's dialogue to the dialogue template; If a dialogue template that matches the user's dialogue exists, the user's dialogue is categorized into domains based on the matching result; If no matching script template exists, the keywords are extracted from the user's script.

3. The method according to claim 2, characterized in that, Keyword extraction from user dialogue includes: The user's speech is segmented to obtain at least one speech segment; Convert the word segmentation of the speech into word vectors; Extract the vector features of the word vectors; The state transition diagram is obtained by calculating the vector features using the conditional random field algorithm. Extract at least one keyword from the state transition diagram.

4. The method according to claim 1, characterized in that, Before performing feature calculations on the feature data of each keyword based on the gradient decision generation tree, the method further includes: The feature data corresponding to each keyword in each domain dataset is obtained, and the feature data of each keyword is uniformly processed; wherein, each domain dataset stores keywords and their corresponding feature data in the same discourse domain.

5. The method according to claim 3, characterized in that, Domain partitioning models are based on deep learning networks; The user's speech is divided into domains based on a domain partitioning model, including: The segmented words of the speech are input into a deep learning network; The utterance segmentation is converted into word vectors using the embedding layer of a deep learning network; The vector features of the word vectors are extracted using the pooling layers of a deep learning network; The vector features are aggregated using fully connected layers of a deep learning network; The collected vector features are normalized using the mapping layer of a deep learning network to obtain the probability value of the domain to which the user's speech belongs. The user's speech is classified into domains based on the probability value of the domain to which the user's speech belongs.

6. A device for dividing a domain, characterized in that, include: The extraction module is used to extract keywords from user statements; The calculation module is used to perform feature calculation on the keywords based on the domain dataset to obtain the predicted domain of the user's speech and its corresponding domain probability; The first segmentation module is used to segment the domain of the user's speech into the predicted domain corresponding to the domain probability when the domain probability is greater than a preset threshold. The second segmentation module is used to segment the user's speech into domains based on the domain segmentation model when the domain probability is less than or equal to a preset threshold. The calculation module is further configured to: perform feature calculation on the feature data of each keyword based on the gradient decision generation tree to obtain the discourse domain of each keyword and its corresponding discourse probability; select the discourse domain corresponding to the highest discourse probability as the prediction domain of the user discourse, and use the highest discourse probability as the domain probability of the prediction domain.

7. The apparatus according to claim 6, characterized in that, It also includes a matching module for: Match the user's dialogue to the dialogue template; If a dialogue template that matches the user's dialogue exists, the user's dialogue is categorized into domains based on the matching result; If no matching script template exists, the keywords are extracted from the user's script.

8. The apparatus according to claim 7, characterized in that, The extraction module is also used for: The user's speech is segmented to obtain at least one speech segment; Convert the word segmentation of the speech into word vectors; Extract the vector features of the word vectors; The state transition diagram is obtained by calculating the vector features using the conditional random field algorithm. Extract at least one keyword from the state transition diagram.

9. An electronic device for domain division, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

Images