Text intent recognition method and device, electronic device, and storage medium

Abstract

The present disclosure provides a text intent recognition method and device, electronic device, and computer-readable storage medium, which relate to the field of artificial intelligence technology. The method includes: extracting keywords from the text to be recognized; generating an initial tensor of the text to be recognized based on the word vector of each keyword; encoding the initial tensor to obtain a feature tensor of the text to be recognized; and processing the feature tensor using an expanded convolutional network model to obtain a recognition result of the text to be recognized. The present disclosure can fully tap the semantic features of the text and improve the accuracy of text intent recognition.

Description

Background Art

[0002] With the development of artificial intelligence technology, text information processing has gradually shifted from being completed manually to being completed by machines. For example, more and more manual customer service staff are being replaced by intelligent assistants. Compared with manual customer service, intelligent assistants have more proactive two-way interaction capabilities, more convenient communication methods, and more professional domain knowledge, and can therefore provide better quality services.

[0003] The realization of intelligent assistants relies on the recognition of user input text through machine learning technology. Compared with the basic semantic recognition of text, it is more difficult to recognize the user's intention reflected in the text. Existing text intent recognition methods cannot achieve a high accuracy rate.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a text intent recognition method and device, an electronic device, and a computer-readable storage medium, thereby overcoming the problem that existing text intent recognition methods cannot achieve a high accuracy rate, at least to a certain extent.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a text intent recognition method is provided, including: extracting keywords from a text to be recognized; generating an initial tensor of the text to be recognized based on word vectors of each of the keywords; encoding the initial tensor to obtain a feature tensor of the text to be recognized; and processing the feature tensor using an expanded convolutional network model to obtain a recognition result of the text to be recognized.

[0008] In an exemplary embodiment of the present disclosure, the generating the initial tensor of the text to be recognized based on the word vector of each of the keywords includes: converting each of the keywords into a word vector to generate a first initial tensor of the text to be recognized; splitting each of the keywords into keywords, and generating a second initial tensor of the text to be recognized based on the word vector of the keyword; and concatenating the first initial tensor and the second initial tensor into the initial tensor.

[0009] In an exemplary embodiment of the present disclosure, the method also includes: extracting position features of each of the keywords in the text to be recognized, generating a third initial tensor of the text to be recognized; extracting context features of each of the keywords, generating a fourth initial tensor of the text to be recognized; and splicing the first initial tensor and the second initial tensor into the initial tensor includes: splicing the first initial tensor, the second initial tensor, the third initial tensor and the fourth initial tensor into the initial tensor.

[0010] In an exemplary embodiment of the present disclosure, encoding the initial tensor to obtain the feature tensor of the text to be recognized includes: processing the initial tensor using an attention model to obtain the feature tensor of the text to be recognized.

[0011] In an exemplary embodiment of the present disclosure, the dilated convolutional network model includes: a Highway layer, used to perform a nonlinear transformation on the feature tensor to obtain an intermediate matrix; a dilated convolution layer (Dilated layer), used to perform a convolution operation on the intermediate matrix using multiple dilated convolution kernels to obtain a convolution layer vector; a fully connected layer, used to perform a fully connected operation on the convolution layer vector to obtain a fully connected vector; a Dropout layer, used to discard part of the dimensions of the fully connected layer vector according to a preset discarding ratio to obtain a Dropout layer vector; an output layer, used to normalize the Dropout layer vector to obtain an output vector.

[0012] In an exemplary embodiment of the present disclosure, the method further includes: obtaining sample texts, and segmenting each sample text to obtain sample words; calculating the internal weight of each sample word through the textrank algorithm; calculating the TF-IDF (term frequency-inverse text frequency) of each sample word; obtaining the comprehensive weight of each sample word according to the internal weight of each sample word and the TF-IDF; screening out sample words whose comprehensive weight is higher than a weight threshold, and generating a keyword library; extracting keywords from the text to be recognized includes: extracting keywords from the text to be recognized based on the keyword library.

[0013] In an exemplary embodiment of the present disclosure, before extracting keywords from the text to be recognized, the method further includes: preprocessing the text to be recognized; wherein the preprocessing includes at least one of word segmentation, removal of stop words, and regular cleaning.

[0014] According to one aspect of the present disclosure, a text intent recognition device is provided, including: an extraction module, used to extract keywords from a text to be recognized; a conversion module, used to generate an initial tensor of the text to be recognized based on the word vector of each of the keywords; an encoding module, used to encode the initial tensor to obtain a feature tensor of the text to be recognized; and a processing module, used to process the feature tensor using an expanded convolutional network model to obtain a recognition result of the text to be recognized.

[0015] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned methods by executing the executable instructions.

[0016] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any one of the above methods is implemented.

[0017] The exemplary embodiments of the present disclosure have the following beneficial effects:

[0018] After extracting keywords from the text to be recognized, the initial tensor is obtained by converting the word vector, and then converted into a feature tensor by encoding, and analyzed and processed using the dilated convolutional network model to obtain the intent recognition result of the text to be recognized. On the one hand, after the text to be recognized is initially vectorized, the features are further extracted by encoding, which can better explore the semantics and intent of the text to be recognized, so that the feature tensor can better represent the global information of the text to be recognized. On the other hand, using the dilated convolutional network model for analysis, the features in the feature tensor can be extracted from the perspective of the machine, and the extraction is more sufficient. The dilated convolution operation on the feature tensor can retain the global information of the text to be recognized or the longer word order feature information, thereby improving the accuracy of text intent recognition. On the other hand, the dilated convolutional network model is based on the neural network model, which is suitable for big data scenarios, and the training process is simple and practical.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 A flowchart of a text intent recognition method in this exemplary embodiment is schematically shown;

[0022] Figure 2 A schematic diagram schematically illustrates a dilated convolution kernel in this exemplary embodiment;

[0023] Figure 3 A schematic diagram schematically illustrates a multi-head attention encoding process in this exemplary embodiment;

[0024] Figure 4 A sub-flow chart of a text intention recognition method in this exemplary embodiment is schematically shown;

[0025] Figure 5 A schematic diagram schematically illustrates a process of establishing a dilated convolutional network model in this exemplary embodiment;

[0026] Figure 6 A structural block diagram of a text intention recognition device in this exemplary embodiment is schematically shown;

[0027] Figure 7 An electronic device for implementing the above method in this exemplary embodiment is schematically shown;

[0028] Figure 8 A computer-readable storage medium for implementing the above method in this exemplary embodiment is schematically shown. DETAILED DESCRIPTION

[0029] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0030] In one solution of the related technology, text intent recognition is performed through a support vector machine model (SVM). Although the support vector machine model can perform relatively effective intent classification based on the characteristics of the text, when applied to big data scenarios, the model training speed is slow, and the characteristics need to be set manually, which has certain limitations and affects the accuracy of the model. In another solution of the related technology, the CoreNLP tool (a natural language processing tool) developed by Stanford University is used to match text with regular expressions to achieve text intent recognition. The disadvantage is that the expressions need to be set manually, which cannot be flexibly applied to online texts in practical applications.

[0031] In view of the above problems, the exemplary embodiments of the present disclosure first propose a text intention recognition method, which can be applied to scenarios such as search and intelligent assistants. By recognizing the intention of the text input by the user, service content can be provided针对性地. Taking the application scenario of an intelligent assistant as an example, the intelligent assistant can be divided into the following 7 business types: commodity query business, order business, after-sales service business, fuzzy discount business, specific commodity discount business, channel direct access business, and unrecognized business. The purpose of this embodiment is to identify which of the above 7 businesses the text intention belongs to, so as to be able to provide services in specific business scenarios.

[0032] The following will结合附 Figure 1 be further described. As Figure 1 shown, the text intention recognition method may include steps S110 to S140:

[0033] In step S110, keywords are extracted from the text to be recognized.

[0034] In this embodiment, text intention recognition is performed on a sentence-by-sentence basis. For a piece of text input by the user, intention recognition can be performed on each sentence separately, or on important sentences (such as the first sentence) among them. Therefore, the text to be recognized is usually a single sentence. A keyword refers to a word that contributes to the semantics or intention in the text to be recognized. Generally, stop words and modal particles in the text to be recognized are removed, and the remaining words are keywords.

[0035] In an exemplary embodiment, preprocessing operations such as word segmentation, stop word removal, and regular cleaning can be performed on the text to be recognized in sequence. For example, jieba word segmentation or the LTP tool (a language analysis tool developed by a team at Harbin Institute of Technology) can be used, and combined with a personalized word library for the application scenario, the text to be recognized is segmented; then stop words are removed, such as meaningless content like "de", "le", "ne", numbers, letters, punctuation marks, etc.; finally, meaningless words that often appear in a specific application scenario can be cleaned up by regular expressions, such as modal particles like "hao de", "haha", "enen", etc., or some common uncivilized terms can be cleaned up using the speech template developed by Stanford University. After the above process, a part of the words remaining in the text to be recognized can be determined as keywords.

[0036] In step S120, an initial tensor of the text to be recognized is generated based on the word vectors of the respective keywords.

[0037] Among them, the word vector can be a one-hot vector determined by the bag-of-words model, or a distributed vector determined by the word2vec model (a natural language processing tool). After each keyword is converted into a word vector, each word vector can be spliced ​​according to the order of the keywords in the text to be recognized. Specifically, it can be spliced ​​into a vector with a higher dimension or a matrix, which is the initial tensor of the text to be recognized. Therefore, the initial tensor can be a one-dimensional tensor (i.e., a vector) or a two-dimensional tensor (i.e., a matrix).

[0038] The feature representation of the text to be recognized is performed by converting keywords into word vectors, which usually can only represent a simple combination of the word meanings of the text to be recognized. Therefore, it is marked as an initial tensor in step S120, and further feature extraction and sorting are required in step S130.

[0039] It should be added that in addition to the word vectors of keywords, other information related to the text to be recognized can also be spliced ​​into the initial tensor in the form of vectors, such as the specific options checked by the user when entering the text to be recognized, the path to enter the text input page, etc. This information may be helpful for text intent recognition.

[0040] In step S130, the initial tensor is encoded to obtain a feature tensor of the text to be recognized.

[0041] Based on the initial tensor obtained in step S120, further feature extraction and sorting are performed on it, that is, the encoding process. The encoding process in the encoder-decoder model can be adopted, for example, encoding can be achieved through the existing convolutional neural network model (CNN), recurrent neural network model (RNN) or long short-term memory network model (LSTM). The encoding process takes into account the relationship between each keyword and the context in the text to be identified, and can fully express the semantic and intentional features of the text to be identified. The initial tensor is encoded to obtain a feature tensor.

[0042] In step S140, the feature tensor is processed using the dilated convolutional network model to obtain a recognition result of the text to be recognized.

[0043] In a general convolutional neural network model, a convolution kernel is used to perform a convolution operation on the input tensor (or pixel matrix of an image) to extract local features. It is usually performed together with a pooling operation to reduce overfitting and reduce the amount of data processing. However, the pooling operation can also cause information loss. In the dilated convolutional network model, a dilated convolution kernel is used for convolution operations. The dilated convolution kernel can be referred to Figure 2 As shown in the figure, the original 3*3 convolution kernel is expanded and the hole in the middle is filled with "0". The size of the expanded convolution kernel = expansion coefficient * (original convolution kernel size - 1) + 1. Figure 2 The expansion coefficient shown is 2, and the expansion coefficient can usually be set to 4, 6, 8, etc. The dilated convolutional network model can increase the receptive field of the convolution kernel while avoiding information loss in pooling, so that the output of each convolution operation contains a larger range of information. Especially in text intent recognition, the global information of the text or longer word order feature information can be reflected in the convolution output, and then the subsequent processing of the neural network model can improve the accuracy of text intent recognition.

[0044] Based on the above description, after extracting keywords from the text to be recognized, the initial tensor is obtained by converting the word vector, and then converted into a feature tensor by encoding, and analyzed and processed using the dilated convolutional network model to obtain the intention recognition result of the text to be recognized. On the one hand, after the text to be recognized is initially vectorized, the features are further extracted by encoding, which can better explore the semantics and intention of the text to be recognized, so that the feature tensor can better represent the global information of the text to be recognized. On the other hand, by using the dilated convolutional network model for analysis, the features in the feature tensor can be extracted from the perspective of the machine, and the extraction is more sufficient, and the dilated convolution operation on the feature tensor can retain the global information of the text to be recognized or the longer word order feature information, thereby improving the accuracy of text intention recognition. On the other hand, the dilated convolutional network model is based on the neural network model, which is suitable for big data scenarios, and the training process is simple and practical.

[0045] In an exemplary embodiment, step S120 may include the following steps:

[0046] Convert each keyword into a word vector to generate the first initial tensor of the text to be recognized;

[0047] Split each keyword into keywords, and generate a second initial tensor of the text to be recognized based on the word vector of the keyword;

[0048] Concatenate the first initial tensor and the second initial tensor into the initial tensor.

[0049] Among them, after the keywords are converted into word vectors, they are concatenated to generate the first initial tensor; after the keywords are converted into character vectors, they are concatenated to generate the second initial tensor. That is, for the text to be recognized, the features are represented from the two aspects of keywords and keywords respectively. The initial tensor is formed by concatenating the first initial tensor and the second initial tensor, and contains the feature representation of keywords and keywords at the same time, which is more sufficient for the representation of the text to be recognized.

[0050] When converting word vectors, you can pre-train the corpus of the scene based on the word2vec model, debug parameters such as the word vector dimension and sliding window size, and train to obtain the distributed vector of each keyword in the corpus, thereby achieving the corresponding conversion.

[0051] Furthermore, the text intent recognition method may also include the following steps:

[0052] Extracting the position features of each keyword in the text to be recognized, and generating a third initial tensor of the text to be recognized;

[0053] Extracting context features of each keyword to generate a fourth initial tensor of the text to be recognized;

[0054] Correspondingly, the first initial tensor, the second initial tensor, the third initial tensor and the fourth initial tensor may be concatenated into an initial tensor.

[0055] Among them, the position feature can be the approximate position of the keyword in the text to be recognized, such as the beginning, middle, or end of the text to be recognized, or the text in which the keyword is a single word. The position feature can also be the sequence number of the keyword in the text to be recognized, etc. The above position feature can be represented in the form of a vector or a matrix. For example, in the word bag of one-hot type keywords, each keyword can be recorded at the beginning (represented by 1), the middle (represented by 2), the end (represented by 3), or as a single word (represented by 4) in the text to be recognized. If it does not appear in the keyword, the corresponding dimension is 0. The position feature of the keyword in the text to be recognized can be represented by an n-dimensional vector, where n is the size of the word bag, and the n-dimensional vector is the third initial tensor.

[0056] Contextual features refer to the feature information of other keywords adjacent to a specific keyword. For example, they can be defined as the first two characters and the last two characters adjacent to each keyword. Through the keyword bag-of-words model, the dimension value of the adjacent keyword hit is 1, and the remaining dimension values ​​are 0. Then, the contextual feature vector of each keyword can be obtained, and the fourth initial vector can be obtained after splicing.

[0057] It should be noted that the specific conversion method of the third initial tensor and the fourth initial tensor is not limited to the above method, and this embodiment does not specifically limit this.

[0058] The third initial tensor and the fourth initial tensor can be regarded as supplementary representations of the text information to be recognized. Concatenating them with the first initial tensor and the second initial tensor into initial tensors can more fully characterize the features of the text to be recognized.

[0059] In an exemplary embodiment, step S130 may include the following steps:

[0060] The initial tensor is processed using the attention model to obtain the feature tensor of the text to be recognized.

[0061] In the process of encoding the initial tensor, the attention model can adjust the degree of attention to each part of the initial tensor and narrow the focus range. For example, assuming the initial tensor input_X = {x1, x2, x3…x m}, the tensor output in the attention layer is Y = {y1,y2,y3…y p}, then we have the following relationship:

[0062]

[0063] a i =Softmax(Similarity({y1,y2,…y i-1},x i ));

[0064]

[0065] Among them, Attention(input_X,Y) is the feature tensor output by the attention model, a i is the weight of the attention mechanism, Softmax is the normalization function, and Similarity is the similarity function.

[0066] Furthermore, the above attention model can be a multi-head attention model. Figure 3 As shown in the figure, in the multi-head attention model, the initial tensor input_X (corresponding to the key and value in the attention model) and the attention tensor Y (corresponding to the query in the attention model) are linearized multiple times, and the scaled dot product is calculated, and then concatenated. It can also be linearized again to output the feature vector. "Multi-head" means that the attention dot product calculation is performed on multiple linear projection subspaces to obtain the attention weight, and then the results of multiple calculations are comprehensively considered to obtain the final feature vector. Its advantage is that it allows the model to learn correlation information in different subspaces, and the information representation of the feature vector is more sufficient.

[0067] In an exemplary embodiment, the dilated convolutional network model may include the following network layers:

[0068] The highway layer is used to perform nonlinear transformation on the feature tensor to obtain the intermediate matrix;

[0069] The dilated convolution layer is used to perform convolution operations on the intermediate matrix using multiple dilated convolution kernels to obtain a convolution layer vector;

[0070] The fully connected layer is used to perform a fully connected operation on the convolutional layer vector to obtain a fully connected vector;

[0071] The Dropout layer is discarded to discard some dimensions of the fully connected layer vector according to a preset discard ratio to obtain a Dropout layer vector;

[0072] The output layer is used to normalize the Dropout layer vector to obtain the output vector.

[0073] The Highway layer can be implemented in two specific ways: one-dimensional convolution (conv1d) and multi-layer perceptron (MLP). Conv1d is suitable for the case where the feature tensor is a vector, and multiple convolution kernels can be defined to concatenate the convolution results into an intermediate matrix; MLP is suitable for the case where the feature tensor is a matrix or a higher-dimensional tensor. The design of the Highway layer adds information channels to the neural network layer, which can be used to capture important semantics in the feature tensor.

[0074] No matter what the dimension of the feature tensor is, it can be converted into an intermediate matrix in the form of a two-dimensional tensor through the Highway layer, and then input into the dilated convolution layer, and the convolution operation can be performed using the two-dimensional dilated convolution kernel. For local feature information in different ranges, multiple dilated convolution kernels with different receptive fields can be designed, and the intermediate matrix can be convolved respectively to obtain the convolution layer vector. In addition, the dilated convolution layer can also include multiple layers, and the intermediate matrix is ​​continuously convolved multiple times. Correspondingly, each of the above-mentioned multiple dilated convolution kernels can be a group of convolution kernels with a sequence, and this embodiment does not specifically limit this.

[0075] In the convolution operation, the local features of the text information are extracted in multiple aspects, so that each local feature can be further learned through the fully connected operation in the fully connected layer. The Dropout layer is designed after the fully connected layer, which is manifested as discarding a part of the neurons in the fully connected layer. In essence, it discards a part of the dimension of the fully connected layer vector, which can prevent overfitting. Usually, the Keep_prob parameter (retention rate) is set in the model, and the retained neurons are randomly selected according to the parameter to obtain the Dropout layer vector. Finally, the normalization operation is performed, such as calculating the normalized value through functions such as Softmax and Sigmoid to obtain the output vector.

[0076] It should be noted that the above are only exemplary lists of several representative network layers of the dilated convolutional network model. In practical applications, other intermediate layers can be added as needed, such as general hidden layers, pooling layers, ReLu (linear rectified unit) activation layers, optimization layers (such as using Adam optimizer), Accuracy layers, etc., and the order of the network layers can also be swapped, which is not particularly limited in this embodiment.

[0077] In an exemplary embodiment, reference Figure 4As shown, the text intent recognition method may further include steps S410 to S450:

[0078] Step S410, obtaining sample texts, and segmenting each sample text to obtain sample words;

[0079] Step S420, calculating the internal weight of each sample word by using the text keyword extraction algorithm TextRank algorithm;

[0080] Step S430, calculating the TF-IDF of each sample word;

[0081] Step S440, obtaining a comprehensive weight of each sample word according to the internal weight of each sample word and TF-IDF;

[0082] Step S450, screening out sample words whose comprehensive weights are higher than the weight threshold, and generating a keyword library;

[0083] After obtaining the keyword library, keywords can be extracted from the text to be recognized based on the keyword library.

[0084] Among them, the sample text is a massive amount of text extracted from the corpus of the application scenario, which can reflect the characteristics of various types of text in the scenario. For example, in the scenario of the intelligent assistant, the first sentence of the user input text can be extracted from the Hive (a database) log table of the intelligent assistant as the sample text. The Textrank algorithm is an internal weight calculated based on the internal statistics of the sample text set, which reflects the internal importance of each sample word; TF-IDF is an index calculated based on internal and external statistics, which reflects the importance of each sample word for a specific scenario. By connecting these two indexes, such as multiplication, multiplication with weighted indexes, averaging, weighted calculation, etc., the comprehensive weight of each sample word can be obtained. The comprehensive weight takes into account multiple indicators such as information gain and document frequency of the text, and can more accurately reflect the representativeness of each sample word in a specific scenario. Therefore, according to the weight threshold, sample words with higher comprehensive weights can be screened out to build a keyword library. The weight threshold can be set to a specific value based on experience and actual usage.

[0085] After obtaining the keyword library, keywords can be extracted from the text to be recognized by matching the text with the word library. Compared with conventional word segmentation tools, keywords extracted through the keyword library are more representative of the scene and can better reflect the user's intention, which is conducive to subsequent text intention recognition.

[0086] In an exemplary embodiment, before extracting keywords from the text to be recognized, the text intent recognition method further includes the following steps:

[0087] Preprocess the text to be recognized; wherein the preprocessing includes at least one of word segmentation, stop word removal, and regular expression cleaning.

[0088] For example, jieba word segmentation or LTP tools can be used, and combined with the personalized word library of the application scenario, the text to be recognized is segmented; stop words can be content without practical meaning such as "de", "le", "ne", numbers, letters, punctuation marks, etc.; regular cleaning can be to remove meaningless words that often appear in specific application scenarios, such as modal particles like "hao de", "haha", "enen", etc., or some common uncivilized terms can also be removed using the speech template developed by Stanford University. After preprocessing, the text to be recognized can retain only the valid information, reducing the subsequent data processing volume, and reducing the interference of invalid information on intent recognition, further improving the recognition accuracy.

[0089] In an exemplary embodiment, the establishment process of the dilated convolutional neural network model (dilated CNN) can refer to Figure 5 as shown. First, sample texts are obtained, segmented, stop words are removed, and regular cleaning is performed to obtain sample words; for the sample words, the textrank weights and TF-IDF are calculated, and the sample words with higher comprehensive weights are screened to construct a keyword library; then the sample texts are matched and processed through the keyword library. For each sample text, after the keywords are matched, the word vectors, character vectors, position features, and context features are extracted, and the sample text is initially represented in the form of an initial tensor; then the multi-head attention model can be used to process the initial tensor to achieve the encoding of the text based on the initial representation. This process can also input the initial category of the text to assist in intent recognition; after obtaining the feature tensor, it is input into the dilated CNN, and the model is trained and the weights are adjusted through the labels of the sample texts. The parameters of the multi-head attention model can also be adjusted during the training process.

[0090] In an exemplary embodiment, the parameters of the dilated CNN can be set as follows:

[0091] Embedding layer dimension Embedding_dim = 64 / / Dimension of each character vector (or word vector)

[0092] Text length Seq_length = 100 / / Analyze and count the text length, and set the length to 100 by Gaussian distribution

[0093] Num_classes = 7 / / Number of intent classifications

[0094] Kernel_sizes = [1, 3, 5] / / Convolution kernel size

[0095] Vocab_size = 5000 / / Size of the keyword library

[0096] Hidden_dim = 128 / / Hidden layer dimension

[0097] Dilated_rate=2 / / dilation coefficient of dilated convolution kernel

[0098] Dropout_keep_prob=0.5 / / Retention rate of Dropout layer

[0099] Learning_rate=0.003 / / Learning rate

[0100] Batch_size = 64 / / Number of training batches

[0101] Num_epochs = 10 / / Number of sample iterations

[0102] Use_attention = True / / Use attention mechanism

[0103] Attention_dim = 128 / / Attention model output dimension

[0104] Use_highway = True / / Use Highway layer

[0105] Highway_type = 'MLP' / / Highway layer uses MLP

[0106] Print_per_batch = 100 / / Output results every 100 batches

[0107] Save_per_batch = 10 / / Save results every 10 batches

[0108] Based on the trained expanded CNN, practical applications can be carried out. For example, it can be applied to the intelligent assistant scenario to classify user intent: product query business, order business, after-sales service business, fuzzy discount business, specific product discount business, channel direct business and unidentified business, so as to provide services in specific business scenarios. The above-mentioned expanded CNN model can be based on the Tensorflow (an artificial intelligence learning system) system, build an algorithm service in Java language, and improve service performance through GPU resources, and cooperate with upstream and downstream joint debugging. In addition, a business monitoring mechanism can also be built in the algorithm service, for example, the user click situation and the change trend of offline indicators in a certain time period can be monitored, and then the data of the time period with poor effect can be extracted from the log table of the intelligent assistant, and targeted data analysis can be performed to optimize the model. The evaluation index of the expanded CNN can adopt F1 value, precision rate, recall rate, AUC or ROC curve, etc. This embodiment does not make special restrictions on this. Taking F1 value as an example, in the application of intelligent assistant, the F1 value of the expanded CNN for the test set can reach 0.92, indicating that the text intent recognition accuracy of this exemplary embodiment is high.

[0109] The exemplary embodiment of the present disclosure also provides a text intention recognition device, referring to Figure 6 As shown, the device 600 may include: an extraction module 610, used to extract keywords from the text to be recognized; a conversion module 620, used to generate an initial tensor of the text to be recognized based on the word vector of each keyword; an encoding module 630, used to encode the initial tensor to obtain a feature tensor of the text to be recognized; and a processing module 640, used to process the feature tensor using an expanded convolutional network model to obtain a recognition result of the text to be recognized.

[0110] In an exemplary embodiment, the conversion module may also include: a first conversion unit, used to convert each keyword into a word vector to generate a first initial tensor of the text to be recognized; a second conversion unit, used to split each keyword into keywords, and generate a second initial tensor of the text to be recognized based on the word vector of the keyword; a splicing unit, used to splice the first initial tensor and the second initial tensor into an initial tensor.

[0111] Furthermore, the conversion module may also include: a third conversion unit, used to extract the position features of each keyword in the text to be recognized, and generate a third initial tensor of the text to be recognized; a fourth conversion unit, used to extract the context features of each keyword, and generate a fourth initial tensor of the text to be recognized; the splicing unit may also be used to splice the first initial tensor, the second initial tensor, the third initial tensor and the fourth initial tensor into an initial tensor.

[0112] In an exemplary embodiment, the encoding module can also be used to process the initial tensor using an attention model to obtain a feature tensor of the text to be recognized.

[0113] In an exemplary embodiment, the dilated convolutional network model may include the following network layers:

[0114] Highway layer, used to perform nonlinear transformation on the feature tensor to obtain the intermediate matrix;

[0115] The dilated convolution layer is used to perform convolution operations on the intermediate matrix using multiple dilated convolution kernels to obtain a convolution layer vector;

[0116] The fully connected layer is used to perform a fully connected operation on the convolutional layer vector to obtain a fully connected vector;

[0117] The Dropout layer is used to discard some dimensions of the fully connected layer vector according to a preset drop ratio to obtain a Dropout layer vector;

[0118] The output layer is used to normalize the Dropout layer vector to obtain the output vector.

[0119] In an exemplary embodiment, the text intent recognition apparatus may further include a vocabulary building module, wherein the vocabulary building module may further include the following units:

[0120] The sample word segmentation unit is used to obtain sample texts and segment each sample text to obtain sample words;

[0121] Textrank calculation unit, used to calculate the internal weight of each sample word through the textrank algorithm;

[0122] TF-IDF calculation unit, used to calculate the TF-IDF of each sample word;

[0123] A comprehensive calculation unit, used to obtain the comprehensive weight of each sample word according to the internal weight of each sample word and TF-IDF;

[0124] A weight screening unit is used to screen out sample words whose comprehensive weight is higher than a weight threshold and generate a keyword library;

[0125] Correspondingly, the extraction module can also be used to extract keywords from the text to be recognized based on the keyword library.

[0126] In an exemplary embodiment, the text intent recognition device may further include: a preprocessing module, used to preprocess the text to be recognized; wherein the preprocessing includes at least one of word segmentation, stop word removal, and regular expression cleaning.

[0127] The specific details of the modules / units of the above-mentioned device have been described in detail in the embodiments of the method part, so they will not be repeated here.

[0128] Exemplary embodiments of the present disclosure also provide an electronic device capable of implementing the above method.

[0129] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0130] Refer to the following Figure 7 An electronic device 700 according to such an exemplary embodiment of the present disclosure is described below. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0131] like Figure 7As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include, but are not limited to: the at least one processing unit 710, the at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), and a display unit 740.

[0132] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 710 can execute Figure 1 Steps S110 to S140 shown in FIG. 1 may also be performed Figure 4 Steps S410 to S450 etc. are shown.

[0133] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 721 and / or a cache memory unit 722 , and may further include a read-only memory unit (ROM) 723 .

[0134] The storage unit 720 may also include a program / utility 724 having a set (at least one) of program modules 725, such program modules 725 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0135] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0136] The electronic device 700 may also communicate with one or more external devices 900 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0137] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the exemplary embodiment of the present disclosure.

[0138] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification.

[0139] refer to Figure 8 As shown, a program product 800 for implementing the above method according to an exemplary embodiment of the present disclosure is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0140] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0141] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0142] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0143] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0144] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0145] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0146] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0147] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A text intent recognition method, characterized in that: include: Extracting keywords from the text to be recognized, wherein the keywords are extracted based on a keyword library, and the keyword library is determined according to the internal weight of the sample words and the word frequency-inverse text frequency TF-IDF; Based on the word vectors of each of the keywords, an initial tensor of the text to be recognized is generated, wherein the initial tensor also includes position features of each of the keywords in the text to be recognized and context features of each keyword obtained by splitting the keywords; Encoding the initial tensor to obtain a feature tensor of the text to be recognized; The feature tensor is processed using an expanded convolutional network model to obtain a recognition result of the text to be recognized, wherein the expanded convolutional network model includes a high-speed network layer and multiple expanded convolution kernels, the high-speed network layer is used to perform a nonlinear transformation on the feature tensor to obtain an intermediate matrix, and the multiple expanded convolution kernels are used to convolve the intermediate matrix according to local feature information in different ranges to obtain a convolution layer vector.

2. The method according to claim 1, characterized in that The step of generating the initial tensor of the text to be recognized based on the word vector of each of the keywords includes: Convert each of the keywords into a word vector to generate a first initial tensor of the text to be recognized; Splitting each of the keywords into keywords, and generating a second initial tensor of the text to be recognized based on the word vectors of the keywords; The first initial tensor and the second initial tensor are concatenated into the initial tensor.

3. The method according to claim 2, characterized in that The method further comprises: Extracting position features of each of the keywords in the text to be recognized, and generating a third initial tensor of the text to be recognized; Extracting context features of each of the keywords to generate a fourth initial tensor of the text to be recognized; The step of concatenating the first initial tensor and the second initial tensor into the initial tensor comprises: The first initial tensor, the second initial tensor, the third initial tensor, and the fourth initial tensor are concatenated into the initial tensor.

4. The method according to claim 1, characterized in that: The encoding of the initial tensor to obtain the feature tensor of the text to be recognized includes: The initial tensor is processed using an attention model to obtain a feature tensor of the text to be recognized.

5. The method according to claim 1, characterized in that The dilated convolutional network model includes: A high-speed network Highway layer is used to perform nonlinear transformation on the feature tensor to obtain an intermediate matrix; A dilated convolution layer, used to perform a convolution operation on the intermediate matrix using a plurality of dilated convolution kernels to obtain a convolution layer vector; A fully connected layer, used to perform a fully connected operation on the convolutional layer vector to obtain a fully connected vector; A Dropout layer is discarded, which is used to discard some dimensions of the fully connected layer vector according to a preset discard ratio to obtain a Dropout layer vector; The output layer is used to normalize the Dropout layer vector to obtain an output vector.

6. The method according to claim 1, characterized in that The method further comprises: Obtain sample texts, and segment each sample text to obtain sample words; The internal weight of each sample word is calculated through the text keyword extraction textrank algorithm; Calculate the term frequency-inverse text frequency TF-IDF of each sample word; Obtaining a comprehensive weight of each sample word according to the internal weight of each sample word and TF-IDF; Screening out sample words whose comprehensive weight is higher than the weight threshold to generate a keyword library; The extracting keywords from the text to be recognized includes: Based on the keyword library, keywords are extracted from the text to be recognized.

7. The method according to claim 1, characterized in that Before extracting keywords from the text to be recognized, the method further includes: Preprocessing the text to be recognized; The preprocessing includes at least one of word segmentation, stop word removal, and regular expression cleaning.

8. A text intention recognition device, characterized in that: include: An extraction module is used to extract keywords from the text to be recognized, wherein the keywords are extracted based on a keyword library, and the keyword library is determined according to the internal weight of the sample words and the word frequency-inverse text frequency TF-IDF; A conversion module, used to generate an initial tensor of the text to be recognized based on the word vector of each keyword, wherein the initial tensor also includes the position feature of each keyword in the text to be recognized and the context feature of each keyword obtained by splitting the keyword; An encoding module, used for encoding the initial tensor to obtain a feature tensor of the text to be recognized; A processing module is used to process the feature tensor using an expanded convolutional network model to obtain a recognition result of the text to be recognized, wherein the expanded convolutional network model includes a high-speed network layer and multiple expanded convolution kernels, the high-speed network layer is used to perform a nonlinear transformation on the feature tensor to obtain an intermediate matrix, and the multiple expanded convolution kernels are used to convolve the intermediate matrix according to local feature information in different ranges to obtain a convolution layer vector.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images