A method and device for identifying conversational semantics
Through the combination of the BERT-Doc2Vec model and textCNN/LSTM network, the problem of inefficient intention recognition in medical conversation speech recognition is solved, more accurate and efficient intention recognition is achieved, and context modeling capabilities are enhanced.
Patent Information
- Application Number
- CN202410315061.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-19
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-03-19
AI Technical Summary
In medical discourse speech recognition, existing keyword matching technology cannot effectively utilize contextual relationships, resulting in inefficient in intention recognition. In addition, existing machine learning models have information loss or rely on a large amount of labeled information when processing complex medical text and long sequence input. .
The BERT-Doc2Vec model combination is used for end-to-end semantic encoding, combining the intention recognition model of textCNN and LSTM networks, and extracting multi-scale and long-distance dependency information of text through multi-level superposition progressive encoding-decoding structure, and generating and keywords and Context-related document vectors to achieve more accurate intent recognition.
It improves the accuracy and efficiency of intention recognition, can perceive text semantics more keenly, enhances context modeling capabilities, and reduces dependence on high-quality annotated information.
Smart Images

Figure CN118211593B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of semantic recognition, and in particular to a method and device for conversation semantic recognition. Background Art
[0002] In the medical conversation speech recognition scenario, it is necessary to analyze the user's voice answer. The current method is keyword matching. The existing keyword matching technology has the following defects:
[0003] (1) No keyword database is established, and the keyword matching is completely relied on, which loses the context;
[0004] (2) No intent recognition model is established in the conversation, and a large amount of manpower is required to listen repeatedly to determine the customer's intention, which is very inefficient.
[0005] Therefore, it is necessary to establish a model based on conversational context to identify the intention of the customer's answer, so as to solve the technical problem that sales staff cannot accurately grasp the customer's intention and improve the accuracy of sales. In theory, the machine learning methods, deep learning models or attention-based models in the existing technology can be used as intent recognition models, but these models either have insufficient generalization ability and cannot adapt well to new situations; or cannot accurately understand the complex sentences and subtle differences in medical texts; or are prone to information loss when processing long sequence inputs; or rely on a large amount of high-quality annotation information. Summary of the invention
[0006] In view of the above defects or deficiencies in the prior art, the present invention provides a method and device for conversation semantics recognition, which can solve the above technical problems in whole or in part. On the one hand, the present invention realizes end-to-end semantic encoding through the BERT-Doc2Vec model combination, losslessly reduces context information, and then obtains document vectors related to both keywords and conversation context. The BERT model preserves word meanings, and the Doc2Vec model integrates word context relationships, and the document vector has both accuracy and abstraction. On the other hand, the document vector is input into an original intention recognition model, which introduces a textCNN network and an LSTM network, and can perform multi-scale modeling of text semantic features, and local and global semantic information on the text can be extracted at the same time. In addition, the encoding-decoding end also introduces an LSTM network, which can learn the long-distance dependency information of the text and enhance the context modeling capability; in addition, a multi-level superimposed progressive encoding-decoding structure is adopted to form an end-to-end semantic abstract graph, and the text semantics can be deeply optimized; by modeling text semantics from multiple perspectives, and transmitting semantic abstraction and refined information, a more acute and accurate perception of text semantics is finally achieved. Compared with a single structure, this progressive encoding-decoding network structure can learn semantic features more deeply and improve text expression capabilities.
[0007] One aspect of the present invention provides a method for identifying conversation semantics, comprising:
[0008] Converting the acquired conversation audio data into conversation text data, and identifying conversation keywords from the conversation text data;
[0009] Input the conversation keywords and conversation sequences into the BERT model to obtain the word vectors of the conversation keywords and the conversation text vectors;
[0010] Constructing a document based on the word vector of the conversation keyword and the conversation text vector, extracting effective information of the constructed document using the Doc2vec model, and generating a document vector related to both the keyword and the conversation context;
[0011] The document vector is input into the intent recognition model to obtain an output conversation recognition result.
[0012] Furthermore, the intent recognition model includes an encoding network, a first textCNN-LSTM network layer, a decoding network and a recognition network;
[0013] The encoding network comprises a first Transformer encoder network layer, a second textCNN-LSTM network layer and a second Transformer encoder network layer connected in sequence; wherein the first Transformer encoder network layer receives an input document vector, and the first Transformer encoder network layer is simultaneously jump-connected to the second Transformer encoder network layer;
[0014] The decoding network includes a first Transformer decoder network layer, a third textCNN-LSTM network layer, and a second Transformer decoder network layer connected in sequence; wherein the first Transformer decoder network layer is also jump-connected to the second Transformer decoder network layer;
[0015] The second Transformer encoder network layer in the encoding network is directly jump-connected to the first Transformer decoder network layer in the decoding network, and the second Transformer encoder network layer is connected to the first Transformer decoder network layer through the first textCNN-LSTM network layer;
[0016] The first textCNN-LSTM network layer, the second textCNN-LSTM network layer and the third textCNN-LSTM network layer all include textCNN modules and LSTM modules connected in sequence;
[0017] The recognition network includes a Softmnax classification layer, the output of the second Transformer decoder network layer in the decoding network is connected to the Softmnax classification layer, and the Softmnax classification layer is used to output the final recognition result.
[0018] Furthermore, the step of identifying conversation keywords from the conversation text data further includes:
[0019] Establishing a word library according to the conversation text data, the word library includes a keyword word library, a stop word word library, a dedicated noun word library and a prohibited word word library;
[0020] The effective information of the conversation text is extracted using a word segmentation tool according to the word library to obtain a word segmentation result.
[0021] Furthermore, the first Transformer encoder network layer and the second Transformer encoder network layer each include a first self-attention module, a first residual and normalization module, a first feedforward network, and a second residual and normalization module connected in sequence;
[0022] The output of the second residual and normalization module of the first Transformer encoder network layer is jump-connected to the first self-attention module of the second Transformer encoder network layer.
[0023] Furthermore, the first Transformer decoder network layer and the second Transformer decoder network layer each include a second self-attention module, a third residual and normalization module, a third self-attention module, a fourth residual and normalization module, a second feedforward network, and a fifth residual and normalization module, which are sequentially connected;
[0024] The output of the second residual and normalization module of the second Transformer encoder network layer is jump-connected to the third self-attention module of the first Transformer decoder network layer.
[0025] Furthermore, the output of the fifth residual and normalization module of the first Transformer decoder network layer is jump-connected to the third self-attention module of the second Transformer decoder network layer.
[0026] Another aspect of the present invention provides a conversation semantics recognition device, comprising:
[0027] A keyword acquisition module, configured to convert the acquired conversation audio data into conversation text data, and identify conversation keywords from the conversation text data;
[0028] A vectorization module is configured to input the conversation keywords and the conversation sequence into the BERT model to obtain the word vectors of the conversation keywords and the conversation text vectors;
[0029] A document vector construction module is configured to construct a document according to the word vector of the conversation keyword and the conversation text vector, extract effective information of the constructed document using a Doc2vec model, and generate a document vector related to both the keyword and the conversation context;
[0030] The conversation recognition module is configured to input the document vector into the intent recognition model to obtain an output conversation recognition result.
[0031] Furthermore, the intent recognition model includes an encoding network, a first textCNN-LSTM network layer, a decoding network and a recognition network;
[0032] The encoding network comprises a first Transformer encoder network layer, a second textCNN-LSTM network layer and a second Transformer encoder network layer connected in sequence; wherein the first Transformer encoder network layer receives an input document vector, and the first Transformer encoder network layer is simultaneously jump-connected to the second Transformer encoder network layer;
[0033] The decoding network includes a first Transformer decoder network layer, a third textCNN-LSTM network layer, and a second Transformer decoder network layer connected in sequence; wherein the first Transformer decoder network layer is also jump-connected to the second Transformer decoder network layer;
[0034] The second Transformer encoder network layer in the encoding network is directly jump-connected to the first Transformer decoder network layer in the decoding network, and the second Transformer encoder network layer is connected to the first Transformer decoder network layer through the first textCNN-LSTM network layer;
[0035] The first textCNN-LSTM network layer, the second textCNN-LSTM network layer and the third textCNN-LSTM network layer all include textCNN modules and LSTM modules connected in sequence;
[0036] The recognition network includes a Softmnax classification layer, the output of the second Transformer decoder network layer in the decoding network is connected to the Softmnax classification layer, and the Softmnax classification layer is used to output the final recognition result.
[0037] Furthermore, the keyword acquisition module is further configured as follows:
[0038] Establishing a word library according to the conversation text data, the word library includes a keyword word library, a stop word word library, a dedicated noun word library and a prohibited word word library;
[0039] The effective information of the conversation text is extracted using a word segmentation tool according to the word library to obtain a word segmentation result.
[0040] Furthermore, the first Transformer encoder network layer and the second Transformer encoder network layer each include a first self-attention module, a first residual and normalization module, a first feedforward network, and a second residual and normalization module connected in sequence;
[0041] The output of the second residual and normalization module of the first Transformer encoder network layer is jump-connected to the first self-attention module of the second Transformer encoder network layer;
[0042] The first Transformer decoder network layer and the second Transformer decoder network layer each include a second self-attention module, a third residual and normalization module, a third self-attention module, a fourth residual and normalization module, a second feedforward network, and a fifth residual and normalization module connected in sequence;
[0043] The output of the second residual and normalization module of the second Transformer encoder network layer is jump-connected to the third self-attention module of the first Transformer decoder network layer;
[0044] The output of the fifth residual and normalization module of the first Transformer decoder network layer is jump-connected to the third self-attention module of the second Transformer decoder network layer.
[0045] The present invention provides a method and device for recognizing conversational semantics, which have the following beneficial effects:
[0046] (1) The BERT-Doc2Vec model combination achieves end-to-end semantic encoding and losslessly reduces context information, thereby obtaining document vectors related to both keywords and conversation context. The BERT model preserves word meanings, and the Doc2Vec model integrates word context relationships, so that document vectors are both accurate and abstract.
[0047] (2) The original intent recognition model introduces the textCNN network and LSTM network, which can perform multi-scale modeling of text semantic features, and extract local and global semantic information on the text at the same time; the LSTM network is also introduced on the encoding and decoding ends, which can learn the long-distance dependency information of the text and enhance the context modeling capability; a multi-level superimposed progressive encoding-decoding structure is adopted to form an end-to-end semantic abstract graph, and the text semantics can be deeply optimized; by modeling text semantics from multiple perspectives and delivering semantic abstraction and refined information, a more sensitive and accurate perception of text semantics can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:
[0049] Figure 1 is a flow chart of a conversation semantics recognition method provided by an embodiment of the present application;
[0050] Figure 2 is a schematic diagram of the structure of an intent recognition model provided by an embodiment of the present application;
[0051] Figure 3 It is a schematic diagram of the structure of the encoding network and the decoding network provided by an embodiment of the present application;
[0052] Figure 4 It is a structural diagram of a conversation semantic recognition device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.
[0055] It should be understood that although the terms first, second, third, etc. may be used to describe the acquisition modules in the embodiments of the present invention, the acquisition modules should not be limited to these terms. These terms are only used to distinguish the acquisition modules from each other.
[0056] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.
[0057] It should be noted that the directional words such as "upper", "lower", "left", and "right" described in the embodiments of the present invention are described at the angles shown in the drawings and should not be understood as limiting the embodiments of the present invention. In addition, in the context, it should also be understood that when it is mentioned that an element is formed "on" or "under" another element, it can not only be formed directly "on" or "under" another element, but also be formed "on" or "under" another element indirectly through an intermediate element.
[0058] See also Figure 1 An embodiment of the present invention provides a conversation semantics recognition method, which is described below using a telephone return visit scenario of an Internet hospital as an example, and includes the following steps:
[0059] Step S101: convert the acquired conversation audio data into conversation text data, and identify conversation keywords from the conversation text data.
[0060] First, obtain the telephone follow-up data of Internet hospitals. The purpose of this step is to collect the original voice interaction text data as training and testing samples for the model, clean the text data, and build various word libraries required for model training.
[0061] Specifically, the telephone callback audio data is converted into conversation text data, and then a keyword thesaurus, a stop word thesaurus, a proprietary noun thesaurus and a banned word thesaurus are established based on the conversation text data, and a word segmentation tool is used to extract effective information from the text to obtain conversation keywords.
[0062] Among them, a dedicated noun database is established. In the conversations of the Internet hospital's telephone follow-up, special medical nouns appear. Special nouns may be misclassified based on the basic vocabulary, and a dedicated vocabulary needs to be established to solve the problem. For example, "compensatory growth", "carpal epiphyseal closure", "follicle-stimulating hormone", "luteinizing hormone" and other dedicated nouns.
[0063] Establish a stop word database. Use the existing conventional stop word database to add user-related information and sensitive information that needs to be filtered based on the requirements of the Internet hospital's telephone follow-up. For example: "telephone number", "age", "address", etc.
[0064] Establish a banned word database. Use the existing banned word database to add special banned words based on the requirements of the Internet hospital telephone follow-up. For example: "fortune-telling", "superstition", "feng shui", "ritual", etc.
[0065] The word segmentation tool used can be based on calculating the word frequency and HMM algorithm in the keyword Trie Tree dictionary to obtain the score of each word and adjacent words, and then extract the text information according to the DAG sequence.
[0066] You can add keywords based on the needs of the business scenario, but you need to use the word frequency to perform word segmentation, for example:
[0067] The conversation was “So right now our main concern is really not how many new patients we treat,…”.
[0068] The word segmentation results are: ['now', 'most', 'mainly', 'really', 'new', 'patient'...].
[0069] The target outcome is: “new patients”, not “patients”.
[0070] The analysis result is: patients 3932n, where 3932 represents the word frequency and n represents the attribute.
[0071] It is necessary to add the keyword "new patient" and adjust the word frequency of "new patient".
[0072] Word segmentation model expression: The input conversation sequence is X = {x1, x2, ..., x n}, the word segmentation result is y={y 11 ,..y 1m ,y2,…,y 2m ,…y n1 ,…y nm},y 11 is the first word extracted from the first group of conversations, the HMM-based word segmentation formula is:
[0073] p(y|x)=Πp(y i |y i-1 ,xi )
[0074] The formula for calculating word frequency is:
[0075]
[0076] in Keywords provided for operations, is the frequency of keywords in the Trie Tree dictionary, b∈Z + The default value is 35, where i∈{1,2,3…}, j∈{1,2,…i-1,i+1,..n}, and the word segmentation structure is modified according to the probability and word frequency of HMM;
[0077] Perform word segmentation on the text data for telephone follow-up data:
[0078] k i ∈R
[0079] where k i is the result after word segmentation i∈{1,2,3…}, k i is a real domain R of vectors.
[0080] Step S102: input the conversation keywords and conversation sequences into the BERT model to obtain the word vectors of the conversation keywords and the conversation text vectors.
[0081] In this step, the word segmentation results and conversation sequences are input into the BERT model. The BERT model is used to extract the word vectors of conversation keywords and the conversation text vectors.
[0082] First, the BERT model is used to vectorize the extracted keywords:
[0083] K={k1,k2,…,k m}
[0084]
[0085] where k i are the keywords extracted from the conversation. is the keyword vector, d is the vector dimension.
[0086] Secondly, the BERT model is also used to implement conversation vectorization:
[0087] X={x1,x2,…,x n}
[0088]
[0089] where x i For the session, is the session vector, d is the vector dimension. BERT uses the Transformer encoding structure to obtain the feature vector h of the text x, H = BERT Transformer (x), where the BERT model obtains the contextual representation of the text by using the Transformer structure.
[0090] The Transformer encoding structure is represented by a multi-head attention mechanism and a feed-forward full link:
[0091] (1) Multi-head attention mechanism:
[0092] Attention(Q,K,V)=Concat(head1,head2,…head n ) o
[0093] in: represents the mapping matrix of the i-th attention head, Q, K, V represent Query, Key, and Value vectors.
[0094] (2) Feedforward full link:
[0095] FFN(x)=max(0,xW1+b1)W2+b2
[0096] where w 0 ,w1,w2,b1,b2 represent the parameters of the fully connected layer.
[0097] Step S103, constructing a document according to the word vector of the conversation keyword and the conversation text vector, extracting effective information of the constructed document using the Doc2vec model, and generating a document vector related to both the keyword and the conversation context.
[0098] Specifically, a document is constructed based on the keyword vector and the conversation text vector obtained in step S102, and then the constructed document is subjected to the Doc2vec model to extract effective information:
[0099]
[0100] in, For the built documentation, The keyword vector extracted for the first session, is the first session vector;
[0101]
[0102] Where i represents the number of sessions i∈{1,2,3…n}, j represents the keyword j∈{1,2,3…m}, represents the Doc2vec vector generated by the jth keyword in the i-th conversation, Each combination is independently input into the Doc2Vec model to generate a vector related to both keywords and conversation context.
[0103] The above method is able to understand the contextual importance of keywords in a document and is particularly effective in extracting document-specific representations of words.
[0104] In summary, the BERT-Doc2Vec model achieves end-to-end semantic encoding and lossless reduction of context information. The BERT model preserves word meanings, and the Doc2Vec model integrates word context relationships. The constructed word_doc document vector is both accurate and abstract.
[0105] Step S104, inputting the document vector into the intent recognition model to obtain an output conversation recognition result.
[0106] Specifically, the word_doc document vector constructed in step S103 is input into the intent recognition model 100 .
[0107] See also Figure 2 . The intent recognition model 100 includes an encoding network 101, a first textCNN-LSTM network layer 102, a decoding network 103 and a recognition network 104. Among them, the first textCNN-LSTM network layer 102 is a network layer composed of a textCNN model and an LSTM model connected in sequence. The intent recognition model 100 of the present invention introduces a textCNN text convolutional network and an LSTM network, which can perform multi-scale modeling of text semantic features, and local and global semantic information on the text can be extracted at the same time. At the same time, the LSTM layer is introduced at both the encoding and decoding ends, which can learn the long-distance dependency information of the text and enhance the context modeling capability.
[0108] See also Figure 3, the encoding network 101 includes a first Transformer encoder network layer 1011, a second textCNN-LSTM network layer 1012 and a second Transformer encoder network layer 1013 connected in sequence; wherein the first Transformer encoder network layer 1011 receives the input word_doc document vector, and the first Transformer encoder network layer 1011 is simultaneously jump-connected to the second Transformer encoder network layer 1013. wherein the second textCNN-LSTM network layer 1012 is a network layer composed of a textCNN model and an LSTM model connected in sequence. the encoding network of the present invention uses a multi-level superimposed progressive encoding structure, that is, the encoder is a multi-layer superimposed structure of more than two layers, to progressively abstract and optimize the text semantics, forming an end-to-end semantic abstract graph, and the text semantics can be deeply optimized.
[0109] See also Figure 3 , the first Transformer encoder network layer 1011 and the second Transformer encoder network layer 1013 each include a first self-attention module 1014, a first residual and normalization module 1015, a first feedforward network 1016, and a second residual and normalization module 1017 connected in sequence. Moreover, the output end of the second residual and normalization module 1017 of the first Transformer encoder network layer 1011 is jump-connected to the first self-attention module 1014 of the second Transformer encoder network layer 1013.
[0110] Furthermore, the second textCNN-LSTM network layer 1012 between the first Transformer encoder network layer 1011 and the second Transformer encoder network layer 1013 is used to extract speech information and context dependencies.
[0111] (1) The textCNN network module in the second textCNN-LSTM network layer 1012:
[0112] Where E is the encoder output (EncoderOutput):
[0113] C=CNN(E)
[0114] Where: C is the output of the CNN layer, and CNN is a collection of convolution operations, including convolution, nonlinear activation functions, and pooling operations.
[0115] (2) LSTM network module in the second textCNN-LSTM network layer 1012:
[0116] L,(h n,c n )=LSTM(C)
[0117] Where L is the output of the LSTM layer, (h n ,c n ) represents the final hidden state and cell state of the LSTM layer, and LSTM is the recurrent operation of the LSTM network module.
[0118] The specific implementation is expressed by the following formula:
[0119] (1) textCNN part:
[0120] C k =pooling(ReLU(E*K k +b k ))
[0121] Among them, “*” is the convolution operation, K k is the kth convolution kernel, b k is the bias, pooling is the pooling operation, and ReLU is the activation function.
[0122] C=|C1;C2;..;C k |
[0123] Among them, C represents the connection of multiple convolution outputs, and “;” represents the connection operation.
[0124] (2) LSTM part:
[0125] (h t ,c t )=LSTM(C t ,h t-1 ,c t-1 )
[0126] Among them, C t is the input at time step t, (h n ,c n ) represents the final hidden state and cell state of the LSTM layer.
[0127] The final LSTM result is:
[0128] L=|h1,h2,…,h T |
[0129] See also Figure 3, the decoding network 103 includes a first Transformer decoder network layer 1031, a third textCNN-LSTM network layer 1032 and a second Transformer decoder network layer 1033 connected in sequence; wherein the first Transformer decoder network layer 1031 is also jump-connected to the second Transformer decoder network layer 1033. Among them, the third textCNN-LSTM network layer 1032 is a network layer composed of a textCNN model and an LSTM model connected in sequence. The decoding network of the present invention uses a multi-level superimposed progressive decoding structure, that is, the decoder is a multi-layer superimposed structure of more than two layers, to gradually abstract and optimize the text semantics, forming an end-to-end semantic abstraction graph, and the text semantics can be deeply optimized.
[0130] Specifically, the first Transformer decoder network layer 1031 is connected to the third textCNN-LSTM network layer 1032 for semantic decoding, which is expressed by the following formula:
[0131] (1) textCNN layer: where D is the decoder output:
[0132] C=CNN(D)
[0133] Where: C is the output of the CNN layer, and CNN is a collection of convolution operations, including convolution, nonlinear activation functions, and pooling operations.
[0134] (2) LSTM layer:
[0135] L,(h n ,c n )=LSTM(C)
[0136] Where L is the output of the LSTM layer, (h n ,c n ) represents the final hidden state and cell state of the LSTM layer, and LSTM is the recurrent operation of the LSTM layer.
[0137] See also Figure 3 , the second Transformer encoder network layer 1013 in the encoding network 101 is directly jump-connected to the first Transformer decoder network layer 1031 in the decoding network 103, and at the same time, the second Transformer encoder network layer 1013 is connected to the first Transformer decoder network layer 1031 through the first textCNN-LSTM network layer 102.
[0138] See also Figure 3The first Transformer decoder network layer 1031 and the second Transformer decoder network layer 1033 both include a second self-attention module 1034, a third residual and normalization module 1035, a third self-attention module 1036, a fourth residual and normalization module 1037, a second feedforward network 1038, and a fifth residual and normalization module 1039, which are connected in sequence.
[0139] See also Figure 3 , the output of the second residual and normalization module 1017 of the second Transformer encoder network layer 1013 is jump-connected to the third self-attention module 1036 of the first Transformer decoder network layer 1031.
[0140] See also Figure 3 , the output of the fifth residual and normalization module 1039 of the first Transformer decoder network layer 1031 is jump connected to the third self-attention module 1036 of the second Transformer decoder network layer 1033.
[0141] For further information, see Figure 3 The recognition network 104 includes a Softmnax classification layer 1041, and the output of the second Transformer decoder network layer 1033 in the decoding network 103 is connected to the Softmnax classification layer 1041, and the Softmnax classification layer 1041 is used to output the final recognition result.
[0142] See the following formula for details:
[0143] I H =softmax(D′ H )
[0144] Among them, D' H is the input, I H To predict intention.
[0145] In summary, the intention recognition model 100 of the present invention introduces the textCNN text convolutional network and the LSTM network, which can perform multi-scale modeling of text semantic features, and the local and global semantic information on the text can be extracted at the same time; at the same time, the LSTM layer is introduced at the encoding-decoding end, which can learn the long-distance dependency information of the text and enhance the context modeling ability; through the multi-level superimposed progressive encoding-decoding structure, that is, the encoder and decoder are a multi-layer superposition structure of more than two layers, the text semantics is gradually abstracted and optimized to form an end-to-end semantic abstract graph, and the text semantics can be deeply optimized; through multi-perspective modeling of text semantics, that is, the integration of multiple network structures such as textCNN and LSTM, the text semantics is modeled from different "perspectives", and the semantic abstraction and refinement information are transmitted, and finally a more acute and accurate perception of the text semantics is achieved. Compared with a single structure, the intention recognition model of the present invention can learn semantic features more deeply and improve the text expression ability.
[0146] Train the model according to the above description until the model reaches the online standard, stop training the model, and obtain the model. Load the trained model and evaluate it in the test set TestSet.
[0147] This paper takes the operation annotation semantic test set TestSet as an example and uses classification accuracy as the evaluation indicator, which is defined as: for a given test data set, the ratio of the number of samples correctly classified by the classifier to the total number of samples. The larger the classification accuracy value, the better the effect of the model, and the lower the value, the worse the effect of the model. See the following table for details:
[0148]
[0149] No. 1: Semantic recognition based on transformer model.
[0150] No. 2: Semantic recognition using the intent recognition model of the present invention.
[0151] From the comparison results in Table 1, we can see that the semantic recognition algorithm proposed in No. 2 is 95.90%-84.91%=10.99% more accurate than the semantic recognition based on the transformer model in No. 1 on the operation annotation semantic dataset TestSet. Therefore, the semantic recognition algorithm with multi-feature fusion proposed in the present invention can achieve higher accuracy and more accurate intent recognition under the same conditions.
[0152] See also Figure 4Another embodiment of the present invention further provides a conversation semantics recognition device 200, including a keyword acquisition module 201, a vectorization module 202, a document vector construction module 203, and a conversation recognition module 204. The conversation semantics recognition device 200 can execute the data communication method in the method embodiment.
[0153] Specifically, the conversation semantics recognition device 200 includes:
[0154] The keyword acquisition module 201 is configured to convert the acquired conversation audio data into conversation text data, and identify conversation keywords from the conversation text data;
[0155] The vectorization module 202 is configured to input the conversation keywords and the conversation sequence into the BERT model to obtain the word vectors of the conversation keywords and the conversation text vectors;
[0156] A document vector construction module 203 is configured to construct a document according to the word vector of the conversation keyword and the conversation text vector, extract effective information of the constructed document using a Doc2vec model, and generate a document vector related to both the keyword and the conversation context;
[0157] The conversation identification module 204 is configured to input the document vector into the intent identification model to obtain an output conversation identification result.
[0158] Furthermore, the intent recognition model includes an encoding network, a first textCNN-LSTM network layer, a decoding network and a recognition network;
[0159] The encoding network includes a first Transformer encoder network layer, a second textCNN-LSTM network layer and a second Transformer encoder network layer connected in sequence; wherein the first Transformer encoder network layer receives an input document vector, and the first Transformer encoder network layer is simultaneously jump-connected to the second Transformer encoder network layer;
[0160] The decoding network includes a first Transformer decoder network layer, a third textCNN-LSTM network layer, and a second Transformer decoder network layer connected in sequence; wherein the first Transformer decoder network layer is also jump-connected to the second Transformer decoder network layer;
[0161] The second Transformer encoder network layer in the encoding network is directly jump-connected to the first Transformer decoder network layer in the decoding network, and the second Transformer encoder network layer is connected to the first Transformer decoder network layer through the first textCNN-LSTM network layer;
[0162] The first textCNN-LSTM network layer, the second textCNN-LSTM network layer and the third textCNN-LSTM network layer all include textCNN modules and LSTM modules connected in sequence;
[0163] The recognition network includes a Softmnax classification layer, the output of the second Transformer decoder network layer in the decoding network is connected to the Softmnax classification layer, and the Softmnax classification layer is used to output the final recognition result.
[0164] Furthermore, the keyword acquisition module 201 is further configured as follows:
[0165] Establishing a word library according to the conversation text data, the word library includes a keyword word library, a stop word word library, a dedicated noun word library and a prohibited word word library;
[0166] The effective information of the conversation text is extracted using a word segmentation tool according to the word library to obtain a word segmentation result.
[0167] Furthermore, the first Transformer encoder network layer and the second Transformer encoder network layer each include a first self-attention module, a first residual and normalization module, a first feedforward network, and a second residual and normalization module connected in sequence;
[0168] The output of the second residual and normalization module of the first Transformer encoder network layer is jump-connected to the first self-attention module of the second Transformer encoder network layer;
[0169] The first Transformer decoder network layer and the second Transformer decoder network layer each include a second self-attention module, a third residual and normalization module, a third self-attention module, a fourth residual and normalization module, a second feedforward network, and a fifth residual and normalization module connected in sequence;
[0170] The output of the second residual and normalization module of the second Transformer encoder network layer is jump-connected to the third self-attention module of the first Transformer decoder network layer;
[0171] The output of the fifth residual and normalization module of the first Transformer decoder network layer is jump-connected to the third self-attention module of the second Transformer decoder network layer.
[0172] It should be noted that the conversation semantics recognition device 200 provided in this embodiment corresponds to a technical solution that can be used to execute each method embodiment, and its implementation principle and technical effect are similar to the method, which will not be repeated here.
[0173] The above description is only a preferred embodiment of the present invention. Those skilled in the art should understand that the disclosure scope involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) by each other to form a technical solution.
Claims
1. A method for identifying conversational semantics, It is characterized in that include: Converting the acquired conversation audio data into conversation text data, and identifying conversation keywords from the conversation text data; Input the conversation keywords and conversation sequences into the BERT model to obtain the word vectors of the conversation keywords and the conversation text vectors; Constructing a document based on the word vector of the conversation keyword and the conversation text vector, extracting effective information of the constructed document using the Doc2vec model, and generating a document vector related to both the keyword and the conversation context; Inputting the document vector into the intent recognition model to obtain an output session recognition result; The intent recognition model includes an encoding network, a first textCNN-LSTM network layer, a decoding network and a recognition network; The encoding network includes a first Transformer encoder network layer, a second textCNN-LSTM network layer and a second Transformer encoder network layer connected in sequence; wherein the first Transformer encoder network layer receives an input document vector, and the first Transformer encoder network layer is simultaneously jump-connected to the second Transformer encoder network layer; The decoding network includes a first Transformer decoder network layer, a third textCNN-LSTM network layer, and a second Transformer decoder network layer connected in sequence; wherein the first Transformer decoder network layer is also jump-connected to the second Transformer decoder network layer; The second Transformer encoder network layer in the encoding network is directly jump-connected to the first Transformer decoder network layer in the decoding network, and the second Transformer encoder network layer is connected to the first Transformer decoder network layer through the first textCNN-LSTM network layer; The first textCNN-LSTM network layer, the second textCNN-LSTM network layer and the third textCNN-LSTM network layer all include textCNN modules and LSTM modules connected in sequence; The recognition network includes a Softmnax classification layer, the output of the second Transformer decoder network layer in the decoding network is connected to the Softmnax classification layer, and the Softmnax classification layer is used to output the final recognition result.
2. A method for identifying conversational semantics according to claim 1, It is characterized in that The step of identifying conversation keywords from the conversation text data further includes: Establishing a word library according to the conversation text data, the word library includes a keyword word library, a stop word word library, a dedicated noun word library and a prohibited word word library; The effective information of the conversation text is extracted using a word segmentation tool according to the word library to obtain a word segmentation result.
3. A conversation semantics recognition method according to claim 1, It is characterized in that The first Transformer encoder network layer and the second Transformer encoder network layer each include a first self-attention module, a first residual and normalization module, a first feedforward network, and a second residual and normalization module connected in sequence; The output of the second residual and normalization module of the first Transformer encoder network layer is jump-connected to the first self-attention module of the second Transformer encoder network layer.
4. A conversation semantics recognition method according to claim 3, It is characterized in that The first Transformer decoder network layer and the second Transformer decoder network layer each include a second self-attention module, a third residual and normalization module, a third self-attention module, a fourth residual and normalization module, a second feedforward network, and a fifth residual and normalization module connected in sequence; The output of the second residual and normalization module of the second Transformer encoder network layer is jump-connected to the third self-attention module of the first Transformer decoder network layer.
5. A conversation semantics recognition method according to claim 4, It is characterized in that The output of the fifth residual and normalization module of the first Transformer decoder network layer is jump-connected to the third self-attention module of the second Transformer decoder network layer.
6. A conversation semantics recognition device, It is characterized in that include: A keyword acquisition module, configured to convert the acquired conversation audio data into conversation text data, and identify conversation keywords from the conversation text data; A vectorization module is configured to input the conversation keywords and the conversation sequence into the BERT model to obtain the word vectors of the conversation keywords and the conversation text vectors; A document vector construction module is configured to construct a document according to the word vector of the conversation keyword and the conversation text vector, extract effective information of the constructed document using a Doc2vec model, and generate a document vector related to both the keyword and the conversation context; A conversation recognition module is configured to input the document vector into an intent recognition model to obtain an output conversation recognition result; The intent recognition model includes an encoding network, a first textCNN-LSTM network layer, a decoding network and a recognition network; The encoding network includes a first Transformer encoder network layer, a second textCNN-LSTM network layer and a second Transformer encoder network layer connected in sequence; wherein the first Transformer encoder network layer receives an input document vector, and the first Transformer encoder network layer is simultaneously jump-connected to the second Transformer encoder network layer; The decoding network includes a first Transformer decoder network layer, a third textCNN-LSTM network layer, and a second Transformer decoder network layer connected in sequence; wherein the first Transformer decoder network layer is also jump-connected to the second Transformer decoder network layer; The second Transformer encoder network layer in the encoding network is directly jump-connected to the first Transformer decoder network layer in the decoding network, and the second Transformer encoder network layer is connected to the first Transformer decoder network layer through the first textCNN-LSTM network layer; The first textCNN-LSTM network layer, the second textCNN-LSTM network layer and the third textCNN-LSTM network layer all include textCNN modules and LSTM modules connected in sequence; The recognition network includes a Softmnax classification layer, the output of the second Transformer decoder network layer in the decoding network is connected to the Softmnax classification layer, and the Softmnax classification layer is used to output the final recognition result.
7. A conversation semantics recognition device according to claim 6, It is characterized in that The keyword acquisition module is further configured as follows: Establishing a word library according to the conversation text data, the word library includes a keyword word library, a stop word word library, a dedicated noun word library and a prohibited word word library; The effective information of the conversation text is extracted using a word segmentation tool according to the word library to obtain a word segmentation result.
8. A conversation semantics recognition device according to claim 6, Features: The first Transformer encoder network layer and the second Transformer encoder network layer each include a first self-attention module, a first residual and normalization module, a first feedforward network, and a second residual and normalization module connected in sequence; The output of the second residual and normalization module of the first Transformer encoder network layer is jump-connected to the first self-attention module of the second Transformer encoder network layer; The first Transformer decoder network layer and the second Transformer decoder network layer each include a second self-attention module, a third residual and normalization module, a third self-attention module, a fourth residual and normalization module, a second feedforward network, and a fifth residual and normalization module connected in sequence; The output of the second residual and normalization module of the second Transformer encoder network layer is jump-connected to the third self-attention module of the first Transformer decoder network layer; The output of the fifth residual and normalization module of the first Transformer decoder network layer is jump-connected to the third self-attention module of the second Transformer decoder network layer.
