Intent recognition and slot filling method, device, readable storage medium and terminal device
By performing joint processing of segmentation of the target text and preset neural network model, the problem of low accuracy of intention recognition and slot filling in the prior art is solved, and a higher accuracy is achieved.
Patent Information
- Application Number
- CN202011022747.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-25
- Publication Date
- 2025-05-02
- Estimated Expiration
- 2040-09-25
AI Technical Summary
The prior art has low accuracy in intention identification and slot filling.
By obtaining the target text, segmenting process is performed to obtain the symbol sequence, and a combination of intent recognition and slot filling is performed using a preset neural network model (such as BERT).
Through the joint processing of intention recognition and slot filling, the accuracy of the final result is greatly improved, and the correlation between intention recognition and slot filling is fully utilized.
Smart Images
Figure CN111966811B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of natural language processing technology, and in particular, relates to an intent recognition and slot filling method, apparatus, computer-readable storage medium, and terminal device. Background Art
[0002] Intent recognition and slot filling are very important tasks in the field of natural language processing (especially in the field of intelligent question answering), and are the key to the quality of intelligent question answering systems. Intent recognition can help intelligent question answering systems understand users, and slot filling can extract and refine the key information required by intelligent question answering systems to meet user intentions. In the existing technology, CNN, RNN, LSTM, etc. are generally used for intent recognition, and Bi-LSTM+CRF, etc. are used for slot filling, with low accuracy. Summary of the invention
[0003] In view of this, the embodiments of the present application provide a method, apparatus, computer-readable storage medium and terminal device for intent recognition and slot filling to solve the problem of low accuracy in the prior art for intent recognition and slot filling.
[0004] A first aspect of an embodiment of the present application provides an intent recognition and slot filling method, which may include:
[0005] Get the target text to be processed;
[0006] Segmenting the target text to obtain a symbol sequence of the target text;
[0007] A preset neural network model is used to perform joint processing of intent recognition and slot filling on the symbol sequence to obtain an intent recognition result and a slot filling result of the target text.
[0008] A second aspect of an embodiment of the present application provides an intent recognition and slot filling device, which may include:
[0009] A text acquisition module is used to acquire the target text to be processed;
[0010] A text segmentation module, used to segment the target text to obtain a symbol sequence of the target text;
[0011] The joint processing module is used to use a preset neural network model to perform intention recognition and slot filling joint processing on the symbol sequence to obtain the intention recognition result and slot filling result of the target text.
[0012] A third aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the following steps are implemented:
[0013] Get the target text to be processed;
[0014] Segmenting the target text to obtain a symbol sequence of the target text;
[0015] A preset neural network model is used to perform joint processing of intent recognition and slot filling on the symbol sequence to obtain an intent recognition result and a slot filling result of the target text.
[0016] A fourth aspect of an embodiment of the present application provides a terminal device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer-readable instructions:
[0017] Get the target text to be processed;
[0018] Segmenting the target text to obtain a symbol sequence of the target text;
[0019] A preset neural network model is used to perform joint processing of intent recognition and slot filling on the symbol sequence to obtain an intent recognition result and a slot filling result of the target text.
[0020] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the embodiments of the present application obtain the target text to be processed; segment the target text to obtain the symbol sequence of the target text; use a preset neural network model to jointly process the symbol sequence with intent recognition and slot filling to obtain the intent recognition result and slot filling result of the target text. Compared with the prior art that completes intent recognition and slot filling as two independent tasks, the embodiments of the present application make full use of the relationship between intent recognition and slot filling, that is, intent recognition has a strong influence on the effect of slot filling, and vice versa. By jointly processing intent recognition and slot filling, the accuracy of the final result is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0022] Figure 1 This is a flow chart of an embodiment of an intent recognition and slot filling method in an embodiment of the present application;
[0023] Figure 2 A schematic flow chart of joint processing of intention recognition and slot filling for the symbol sequence;
[0024] Figure 3 A schematic diagram for generating a superposition sequence corresponding to the symbol sequence;
[0025] Figure 4 Schematic diagram of the attention mechanism;
[0026] Figure 5 This is a structural diagram of an embodiment of an intention recognition and slot filling device in an embodiment of the present application;
[0027] Figure 6 This is a schematic block diagram of a terminal device in an embodiment of the present application. DETAILED DESCRIPTION
[0028] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0029] See also Figure 1 In the embodiments of the present application, an embodiment of an intent recognition and slot filling method may include:
[0030] Step S101: Obtain target text to be processed.
[0031] In a specific implementation of an embodiment of the present application, a user can directly input text through a physical key or a virtual key in a designated human-computer interaction interface and send it to a preset terminal device. After receiving the text input by the user, the terminal device can process the text as the target text.
[0032] In another specific implementation of the embodiment of the present application, the user can also interact with the terminal device in the form of voice. The terminal device collects the user's conversation voice through its own microphone, microphone and other voice collection devices, and text-processes the collected conversation voice to obtain the target text.
[0033] Step S102: segment the target text to obtain a symbol sequence of the target text.
[0034] For the target text, it is first necessary to pre-process it and divide it into individual sentences. In an embodiment of the present application, commas, periods, question marks, exclamation marks, etc. can be used as sentence separators to segment the target text. In order to facilitate subsequent processing, an embodiment of the present application can insert special symbols into the target text to mark the segmentation results. For example, a preset first symbol (the symbol is denoted as [CLS]) can be inserted before the first sentence after segmentation, and a preset second symbol (the symbol is denoted as [SEP]) can be added after each sentence after segmentation, and [CLS] and [SEP] and the characters therebetween are used as a group of token sequences, that is, the symbol sequence of the target text.
[0035] It should be noted that the "character" mentioned in the embodiments of the present application is the smallest unit of natural language processing. Generally, for English text or other alphabetic phonetic text, each word can be regarded as a character, and for Chinese text, each Chinese character can be regarded as a character.
[0036] Step S103: Use a preset neural network model to jointly process the symbol sequence for intent recognition and slot filling to obtain an intent recognition result and a slot filling result for the target text.
[0037] In a specific implementation of the embodiment of the present application, artificial intelligence technology is used, and the neural network model can be a BERT (Bidirectional Encoder Representation from Transformers) model. The model architecture of BERT is based on multi-layer bidirectional conversion decoding. Because the decoder cannot obtain the information to be predicted, the main innovations of the model are in the pre-training method, that is, the Masked LM and Next SentencePrediction methods are used to capture the expressions at the word and sentence levels respectively. Among them, "bidirectional" means that when the model processes a certain word, it can simultaneously use the two parts of information of the previous word and the following word. The source of this "bidirectional" is that BERT is different from the traditional language model. It does not predict the most likely current word under the condition of giving all the previous words, but randomly masks some words and uses all the unmasked words for prediction. The BERT model is pre-trained in a large amount of unlabeled text using two strategies, one is the masked language model and the other is the next sequence prediction. The pre-trained BERT model provides a powerful sentence representation containing contextual dependencies, which can be used to process a variety of natural language processing (NLP) tasks, such as intent recognition and word slot filling tasks in the embodiments of the present application.
[0038] like Figure 2 As shown, step S103 may specifically include the following steps:
[0039] Step S1031: Generate a superposition sequence corresponding to the symbol sequence.
[0040] Specifically, a word embedding sequence, a segment embedding sequence and a position embedding sequence corresponding to the symbol sequence can be generated respectively, wherein the word embedding sequence includes the word vector of each symbol in the symbol sequence, the segment embedding sequence includes the segment information to which each symbol in the symbol sequence belongs, and the position embedding sequence includes the position information of each symbol in the symbol sequence;
[0041] Then, the word embedding sequence, the segment embedding sequence and the position embedding sequence are superimposed to generate the superimposed sequence.
[0042] For each symbol in the symbol sequence of the target text (including characters and inserted special symbols), there are three embedding forms, namely, token embedding, segment embedding and position embedding. Among them, token embedding represents the word vector of the symbol. The first symbol is [CLS], which can be used for subsequent classification tasks; segment embedding is used to distinguish different sentences. If the target text is composed of multiple sentences, then each sentence has an embedding item of the entire sentence corresponding to each symbol; position embedding represents position information, because the order of words in NLP is a very important feature, and the position information needs to be encoded here; superimposing the three embeddings corresponding to each symbol forms a superimposed sequence corresponding to the symbol sequence.
[0043] like Figure 3As shown in the figure, the symbol sequence has two natural sentences, sentence A [my dog is cute] and sentence B [he likes playing]. First, each symbol and special symbol needs to be converted into a word embedding vector, because neural networks can only perform numerical calculations. Among them, the special symbol [SEP] is a symbol used to separate the two sentences. The first half of the sentence will be added with the segmentation code A, and the second half will be added with the segmentation code B. Because the relationship between sentences needs to be modeled, BERT has a task to predict whether sentence B is the sentence after sentence A, and this classification task will be achieved with the help of the special symbol [CLS] at the front of sentence A / B. This special symbol can be regarded as a collection of representations of the entire symbol sequence. The final position encoding is determined by the encoder architecture itself, because the method based on full attention cannot encode the positional relationship between words like CNN or RNN, but it is precisely because of this property that the relationship between two words can be modeled regardless of the distance. Therefore, in order to make the encoder perceive the positional relationship between words, position encoding is needed here to add position information to each symbol.
[0044] Step S1032: Perform hidden layer processing on the superimposed sequence to obtain a hidden layer state of each symbol in the superimposed sequence.
[0045] In an embodiment of the present application, the superposition sequence is used as the input of BERT, and after the hidden layer processing of BERT, the hidden layer state of each symbol in the superposition sequence can be obtained. Here, the hidden layer state of the first symbol in the superposition sequence, that is, [CLS], can be recorded as h_1, the hidden layer state of the second symbol in the superposition sequence can be recorded as h_2, the hidden layer state of the third symbol in the superposition sequence can be recorded as h_3, ..., the hidden layer state of the last symbol in the superposition sequence can be recorded as h_T, and so on, where T is the total number of symbols in the superposition sequence.
[0046] Step S1033: Generate the intention matrix of the target text according to the hidden layer state of the first symbol in the superposition sequence.
[0047] The hidden layer state of the first symbol in the superimposed sequence is a 768-dimensional vector, as shown below:
[0048] [1,1,1,…]
[0049] The hidden layer state of the first symbol in the superimposed sequence is processed by the fully connected layer of BERT to obtain the output vectors of each fully connected layer, that is, 12 768-dimensional vectors, as shown below:
[0050] [1,1,1,…]
[0051] [1,1,1,…]
[0052] …
[0053] By concatenating the output vectors of each fully connected layer, the intent matrix of the target text can be obtained. The intent matrix is a matrix with 12 rows and 758 columns, each row of which is a fully connected layer output vector. The intent matrix is the information representation of intent classification, as shown below:
[0054] [1,1,1,…
[0055] 1,1,1,…
[0056] …
[0057] 1,1,1,…]
[0058] It should be noted that the above process is described using an all-1 vector and an all-1 matrix as an example. Those skilled in the art should understand that this is just a special example, not a limitation on the specific values of the vector and matrix.
[0059] Step S1034: Use the attention mechanism to process the hidden layer states of the intention matrix and each symbol in the superimposed sequence to obtain the attention matrix of the target text.
[0060] The attention mechanism is a technology that enables the model to focus on and learn important information, and is often used in seq2seq models. In the model encoding stage, the attention mechanism provides the hidden state of all nodes as context to the decoder. In the decoding stage, the decoder adopts a selection mechanism to select the hidden state that best matches the current position. The mechanism includes the following three steps: determine which hidden state is most closely related to the current node; calculate the score value of each hidden state; and perform softmax calculation on each score. Figure 4 As shown in the figure, the essence of the attention mechanism can be described as a mapping from a query to a series of (key-value pairs). First, the query and each key are similarly calculated to obtain the weight, then the weights are normalized by softmax, and finally the weights and the corresponding key values are weighted and summed to obtain the attention value.
[0061] The embodiment of the present application integrates the information of intent classification into the process of slot filling, and focuses on the token sequence through the vector of intent classification, that is, adopts the attention mechanism to enhance the effect of slot filling, and better utilizes the potential information association between intent and slot by representing and jointly training the intent recognition and slot filling in one model architecture.
[0062] Specifically, in the embodiment of the present application, the attention matrix can be calculated according to the following formula:
[0063] H ～ =Attention(Q,K,V)=Attention(H_(2:T),Intent,Intent)
[0064] =softmax(H_(2:T)*Intent^T)*Intent
[0065] Wherein, H_(2:T) is the hidden layer state of each symbol in the superposition sequence except the first symbol, Intent is the intention matrix, softmax is the softmax function, Attention is the attention mechanism processing function, Q is the query parameter in the attention mechanism, and Q=H_(2:T), K is the key parameter in the attention mechanism, and K=Intent, V is the value parameter in the attention mechanism, and V=Intent, H ～ is the attention matrix, and H ～ =[(h_2) ～ ,…,(h_T) ～ ],(h_2) ～ is the attention vector of the second symbol in the superimposed sequence, (h_3) ～ is the attention vector of the third symbol in the superimposed sequence, ..., (h_T) ～ is the attention vector of the Tth symbol in the superimposed sequence, and so on.
[0066] Step S1035: determine the intent recognition result and slot filling result of the target text according to the attention matrix of the target text.
[0067] Specifically, the attention matrix can be input into a softmax function to obtain a classification result, that is, an intent recognition result and a slot filling result of the target text.
[0068] Corresponding to the intent recognition and slot filling joint model architecture proposed in the above process, in the embodiment of the present application, the following objective function can be used to jointly train it:
[0069]
[0070] Wherein, x is a training text, n is a word number in the training text, 1≤n≤N, N is the total number of words in the training text, i is the number of various intent recognition results of the training text, s is the number of various slot filling results of the training text, is the probability that the nth word in the training text is the sth slot filling result, p(yi |x) is the probability that the training text is the i-th intention recognition result, p(y i ,y s |x) is the probability that the training text is the i-th intent recognition result and the s-th slot filling result.
[0071] The ultimate goal of joint training is to maximize p(y i ,y s |x) is fine-tuned end-to-end by minimizing the cross entropy loss to obtain the final model, which can then be used to perform joint analysis of text intent recognition and slot filling.
[0072] Preferably, in order to further ensure the privacy and security of the above-mentioned target text's intent recognition results and slot filling results, the target text's intent recognition results and slot filling results can also be stored in a node of a blockchain.
[0073] It should be noted that the blockchain referred to in the present invention is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the blockchain underlying platform, platform product service layer, and application service layer.
[0074] At the same time, this solution can be applied in the field of smart cities, thereby promoting the construction of smart cities.
[0075] In summary, the embodiment of the present application obtains a target text to be processed; performs segmentation processing on the target text to obtain a symbol sequence of the target text; uses a preset neural network model to jointly process the symbol sequence with intent recognition and slot filling to obtain an intent recognition result and a slot filling result of the target text. Compared to the prior art in which intent recognition and slot filling are completed as two independent tasks, the embodiment of the present application makes full use of the relationship between intent recognition and slot filling, that is, intent recognition has a strong influence on the effect of slot filling, and vice versa. By jointly processing intent recognition and slot filling, the accuracy of the final result is greatly improved.
[0076] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0077] Corresponding to the intent recognition and slot filling method described in the above embodiment, Figure 5 A structural diagram of an embodiment of an intent recognition and slot filling device provided in an embodiment of the present application is shown.
[0078] In this embodiment, an intent recognition and slot filling device may include:
[0079] The text acquisition module 501 is used to acquire the target text to be processed;
[0080] A text segmentation module 502 is used to segment the target text to obtain a symbol sequence of the target text;
[0081] The joint processing module 503 is used to use a preset neural network model to perform intention recognition and slot filling joint processing on the symbol sequence to obtain the intention recognition result and slot filling result of the target text.
[0082] Furthermore, the joint processing module may include:
[0083] A superposition sequence generating unit, used to generate a superposition sequence corresponding to the symbol sequence;
[0084] A hidden layer processing unit, used to perform hidden layer processing on the superimposed sequence to obtain a hidden layer state of each symbol in the superimposed sequence;
[0085] An intention matrix generating unit, used for generating an intention matrix of the target text according to the hidden layer state of the first symbol in the superposition sequence;
[0086] An attention matrix calculation unit, used for processing the hidden layer states of the intention matrix and each symbol in the superposition sequence using an attention mechanism to obtain an attention matrix of the target text;
[0087] A result determination unit is used to determine the intent recognition result and slot filling result of the target text according to the attention matrix of the target text.
[0088] Further, the superposition sequence generating unit may include:
[0089] A sequence generation subunit, configured to respectively generate a word embedding sequence, a segment embedding sequence and a position embedding sequence corresponding to the symbol sequence, wherein the word embedding sequence includes a word vector of each symbol in the symbol sequence, the segment embedding sequence includes segment information to which each symbol in the symbol sequence belongs, and the position embedding sequence includes position information of each symbol in the symbol sequence;
[0090] A sequence superposition subunit is used to superimpose the word embedding sequence, the segment embedding sequence and the position embedding sequence to generate the superimposed sequence.
[0091] Further, the intention matrix generating unit may include:
[0092] A fully connected layer processing subunit, used for performing fully connected layer processing on the hidden layer state of the first symbol in the superposition sequence to obtain each fully connected layer output vector;
[0093] The concatenation processing subunit is used to concatenate the output vectors of each fully connected layer to obtain the intention matrix of the target text.
[0094] Furthermore, the attention matrix calculation unit is specifically used to calculate the attention matrix according to the following formula:
[0095] H ～ =Attention(Q,K,V)=Attention(H_(2:T),Intent,Intent)
[0096] =softmax(H_(2:T)*Intent^T)*Intent
[0097] Wherein, T is the total number of symbols in the superposition sequence, H_(2:T) is the hidden layer state of each symbol in the superposition sequence except the first symbol, Intent is the intention matrix, softmax is the softmax function, Attention is the attention mechanism processing function, Q is the query parameter in the attention mechanism, and Q=H_(2:T), K is the key parameter in the attention mechanism, and K=Intent, V is the value parameter in the attention mechanism, and V=Intent, H ～ is the attention matrix, and H ～ =[(h_2) ～ ,…,(h_T) ～ ],(h_2) ～ is the attention vector of the second symbol in the superimposed sequence, (h_3) ～ is the attention vector of the third symbol in the superimposed sequence, ..., (h_T) ～ is the attention vector of the Tth symbol in the superimposed sequence, and so on.
[0098] Furthermore, the intention identification and slot filling device may also include:
[0099] The model training module is used to train the initial neural network model using a preset objective function to obtain a trained neural network model; the objective function is:
[0100]
[0101] Wherein, x is a training text, n is a word number in the training text, 1≤n≤N, N is the total number of words in the training text, i is the number of various intent recognition results of the training text, s is the number of various slot filling results of the training text, is the probability that the nth word in the training text is the sth slot filling result, p(y i |x) is the probability that the training text is the i-th intention recognition result, p(y i ,y s |x) is the probability that the training text is the i-th intent recognition result and the s-th slot filling result.
[0102] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0103] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0104] Figure 6 A schematic block diagram of a terminal device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.
[0105] In this embodiment, the terminal device 6 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device 6 may include: a processor 60, a memory 61, and computer-readable instructions 62 stored in the memory 61 and executable on the processor 60, such as computer-readable instructions for executing the above-mentioned intent recognition and slot filling method. When the processor 60 executes the computer-readable instructions 62, the steps in the above-mentioned intent recognition and slot filling method embodiments are implemented, such as Figure 1 Alternatively, when the processor 60 executes the computer-readable instructions 62, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 5 Functions of modules 501 to 503 are shown.
[0106] Exemplarily, the computer-readable instruction 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of completing a specific function, which are used to describe the execution process of the computer-readable instruction 62 in the terminal device 6.
[0107] The processor 60 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0108] The memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or memory of the terminal device 6. The memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 6. Further, the memory 61 may also include both an internal storage unit of the terminal device 6 and an external storage device. The memory 61 is used to store the computer-readable instructions and other instructions and data required by the terminal device 6. The memory 61 may also be used to temporarily store data that has been output or is to be output.
[0109] Each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0110] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of computer-readable instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store computer-readable instructions.
[0111] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. An intent recognition and slot filling method, characterized in that: include: Get the target text to be processed; Segmenting the target text to obtain a symbol sequence of the target text; generating a superposition sequence corresponding to the symbol sequence; Performing hidden layer processing on the superimposed sequence to obtain a hidden layer state of each symbol in the superimposed sequence; Generate an intention matrix of the target text according to the hidden layer state of the first symbol in the superposition sequence; The intention matrix and the hidden layer states of each symbol in the superimposed sequence are processed using an attention mechanism to obtain an attention matrix of the target text; wherein the query parameter in the attention mechanism is the hidden layer state of each symbol in the superimposed sequence except the first symbol, and the key parameter and value parameter in the attention mechanism are both the intention matrix; The intent recognition result and slot filling result of the target text are determined according to the attention matrix of the target text.
2. The intent recognition and slot filling method according to claim 1, characterized in that: Generating a superposition sequence corresponding to the symbol sequence comprises: Respectively generate a word embedding sequence, a segment embedding sequence and a position embedding sequence corresponding to the symbol sequence, wherein the word embedding sequence includes the word vector of each symbol in the symbol sequence, the segment embedding sequence includes the segment information to which each symbol in the symbol sequence belongs, and the position embedding sequence includes the position information of each symbol in the symbol sequence; The word embedding sequence, the segment embedding sequence and the position embedding sequence are superimposed to generate the superimposed sequence.
3. The intent recognition and slot filling method according to claim 1, characterized in that: The generating the intention matrix of the target text according to the hidden layer state of the first symbol in the superposition sequence comprises: Performing full connection layer processing on the hidden layer state of the first symbol in the superposition sequence to obtain each full connection layer output vector; The output vectors of each fully connected layer are concatenated to obtain the intent matrix of the target text.
4. The intent recognition and slot filling method according to claim 1, characterized in that: The use of the attention mechanism to process the hidden layer states of the intention matrix and each symbol in the superposition sequence to obtain the attention matrix of the target text includes: The attention matrix is calculated according to the following formula: H ～ =Attention(Q,K,V)=Attention(H_(2:T),Intent,Intent) =softmax(H_(2:T)*Intent^T)*Intent Among them, T is the total number of symbols in the superposition sequence, H_(2:T) is the hidden layer state of each symbol in the superposition sequence except the first symbol, Intent is the intention matrix, softmax is the softmax function, Attention is the attention mechanism processing function, Q is the query parameter in the attention mechanism, and Q = H_(2:T), K is the key parameter in the attention mechanism, and K = Intent, V is the value parameter in the attention mechanism, and V = Intent, H~ is the attention matrix.
5. The method for intention identification and slot filling according to any one of claims 1 to 4, characterized in that: Also includes: The initial neural network model is trained using a preset objective function to obtain a trained neural network model; the objective function is: Where x is the training text, n is the word number in the training text, 1≤n≤N, N is the total number of words in the training text, i is the number of various intent recognition results of the training text, s is the number of various slot filling results of the training text, p(y n s |x) is the probability that the nth word in the training text is the sth slot filling result, p(y i |x) is the probability that the training text is the i-th intention recognition result, p(y i ,y s |x) is the probability that the training text is the i-th intent recognition result and the s-th slot filling result.
6. An intent recognition and slot filling device, characterized in that include: A text acquisition module is used to acquire the target text to be processed; A text segmentation module, used to segment the target text to obtain a symbol sequence of the target text; A joint processing module is used to generate a superposition sequence corresponding to the symbol sequence; perform hidden layer processing on the superposition sequence to obtain the hidden layer state of each symbol in the superposition sequence; and generate the intention matrix of the target text according to the hidden layer state of the first symbol in the superposition sequence; The intention matrix and the hidden layer states of each symbol in the superimposed sequence are processed using an attention mechanism to obtain the attention matrix of the target text; wherein the query parameter in the attention mechanism is the hidden layer state of each symbol in the superimposed sequence except the first symbol, and the key parameter and value parameter in the attention mechanism are both the intention matrix; the intention recognition result and slot filling result of the target text are determined according to the attention matrix of the target text.
7. A computer-readable storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by a processor, the steps of the intent recognition and slot filling method according to any one of claims 1 to 5 are implemented.
8. A terminal device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer-readable instructions, the processor implements the steps of the intent recognition and slot filling method according to any one of claims 1 to 5.