A training method and device of an intent recognition model, and an intent recognition method and device

Abstract

The present disclosure provides a training method and device of an intent recognition model, and an intent recognition method and device, and relates to the technical field of computers, in particular to the technical fields of artificial intelligence, natural language processing, deep learning and the like. The specific implementation scheme is as follows: obtaining a text sequence of each training text in a plurality of training texts; inputting the text sequence of each training text into an encoding layer included in an initial intent recognition model to obtain a target feature of each training text, the target feature of each training text being used to represent an association relationship between each word included in each training text and other words included in the training text; inputting the target feature of each training text into a network layer included in the initial intent recognition model to obtain a prediction result of each training text; updating parameters included in the initial intent recognition model based on a target loss function to obtain a target intent recognition model.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to the fields of artificial intelligence, natural language processing, and deep learning. Specifically, it provides a method for training an intent recognition model, an intent recognition method, and an apparatus. Background Technology

[0002] Currently, an intent recognition model can be built on a server based on a neural network. This intent recognition model can be used to identify the intent in text, specifically whether the text contains an intent. Summary of the Invention

[0003] This disclosure provides a training method, an intent recognition method, and an apparatus for a more accurate intent recognition model.

[0004] According to one aspect of this disclosure, a training method for an intent recognition model is provided. The method includes: acquiring a text sequence for each training text from a plurality of training texts, wherein the text sequence of each training text includes: word vectors for each word included in each training text; inputting the text sequence of each training text into an encoding layer included in an initial intent recognition model to obtain target features for each training text, wherein the target features of each training text are used to characterize the association between each word included in each training text and other words included in each training text; inputting the target features of each training text into a network layer included in the initial intent recognition model to obtain a prediction result for each training text; updating the parameters included in the initial intent recognition model based on a target loss function to obtain a target intent recognition model, wherein the target intent recognition model is used to predict whether a text contains intent, and the target loss function is used to characterize the degree of inconsistency between the prediction result and the actual result of each training text.

[0005] According to another aspect of this disclosure, an intent recognition method is provided. It includes: acquiring a text sequence of a text to be recognized, the text sequence including word vectors for each word in the text; inputting the text sequence into an encoding layer of a target intent recognition model to obtain target features of the text, the target features characterizing the association between each word in the text and other words in the text, the target intent recognition model being used to identify whether a text contains intent; and inputting the target features into a network layer of the target intent recognition model to determine whether the text contains intent.

[0006] According to another aspect of this disclosure, a training apparatus for an intent recognition model is provided. It includes: an acquisition module and a processing module; the acquisition module is configured to acquire a text sequence of each training text from a plurality of training texts, the text sequence of each training text including: word vectors of each word included in each training text; the processing module is configured to input the text sequence of each training text into an encoding layer included in an initial intent recognition model to obtain target features of each training text, the target features of each training text being used to characterize the association relationship between each word included in each training text and other words included in each training text; the processing module is further configured to input the target features of each training text into a network layer included in the initial intent recognition model to obtain a prediction result for each training text; the processing module is further configured to update the parameters included in the initial intent recognition model based on a target loss function to obtain a target intent recognition model, the target intent recognition model being used to predict whether a text contains intent, the target loss function being used to characterize the degree of inconsistency between the prediction result of each training text and the actual result of each training text.

[0007] According to another aspect of this disclosure, an intent recognition device is provided. It includes: an acquisition module, a processing module, and a determination module; the acquisition module is used to acquire a text sequence of text to be recognized, the text sequence of text to be recognized including: word vectors of each word included in the text to be recognized; the processing module is used to input the text sequence of text to be recognized into an encoding layer in a target intent recognition model to obtain target features of the text to be recognized, the target features of the text to be recognized being used to characterize the association relationship between each word included in the text to be recognized and other words included in the text to be recognized, the target intent recognition model being used to identify whether a text contains intent; the determination module is used to input the target features of the text to be recognized into a network layer included in the target intent recognition model to determine whether the text to be recognized contains intent.

[0008] According to another aspect of this disclosure, an electronic device is provided. It includes: at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform a training method for an intent recognition model provided in this disclosure or to perform an intent recognition method provided in this disclosure.

[0009] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to execute a training method for the intent recognition model provided in this disclosure or to execute an intent recognition method provided in this disclosure.

[0010] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the training method of the intent recognition model provided in this disclosure or implements the intent recognition method provided in this disclosure.

[0011] Based on any of the above aspects, in this disclosure, an electronic device can acquire the text sequence of each training text from multiple training texts; then, the electronic device can input the text sequence of each training text into the encoding layer included in the initial intent recognition model to obtain the target features of each training text; and input the target features of each training text into the network layer included in the initial intent recognition model to obtain the prediction result of each training text; subsequently, the electronic device can update the parameters included in the initial intent recognition model based on the target loss function to obtain the target intent recognition model. In this disclosure, since the target features of a training text are used to characterize the association between each word included in the training text and other words included in the training text, the target features of the training text incorporate the semantic information of each word included in the training text, and can accurately characterize the semantic information of the training text. Thus, by updating the parameters included in the initial intent recognition model based on the target loss function (i.e., the degree of inconsistency between the prediction result of each training text in multiple training texts and the actual result of each training text), the electronic device can generate a more accurate target intent recognition model, which can identify whether a long text contains an intent (or key segment) with high accuracy.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0013] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0014] Figure 1 A flowchart illustrating a training method for an intent recognition model provided in an embodiment of this disclosure is shown.

[0015] Figure 2 A flowchart illustrating another training method for an intent recognition model provided in an embodiment of this disclosure is shown.

[0016] Figure 3 A schematic diagram of constructing an interest pool according to an embodiment of this disclosure is shown;

[0017] Figure 4 A schematic diagram illustrating an embodiment of the present disclosure for obtaining cross-entropy loss is shown;

[0018] Figure 5 A flowchart illustrating an intent recognition method provided in an embodiment of this disclosure is shown.

[0019] Figure 6 A schematic diagram of the structure of a training device for an intent recognition model provided in this disclosure embodiment is shown;

[0020] Figure 7 A schematic diagram of the structure of an intent recognition device provided in this disclosure embodiment is shown below;

[0021] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0022] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0023] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0024] It should also be understood that the term "comprising" indicates the presence of the described feature, whole, step, operation, element and / or component, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements and / or components.

[0025] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0026] In related technologies, an intent recognition model can be built on a server based on a neural network. Based on this intent recognition model, intent recognition can be performed on text, specifically to identify whether the text contains intent.

[0027] Currently, employees may encounter various work challenges. Due to insufficient experience, ability, or limited resources, they may be unable to obtain relevant solutions in a short time, which could affect the orderly progress of work. Therefore, identifying the problems employees encounter at work and efficiently providing them with solutions is an important way to improve employee work efficiency.

[0028] Text is an important medium for employees to record their work content. For example, weekly work reports usually record the content of employees' current work and the problems that need to be solved. However, weekly work reports contain a lot of content and are quite long. It is challenging and urgent to find the problems encountered by employees in long texts.

[0029] Although related technologies have constructed intent recognition models, and these models can identify whether text contains intent (which can also be understood as whether the text contains a problem that employees need to solve), these intent recognition models are based on traditional neural networks such as long short-term memory (LSTM) and convolutional neural networks (CNN). These traditional neural networks have limited text representation capabilities and may not accurately represent the semantics of the text (including each word within the text), thus failing to accurately identify whether the text contains intent.

[0030] Based on this, embodiments of this disclosure provide a training method for an intent recognition model. In this method, since the target features of a training text are used to characterize the association between each word included in the training text and other words included in the training text, the target features of the training text incorporate the semantic information of each word included in the training text, thus accurately representing the semantic information of the training text. In this way, the electronic device updates the parameters included in the initial intent recognition model based on the target loss function (i.e., the degree of inconsistency between the prediction result of each training text and the actual result of each training text), thereby generating a more accurate target intent recognition model. This target intent recognition model can identify whether a long text contains an intent (or key segment) with high accuracy.

[0031] The technical solutions provided in this disclosure can be applied to intent recognition scenarios, specifically deep learning applications such as search engines and dialogue systems. For example, when this technical solution is applied to a search engine scenario, the electronic device can input text into an intent recognition model. If the intent recognition model determines that the text contains an intent, the electronic device can use the intent (which can also be understood as keywords) contained in the text as the keywords to be searched, thereby retrieving content related to those keywords.

[0032] For example, the electronic device implementing the technical solutions provided in the embodiments of this disclosure can be various Internet of Things (IoT) devices such as mobile phones, smart cars, robot vacuum cleaners, and televisions. The electronic device can also be a tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, etc. This disclosure does not impose any special limitations on the specific form of the electronic device. It can interact with the user through one or more methods such as a keyboard, touchpad, touchscreen, remote control, voice interaction, or handwriting devices.

[0033] Optionally, the electronic device can also be a server, and this disclosure does not limit the scope of the embodiments. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN) services, and big data and artificial intelligence platforms.

[0034] The technical solutions provided by the embodiments of this disclosure are described in detail below with reference to the accompanying drawings.

[0035] like Figure 1 As shown, the training method for the intent recognition model provided in this embodiment may include S101-S104.

[0036] S101. The electronic device acquires the text sequence of each training text from multiple training texts.

[0037] The text sequence of each training text includes: the word vector of each word included in each training text.

[0038] It should be understood that, for any one of the multiple training texts mentioned above, the electronic device can first perform word segmentation on the training text to obtain at least one word included in the training text. Then, the electronic device can map this at least one word into a vector, thus obtaining the text sequence of the training text. This text sequence can also be understood as the initial representation of the training text.

[0039] S102. The electronic device inputs the text sequence of each training text into the encoding layer included in the initial intent recognition model to obtain the target features of each training text.

[0040] The target features of each training text are used to characterize the relationship between each word included in each training text and the other words included in each training text.

[0041] It should be understood that the target features of each training text incorporate the semantic information of each word included in each training text, and can accurately represent the semantic information of each training text.

[0042] S103. The electronic device inputs the target features of each training text into the network layers included in the initial intent recognition model to obtain the prediction results for each training text.

[0043] It should be understood that the network layers included in the initial intent recognition model can also be understood as the output layers of the initial intent recognition model, and these network layers can be fully connected networks. Through this fully connected network, the target features of each training text can be categorized into the category space to predict which category each training text belongs to.

[0044] In this embodiment of the disclosure, the training texts are categorized into two types: those containing intent and those not containing intent. That is, the prediction result for each training text is either containing intent or not containing intent.

[0045] S104. The electronic device updates the parameters included in the initial intent recognition model based on the target loss function to obtain the target intent recognition model.

[0046] The target intent recognition model is used to predict whether a text contains intent, and the target loss function is used to characterize the degree of inconsistency between the prediction result of each training text and the actual result of each training text.

[0047] It should be understood that the actual results for each training text can be pre-stored in the electronic device.

[0048] It is understandable that the electronic device updates the parameters included in the initial intent recognition model based on the target loss function by backpropagating the target loss function to update the parameters included in the initial intent recognition model.

[0049] In this embodiment of the disclosure, updating the parameters included in the initial intent recognition model of the electronic device may specifically include updating the parameters in the coding layer and the parameters in the network layer.

[0050] The technical solution provided by the above embodiments can bring at least the following beneficial effects: As shown in S101-S104, the electronic device can acquire the text sequence of each training text in multiple training texts; then, the electronic device can input the text sequence of each training text into the encoding layer included in the initial intent recognition model to obtain the target features of each training text; and input the target features of each training text into the network layer included in the initial intent recognition model to obtain the prediction result of each training text; afterwards, the electronic device can update the parameters included in the initial intent recognition model based on the target loss function to obtain the target intent recognition model. In this embodiment of the present disclosure, since the target features of a training text are used to characterize the association relationship between each word included in the training text and other words included in the training text, the target features of the training text integrate the semantic information of each word included in the training text, and can accurately characterize the semantic information of the training text. In this way, the electronic device updates the parameters included in the initial intent recognition model based on the target loss function (i.e., the degree of inconsistency between the prediction result of each training text in multiple training texts and the actual result of each training text), and can generate a more accurate target intent recognition model, which can identify whether long texts contain intent (or key segments) with high accuracy.

[0051] Since the training method for the intent recognition model is supervised, the first step in training the model is to mine training data. Analysis of a large amount of user text reveals that text with intent often contains the user's points of interest. For example, in the text "What are some deep learning frameworks?", "deep learning frameworks" represents the user's point of interest. In this embodiment, the electronic device can identify text fragments containing user points of interest as intent fragments.

[0052] In addition, the most direct way to build training data is through manual annotation, but manual annotation requires high manpower costs, and users have a wide range of interests, making it difficult to accurately and effectively annotate data manually.

[0053] Based on this, such as Figure 2 As shown, the training method for the intent recognition model provided in this embodiment may include S201-S208.

[0054] S201. The electronic device acquires the text sequence of each training text from multiple training texts.

[0055] S202. The electronic device inputs the text sequence of each training text into the encoding layer included in the initial intent recognition model to obtain the target features of each training text.

[0056] S203. The electronic device inputs the target features of each training text into the network layers included in the initial intent recognition model to obtain the prediction results for each training text.

[0057] It should be noted that the explanations in S201-S203 above are the same as or similar to the descriptions in S101-S103 above, and will not be repeated here.

[0058] S204. Electronic devices determine whether key segments exist in each training text.

[0059] This key segment is used to characterize the segments that the user is interested in.

[0060] Optionally, an interest pool can be constructed in the electronic device, which may include (or store) segments (i.e., key segments) that the user is interested in. These key segments can also be understood as points of interest (or segments of interest) that the user is interested in. Specifically, the types of key segments can be advanced technical terms and product keywords, etc.

[0061] In one alternative implementation, the electronic device can construct an interest pool based on the Open Domain Information Extraction Model (Monet model). For details on constructing the interest pool, please refer to [link to relevant documentation]. Figure 3 As shown in the image.

[0062] First, a small amount of manually labeled data can be used, specifically a small amount of text, with manual annotation to determine whether it contains key segments. Then, the Monet model is trained using this small amount of manually labeled data. After acquiring enterprise text data, the trained Monet model is used to infer whether the enterprise text data contains key segments. The key segments contained in the enterprise text data are then post-processed, and finally stored in an interest pool.

[0063] Alternatively, the above post-processing can be understood as filtering (or deleting) low-quality text fragments, such as names, place names, and phone numbers.

[0064] S205. When a key segment exists in each training text, the electronic device determines that the true result of each training text is an intent.

[0065] It should be understood that when each training text contains a key segment, it indicates that each training text includes points of interest (or segments of interest) that the user is interested in. In this case, the electronic device can determine that each training text contains intent. In this embodiment of the disclosure, the electronic device can label training texts that actually contain intent as positive examples.

[0066] S206. When no key segment exists in each training text, the electronic device determines that the true result of each training text is that it does not contain intent.

[0067] It is understandable that when no key segment exists in each training text, it indicates that each training text does not contain any points of interest (or segments of interest) that the user is interested in. In this case, the electronic device can determine that each training text does not contain any intent. In this embodiment of the present disclosure, the electronic device can label training texts that actually show no intent as negative examples.

[0068] Optionally, when a training text is long, it may be difficult to perform intent recognition or annotation on that text. In this case, the electronic device can decompose the training text, that is, cut the long text into short text segments, in order to complete the intent recognition or annotation process.

[0069] S207. The electronic device determines the target loss function based on the prediction results and the actual results of each training text.

[0070] Alternatively, the electronic device may use a cross-entropy function to determine the target loss function.

[0071] S208. The electronic device updates the parameters included in the initial intent recognition model based on the target loss function to obtain the target intent recognition model.

[0072] In the training method of the intent recognition model provided in the above-described embodiments, the training data can be automatically labeled by keyword matching based on the constructed interest pool, without relying on manual labeling. This allows for accurate and effective labeling of the training data, thereby improving the training efficiency of the intent recognition model.

[0073] The training method for the intent recognition model provided in this disclosure can be based on the pre-trained language ERNIE. It abandons the common practice of previous pre-trained models by integrating a convolutional neural network model into the model architecture for fine-tuning. ERNIE has strong semantic representation capabilities, and convolutional neural networks also have their own advantages in intent recognition tasks, capable of capturing n-gram information in text sequences. This n-gram information is particularly important for classification tasks. Keywords and key features in text are typically arranged and combined in n-gram form. Therefore, this disclosure combines ERNIE and convolutional neural networks for training, ultimately obtaining an intent recognition model based on user interests.

[0074] Based on this, in one implementation of the present disclosure, the text sequence of each training text further includes a start label, and the encoding layer includes multiple sub-encoding layers. The electronic device inputs the target features of each training text into the network layers included in the initial intent recognition model to obtain the prediction result of each training text, which may specifically include steps A-C.

[0075] Step A: The electronic device inputs the text sequence of the first training text into each of the multiple sub-coding layers to obtain the multi-head attention result of the first training text in each sub-coding layer.

[0076] The first training text is one of the aforementioned multiple training texts, and the multi-head attention result of the first training text in a sub-encoding layer includes the representation of each word included in the first training text in the sub-encoding layer and the representation of the start label included in the first training text in the sub-encoding layer.

[0077] For example, a sub-coding layer can be a transformer layer.

[0078] Optionally, the text sequence of each training text may also include an end label, and the multi-head attention result of the first training text in a sub-coding layer may also include the representation of the end label included in the first training text in the sub-coding layer.

[0079] For example, the start tag can be a CLS tag and the end tag can be a SEP tag.

[0080] It should be understood that a key component of each sub-coding layer is the multi-head attention mechanism, which consists of several self-attention mechanisms. Specifically, the aforementioned encoding layer includes multiple attention heads. The electronic device inputs the text sequence of the first training text into each of the multiple sub-coding layers to obtain the multi-head attention result of the first training text in each sub-coding layer, which may specifically include steps A1-A3.

[0081] Step A1: The electronic device inputs the text sequence of the first training text into each sub-encoding layer to determine the query vector, the key vector, and the value vector of the first training text based on each attention head in each sub-encoding layer.

[0082] It should be understood that while the structure of each of the multiple sub-coding layers is the same, the learnable parameters (or weights) differ. Similarly, for each of the multiple attention heads, the learnable parameters (or weights) for the query vector, key vector, and value vector are different. Specifically, by performing a linear transformation on the text sequence of the first training text based on different parameters, the electronic device can obtain the query vector, key vector, and value vector of the first training text based on each attention head in each sub-coding layer.

[0083] Step A2: The electronic device determines the single-head attention result of the first training text in each sub-encoding layer based on each attention head, according to the query vector obtained by the first training text in each sub-encoding layer based on each attention head, the key vector obtained by the first training text in each sub-encoding layer based on each attention head, and the value vector obtained by the first training text in each sub-encoding layer based on each attention head.

[0084] Optionally, the electronic device can determine that the single-head attention result obtained by the first training text based on the i-th attention head in a sub-coding layer satisfies the following formula:

[0085]

[0086] Among them, Head i (Q,K,V) represents the single-head attention result obtained by the first training text based on the i-th attention head in this sub-encoding layer, where Q i This indicates that the query vector obtained by the first training text in this sub-encoding layer is based on the i-th attention head. V represents the transpose of the key vector obtained from the i-th attention head for the first training text in this sub-encoding layer. i This represents the value vector obtained by the first training text in the sub-encoding layer based on the i-th attention head. This represents the dimension of the key vector obtained by the first training text in the sub-encoding layer based on the i-th attention head, where i is an integer greater than or equal to 1.

[0087] Step A3: The electronic device performs a concatenation operation on the single-head attention results of the first training text in each sub-coding layer based on each attention head, to obtain the multi-head attention results of the first training text in each sub-coding layer.

[0088] For example, the number of attention heads mentioned above can be eight. The electronic device can concatenate the eight single-head attention results to obtain one multi-head attention result.

[0089] In this embodiment, the electronic device can determine the single-head attention result of the first training text based on each attention head in each sub-encoding layer based on the query vector, key vector, and value vector of the first training text based on each attention head in each sub-encoding layer; and concatenate the single-head attention result to obtain the multi-head attention result of the first training text in each sub-encoding layer. This allows for the optimization of different feature parts of each word in the first training text based on each attention head, thereby balancing the potential biases produced by the same attention head, giving the first training text a more diverse and accurate semantic expression, and ultimately improving the model training effect.

[0090] Step B: The electronic device determines the representation of the start label included in the first training text in each sub-coding layer as the output result of the first training text in each sub-coding layer.

[0091] It should be understood that for a given training text (e.g., the first training text), the start label included in the first training text does not contain obvious speech information. Compared to each word included in the first training text, the start label can more fairly integrate the semantic information of each word included in the first training text, and can accurately represent the semantic information of the first training text.

[0092] Step C: The electronic device performs preset operations on the output results of the first training text at each sub-coding layer to obtain the target features of the first training text.

[0093] The preset operation includes one or more of the following: splicing operation, convolution operation, and pooling operation.

[0094] It should be understood that for each of the aforementioned sub-coding layers, a first training text output can be obtained. The electronic device can concatenate the multiple outputs of the first training text. Then, the electronic device can perform a convolution operation on the feature vector obtained after the concatenation operation, and perform a pooling operation on the feature vector obtained after the convolution operation to obtain the target features of the first training text.

[0095] Optionally, the electronic device can perform convolution operations based on convolution kernels of various sizes, and the pooling operation mentioned above can specifically be a max pooling operation.

[0096] For example, the size of the convolution kernel can be 2, 3, or 5.

[0097] In the training method of the intent recognition model provided in this disclosure embodiment, the electronic device can input the text sequence of the first training text into each of the multiple sub-encoding layers to obtain the multi-head attention result of the first training text in each sub-encoding layer; and determine the representation of the start label included in the first training text in each sub-encoding layer as the output result of the first training text in each sub-encoding layer; then the electronic device can perform prediction operations (including one or more of concatenation, convolution, and pooling operations) on the output result of the first training text in each sub-encoding layer to obtain the target features of the first training text. In this disclosure embodiment, for a certain training text (e.g., the first training text), the start label included in the first training text has no obvious speech information. Compared with each word included in the first training text, the start label can more fairly integrate the semantic information of each word included in the first training text, and can accurately represent the semantic information of the first training text. That is, the electronic device can generate target features that can accurately represent the semantic information of the text, thereby improving the accuracy of the intent recognition model.

[0098] The following example illustrates the training process of the intent recognition model provided in this disclosure.

[0099] like Figure 4 As shown, the initial intent recognition model includes 12 sub-coding layers (including sub-coding layer 1, sub-coding layer 2, ... and sub-coding layer 12), a convolutional neural network, and a fully connected layer.

[0100] The electronic device can input the text sequence of the first training text into the 12 sub-coding layers respectively to obtain 12 CLS (including CLS1, CLS2, ... and CLS12). Each CLS is the output result of the first training text in a sub-coding layer.

[0101] After the electronic device concatenates the 12 CLSs, it can input the resulting feature vector into a convolutional neural network. This convolutional neural network is used to perform convolution and pooling operations on the feature vector obtained from the concatenation operation to obtain the target features of the first training text.

[0102] Then, the electronic device can input the target features of the first training text into the fully connected layer to obtain the prediction result of the first training text. Based on the prediction result of each training text and the actual result of each training text, the electronic device determines the cross-entropy loss (i.e., the target loss function determined based on cross-entropy), and then updates the parameters in the 12 sub-encoding layers, the parameters in the convolutional neural network, and the parameters in the fully connected network based on the cross-entropy loss to obtain the target intent recognition model.

[0103] like Figure 5 As shown in the embodiments of this disclosure, an intent recognition method is also provided. This intent recognition method may further include steps S501-S503.

[0104] S501, The electronic device acquires the text sequence of the text to be recognized.

[0105] The text sequence of the text to be identified includes the word vector of each word in the text.

[0106] S502. The electronic device inputs the text sequence of the text to be recognized into the encoding layer of the target intent recognition model to obtain the target features of the text to be recognized.

[0107] The target features of the text to be identified are used to characterize the relationship between each word in the text and other words in the text, and the target intent recognition model is used to identify whether a text contains intent.

[0108] It should be understood that the target features of the text to be identified incorporate the semantic information of each word included in the text to be identified, and can accurately represent the semantic information of the text to be identified.

[0109] It should be noted that the explanation of how electronic devices input the text sequence of the text to be recognized into the encoding layer of the target intent recognition model to obtain the target features of the text to be recognized is the same as or similar to the specific description of how electronic devices input the text sequence of each training text into the encoding layer included in the initial intent recognition model to obtain the target features of each training text, and will not be repeated here.

[0110] In this embodiment of the disclosure, the initial intent recognition model can be understood as an intent recognition model that is being trained, and the target intent recognition model can be understood as an intent recognition model that has already been trained.

[0111] S503. The electronic device inputs the target features of the text to be recognized into the network layer included in the target intent recognition model to determine whether the text to be recognized contains intent.

[0112] The technical solution provided by the above embodiments can bring at least the following beneficial effects: As shown in S501-S503, the electronic device can acquire the text sequence of the text to be recognized; and input the text sequence of the text to be recognized into the encoding layer included in the target intent recognition model to obtain the target features of the text to be recognized; then the electronic device can input the target features of the text to be recognized into the network layer included in the target intent recognition model to determine whether the text to be recognized contains intent. In this embodiment of the present disclosure, since the target features of the text to be recognized are used to characterize the association relationship between each word included in the text to be recognized and other words included in the text to be recognized, the target features of the text to be recognized incorporate the semantic information of each word included in the text to be recognized, and can accurately characterize the semantic information of the text to be recognized. Thus, when the electronic device inputs the target features of the text to be recognized into the network layer included in the target intent recognition model, it can accurately determine whether the text to be recognized contains intent, thereby improving the accuracy of intent recognition.

[0113] It is understood that, in practical implementation, the electronic device described in the embodiments of this disclosure may include one or more hardware structures and / or software modules for implementing the training method and intent recognition method of the aforementioned corresponding intent recognition model. These hardware structures and / or software modules can constitute an electronic device. Those skilled in the art should readily recognize that, based on the algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0114] Based on this understanding, the present disclosure also provides a training device for an intent recognition model. Figure 6 A schematic diagram of the structure of a training apparatus for an intent recognition model provided in an embodiment of this disclosure is shown. Figure 6 As shown, the training device 60 for the intent recognition model may include an acquisition module 601 and a processing module 602.

[0115] The acquisition module 601 is used to acquire the text sequence of each training text in a plurality of training texts, wherein the text sequence of each training text includes: the word vector of each word included in each training text.

[0116] The processing module 602 is used to input the text sequence of each training text into the encoding layer included in the initial intent recognition model to obtain the target features of each training text. The target features of each training text are used to characterize the association between each word included in each training text and other words included in each training text.

[0117] The processing module 602 is further configured to input the target features of each training text into the network layers included in the initial intent recognition model to obtain the prediction results of each training text.

[0118] The processing module 602 is further configured to update the parameters included in the initial intent recognition model based on the target loss function to obtain a target intent recognition model. The target intent recognition model is used to predict whether a text contains an intent. The target loss function is used to characterize the degree of inconsistency between the prediction result of each training text and the actual result of each training text.

[0119] Optionally, the training apparatus for the intent recognition model further includes a determination module 603.

[0120] The determination module 603 is used to determine whether there are key segments in each training text, which are used to characterize segments that the user is interested in.

[0121] The determination module 603 is further configured to determine that the true result of each training text contains intent when the key segment exists in each training text.

[0122] The determination module 603 is further configured to determine that the true result of each training text does not contain intent when the key segment does not exist in each training text.

[0123] The determination module 603 is further configured to determine the target loss function based on the prediction result of each training text and the true result of each training text.

[0124] Optionally, the text sequence of each training text also includes a start label, and the encoding layer includes multiple sub-encoding layers.

[0125] The processing module 602 is specifically used to input the text sequence of the first training text into each of the plurality of sub-coding layers to obtain the multi-head attention result of the first training text in each sub-coding layer. The first training text is one of the plurality of training texts. The multi-head attention result of the first training text in a sub-coding layer includes the representation of each word included in the first training text in the sub-coding layer and the representation of the start label included in the first training text in the sub-coding layer.

[0126] The determining module 603 is further configured to determine the representation of the start label included in the first training text in each sub-coding layer as the output result of the first training text in each sub-coding layer.

[0127] The processing module 602 is further configured to perform preset operations on the output of the first training text in each sub-coding layer to obtain the target features of the first training text. The preset operations include one or more of the following: concatenation operation, convolution operation, and pooling operation.

[0128] Optionally, the coding layer includes multiple attention heads.

[0129] The determining module 603 is further configured to input the text sequence of the first training text into each of the sub-encoding layers respectively, and determine the query vector obtained by the first training text based on each of the plurality of attention heads in each sub-encoding layer, the key vector obtained by the first training text based on each attention head in each sub-encoding layer, and the value vector obtained by the first training text based on each attention head in each sub-encoding layer.

[0130] The determining module 603 is further configured to determine the single-head attention result obtained by the first training text based on each attention head in each sub-encoding layer based on the query vector obtained by the first training text based on each attention head in each sub-encoding layer, the key vector obtained by the first training text based on each attention head in each sub-encoding layer, and the value vector obtained by the first training text based on each attention head in each sub-encoding layer.

[0131] The processing module 602 is further configured to perform a concatenation operation on the single-head attention results of the first training text obtained based on each attention head in each sub-coding layer, so as to obtain the multi-head attention results of the first training text in each sub-coding layer.

[0132] Figure 7 This is a structural example diagram of an intent recognition device provided in an embodiment of this disclosure. Figure 7 As shown, the intent recognition device 70 may include: an acquisition module 701, a processing module 702, and a determination module 703.

[0133] The acquisition module 701 is used to acquire the text sequence of the text to be identified, which includes the word vector of each word included in the text to be identified.

[0134] The processing module 702 is used to input the text sequence of the text to be identified into the encoding layer of the target intent recognition model to obtain the target features of the text to be identified. The target features of the text to be identified are used to characterize the relationship between each word included in the text to be identified and other words included in the text to be identified. The target intent recognition model is used to identify whether a text contains an intent.

[0135] The determination module 703 is used to input the target features of the text to be identified into the network layer included in the target intent recognition model to determine whether the text to be identified contains intent.

[0136] Optionally, the text sequence of the text to be identified may also include a start label, and the encoding layer may include multiple sub-encoding layers.

[0137] The processing module 702 is specifically used to input the text sequence of the text to be recognized into each of the multiple sub-coding layers to obtain the multi-head attention result of the text to be recognized in each sub-coding layer. The multi-head attention result of the text to be recognized in a sub-coding layer includes the representation of each word in the text to be recognized in the sub-coding layer and the representation of the start label in the text to be recognized in the sub-coding layer.

[0138] The determination module 703 is further configured to determine the representation of the start tag included in the text to be recognized in each sub-encoding layer as the output result of the text to be recognized in each sub-encoding layer.

[0139] The processing module 702 is further configured to perform preset operations on the output results of the text to be recognized in each sub-encoding layer to obtain the target features of the text to be recognized. The preset operations include one or more of the following: concatenation operation, convolution operation, and pooling operation.

[0140] Optionally, the coding layer includes multiple attention heads.

[0141] The determining module 703 is further configured to input the text sequence of the text to be identified into each of the sub-encoding layers respectively, and determine the query vector obtained by the text to be identified based on each of the plurality of attention heads in each sub-encoding layer, the key vector obtained by the text to be identified based on each attention head in each sub-encoding layer, and the value vector obtained by the text to be identified based on each attention head in each sub-encoding layer.

[0142] The determining module 703 is further configured to determine the single-head attention result of the text to be identified in each sub-encoding layer based on each attention head, the query vector obtained by the text to be identified in each sub-encoding layer based on each attention head, the key vector obtained by the text to be identified in each sub-encoding layer based on each attention head, and the value vector obtained by the text to be identified in each sub-encoding layer based on each attention head.

[0143] The processing module 702 is further configured to perform a concatenation operation on the single-head attention results of the text to be recognized in each sub-coding layer based on each attention head, so as to obtain the multi-head attention results of the text to be recognized in each sub-coding layer.

[0144] As described above, the embodiments of this disclosure can divide the training device for the intent recognition model and the intent recognition device into functional modules according to the above method examples.

[0145] Furthermore, it should be noted that the module division in this embodiment is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. For example, different functional modules may be defined for each function, or two or more functions may be integrated into one processing module.

[0146] The specific methods by which each module performs its operation and the beneficial effects of the intent recognition model training device and intent recognition device in the above embodiments have been described in detail in the foregoing method embodiments, and will not be repeated here.

[0147] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0148] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0149] like Figure 8As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0150] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0151] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the training method for the intent recognition model and the intent recognition method. For example, in some embodiments, the training method for the intent recognition model and the intent recognition method can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the training method for the intent recognition model and the intent recognition method described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform the training method of the intent recognition model and the intent recognition method.

[0152] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0153] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0154] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0155] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0156] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0157] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0158] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0159] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for training an intent recognition model, comprising: Obtain the text sequence of each training text from a plurality of training texts, wherein the text sequence of each training text includes: word vectors of each word included in each training text; and the text sequence of each training text also includes start tags; The text sequence of the first training text is input into each of the multiple sub-encoding layers included in the initial intent recognition model to obtain the multi-head attention result of the first training text in each sub-encoding layer. The first training text is one of the multiple training texts. The multi-head attention result of the first training text in a sub-encoding layer includes the representation of each word included in the first training text in the sub-encoding layer and the representation of the start label included in the first training text in the sub-encoding layer. The representation of the start label included in the first training text in each sub-coding layer is determined as the output result of the first training text in each sub-coding layer; The output of the first training text at each sub-encoding layer is subjected to a preset operation to obtain the target features of the first training text. Each sub-encoding layer has the same structure but different learnable parameters. The preset operation includes: concatenating the output of the first training text at each sub-encoding layer, inputting the concatenated feature vector into a convolutional neural network, and performing convolution and pooling operations by the convolutional neural network to obtain the target features of the first training text. The target features of each training text are used to characterize the association between each word included in each training text and other words included in each training text. The target features of each training text are input into the network layers included in the initial intent recognition model to obtain the prediction result of each training text; The parameters included in the initial intent recognition model are updated based on the target loss function to obtain the target intent recognition model. The target intent recognition model is used to predict whether a text contains intent. The target loss function is used to characterize the degree of inconsistency between the prediction result of each training text and the actual result of each training text.

2. The method according to claim 1, further comprising: Determine whether there are key segments in each training text, the key segments being used to characterize segments that the user is interested in; When the key segment exists in each training text, the true result of each training text is determined to contain intent; Alternatively, when the key segment does not exist in each of the training texts, the true result of each training text is determined to be that it does not contain intent; The target loss function is determined based on the prediction result and the actual result of each training text.

3. The method according to claim 1, wherein the coding layer comprises a plurality of attention heads, wherein, The step of inputting the text sequence of the first training text into each of the plurality of sub-coding layers to obtain the multi-head attention result of the first training text in each sub-coding layer includes: The text sequence of the first training text is input into each of the sub-encoding layers to determine the query vector obtained by the first training text based on each of the plurality of attention heads in each sub-encoding layer, the key vector obtained by the first training text based on each attention head in each sub-encoding layer, and the value vector obtained by the first training text based on each attention head in each sub-encoding layer. Based on the query vector obtained by the first training text in each sub-encoding layer based on each attention head, the key vector obtained by the first training text in each sub-encoding layer based on each attention head, and the value vector obtained by the first training text in each sub-encoding layer based on each attention head, determine the single-head attention result obtained by the first training text in each sub-encoding layer based on each attention head; The single-head attention results of the first training text obtained based on each attention head in each sub-encoding layer are concatenated to obtain the multi-head attention results of the first training text in each sub-encoding layer.

4. An intent recognition method, comprising: Obtain the text sequence of the text to be identified, wherein the text sequence of the text to be identified includes: the word vector of each word included in the text to be identified, and the text sequence of the text to be identified also includes a start tag; The text sequence of the text to be identified is input into each of the multiple sub-coding layers in the target intent recognition model to obtain the multi-head attention result of the text to be identified in each sub-coding layer. The multi-head attention result of the text to be identified in a sub-coding layer includes the representation of each word included in the text to be identified in the sub-coding layer and the representation of the start label included in the text to be identified in the sub-coding layer. The representation of the start tag included in the text to be recognized in each sub-encoding layer is determined as the output result of the text to be recognized in each sub-encoding layer; The target features of the text to be identified are obtained by performing preset operations on the output results of each sub-encoding layer. Each sub-encoding layer has the same structure but different learnable parameters. The preset operations include: concatenating the output results of the text to be identified in each sub-encoding layer, inputting the concatenated feature vector into a convolutional neural network, and performing convolution and pooling operations by the convolutional neural network to obtain the target features of the text to be identified. The target features of the text to be identified are used to characterize the association between each word in the text to be identified and other words in the text to be identified. The target intent recognition model is used to identify whether a text contains intent. The target features of the text to be identified are input into the network layers included in the target intent recognition model to determine whether the text to be identified contains intent.

5. The method according to claim 4, wherein the coding layer comprises a plurality of attention heads, wherein, The step of inputting the text sequence of the text to be identified into each of the plurality of sub-coding layers to obtain the multi-head attention result of the text to be identified in each sub-coding layer includes: The text sequence of the text to be identified is input into each of the sub-encoding layers to determine the query vector, the key vector, and the value vector of the text to be identified based on each attention head in each of the multiple attention heads in each sub-encoding layer. Based on the query vector obtained by the text to be identified in each sub-encoding layer based on each attention head, the key vector obtained by the text to be identified in each sub-encoding layer based on each attention head, and the value vector obtained by the text to be identified in each sub-encoding layer based on each attention head, the single-head attention result obtained by the text to be identified in each sub-encoding layer based on each attention head is determined; The single-head attention results of the text to be identified in each sub-coding layer based on each attention head are concatenated to obtain the multi-head attention results of the text to be identified in each sub-coding layer.

6. A training device for an intent recognition model, comprising: Acquisition module, determination module, and processing module; The acquisition module is used to acquire the text sequence of each training text in a plurality of training texts, wherein the text sequence of each training text includes: the word vector of each word included in each training text; the text sequence of each training text also includes a start label; The processing module is used to input the text sequence of the first training text into each of the multiple sub-coding layers in the initial intent recognition model to obtain the multi-head attention result of the first training text in each sub-coding layer. The first training text is one of the multiple training texts. The multi-head attention result of the first training text in a sub-coding layer includes the representation of each word included in the first training text in the sub-coding layer and the representation of the start label included in the first training text in the sub-coding layer. The determining module is further configured to determine the representation of the start tag included in the first training text in each sub-coding layer as the output result of the first training text in each sub-coding layer; The processing module is further configured to perform preset operations on the output results of the first training text at each sub-encoding layer to obtain the target features of the first training text. Each sub-encoding layer has the same structure but different learnable parameters. The preset operations include: concatenating the output results of the first training text at each sub-encoding layer, inputting the concatenated feature vector into a convolutional neural network, and performing convolution and pooling operations by the convolutional neural network to obtain the target features of the first training text. The target features of each training text are used to characterize the association between each word included in each training text and other words included in each training text. The processing module is further configured to input the target features of each training text into the network layer included in the initial intent recognition model to obtain the prediction result of each training text; The processing module is further configured to update the parameters included in the initial intent recognition model based on the target loss function to obtain a target intent recognition model. The target intent recognition model is used to predict whether a text contains an intent. The target loss function is used to characterize the degree of inconsistency between the prediction result of each training text and the actual result of each training text.

7. The apparatus according to claim 6, further comprising: Determine the module; The determining module is used to determine whether there are key segments in each training text, and the key segments are used to characterize segments that the user is interested in; The determining module is further configured to determine that the true result of each training text contains intent when the key segment exists in each training text; The determining module is further configured to determine that the true result of each training text does not contain intent when the key segment does not exist in each training text; The determining module is further configured to determine the target loss function based on the prediction result of each training text and the actual result of each training text.

8. The apparatus of claim 6, wherein the coding layer comprises a plurality of attention heads; The determining module is further configured to input the text sequence of the first training text into each of the sub-encoding layers respectively, and determine the query vector obtained by the first training text based on each of the plurality of attention heads in each sub-encoding layer, the key vector obtained by the first training text based on each attention head in each sub-encoding layer, and the value vector obtained by the first training text based on each attention head in each sub-encoding layer. The determining module is further configured to determine the single-head attention result obtained by the first training text based on each attention head in each sub-encoding layer based on the query vector obtained by the first training text based on each attention head in each sub-encoding layer, the key vector obtained by the first training text based on each attention head in each sub-encoding layer, and the value vector obtained by the first training text based on each attention head in each sub-encoding layer. The processing module is further configured to perform a concatenation operation on the single-head attention results of the first training text obtained based on each attention head in each sub-coding layer, so as to obtain the multi-head attention results of the first training text in each sub-coding layer.

9. An intent recognition device, comprising: The module includes an acquisition module, a processing module, and a determination module. The acquisition module is used to acquire a text sequence of the text to be identified, wherein the text sequence of the text to be identified includes: word vectors of each word included in the text to be identified, and the text sequence of the text to be identified also includes start tags; The processing module is specifically used to input the text sequence of the text to be identified into each of the multiple sub-coding layers in the target intent recognition model, and to obtain the multi-head attention result of the text to be identified in each sub-coding layer. The multi-head attention result of the text to be identified in a sub-coding layer includes the representation of each word included in the text to be identified in the sub-coding layer and the representation of the start label included in the text to be identified in the sub-coding layer. The determining module is further configured to determine the representation of the start tag included in the text to be identified in each sub-encoding layer as the output result of the text to be identified in each sub-encoding layer; The processing module is further configured to perform preset operations on the output results of the text to be recognized at each sub-encoding layer to obtain the target features of the text to be recognized. Each sub-encoding layer has the same structure but different learnable parameters. The preset operations include: concatenating the output results of the text to be recognized at each sub-encoding layer, inputting the concatenated feature vector into a convolutional neural network, and performing convolution and pooling operations by the convolutional neural network to obtain the target features of the text to be recognized. The target features of the text to be recognized are used to characterize the association between each word included in the text to be recognized and other words included in the text to be recognized. The target intent recognition model is used to identify whether a text contains intent. The determining module is used to input the target features of the text to be identified into the network layer included in the target intent recognition model to determine whether the text to be identified contains intent.

10. The apparatus of claim 9, wherein the coding layer comprises a plurality of attention heads; The determining module is further configured to input the text sequence of the text to be identified into each of the sub-encoding layers respectively, and determine the query vector obtained by the text to be identified based on each of the plurality of attention heads in each sub-encoding layer, the key vector obtained by the text to be identified based on each attention head in each sub-encoding layer, and the value vector obtained by the text to be identified based on each attention head in each sub-encoding layer; The determining module is further configured to determine the single-head attention result of the text to be identified in each sub-encoding layer based on each attention head, according to the query vector obtained by the text to be identified in each sub-encoding layer based on each attention head, the key vector obtained by the text to be identified in each sub-encoding layer based on each attention head, and the value vector obtained by the text to be identified in each sub-encoding layer based on each attention head; The processing module is further configured to perform a concatenation operation on the single-head attention result of the text to be identified in each sub-coding layer based on each attention head, so as to obtain the multi-head attention result of the text to be identified in each sub-coding layer.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the training method of the intent recognition model according to any one of claims 1-3 or to perform the intent recognition method according to any one of claims 4-5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the training method of the intent recognition model according to any one of claims 1-3 or to execute the intent recognition method according to any one of claims 4-5.

13. A computer program product comprising a computer program that, when executed by a processor, implements a training method for an intent recognition model according to any one of claims 1-3 or implements an intent recognition method according to any one of claims 4-5.

Images