Intention recognition method and device, storage medium, program product and computer equipment

Abstract

The invention discloses an intention recognition method and device, a storage medium, a program product and computer equipment. The method comprises the steps of obtaining to-be-recognized data; calling a hidden layer in the neural network model, and determining first hidden state information matched with the to-be-identified data; determining a semantic difference between the first hidden state information and preset semantic reference information; under the condition that the semantic difference meets a preset semantic drift condition, adjusting the neural network model to obtain an adjusted neural network model; and calling the adjusted neural network model, and determining an intention recognition result matched with the to-be-recognized data, so that whether semantic drift occurs or not can be judged by performing difference analysis on the hidden state information and the preset semantic reference information. When the semantic drift occurs, the model can be subjected to related adjustment to improve the recognition capability of the model for the change corresponding to the semantic drift, and the adjusted model is reused to complete intention recognition, so that the intention recognition accuracy can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an intent recognition method, apparatus, storage medium, program product, and computer equipment. Background Technology

[0002] Intent recognition refers to the analysis of user-related data to identify the intent that the user wants to express.

[0003] In related technologies, general-purpose deep learning models are typically used to perform intent analysis. However, when the intent a user intends to express undergoes a slight change, there may still be some correlation between the intent before and after the change, but there are indeed subtle differences. This is called semantic drift (SD). Deep learning models generally have difficulty directly and accurately distinguishing the changes corresponding to this semantic drift, resulting in low accuracy in intent recognition. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application propose an intent recognition method, apparatus, storage medium, program product, and computer device, which can improve the accuracy of intent recognition.

[0005] In a first aspect, embodiments of this application provide an intent recognition method, including: Obtain the data to be identified; The hidden layer in the neural network model is invoked to determine the first hidden state information that matches the data to be identified; Determine the semantic difference between the first hidden state information and the preset semantic benchmark information; When the semantic differences meet the preset semantic drift conditions, the neural network model is adjusted to obtain the adjusted neural network model. The adjusted neural network model is invoked to determine the intent recognition result that matches the data to be identified.

[0006] Optionally, adjusting the neural network model includes: The neural network model is adjusted based on the aforementioned semantic differences.

[0007] Optionally, adjusting the neural network model based on the semantic differences includes: Based on the aforementioned semantic differences, the degree of semantic drift is determined; The neural network model is adjusted based on the degree of semantic drift.

[0008] Optionally, adjusting the neural network model based on the degree of semantic drift includes: Based on the degree of semantic drift, determine the model weight adjustment coefficient; Based on the model weight adjustment coefficient and the model weight of the neural network model, the adjusted model weight is determined; The neural network model is adjusted based on the adjusted model weights.

[0009] Optionally, adjusting the neural network model includes: The hidden layers in the neural network model are adjusted.

[0010] Optionally, the step of invoking the hidden layer in the neural network model to determine the first hidden state information matching the data to be identified includes: Determine the feature information of the data to be identified; Based on the feature information, the hidden layer in the neural network model is invoked to determine the first hidden state information.

[0011] Optionally, determining the feature information of the data to be identified includes: Based on the data to be identified, a feature extraction model is invoked to determine the feature information.

[0012] Optionally, the data to be identified includes the text to be identified, the feature extraction model includes a pre-trained word embedding model, and the feature information includes the word embedding feature information of the text to be identified; The step of determining the feature information by calling a feature extraction model based on the data to be identified includes: The word embedding model is invoked to convert each text unit in the text to be identified into a word embedding feature corresponding to that text unit, so as to obtain the word embedding feature information containing each word embedding feature.

[0013] Optionally, the hidden layer includes a forward long short-term memory network (LSTM) and a backward LSTM. The step of calling the hidden layer in the neural network model based on the feature information to determine the first hidden state information includes: Based on the feature information, the forward LSTM in the neural network model is invoked to determine the first forward hidden state, and the backward LSTM in the neural network model is invoked to determine the first backward hidden state. Based on the first forward hiding state and the first backward hiding state, the first hiding state information is determined.

[0014] Optionally, determining the first hidden state information based on the first forward hidden state and the first backward hidden state includes: The first forward hidden state and the first backward hidden state are fused together to obtain the first hidden state information.

[0015] Optionally, the hidden layer includes a forward LSTM and a backward LSTM; The step of calling the adjusted neural network model to determine the intent recognition result matching the data to be recognized includes: Based on the feature information, the forward LSTM in the adjusted neural network model is invoked to determine the second forward hidden state, and the backward LSTM in the adjusted neural network model is invoked to determine the second backward hidden state. The intent recognition result is determined based on the second forward hiding state and the second backward hiding state.

[0016] Optionally, determining the intent recognition result based on the second forward hiding state and the second backward hiding state includes: Based on the second forward hiding state and the second backward hiding state, determine the second hidden state information; Based on the second hidden state information, the intent recognition result is determined.

[0017] Optionally, determining the second hidden state information based on the second forward hidden state and the second backward hidden state includes: The second hidden state information is obtained by fusing the second forward hidden state and the second backward hidden state.

[0018] Optionally, the fusion process includes a stitching process.

[0019] Optionally, determining the intent recognition result based on the second hidden state information includes: Based on the second hidden state information, the output layer in the adjusted neural network model is invoked to determine the intent recognition result.

[0020] Optionally, the preset semantic reference information includes at least one semantic anchor point, which is adapted to indicate the word features of the preset intent keyword; Determining the semantic difference between the first hidden state information and the preset semantic benchmark information includes: Determine the difference information between the first hidden state information and each of the semantic anchor points; Based on the aforementioned difference information, the semantic differences are determined.

[0021] Optionally, determining the difference information between the first hidden state information and each of the semantic anchor points includes: Calculate the similarity between the first hidden state information and each of the semantic anchor points to obtain the difference information of each of the semantic anchor points.

[0022] Optionally, the similarity includes cosine similarity.

[0023] Optionally, determining the semantic differences based on each of the difference information includes: Calculate the average value of each of the aforementioned differences; The semantic differences are determined based on the average value.

[0024] Optionally, the preset semantic drift condition includes the semantic difference being higher than a preset difference threshold.

[0025] Optionally, the preset semantic benchmark information is related to the intent recognition scenario.

[0026] Optionally, before invoking the hidden layer in the neural network model to determine the first hidden state information matching the data to be identified, the method further includes: The data to be identified is preprocessed.

[0027] Secondly, embodiments of this application provide an intent recognition device, including: The data acquisition module is used to acquire the data to be identified. The first hidden state determination module is used to call the hidden layer in the neural network model to determine the first hidden state information that matches the data to be identified. A semantic difference determination module is used to determine the semantic difference between the first hidden state information and the preset semantic benchmark information; The model adjustment module is used to adjust the neural network model when the semantic differences meet the preset semantic drift conditions, so as to obtain the adjusted neural network model. The recognition module is used to call the adjusted neural network model to determine the intent recognition result that matches the data to be recognized.

[0028] Thirdly, embodiments of this application provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the above-mentioned embodiments.

[0029] Fourthly, embodiments of this application provide a computer program product, including computer instructions that, when executed by a processor, implement the steps of the method described in any of the above-described embodiments.

[0030] Fifthly, embodiments of this application provide a computer device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the preceding claims.

[0031] In summary, the embodiments of this application have at least the following beneficial effects: By employing the embodiments of this application, the following steps are taken: First, data to be identified is acquired; then, a hidden layer in a neural network model is invoked to determine first hidden state information matching the data to be identified; the semantic difference between the first hidden state information and preset semantic benchmark information is determined; if the semantic difference satisfies a preset semantic drift condition, the neural network model is adjusted to obtain an adjusted neural network model; finally, the adjusted neural network model is invoked to determine the intent recognition result matching the data to be identified. In this way, by performing difference analysis between the hidden state information and the preset semantic benchmark information, it is possible to determine whether semantic drift has occurred. When semantic drift occurs, the model can be adjusted to improve its ability to recognize changes corresponding to semantic drift, and the adjusted model can be reused to complete intent recognition, thereby improving the accuracy of intent recognition. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the intent recognition method provided in an embodiment of this application; Figure 2 This is a schematic diagram of intent recognition data processing provided in an embodiment of this application; Figure 3 This is yet another schematic diagram of intent recognition data processing provided in the embodiments of this application; Figure 4 This is another schematic diagram of intent recognition data processing provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the intent recognition device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments / examples are only a part of the embodiments / examples of this application, and not all of the embodiments / examples. Based on the embodiments / examples in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0034] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "multiple" means two or more. In the description of this application, the term "comprising" and its variations are open-ended, meaning "including but not limited to." The term "based on" means "at least partially based on." The term "according to" means "at least partially according to." The term "one embodiment / example" means "at least one embodiment / example"; the term "another embodiment / example" means "at least one additional embodiment / example"; the term "some embodiments / examples" means "at least some embodiments / examples."

[0035] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0036] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this application is for the purpose of describing specific embodiments only and is not intended to limit the application. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0037] In intelligent customer service systems, accurately identifying user intent is a core element for achieving efficient service. When interacting with intelligent customer service, users express their needs, or user intent, through natural language. However, users often use different ways to express the same or similar intents. For example, regarding the intent to inquire about an order, a user might say "I want to check the order status" or "I want to know the details of my order." These expressions are semantically related but also have subtle differences, known as semantic drift.

[0038] In related technologies, general-purpose deep learning models such as bidirectional LSTM can be used. These models perform reasonably well when processing texts with significant semantic differences. However, for semantically similar texts with subtle variations, they often struggle to accurately capture semantic drift, leading to errors in intent recognition and impacting the service quality and user experience of intelligent customer service systems. For example, the aforementioned deep learning models are generally not sensitive enough to subtle semantic changes and struggle to detect semantic drift. When encountering sentences with similar meanings but potentially different intents, the model cannot adjust its parameters in a timely manner, resulting in low intent recognition accuracy and poor robustness.

[0039] Therefore, how to effectively detect this semantic drift and adjust the model accordingly to ensure the accuracy of intent recognition has become an urgent problem to be solved in intelligent customer service systems.

[0040] This application provides an intent recognition method, apparatus, storage medium, program product, and computer device, which aim to at least partially address the shortcomings of the aforementioned related technologies.

[0041] Firstly, see [the following] Figure 1 The diagram shows a flowchart of an intent recognition method provided in an embodiment of this application. The intent recognition method can be applied to a computer device with data processing capabilities. The method includes steps S101-S105, as detailed below.

[0042] S101, Obtain the data to be identified.

[0043] In some examples, the data to be identified may include at least one of the following: user-inputted text, user-captured audio, user-captured image, or user-captured video. It is understood that data from different modalities can generally be converted into semantic information, which will not be elaborated upon here.

[0044] In an optional implementation, before invoking the hidden layer in the neural network model to determine the first hidden state information matching the data to be identified, the method further includes: The data to be identified is preprocessed.

[0045] In some examples, the preprocessing may include at least one of the following: data cleaning, word segmentation, stop word removal, etc.

[0046] For example, data cleaning can involve removing special symbols (such as @, #, $, etc.), irrelevant numbers (such as random phone number fragments), and / or whitespace characters from text. For instance, cleaning the text "I want to check the order status #123" into "I want to check the order status".

[0047] Exemplarily, the word segmentation process can be to use a word segmentation tool (such as Jieba) to split the cleaned text into independent words. For example, segment "I want to query the order status" into "I - want - query - order - status".

[0048] Exemplarily, the stop word removal process can be to remove the words without actual semantics in the text, such as "de", "le", "ne", etc. For example, after removing the stop words, "I want to know the situation of my order" becomes "I - want - know - order - situation".

[0049] S102, call the hidden layer in the neural network model to determine the first hidden state information matching the to - be - recognized data.

[0050] In some examples, the neural network model can include an input layer, a hidden layer, and an output layer. Among them, the input layer can be used to receive the data input into the neural network model and output the output result of the input layer to the hidden layer. The hidden layer can be used to process the output result of the input layer to obtain the hidden state information and output it to the output layer. The output layer can be used to map the hidden state information output by the hidden layer to the intention recognition result.

[0051] Exemplarily, feature information can be extracted from the to - be - recognized data and input into the above - mentioned input layer, and then the first hidden state information output by the hidden layer can be obtained.

[0052] In some examples, the neural network model can be pre - trained, and its training process is as follows: Take the pre - processed text data and its corresponding intention labels as training data, where the intention labels are pre - defined user intention categories, such as "query order", "complain about problems", etc.

[0053] Taking the neural network model using an improved bidirectional LSTM model as an example, randomly initialize the parameters such as weights and biases of the improved bidirectional LSTM model. For example, initialize the hidden layer weights as (0.1, 0.3, 0.2).

[0054] Adopt the cross - entropy loss function, and the formula is L = -Σ(ytrue×log(ypred)), where ytrue is the true intention label and ypred is the intention probability distribution predicted by the model.

[0055] Use the Adam optimizer, set the learning rate to 0.001, and perform model training. During the training process, continuously adjust the model parameters through backpropagation to minimize the loss function value. Set the training iteration times to 100 times, and use samples with a batch size of 32 for training each time.

[0056] S103, determine the semantic difference between the first hidden state information and the preset semantic benchmark information.

[0057] In some examples, the preset semantic baseline information can be configured according to the preset intent recognition scenario. For example, in the intelligent customer service scenario, when users interact with intelligent customer service, they will express their needs through natural language, that is, express their intentions by inputting text to be recognized. In this way, the feature information of relevant keywords that are commonly used and / or easily cause semantic confusion in this scenario can be added to the preset semantic baseline information so as to use the semantic difference to measure whether semantic drift has occurred.

[0058] In some examples, this semantic difference can be determined by one or more types of similarity.

[0059] S104, when the semantic difference meets the preset semantic drift condition, the neural network model is adjusted to obtain the adjusted neural network model.

[0060] In some examples, in step S104, adjusting the neural network model may include randomly adjusting the neural network model, calling the randomly adjusted model to re-acquire semantic differences, and judging whether the direction of the random adjustment is conducive to improving the model's recognition performance for the semantic drift that has occurred based on the re-acquired semantic differences. Then, the neural network model can be further adjusted according to the adjustment direction until the acquired semantic differences no longer meet the preset semantic drift conditions (i.e., it can be considered that no semantic drift has occurred), thereby using the neural network model that does not meet the preset semantic drift conditions as the adjusted neural network model.

[0061] As can be seen, the implementation methods adopted when adjusting the neural network model in the embodiments of this application can be diverse, and the embodiments of this application do not specifically limit them.

[0062] In some examples, adjusting the neural network model in step S104 may include adjusting the relevant model parameters of the neural network model without changing the model structure of the neural network model. Therefore, the neural network model before and after adjustment may contain hidden layers. And / or, adjusting the neural network model may include adjusting at least part of the model structure of the neural network model, such as adjusting the number of different types of layers in the neural network model. In this case, the neural network model before and after adjustment may contain at least one hidden layer.

[0063] In one optional implementation, the preset semantic drift condition includes the semantic difference being higher than a preset difference threshold. For example, if the semantic difference is represented by similarity, then the semantic difference being higher than the preset difference threshold may mean that the similarity is lower than a preset similarity threshold.

[0064] S105, invoke the adjusted neural network model to determine the intent recognition result that matches the data to be identified.

[0065] In some examples, the neural network model may include an input layer, a hidden layer, and an output layer. The input layer may be used to receive data input into the neural network model and output the output of the input layer to the hidden layer. The hidden layer may be used to process the output of the input layer to obtain hidden state information and output it to the output layer. The output layer may be used to map the hidden state information output by the hidden layer to an intent recognition result.

[0066] In one optional implementation, the step of invoking the hidden layer in the neural network model to determine the first hidden state information matching the data to be identified includes: Determine the feature information of the data to be identified; Based on the feature information, the hidden layer in the neural network model is invoked to determine the first hidden state information.

[0067] In one optional implementation, determining the feature information of the data to be identified includes: Based on the data to be identified, a feature extraction model is invoked to determine the feature information.

[0068] In one optional implementation, the data to be identified includes text to be identified, the feature extraction model includes a pre-trained word embedding model, and the feature information includes word embedding feature information of the text to be identified; The step of determining the feature information by calling a feature extraction model based on the data to be identified includes: The word embedding model is invoked to convert each text unit in the text to be identified into a word embedding feature corresponding to that text unit, so as to obtain the word embedding feature information containing each word embedding feature.

[0069] In some examples, the word embedding model may include the Word2Vec model.

[0070] In some examples, this word embedding model can be used to convert each text unit in the text to be identified into a fixed-dimensional word embedding vector, which serves as the word embedding feature corresponding to the text unit. For example, the fixed dimension can be d, here d=300, for example, the word embedding vector corresponding to "query" is v1=(v11, v12,...,v1300), the word embedding vector corresponding to "order" is v2=(v21, v22,...,v2300), etc.

[0071] In one alternative implementation, the hidden layer includes a forward LSTM (Long Short-Term Memory) network and a backward LSTM. The step of calling the hidden layer in the neural network model based on the feature information to determine the first hidden state information includes: Based on the feature information, the forward LSTM in the neural network model is invoked to determine the first forward hidden state, and the backward LSTM in the neural network model is invoked to determine the first backward hidden state. Based on the first forward hiding state and the first backward hiding state, the first hiding state information is determined.

[0072] In some examples, the neural network model can be a bidirectional LSTM model, which can include an input layer, a hidden layer, and an output layer. The hidden layer incorporates a Semantic Anchor (SA) mechanism.

[0073] The input layer can be used to receive the above feature information, such as the word embedding vector sequence in the word embedding feature information of the text to be identified, denoted as the input sequence. Here, let the input sequence be x=[x1,x2,...,xt], where xt is the word embedding vector of the t-th word and t is the sequence length.

[0074] See also Figure 2 The hidden layer can contain a forward LSTM and a backward LSTM. The forward LSTM is used to process the input sequence from left to right to obtain the forward hidden state →ht; the backward LSTM is used to process the input sequence from right to left to obtain the backward hidden state ←ht.

[0075] Suppose that at a certain time t, the forward hidden state →ht=(0.3,0.4,0.2) and the backward hidden state ←ht=(0.5,0.1,0.6), then ht=(0.3,0.4,0.2,0.5,0.1,0.6). (Subsequent calculations will use the first 3 dimensions as examples).

[0076] In one optional implementation, determining the first hidden state information based on the first forward hidden state and the first backward hidden state includes: The first forward hidden state and the first backward hidden state are fused together to obtain the first hidden state information.

[0077] In some examples, see Figure 2 The fusion process includes splicing, so the hidden state ht of the bidirectional LSTM can be a splice of the forward hidden state and the backward hidden state, i.e., ht=[→ht;←ht].

[0078] In one alternative implementation, the hidden layer includes a forward LSTM and a backward LSTM; The step of calling the adjusted neural network model to determine the intent recognition result matching the data to be recognized includes: Based on the feature information, the forward LSTM in the adjusted neural network model is invoked to determine the second forward hidden state, and the backward LSTM in the adjusted neural network model is invoked to determine the second backward hidden state. The intent recognition result is determined based on the second forward hiding state and the second backward hiding state.

[0079] It is understood that the forward LSTM and backward LSTM contained in the adjusted neural network model in this embodiment may be structurally similar to or identical to the forward LSTM and backward LSTM contained in the original neural network model. The main differences are: the weights and other parameters of the forward LSTM in the neural network model are different before and after adjustment, and / or the weights and other parameters of the backward LSTM in the neural network model are different before and after adjustment. However, the calculation principles of the forward LSTM and backward LSTM are the same, and can be referred to the above-mentioned embodiments.

[0080] In one optional implementation, determining the intent recognition result based on the second forward hiding state and the second backward hiding state includes: Based on the second forward hiding state and the second backward hiding state, determine the second hidden state information; Based on the second hidden state information, the intent recognition result is determined.

[0081] In one optional implementation, determining the second hidden state information based on the second forward hidden state and the second backward hidden state includes: The second hidden state information is obtained by fusing the second forward hidden state and the second backward hidden state.

[0082] In one alternative implementation, the fusion process includes a stitching process.

[0083] In one optional implementation, determining the intent recognition result based on the second hidden state information includes: Based on the second hidden state information, the output layer in the adjusted neural network model is invoked to determine the intent recognition result.

[0084] In one optional implementation, the preset semantic reference information includes at least one semantic anchor, which is adapted to indicate the word features of a preset intent keyword; Determining the semantic difference between the first hidden state information and the preset semantic benchmark information includes: Determine the difference information between the first hidden state information and each of the semantic anchor points; Based on the aforementioned difference information, the semantic differences are determined.

[0085] In some examples, the semantic anchor can be a representative word embedding vector selected from pre-trained word embedding vectors. This at least one semantic anchor can be used to represent core words in common user intents. For example, the selection of semantic anchors can include: statistically analyzing the frequency of each word in the pre-processed text, and selecting the word embedding vectors corresponding to words with high frequency and that can represent the core of a specific intent as semantic anchors.

[0086] In some examples, semantic anchors can be stored in a semantic anchor storage module to complete the pre-setting of semantic anchors.

[0087] For example, for order-related intents, word embedding vectors of words such as "order", "query", "status", "logistics", and "details" can be selected as semantic anchors, denoted as a1, a2, a3, a4, and a5 respectively. Their specific vector values ​​are as follows (taking a 3-dimensional vector as an example).

[0088] a1 (orders) = (0.2, 0.5, 0.3) a2 (query) = (0.4, 0.1, 0.6) a3 (state) = (0.3, 0.6, 0.2) a4 (Logistics) = (0.5, 0.2, 0.7) a5 (details) = (0.1, 0.7, 0.4) Furthermore, for the hidden state ht at each time t, its similarity to the semantic anchor ai (i=1, 2, 3...) can be calculated, and the hidden state can be dynamically adjusted based on the similarity.

[0089] In this embodiment, the similarity calculation formula can use cosine similarity, that is, for the hidden state ht and the semantic anchor ai, the similarity sim (ht, ai) = (ht, ai) ai) / (||ht|| ||ai||).

[0090] in Denotes the vector dot product, || || represents the magnitude of the vector.

[0091] Taking ht=(0.3,0.4,0.2) (taking the first 3 dimensions) as an example, the similarity between it and each anchor point is calculated as follows.

[0092] sim(ht,a1)=(0.3×0.2+0.4×0.5+0.2×0.3) / (√(0.3²+0.4²+0.2²)×√(0.2²+0.5²+0.3 ²))=(0.06+0.2+0.06) / (√0.29×√0.38)≈0.32 / (0.5385×0.6164)≈0.32 / 0.332≈0.964.

[0093] sim(ht,a2)=(0.3×0.4+0.4×0.1+0.2×0.6) / (√0.29×√(0.4²+0.1²+0.6²))=(0. 12+0.04+0.12) / (0.5385×√0.53)≈0.28 / (0.5385×0.728)≈0.28 / 0.392≈0.714.

[0094] sim(ht,a3)=(0.3×0.3+0.4×0.6+0.2×0.2) / (√0.29×√(0.3²+0.6²+0.2²))=(0 .09+0.24+0.04) / (0.5385×√0.49)≈0.37 / (0.5385×0.7)≈0.37 / 0.377≈0.981.

[0095] sim(ht,a4)=(0.3×0.5+0.4×0.2+0.2×0.7) / (√0.29×√(0.5²+0.2²+0.7²))=(0. 15+0.08+0.14) / (0.5385×√0.78)≈0.37 / (0.5385×0.883)≈0.37 / 0.476≈0.777.

[0096] sim(ht,a5)=(0.3×0.1+0.4×0.7+0.2×0.4) / (√0.29×√(0.1²+0.7²+0.4²))=(0. 03+0.28+0.08) / (0.5385×√0.66)≈0.39 / (0.5385×0.812)≈0.39 / 0.437≈0.892.

[0097] The formula for calculating the adjusted hidden state ht' is as follows.

[0098] ht'=ht+α×Σ(sim(ht,ai)×(ai-ht)), where α is an adjustment coefficient with a value of 0.5.

[0099] Calculate Σ(sim(ht,ai)×(ai-ht)): sim(ht,a1)×(a1-ht)=0.964×[(0.2-0.3),(0.5-0.4),(0.3-0.2)]=0.964×(-0.1,0.1,0.1)=(-0.0964,0.0964,0.0964).

[0100] sim(ht,a2)×(a2-ht)=0.714×[(0.4-0.3),(0.1-0.4),(0.6-0.2)]=0.714×(0.1,-0.3,0.4)=(0.0714,-0.2142,0.2856).

[0101] sim(ht,a3)×(a3-ht)=0.981×[(0.3-0.3),(0.6-0.4),(0.2-0.2)]=0.981×(0,0.2,0)=(0,0.1962,0).

[0102] sim(ht,a4)×(a4-ht)=0.777×[(0.5-0.3),(0.2-0.4),(0.7-0.2)]=0.777×(0.2,-0.2,0.5)=(0.1554,-0.1554,0.3885).

[0103] sim(ht,a5)×(a5-ht)=0.892×[(0.1-0.3),(0.7-0.4),(0.4-0.2)]=0.892×(-0.2,0.3,0.2)=(-0.1784,0.2676,0.1784).

[0104] Summing yields: (0.0714-0.0964+0-0.1784+0.1554,0.0964-0.2142+0.1962-0.1554+0.2676,0.0964+0.2856+0+0.3885+0.1784)=(-0.048,0.1906,0.9489) The adjusted hidden state ht' is...

[0105] ht'=(0.3,0.4,0.2)+0.5×(-0.048,0.1906,0.9489) =(0.3-0.024,0.4+0.0953,0.2+0.4745) =(0.276,0.4953,0.6745) In one optional implementation, determining the difference information between the first hidden state information and each of the semantic anchor points includes: Calculate the similarity between the first hidden state information and each of the semantic anchor points to obtain the difference information of each of the semantic anchor points.

[0106] In some examples, the similarity between the first hidden state information ht and each semantic anchor can be directly calculated as the difference information of each semantic anchor; or, after calculating the similarity between the first hidden state information ht and each semantic anchor, the first hidden state information ht can be dynamically adjusted according to the similarity, and the similarity between the adjusted first hidden state information ht' and each semantic anchor can be calculated as the difference information of each semantic anchor. For ease of description, the case of ht' will be used as an example for illustration below.

[0107] In one alternative implementation, the similarity includes cosine similarity.

[0108] In one optional implementation, determining the semantic differences based on each of the difference information includes: Calculate the average value of each of the aforementioned differences; The semantic differences are determined based on the average value.

[0109] In some examples, when a new user enters the text "I want to know my order information", it is assumed that the adjusted hidden state ht' (taking a 3-dimensional example) can be obtained as (0.276, 0.4953, 0.6745). See also... Figure 3 Here, we can further calculate the similarity between ht' and each semantic anchor point, where sim(ht',ai) is calculated. The process is shown in Table 1 below: Table 1

[0110] Therefore, the average similarity avgsim = (0.930 + 0.880 + 0.835 + 0.912 + 0.900) / 5 = 4.457 / 5 = 0.891.

[0111] A similarity threshold of θ=0.7 is set. Since 0.891≥0.7, it is determined that the user input text does not have semantic drift.

[0112] Taking the user's input text "I want to cancel my order payment" as an example, after processing, the adjusted hidden state ht'=(0.6,0.2,0.1) is obtained.

[0113] The similarity between it and each semantic anchor point is calculated as shown in Table 2 below.

[0114] Table 2

[0115] The average similarity avgsim = (0.635 + 0.687 + 0.714 + 0.726 + 0.462) / 5 = 3.224 / 5 = 0.645. Since 0.645 < 0.7, the user input text is determined to have semantic drift.

[0116] In one optional implementation, adjusting the neural network model includes: The neural network model is adjusted based on the aforementioned semantic differences.

[0117] In some examples, the model parameters and / or model structure of a neural network model can be adjusted based on semantic differences.

[0118] In one optional implementation, adjusting the neural network model based on the semantic differences includes: Based on the aforementioned semantic differences, the degree of semantic drift is determined; The neural network model is adjusted based on the degree of semantic drift.

[0119] In one optional implementation, adjusting the neural network model based on the degree of semantic drift includes: Based on the degree of semantic drift, determine the model weight adjustment coefficient; Based on the model weight adjustment coefficient and the model weight of the neural network model, the adjusted model weight is determined; The neural network model is adjusted based on the adjusted model weights.

[0120] In one optional implementation, adjusting the neural network model includes: The hidden layers in the neural network model are adjusted.

[0121] In some examples, the hidden layers in a neural network model can be adjusted based on the adjusted model weights.

[0122] For example, when a user inputs the text "I want to know my order information", the model weights remain unchanged because no semantic drift is detected. Assuming the current weights of the hidden layer are wold=(0.1,0.3,0.2), the adjusted weights will still be (0.1,0.3,0.2).

[0123] For example, when a user inputs the text "I want to cancel my order payment", a semantic drift is detected, with a drift degree of driftdegree=1-avgsim=1-0.645=0.355.

[0124] The formula for adjusting the weights is: w_new = w_old × (1 + β × driftdegree).

[0125] Where β is the weight adjustment coefficient, with a value of 0.1, and the weights of the hidden layer before adjustment are w_old=(0.1,0.3,0.2).

[0126] The specific calculations are as follows: w_new1=0.1×(1+0.1×0.355)=0.1×(1+0.0355)=0.1×1.0355=0.10355 w_new2=0.3×(1+0.1×0.355)=0.3×1.0355=0.31065 w_new3=0.2×(1+0.1×0.355)=0.2×1.0355=0.2071 Therefore, the adjusted weights are (0.10355, 0.31065, 0.2071).

[0127] Intent recognition output: See also Figure 4 The adjusted model processes the user's input text and outputs the intent recognition result.

[0128] For the statement "I want to know my order information", the model processes it based on the original weights, and the output intent is "query order". The probability calculation process is as follows: The model processes the logits of the output layer using a softmax function to obtain the probability of each intent category. Assuming the logit value for the "query order" category in the output layer is 2.4, and the logit values ​​for other intent categories are 1.2, 0.8, and 0.5 respectively, then the probability of "query order" is:

[0129] After internal model optimization and matching with training data, the final output probability is 0.92.

[0130] For the statement "I want to cancel my order payment", the model processes it based on the adjusted weights (0.10355, 0.31065, 0.2071).

[0131] The output intent is "Cancel order payment", and the probability calculation process is as follows: The output layer has a logit value of 2.1 for the "Cancel Order Payment" category, and logit values ​​for other intent categories are 1.0, 0.7, and 0.6 respectively. Therefore, the probability of "Cancel Order Payment" is:

[0132] Thus, after model optimization, the final output probability is 0.89.

[0133] In one alternative implementation, the preset semantic benchmark information is related to the intent recognition scenario.

[0134] Secondly, correspondingly, this application also provides an intent recognition device capable of implementing all the processes of the intent recognition method provided in the above embodiments.

[0135] See Figure 5 The diagram shows a schematic representation of an intent recognition device 500 provided in an embodiment of this application. The intent recognition device 500 includes: Data acquisition module 501 is used to acquire data to be identified; The first hidden state determination module 502 is used to call the hidden layer in the neural network model to determine the first hidden state information that matches the data to be identified. The semantic difference determination module 503 is used to determine the semantic difference between the first hidden state information and the preset semantic benchmark information; The model adjustment module 504 is used to adjust the neural network model when the semantic difference meets the preset semantic drift condition, so as to obtain the adjusted neural network model. The recognition module 505 is used to call the adjusted neural network model to determine the intent recognition result that matches the data to be recognized.

[0136] In one optional implementation, adjusting the neural network model includes: The neural network model is adjusted based on the aforementioned semantic differences.

[0137] In one optional implementation, adjusting the neural network model based on the semantic differences includes: Based on the aforementioned semantic differences, the degree of semantic drift is determined; The neural network model is adjusted based on the degree of semantic drift.

[0138] In one optional implementation, adjusting the neural network model based on the degree of semantic drift includes: Based on the degree of semantic drift, determine the model weight adjustment coefficient; Based on the model weight adjustment coefficient and the model weight of the neural network model, the adjusted model weight is determined; The neural network model is adjusted based on the adjusted model weights.

[0139] In one optional implementation, adjusting the neural network model includes: The hidden layers in the neural network model are adjusted.

[0140] In one optional implementation, the step of invoking the hidden layer in the neural network model to determine the first hidden state information matching the data to be identified includes: Determine the feature information of the data to be identified; Based on the feature information, the hidden layer in the neural network model is invoked to determine the first hidden state information.

[0141] In one optional implementation, determining the feature information of the data to be identified includes: Based on the data to be identified, a feature extraction model is invoked to determine the feature information.

[0142] In one optional implementation, the data to be identified includes text to be identified, the feature extraction model includes a pre-trained word embedding model, and the feature information includes word embedding feature information of the text to be identified; The step of determining the feature information by calling a feature extraction model based on the data to be identified includes: The word embedding model is invoked to convert each text unit in the text to be identified into a word embedding feature corresponding to that text unit, so as to obtain the word embedding feature information containing each word embedding feature.

[0143] In one alternative implementation, the hidden layer includes a forward long short-term memory network (LSTM) and a backward LSTM. The step of calling the hidden layer in the neural network model based on the feature information to determine the first hidden state information includes: Based on the feature information, the forward LSTM in the neural network model is invoked to determine the first forward hidden state, and the backward LSTM in the neural network model is invoked to determine the first backward hidden state. Based on the first forward hiding state and the first backward hiding state, the first hiding state information is determined.

[0144] In one optional implementation, determining the first hidden state information based on the first forward hidden state and the first backward hidden state includes: The first forward hidden state and the first backward hidden state are fused together to obtain the first hidden state information.

[0145] In one alternative implementation, the hidden layer includes a forward LSTM and a backward LSTM; The step of calling the adjusted neural network model to determine the intent recognition result matching the data to be recognized includes: Based on the feature information, the forward LSTM in the adjusted neural network model is invoked to determine the second forward hidden state, and the backward LSTM in the adjusted neural network model is invoked to determine the second backward hidden state. The intent recognition result is determined based on the second forward hiding state and the second backward hiding state.

[0146] In one optional implementation, determining the intent recognition result based on the second forward hiding state and the second backward hiding state includes: Based on the second forward hiding state and the second backward hiding state, determine the second hidden state information; Based on the second hidden state information, the intent recognition result is determined.

[0147] In one optional implementation, determining the second hidden state information based on the second forward hidden state and the second backward hidden state includes: The second hidden state information is obtained by fusing the second forward hidden state and the second backward hidden state.

[0148] In one alternative implementation, the fusion process includes a stitching process.

[0149] In one optional implementation, determining the intent recognition result based on the second hidden state information includes: Based on the second hidden state information, the output layer in the adjusted neural network model is invoked to determine the intent recognition result.

[0150] In one optional implementation, the preset semantic reference information includes at least one semantic anchor, which is adapted to indicate the word features of a preset intent keyword; Determining the semantic difference between the first hidden state information and the preset semantic benchmark information includes: Determine the difference information between the first hidden state information and each of the semantic anchor points; Based on the aforementioned difference information, the semantic differences are determined.

[0151] In one optional implementation, determining the difference information between the first hidden state information and each of the semantic anchor points includes: Calculate the similarity between the first hidden state information and each of the semantic anchor points to obtain the difference information of each of the semantic anchor points.

[0152] In one alternative implementation, the similarity includes cosine similarity.

[0153] In one optional implementation, determining the semantic differences based on each of the difference information includes: Calculate the average value of each of the aforementioned differences; The semantic differences are determined based on the average value.

[0154] In one optional implementation, the preset semantic drift condition includes the semantic difference being higher than a preset difference threshold.

[0155] In one alternative implementation, the preset semantic benchmark information is related to the intent recognition scenario.

[0156] In an optional embodiment, the apparatus further includes a preprocessing module, which is used for: Before determining the first hidden state information that matches the data to be identified in the hidden layer of the invoked neural network model, the data to be identified is preprocessed.

[0157] Thirdly, embodiments of this application provide a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any of the above-mentioned embodiments.

[0158] Fourthly, embodiments of this application provide a computer program product, including computer instructions that, when executed by a processor, implement the steps of the method described in any of the above-described embodiments.

[0159] Fifthly, embodiments of this application provide a computer device including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the steps of the method described in any of the preceding claims.

[0160] See Figure 6 The computer device in this embodiment includes a processor 601, a memory 602, and a computer program, such as an intent recognition program, stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program, it implements the steps in the various intent recognition method embodiments described above, for example... Figure 1 The steps S101-S105 are shown.

[0161] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 602 and executed by the processor 601 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the computer device.

[0162] The computer device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device may include, but is not limited to, a processor 601 and a memory 602. Those skilled in the art will understand that the schematic diagram is merely an example of a computer device and does not constitute a limitation on the computer device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0163] The processor 601 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, or processor 601 can be any conventional processor. The processor 601 is the control center of the computer device, connecting various parts of the entire computer device through various interfaces and lines.

[0164] The memory 602 can be used to store the computer programs and / or modules. The processor 601 implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory 602 and calling the data stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0165] Wherein, if the modules / units integrated into the computer device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a non-transitory computer-readable storage medium. When the computer program is executed by the processor 601, it can implement the steps of the various method embodiments described above. Wherein, the computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0166] In summary, the embodiments of this application have at least the following beneficial effects: By employing the embodiments of this application, the following steps are taken: First, data to be identified is acquired; then, a hidden layer in a neural network model is invoked to determine first hidden state information matching the data to be identified; the semantic difference between the first hidden state information and preset semantic benchmark information is determined; if the semantic difference satisfies a preset semantic drift condition, the neural network model is adjusted to obtain an adjusted neural network model; finally, the adjusted neural network model is invoked to determine the intent recognition result matching the data to be identified. In this way, by performing difference analysis between the hidden state information and the preset semantic benchmark information, it is possible to determine whether semantic drift has occurred. When semantic drift occurs, the model can be adjusted to improve its ability to recognize changes corresponding to semantic drift, and the adjusted model can be reused to complete intent recognition, thereby improving the accuracy of intent recognition.

[0167] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware platforms, or it can be implemented entirely by hardware. Based on this understanding, all or part of the technical solutions of this application that contribute to the background technology can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0168] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. An intention recognition method characterized by, The method comprises: obtaining to-be-recognized data; calling a hidden layer in a neural network model to determine first hidden state information matched with the to-be-recognized data; determining a semantic difference between the first hidden state information and preset semantic reference information; in a case where the semantic difference meets a preset semantic drift condition, adjusting the neural network model to obtain an adjusted neural network model; calling the adjusted neural network model to determine an intent recognition result matched with the to-be-recognized data.

2. The method of claim 1, wherein, The adjusting of the neural network model comprises: adjusting the neural network model based on the semantic difference.

3. The method of claim 2, wherein, The adjusting of the neural network model based on the semantic difference comprises: determining a semantic drift degree based on the semantic difference; adjusting the neural network model based on the semantic drift degree.

4. The method of claim 3, wherein, The adjusting of the neural network model based on the semantic drift degree comprises: determining a model weight adjustment coefficient based on the semantic drift degree; determining an adjusted model weight based on the model weight adjustment coefficient and a model weight of the neural network model; adjusting the neural network model based on the adjusted model weight.

5. The method according to any one of claims 1 to 4, characterized in that, The adjusting of the neural network model comprises: adjusting a hidden layer in the neural network model.

6. The method of claim 1, wherein, The calling of the hidden layer in the neural network model to determine the first hidden state information matched with the to-be-recognized data comprises: determining feature information of the to-be-recognized data; calling the hidden layer in the neural network model to determine the first hidden state information based on the feature information.

7. The method of claim 6, wherein, The determining of the feature information of the to-be-recognized data comprises: calling a feature extraction model to determine the feature information based on the to-be-recognized data.

8. The method of claim 7, wherein, The to-be-recognized data comprises to-be-recognized text, the feature extraction model comprises a pre-trained word embedding model, and the feature information comprises word embedding feature information of the to-be-recognized text; The calling of the feature extraction model to determine the feature information based on the to-be-recognized data comprises: calling the word embedding model to convert each text unit in the to-be-recognized text into a word embedding feature corresponding to the text unit to obtain the word embedding feature information comprising the word embedding features.

9. The method of claim 6, wherein, The hidden layer comprises a forward long short-term memory (LSTM) and a backward LSTM; The calling of the hidden layer in the neural network model to determine the first hidden state information based on the feature information comprises: calling the forward LSTM in the neural network model to determine a first forward hidden state based on the feature information, and calling the backward LSTM in the neural network model to determine a first backward hidden state; determining the first hidden state information based on the first forward hidden state and the first backward hidden state.

10. The method of claim 9, wherein, The determining of the first hidden state information based on the first forward hidden state and the first backward hidden state comprises: performing fusion processing on the first forward hidden state and the first backward hidden state to obtain the first hidden state information.

11. The method of claim 6, wherein, The hidden layer comprises a forward LSTM and a backward LSTM; The calling the adjusted neural network model to determine the intent recognition result matched with the to-be-identified data comprises: Based on the feature information, calling a forward LSTM in the adjusted neural network model to determine a second forward hidden state, and calling a backward LSTM in the adjusted neural network model to determine a second backward hidden state; Based on the second forward hidden state and the second backward hidden state, determining the intent recognition result.

12. The method of claim 11, wherein, The determining the intent recognition result based on the second forward hidden state and the second backward hidden state comprises: Based on the second forward hidden state and the second backward hidden state, determining second hidden state information; Based on the second hidden state information, determining the intent recognition result.

13. The method of claim 12, wherein, The determining the second hidden state information based on the second forward hidden state and the second backward hidden state comprises: Based on the second forward hidden state and the second backward hidden state, performing fusion processing to obtain the second hidden state information.

14. The method of claim 10 or 13, wherein The fusion processing comprises splicing processing.

15. The method of claim 12, wherein, The determining the intent recognition result based on the second hidden state information comprises: Based on the second hidden state information, calling an output layer in the adjusted neural network model to determine the intent recognition result.

16. The method of claim 1, wherein, The preset semantic reference information comprises at least one semantic anchor point, and the semantic anchor point is adapted to indicate a word feature of a preset intent keyword. The determining the semantic difference between the first hidden state information and the preset semantic reference information comprises: Determining difference information between the first hidden state information and each semantic anchor point; Based on the difference information, determining the semantic difference.

17. The method of claim 16, wherein, The determining the difference information between the first hidden state information and each semantic anchor point comprises: Calculating a similarity between the first hidden state information and each semantic anchor point to obtain difference information of each semantic anchor point.

18. The method of claim 17, wherein The similarity comprises a cosine similarity.

19. The method of claim 16, wherein, The determining the semantic difference based on the difference information comprises: Calculating an average value of the difference information; Based on the average value, determining the semantic difference.

20. The method of claim 1, wherein The preset semantic drift condition comprises that the semantic difference is higher than a preset difference threshold.

21. The method of any one of claims 1-4 or 6-13 or 15-20, wherein The preset semantic reference information is related to an intent recognition scenario.

22. The method of any one of claims 1-4 or 6-13 or 15-20, wherein, Before the calling a hidden layer in the neural network model to determine the first hidden state information matched with the to-be-identified data, the method further comprises: Preprocessing the to-be-identified data.

23. An intention recognition apparatus characterized by comprising: Comprise: A data acquisition module is configured to acquire to-be-identified data; A first hidden state determination module is configured to call a hidden layer in the neural network model to determine first hidden state information matched with the to-be-identified data. A semantic difference determination module is configured to determine a semantic difference between the first hidden state information and preset semantic reference information. A model adjustment module is configured to adjust the neural network model to obtain an adjusted neural network model when the semantic difference satisfies a preset semantic drift condition. An identification module is configured to call the adjusted neural network model to determine an intent recognition result matched with the to-be-identified data. 24.A non-transitory computer-readable storage medium having stored thereon a computer program. The computer program is executed by a processor to implement the method in any one of claims 1-22.

25. A computer program product comprising computer instructions, characterized in that, The computer program is executed by a processor to implement the method in any one of claims 1-22.

26. A computer device, comprising: The computer program is executed by a processor to implement the method in any one of claims 1-22.

Images