Intention recognition method and device, electronic equipment, storage medium and program product

Abstract

The application provides an intention recognition method, device, electronic equipment, storage medium and program product, and belongs to the technical field of natural language processing. The method comprises the following steps: constructing an input vector based on the latest round of requests and historical dialogue data input by a user in a multi-round dialogue scenario; inputting the input vector into a pre-trained request rewriting model to obtain an output to-be-recognized request; the to-be-recognized request is a first rewritten request composed of a stop generation mark and the latest round of requests or a second rewritten request composed of a non-stop generation mark and a rewritten request; identifying the target intention of the user based on the to-be-recognized request; and the request rewriting model is used for judging whether the latest round of requests depends on the historical dialogue data, and outputting the first rewritten request or the second rewritten request according to the judgment result. According to the application, the historical dialogue dependence judgment and the request rewriting task are simultaneously completed in a single inference by one request rewriting model, so that the intention recognition is more efficient, more accurate and less computationally intensive.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to an intent recognition method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] Intent recognition systems based on Large Language Models (LLMs) are widely applicable to user request parsing and intent determination in multi-turn dialogue scenarios.

[0003] Currently, the main approach to achieving intent recognition is a two-stage processing scheme, which involves designing and training a discriminative model (such as a rule engine or a small classifier) ​​and a rewriting model (such as a referential resolution model). The discriminative model determines the dependence of the user's latest request on the historical dialogue in a multi-turn dialogue scenario, and selectively calls the rewriting model based on the judgment result. The rewriting module then rewrites the user's latest request based on the historical dialogue.

[0004] However, the two-stage processing scheme prolongs the intent recognition process, complicates the intent recognition process, and increases the risk of error propagation. Summary of the Invention

[0005] This invention provides an intent recognition method, apparatus, electronic device, storage medium, and program product to address the shortcomings of existing two-stage processing schemes, such as complex processes and increased risk of error propagation, and to achieve an intent recognition scheme with a simple process that does not increase the risk of error propagation.

[0006] This invention provides an intent recognition method, comprising:

[0007] Based on the latest request and historical dialogue data input by the user in a multi-turn dialogue scenario, construct an input vector;

[0008] The input vector is fed into a pre-trained request rewriting model to obtain the request to be identified output by the request rewriting model; the request to be identified is either a first rewriting request or a second rewriting request.

[0009] Based on the request to be identified, the user's target intent is identified;

[0010] The request rewriting model is used to determine whether the latest round of requests depends on the historical dialogue data, and outputs the first rewriting request or the second rewriting request based on the determination result; the first rewriting request consists of a stop generation flag and the latest round of requests; the second rewriting request consists of a non-stop generation flag and a rewritten request; the rewritten request is a request based on the historical dialogue data after rewriting the latest round of requests.

[0011] According to an intent recognition method provided by the present invention, the request rewriting model is pre-trained based on multiple rewriting training samples; wherein, the rewriting training samples include input vector samples and their corresponding request annotation data; the input vector samples are determined based on user request samples and historical dialogue samples; for each rewriting training sample, when the user request sample does not depend on the historical dialogue samples, the request annotation data corresponding to the input vector sample consists of the stop generation flag and the user request sample; when the user request sample depends on the historical dialogue samples, the request annotation data corresponding to the input vector sample consists of the non-stop generation flag and the rewritten request sample; the rewritten request sample is a request sample after rewriting the user request sample based on the historical dialogue samples.

[0012] According to an intent recognition method provided by the present invention, when the user request sample does not depend on the historical dialogue sample, the stop generation marker is located at the beginning of the sequence of the request annotation data; when the user request sample depends on the historical dialogue sample, the non-stop generation marker is located at the beginning of the sequence of the request annotation data; when the request to be identified output by the request rewriting model is the first rewriting request, the stop generation marker is located at the beginning of the sequence of the first rewriting request; when the request to be identified output by the request rewriting model is the second rewriting request, the non-stop generation marker is located at the beginning of the sequence of the second rewriting request.

[0013] According to an intent recognition method provided by the present invention, before constructing the input vector based on the latest round request and historical dialogue data input by the user in a multi-turn dialogue scenario, the method further includes: adding the stop generation marker and the non-stop generation marker as special lexical units to the lexical file of the request rewriting model, and simultaneously expanding the first dimension of the original weight matrix of the embedding layer of the request rewriting model; and explicitly adding the stop generation marker and the non-stop generation marker as special lexical units of the word segmenter.

[0014] According to an intent recognition method provided by the present invention, the step of recognizing the user's target intent based on the request to be recognized includes: inputting the request to be recognized into a pre-trained intent recognition model to obtain the target intent output by the intent recognition model; the intent recognition model is implemented based on a large-scale language model.

[0015] According to an intent recognition method provided by the present invention, the rewriting training samples include simple referential rewriting training samples and implicit dependency rewriting samples.

[0016] The present invention also provides an intent recognition device, comprising:

[0017] The input vector construction module is used to construct input vectors based on the latest request and historical dialogue data input by the user in a multi-turn dialogue scenario;

[0018] The user request rewriting module is used to input the input vector into a pre-trained request rewriting model to obtain the request to be identified output by the request rewriting model; the request to be identified is a first rewriting request or a second rewriting request.

[0019] The target intent recognition module is used to recognize the user's target intent based on the request to be recognized;

[0020] The request rewriting model is used to determine whether the latest round of requests depends on the historical dialogue data, and outputs the first rewriting request or the second rewriting request based on the determination result; the first rewriting request consists of a stop generation flag and the latest round of requests; the second rewriting request consists of a non-stop generation flag and a rewritten request; the rewritten request is a request based on the historical dialogue data after rewriting the latest round of requests.

[0021] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the intent recognition method as described above.

[0022] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the intent recognition method as described above.

[0023] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the intent recognition method as described above.

[0024] The intent recognition method, apparatus, electronic device, storage medium, and program product provided by this invention simultaneously complete the historical dialogue dependency judgment and request rewriting tasks in a single inference by using a pre-trained request rewriting model. Depending on the result of the historical dialogue dependency judgment, the latest round of requests may or may not be rewritten. This optimizes the two-stage processing flow, which eliminates the interference of unnecessary historical information on the intent recognition of the latest round of requests, into a single step, simplifying the system workflow and avoiding the risk of increased error propagation caused by the two-stage processing. It provides an intent recognition method that requires less computation, is more efficient, and is more accurate. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is one of the flowcharts illustrating the intent recognition method provided by the present invention.

[0027] Figure 2 This is the second flowchart of the intent recognition method provided by the present invention.

[0028] Figure 3 This is a schematic diagram of the intent recognition device provided by the present invention.

[0029] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0031] It should be noted that, in the description of this invention, the term "comprising" or any other variations thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Those skilled in the art will understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0032] The terms "first," "second," etc., used in this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more.

[0033] The following is combined with Figures 1-4 This invention describes the intent recognition method, apparatus, electronic device, storage medium, and program product provided by the present invention.

[0034] LLM-based intent recognition systems are widely applicable to user request parsing and intent determination in multi-turn dialogue scenarios.

[0035] Initially, the approach to intent recognition involved directly concatenating the user's latest request and complete historical dialogue data as the entire context, inputting this into an LLM-based intent recognition system. The system then used an attention mechanism to capture long-range dependencies. However, because this approach inputs the entire context into the intent recognition system, even if the user's latest request is an independent intent unrelated to historical dialogue data, the historical dialogue data is still included as part of the input sequence. This requires the intent recognition system to consume significant computational resources to process redundant information, increasing unnecessary inference resources and resulting in wasted computational resources.

[0036] Furthermore, long contexts can dilute the attention of LLM-based intent recognition systems. When the user's latest request is unrelated to historical dialogue data, redundant information may interfere with the intent recognition of the user's latest request, increasing the difficulty of intent recognition.

[0037] To reduce the computational load and difficulty of intent recognition, a two-stage processing scheme is adopted. This involves designing and training a discriminative model (such as a rule engine or a small classifier) ​​and a rewriting model (such as a pronoun resolution model). The discriminative model determines the dependency of the user's latest request relative to historical dialogues in a multi-turn dialogue scenario. Based on the determination result, the rewriting model is selectively invoked. The rewriting module then rewrites the user's latest request based on historical dialogues. For example, personal pronouns are first detected using regular expression matching, and then the pronoun resolution module is invoked to rewrite the user's latest request.

[0038] However, the two-stage processing scheme prolongs the intent recognition chain, complicates the intent recognition process, increases the risk of error propagation, and makes it difficult to handle complex semantic dependencies.

[0039] Figure 1 This is one of the flowcharts illustrating the intent recognition method provided by the present invention, such as... Figure 1 As shown, the intent recognition method includes, but is not limited to, steps 101 to 103.

[0040] It should be noted that the execution subject of the intent recognition method provided by the present invention is the corresponding intent recognition device, which can be a server, computer equipment, such as mobile phone, tablet computer, laptop computer, handheld computer, vehicle electronic equipment, wearable device, ultra-mobile personal computer (UMPC), netbook or personal digital assistant (PDA), etc.

[0041] Step 101: Construct an input vector based on the latest request and historical dialogue data input by the user in a multi-turn dialogue scenario.

[0042] Specifically, in multi-turn dialogue scenarios, the system receives the latest request from the user and constructs an input vector using the latest request and historical dialogue data (including historical request data from multiple rounds of user input and historical response data from the system).

[0043] For example, the latest user request and historical dialogue data can be directly concatenated to obtain the constructed input vector.

[0044] Step 102: Input the input vector into the pre-trained request rewriting model to obtain the request to be identified output by the request rewriting model.

[0045] The request to be identified is either a first rewrite request or a second rewrite request.

[0046] The request rewriting model is used to determine whether the latest round of requests depends on the historical dialogue data, and outputs the first rewriting request or the second rewriting request based on the determination result; the first rewriting request consists of a stop generation flag and the latest round of requests; the second rewriting request consists of a non-stop generation flag and a rewritten request; the rewritten request is a request based on the historical dialogue data after rewriting the latest round of requests.

[0047] Specifically, the input vector, constructed using the latest request from the user and historical dialogue data, is fed into a pre-trained request rewriting model. When processing the input vector, the request rewriting model has the ability to determine whether the latest request from the user depends on historical dialogue data. Therefore, the request rewriting model can output either the first rewritten request or the second rewritten request as the request to be identified based on the determination result of whether the latest request depends on historical dialogue data.

[0048] Specifically, if the request rewriting model determines that the latest round of requests does not depend on historical dialogue data, that is, the latest round of requests is independent, it outputs a stop generation flag, triggering the request rewriting model to stop rewriting the latest round of requests input by the user, and directly continues to output the latest round of requests after the output stop generation flag, thus obtaining the first rewritten request composed of the stop generation flag and the latest round of requests as the request to be identified.

[0049] If the request rewriting model determines that the latest round of requests depends on historical dialogue data, it outputs a non-stop generation marker. The request rewriting model then rewrites the latest round of requests based on the historical dialogue data to obtain the rewritten request. After outputting the non-stop generation marker, it continues to output the rewritten request, resulting in a second rewritten request consisting of the non-stop generation marker and an independent, complete rewritten request, which is used as the request to be identified.

[0050] Optionally, the request rewriting model is built upon a Small Language Model (SLM). By building the request rewriting model based on SLM, when the request rewriting model determines that the latest round of requests depends on historical dialogue data, after outputting the non-stopping generation flag, the request rewriting model can simultaneously rewrite the latest round of requests and output the rewritten requests, ultimately outputting a complete second rewritten request consisting of the non-stopping generation flag and the rewritten requests.

[0051] As can be seen, the request rewriting model provided by this invention can complete two tasks simultaneously: dependency determination (i.e., whether the latest round of requests depends on historical dialogue data) and request rewriting, through a single model.

[0052] It should be noted that the request rewriting model can rewrite the latest round of requests based on historical dialogue data in various ways, including but not limited to various complete rewriting methods such as eliminating references and completing omissions. This invention does not limit the request rewriting method.

[0053] For example, if the historical dialogue data is "User: Open the animation channel. Assistant: Okay, the animation channel has been opened for you", and the latest request is "How's the weather today?".

[0054] The input vector, concatenated from historical dialogue data and the latest request, is fed into the request rewriting model. The model determines that the user's latest request does not depend on historical dialogue data and outputs the first rewritten request, "[0] How's the weather today?", as the request to be identified. Here, "[0]" is a stop generation marker.

[0055] In other words, during the inference phase, by setting [0] as the stop generation flag, if the request rewriting model determines that the latest round of requests is independent, the rewriting will stop immediately after generating [0], and [0] and the original latest round of requests will be directly passed to the downstream intent recognition, thus truncating the request rewriting model from unnecessary rewriting and improving inference efficiency.

[0056] For example, if the historical dialogue data is "User: I want to watch A's movie. Assistant: Okay, I recommend 'BBB' and 'CCCC' for you", and the latest request is "What songs has he sung?".

[0057] The input vector, which is a concatenation of historical dialogue data and the latest round of requests, is fed into the request rewriting model. The request rewriting model determines that the user's latest round of requests depends on the historical dialogue data and outputs the second rewritten request "[1] What songs has A sung?" as the request to be identified. Here, "[1]" is a non-stopping generation marker.

[0058] Step 103: Based on the request to be identified, identify the user's target intent.

[0059] Specifically, the request rewriting model is used to determine whether the user's latest request depends on the output of the request to be identified after historical dialogue data, thereby identifying the target intent of the user's latest request.

[0060] For example, after the user's latest request and historical dialogue data are concatenated and input into the trained request rewriting model, the request rewriting model determines whether the latest request depends on the historical dialogue data.

[0061] If the latest round of requests is determined to be independent, the first rewrite request, consisting of the stop generation flag and the latest round of requests, is directly used as the request to be identified and input into the downstream intent recognition stage to identify the user's target intent. In this case, there is no need to use historical dialogue data to rewrite the latest round of requests, which can reduce redundant and interfering information in intent recognition.

[0062] If the latest round of requests depends on historical dialogue data, the request rewriting model continues to integrate historical dialogue information to perform the rewriting task after outputting the non-stop generation marker. Based on the historical dialogue data, the latest round of requests is completely rewritten into a rewritten request and output. The resulting second rewritten request, consisting of the non-stop generation marker and the rewritten request, is used as the request to be identified and input into the downstream intent recognition stage to identify the user's target intent.

[0063] The intent recognition method provided by this invention simultaneously completes the historical dialogue dependency judgment and request rewriting tasks in a single inference by using a pre-trained request rewriting model. Depending on the result of the historical dialogue dependency judgment, the latest round of requests is rewritten or not rewritten. This optimizes the two-stage processing flow that eliminates the interference of unnecessary historical information on the intent recognition of the latest round of requests into a single step, simplifying the system workflow and avoiding the risk of increased error propagation caused by the two-stage processing. It provides an intent recognition method that requires less computation, is more efficient, and is more accurate.

[0064] Based on the above embodiments, as an optional embodiment, the request rewriting model is obtained by pre-training based on multiple rewriting training samples;

[0065] The rewritten training samples include input vector samples and their corresponding request annotation data; the input vector samples are determined based on user request samples and historical dialogue samples.

[0066] For each rewritten training sample, when the user request sample does not depend on the historical dialogue sample, the request annotation data corresponding to the input vector sample consists of the stop generation marker and the user request sample; when the user request sample depends on the historical dialogue sample, the request annotation data corresponding to the input vector sample consists of the non-stop generation marker and the rewritten request sample; the rewritten request sample is a request sample that rewrites the user request sample based on the historical dialogue sample.

[0067] Specifically, before using the request rewriting model to actually complete the historical dialogue dependency judgment and request rewriting tasks, it is necessary to build the request rewriting model based on SLM in advance and complete the pre-training of the request rewriting model.

[0068] Before pre-training the request rewriting model, historical multi-turn dialogue data is acquired. For each turn of dialogue, it is necessary to label whether the user request in that turn depends on the historical dialogue information to obtain multiple rewriting training samples. Specifically, for each turn of dialogue, the user request in that turn is used as the user request sample, and the historical dialogue before that turn is used as the historical dialogue sample. Input vector samples are constructed using the user request samples and historical dialogue samples (e.g., by directly concatenating the user request samples and historical dialogue samples). It is further determined whether the user request samples in that turn of dialogue depend on the historical dialogue samples before that turn of dialogue. If they do not depend on each other, the stop generation marker and the user request samples in that turn of dialogue are determined as the request annotation data corresponding to the input vector samples of that turn of dialogue. If they depend on each other, the user request samples are rewritten based on the historical dialogue samples of that turn of dialogue to obtain the rewritten request samples. The non-stop generation marker and the rewritten request samples are determined as the request annotation data corresponding to the input vector samples of that turn of dialogue, and finally, the rewriting training samples for that turn of dialogue are obtained.

[0069] Repeat the steps of obtaining rewrite training samples. After obtaining multiple rewrite training samples, use the multiple rewrite training samples to train the request rewrite model until the accuracy of the request rewrite model meets certain requirements, and obtain the pre-trained request rewrite model.

[0070] For example, if the user request sample is "How's the weather today?", and the historical dialogue sample is "User: Open the animation channel. Assistant: Okay, the animation channel has been opened for you", then the user request sample is independent of the historical dialogue sample. Therefore, the request annotation data corresponding to the input vector sample composed of the user request sample and the historical dialogue sample is "[0] How's the weather today?". Here, "[0]" is the stop generation marker.

[0071] For example, if the user request sample is "What songs has he sung?", and the historical dialogue sample is "User: I want to watch A's movie. Assistant: Okay, I recommend 'BBB' and 'CCCC'", then the user request sample depends on the historical dialogue sample. Therefore, the request annotation data corresponding to the input vector sample composed of the user request sample and the historical dialogue sample is "[1] What songs has A sung?". Among them, "[1]" is a non-stopping generation marker.

[0072] As an optional implementation, negative sample enhancement can be achieved by adding a certain percentage (such as 20%, 30%, 35%, etc.) of noisy historical dialogues to independent user request samples.

[0073] The intent recognition method provided by this invention adds a stop generation marker or non-stop generation marker as a special marker to the request annotation data based on whether the user request sample depends on the historical dialogue sample during the rewriting training sample, forming a visible dependency prompt. This enables the trained request rewriting model to simultaneously complete the two tasks of dependency judgment and complete rewriting, providing a more computationally efficient and accurate intent recognition method.

[0074] Based on the above embodiments, as an optional embodiment, when the user request sample does not depend on the historical dialogue sample, the stop generation flag is located at the beginning of the sequence of the input vector sample;

[0075] When the user request sample depends on the historical dialogue sample, the non-stop generation tag is located at the beginning of the sequence of the input vector sample;

[0076] If the request to be identified output by the request rewriting model is the first rewriting request, the stop generation flag is located at the beginning of the sequence of the first rewriting request;

[0077] When the request to be identified output by the request rewriting model is the second rewriting request, the non-stop generation flag is located at the beginning of the sequence of the second rewriting request.

[0078] Specifically, during the pre-training process of the request rewrite model, the special markers for dependency judgments (i.e., stop generation markers and non-stop generation markers) are always forced to be placed at the beginning of a specific sequence of the requested labeled data in the rewritten training samples, serving as a prefix for the labeled data. This allows the request rewrite model to quickly establish a stable association between the special markers for stop generation markers or non-stop generation markers and the context structure during pre-training. The request rewrite model also does not need to learn the non-fixed occurrence positions of stop generation markers or non-stop generation markers during pre-training. In other words, the request rewrite model does not need to learn more complex position-invariant representations of special markers, which makes the training process faster and easier to converge.

[0079] Furthermore, when building a request rewriting model based on SLM, the Transformer in the model is naturally sensitive to sequence position, and the attention module in the model is prone to "position bias" in the early stage of training. When both the stop generation marker and the non-stop generation marker are located at the beginning of the sequence in the request labeled data, the model can establish a consistent relationship with other positions in the sentence by using the attention weight of the first position during the initial training. This makes it easier for the stop generation marker and the non-stop generation marker to play the role of control characters, instruction prefixes and semantic cues.

[0080] By consistently placing a special marker at the beginning of the sequence of requested labeled data in the rewritten training samples during the training process of the request rewriting model, the pre-trained request rewriting model will, during inference, have a stop generation marker at the beginning of the first rewriting request sequence and a non-stop generation marker at the beginning of the second rewriting request sequence.

[0081] Understandably, when the above design is made based on the decoder architecture model inference method, if the request rewriting model determines that the latest round of requests does not depend on historical dialogue data during inference, the first digit of the output first rewriting request is the stop generation marker (such as "[0]"). At this time, the request rewriting model can immediately stop inference, and the downstream intent model can directly use the current request without waiting for the rewriting result to be output. By stopping the rewriting process in advance, the computational load of the request rewriting model can be reduced, that is, it only needs to calculate the output of the stop generation marker at the beginning of the sequence, without calculating the rewriting result after the output is completed; the downstream intent recognition model does not need to wait for the complete rewriting result, and directly uses the stop generation marker and the latest round of requests as input, thus accelerating the overall process of intent recognition.

[0082] The intent recognition method provided by this invention obtains rewritten training samples by controlling the position of the stop generation marker or the non-stop generation marker to be at the beginning of the sequence during the construction of rewritten training samples. This enables the pre-trained request rewriting model to simultaneously complete the two tasks of dependency judgment and complete rewriting. Furthermore, it can immediately truncate the rewriting process when it is determined that the latest round of requests does not depend on the historical dialogue, thus providing a more computationally efficient and accurate intent recognition method.

[0083] Based on the above embodiments, as an optional embodiment, before constructing the input vector based on the latest round request and historical dialogue data input by the user in a multi-turn dialogue scenario, the method further includes:

[0084] Add the stop generation marker and the non-stop generation marker as special lexical units to the vocabulary file of the request rewriting model, and simultaneously expand the first dimension of the original weight matrix of the embedding layer of the request rewriting model;

[0085] The stop generation marker and the non-stop generation marker are explicitly added as special tokens of the tokenizer.

[0086] Specifically, before constructing the input vector based on the latest request and historical dialogue data input by the user in a multi-turn dialogue scenario, after constructing the request rewriting model based on SLM and before pre-training the request rewriting model, considering that a stop generation token or a non-stop generation token needs to be added to the beginning of the sequence of request labeled data in the rewritten training samples, a stop generation token (such as "[0]") and a non-stop generation token (such as "[1]") are explicitly added to the vocabulary file of the request rewriting model as special tokens, which are used to represent independence and dependency respectively, such as [0] representing independence and [1] representing dependency.

[0087] After expanding the vocabulary of the requested rewrite model, the first-dimensional original weight matrix of the embedding layer of the requested rewrite model is simultaneously expanded to ensure that the stopping and non-stopping generation tokens, as special lexical units, can be learned by the requested rewrite model to have their own semantic representations. Specifically, the size of the first-dimensional original weight matrix of the embedding layer of the requested rewrite model is increased from... Expand to ,in, This is the original vocabulary size. is the embedding vector dimension of the embedding layer. During pre-training, the vector representation of a special token is initialized with random values ​​and is jointly optimized with other word vectors during subsequent training, enabling the model to gradually capture the intrinsic relationship between special tokens and contextual dependencies.

[0088] In addition, stop generation markers and non-stop generation markers will be explicitly added as special terms for the word segmenter.

[0089] The vector parameters of the newly added stop generation token and non-stop generation token in the embedding layer are initially randomly initialized using a Gaussian distribution to ensure that their parameter distribution has the same scale as the original pre-trained weights, thereby avoiding gradient perturbation caused by requesting to rewrite the model in the early stage of training.

[0090] During the training of downstream tasks that request model rewriting, the embedding parameters of the newly added special tokens are not frozen, but participate in gradient backpropagation and joint optimization together with the embedding weights of the original vocabulary. This allows the model to gradually learn the semantic dependencies between the two newly added special tokens and the context during end-to-end training.

[0091] The intent recognition method provided by this invention adds stop generation and non-stop generation tokens as unique tokens exclusive to the model vocabulary, explicitly adds them as special words in the word segmenter, and simultaneously expands the original weight matrix of the embedding layer. Compared to directly using existing tokens to express semantics, adding unique tokens and training their vector representations avoids semantic interference. Furthermore, the position of the unique tokens is fixed at the beginning of the sequence, which helps the request rewriting model converge quickly and improves the performance of the request rewriting model.

[0092] Figure 2 This is a second schematic diagram of the intent recognition method provided by the present invention, as shown below. Figure 2 As shown, identifying the user's intent based on the request to be identified includes:

[0093] The request to be identified is input into a pre-trained intent recognition model to obtain the target intent output by the intent recognition model; the intent recognition model is implemented based on LLM.

[0094] Specifically, an intent recognition model is pre-built based on LLM, and multiple recognition training samples consisting of recognition request samples and their corresponding intent labels are collected. The pre-training of the intent recognition model is completed using multiple recognition training samples.

[0095] During the actual intent recognition of the user's latest request, the concatenated input vector of historical dialogue data and the latest request is fed into the request rewriting model. The model determines whether the latest request depends on historical dialogue data. If it does, the first output of the request rewriting model is a non-stop generation flag. Based on the historical dialogue data, it rewrites the latest request into a rewritten request and outputs the rewritten request. The non-stop generation flag and the rewritten request are then input as an independent and complete second rewritten request into a pre-trained intent recognition model, which identifies the user's target intent in the latest dialogue. If the request rewriting model does not depend on or is independent, the first output of the request rewriting model is a stop generation flag, triggering immediate cessation of rewriting inference. The stop generation flag and the latest request are then input as an independent and complete first rewritten request into a pre-trained intent recognition model, which identifies the user's target intent in the latest dialogue.

[0096] The intent recognition method provided by this invention utilizes an LLM-based intent recognition model to identify the user intent of the latest round of requests. Combined with a request rewriting model that can simultaneously perform historical dependency judgment and request rewriting tasks, it simplifies the system workflow, reduces the risk of error accumulation and propagation, and provides a more efficient and accurate intent recognition method with less computation.

[0097] Based on the above embodiments, as an optional embodiment, the rewritten training samples include simple reference rewritten training samples and implicit dependency rewritten samples.

[0098] Specifically, the rewrite training samples used to pre-train the request rewrite model include simple reference rewrite training samples and implicit dependency rewrite samples.

[0099] Among them, simple pronoun rewriting training samples refer to rewriting training samples that need to complete the mapping from explicit pronouns (such as "he", "she", "it") to entities (such as people, things, etc.); implicit dependency rewriting samples refer to rewriting training samples that need to complete the mapping without explicit pronouns or entities.

[0100] For example, in a simple reference rewriting training sample, the historical dialogue sample is “User: Introduce A; Assistant: A is a famous movie actor, ...", the user request sample is “Has he ever won the Best Actor award?”, and the corresponding request annotation data is “[1]Has A ever won the Best Actor award?”.

[0101] In this simple reference rewriting training sample, it is only necessary to complete the mapping from the pronoun "he" to the person entity "A".

[0102] For example, in a hidden dependency rewriting sample, the historical dialogue sample is "User: Find some animations suitable for eight-year-old children to watch; Assistant: I recommend Frozen, Little Warriors, and Boonie Bears. Which one do you like?" The user request sample is "Is there anything else?", and the corresponding request annotation data is "[1] Besides Frozen, Little Warriors, and Boonie Bears, what other animations are suitable for eight-year-old children to watch?".

[0103] In this implicit dependency rewriting sample, the user request sample used a question / rhetorical question format and did not contain explicit pronouns or entity words, which actually implied the intention of "excluding existing search results and searching for more results according to the original conditions".

[0104] In one embodiment, the ratio between the simple referential rewriting training samples and the implicit dependency rewriting samples is determined based on the downstream business scenario of the intent recognition model.

[0105] For example, the higher the difficulty of rewriting downstream business scenarios, the higher the proportion of implicit dependency rewriting samples in all rewriting training samples.

[0106] The intent recognition method provided by this invention constructs rewriting training samples from simple reference rewriting training samples and implicit dependency rewriting samples with different rewriting difficulties. Then, it pre-trains a request rewriting model using these graded rewriting training samples, which improves the parsing accuracy of complex semantic dependencies. In complete rewriting, it eliminates references and completes omissions, helping to reduce the recognition difficulty of downstream intent recognition models and more accurately understand user intent. Furthermore, by determining the ratio between simple reference rewriting training samples and implicit dependency rewriting samples based on the downstream business scenarios of the intent recognition model, the pre-trained request rewriting model can adapt to intent recognition scenarios with different rewriting difficulties.

[0107] Overall, compared to traditional architectures where dependency information is implicitly encoded in feature representations, the intent recognition method provided by this invention achieves multi-task completion in a single inference, highly interpretable context rewriting, soft linking mechanism, and dynamic stopping mechanism.

[0108] Regarding the completion of multiple tasks in a single inference, by introducing two special tokens, a stop generation token ([0]) and a non-stop generation token ([1]), during the training phase of the request rewrite model, and using one of the special tokens, the stop generation token, as a stop token during the inference phase, it is possible to complete both dependency judgment and complete rewrite tasks in a single inference, thereby improving the efficiency of intent recognition.

[0109] Regarding the highly interpretable context rewriting, by explicitly embedding the stop generation marker ([0]) and the non-stop generation marker ([1]) into the sequence, key markers are formed on the model's "behavioral causal chain," which facilitates the later interpretation and tracing of the generated content, provides clear signals for system decision path log tracking and human understanding, provides a foundation for system debugging, log tracking, and plug-and-play policy configuration, and can have good transparency and controllability in multi-module collaborative systems, which is conducive to industrial-grade deployment and module reuse.

[0110] Regarding the soft link mechanism, compared with the hard control signals (such as if-else) commonly found in traditional modular design, this system uses a special Token[0] as a soft link medium to inject the dependency judgment result into the text sequence, so that the rewriting module can perceive the dependency information in the Transformer layer structure, taking into account both the module decoupling and the efficiency of information transmission.

[0111] Regarding the dynamic stopping mechanism, the early-exit mechanism is implemented by using a request rewriting model based on a small model to output a special token at the first position. This significantly shortens the inference time for request rewriting and intent recognition, improves the system response speed, and gives the intent recognition model stronger context awareness and control capabilities.

[0112] Figure 3This is a schematic diagram of the intent recognition device provided by the present invention, as shown below. Figure 3 As shown, the intent recognition device includes, but is not limited to, an input vector construction module 301, a user request rewriting module 302, and a target intent recognition module 303.

[0113] The input vector construction module 301 is used to construct input vectors based on the latest request and historical dialogue data input by the user in a multi-turn dialogue scenario.

[0114] User request rewriting module 302 is used to input the input vector into a pre-trained request rewriting model to obtain the request to be identified output by the request rewriting model; the request to be identified is a first rewriting request or a second rewriting request.

[0115] The target intent recognition module 303 is used to recognize the user's target intent based on the request to be recognized.

[0116] The request rewriting model is used to determine whether the latest round of requests depends on the historical dialogue data, and outputs the first rewriting request or the second rewriting request based on the determination result; the first rewriting request consists of a stop generation flag and the latest round of requests; the second rewriting request consists of a non-stop generation flag and a rewritten request; the rewritten request is a request based on the historical dialogue data after rewriting the latest round of requests.

[0117] It should be noted that the intent recognition device provided by the present invention can execute the intent recognition method described in any of the above embodiments during specific operation, and this embodiment will not elaborate on this.

[0118] The intent recognition device provided by this invention simultaneously completes historical dialogue dependency judgment and request rewriting tasks in a single inference by using a pre-trained request rewriting model. Depending on the result of the historical dialogue dependency judgment, the latest round of requests is rewritten or not rewritten. This optimizes the two-stage processing flow that eliminates the interference of unnecessary historical information on the intent recognition of the latest round of requests into a single step, simplifying the system workflow and avoiding the risk of increased error propagation caused by the two-stage processing. It provides an intent recognition device with less computation, higher efficiency, and higher accuracy.

[0119] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communications bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other through the communications bus 440. The processor 410 can call logical instructions in the memory 430 to execute the intent recognition method provided in any of the above embodiments. The intent recognition method includes, but is not limited to, the following steps: constructing an input vector based on the latest request and historical dialogue data input by the user in a multi-turn dialogue scenario; inputting the input vector into a pre-trained request rewriting model to obtain the request to be recognized output by the request rewriting model; the request to be recognized is a first rewritten request or a second rewritten request; identifying the user's target intent based on the request to be recognized; the request rewriting model is used to determine whether the latest request depends on the historical dialogue data, and outputs the first rewritten request or the second rewritten request according to the determination result; the first rewritten request consists of a stop generation flag and the latest request; the second rewritten request consists of a non-stop generation flag and a rewritten request; the rewritten request is a request after rewriting the latest request based on the historical dialogue data.

[0120] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the intent recognition method provided in any of the above embodiments. The intent recognition method includes, but is not limited to, the following steps: constructing an input vector based on the latest round request and historical dialogue data input by the user in a multi-turn dialogue scenario; inputting the input vector into a pre-trained request rewriting model to obtain a request to be recognized output by the request rewriting model; the request to be recognized is a first rewritten request or a second rewritten request; recognizing the user's target intent based on the request to be recognized; the request rewriting model is used to determine whether the latest round request depends on the historical dialogue data, and outputs the first rewritten request or the second rewritten request according to the determination result; the first rewritten request consists of a stop generation flag and the latest round request; the second rewritten request consists of a non-stop generation flag and a rewritten request; the rewritten request is a request after rewriting the latest round request based on the historical dialogue data.

[0122] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the intent recognition method provided in any of the above embodiments. The intent recognition method includes, but is not limited to, the following steps: constructing an input vector based on the latest round request and historical dialogue data input by the user in a multi-turn dialogue scenario; inputting the input vector into a pre-trained request rewriting model to obtain a request to be recognized output by the request rewriting model; the request to be recognized is a first rewritten request or a second rewritten request; recognizing the user's target intent based on the request to be recognized; the request rewriting model is used to determine whether the latest round request depends on the historical dialogue data, and outputs the first rewritten request or the second rewritten request according to the determination result; the first rewritten request consists of a stop generation flag and the latest round request; the second rewritten request consists of a non-stop generation flag and a rewritten request; the rewritten request is a request after rewriting the latest round request based on the historical dialogue data.

[0123] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0124] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intent recognition method, characterized in that, include: Based on the latest request and historical dialogue data input by the user in a multi-turn dialogue scenario, construct an input vector; The input vector is fed into a pre-trained request rewriting model to obtain the request to be identified output by the request rewriting model. The request to be identified is either a first rewrite request or a second rewrite request; Based on the request to be identified, the user's target intent is identified; The request rewriting model is used to determine whether the latest round of requests depends on the historical dialogue data, and outputs the first rewriting request or the second rewriting request based on the determination result; the first rewriting request consists of a stop generation flag and the latest round of requests; the second rewriting request consists of a non-stop generation flag and a rewritten request; the rewritten request is a request based on the historical dialogue data after rewriting the latest round of requests. The request rewriting model is pre-trained based on multiple rewriting training samples; the rewriting training samples include input vector samples and their corresponding request annotation data; the input vector samples are determined based on user request samples and historical dialogue samples. For each rewritten training sample, when the user request sample does not depend on the historical dialogue sample, the request annotation data corresponding to the input vector sample consists of the stop generation marker and the user request sample; when the user request sample depends on the historical dialogue sample, the request annotation data corresponding to the input vector sample consists of the non-stop generation marker and the rewritten request sample; the rewritten request sample is a request sample that rewrites the user request sample based on the historical dialogue sample.

2. The intent recognition method according to claim 1, characterized in that, When the user request sample does not depend on the historical dialogue sample, the stop generation marker is located at the beginning of the sequence of the request annotation data; When the user request sample depends on the historical dialogue sample, the non-stop generation tag is located at the beginning of the sequence of the request annotation data; If the request to be identified output by the request rewriting model is the first rewriting request, the stop generation flag is located at the beginning of the sequence of the first rewriting request; When the request to be identified output by the request rewriting model is the second rewriting request, the non-stop generation flag is located at the beginning of the sequence of the second rewriting request.

3. The intent recognition method according to claim 1, characterized in that, Before constructing the input vector based on the latest request and historical dialogue data input by the user in a multi-turn dialogue scenario, the following is also included: Add the stop generation marker and the non-stop generation marker as special terms to the vocabulary file of the request rewriting model, and simultaneously expand the first dimension of the original weight matrix of the embedding layer of the request rewriting model; The stop generation marker and the non-stop generation marker are explicitly added as special tokens of the word segmenter.

4. The intent recognition method according to claim 1, characterized in that, The step of identifying the user's target intent based on the request to be identified includes: The request to be identified is input into a pre-trained intent recognition model to obtain the target intent output by the intent recognition model; the intent recognition model is implemented based on a large language model.

5. The intent recognition method according to claim 4, characterized in that, The rewritten training samples include simple reference rewritten training samples and implicit dependency rewritten samples.

6. An intent recognition device, characterized in that, include: The input vector construction module is used to construct input vectors based on the latest request and historical dialogue data input by the user in a multi-turn dialogue scenario; The user request rewriting module is used to input the input vector into a pre-trained request rewriting model to obtain the request to be identified output by the request rewriting model; the request to be identified is a first rewriting request or a second rewriting request. The target intent recognition module is used to recognize the user's target intent based on the request to be recognized; The request rewriting model is used to determine whether the latest round of requests depends on the historical dialogue data, and outputs the first rewriting request or the second rewriting request based on the determination result; the first rewriting request consists of a stop generation flag and the latest round of requests; the second rewriting request consists of a non-stop generation flag and a rewritten request; the rewritten request is a request based on the historical dialogue data after rewriting the latest round of requests. The request rewriting model is pre-trained based on multiple rewriting training samples; the rewriting training samples include input vector samples and their corresponding request annotation data; the input vector samples are determined based on user request samples and historical dialogue samples. For each rewritten training sample, when the user request sample does not depend on the historical dialogue sample, the request annotation data corresponding to the input vector sample consists of the stop generation marker and the user request sample; when the user request sample depends on the historical dialogue sample, the request annotation data corresponding to the input vector sample consists of the non-stop generation marker and the rewritten request sample; the rewritten request sample is a request sample that rewrites the user request sample based on the historical dialogue sample.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the intent recognition method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the intent recognition method as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the intent recognition method as described in any one of claims 1 to 5.

Images