Training method of intent recognition model, intent recognition method and device

Abstract

The application provides a training method and device of an intent recognition model, and an intent recognition method and device. The method comprises: obtaining a training sample set; a sample in the training sample set comprises a question sentence labeled with an intent label; a pre-training model is trained using the sample in the training sample set; an initial output vector corresponding to the sample is obtained; an initial back propagation gradient of the pre-training model is determined according to the initial output vector; a preset number of disturbances are added to the initial output vector based on the initial back propagation gradient to obtain a target back propagation gradient; and a model parameter of the pre-training model is updated according to the target back propagation gradient to obtain an intent recognition model. The application uses the gradient of the back propagation in the pre-training model to perform adversarial disturbance on the output vector corresponding to the model sample, i.e. the word embedding layer vector. This adversarial training method can improve the robustness of the model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method for an intent recognition model, an intent recognition method, and an apparatus. Background Technology

[0002] In existing technologies, intent recognition is typically handled using intent recognition models within chatbot modules. This primarily involves determining the user's intent through their text input and is generally designed as a classification model. Since chatbots deal with everyday scenarios such as customer service, sales, and follow-up calls, user text input is often conversational. This presents a significant challenge to the model, as the same intent can be expressed in many different ways, requiring the intent recognition model to have strong generalization capabilities.

[0003] Currently, in order to improve the generalization ability of the model, it is common practice to manually add noise interference to the text content (such as replacing typos, synonyms, adding or removing modal particles, adverbs, etc.) for adversarial training. However, the noise interference added in this way is limited, and there may be situations where the perturbation is too large, which is not conducive to adversarial training. Therefore, it still cannot effectively improve the generalization ability of the intent recognition model. Summary of the Invention

[0004] The purpose of this application is to provide a training method, an intent recognition method and apparatus for an intent recognition model. By utilizing the gradient of backpropagation in a pre-trained language model, adversarial perturbation is applied to the output vector corresponding to the model sample, i.e., the word embedding layer vector. This adversarial training method can improve the robustness of the model.

[0005] In a first aspect, embodiments of this application provide a training method for an intent recognition model. The method includes: acquiring a training sample set; the samples in the training sample set include: questions labeled with intent tags; training a pre-trained model using the samples in the training sample set; acquiring an initial output vector corresponding to the sample; determining an initial backpropagation gradient of the pre-trained model based on the initial output vector; adding a preset number of perturbations to the initial output vector based on the initial backpropagation gradient to obtain a target backpropagation gradient; and updating the model parameters of the pre-trained model based on the target backpropagation gradient to obtain an intent recognition model.

[0006] In a preferred embodiment of this application, the step of adding a preset number of perturbations to the initial output vector based on the initial backpropagation gradient to obtain the target backpropagation gradient includes: using the initial backpropagation gradient as the current gradient, performing the following perturbation operations: calculating the current perturbation vector based on the current gradient; using the sum of the current perturbation vector and the initial output vector as the target output vector; determining the current gradient increment based on the target output vector and the current number of perturbations; updating the current gradient with the current gradient increment and the sum of the current gradients, and continuing to perform the perturbation operations until the current number of perturbations reaches the preset number, and then using the current gradient increment and the sum of the current gradients as the target backpropagation gradient.

[0007] In a preferred embodiment of this application, the step of calculating the current perturbation vector based on the current gradient includes: calculating the current perturbation vector according to the following formula:

[0008] ;

[0009] in, Indicates the current perturbation vector; This represents a pre-defined positive number, a very small positive number; Represents a symbolic function; Indicates the current gradient; Indicates the initial output vector; Represents the loss function; These represent the model parameters, model input vector, and model output vector, respectively.

[0010] In a preferred embodiment of this application, the step of determining the current gradient increment based on the target output vector and the current perturbation number includes: calculating the target loss value based on the target output vector and the loss function; performing forward and backward propagation based on the target loss value to obtain the target gradient; and calculating the current gradient increment using the following formula:

[0011] ;

[0012] in, This represents the current gradient increment. Represents the target gradient; Represents the target output vector; Indicates the attenuation factor; This indicates the current number of disturbances.

[0013] In a preferred embodiment of this application, the step of determining the initial backpropagation gradient of the pre-trained model based on the initial output vector includes: calculating the initial loss value corresponding to the sample based on the initial output vector and the loss function; and performing backpropagation based on the initial loss value to obtain the initial backpropagation gradient of the pre-trained model.

[0014] In a preferred embodiment of this application, the step of obtaining the training sample set includes: obtaining multiple user input texts from the dialogue system log; preprocessing the multiple user input texts to obtain multiple target texts expressing user intent; and labeling each target text with an intent tag to obtain a training sample set containing multiple questions labeled with intent tags.

[0015] In a preferred embodiment of this application, after the step of updating the model parameters of the pre-trained model according to the target backpropagation gradient, the method further includes: obtaining a validation sample set; validating the model with updated parameters using the validation sample set; and using the model with updated parameters as the intent recognition model when the validation result meets specified conditions.

[0016] Secondly, embodiments of this application also provide an intent recognition method, the method comprising: acquiring text to be recognized; inputting the text to be recognized into a preset intent recognition model; the intent recognition model being a model trained using the method described in the first aspect; and outputting an output vector representing the intent corresponding to the text to be recognized through the intent recognition model.

[0017] Thirdly, embodiments of this application also provide a training apparatus for an intent recognition model. The apparatus includes: a sample set acquisition module for acquiring a training sample set; the samples in the training sample set include: questions labeled with intent tags; an output vector acquisition module for training a pre-trained model using samples from the training sample set to acquire an initial output vector corresponding to the sample; a gradient determination module for determining the initial backpropagation gradient of the pre-trained model based on the initial output vector; a perturbation module for adding a preset number of perturbations to the initial output vector based on the initial backpropagation gradient to obtain a target backpropagation gradient; and a parameter update module for updating the model parameters of the pre-trained model based on the target backpropagation gradient to obtain an intent recognition model.

[0018] Fourthly, embodiments of this application also provide an intent recognition device, the device comprising: a text acquisition module for acquiring text to be recognized; a text input module for inputting the text to be recognized into a preset intent recognition model; the intent recognition model is a model trained using the method described in the second aspect; and a model prediction module for predicting the text to be recognized through the intent recognition model and outputting an output vector representing the intent corresponding to the text to be recognized.

[0019] The intention recognition model training method, intention recognition method, and apparatus provided in this application first acquire a training sample set. The samples in this training sample set include: questions labeled with intention tags; the pre-trained model is trained using samples from the training sample set; the initial output vector corresponding to the sample is acquired; the initial backpropagation gradient of the pre-trained model is determined based on the initial output vector; based on the initial backpropagation gradient, the initial output vector is perturbed a preset number of times to obtain the target backpropagation gradient; the model parameters of the pre-trained model are updated based on the target backpropagation gradient to obtain the intention recognition model. This application utilizes the backpropagation gradient in the pre-trained language model to adversarially perturb the output vector corresponding to the model samples, i.e., the word embedding layer vector. This adversarial training method can improve the model's robustness, thus enabling the model to have strong generalization ability for input colloquial text. Furthermore, during model training, since only a small amount of labeled data is needed and it does not rely on external data resources, compared to traditional models, it does not require a large amount of manual labeling costs, reducing training time and making it highly practical. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating a training method for an intent recognition model provided in an embodiment of this application;

[0022] Figure 2 A flowchart illustrating another training method for an intent recognition model provided in an embodiment of this application;

[0023] Figure 3 A flowchart illustrating an intent recognition method provided in this application embodiment;

[0024] Figure 4 A structural block diagram of a training device for an intent recognition model provided in an embodiment of this application;

[0025] Figure 5 A structural block diagram of an intent recognition device provided in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solution of the present application will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0028] In the prior art, in order to improve the generalization ability of the model, the method of adding noise interference to the text content manually is usually adopted for adversarial training. For example, a way to generate adversarial samples is to add noise interference to the text content manually, and the forms are as follows: 1. Replace misspelled words, for example: change "I haven't taken the third dose of the vaccine" to "I haven't taken the di third dose of the vaccine"; 2. Add modal particles, adverbs, etc., for example: change "Where can I operate the APP download" to "Where can I operate the APP download"; 3. Reduce particles, adverbs, etc., for example: change "Where can I operate the APP download" to "Where can I operate the APP download"; 4. Replace synonyms, for example: change "car insurance quote" to "car insurance price calculation". The noise interference added by this method is limited, and there may be a situation where the perturbation is too large and not conducive to adversarial training, and it still cannot well improve the generalization ability of the intent recognition model.

[0029] Based on this, the embodiments of the present application provide a training method, an intent recognition method and a device for an intent recognition model, which use the gradient of backpropagation in the pre-trained language model to perform adversarial perturbation on the output vector corresponding to the model sample, that is, the word embedding layer vector. This adversarial training method can improve the robustness of the model.

[0030] For the convenience of understanding this embodiment, a training method for an intent recognition model disclosed in the embodiments of the present application will be introduced in detail first.

[0031] Figure 1 FIG. is a flowchart of a training method for an intent recognition model provided by an embodiment of the present application. The method specifically includes the following steps:

[0032] Step S102, obtain a training sample set; the samples in the training sample set include: questions marked with intent labels.

[0033] For example, user input text, i.e., various user-entered questions, is collected from the logs of the dialogue system. Because the raw data contains a lot of noise, such as meaningless special characters, spaces, and garbled characters, regular expressions are used for cleanup. Excessively long dialogue text is truncated. Spoken expressions often contain semantically incomplete or syntactically incorrect information; dependency parsing and part-of-speech tagging are used to filter this information, leaving only the text that clearly and completely expresses the user's intent. Then, the intent is manually labeled. The intent classification system is related to the specific application scenario; for example, common intents in the insurance field include: "buy insurance," "cancel insurance," "check policy," and "policy loan," etc.

[0034] Step S104: Use samples from the training sample set to train the pre-trained model and obtain the initial output vector corresponding to the sample; that is, input the sample into the pre-trained model and output the output vector corresponding to the sample through the pre-trained model, which is the word embedding layer vector.

[0035] Step S106: Determine the initial backpropagation gradient of the pre-trained model based on the initial output vector.

[0036] In practice, the initial loss value corresponding to the sample can be calculated first based on the initial output vector and the loss function; then backpropagation can be performed based on the initial loss value to obtain the initial backpropagation gradient of the pre-trained model.

[0037] Step S108: Based on the initial backpropagation gradient, the initial output vector is perturbed a preset number of times to obtain the target backpropagation gradient.

[0038] The preset number of times can be set differently according to the actual situation. The specific process of increasing the disturbance includes multiple loop operations, which will be described in detail later.

[0039] Step S110: Update the model parameters of the pre-trained model according to the target backpropagation gradient to obtain the intent recognition model.

[0040] It should be noted that during the training process of the above model, a preset number of training iterations can be set for each sample. The final model can be determined based on the model's predictive performance on the validation set or the number of training iterations for the samples. For example, when the model's predictive performance reaches a certain value or the number of training iterations reaches a certain value, the model training ends, and the intent recognition model is obtained.

[0041] The training method for the intent recognition model provided in this application utilizes the gradient from backpropagation in the pre-trained language model to adversarially perturb the output vector corresponding to the model sample, i.e., the word embedding layer vector. This adversarial training method can improve the robustness of the model, thus enabling it to have strong generalization ability for colloquial input text. Furthermore, since the model training process requires only a small amount of labeled data and does not rely on external data resources, it reduces training time compared to traditional models by eliminating the need for extensive manual labeling and making it highly practical.

[0042] Based on the above method embodiments, this application also provides a training method for an intent recognition model. This method is implemented based on the previous embodiment. In this application embodiment, the model training part is divided into six parts in sequence: system-related text data collection, data processing, preparation of training and validation sets, preparation of the open-source pre-trained model BERT, model training, and output of model files. This embodiment focuses on describing the method for determining the backpropagation gradient during the model training process, as well as the method for adding perturbations.

[0043] 1. Collect system-related text data, such as obtaining multiple user input texts from the dialogue system logs.

[0044] 2. Data processing: The collected raw data contains a lot of noise, such as meaningless special characters, spaces, and garbled characters. Preprocessing is required, which means preprocessing multiple user input texts to obtain multiple target texts that express the user's intent.

[0045] The specific preprocessing process is as follows: For example, regular expressions are used for cleaning, and dialogue text that is too long is truncated; in colloquial expressions often contain semantic incompleteness or syntactic errors, so dependency parsing, part-of-speech recognition and other methods are used to filter out the text that can fully and clearly express the user's intent.

[0046] Then, each target text is labeled with an intent tag, resulting in a training sample set containing multiple questions labeled with intent tags. The annotation of intent tags needs to be done manually, and the intent classification system is related to the specific application scenario. For example, commonly used intents in the insurance field include: "buy insurance", "cancel insurance", "inquire about policy", "policy loan", etc.

[0047] For example, the question "Where can I see [name]?" corresponds to the intent tag "sales channel"; another example is the question "Do you currently discontinue [name]?", which corresponds to the intent tag "sales status", and so on. Here, [name] represents the product name.

[0048] 3. Preparation of training and validation sets: Use a Python script to divide the processed dataset into appropriately proportioned sets. The training set is used for model training, and the validation set is used to verify the model's performance during training.

[0049] 4. Preparing the open-source pre-trained model BERT: Download the BERT open-source pre-trained model file from the official website. BERT stands for Bidirectional Encoder Representations from Transformer. It is a pre-trained language representation model that emphasizes a departure from traditional unidirectional language models or shallow concatenation of two unidirectional language models for pre-training. Instead, it employs a new masked language model (MLM) to generate deep bidirectional language representations.

[0050] 5. During model training, a validation set is used to evaluate the training effectiveness. For example, a validation sample set is obtained; the model with updated parameters is validated using the validation sample set; when the validation results meet specified conditions, the model with updated parameters is used as the intent recognition model. The validation results can be the model's prediction accuracy or other metrics.

[0051] 6. Output of template and model files: After training is complete, save the trained model parameters in binary format for use in the online system for intent recognition.

[0052] The model training process in step 5 above will be explained in detail below:

[0053] For each sample, perform the following steps:

[0054] Step S201: Train the pre-trained model using samples to obtain the initial output vector corresponding to the sample; that is, input the sample into the pre-trained model, and the pre-trained model outputs the output vector corresponding to the sample, i.e., the word embedding layer vector. .

[0055] Step S202: Calculate the initial loss value corresponding to the sample based on the initial output vector and the loss function. and loss function Calculate the loss value .

[0056] Step S203: Perform backpropagation based on the initial loss value to obtain the initial backpropagation gradient of the pre-trained model. (Loss value) Perform backpropagation to obtain the initial backpropagation gradient of the pre-trained model. .

[0057] Step S204: Using the initial backpropagation gradient as the current gradient, perform the following perturbation operation:

[0058] Step S2042: Calculate the current perturbation vector based on the current gradient.

[0059] Specifically, the current disturbance vector is calculated according to the following formula:

[0060] ;

[0061] in, Indicates the current perturbation vector; This represents a pre-defined positive number, a very small positive number; Represents a symbolic function; Indicates the current gradient; Represents the loss function; These represent the model parameters, model input vector, and model output vector, respectively.

[0062] Step S2044: Use the sum of the current perturbation vector and the initial output vector as the target output vector. That is... .

[0063] Step S2046: Determine the current gradient increment based on the target output vector and the current number of perturbations.

[0064] Based on the target output vector and loss function Calculate the target loss value Based on the target loss value Perform forward and backward propagation to obtain the target gradient. The current gradient increment is calculated using the following formula:

[0065] ;

[0066] in, This represents the current gradient increment. Represents the target gradient; Indicates the attenuation factor; This indicates the current number of disturbances.

[0067] Step S2048: Update the current gradient with the current gradient increment and the sum of the current gradients, and continue to perform the perturbation operation until the current perturbation count reaches the preset number. Then, use the current gradient increment and the sum of the current gradients as the target backpropagation gradient.

[0068] For example, before the disturbance, by gradient That is, the initial gradient is ;

[0069] Depend on The current disturbance vector can be calculated. , ;Depend on The gradient can be calculated. ;Depend on The gradient increment is calculated using the number of perturbations. Then, after the first perturbation, the gradient is: After the second perturbation, the gradient is: By analogy, the target backpropagation gradient after a preset number of perturbations can be determined.

[0070] Step S205: Update the model parameters of the pre-trained model according to the backpropagation gradient of the target to obtain the intent recognition model.

[0071] The intent recognition model training method provided in this application utilizes the gradient from backpropagation in the BERT pre-trained language model to adversarially perturb the output vector corresponding to the model sample, i.e., the word embedding layer vector. This adversarial training method can improve the robustness of the model, thus enabling the model to have strong generalization ability for colloquial input text. Furthermore, since the model training process requires only a small amount of labeled data and does not rely on external data resources, it reduces the need for extensive manual annotation compared to traditional models, thus shortening training time and making it highly practical.

[0072] Based on the above-described training method embodiments for the intent recognition model, this application also provides an intent recognition method, see [link to relevant documentation]. Figure 3 As shown, the method specifically includes the following steps:

[0073] Step S302: Obtain the text to be recognized;

[0074] Step S304: Input the text to be recognized into the preset intent recognition model; the intent recognition model is a model trained using the intent recognition model training method.

[0075] Step S306: Output an output vector representing the intent corresponding to the text to be recognized through the intent recognition model.

[0076] Based on the above-described training method embodiments for the intent recognition model, this application also provides a training apparatus for the intent recognition model, see [link to relevant documentation]. Figure 4 As shown, the device includes:

[0077] The sample set acquisition module 41 is used to acquire a training sample set; the samples in the training sample set include: questions labeled with intent tags; the output vector acquisition module 42 is used to train the pre-trained model using samples from the training sample set and acquire the initial output vector corresponding to the sample; the gradient determination module 43 is used to determine the initial backpropagation gradient of the pre-trained model based on the initial output vector; the perturbation module 44 is used to add a preset number of perturbations to the initial output vector based on the initial backpropagation gradient to obtain the target backpropagation gradient; the parameter update module 45 is used to update the model parameters of the pre-trained model based on the target backpropagation gradient to obtain the intent recognition model.

[0078] In a preferred embodiment of this application, the perturbation module 44 is configured to: use the initial backpropagation gradient as the current gradient and perform the following perturbation operations: calculate the current perturbation vector based on the current gradient; use the sum of the current perturbation vector and the initial output vector as the target output vector; determine the current gradient increment based on the target output vector and the current perturbation count; update the current gradient with the current gradient increment and the sum of the current gradient, and continue to perform the perturbation operation until the current perturbation count reaches a preset number, and then use the current gradient increment and the sum of the current gradient as the target backpropagation gradient.

[0079] In a preferred embodiment of this application, the disturbance module 44 is used to calculate the current disturbance vector according to the following formula:

[0080] ;

[0081] in, Indicates the current perturbation vector; This represents a pre-defined positive number, a very small positive number; Represents a symbolic function; Indicates the current gradient; Indicates the initial output vector; Represents the loss function; These represent the model parameters, model input vector, and model output vector, respectively.

[0082] In a preferred embodiment of this application, the perturbation module 44 is configured to: calculate the target loss value based on the target output vector and the loss function; perform forward and backward propagation based on the target loss value to obtain the target gradient; and calculate the current gradient increment using the following formula:

[0083] ;

[0084] in, This represents the current gradient increment. Represents the target gradient; Represents the target output vector; Indicates the attenuation factor; This indicates the current number of disturbances.

[0085] In a preferred embodiment of this application, the gradient determination module 43 is used to calculate the initial loss value corresponding to the sample based on the initial output vector and the loss function; and to perform backpropagation based on the initial loss value to obtain the initial backpropagation gradient of the pre-trained model.

[0086] In a preferred embodiment of this application, the sample set acquisition module 41 is used to acquire multiple user input texts from the dialogue system log; preprocess the multiple user input texts to obtain multiple target texts expressing user intent; and label each target text with an intent tag to obtain a training sample set containing multiple questions labeled with intent tags.

[0087] In a preferred embodiment of this application, the above-mentioned apparatus further includes a verification module, configured to: acquire a verification sample set; verify the model with updated parameters using the verification sample set; and when the verification result meets specified conditions, use the model with updated parameters as the intent recognition model.

[0088] The device provided in this application embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts of the device embodiment not mentioned can be referred to the corresponding content in the aforementioned method embodiment.

[0089] Based on the above embodiments of the intent recognition method, this application also provides an intent recognition device, see [link to relevant documentation]. Figure 5 As shown, the device includes:

[0090] The text acquisition module 51 is used to acquire the text to be recognized; the text input module 52 is used to input the text to be recognized into a preset intent recognition model; the intent recognition model is a model trained by the method described in the second aspect; the model prediction module 53 is used to predict the text to be recognized through the intent recognition model and output an output vector representing the intent corresponding to the text to be recognized.

[0091] The device provided in this application embodiment has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts of the device embodiment not mentioned can be referred to the corresponding content in the aforementioned method embodiment.

[0092] This application also provides an electronic device, such as... Figure 6 The diagram shows the structure of the electronic device, which includes a processor 61 and a memory 60. The memory 60 stores computer-executable instructions that can be executed by the processor 61, and the processor 61 executes the computer-executable instructions to implement the above-described method.

[0093] exist Figure 6 In the illustrated embodiment, the electronic device further includes a bus 62 and a communication interface 63, wherein the processor 61, the communication interface 63, and the memory 60 are connected via the bus 62.

[0094] The memory 60 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 63 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 62 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 62 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0095] Processor 61 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 61 or by instructions in software form. Processor 61 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory, and the processor 61 reads the information in the memory and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiment.

[0096] This application also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions cause the processor to implement the above-described method. For specific implementation details, please refer to the foregoing method embodiments, which will not be repeated here.

[0097] The computer program products of the methods, apparatus, and electronic devices provided in the embodiments of this application include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementations, please refer to the method embodiments, which will not be repeated here.

[0098] Unless otherwise specifically stated, the relative steps, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of this application.

[0099] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] In the description of this application, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0101] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the technical scope disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for training an intent recognition model, the method comprising: The method comprises: obtaining a training sample set; samples in the training sample set comprise: a question sentence labeled with an intention label; training a pre-training model using the samples in the training sample set; obtaining an initial output vector corresponding to the sample; determining an initial back propagation gradient of the pre-training model according to the initial output vector; based on the initial back propagation gradient, adding a preset number of perturbations to the initial output vector to obtain a target back propagation gradient; updating the model parameters of the pre-training model according to the target back propagation gradient to obtain an intention recognition model; the step of obtaining the target back propagation gradient based on the initial back propagation gradient, adding a preset number of perturbations to the initial output vector, comprising: taking the initial back propagation gradient as the current gradient, and performing the following perturbation operation: calculate the current perturbation vector according to the current gradient; taking the sum of the current perturbation vector and the initial output vector as the target output vector; determine the current gradient increment according to the target output vector and the current perturbation number; update the current gradient by taking the sum of the current gradient increment and the current gradient, continue to execute the perturbation operation, and when the current perturbation number reaches the preset number, take the sum of the current gradient increment and the current gradient as the target back propagation gradient; the step of calculating the current perturbation vector according to the current gradient, comprising: calculate the current perturbation vector according to the following formula: ； wherein, denotes the current perturbation vector; denotes a predetermined positive number, a very small positive number; denotes a sign function; denotes the current gradient; denotes the initial output vector; denotes the loss function; denote a model parameter, a model input vector and a model output vector, respectively; the step of determining the current gradient increment according to the target output vector and the current perturbation number, comprising: calculate the target loss value according to the target output vector and the loss function; based on the target loss value, perform forward and backward propagation processing to obtain the target gradient; calculate the current gradient increment by using the following formula: ； wherein, denotes the current gradient increment, denotes the target gradient; denotes the target output vector; denotes the decay factor; denotes the current disturbance number.

2. The method of claim 1, wherein, the step of determining the initial back propagation gradient of the pre-training model according to the initial output vector, comprising: calculate the initial loss value corresponding to the sample according to the initial output vector and the loss function; perform back propagation according to the initial loss value to obtain the initial back propagation gradient of the pre-training model.

3. The method of claim 1, wherein, the step of obtaining the training sample set, comprising: obtain a plurality of user input texts from the dialogue system log; preprocess a plurality of the user input texts to obtain a plurality of target texts expressing user intentions; label the intention label for each target text to obtain a training sample set containing a plurality of question sentences labeled with intention labels.

4. The method of claim 1, wherein, after the step of updating the model parameters of the pre-training model according to the target back propagation gradient, further comprising: obtaining a verification sample set; verify the model with updated parameters through the verification sample set; when the verification result meets the specified condition, the model with updated parameters is taken as the intention recognition model.

5. An intention recognition method characterized by, the method comprises: obtaining a to-be-recognized text; input the to-be-recognized text into a preset intention recognition model; the intention recognition model is a model trained by the method of any one of claims 1-4; output an output vector for representing the intention corresponding to the to-be-recognized text through the intention recognition model. 6.A device for training an intention recognition model, comprising: the device for implementing the training method of the intention recognition model in any one of claims 1-4, the device comprises: The sample set obtaining module is configured to obtain a training sample set; samples in the training sample set include a question sentence labeled with an intention label; The output vector obtaining module is configured to train a pre-training model by using the samples in the training sample set, and obtain an initial output vector corresponding to the samples. The gradient determining module is configured to determine an initial back propagation gradient of the pre-training model according to the initial output vector. The perturbation module is configured to add a preset number of perturbations to the initial output vector based on the initial back propagation gradient, and obtain a target back propagation gradient. The parameter updating module is configured to update model parameters of the pre-training model according to the target back propagation gradient, and obtain an intention recognition model.

7. An intention recognition apparatus characterized by comprising: The device comprises: The text obtaining module is configured to obtain a text to be recognized. The text input module is configured to input the text to be recognized into a preset intention recognition model; the intention recognition model is a model trained by using the method in any one of claims 1-4. The model prediction module is configured to predict the text to be recognized by using the intention recognition model, and output an output vector used to represent an intention corresponding to the text to be recognized.

Images