A learning method, device and equipment for intention recognition model

Abstract

A learning method, device and equipment for an intent recognition model, relating to the field of communication technology, is beneficial to improving the accuracy of the intent recognition model in a human-computer dialogue system, improving the accuracy of the human-computer dialogue system in executing tasks, and improving the user experience. The method comprises: a server receives positive data corresponding to a first skill input by a skill developer (S101); the server generates negative data corresponding to the first skill based on the positive data corresponding to the first skill (S102); the server determines a second skill similar to the first skill (S103); the server obtains data corresponding to each second skill (S104); the server generates a second base model based on the data corresponding to the second skill and the first base model stored in the server (S105); the server learns based on the positive data and negative data corresponding to the first skill and the second base model (S106) to generate an intent recognition model.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a learning method, device and equipment for an intent recognition model. Background Art

[0002] The human-computer dialogue system, also known as the human-computer dialogue platform, chatbot, etc., is a new generation of human-computer interaction interface. Specifically, the human-computer dialogue system is divided into open-domain chatbots and task-oriented chatbots according to the fields involved.

[0003] Among them, task-oriented chatbots can provide terminal users with services such as ordering food, booking tickets, and taking taxis. For example, the providers of these services input some training data corresponding to function A (such as user statements, also known as corpus) into the server in advance, and the server uses the ensemble learning method to train a model corresponding to function A based on the input training data. The model corresponding to function A can be used to predict new user statements input by the terminal user to determine the user's intention, that is, whether the server provides the service corresponding to function A to the terminal user.

[0004] It should be noted that in the process of server integrated learning, the server uses some pre-set base learners (also called base models) to learn the training data input by the service provider, and uses certain rules to integrate the models trained by each base learner, thereby obtaining a more accurate model than a single base learner.

[0005] It can be seen that the richness and accuracy of the training data input by the service provider, as well as the rationality of the pre-set base model, directly affect the accuracy of the model obtained after ensemble learning. If the input training data is small, or there are some inaccuracies in the input training data, or there are some unreasonable situations in the pre-set base model of the server, it will seriously affect the accuracy of the model obtained by ensemble learning, and then affect the accuracy of the chatbot when performing specific tasks, and affect the experience of end users. Summary of the invention

[0006] The present application provides a method, device and equipment for learning an intent recognition model, which can improve the accuracy of the intent recognition model in the human-computer dialogue system, improve the accuracy of the human-computer dialogue system in executing tasks, and improve the user experience.

[0007] In a first aspect, an embodiment of the present application provides a method for learning an intent recognition model, the method comprising: a server receives positive data corresponding to each intent in a first skill input by a skill developer; the server generates negative data corresponding to each intent in the first skill based on the positive data corresponding to each intent in the first skill; the server obtains training data corresponding to a second skill, the second skill is a skill similar to the first skill, and the number of second skills is at least one; the server learns based on the training data corresponding to the second skill and a pre-set first base model to generate a second base model, and the number of second base models is at least one; the server learns based on the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model to generate an intent recognition model.

[0008] As can be seen from the above, in the embodiment of the present application, since negative data is introduced when training the intent recognition model, it is helpful to reduce the misrecognition caused by training with only positive data, and it is helpful to improve the accuracy of the learned intent recognition model. In addition, since training data of other similar voice skills is also introduced when training the intent recognition model, more base models are learned, and the types of base models are enriched, the accuracy of the trained intent recognition model can be improved.

[0009] In one possible implementation, the server generates negative data corresponding to each intent in the first skill based on the positive data corresponding to each intent in the first skill, including: the server extracts the keywords corresponding to each intent in the first skill for each intent in the first skill, wherein the keywords are key features that affect the weight of the first base model; the keywords corresponding to different intents in the first skill are combined, or the keywords corresponding to different intents are combined with words that are not related to the first skill, and the combined words are determined as the negative data corresponding to the different intents.

[0010] It should be noted that the above keywords are the most significant features that are most likely to affect the weight of classification (recognition intent) features. In the prior art, the reasons for misidentification are that the significant features of other categories (other intents) and the significant features of this category (this intent) are highly overlapped. Figure 1 To open the app, Figure 2 To close the app. Figure 1 Harmony Figure 2 "app" is a high-frequency salient feature, but it is not a salient feature of each category. However, if this high-frequency salient feature is still used in classification, it is easy to cause misidentification. Therefore, the embodiment of the present application improves the accuracy by reducing the weight of these salient features contained in each category (intent) during classification.

[0011] In a possible implementation, the server generates negative data corresponding to each intent in the first skill based on the positive data corresponding to each intent in the first skill, and also includes: the server determines a first set from the full set of training data stored on the server based on the classification of the first skill, and the training data in the first set includes negative data corresponding to each intent in the first skill; wherein the training data includes positive data and negative data; a preset number of training data are sampled from the first set; and manual labeling and / or clustering algorithms are used to determine the negative data corresponding to each intent in the first skill from the sampled training data.

[0012] In a possible implementation, the server obtains data corresponding to the second skill, including: the server determines the second skill according to the classification of the first skill and / or the positive data of the first skill; and obtains training data corresponding to the second skill through the shared layer.

[0013] Since training data of other similar voice skills is also introduced when training the intent recognition model, it helps to solve the problem that the first skill developer inputs less training data and finds it difficult to learn a more accurate intent recognition model.

[0014] In one possible implementation, the server learns according to the training data corresponding to the second skill and the pre-set first base model to generate the second base model, including: the server learns according to the training data corresponding to the second skill and the pre-set first base model using a multi-task learning method to generate the second base model.

[0015] In one possible implementation, the server learns according to the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model to generate an intent recognition model, including: the server uses an integrated learning method to learn according to the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model to generate an intent recognition model.

[0016] In one possible implementation, the server generates an intent recognition model specifically including: the server generates multiple intermediate models in the process of generating the intent recognition model; the server determines that the first generated intermediate model is the fastest intent recognition model; and / or the server determines the best intent recognition model after selecting models and adjusting parameters of multiple intermediate models.

[0017] The embodiments of the present application provide a variety of return mechanisms, such as: returning the fastest generated intent recognition model, returning the best intent recognition model, etc. For example: when returning the fastest generated intent recognition model, since the server 300 obtains multiple intermediate models in the process of learning the intent recognition model, these intermediate models can also realize the functions of the intent recognition model, so the first intermediate model obtained can be determined as the intent recognition model returned to the voice skill developer or user, so that the voice skill developer or user can obtain the intent recognition model as quickly as possible to understand the functions and performance of the intent recognition model. For another example: when returning the best intent recognition model, the server 300 can adjust the parameters of the multiple intermediate models obtained in the process of learning the intent recognition model to determine the model with higher accuracy as the intent recognition model. It can meet the various needs of users.

[0018] In one possible implementation, the server determines the optimal intent recognition model including: the server calculates a first accuracy rate for each intermediate model based on positive data corresponding to each intent in the first skill; the server calculates a second accuracy rate for each intermediate model based on negative data corresponding to each intent in the first skill; the server determines the optimal intent recognition model after selecting and adjusting parameters of multiple intermediate models based on the first accuracy rate, the second accuracy rate, and the weights input by the skill developer.

[0019] In the embodiments of the present application, not only the corpus within the voice skill, such as the input training data of the skill developer, is considered to test the accuracy of the model, but also some corpus outside the voice skill determined by the server, such as automatically generated negative data, is considered to test the accuracy of the model. Since more test data is introduced when testing the accuracy of the model, the test results are more accurate. Then, after adjusting the parameters in the model according to the more accurate test results, the best intent recognition model can be obtained. In other words, the technical solution provided in the embodiments of the present application is conducive to improving the accuracy of the intent recognition model.

[0020] In a second aspect, an embodiment of the present application provides a server, including: a processor, a memory, and a communication interface; the memory is used to store computer program codes, the computer program codes include computer instructions, and when the processor reads the computer instructions from the memory, the server performs the following operations:

[0021] Receive positive data corresponding to each intent in the first skill input by the skill developer through the communication interface; generate negative data corresponding to each intent in the first skill according to the positive data corresponding to each intent in the first skill; obtain training data corresponding to the second skill, the second skill is a skill similar to the first skill, and the number of second skills is at least one; learn according to the training data corresponding to the second skill and a pre-set first base model to generate a second base model, and the number of second base models is at least one; learn according to the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model to generate an intent recognition model.

[0022] In a possible implementation, the server generates negative data corresponding to each intent in the first skill based on the positive data corresponding to each intent in the first skill, specifically including: the server extracts the keywords corresponding to each intent in the first skill for each intent in the first skill, wherein the keywords are key features that affect the weight of the first base model; the keywords corresponding to different intents in the first skill are combined, or the keywords corresponding to different intents are combined with words that are not related to the first skill, and the combined words are determined as the negative data corresponding to the different intents.

[0023] In a possible implementation, the server generates negative data corresponding to each intent in the first skill based on the positive data corresponding to each intent in the first skill, and specifically includes: the server determines a first set from the full set of training data stored on the server according to the classification of the first skill, and the training data in the first set includes negative data corresponding to each intent in the first skill; wherein the training data includes positive data and negative data; a preset number of training data are sampled from the first set; and manual labeling and / or clustering algorithms are used to determine the negative data corresponding to each intent in the first skill from the sampled training data.

[0024] In a possible implementation, the server obtains data corresponding to the second skill, including: the server determines the second skill according to the classification of the first skill and / or the positive data of the first skill; and obtains training data corresponding to the second skill through the shared layer.

[0025] In one possible implementation, the server learns according to the training data corresponding to the second skill and the pre-set first base model to generate the second base model, including: the server learns according to the training data corresponding to the second skill and the pre-set first base model using a multi-task learning method to generate the second base model.

[0026] In one possible implementation, the server learns according to the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model to generate an intent recognition model, including: the server uses an integrated learning method to learn according to the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model to generate an intent recognition model.

[0027] In one possible implementation, the server generates an intent recognition model and specifically includes: the server generates multiple intermediate models in the process of generating the intent recognition model; the server determines that the first generated intermediate model is the fastest intent recognition model; and / or the server determines the best intent recognition model after selecting models and adjusting parameters of multiple intermediate models.

[0028] In one possible implementation, the server determines the optimal intent recognition model including: the server calculates a first accuracy rate for each intermediate model based on positive data corresponding to each intent in the first skill; the server calculates a second accuracy rate for each intermediate model based on negative data corresponding to each intent in the first skill; the server determines the optimal intent recognition model after selecting and adjusting parameters of multiple intermediate models based on the first accuracy rate, the second accuracy rate, and the weights input by the skill developer.

[0029] In a third aspect, a computer storage medium comprises computer instructions, which, when executed on a server, cause the server to execute the method as described in the first aspect and any possible implementation thereof.

[0030] A fourth aspect is a computer program product. When the computer program product is run on a computer, it enables the computer to execute the method as described in the first aspect and any possible implementation thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a human-computer dialogue system provided in an embodiment of the present application Figure 1 ;

[0032] Figure 2 A schematic diagram of a learning method for an intent recognition model in the prior art;

[0033] Figure 3 A schematic diagram of a usage scenario of an intent recognition model provided in an embodiment of the present application;

[0034] Figure 4 A schematic diagram of a human-computer dialogue system provided in an embodiment of the present application Figure 2 ;

[0035] Figure 5 A schematic diagram of a learning method for an intent recognition model provided in an embodiment of the present application Figure 1 ;

[0036] Figure 6 A schematic diagram of a learning method for an intent recognition model provided in an embodiment of the present application Figure 2 ;

[0037] Figure 7 A schematic diagram of a learning method for an intent recognition model provided in an embodiment of the present application Figure 3 ;

[0038] Figure 8 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0039] Fig. 9 A flowchart of a method for learning an intent recognition model provided in an embodiment of the present application;

[0040] Fig.10 A schematic diagram of a learning method for an intent recognition model provided in an embodiment of the present application Figure 4 . DETAILED DESCRIPTION

[0041] The technical solution in the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application. In the description of the embodiment of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0042] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0043] like Figure 1As shown, it is a schematic diagram of the composition of a human-computer dialogue system provided in an embodiment of the present application. The human-computer dialogue system includes: one or more electronic devices 100, one or more servers 200 and one or more servers 300. A communication connection is established between the electronic device 100 and the server 300. The server 300 establishes a communication connection with the electronic device 100 and the server 200 respectively. Optionally, the electronic device 100 can also establish a communication connection with the server 200. The communication connection can be established using a telecommunication network (communication networks such as 3G / 4G / 5G) or a WIFI network, etc., which is not limited in the embodiments of the present application.

[0044] Among them, the electronic device 100 can be a mobile phone, a tablet computer, a personal computer (PC), a personal digital assistant (PDA), a smart watch, a netbook, a wearable electronic device, an augmented reality technology (AR) device, a virtual reality (VR) device, a vehicle-mounted device, a smart car, a smart speaker, a robot, etc. The present application does not impose any special restrictions on the specific form of the electronic device.

[0045] The server 200 may be a server of a third-party application, and is used to provide services of the third-party application, wherein the third-party application may be, for example, an application such as Meituan, Amazon, or Didi Taxi.

[0046] The server 300 may be a server of the manufacturer of the electronic device 100, for example, it may be a cloud server of a voice assistant in the electronic device 100, etc. The server 300 may also be other servers, which is not limited in the embodiments of the present application.

[0047] Taking the development of a new voice application or function (which may be called a voice skill) by a third-party application as an example, the human-computer dialogue scenario of the application of the technical solution provided in the embodiment of the present application is explained.

[0048] The voice skill may refer to a function in which a user can request a third-party application to provide one or more services in the application by using a conversational interaction between the electronic device 100 and the server 200 of the third-party application. This conversational interaction process simulates the interaction scenario in the user's real life, making the interaction between the user and the electronic device as natural as interaction with a person.

[0049] In the above-mentioned conversational interaction process, each sentence spoken by the user corresponds to an intent, which is the purpose of the user's saying this sentence. It should be noted that each voice skill is composed of several intents. The server 300 matches each sentence spoken by the user with the intent in the voice skill to understand the user's needs and provide corresponding services, such as providing services such as ordering meals, booking tickets, and taking taxis.

[0050] The above process of matching the user's words with the intention in the voice skill is the process of intent recognition. The server 300 can build an intent recognition model, which can automatically identify the intention corresponding to the user's words based on the user's input.

[0051] like Figure 2 As shown, it is a schematic diagram of the process of training the intent recognition model of the server 300. Specifically, the developer of the voice skill, such as the server 200, needs to first input some training data corresponding to the new voice skill to the server 300 (which may include the correspondence between the user's statement and the user's intention). According to these training data, the server 300 can adopt a corresponding learning method, such as an integrated learning method, to obtain the intent recognition model corresponding to the new voice skill. For example: the corpus A corresponding to the voice skill 1 is input into the model learning framework 1 for learning to obtain the intent recognition model corresponding to the voice skill 1. Among them, the model learning framework 1 can adopt an integrated learning framework, for example, and the smallest learning unit in the model learning framework is the base model. The integrated learning framework contains multiple base models, which are recorded as base model 1. There are differences between these base models 1. For example, different base models 1 may belong to different models, such as support vector machines (SVM) and logistic regression (LR) models; they may also belong to the same model but use different hyperparameters (for example, SVM support vector machine models using different hyperparameter C values); they may also belong to the same model but use different data (for example, the same SVM model uses different original data subsets, etc.).

[0052] In addition, for the specific technical solutions of the integrated learning method involved in the embodiments of this application, please refer to the patent application with application number CN 102521599A and titled “A Pattern Training and Recognition Method Based on Integrated Learning” published by the State Intellectual Property Office of China. Please also refer to the patent application with application number CN107491531A and titled “Chinese Network Comment Sentiment Classification Method Based on Integrated Learning Framework” published by the State Intellectual Property Office of China, as well as other implementations of the integrated learning method in the prior art, which are not limited by the embodiments of this application.

[0053] After the training of the intent recognition model corresponding to the new voice skill is completed, when the electronic device 100 receives the user input, it sends the user input to the server 300, and the server 300 makes a prediction based on the user's statement and the trained intent recognition model to determine the user's intention corresponding to the user's statement, and then determine the third-party application that provides services for the user and the specific functions in the third-party application.

[0054] For example: Figure 3 As shown, it is a schematic diagram of the process of server 300 determining to provide a third-party application service to a user. It is assumed that in addition to the newly learned intent recognition model corresponding to voice skill 1 (belonging to one of the voice skills of third-party application 1), server 300 also stores the intent recognition model corresponding to voice skill 2 of third-party application 1, the intent recognition model corresponding to voice skill 3 of third-party application 2, and so on.

[0055] Specifically, the user inputs voice 1 through the electronic device 100, and the electronic device 100 can convert the voice 1 into corresponding text, that is, user speech 1, and send it to the server 300, or the electronic device 100 can directly send the voice 1 to the server 300, and the server 300 converts the voice 1 into text (user speech 1). The server 300 distributes the user speech 1 to the intention recognition model corresponding to each voice skill, and the intention recognition model corresponding to each voice skill performs calculations to determine that the user speech 1 corresponds to the voice skill 1, that is, it is necessary to provide the user with services corresponding to the voice skill 1 of the third-party application.

[0056] In some embodiments, after determining the third-party application that provides services to the user, the server 300 can return the information of the determined third-party application to the electronic device 100. The information of the third-party application can include the address information of the server of the third-party application. The information of the third-party application can also include the information of the specific services that need to be provided to the user, such as ordering or querying orders. Then, the electronic device 100 can establish a communication connection with the server 200 of the third-party application, and the server 200 of the third-party application provides the corresponding service to the user. In other embodiments, the server 200 can establish a communication connection with the server 200 of the corresponding third-party application, and send the service request information of the user to the server 200 of the third-party application. The server 200 of the third-party application can interact with the electronic device 100 through the server 300 to provide the corresponding service to the user. Among them, the service request information can include: information of the specific service requested by the user, such as ordering or querying orders. The embodiment of the present application does not limit the specific way in which the server 200 of the third-party application provides services to the user of the electronic device 100.

[0057] It should be noted that if Figure 4 FIG. 1 is a schematic diagram of another human-computer dialogue system provided in an embodiment of the present application. The human-computer dialogue system includes: one or more electronic devices 100 and one or more servers 200. The electronic devices 100 and the servers 200 can refer to Figure 1 Related description in .

[0058] The difference is that the above-mentioned process of learning the intention recognition model corresponding to the new voice skill based on the training data corresponding to the new voice skill input by the voice skill developer can also be executed on the server 200, or when the computing function of the electronic device 100 can support the operation of the process, it can also be executed on the electronic device 100. The method provided in the embodiment of the present application does not limit the execution subject of the training intention recognition model. If the server 200 of a third-party application executes the model for training intention recognition, then the server 200 of the third-party application trains the intention recognition models of multiple voice skills of the third-party application itself. The following is explained by taking the server 300 executing the training intention recognition model as an example.

[0059] It should be noted that when the server 300 is training the intent recognition model, if the voice skill developer cannot provide enough training data, or the voice skill developer cannot guarantee the accuracy of the training data, the accuracy of the target model trained by the server 300 cannot be guaranteed, which will affect the accuracy of the server 300 in recognizing the user's intent and the user experience of the electronic device. Due to the various scenarios and types of voice skills, the base model pre-set by the server 300 will inevitably have some unreasonable situations, such as: not applicable to certain scenarios or types of voice skills, which will also affect the accuracy of the server 300 training the target model.

[0060] It should be noted that the server 300 can provide a common human-computer dialogue platform for multiple third-party applications. The developer or maintainer of the human-computer dialogue platform can pre-set some basic skill templates (including some pre-set training data, base models used in learning, etc.) on the human-computer dialogue platform. These skill templates cover some common usage scenarios. Then, third-party applications can modify these basic skill models to achieve their own personalized needs, or third-party applications can add customized skills according to their own service content.

[0061] To this end, the technical solution provided in the embodiment of the present application can be applied to the process in which the server 300 learns the intention recognition model corresponding to the new voice skill, and can improve the accuracy of the intention recognition model learned by the server 300 based on the existing technology.

[0062] like Figure 5As shown, it is a schematic diagram of a method for learning an intent recognition model provided by an embodiment of the present application. The server 300 can automatically generate some other training data based on the training data corresponding to the input new voice skill. Then, the training data input by the voice skill developer and the other automatically generated training data are input into the server 300 together, and the server 300 learns. Since more training data are introduced in the learning process, the accuracy of the intent recognition model obtained by the server 300 after learning can be improved.

[0063] For example, the training data of voice skill 1 is corpus A. The server generates negative data, namely corpus B, based on the positive data in corpus A. Then, corpus A and corpus B are input into the model learning framework for learning, and the intention recognition model corresponding to voice skill 1 is obtained. Among them, positive data is the data of user statements that can trigger the server 300 to perform corresponding operations. Negative data is some data similar to positive data but not positive data, that is, data of user statements that the server 300 does not perform corresponding operations.

[0064] In other embodiments of the present application, the server 300 can learn the intention recognition models corresponding to different voice skills of the same third-party application, and can also learn the intention recognition models corresponding to different voice skills of different third-party applications. That is to say, the server 300 can store the intention recognition models of different voice skills of the same third-party application, and the intention recognition models corresponding to different voice skills of different third-party applications. Considering that the intention recognition models corresponding to various types of voice skills of various topics can be stored on the server 300, there will be some similarities between these large numbers of voice skills, so the corpus corresponding to these voice skills can also be reused. Therefore, the technical solution provided in the embodiment of the present application can also be combined with the training data of other voice skills similar to the voice skill when training the intention recognition model corresponding to a certain voice skill. For example: the server 300 can expand the base model in the model learning framework according to the training data of other voice skills similar to the voice skill. Among them, other voice skills similar to the voice skill can belong to the same third-party application as the voice skill, or can belong to different third-party applications from the voice skill. The embodiment of the present application does not limit the type and number of similar voice skills.

[0065] For example: Figure 6 As shown, it is a schematic diagram of another process of learning intention recognition model provided by the embodiment of the present application. Take the example that the server 300 determines that voice skill 2 and voice skill 3 are other voice skills similar to voice skill 1 for explanation.

[0066] The server 300 can input the corpus C corresponding to the speech skill 2 into the model learning framework 2 to learn one or more base models 2. The server 300 inputs the corpus D corresponding to the speech skill 3 into the model learning framework 2 to learn one or more base models 2. The model learning framework 2 includes one or more base models 1 pre-stored by the server 300. The multiple base models 2 obtained according to the corpus C and the corpus D, as well as the model learning framework 2, are the base models obtained after the server 300 expands, and can be used as the base models in the model learning framework 3. The model learning framework 2 can, for example, adopt a multi-task learning framework, and the model learning framework 3 can, for example, adopt an integrated learning framework.

[0067] The server 300 can input the corpus A corresponding to the voice skill 1 input by the skill developer and the automatically generated corpus B into the model learning framework 3 for learning, and obtain the intention recognition model corresponding to the voice skill 1. Optionally, the server 300 can also directly input the corpus A corresponding to the voice skill 1 input by the skill developer into the model learning framework 3 for learning, and obtain the intention recognition model corresponding to the voice skill 1. This embodiment of the application is not limited to this.

[0068] It can be seen that since training data of other similar voice skills are introduced when training the intent recognition model, more base models are learned and the types of base models are enriched, the accuracy of the trained intent recognition model can be improved.

[0069] In some other embodiments of the present application, Figure 7As shown, it is a schematic diagram of the process of another server 300 learning the intention recognition model provided by the embodiment of the present application. Specifically, in the process of the server 300 learning the intention recognition model, multiple intermediate models will be obtained successively, and the server 300 will select the model and adjust the parameters of these intermediate models to obtain the intention recognition model with the highest accuracy finally generated. It should be noted that these intermediate models can all realize the function of the intention recognition model corresponding to the voice skill 1. Taking into account the different needs of voice skill developers, the embodiment of the present application provides a variety of return mechanisms, such as: returning the fastest generated intention recognition model, returning the best intention recognition model, etc. For example: when returning the fastest generated intention recognition model, since the server 300 obtains multiple intermediate models in the process of learning the intention recognition model, these intermediate models can also realize the function of the intention recognition model, so the first intermediate model obtained can be determined as the intention recognition model returned to the voice skill developer or user, so that the voice skill developer or user can obtain the intention recognition model as quickly as possible to understand the function and performance of the intention recognition model. For another example, when returning the best intent recognition model, the server 300 may adjust the parameters of the multiple intermediate models obtained during the process of learning the intent recognition model to determine the model with higher accuracy as the intent recognition model.

[0070] Due to the technical solution provided by the embodiment of the present application, when training the intention recognition model corresponding to a certain voice skill, combined with the training data of other voice skills similar to the voice skill, the embodiment of the present application also provides a standard for determining the best model, that is, not only considering the corpus within the voice skill, such as: the input training data of the skill developer, to test the accuracy of the model, but also considering using the server to determine some corpus outside the voice skill, such as automatically generated negative data, to test the accuracy of the model. Since more test data is introduced when testing the accuracy of the model, the test results are more accurate. Then, after adjusting the parameters in the model according to the more accurate test results, the best intention recognition model can be obtained. In other words, the technical solution provided by the embodiment of the present application is conducive to improving the accuracy of the intention recognition model. The specific implementation process will be described in detail below.

[0071] like Figure 8 , which is a schematic diagram of the hardware structure of a server 300 provided in an embodiment of the present application, wherein the server 300 includes at least one processor 301, at least one memory 302, and at least one communication interface 303. Optionally, the server 300 may also include an output device and an input device, which are not shown in the figure.

[0072] The processor 301, the memory 302 and the communication interface 303 are connected via a bus. The processor 301 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application. The processor 301 may also include multiple CPUs, and the processor 301 may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. The processor here may refer to one or more devices, circuits or processing cores for processing data (such as computer program instructions).

[0073] In an embodiment of the present application, the processor 301 can be specifically used to automatically generate negative data of a new voice skill based on the training data of the new voice skill input by the voice skill developer. The processor 301 can also be specifically used to determine other voice skills similar to the new voice skill, obtain data of other voice skills, and expand the base model in the model learning framework. The processor 301 can also learn based on the expanded base model and the training data input by the developer, as well as the negative data automatically generated by the processor 301, to obtain an intent recognition model. The processor 301 can also be specifically used to adjust the parameters of multiple intermediate models generated during the learning process in order to generate an optimal intent recognition model, etc.

[0074] The memory 302 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 302 may exist independently and be connected to the processor 301 via a bus. The memory 302 may also be integrated with the processor 301. Among them, the memory 302 is used to store the application code that executes the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the computer program code stored in the memory 302, so as to implement the data transmission method described in the embodiment of the present application.

[0075] In an embodiment of the present application, the memory 302 can be used to store data of the base model pre-set in the learning framework in the server 300, and can also be used to store negative data corresponding to each voice skill automatically generated by the server 300, various intermediate models generated by the processor 301 during the learning process, and various data for training or testing, as well as models of intent recognition corresponding to each learned voice skill, etc.

[0076] The communication interface 303 can be used to communicate with other devices or communication networks, such as Ethernet, wireless local area networks (WLAN), etc.

[0077] In the embodiment of the present application, the communication interface 303 can be specifically used to communicate with the electronic device 100 to achieve interaction with the user of the electronic device. The communication interface 303 can also be specifically used to communicate with the server 200 of the third-party application. For example, the server 300 can receive the training data corresponding to the new voice skill input by the service 200 of the third-party application, or the server 300 can send the determined user service request to the server 200 of the third-party application, so that the server 200 of the third-party application provides the corresponding service to the user of the electronic device.

[0078] The output device communicates with the processor and can display information in a variety of ways. For example, the output device can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. The input device communicates with the processor and can receive user input in a variety of ways. For example, the input device can be a mouse, a keyboard, a touch screen device, or a sensor device.

[0079] The technical solutions involved in the following embodiments can all be implemented in the server 300 having the above hardware architecture.

[0080] like Fig. 9 As shown, a flow chart of a method for learning an intent recognition model is provided in an embodiment of the present application, which specifically includes:

[0081] S101. The server receives positive data corresponding to the first skill input by a skill developer.

[0082] Among them, the skill developer can be a developer of a third-party application or a manufacturer of an electronic device, which is not limited in the embodiments of the present application.

[0083] In some embodiments of the present application, the server provides a skill development platform for skill developers. Skill developers can log in to the skill developer's platform through a telecommunications network (communication networks such as 3G / 4G / 5G) or a WIFI network, register an account, and start creating a new voice skill, namely the first skill. The skill developer can enter basic information such as the name of the first skill, the classification of the first skill, and then enter the intent contained in the first skill, and the positive data corresponding to each intent. Among them, the positive data corresponding to each intent includes: the server needs to perform corresponding operations in response to receiving the positive data, such as triggering the corresponding service, or returning the corresponding dialogue to the electronic device. In other words, the data of the user's statement that can trigger the server to perform the corresponding operation is called positive data, and can also be called positive corpus, positive example, positive example data, skill data, etc.

[0084] In other embodiments of the present application, the skill developer may also input some negative data corresponding to each intent when inputting the positive data corresponding to each intent of the first skill. Among them, negative data is data similar to positive data, but not positive data. Among them, negative data can also be called negative corpus, negative examples, negative example data, data outside of skills, etc., which are not limited in the embodiments of the present application.

[0085] For example, the skill developer is Meituan App, and the first skill developed by Meituan App is meal ordering service. Meituan App can input positive data corresponding to meal ordering service, such as ordering takeout, ordering meals, etc. Meituan App can also input negative data, such as ordering air tickets, etc.

[0086] S102. The server generates negative data corresponding to the first skill based on the positive data corresponding to the first skill.

[0087] The first skill includes multiple intents, each of which may correspond to one or more positive data. For each intent, the server may generate corresponding negative data based on the positive data of each intent. This is conducive to improving the accuracy of the intent recognition model trained later.

[0088] This is because, in the prior art, when there is only one type of training data (positive data) input to the server, the intent recognition model trained by the server uses the threshold set by the skill developer as the only criterion for judging a user's statement. In other words, when the confidence of a statement is lower than the threshold, the statement is considered not to be the intention within the first skill. When the confidence of a statement reaches the threshold, the statement is considered to be the intention within the first skill. In actual scenarios, since the thresholds of different voice skills depend on factors such as the similarity between different voice skills, this threshold is difficult to set. In an embodiment of the present application, other types of training data are introduced, for example, negative data is introduced, and category discrimination can also be performed, which is beneficial to avoid some misidentification situations and to improve the accuracy of the trained intent recognition model.

[0089] Exemplarily, the present application provides two methods for generating negative data corresponding to the first skill based on positive data corresponding to the first skill, as follows:

[0090] Method 1: For each intent in the first skill, the server extracts keywords of each intent from the positive data corresponding to each intent, and combines the extracted keywords with other irrelevant words to form negative data corresponding to each intent.

[0091] It should be noted that the above keywords are the most significant features that are most likely to affect the weight of classification (recognition intent) features. In the prior art, the reasons for misidentification are that the significant features of other categories (other intents) and the significant features of this category (this intent) are highly overlapped. Figure 1 To open the app, Figure 2 To close the app. Figure 1 Harmony Figure 2"app" is a high-frequency salient feature, but it is not a salient feature of each category. However, if this high-frequency salient feature is still used in classification, it is easy to cause misidentification. Therefore, the embodiment of the present application improves the accuracy by reducing the weight of these salient features contained in each category (intent) during classification.

[0092] Among them, the keywords extracted from the above-mentioned positive data can be manually marked by skill developers, or can be extracted using relevant mining technology of sampled keywords. For example, the keywords in the positive data are also a feature in the positive data, so feature selection methods can be used for extraction, such as the LR (logistic regression) method based on L1 regularization. The embodiment of the present application can also adopt a wrapper feature selection method, such as: LVW (Las Vegas Wrapper), a sparse representation method based on dictionary learning, and a simple heuristic algorithm (Heuristic Algorithm). The embodiment of the present application does not limit the specific method of extracting keywords.

[0093] For example, the LR method based on L1 regularization is used to briefly introduce the process of extracting keywords from positive data. Figure 1 There are K categories in total (i.e., the positive data input by the user corresponds to K intents, i.e., one intent can be understood as one category). K "1 vs the rest" classifiers are constructed respectively. The L1 norm can be used to ignore features that have little impact on the classification weight, thereby retaining features that have an impact on the classification. Given that each positive data is converted into two categories and the features are 0 / 1 discrete features, significant features can be obtained by screening and sorting through the weight sign and absolute value, that is, the keywords with the largest weight for each intent.

[0094] The above irrelevant words may be words (or word combinations or sentences, etc.) that are similar in form to the original positive data but deviate in semantics after being combined with the extracted keywords, or words (or word combinations or sentences, etc.) that are semantically irrelevant to the original positive data. For example, a sentence similarity method may be used to determine whether the combined words (or word combinations or sentences, etc.) are semantically deviated from or irrelevant to the original positive data.

[0095] Exemplarily, the server may randomly combine keywords corresponding to different intents to generate negative data. The server may also randomly combine keywords in each intent with some irrelevant words such as entity words in the first skill to generate negative data. The embodiment of the present application does not limit the server's selection method for irrelevant words, nor does it limit the way of combining keywords for each intent with other irrelevant words.

[0096] For example, the positive data of a skill contains "open menu" and "close menu", which correspond to two intentions. Figure 1 Harmony Figure 2 After extracting the keywords of the positive data, we can get the meaning Figure 1 The keyword is "open", meaning Figure 2 The keyword of the skill is "close". Then, "open close" obtained by combining keywords of different intents, and "open drawer" and "close WeChat" obtained by combining keywords of each intent with other irrelevant words constitute the negative data of the skill.

[0097] Method 2: In an embodiment of the present application, the server stores a large amount of training data for voice skills, including data for different voice skills of the same third-party application and data for different voice skills of different third-party applications. These massive amounts of training data may belong to different fields and different classifications, constituting a complete set of training data on the server. For the first skill, the server may determine a first set from the complete set of training data stored on the server according to the classification of the first skill, wherein the corpus in the first set is a set of corpus that may conflict with the forward data of the first skill. When determining the first set, a method based on an intra-class classifier or a method based on similarity retrieval may be used, such as an inverted index, sentence similarity calculation, etc., to determine the corpus that conflicts with the forward data, or to determine suspicious corpus.

[0098] It should be noted that when determining the first set, the server can directly determine the first set from the full set of training data, or gradually determine the first set from the full set of training data. That is to say, the server can first determine a set A from the full set of training data, where set A is smaller than the full set of training data. Then determine a set B from set A, where set B is smaller than set A, and so on, until the first set is determined. The embodiment of the present application does not limit the specific method for the server to determine the first set.

[0099] The corpus in the first set may be corpus similar to the positive data, for example, containing the same or similar keywords as the positive data, or containing the same or similar entity words (for example, nouns) as the positive data. For example, if the positive data is "open menu", and the keyword of the corpus is "open", then the first set may contain, for example, "turn on the radio", "open menu", "hide menu", etc.

[0100] The server may sample a certain amount of corpus from the first set, and further determine whether the adopted corpus is negative data corresponding to the first skill.

[0101] The above-mentioned certain number (i.e., the number of sampled corpora) can be determined according to the number of positive data. For example, the number of sampled corpora can be determined to be the same as the number of positive data, or the maximum number of corpora corresponding to each intent, or the average number of corpora, or a preset number, etc. The embodiment of the present application does not specifically limit the number of sampled corpora.

[0102] Among them, the sampling process can adopt random sampling or sampling according to topic distribution, and the embodiments of the present application do not limit the specific sampling method.

[0103] When the server determines whether the sampled corpus is the negative data of the first skill, it can also be understood as classifying the sampled corpus. If the semantics are the same as the positive data, it is positive data; if they are different, it is negative data. Whether the sampled corpus is the negative data of the first skill can be determined by manual definition or by clustering algorithm and manual iteration. The clustering algorithm can be hierarchical clustering or a text clustering classification method based on a topic model. The clustering algorithm and manual iteration method can be, for example, manual annotation of some features that cannot be recognized by the clustering algorithm, such as the brightness of the color, etc. For some suspicious corpus that cannot be determined by the clustering algorithm, manual confirmation is performed to determine whether it is negative data. The clustering algorithm can also learn the rules of manual annotation, and then use the rules to determine whether other sampled corpus are negative data.

[0104] It should be noted that when the server generates negative data based on positive data, it can adopt any one of the above methods, or use both methods, or use other methods. The embodiment of the present application does not limit the method of generating negative data based on positive data.

[0105] S103. The server determines a second skill that is similar to the first skill, and the number of the second skill is at least one.

[0106] It should be noted that, considering that a large amount of data of different voice skills is stored on the server, and the data storage similarities between these large amounts of different voice skills, the server can share the data between different voice skills. Specifically, the server can share the data of each voice skill after obtaining the consent of each voice skill developer. The sharing method can be based on classification, for example: data can be shared between different voice skills in the same category, and data cannot be shared between different voice skills in different categories. In some embodiments, the server can add a sharing layer between voice skills that can share data, and different voice skills can obtain data of other voice skills through the sharing layer. In other embodiments, the server can also have some built-in data that can be used by different voice skills. In this way, when the server learns the intent recognition model corresponding to a new voice skill, if the voice skill developer does not input the training data corresponding to the voice skill, or the input training data is less, the server can also learn by sharing the data of other voice skills and the data built into the server to obtain the intent recognition model.

[0107] To this end, in steps S103 and S104, the server can determine other second skills on the server that are similar to the first skill based on, for example, the classification of the first skill, or the similarity between the training data of the first skill and the training data of other skills. The second skill can be other voice skills developed by the developer of the first skill, or other voice skills developed by other voice skill developers. The embodiments of the present application do not limit the method for extracting the second skill and the number of extractions.

[0108] S104. The server obtains data corresponding to each second skill.

[0109] For example, the server can obtain the training data corresponding to each second skill through the shared layer, which may include positive data and negative data. Specifically, it may include the positive data and negative data input by the developer of the second skill, and it may also include negative data automatically generated by the server in the process of learning the intent recognition model or other models corresponding to the second skill.

[0110] S105. The server generates a second base model according to the data corresponding to the second skill and the first base model stored in the server.

[0111] When the server learns the intent recognition model corresponding to each skill, a model learning framework will be pre-set, including one or more first base models used, and the organizational relationship between each first base model (which may include the weights and hierarchical relationships between each first base model).

[0112] In an embodiment of the present application, the server inputs the corpus corresponding to each second skill into the first base model respectively, and learns multiple second base models. In some embodiments, for example, a multi-task learning method can be adopted, wherein the multi-task learning method can be implemented based on models such as deep learning convolutional neural networks (CNN), bidirectional long short-term memory networks (LSTM). In other embodiments, adversarial training can also be added to the process of multi-task learning, which can further realize the purification of shared data, improve the accuracy of the second base model, and then improve the accuracy of the intent recognition model.

[0113] For example: Figure 6 As shown, the first skill is voice skill 1, and the input training data of the first skill is corpus A. Assume that the server determines that the voice skills similar to voice skill 1 are voice skill 2 and voice skill 3. Among them, the training data corresponding to voice skill 2 is corpus C, and the training data corresponding to voice skill 3 is corpus D.

[0114] The server inputs corpus C into the model learning framework 2 for learning, wherein the model learning framework 2 includes one or more first base models, namely base model 1. The model learning framework 2 may be, for example, a multi-task learning framework. Specifically, corpus C is input into the multi-task learning framework to generate multiple second base models, namely, multiple base models 2. At this point, the server is equivalent to task learning for speech skill 2. Corpus D is input into the multi-task learning framework to generate multiple second base models. At this point, the server is equivalent to task learning for speech skill 3.

[0115] That is to say, when the server is learning the intent recognition model corresponding to voice skill 1, it also introduces the training data of other voice skills (voice skill 2 or voice skill 3), and performs multi-task joint training on the training data of other voice skills to generate a second base model with a different representation space. The second base model of the different representation space is then input into the integrated learning framework for learning, and finally the intent recognition model is obtained. Compared with the method in the prior art of simply transforming the features of the training data or replacing the first base model, the technical solution provided in the embodiment of the present application is more conducive to improving the accuracy of the intent recognition model corresponding to the learned voice skill.

[0116] S106: The server learns based on the positive data and negative data corresponding to the first skill and the second base model. If the fastest intent recognition model is determined to be returned according to the strategy, step S107 is executed; if the best intent recognition model is determined to be returned according to the strategy, step S108 is executed.

[0117] The above strategy can be a strategy preset by the server, that is, which model is returned or which two models are returned; it can also be a strategy recommended by the server to the user, that is, different models can be recommended to the user according to different situations. It can also be a return strategy received by the server from the user to determine the corresponding model to be returned. The embodiment of the present application does not limit the strategy.

[0118] In some embodiments of the present application, the server may adopt an ensemble learning method for learning, such as a stacking ensemble learning method. During the server learning process, some intermediate models will be generated successively.

[0119] For example: Fig.10 As shown, it is a schematic diagram of the process of learning another intention recognition model provided by the embodiment of the present application. When the model learning framework 3 adopts the stacking ensemble learning framework, the server inputs the positive data and negative data corresponding to the first skill into the stacking ensemble learning framework, and learns the intermediate model of the first layer, which is recorded as intermediate model 1. Then, the server inputs the features obtained from the intermediate model of the first layer into the stacking ensemble learning framework, and learns the intermediate model of the second layer, and so on, until the intention recognition model is finally generated.

[0120] S107. The server determines the fastest intent recognition model.

[0121] During the learning process of step S106, the server will successively generate multiple intermediate models, and these intermediate models can independently realize the function of the intent recognition model corresponding to the first skill. Therefore, based on the needs of the skill developer, the server can first return some intermediate models as the fastest generated models. In some embodiments, the server can determine and return the first generated intermediate model as the fastest intent recognition model. Fig.10 As shown in , the service can determine that the first intermediate model generated first is the fastest intent recognition model. In some other embodiments, the server can determine and return a model generated using a specified method as the fastest intent recognition model. In this way, skill developers can obtain the intent recognition model as quickly as possible to understand the function, performance, etc. of the intent recognition model.

[0122] S108. The server determines the best intent recognition model.

[0123] It is generally believed that after the server completes learning based on the positive data and negative data corresponding to the first skill and the second base model, the final intent recognition model is the best intent recognition model. This is because during the learning process, the server will select models and adjust parameters based on the multiple intermediate models generated successively, so that the final intent recognition model has the highest accuracy. For example, the grid search method or Bayesian optimization method can be used for parameter tuning.

[0124] It should be noted that the parameter tuning in the prior art is based on the training data corresponding to the first skill input by the first skill developer. Therefore, the accuracy of the intent recognition model finally obtained only reflects the accuracy of prediction using the data in the first skill.

[0125] However, in an embodiment of the present application, when training the intent recognition model, the server uses data outside the first skill (for example, automatically generated negative data of the first skill, and when expanding the base model, data of the second skill similar to the first skill is also used) in addition to the data within the first skill (for example, the positive data of the first skill input by the first skill developer).

[0126] Therefore, if Fig.10 As shown, when the server selects the best intent recognition model, it needs to consider the data within the first skill (for example: the positive data of the first skill), as well as the data outside the first skill (for example: the negative data of the first skill automatically generated by the service). There are two parts of accuracy that the server needs to consider. One is the accuracy of prediction using the data within the first skill, that is, the accuracy of the first skill data. When predicting with the data within the first skill, if the prediction is a positive class (that is, it can be identified as the intention to call the first skill), the prediction is correct. The second is the accuracy of prediction using data outside the first skill, that is, the accuracy of data outside the first skill. When predicting with data outside the first skill, if the prediction is a negative class (that is, it can be identified as the intention not to call the first skill), the prediction is correct.

[0127] In some embodiments, the false recall rate of data outside the first skill can also be used to reflect the accuracy of prediction using data outside the first skill. The lower the false recall rate of data outside the first skill, the higher the accuracy of prediction using data outside the first skill. It can be seen that the accuracy of the first skill data is inversely proportional to the accuracy of the data outside the first skill, and directly proportional to the false recall rate of data outside the first skill. In other words, the accuracy of the first skill data is high, and the false recall rate of the first skill is also high. Then, it is impossible to make the intent recognition model finally trained to achieve the best in both aspects. Therefore, an embodiment of the present application provides a method for evaluating the best intent recognition model.

[0128] The server can set a confidence level for the first skill, which can be set by the user (e.g., the developer of the first skill). The confidence level parameter can reflect the user's relative requirements for the accuracy of the data within the first skill and the false recall rate of the data outside the first skill. For example, the greater the confidence level, the lower the false recall rate of the data outside the first skill, that is, the higher the prediction accuracy of the data outside the first skill. Therefore, the following formula 1 can be used to evaluate the best intent recognition model, as follows:

[0129] score=accuracyIn*(1-C)+accuracyOut*C(Formula 1)

[0130] Among them, score is the score of the best intent recognition model. The higher the score, the higher the accuracy of the intent recognition model, and the more it meets the user's requirements. accuracyIn is the average accuracy of the data in the first skill. accuracyOut is the average accuracy of the data outside the first skill. C is the confidence value set by the user, which is greater than or equal to zero and less than or equal to 1.

[0131] When the server calculates the average accuracy of the data within the first skill and the average accuracy of the data outside the first skill, for example, a K-fold cross validation method may be used. The specific method of the K-fold cross validation can refer to the prior art and will not be described in detail here.

[0132] In other embodiments of the present application, in order to avoid unreasonable settings of confidence by the user, the accuracy of the trained intent recognition model will be greatly affected. For example, when the confidence set by the user is too high, the accuracy of the data within the skill will be ignored. To this end, the server can also set a parameter to control the confidence set by the user. Then, the following formula 2 can be used to evaluate the best intent recognition model, as follows:

[0133] score=accuracyIn*(1-C*P)+accuracyOut*C*P(Formula 2)

[0134] The meanings of the score, accuracyIn, accuracyOut, and C parameters are the same as those in Formula 1 and are not repeated here. P is a parameter set by the server, and P is greater than or equal to zero and less than or equal to 1. The P value can control the degree of influence of the confidence set by the user on the score. The larger the P value, the greater the influence of the confidence set by the user on the score. The smaller the P value, the smaller the influence of the confidence set by the user on the score.

[0135] It should be noted that, when executing steps S107 and S108, the server can share some common features of the intermediate model through the shared layer to reduce the amount of calculation and improve the efficiency of model training.

[0136] Then, after learning the fastest or best intention recognition model, it can be used to predict new user statements input by the user of the electronic device and determine the user's intention so as to provide corresponding services to the user.

[0137] For example: Figure 1 The human-computer dialogue system shown can deploy the fastest or best learned intent recognition model on server 300 or server 200. When deployed on server 300, server 300 can store intent recognition models corresponding to different voice skills of multiple third-party applications (e.g., third-party application 1 and third-party application 2). When deployed on server 200, server 200 can store intent recognition models corresponding to different voice skills of the third-party application.

[0138] Another example: Figure 4 The human-computer dialogue system shown can deploy the fastest or best learned intent recognition model on the server 200, and the server 200 can store the intent recognition models corresponding to different voice skills of the third-party application 1.

[0139] The application of the intent recognition model can be referred to the previous description and will not be repeated here.

[0140] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0141] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0143] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks or optical disks.

[0145] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A learning method for an intent recognition model, characterized in that: The method comprises: The server receives positive data corresponding to each intent in the first skill input by the skill developer; The server generates negative data corresponding to each intention in the first skill according to the positive data corresponding to each intention in the first skill; The server obtains training data corresponding to a second skill, where the second skill is a skill similar to the first skill, and the number of the second skill is at least one; The server learns according to the training data corresponding to the second skill and the preset first base model to generate a second base model, wherein the number of the second base model is at least one; the server learns according to the positive data corresponding to each intent in the first skill and the negative data corresponding to each intent in the first skill, and the second base model to generate an intent recognition model; The server generates negative data corresponding to each intention in the first skill according to the positive data corresponding to each intention in the first skill, including: The server extracts keywords corresponding to each intent in the first skill for each intent in the first skill, wherein the keywords are key features that affect the weight of the first base model; The keywords corresponding to different intentions in the first skill are combined, or the keywords corresponding to different intentions are combined with words not related to the first skill, and the combined words are determined as negative data corresponding to the different intentions.

2. The method for learning an intention recognition model according to claim 1, characterized in that: The server generating negative data corresponding to each intent in the first skill according to the positive data corresponding to each intent in the first skill also includes: The server determines a first set from the entire set of training data stored on the server according to the classification of the first skill, wherein the training data in the first set includes negative data corresponding to each intent in the first skill; wherein the training data includes positive data and negative data; Sampling a preset amount of training data from the first set; Manual labeling and / or clustering algorithms are used to determine negative data corresponding to each intention in the first skill from the sampled training data.

3. The method for learning an intention recognition model according to claim 1 or 2, characterized in that: The server acquiring data corresponding to the second skill includes: The server determines the second skill according to the classification of the first skill and / or the positive data of the first skill; The training data corresponding to the second skill is obtained through the shared layer.

4. The method for learning an intention recognition model according to claim 1 or 2, characterized in that: The server performs learning according to the training data corresponding to the second skill and the preset first base model to generate the second base model, including: The server uses a multi-task learning method to learn according to the training data corresponding to the second skill and the preset first base model to generate a second base model.

5. The method for learning an intention recognition model according to claim 1 or 2, characterized in that: The server generates an intent recognition model by learning according to the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model, including: The server uses an integrated learning method to learn based on the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model to generate an intent recognition model.

6. The method for learning an intention recognition model according to claim 5, characterized in that: The server generates an intention recognition model specifically including: The server generates a plurality of intermediate models in the process of generating the intention recognition model; The server determines that the first generated intermediate model is the fastest intent recognition model; and / or, the server determines the best intent recognition model after performing model selection and parameter adjustment on the multiple intermediate models.

7. The method for learning an intention recognition model according to claim 6, characterized in that: The server determines that the best intent recognition model includes: The server calculates a first accuracy rate of each intermediate model according to positive data corresponding to each intent in the first skill; the server calculates a second accuracy rate of each intermediate model according to negative data corresponding to each intent in the first skill; The server determines the best intent recognition model after selecting models and adjusting parameters of the multiple intermediate models according to the first accuracy rate, the second accuracy rate, and the weights input by the skill developer.

8. A server, characterized in that: include: A processor, a memory and a communication interface; the memory is used to store computer program codes, the computer program codes include computer instructions, when the processor reads the computer instructions from the memory, the server performs the following operations: Receiving, through the communication interface, positive data corresponding to each intention in the first skill input by the skill developer; generating negative data corresponding to each intention in the first skill according to the positive data corresponding to each intention in the first skill; Acquire training data corresponding to a second skill, where the second skill is a skill similar to the first skill, and the number of the second skill is at least one; Learning according to the training data corresponding to the second skill and the preset first base model to generate a second base model, where the number of the second base model is at least one; Generate an intent recognition model by learning according to the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model; The server generates negative data corresponding to each intention in the first skill according to the positive data corresponding to each intention in the first skill, specifically including: The server extracts keywords corresponding to each intent in the first skill for each intent in the first skill, wherein the keywords are key features that affect the weight of the first base model; The keywords corresponding to different intentions in the first skill are combined, or the keywords corresponding to different intentions are combined with words not related to the first skill, and the combined words are determined as negative data corresponding to the different intentions.

9. The server according to claim 8, characterized in that: The server generating negative data corresponding to each intention in the first skill according to the positive data corresponding to each intention in the first skill further specifically includes: The server determines a first set from the entire set of training data stored on the server according to the classification of the first skill, wherein the training data in the first set includes negative data corresponding to each intent in the first skill; wherein the training data includes positive data and negative data; Sampling a preset amount of training data from the first set; Manual labeling and / or clustering algorithms are used to determine negative data corresponding to each intention in the first skill from the sampled training data.

10. The server according to claim 8 or 9, characterized in that: The server acquiring data corresponding to the second skill includes: The server determines the second skill according to the classification of the first skill and / or the positive data of the first skill; The training data corresponding to the second skill is obtained through the shared layer.

11. The server according to claim 8 or 9, characterized in that: The server performs learning according to the training data corresponding to the second skill and the preset first base model to generate the second base model, including: The server uses a multi-task learning method to learn according to the training data corresponding to the second skill and the preset first base model to generate a second base model.

12. The server according to claim 8 or 9, characterized in that: The server generates an intent recognition model by learning according to the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model, including: The server uses an integrated learning method to learn based on the positive data corresponding to each intent in the first skill, the negative data corresponding to each intent in the first skill, and the second base model to generate an intent recognition model.

13. The server according to claim 12, characterized in that: The server-generated intent recognition model also specifically includes: The server generates a plurality of intermediate models in the process of generating the intention recognition model; The server determines that the first generated intermediate model is the fastest intent recognition model; and / or, the server determines the best intent recognition model after performing model selection and parameter adjustment on the multiple intermediate models.

14. The server according to claim 13, characterized in that: The server determines that the best intent recognition model includes: The server calculates a first accuracy rate of each intermediate model according to positive data corresponding to each intent in the first skill; the server calculates a second accuracy rate of each intermediate model according to negative data corresponding to each intent in the first skill; The server determines the best intent recognition model after selecting models and adjusting parameters of the multiple intermediate models according to the first accuracy rate, the second accuracy rate, and the weights input by the skill developer.

Images