Intention recognition method, system, terminal and medium based on human-computer dialogue

Abstract

The present invention provides an intent recognition method, system, terminal and medium based on human-computer dialogue, aiming to obtain a high-confidence dialogue data set and a low-confidence dialogue data set based on online user historical characteristics, and to construct a new standard question training data set and an existing standard question training data set respectively, so as to train an intent recognition model; the present invention can use a small amount (for example, 20-50) of labeled samples to annotate online historical data into training data, realize semi-automatic update of the online intent recognition model, effectively reduce the company's manpower expenditure on model training sample annotators, while increasing the model iteration speed and improving the product experience of knowledge base operators.

Description

Technical Field

[0001] The present invention relates to the fields of intelligent customer service and machine learning, and in particular to an intention recognition method, system, terminal and medium based on human-computer dialogue. Background Art

[0002] In order to improve user experience and solve some pre-sales and after-sales problems, current C-end application software will have an intelligent conversation robot to provide assistance to users. The technical core of task-based intelligent conversation robots is intent recognition. The mainstream industrial-grade intent recognition solution is a supervised task, that is, the intention of the user's expression is recognized by manually labeling the conversation type label. This solution has accurate recognition effect, but the disadvantage is that it requires a lot of manpower to label historical conversations, which will result in slow system iteration and high maintenance costs. Summary of the invention

[0003] In view of the above-mentioned defects of the prior art, the present invention provides an intention recognition method, system, terminal and medium based on human-computer dialogue, which is used to solve the problem that the existing intention recognition solution requires a large amount of manpower to mark historical dialogues, which will result in slow system iteration and update speed and high maintenance cost.

[0004] To achieve the above-mentioned objectives, the present invention provides an intention recognition method based on human-computer dialogue, comprising: obtaining online user historical features; based on the online user historical features, screening a high-confidence dialogue data set and a low-confidence dialogue data set; adding a number of standard questions and corresponding similar questions, and obtaining a newly added standard question training data set based on the low-confidence dialogue data set; obtaining existing online standard questions and corresponding similar questions, and obtaining an existing standard question training data set based on the high-confidence dialogue data set; merging the newly added standard question training data set and the existing standard question training data set to obtain a total training set, so as to perform model training and obtain an intention recognition model.

[0005] In a preferred embodiment of the present invention, the screening of high-confidence conversation data sets and low-confidence conversation data sets based on the online user historical features includes: clustering the historical conversation data into multiple conversation data sets according to user numbers; filtering each of the conversation data sets according to preset filtering rules to obtain a filtered conversation data set; traversing the filtered conversation data sets, and performing secondary filtering on each of the filtered conversation data sets according to whether the user conversation is unclear and whether there are multiple intent expressions, to obtain a conversation data set expressed by a single intent user; and classifying the conversation data set expressed by a single intent user into a high-confidence conversation data set and a low-confidence conversation data set according to preset confidence classification rules.

[0006] In another preferred embodiment of the present invention, the preset filtering rule includes: for the same user ID, if the interval between two adjacent conversations is less than a preset time interval, only the most recent conversation is retained, otherwise all conversations are retained.

[0007] In another preferred embodiment of the present invention, the secondary filtering of each of the filtered conversation data sets includes: setting a high character count threshold and a low character count threshold; if the number of characters in the user conversation is lower than the low character count threshold, it is determined that the current user conversation is not clearly expressed; if the number of characters in the user conversation is higher than the high character count threshold, it is determined that the current user conversation has multiple intent expressions.

[0008] In another preferred embodiment of the present invention, the method of obtaining a new standard question training data set based on the low-confidence dialogue data set includes: training multiple base models based on several newly added standard questions and corresponding similar questions; inputting the low-confidence dialogue data set into the multiple base models respectively to obtain the probability distribution of the user's expression for each standard question, and outputting the standard questions greater than a preset probability threshold as candidate tags for the current user expression; and outputting the candidate tags whose proportion among all candidate tags exceeds the preset proportion threshold as the true tags for the current user expression.

[0009] In another preferred embodiment of the present invention, the method of obtaining an existing standard question training dataset based on the high-confidence dialogue dataset includes: using existing online standard questions and their corresponding similar questions to train a classifier model; inputting the high-confidence dialogue dataset into the classifier model to obtain the distribution probability of similar questions for each standard question, and outputting the standard questions with a probability greater than a preset probability threshold as candidate tags for the current similar question; and outputting the candidate tags whose proportion among all candidate tags exceeds the preset proportion threshold as the true tags of the current similar question.

[0010] In another preferred embodiment of the present invention, after obtaining the existing standard question training data set, noise filtering is performed on it, and the filtering process includes: counting the sum of the probabilities of all similar questions under each standard question to the standard question, and calculating the average probability value as the probability threshold of the standard question, and eliminating similar questions with probabilities lower than the probability threshold; correcting the joint count matrix of the real label and the online label to compensate for the total number of similar question sets after eliminating some similar questions; normalizing the joint count matrix to obtain the joint probability distribution of the real label and the online label; and obtaining the existing standard question training data set after noise filtering based on the joint probability distribution.

[0011] To achieve the above-mentioned objectives, the present invention provides an intention recognition system based on human-computer dialogue, comprising: a feature acquisition module, used to acquire online user historical features; a data set generation module, used to screen a high-confidence dialogue data set and a low-confidence dialogue data set based on the online user historical features; a training data set generation module, used to add a number of standard questions and corresponding similar questions, and obtain a newly added standard question training data set based on the low-confidence dialogue data set; and to acquire existing online standard questions and corresponding similar questions, and obtain an existing standard question training data set based on the high-confidence dialogue data set; an intention recognition model generation module, used to merge the newly added standard question training data set and the existing standard question training data set to obtain a total training set, so as to perform model training and obtain an intention recognition model.

[0012] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for intention recognition based on human-computer dialogue is implemented.

[0013] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides an electronic terminal, including: a processor and a memory; the memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that the terminal executes the intention recognition method based on human-computer dialogue.

[0014] The human-computer dialogue-based intention recognition method, system, terminal and medium provided by the present invention have the following technical effects: the present invention obtains a high-confidence dialogue data set and a low-confidence dialogue data set based on online user historical characteristics, and constructs a new standard question training data set and an existing standard question training data set respectively, thereby training an intention recognition model; the present invention can use a small amount (for example, 20-50) of labeled samples to annotate online historical data into training data, realizes semi-automatic update of the online intention recognition model, and can effectively reduce the company's manpower expenditure on model training sample annotators, while increasing the model iteration speed and improving the product experience of knowledge base operators.

[0015] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a flowchart of an intention recognition method based on human-computer dialogue in an embodiment of the present invention.

[0017] Figure 2 It is a structural diagram of an intention recognition system based on human-computer dialogue in an embodiment of the present invention.

[0018] Figure 3 It is a structural schematic diagram of an electronic terminal in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0020] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present invention in a schematic manner, and thus the illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0021] While some exemplary embodiments of the present invention have been described for the purpose of illustration, it should be understood that the present invention may be implemented in other ways not specifically shown in the drawings.

[0022] In order to solve the problem in the prior art that data labeling requires a lot of manpower investment, the present application provides an intent recognition method, system, terminal and medium based on human-computer dialogue, aiming to utilize online user behavior characteristics and a small amount of data configured by knowledge base operators to realize automatic labeling and reflux of online historical dialogue data for training and iteration of new intent recognition models. The dialogue system transformed by this method can greatly reduce the manpower investment of annotators (personnel who perform data labeling), while increasing the speed of model iteration and improving the product experience of knowledge base operators.

[0023] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the technical solution in the embodiments of the present invention is further described in detail through the following embodiments and in combination with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the invention.

[0024] like Figure 1 As shown, a flow chart of an intention recognition method based on human-computer dialogue in an embodiment of the present invention is shown.

[0025] In this embodiment, the intention recognition method based on human-computer dialogue mainly includes steps S11 to S16. It should be noted that the above method can be applied to a variety of hardware devices, such as ARM (Advanced RISC Machines) controller, FPGA (Field Programmable Gate Array) controller, SoC (System on Chip) controller, DSP (Digital Signal Processing) controller, or MCU (Microcontroller Unit) controller, etc.; for another example, it can be applied to personal computers such as desktop computers, laptops, tablet computers, smart phones, smart bracelets, smart watches, smart helmets, etc.; for another example, it can be applied to servers, and the servers can be arranged on one or more physical servers according to various factors such as functions and loads, or can be composed of distributed or centralized server clusters.

[0026] Hereinafter, each step of the method for intention recognition based on human-computer dialogue of this embodiment will be explained in detail.

[0027] Step S11: Obtain online user historical features.

[0028] Specifically, the online user history characteristics include but are not limited to user click behavior characteristics, characteristics of users re-entering the customer service robot within a certain period of time, characteristics of user conversation time intervals, characteristics of the number of questions asked by users, characteristics of the robot's recommended answers or question lists, characteristics of whether the user is transferred to a manual operator, characteristics of the number of business question categories asked by users, characteristics of the number of characters in the user's expression, and the like.

[0029] It should be noted that user input and user behavior are for robots. The above-mentioned robots can be industrial robots or special robots; industrial robots refer to multi-joint manipulators or multi-degree-of-freedom robots for the industrial field; special robots are various advanced robots used in non-manufacturing industries and serving humans, including: service robots, underwater robots, entertainment robots, military robots, agricultural robots, etc., which are not specifically limited in this embodiment.

[0030] Step S12: based on the online user historical features, screening high-confidence dialogue data sets and low-confidence dialogue data sets.

[0031] It should be understood that confidence is also called confidence level. In statistics, the confidence interval of a probability sample is an interval estimate of a population parameter of the sample; the confidence interval shows the degree to which the true value of the parameter has a certain probability of falling around the measurement result; the confidence interval gives the range of credibility of the measured value of the measured parameter, that is, the "certain probability" required above, and this probability is called the confidence level. On this basis, the high-confidence dialogue data set in this embodiment refers to a data set with high confidence, and the low-confidence dialogue data set refers to a data set with low confidence.

[0032] In this embodiment, the screening process of the high-confidence dialogue data set and the low-confidence dialogue data set is as follows:

[0033] Step a: Cluster the historical conversation data into multiple conversation data sets according to the user ID. For ease of description, the multiple conversation data sets obtained by clustering can be recorded as data set D = {D1, D2, ..., D n}, where dataset D1 corresponds to a conversation dataset based on the conversation between the user with user ID1 and the robot, dataset D2 corresponds to a conversation dataset based on the conversation between the user with user ID2 and the robot, and dataset D n The corresponding ID is based on the user number n The conversation dataset is obtained by the conversation between users and robots.

[0034] Step b: Filter each of the conversation data sets according to the preset filtering rules to obtain a filtered conversation data set. For ease of description, the filtered conversation data set can be recorded as S = {S1, S2, ..., S n}.

[0035] In some examples, the preset filtering rule includes: for the same user ID, if the interval between two adjacent conversations is less than a preset time interval, only the most recent conversation is retained, otherwise all conversations are retained.

[0036] Step c: traverse the filtered conversation data set, and perform secondary filtering on each of the filtered conversation data sets according to whether the user conversation is unclear and whether there are multiple intent expressions, to obtain a conversation data set of a single intent user expression.

[0037] In some examples, the method of performing secondary filtering on each of the filtered conversation data sets includes: setting a high character count threshold and a low character count threshold; if the number of characters in the user conversation is lower than the low character count threshold, it is determined that the current user conversation is not clearly expressed; if the number of characters in the user conversation is higher than the high character count threshold, it is determined that the current user conversation has multiple intent expressions.

[0038] Step d: Classify the conversation data set expressed by the single-intent user into a high-confidence conversation data set and a low-confidence conversation data set according to a preset confidence classification rule.

[0039] The preset confidence classification rules include: confidence classification based on whether it is clicked by the user and whether the dialogue in which the expression is located is transferred to manual service. The specific content is as follows: the user expressions that are clicked by the user and not transferred to manual service are merged to obtain a high-confidence dialogue candidate data set, and the remaining user expressions are merged to obtain a low-confidence dialogue candidate data set. Among them, each sample in each candidate data set contains two parts, namely, the user dialogue expression and the intention recognition standard sentence; the user dialogue expression refers to the original words of the user in the human-computer dialogue. Since each user has different expressions, these user dialogue expressions are not standard; and the intention recognition standard sentence refers to the standard sentence extracted after the intention recognition of the user dialogue expressions with the same meaning but different expressions. For example, user dialogue expression A is "Excuse me, where is the convenience store nearby?", and user dialogue expression B is "I want to go to the convenience store to buy snacks." Although the expressions are different, the meaning is the same. Therefore, after the intention recognition, the user dialogue expression A and the user dialogue expression B can be refined to obtain the same standard sentence C "the nearest convenience store". Based on this example, two samples can be generated. The first sample includes user dialogue expression A and intent recognition standard sentence C; the second sample includes user dialogue expression B and intent recognition standard sentence C.

[0040] Step S13: adding a number of standard questions and corresponding similar questions, and obtaining a new standard question training data set based on the low-confidence dialogue data set.

[0041] In some examples, the method of adding several standard questions and corresponding similar questions includes: the online dialogue robot counts the conversion rate of standard questions in the knowledge base, so that the knowledge base operator can manually split, merge, add and other optimization operations on the standard questions based on the indicators. At the same time, a certain number of typical user expressions are configured for each standard question, and this number is usually 20 to 50, but this embodiment does not specifically limit this number. All these typical user expressions are called similar question sets.

[0042] Furthermore, multiple base models are trained based on several newly added standard questions and corresponding similar questions. These models include but are not limited to pre-trained models such as Robert, Bert, and xlnet.

[0043] Subsequently, the low confidence dialogue data set in step S12 is respectively input into the above-mentioned multiple pre-trained models, and the model outputs the probability distribution of the user's expression for each standard question. A probability threshold Q is set, and the standard question greater than the probability threshold will be selected as the candidate label output of the current user expression.

[0044] Next, vote on the output results of all models and set a ratio r. If the recognition ratio of the standard question in N models is greater than r, the output candidate label is used as the true label of the user statement. Each dialogue data in the low-confidence dialogue data set is operated according to the above method to obtain the newly added standard question training data set T newfaq For example, 5 base models are trained and the recognition ratio r is set to 0.5. Each base model outputs a candidate label (for example, the output candidate labels are set to be candidate label A, candidate label A, candidate label C, candidate label D, and candidate label A). Since the number of candidate label A accounts for 0.6 of the total number of candidate labels, which is greater than the recognition ratio r (0.5), candidate label A can be used as the true label of the user statement. Therefore, a user statement corresponds to a true label as a record, and all the records eventually form the newly added standard training data set T. newfaq .

[0045] Step S14: Obtain existing online standard questions and corresponding similar questions, and obtain an existing standard question training dataset based on the high-confidence dialogue dataset.

[0046] Specifically, the existing online standard questions and corresponding similar questions are used to train the classifier model. The loss function used by the classifier model can be the cross entropy loss function, and its formula is as follows:

[0047]

[0048] Among them, m represents the number of samples, n represents the number of categories of online standard questions, y represents the true standard question distribution, and p represents the predicted probability distribution.

[0049] After the classifier model is obtained, the high confidence dialogue data set in step S12 is input into the classifier model, and the output of the model softmax layer is taken as the model output result to obtain the distribution probability P of the sample in the standard question.

[0050] Furthermore, since the high-confidence conversation data set is obtained based on the online user historical features, and the user input and user behavior usually include abnormal data such as random user clicks, that is, there is a lot of noise data in the data. Therefore, in order to filter the noise in the data, the standard question label Y in the high-confidence data and the distribution probability P of the model output are calculated for confidence, and the statistically reliable data is obtained as the training data set T oldfaq Among them, the specific steps of confidence calculation are as follows:

[0051] Step 1: Count the sum of the probabilities of all similar questions under each standard question to the standard question, and calculate the average probability value as the probability threshold of the standard question, and eliminate similar questions with probabilities lower than the probability threshold.

[0052] The calculation formula of the probability threshold is as follows:

[0053]

[0054] Among them, t j is the threshold of standard question j, the total number of standard questions is n, m is the total number of similar questions, p i is the probability value of similar question i for standard question j. Take the maximum probability p of the similar question model j Greater than t j The corresponding standard question is taken as the true label. If the maximum probability p j No more than t j , it should be eliminated; then count the joint count matrix of the real label and the online label (i.e., the online data obtained in the above step S12) Where y is the true label, y * Online tags are obtained by filtering based on user behavior.

[0055] Step 2: Correct the joint count matrix of the real label and the online label to compensate for the total number of similar questions after removing some similar questions. Since some similar questions have been removed, the total number of sample sets of the real label and the online label is inconsistent, so matrix correction is required to obtain a new count matrix. The calculation formula is as follows:

[0056]

[0057] Among them, |X y＝i | is the total number of samples with online label y=i.

[0058] Step 3: Normalize the joint count matrix to obtain the joint probability distribution of the real label and the online label. The calculation formula is as follows:

[0059]

[0060] in, is the corrected count matrix.

[0061] Step 4: Based on the joint probability distribution, obtain the existing standard training data set after noise filtering.

[0062] Specifically, for each type of standard question, select the non-diagonal unit value corresponding to the joint probability distribution and multiply it by the number of similar questions of this category in the online data. The value is rounded down to determine the number of similar questions x to be filtered. Sort by the lowest probability and remove x samples. All categories are processed to obtain the existing standard question training data set after noise filtering.

[0063] Step S15: After merging the newly added standard question training data set and the existing standard question training data set, a total training set is obtained to perform model training and obtain an intent recognition model. That is, the data set T newfaq and dataset T oldfaq Merge to get T train The online intent classification model is trained using the data set to obtain a new intent recognition model, which is then automatically updated after being deployed online.

[0064] Based on the above method steps, a small amount (for example, 20-50) of labeled samples can be used to annotate online historical data into training data, thereby realizing semi-automatic updating of the online intent recognition model. This can effectively reduce the company's manpower expenditure on model training sample labeling personnel, while increasing the speed of model iteration and improving the product experience of knowledge base operators.

[0065] like Figure 2 , which shows a schematic diagram of the structure of an intention recognition system based on human-computer dialogue in an embodiment of the present invention. The intention recognition system 200 in this embodiment includes: a feature acquisition module 201, a data set generation module 202, a training data set generation module 203, and an intention recognition model generation module 204.

[0066] The feature acquisition module 201 is used to acquire online user historical features.

[0067] Specifically, the user input and user behavior characteristics include but are not limited to: user click behavior, characteristics of the user re-entering the customer service robot within a certain period of time, characteristics of the time interval of user conversations, characteristics of the number of times the user asks questions, characteristics of the robot's recommended answers or question lists, characteristics of whether the user is transferred to manual operation, characteristics of the number of business question categories asked by the user, characteristics of the number of characters in the user's expression, etc.

[0068] The data set generation module 202 is used to screen the high-confidence conversation data set and the low-confidence conversation data set based on the online user history features.

[0069] In this embodiment, the screening process of the high-confidence dialogue data set and the low-confidence dialogue data set is as follows:

[0070] Step a: Cluster the historical conversation data into multiple conversation data sets according to the user ID. For ease of description, the multiple conversation data sets obtained by clustering can be recorded as data set D = {D1, D2, ..., D n}, where dataset D1 corresponds to a conversation dataset based on the conversation between the user with user ID1 and the robot, dataset D2 corresponds to a conversation dataset based on the conversation between the user with user ID2 and the robot, and dataset D n The corresponding ID is based on the user number nThe conversation dataset is obtained by the conversation between users and robots.

[0071] Step b: Filter each of the conversation data sets according to the preset filtering rules to obtain a filtered conversation data set. For ease of description, the filtered conversation data set can be recorded as S = {S1, S2, ..., S n}.

[0072] In some examples, the preset filtering rule includes: for the same user ID, if the interval between two adjacent conversations is less than a preset time interval, only the most recent conversation is retained, otherwise all conversations are retained.

[0073] Step c: traverse the filtered conversation data set, and perform secondary filtering on each of the filtered conversation data sets according to whether the user conversation is unclear and whether there are multiple intent expressions, to obtain a conversation data set of a single intent user expression.

[0074] In some examples, the method of performing secondary filtering on each of the filtered conversation data sets includes: setting a high character count threshold and a low character count threshold; if the number of characters in the user conversation is lower than the low character count threshold, it is determined that the current user conversation is not clearly expressed; if the number of characters in the user conversation is higher than the high character count threshold, it is determined that the current user conversation has multiple intent expressions.

[0075] Step d: Classify the conversation data set expressed by the single-intent user into a high-confidence conversation data set and a low-confidence conversation data set according to a preset confidence classification rule.

[0076] The preset confidence classification rules include: confidence classification based on whether it is clicked by the user and whether the dialogue in which the expression is located is transferred to manual service. The specific content is as follows: the user expressions that are clicked by the user and not transferred to manual service are merged to obtain a high-confidence dialogue candidate data set, and the remaining user expressions are merged to obtain a low-confidence dialogue candidate data set. Each sample in each candidate data set contains two parts, namely the user dialogue expression and the intention recognition standard sentence.

[0077] The training data set generation module 203 is used to add a number of standard questions and corresponding similar questions, and obtain a new standard question training data set based on the low-confidence dialogue data set.

[0078] In some examples, the method of adding several standard questions and corresponding similar questions includes: the online dialogue robot counts the conversion rate of standard questions in the knowledge base, so that the knowledge base operator can manually split, merge, add and other optimization operations on the standard questions based on the indicators. At the same time, a certain number of typical user expressions are configured for each standard question, and this number is usually 20 to 50, but this embodiment does not specifically limit this number. All these typical user expressions are called similar question sets.

[0079] Furthermore, multiple base models are trained based on several newly added standard questions and corresponding similar questions. These models include but are not limited to pre-trained models such as Robert, Bert, and xlnet.

[0080] Subsequently, the low confidence dialogue data set in step S12 is respectively input into the above-mentioned multiple pre-trained models, and the model outputs the probability distribution of the user's expression for each standard question. A probability threshold Q is set, and the standard question greater than the probability threshold will be selected as the candidate label output of the current user expression.

[0081] Next, vote on the output results of all models and set a ratio r. If the recognition ratio of the standard question in N models is greater than r, the output candidate label is used as the true label of the user statement. Each dialogue data in the low-confidence dialogue data set is operated according to the above method to obtain the newly added standard question training data set T newfaq For example, 5 base models are trained and the recognition ratio r is set to 0.5. Each base model outputs a candidate label (for example, the output candidate labels are set to be candidate label A, candidate label A, candidate label C, candidate label D, and candidate label A). Since the number of candidate label A accounts for 0.6 of the total number of candidate labels, which is greater than the recognition ratio r (0.5), candidate label A can be used as the true label of the user statement. Therefore, a user statement corresponds to a true label as a record, and all the records eventually form the newly added standard training data set T. newfaq .

[0082] The training data set generation module 203 is further used to obtain existing online standard questions and corresponding similar questions, and obtain an existing standard question training data set based on the high-confidence dialogue data set.

[0083] Specifically, the existing online standard questions and corresponding similar questions are used to train the classifier model. The loss function used by the classifier model can be the cross entropy loss function, and its formula is as follows:

[0084]

[0085] Among them, m represents the number of samples, n represents the number of categories of online standard questions, y represents the true standard question distribution, and p represents the predicted probability distribution.

[0086] After the classifier model is obtained, the high confidence dialogue data set in step S12 is input into the classifier model, and the output of the model softmax layer is taken as the model output result to obtain the distribution probability P of the sample in the standard question.

[0087] Furthermore, since the high-confidence conversation data set is obtained based on the online user historical features, and the user input and user behavior usually include abnormal data such as random user clicks, that is, there is a lot of noise data in the data. Therefore, in order to filter the noise in the data, the standard question label Y in the high-confidence data and the distribution probability P of the model output are calculated for confidence, and the statistically reliable data is obtained as the training data set T oldfaq .

[0088] The intent recognition model generation module 204 is used to merge the newly added standard question training data set and the existing standard question training data set to obtain a total training set, so as to perform model training and obtain an intent recognition model.

[0089] It should be understood that the division of the modules of the above system is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the data set generation module can be a separately established processing element, or it can be integrated in a chip of the above system. In addition, it can also be stored in the memory of the above system in the form of program code, and called and executed by a processing element of the above system. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0090] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0091] like Figure 3 As shown, a structural schematic diagram of an electronic terminal provided in this example is shown, including: a processor 31, a memory 32, and a communicator 33; the memory 32 is connected to the processor 31 and the communicator 33 through a system bus and completes communication with each other, the memory 32 is used to store computer programs, the communicator 33 is used to communicate with other devices, and the processor 31 is used to run computer programs, so that the electronic terminal executes the various steps of the intention recognition method based on human-computer dialogue as mentioned above.

[0092] The system bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The communication interface is used to realize the communication between the database access device and other devices (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage.

[0093] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0094] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the intention recognition method based on human-computer dialogue is implemented.

[0095] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0096] In the embodiments provided in the present application, the computer readable and writable storage medium may include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a disk storage device or other magnetic storage device, a flash memory, a USB flash drive, a mobile hard disk, or any other medium that can be used to store a desired program code in the form of an instruction or data structure and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if the instruction is sent from a website, a server or other remote source using a coaxial cable, an optical fiber cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, optical fiber cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. However, it should be understood that computer readable and writable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but are intended to be non-temporary, tangible storage media. Disk and disc, as used in this application, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.

[0097] In summary, the present invention provides an intent recognition method, system, terminal and medium based on human-computer dialogue, aiming to obtain a high-confidence dialogue data set and a low-confidence dialogue data set based on online user historical characteristics, and to construct a new standard question training data set and an existing standard question training data set respectively, so as to train an intent recognition model; the present invention can use a small amount (for example, 20-50) of labeled samples to annotate online historical data into training data, realize semi-automatic update of the online intent recognition model, and can effectively reduce the company's manpower expenditure on model training sample annotators, while increasing the model iteration speed and improving the product experience of knowledge base operators.

[0098] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for intention recognition based on human-computer dialogue, characterized in that: include: Obtain historical characteristics of online users; Based on the online user historical features, screening a high-confidence dialogue data set and a low-confidence dialogue data set; Adding several standard questions and corresponding similar questions, and obtaining a new standard question training data set based on the low-confidence dialogue data set; Obtain existing online standard questions and corresponding similar questions, and obtain an existing standard question training dataset based on the high-confidence dialogue dataset; The newly added standard question training data set and the existing standard question training data set are combined to obtain a total training set to perform model training and obtain an intent recognition model; The step of obtaining a new standard question training data set based on the low-confidence dialogue data set includes: Based on several newly added standard questions and corresponding similar questions, multiple base models are trained; Inputting the low-confidence dialogue data sets into the multiple base models respectively, obtaining the probability distribution of the user's expression for each standard question, and outputting the standard questions greater than a preset probability threshold as candidate tags for the current user's expression; The candidate tags whose proportion among all candidate tags exceeds the preset proportion threshold are output as the true tags expressed by the current user.

2. The intention recognition method based on human-computer dialogue as claimed in claim 1, characterized in that: The step of screening the high-confidence conversation data set and the low-confidence conversation data set based on the online user historical features includes: Cluster the historical conversation data into multiple conversation data sets according to user IDs; Filtering each of the conversation data sets according to a preset filtering rule to obtain a filtered conversation data set; Traversing the filtered conversation data sets, performing secondary filtering on each of the filtered conversation data sets according to whether the user conversation is unclear and whether there are multiple intent expressions, to obtain a conversation data set of a single intent user expression; According to a preset confidence classification rule, the conversation data set expressed by the single-intent user is classified into a high-confidence conversation data set and a low-confidence conversation data set.

3. The intention recognition method based on human-computer dialogue as claimed in claim 2, characterized in that: The preset filtering rule includes: for the same user number, if the interval between two adjacent conversations is less than a preset time interval, only the most recent conversation is retained, otherwise all conversations are retained.

4. The method for intention recognition based on human-computer dialogue as claimed in claim 2, characterized in that: The secondary filtering of each of the filtered conversation data sets includes: setting a high character count threshold and a low character count threshold; if the number of characters in the user conversation is lower than the low character count threshold, determining that the current user conversation is not clearly expressed; if the number of characters in the user conversation is higher than the high character count threshold, determining that the current user conversation has multiple intent expressions.

5. The method for intention recognition based on human-computer dialogue as claimed in claim 1, characterized in that: The obtaining of an existing standard question training data set based on the high confidence dialogue data set includes: Use existing online standard questions and their corresponding similar questions to train the classifier model; Input the high-confidence dialogue data set into the classifier model to obtain the distribution probability of similar questions for each standard question, and output the standard questions with probability greater than a preset threshold as candidate labels for the current similar question; The candidate tags whose proportion among all candidate tags exceeds the preset proportion threshold are output as the true tags of the current similar question.

6. The intention recognition method based on human-computer dialogue as claimed in claim 5, characterized in that: After obtaining the existing standard training data set, the noise is filtered out, and the filtering process includes: The sum of the probabilities of all similar questions under each standard question with respect to the standard question is calculated, and the average probability value is calculated as the probability threshold of the standard question, and similar questions with probabilities lower than the probability threshold are eliminated; Correct the joint count matrix of the real labels and online labels to compensate for the total number of similar questions after removing some similar questions; Normalizing the joint count matrix to obtain a joint probability distribution of real labels and online labels; Based on the joint probability distribution, an existing standard training data set is obtained after noise filtering.

7. An intention recognition system based on human-computer dialogue, characterized in that: include: Feature acquisition module, used to obtain historical features of online users; A data set generation module, used to screen a high-confidence conversation data set and a low-confidence conversation data set based on the online user historical features; A training data set generation module, used to add a number of standard questions and corresponding similar questions, and obtain a new standard question training data set based on the low-confidence dialogue data set; and obtaining existing online standard questions and corresponding similar questions, and obtaining an existing standard question training dataset based on the high-confidence dialogue dataset; An intent recognition model generation module is used to merge the newly added standard question training data set and the existing standard question training data set to obtain a total training set to perform model training and obtain an intent recognition model; The training data set generation module obtains a new standard question training data set based on the low-confidence dialogue data set, including: Based on several newly added standard questions and corresponding similar questions, multiple base models are trained; Inputting the low-confidence dialogue data sets into the multiple base models respectively, obtaining the probability distribution of the user's expression for each standard question, and outputting the standard questions greater than a preset probability threshold as candidate tags for the current user's expression; The candidate tags whose proportion among all candidate tags exceeds the preset proportion threshold are output as the true tags expressed by the current user.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for intention recognition based on human-computer dialogue described in any one of claims 1 to 6 is implemented.

9. An electronic terminal, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory, so that the terminal executes the intention recognition method based on human-computer dialogue as described in any one of claims 1 to 6.

Images