Intention recognition method and device, equipment, storage medium and program product

Abstract

The invention provides an intention recognition method, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining a message text of a customer in a customer service process; based on an intention recognition model, the client intention corresponding to the message text is judged, and the intention recognition model is obtained through pre-training and comprises the steps that the to-be-trained question text and the client preference data are preprocessed; based on the preprocessed question text content, text representation based on a self-attention mechanism is obtained; based on an interactive attention mechanism, mapping the preprocessed customer preference data into question text representation to obtain question text representation fused with customer preferences; splicing the text representation based on the self-attention mechanism and the question text representation fused with the customer preference to obtain a target representation of a to-be-trained question text; and training the target representation to obtain an intention recognition model. The invention further provides an intention recognition device, intention recognition equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and more specifically to an intent recognition method, apparatus, device, storage medium, and program product. Background Art

[0002] In the current digital economy and information age, the quality and efficiency of customer service are increasingly valued by enterprises. With the rapid development of online service platforms, the communication methods between customers and enterprises are becoming increasingly diverse, and customer needs and intentions are becoming more complex. Whether in e-commerce, finance or technical support, how to accurately understand customer intentions and respond quickly is the key to improving customer satisfaction and loyalty.

[0003] With the vigorous development of digital technologies such as big data and artificial intelligence, banks and other financial institutions are also constantly promoting digital transformation and creating new windows and new models for customer service in the digital ecosystem. Among them, intelligent customer service understands business demands and analyzes target documents through natural language interaction mode, and can automatically give reasonable answers based on customer questions, creating an online and humanized customer service system, which is an important way to provide more comprehensive, professional and accurate services. Intent recognition, as an important technology in natural language processing, aims to extract the real needs and purposes behind the user's input text. Traditional intent recognition methods mainly rely on rule-based pattern matching or simple machine learning algorithms. These methods often have limitations when processing complex and changeable customer requests and cannot meet the personalized needs of customers. Summary of the invention

[0004] In view of the above problems, the present disclosure provides an intent recognition method, apparatus, device, storage medium and program product.

[0005] According to a first aspect of the present disclosure, there is provided an intent recognition method, including: obtaining a message text of a customer during a customer service process; judging the customer intent corresponding to the message text based on an intent recognition model, wherein the intent recognition model is obtained through pre-training, including: pre-processing a question text to be trained and customer preference data respectively; obtaining a text representation based on a self-attention mechanism based on the preprocessed question text content; mapping the preprocessed customer preference data to the question text representation based on an interactive attention mechanism to obtain a question text representation that integrates customer preferences; concatenating the text representation based on the self-attention mechanism and the question text representation that integrates customer preferences to obtain a target representation of the question text to be trained; and training the target representation to obtain an intent recognition model.

[0006] According to an embodiment of the present disclosure, the question text to be trained is preprocessed, including: vectorizing the question text content based on a pre-trained processing model to obtain a word vector representation of the question text.

[0007] According to an embodiment of the present disclosure, customer preference data is preprocessed, including: obtaining authorization to collect customer preference data; obtaining the customer preference data after obtaining authorization, and merging, cleaning and encoding the data to obtain a customer preference information representation vector.

[0008] According to an embodiment of the present disclosure, based on the preprocessed question text content, a text representation based on the self-attention mechanism is obtained, including: inputting the word vector representation of the question text into the long-short term recurrent neural network to obtain the question text representation of the integrated context content; based on the self-attention mechanism, weighting the question text representation of the integrated context content is assigned to obtain the text representation based on the self-attention mechanism.

[0009] According to an embodiment of the present disclosure, based on the interactive attention mechanism, the preprocessed customer preference data is mapped to the question text representation to obtain the question text representation that integrates the customer preferences, including: calculating the importance weights of the customer preference information representation vector to the words in the question text through the interactive attention mechanism; combining the question text representation of the fused context content weighted by attention with the customer preference information representation vector to obtain the question text representation that integrates the customer preferences.

[0010] According to an embodiment of the present disclosure, a target representation is trained to obtain an intent recognition model, including: selecting a deep learning model architecture for intent recognition; using a cross-entropy loss function as an objective function to calculate the cross-entropy loss; inputting the target representation into the architecture of the deep learning model for forward propagation, and training to obtain the intent recognition model.

[0011] According to an embodiment of the present disclosure, training the target representation to obtain the intent recognition model also includes: during the training process, calculating the gradient through the back propagation algorithm, and using the optimization algorithm to update the model parameters.

[0012] The second aspect of the present disclosure provides an intent recognition device, including: a message text acquisition module, used to obtain the customer's message text during the customer service process; a customer intention judgment module, used to judge the customer intention corresponding to the message text based on an intent recognition model, wherein the intent recognition model is obtained through pre-training.

[0013] According to an embodiment of the present disclosure, a customer intention judgment module includes: a preprocessing unit, which is used to preprocess the question text to be trained and the customer preference data respectively; a text representation acquisition unit of the self-attention mechanism, which is used to obtain a text representation based on the self-attention mechanism based on the preprocessed question text content; a question text representation acquisition unit that integrates customer preferences, which is used to map the preprocessed customer preference data to the question text representation based on the interactive attention mechanism to obtain a question text representation that integrates customer preferences; a target representation acquisition unit for the question text to be trained, which is used to splice the text representation based on the self-attention mechanism and the question text representation that integrates customer preferences to obtain a target representation of the question text to be trained; and an intention recognition model acquisition unit, which is used to train the target representation to obtain an intention recognition model.

[0014] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0015] The fourth aspect of the present disclosure further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the above computer program or instructions are executed by a processor.

[0016] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above contents and other purposes, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0018] Figure 1 The application scenario diagram of the intention recognition method according to the embodiment of the present disclosure is schematically shown;

[0019] Figure 2 A flowchart of an intention recognition method according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 3 A flowchart of obtaining an intent recognition model through pre-training according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 4 A flowchart of obtaining text representation based on a self-attention mechanism according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 5A flowchart of obtaining a question text representation integrating customer preferences according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 6 Schematically illustrates an intention recognition device according to an embodiment of the present disclosure; and

[0024] Figure 7 A block diagram of an electronic device suitable for implementing the intention recognition method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0026] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0027] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0028] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0029] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs, and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0031] An embodiment of the present disclosure provides an intent recognition method, which includes: obtaining a message text of a customer during a customer service process; judging the customer intent corresponding to the message text based on an intent recognition model, wherein the intent recognition model is obtained through pre-training, including: pre-processing a question text to be trained and customer preference data respectively; obtaining a text representation based on a self-attention mechanism based on the preprocessed question text content; mapping the preprocessed customer preference data to the question text representation based on an interactive attention mechanism to obtain a question text representation that integrates customer preferences; concatenating the text representation based on the self-attention mechanism and the question text representation that integrates customer preferences to obtain a target representation of the question text to be trained; and training the target representation to obtain an intent recognition model.

[0032] Through the embodiments of the present disclosure, the method can effectively capture important semantic information in the text and the individual preferences of customers by combining the self-attention mechanism and the interactive attention mechanism, thereby significantly improving the accuracy of the intent recognition model and ensuring a more accurate understanding of customer needs; by combining the long short-term memory network with the self-attention mechanism, the model can conduct in-depth analysis of the context of the question text and optimize the information extraction process, so that it has stronger context understanding capabilities when processing complex customer messages; by introducing customer preference data and integrating it into the intent recognition process, personalized responses and services can be provided.

[0033] Figure 1The application scenario diagram of the intention recognition method according to an embodiment of the present disclosure is schematically shown.

[0034] like Figure 1 As shown, the application scenario 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0035] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0037] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0038] It should be noted that the intent recognition method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the intent recognition device provided in the embodiment of the present disclosure can generally be set in the server 105. The intent recognition method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the intent recognition device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0039] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0040] The following will be based on Figure 1 The scene described by Figure 2~Figure 5 The intent recognition method of the disclosed embodiment is described in detail.

[0041] Figure 2 The flowchart of the intention recognition method according to the embodiment of the present disclosure is schematically shown.

[0042] like Figure 2 As shown, in an embodiment of the present disclosure, the intention recognition method specifically includes operations S210 to S220.

[0043] In operation S210, a message text of a customer during a customer service process is obtained.

[0044] In the embodiments of the present disclosure, the customer's message text is obtained during the customer service process. For example, it can be applied to a dialogue scenario in the financial field. The dialogue scenario can be specifically: the customer initiates a consultation dialogue with the customer service or intelligent customer service, and the customer service or intelligent customer service obtains the customer's message text at this time.

[0045] In operation S220, the customer intent corresponding to the message text is determined based on the intent recognition model, wherein the intent recognition model is obtained through pre-training.

[0046] After obtaining the customer's message text, the backend of the intelligent customer service determines the customer's intention corresponding to the message text consulted by the customer based on the pre-trained intent recognition model, and the customer service gives the customer a corresponding reply based on the customer's consultation dialogue. In order to better solve user demands and improve customer satisfaction, intelligent customer service needs to accurately understand customer intentions, thereby improving the accuracy of answers. For example, during the consultation process, there may be customers who ask the same question but may have different query intentions, which also shows that customer intentions may be related to personal information such as the customer's assets, product holdings, and transaction behaviors.

[0047] The following is a detailed description of the specific process of obtaining the intent recognition model through pre-training.

[0048] Figure 3 The flowchart of obtaining the intent recognition model through pre-training according to an embodiment of the present disclosure is schematically shown.

[0049] like Figure 3 As shown, in an embodiment of the present disclosure, an intent recognition model is obtained through pre-training, which specifically includes operations S310 to S350.

[0050] In operation S310, the training question text and the customer preference data are preprocessed respectively.

[0051] In the embodiment of the present disclosure, firstly, the text content data of the questions asked by the intelligent customer service during the customer service process is collected, and the question intention category is marked as the question text to be trained. Then, the question text to be trained is preprocessed.

[0052] In an embodiment of the present disclosure, the question text to be trained is preprocessed, including: vectorizing the question text content based on a pre-trained processing model to obtain a word vector representation of the question text.

[0053] Specifically, the text content of the customer's question is unstructured data, which can be represented as ,in Is the first Therefore, we can use a pre-trained processing model, such as the word2vec pre-trained model, to vectorize the words in the question and obtain the representation of the question text. ,in Is the first The word vector of each word. Word2vec training usually uses a large-scale corpus to ensure that the model can learn rich semantic information. After training, the model can represent each word as a vector of fixed dimension, which can be used for various downstream tasks.

[0054] Furthermore, customer preference data is collected, where the customer preference data includes customer basic information, customer asset information, customer product holding information and customer historical risk data.

[0055] Specifically, basic customer information includes, for example, region, age, gender, education level, occupation, whether a customer is on the blacklist, the economic nature of the work unit, and whether a customer is a mobile banking customer. Customer asset information includes, for example, total assets, total liabilities, total deposits, total loans, average daily assets this month, average daily assets this year, average daily assets in the past three months, average daily assets in the past six months, average daily assets in the past twelve months, financial management customer mark, and default customer mark. Customer product information includes customer loan balance information, such as personal asset amount segmentation mark, personal consumption loan amount, personal housing loan amount, personal business loan amount, customer product balance and product mark, amount, etc. Customer historical risk data includes, for example, credit risk level, potential credit risk customer mark, customer mark with multiple overdue payments, current non-performing loan customer mark, operational risk mark, customer mark with attention, suspected abnormal customer mark, incomplete information mark, expired certificate mark, information pending verification mark, whether a cross-default customer, whether an anti-money laundering customer, whether a gray-listed customer, etc.

[0056] In an embodiment of the present disclosure, the customer preference data is preprocessed, including: obtaining authorization to collect the customer preference data; obtaining the customer preference data after obtaining the authorization, and merging, cleaning and encoding the data to obtain a customer preference information representation vector.

[0057] Specifically, after obtaining authorization, customer preference data is obtained. Customer preference data belongs to structured data, and the structured data is merged, cleaned and encoded.

[0058] Exemplarily, the collected data from different sources are integrated. For example, based on the customer ID or other related fields, the data from different sources are merged into a unified data frame to ensure that there are no duplicate records in the merged data.

[0059] Exemplarily, the acquired data is cleaned again, for example, including deleting or filling missing values, then removing outliers, and finally unifying the data format, such as date format, string case, and other standardized format processing.

[0060] Exemplarily, the cleaned data is converted into a format suitable for a machine learning model to obtain a customer preference information representation vector. For example, one-hot encoding or label encoding may be used.

[0061] Through the embodiments of the present disclosure, preference data is obtained from customers, merged, cleaned and encoded, and finally a representation vector of customer preference information is obtained. Such data is conducive to the use of application scenarios such as personalized recommendation and customer segmentation.

[0062] In operation S320, a text representation based on a self-attention mechanism is obtained based on the preprocessed question text content.

[0063] Figure 4 A flowchart for obtaining text representation based on a self-attention mechanism according to an embodiment of the present disclosure is schematically shown.

[0064] like Figure 4 As shown, in an embodiment of the present disclosure, based on the preprocessed question text content, a text representation based on a self-attention mechanism is obtained, which specifically includes operations S410 to S420.

[0065] In operation S410, the word vector representation of the question text is input into the long short-term recurrent neural network to obtain the question text representation integrated with the context content.

[0066] In operation S420, based on the self-attention mechanism, weights are assigned to the question text representation integrated with the context content to obtain a text representation based on the self-attention mechanism.

[0067] Specifically, in order to obtain the contextual dependency of words in the question text, the embodiment of the present disclosure uses a long-term and short-term recurrent neural network to obtain the question text representation that integrates the context content. As the input of the long-term and short-term recurrent neural network, after learning the long-term and short-term recurrent neural network, the text representation that integrates the context information can be obtained The long short-term recurrent neural network (LSTM) is used to model the question text, which can effectively capture the long-distance dependencies in the text, thereby generating a question text representation that integrates the contextual content. This representation covers the semantic depth and contextual information of the question, ensuring that the model fully understands the question.

[0068] In traditional models, maximum pooling or average pooling is usually used to fuse information from multiple hidden states. Considering the different importance of each word in the text, keywords should be given a higher weight. In the embodiment of the present disclosure, a self-attention mechanism is used to calculate the weight of each word, and the information of the states is fused according to different weights to obtain a text representation based on the self-attention mechanism. ,in is the self-attention weight vector of the text.

[0069] Through the embodiments of the present disclosure, the weight assignment process based on the self-attention mechanism enables the model to dynamically adjust the importance of each word in the question text representation. This flexible weight assignment can highlight the key parts and suppress redundant information, thereby improving the performance of the model in subsequent tasks (such as question answering, etc.).

[0070] In operation S330, based on the interactive attention mechanism, the pre-processed customer preference data is mapped to the question text representation to obtain the question text representation integrating the customer preference.

[0071] Figure 5 The flowchart of obtaining a question text representation integrating customer preferences according to an embodiment of the present disclosure is schematically shown.

[0072] like Figure 5 As shown, in an embodiment of the present disclosure, based on the interactive attention mechanism, the preprocessed customer preference data is mapped to the question text representation to obtain the question text representation integrating the customer preference, which specifically includes operations S510 to S520.

[0073] In operation S510, the importance weights of the customer preference information representation vector to the words in the question text are calculated through an interactive attention mechanism.

[0074] In operation S520, the question text representation of the fused context content after attention weighting is combined with the customer preference information representation vector to obtain the question text representation fused with the customer preference.

[0075] For example, considering that customers who ask the same question may have different query intentions, the customer's personal information such as asset status, trading behavior and investment preferences are mapped to the representation of the customer's question content based on the interactive attention mechanism to obtain the question content representation that integrates the customer's preferences. , Indicates the importance of the i-th word in characterizing customer preferences.

[0076] Through the embodiments of the present disclosure, by paying attention to customer preference information, the model can better capture keywords in the question text, thereby improving the accurate recognition of customer intentions; combining the attention-weighted question text representation with customer preference information can effectively integrate personalized information into text parsing, so that the generated representation is more in line with the specific needs of customers; the interactive attention mechanism makes the model have good adaptability, and by dynamically adjusting the importance weight of each word, the model can grasp the core content of the question more accurately, thereby providing more targeted responses when answering and improving customer experience.

[0077] In operation S340, the text representation based on the self-attention mechanism is concatenated with the question text representation integrating the customer preference to obtain a target representation of the question text to be trained.

[0078] In the embodiment of the present disclosure, the text representation obtained based on the self-attention mechanism and the text representation obtained by integrating customer preferences are concatenated as the target representation of the question text. For intent recognition, i.e. .

[0079] In operation S350, the target representation is trained to obtain an intent recognition model.

[0080] In an embodiment of the present disclosure, a target representation is trained to obtain an intent recognition model, including: selecting a deep learning model architecture for intent recognition; using a cross-entropy loss function as an objective function to calculate the cross-entropy loss; inputting the target representation into the architecture of the deep learning model for forward propagation, and training to obtain the intent recognition model.

[0081] For example, choose a deep learning model architecture suitable for intent recognition, such as a multi-layer perceptron (MLP), a convolutional neural network (CNN), or a long short-term memory network (LSTM), or even a transformer-based structure; the model input is the target representation after the self-attention mechanism and the fusion of customer preferences; define a loss function, such as using a cross-entropy loss function to evaluate the difference between the model's prediction and the actual intent.

[0082] During the training process, the target representation (i.e., the question text representation after the interactive attention mechanism) is input into the selected deep learning model. After several layers of calculation, the probability distribution of the intent classification is output, and the cross entropy loss of the model output is calculated. During the training process, the gradient is calculated through the back propagation algorithm, and the model parameters are updated using the optimization algorithm.

[0083] Through the embodiments of the present disclosure, by using a deep learning model architecture and a cross-entropy loss function, the accuracy of intent recognition can be significantly improved, so that the model can more accurately understand the questions raised by users; the model can continuously learn new intent categories during the training process, has good adaptability, and can integrate new data for retraining at any time to quickly respond to changes in user needs.

[0084] Based on the above intention recognition method, a more accurate representation of the question content is obtained through the self-attention mechanism. Based on the interactive attention mechanism, the customer's personal information such as asset status, trading behavior and investment preferences are mapped to the representation of the customer's question text content, and then spliced ​​to obtain the question content representation that integrates the customer's preferences. This can effectively capture important semantic information in the text and the customer's individual preferences, thereby significantly improving the accuracy of the intent recognition model, ensuring a more accurate understanding of customer needs, and realizing personalized intent recognition.

[0085] Based on the above intention recognition method, the present disclosure also provides an intention recognition device. Figure 6 The device is described in detail.

[0086] Figure 6 The structural block diagram of the intention recognition device according to an embodiment of the present disclosure is schematically shown.

[0087] like Figure 6 As shown, the intention recognition device 600 of this embodiment includes a message text acquisition module 610 and a customer intention judgment module 620.

[0088] The message text acquisition module 610 is used to acquire the message text of the customer during the customer service process. In one embodiment, the message text acquisition module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0089] The customer intention determination module 620 is used to determine the customer intention corresponding to the message text based on the intention recognition model, wherein the intention recognition model is obtained through pre-training. In one embodiment, the customer intention determination module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0090] According to an embodiment of the present disclosure, the customer intention judgment module 620 includes a preprocessing unit, a text representation acquisition unit of a self-attention mechanism, a question text representation acquisition unit integrating customer preferences, a target representation acquisition unit of the question text to be trained, and an intention recognition model acquisition unit.

[0091] The preprocessing unit is used to preprocess the training question text and the customer preference data respectively. In one embodiment, the preprocessing unit can be used to perform the operation S310 described above, which will not be repeated here.

[0092] The text representation acquisition unit of the self-attention mechanism is used to obtain the text representation based on the self-attention mechanism based on the preprocessed question text content. In one embodiment, the text representation acquisition unit of the self-attention mechanism can be used to perform the operation S320 described above, which will not be repeated here.

[0093] The question text representation acquisition unit integrating customer preferences is used to map the preprocessed customer preference data to the question text representation based on the interactive attention mechanism to obtain the question text representation integrating customer preferences. In one embodiment, the question text representation acquisition unit integrating customer preferences can be used to perform the operation S330 described above, which will not be repeated here.

[0094] The target representation acquisition unit of the question text to be trained is used to splice the text representation based on the self-attention mechanism with the question text representation integrated with the customer preference to obtain the target representation of the question text to be trained. In one embodiment, the target representation acquisition unit of the question text to be trained can be used to perform the operation S340 described above, which will not be repeated here.

[0095] The intention recognition model acquisition unit is used to train the target representation to obtain the intention recognition model. In one embodiment, the intention recognition model acquisition unit can be used to perform the operation S350 described above, which will not be repeated here.

[0096] According to an embodiment of the present disclosure, any multiple modules in the message text acquisition module 610 and the customer intention judgment module 620 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the message text acquisition module 610 and the customer intention judgment module 620 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the message text acquisition module 610 and the customer intention judgment module 620 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0097] Figure 7 A block diagram of an electronic device suitable for implementing the intention recognition method according to an embodiment of the present disclosure is schematically shown.

[0098] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0099] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.

[0100] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0101] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0102] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0103] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the intention recognition method provided by the embodiment of the present disclosure.

[0104] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 701. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0105] In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and downloaded and installed through the communication part 709, and / or installed from a removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0106] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0107] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0108] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0110] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for intention recognition, characterized in that: The method comprises: Get the customer's message text during the customer service process; Based on the intention recognition model, the customer intention corresponding to the message text is determined, wherein the intention recognition model is obtained through pre-training, including: Pre-process the training question text and customer preference data separately; Based on the preprocessed question text content, a text representation based on the self-attention mechanism is obtained; Based on the interactive attention mechanism, the pre-processed customer preference data is mapped to the question text representation to obtain the question text representation that integrates customer preferences. The text representation based on the self-attention mechanism is concatenated with the question text representation integrating customer preferences to obtain the target representation of the question text to be trained; The target representation is trained to obtain the intent recognition model.

2. The method for intention recognition according to claim 1, characterized in that: The preprocessing of the training question text includes: The question text content is vectorized and represented based on a pre-trained processing model to obtain a word vector representation of the question text.

3. The method for identifying intention according to claim 2, characterized in that: The preprocessing of the customer preference data includes: Obtain authorization to collect customer preference data; After obtaining authorization, the customer preference data is obtained, and merged, cleaned and encoded to obtain a customer preference information representation vector.

4. The intention recognition method according to claim 1 or 2, characterized in that: The text representation based on the self-attention mechanism is obtained based on the preprocessed question text content, including: Inputting the word vector representation of the question text into the long-short term recurrent neural network to obtain the question text representation integrating the context content; Based on the self-attention mechanism, weights are assigned to the question text representation that integrates the context content to obtain a text representation based on the self-attention mechanism.

5. The intention recognition method according to claim 1 or 3, characterized in that: Based on the interactive attention mechanism, the pre-processed customer preference data is mapped to the question text representation to obtain the question text representation integrating the customer preference, including: Calculating the importance weight of the customer preference information representation vector to the words in the question text through an interactive attention mechanism; The question text representation of the fused context content after attention weighting is combined with the customer preference information representation vector to obtain the question text representation of the fused customer preference.

6. The method for intention recognition according to claim 5, characterized in that: The training of the target representation to obtain the intention recognition model includes: Select the deep learning model architecture for intent recognition; Calculate the cross entropy loss using the cross entropy loss function as the objective function; The target representation is input into the architecture of the deep learning model for forward propagation, and the intent recognition model is trained to obtain the intent recognition model.

7. The intention recognition method according to claim 1 or 6, characterized in that: The training of the target representation to obtain the intention recognition model further includes: During the training process, the gradient is calculated through the back-propagation algorithm, and the model parameters are updated using the optimization algorithm.

8. An intention recognition device, characterized in that: The device comprises: The message text acquisition module is used to obtain the customer's message text during the customer service process; The customer intention judgment module is used to judge the customer intention corresponding to the message text based on the intention recognition model, wherein the intention recognition model is obtained through pre-training.

9. The intention recognition device according to claim 8, characterized in that: The customer intention judgment module includes: A preprocessing unit, used to preprocess the training question text and customer preference data respectively; A text representation acquisition unit of a self-attention mechanism, used to obtain a text representation based on a self-attention mechanism based on the preprocessed question text content; A question text representation acquisition unit integrating customer preferences is used to map the pre-processed customer preference data to the question text representation based on an interactive attention mechanism to obtain the question text representation integrating customer preferences; A target representation acquisition unit for the question text to be trained, used for concatenating the text representation based on the self-attention mechanism with the question text representation integrating customer preferences to obtain the target representation of the question text to be trained; The intention recognition model acquisition unit is used to train the target representation to obtain the intention recognition model.

10. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

11. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images