Personalized dynamic intent recognition method and apparatus, and related device

Abstract

Provided are a personalized dynamic intent recognition method and apparatus, and a related device, related to the technical field of computers. The method comprises: acquiring historical dialog data of at least one user with customer service (S202); on the basis of the historical dialog data, constructing empirical tuple data for training a deep reinforcement learning model, the deep reinforcement learning model being used for predicting a value of an agent from one state to one action, the empirical tuple data comprising a current state, an action, a reward value, and a next state of the agent (S204); acquiring personalized weight information of each user, and, in the process of the agent training the deep reinforcement learning model by using the empirical tuple data, using the personalized weight information of each user to update a prediction value of the deep reinforcement learning model, until a personalized intent recognition model satisfying a preset condition is obtained. The present method can dynamically and accurately identify a personalized intent of a user, so as to more conveniently provide the user with a higher-quality personalized service.

Description

Method and device for dynamically identifying personalized intention, and related equipment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] The present disclosure claims priority to Chinese Patent Application No. 202410773527.1, filed on June 14, 2024, entitled "Method and device for dynamically identifying personalized intention, and related equipment", the entire contents of which are incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present disclosure relates to the field of computer technology, and more particularly, to a method and device for dynamically identifying personalized intention, and related equipment. BACKGROUND

[0004] With the rapid progress of technology and the continuous development of artificial intelligence, enterprises are also constantly pursuing higher efficiency and better service in order to provide users with a better user experience. In the field of artificial intelligence, natural language processing technology has made significant progress, and machines can better understand human language, thereby achieving more intelligent and efficient service. In practical applications, intention recognition is an important task in natural language processing, aiming to determine the user's intention or purpose in a conversation or text. Intention recognition technology is commonly used to build chat robots, intelligent customer service systems, virtual assistants, etc., by analyzing the natural language input by users to understand their intentions and needs, so as to provide more accurate and targeted answers or services. However, traditional methods have limitations in understanding user intentions and cannot meet the personalized needs of users. Therefore, how to propose a new intention recognition method to identify user intentions individually is a technical problem that needs to be solved.

[0005] It should be noted that the information disclosed in the above BACKGROUND section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0006] The present disclosure provides a method and device for dynamically identifying personalized intention, and related equipment, which at least partially overcomes the problem of not being able to identify user intentions individually in the related art.

[0007] Other characteristics and advantages of the present disclosure will become apparent from the following detailed description, or will be learned by practice of the present disclosure.

[0008] According to one aspect of the present disclosure, a method for dynamically identifying personalized intention is provided, comprising: obtaining historical conversation data of at least one user with a customer service;

[0009] According to historical dialogue data, experience tuple data for training a deep reinforcement learning model is constructed, wherein the deep reinforcement learning model is used to predict the value of an agent from a state to an action, and the experience tuple data includes the current state of the agent, the action, the reward value and the next state.

[0010] The personalized weight information of each user is obtained, and the personalized weight information of each user is used to update the prediction value of the deep reinforcement learning model in the process of training the deep reinforcement learning model by the agent using the experience tuple data, until a personalized intent recognition model that meets a preset condition is obtained, wherein the personalized intent recognition model is used to identify the personalized intent of each user.

[0011] In some embodiments, the customer service is an agent, and according to historical dialogue data, experience tuple data for training a deep reinforcement learning model is constructed, including:

[0012] The historical dialogue data is taken as the current state of the agent;

[0013] The action of the agent is determined according to the current state;

[0014] The reward value of the agent is calculated according to the feedback of the user based on the action of the agent and a pre-designed reward function, and the next state of the agent is obtained.

[0015] In some embodiments, the reward value of the agent is calculated according to the feedback of the user based on the action of the agent and a pre-designed reward function, including:

[0016] The feedback information of the user is obtained, and the feedback information includes completion degree information and satisfaction degree information;

[0017] The reward value of the agent is calculated according to the completion degree information and the satisfaction degree information.

[0018] In some embodiments, the personalized weight information of each user is obtained, including:

[0019] User feature information is obtained;

[0020] The personalized weight of the user is calculated according to the user feature information.

[0021] In some embodiments, after obtaining the historical dialogue data between at least one user and the customer service, the method further includes:

[0022] Data preprocessing is performed on the historical dialogue data;

[0023] After data preprocessing, the historical dialogue data is extracted to obtain keyword information, and the keywords are used to construct experience tuple data for training a deep reinforcement learning model.

[0024] In some embodiments, the customer service is a dialogue robot, which is obtained based on a pre-acquired operation service field knowledge training semantic dialogue large model.

[0025] According to another aspect of the present disclosure, a personalized intent dynamic identification method is also provided, including: acquiring historical dialogue data of at least one user and a customer service;

[0026] constructing experience tuple data for training a deep reinforcement learning model according to the historical dialogue data, wherein the deep reinforcement learning model is used to predict a value of an agent from a state to an action, and the experience tuple data includes a current state, an action, a reward value and a next state of the agent;

[0027] training a personalized intent recognition model, including: acquiring personalized weight information of each user, and updating a prediction value of the deep reinforcement learning model by using the personalized weight information of each user in a process of training the deep reinforcement learning model by the agent using the experience tuple data, until a personalized intent recognition model meeting a preset condition is obtained, wherein the personalized intent recognition model is used to identify a personalized intent of each user.

[0028] According to another aspect of the present disclosure, an electronic device is also provided, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the above-mentioned any one of the personalized intent dynamic identification methods by executing the executable instructions.

[0029] According to another aspect of the present disclosure, a computer readable storage medium is also provided, having a computer program stored thereon, the computer program being executed by a processor to implement the above-mentioned any one of the personalized intent dynamic identification methods.

[0030] According to another aspect of the present disclosure, a computer program product is also provided, including a computer program, the computer program being executed by a processor to implement the above-mentioned any one of the personalized intent dynamic identification methods.

[0031] The personalized intent dynamic recognition method, device and related equipment provided in the embodiments of the present disclosure, by obtaining the historical dialogue data of the user and the customer service, using the deep reinforcement learning technology, constructing the experience tuple data, and obtaining the personalized weight information of each user. In the process of training the personalized intent dynamic recognition model, the personalized weight information is used to update the prediction value of the deep reinforcement learning model until the personalized intent recognition model that meets the preset condition is obtained. The personalized intent recognition model of the embodiments of the present disclosure can accurately recognize the personalized intent of each user, provide more personalized and efficient service for the user, and meet the personalized needs of the user.

[0032] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0033] The drawings incorporated into the specification and forming part of the specification, show embodiments consistent with the present disclosure, and together with the specification, serve to explain the principles of the present disclosure. It is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained from these drawings without creative labor for those skilled in the art.

[0034] FIG. 1 shows an exemplary application system architecture schematic diagram of applying the personalized intent dynamic recognition method in the embodiments of the present disclosure;

[0035] FIG. 2 shows a flowchart of a personalized intent dynamic recognition method in the embodiments of the present disclosure;

[0036] FIG. 3 shows a flowchart of another personalized intent dynamic recognition method in the embodiments of the present disclosure;

[0037] FIG. 4 shows a flowchart of another personalized intent dynamic recognition method in the embodiments of the present disclosure;

[0038] FIG. 5 shows a flowchart of a specific personalized intent recognition method in the embodiments of the present disclosure;

[0039] FIG. 6 shows a personalized intent dynamic recognition system applying the personalized intent recognition method in the embodiments of the present disclosure;

[0040] FIG. 7 shows a schematic diagram of a personalized intent dynamic recognition device in the embodiments of the present disclosure;

[0041] FIG. 8 shows a structural block diagram of an electronic device in the embodiments of the present disclosure. DETAILED DESCRIPTION

[0042] Example implementations are now described with reference to the drawings. Example implementations can, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these examples are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art. The described features, structures, or characteristics can be combined in one or more implementations.

[0043] Furthermore, the accompanying drawings are only intended to show some schematic views of the present disclosure, and are not necessarily drawn to scale. The same reference numbers in the drawings represent the same or similar parts, and thus repetitive description thereof will be omitted. Some block diagrams shown in the drawings are functional entities, and do not necessarily have to correspond to physically or logically independent entities. These functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0044] The specific implementation of the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0045] FIG. 1 shows an exemplary application system architecture schematic diagram to which the personalized intent dynamic identification method in the embodiments of the present disclosure can be applied. As shown in FIG. 1, the system architecture can include a terminal device 101, a network 102, and a server 103.

[0046] The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103, and can be a wired network or a wireless network.

[0047] Optionally, the wireless or wired networks described above use standard communications technologies and / or protocols. The network can be the Internet, but can also include any combination of LANs, MANs, WANs, mobile, wired, or wireless networks, networks of networks, and / or intranets. In some embodiments, technologies used include, without limitation, Hypertext Markup Language (HTML), Extensible Markup Language (XML), and / or the like, to create and transmit data over the network. In addition, conventional encryption technologies such as the Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), Internet Protocol Security (IPSec), and / or the like, can be used to encrypt all or some of the links between nodes in the network. In other embodiments, custom and / or proprietary data communications technologies can be used in place of, or in addition to, the above technologies.

[0048] The terminal device 101 can be various electronic devices, including but not limited to a smartphone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a wearable device, an augmented reality device, a virtual reality device, and / or the like.

[0049] Optionally, the application clients installed in different terminal devices 101 are the same, or are clients of the same type of application based on different operating systems. Depending on the terminal platform, the specific form of the application client can also be different, for example, the application client can be a mobile phone client, a PC client, and / or the like.

[0050] The server 103 can be a server that provides various services, for example, a background management server that provides support for the operation of the device by the user using the terminal device 101. The background management server can analyze and / or the like process the received request and / or the like data, and feed back the processing result to the terminal device.

[0051] Optionally, the server can be a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms.

[0052] Those skilled in the art can know that the number of terminal devices, networks and servers in FIG. 1 is only illustrative, and any number of terminal devices, networks and servers can be provided according to actual needs. The embodiments of the present disclosure do not limit this.

[0053] Under the above system architecture, the embodiments of the present disclosure provide a personalized intention dynamic identification method, which can be executed by any electronic device with computing processing capability.

[0054] In some embodiments, the personalized intention dynamic identification method provided in the embodiments of the present disclosure can be executed by the terminal device of the above system architecture; in other embodiments, the personalized intention dynamic identification method provided in the embodiments of the present disclosure can be executed by the server in the above system architecture; in other embodiments, the personalized intention dynamic identification method provided in the embodiments of the present disclosure can be realized by the terminal device and the server in the above system architecture through interaction.

[0055] FIG. 2 shows a flowchart of a personalized intention dynamic identification method in the embodiments of the present disclosure. As shown in FIG. 2, the personalized intention dynamic identification method provided in the embodiments of the present disclosure includes the following steps:

[0056] S202, obtaining historical dialogue data of at least one user and a customer service.

[0057] It should be noted that the customer service of the embodiments of the present disclosure can be any customer service that can communicate with the user, including but not limited to artificial customer service, robot customer service, etc. In the embodiments of the present disclosure, the customer service can effectively communicate with the user and collect historical dialogue data for subsequent training and optimization of the deep reinforcement learning model.

[0058] The historical dialogue data of at least one user and a customer service can be obtained, specifically, the historical dialogue data can be extracted from a customer service system. The historical dialogue data can be any dialogue data stored in text, voice or video format. The historical dialogue data records the interactive content of the user when seeking help, asking questions or expressing needs with the customer service, including product inquiries, technical support, reservation services, complaint handling, and some substantive content about the user himself.

[0059] By obtaining these historical dialogue data, a deep reinforcement learning model is trained. The deep reinforcement learning model will use these historical dialogue data to learn how to identify the real intention of the user from the new input utterance of the user and provide appropriate responses or suggestions according to the intention. With the increase of data volume, the prediction ability of the personalized intention recognition model will also be improved accordingly, so as to provide more personalized and accurate services.

[0060] In S204, experience tuple data for training the deep reinforcement learning model is constructed according to the historical dialogue data, wherein the deep reinforcement learning model is used to predict the value of an agent from a state to an action, and the experience tuple data includes the current state of the agent, the action, the reward value and the next state.

[0061] It should be noted that in deep reinforcement learning, experience tuple data is a data unit for training an agent. Each experience tuple usually contains four elements: the current state of the agent, the action, the reward value and the next state. The experience tuple data defines the interaction process of the agent in the environment. In the embodiments of the present disclosure, the process of the agent as a robot customer communicating with the user is the interaction process of the agent in the environment.

[0062] In addition, in deep reinforcement learning, the value refers to the long-term reward that the agent is expected to obtain by taking a certain action in a specific state. The goal of the deep reinforcement learning model is to learn a value function that can predict a value for each state-action pair, i.e., the cumulative reward that the agent is expected to obtain after taking the action. Taking the deep Q-network (DQN) model as an example, the value refers to the Q value.

[0063] In the embodiments of the present disclosure, the agent as a robot customer, the process of communicating with the user is its interaction process in the environment. Each dialogue round of the agent can be regarded as an experience tuple, which specifically includes the following steps:

[0064] 1. The historical dialogue data is taken as the current state of the agent.

[0065] 2. The action of the agent is determined according to the current state.

[0066] 3. The reward value of the agent is calculated according to the feedback of the user based on the action of the agent and the pre-designed reward function, and the next state of the agent is obtained.

[0067] It should be noted that the historical dialogue data is used as the current state of the agent in the personalized intent dynamic identification method. The agent, as a kind of robot customer service, its current state can include the user's input question or request in the historical dialogue data and the context of the dialogue. Among them, the user's input question and request are the latest input content of the user, which can be a query, complaint, suggestion or help needed for a certain product or service. The context of the dialogue includes the historical information of the current dialogue, such as the user's previous questions and the agent's responses. The context is very important for understanding the user's intent and maintaining the consistency of the dialogue.

[0068] After determining the current state of the agent, the agent selects an action under the given current state according to its internal strategy. In the context of customer service, the agent's actions can include:

[0069] Responding to user's question: providing direct and relevant answers.

[0070] Requesting more information: when more detailed information is needed to understand the user's intent, the agent may ask further questions.

[0071] Guiding the dialogue: the agent may guide the dialogue to a more specific topic or sub-question.

[0072] Providing suggestions or solutions: for some user requests, the agent may provide a series of suggestions or solutions for the user to choose from.

[0073] The goal of the agent is to maximize the expected long-term reward by taking appropriate actions. During the training process, the agent will try different actions and learn which actions can bring the maximum cumulative reward in a specific state. In this way, the agent can gradually optimize its strategy to more effectively achieve its goals in future interactions.

[0074] The reward value of the agent is calculated according to the user's feedback based on the agent's action and the pre-designed reward function. After the agent takes an action, the user's feedback is an immediate evaluation of this action, which can be expressed as the user's satisfaction with the agent's action. This feedback is combined with the pre-designed reward function to calculate the agent's reward value. The design of the reward function should be consistent with the goal the agent wants to optimize, for example, improving user satisfaction.

[0075] The agent's action will affect the environment, resulting in a new state. In the customer service system, the next state is the user's next round of response or question to the agent's response.

[0076] In this way, the agent continuously learns from interactions with the user, optimizing its policy so that it can take more appropriate actions when encountering similar states in the future. The method of the embodiments of the present disclosure can adapt to the personalized needs of different users, providing more customized services. As the agent receives more training data, the performance of the agent gradually improves, enabling it to more accurately identify and respond to the user's intent.

[0077] According to the current state, action, reward value and next state obtained from the historical dialogue data above, an experience tuple data can be constructed, which is used to train a deep reinforcement learning model. The agent gradually improves its policy through continuous trial and learning, improving its ability to evaluate future states and choose actions. The agent can better adapt to complex dialogue environments and provide more personalized and effective services. As the training data increases, the performance of the agent will further improve, enabling it to more accurately identify and respond to the user's intent.

[0078] In S206, personalized weight information of each user is obtained, and in the process of training the deep reinforcement learning model by the agent using the experience tuple data, the personalized weight information of each user is used to update the predicted value of the deep reinforcement learning model until a personalized intent recognition model that meets the preset condition is obtained, wherein the personalized intent recognition model is used to identify the personalized intent of each user.

[0079] Obtaining personalized weight information can help the personalized intent recognition model better understand the user's needs and preferences. The personalized weight information can include the user's purchase history, preference settings, behavior patterns, interaction history, etc. Through the personalized weight information, a personalized weight can be assigned to each user to reflect their unique behavior and needs.

[0080] In the training process of the deep reinforcement learning model, the personalized intent recognition model learns to predict the value of actions based on experience tuple data (current state, action, reward value and next state). By introducing personalized weight information, the model can adjust its predictions to better adapt to the personalized needs of each user. After multiple rounds of training, the personalized intent recognition model can identify and understand the personalized intent of different users. The personalized intent recognition model of the embodiments of the present disclosure can more accurately predict the user's needs and provide more personalized services by learning the personalized weight information of each user.

[0081] The embodiments of the present disclosure improve the accuracy of intent recognition and achieve personalized services through deep reinforcement learning technology combined with historical dialogue data and personalized weight information of users. As the model receives more training data, its performance gradually improves, enabling it to more accurately identify and respond to the user's intent, achieving the ability to dynamically identify the user's personalized intent.

[0082] It should be noted that the acquisition, storage, use, processing, etc. of data in the technical solutions of the present disclosure comply with relevant provisions of national laws and regulations. The various types of data, such as personal identity data, operation data, and behavior data of individuals, customers, and crowds, obtained in the embodiments of the present disclosure have been authorized.

[0083] In some embodiments of the present disclosure, as shown in FIG. 3, the reward value of the agent is calculated according to the feedback of the user based on the action of the agent and the pre-designed reward function, and further includes the following steps:

[0084] S302, obtaining feedback information of the user, the feedback information including completion degree information and satisfaction degree information;

[0085] S304, calculating the reward value of the agent according to the completion degree information and the satisfaction degree information.

[0086] It should be noted that the reward value of the agent is calculated according to the feedback of the user based on the action of the agent and the pre-designed reward function. This reward value reflects the effect of the action of the agent and is used to guide the behavior selection of the agent in subsequent dialogues. Specifically, feedback information is obtained from the user. The feedback information includes completion degree information and satisfaction degree information.

[0087] The completion degree information refers to information about whether the agent has completed an action completely, and the satisfaction degree information refers to the satisfaction of the user with the action of the agent in the next input information.

[0088] According to the obtained completion degree information and satisfaction degree information, a pre-designed reward function is used to calculate the reward value of the agent. This reward function converts the information of completion degree and satisfaction degree into a reward value, reflecting the effect of the action of the agent. The agent adjusts its strategy according to the calculated reward value to take more effective actions in future dialogues. The positive and negative and size of the reward value directly affect the learning process of the agent, prompting the agent to take actions that can bring higher rewards.

[0089] In the embodiments of the present disclosure, the agent can continuously optimize its behavior according to the feedback of the user, thereby providing more accurate and satisfactory answers. The agent can adjust its behavior actions in real time according to the feedback of the user, making it more adaptable to the personalized needs of the user. With the training, the behavior of the agent will be more in line with the expectations of the user, thereby improving the user experience and satisfaction.

[0090] In some embodiments of the present disclosure, as shown in FIG. 4, the personalized weight information of each user is obtained, and further includes the following steps:

[0091] S402, obtaining user feature information;

[0092] S404, calculating the individualized weight of the user according to the user feature information.

[0093] It should be noted that the individualized weight information of each user is obtained in order to better understand the unique needs and preferences of the user, so as to provide more personalized services for the user. Specifically, the user feature information related to the user needs to be collected, which can include but is not limited to the user's purchase history, preference settings, behavior patterns, interaction history, etc. Through these information, the system can better understand the user's behavior and needs. Based on the obtained user feature information, the system will use some algorithm or model to calculate the individualized weight of each user. The individualized weight reflects the unique needs and preferences of the user, and is used to guide the behavior selection of the agent in the dialogue.

[0094] The embodiments of the present disclosure can provide personalized services for each user. The agent will adjust its behavior according to the individualized weight information of each user during the dialogue process, in order to better adapt to the needs and preferences of the user.

[0095] In some embodiments of the present disclosure, after obtaining the historical dialogue data of at least one user and the customer service, the method further comprises the steps of:

[0096] data preprocessing is performed on the historical dialogue data;

[0097] extracting keyword information from the historical dialogue data after data preprocessing, the keywords are used to construct experience tuple data for training the deep reinforcement learning model.

[0098] Data preprocessing is to clean and format the collected historical dialogue data for subsequent analysis and processing. Data preprocessing may include steps such as removing noise, standardizing text, extracting key information, classifying dialogue, etc.

[0099] Key information is extracted from the historical dialogue data after data preprocessing. These keywords are the key part of the experience tuple data used to build the training deep reinforcement learning model. The selection and extraction of keywords help the personalized intent recognition model better understand the content of the dialogue and learn the user's intent and behavior patterns from it. Since the agent in the embodiments of the present disclosure is a robot customer service, the current state, action, next state and reward value are all based on the dialogue data between the user and the customer, so it is crucial to extract the keywords in the dialogue data to identify the keywords to complete the definition of the current state, action, next state and reward value of the agent. Keywords can help identify the current state of the agent, i.e., the current content of the dialogue, the user's intent and the context of the dialogue. The action of the agent is selected according to the current state, and the keywords of the agent's reply can define the action selected by the agent. The user's response or feedback will form new dialogue content, and the keywords in these contents will define the next state of the agent. The reward value is calculated according to the user's feedback on the agent's action and the pre-designed reward function, and the keywords in the user's feedback can help evaluate the effect of the agent's action, so as to determine the positive and negative and size of the reward value. In some embodiments of the present disclosure, keywords can be extracted based on the BERT (Bidirectional Encoder Representations from Transformers) model, which is a powerful pre-trained language representation model that can understand the complexity of language, including vocabulary, syntax and semantics, etc. The BERT model can be used for various natural language processing tasks such as text classification, named entity recognition, sentiment analysis, etc. In terms of keyword extraction, the BERT model can provide very valuable help.

[0100] The embodiments of the present disclosure can provide high-quality data input for deep reinforcement learning models, thereby improving the training effect and performance of the model. By extracting keywords from dialogue data, the agent can better understand the content of the dialogue and the user's intent, thereby making more accurate decisions.

[0101] In some embodiments of the present disclosure, the customer service is a dialogue robot, which is trained based on a pre-collected knowledge in the field of operation service to obtain a semantic dialogue large model. Before training the dialogue robot, extensive knowledge related to operation service needs to be collected. The operation service field knowledge can include product information, service process, frequently asked questions, laws and regulations, etc. The dialogue robot uses these pre-collected operation service field knowledge to train a semantic dialogue large model. When the customer service dialogues with the user, it is convenient to understand and generate dialogue content related to operation service, including answering user questions, providing help and suggestions, etc. In this way, the dialogue robot can generate appropriate answers according to the user's questions or requests by using the semantic dialogue large model it has trained. The embodiments of the present disclosure can provide more accurate and relevant answers, thereby improving user experience and service quality.

[0102] In some embodiments of the present disclosure, taking the field of telecommunications operation service as an example, the specific telecommunications operation service personalized intent dynamic identification system is applied, as shown in FIGS. 5-6, which shows a specific example of the personalized intent dynamic identification method in the embodiments of the present disclosure.

[0103] FIG. 5 shows a specific personalized intent identification method flowchart in the embodiments of the present disclosure, which is applied to the personalized intent dynamic identification system shown in FIG. 6, as shown in FIG. 5, the method includes:

[0104] S502, User-Customer Dialogue: Collecting multi-round dialogue data between users of an operator and artificial customer service or robot customer service in the system.

[0105] S504, Data Cleaning: Data desensitization and cleaning of mobile phone dialogue data.

[0106] S506, Keyword Extraction: Based on the dialogue with the user, at least two keywords are first extracted based on the BERT model as the current state and the next state of the agent. Then a random action is selected, and the reward value is calculated based on the set reward function to build the data required for training the deep Q network. The data style is composed of: (current state s, action a, reward value, next state), which means that the agent will obtain the reward r by adopting the action a to enter the next state from the current state.

[0107] S508, User Feature Data: Collecting basic information, social relationships, product purchase records, etc. of the user and performing desensitization processing.

[0108] S510, Personalized Weight Extraction: Using the collected user information to build a social network model, in which the user is a node and the relationship between the users is an edge. According to the similarity between users, social relationship and user historical behavior, the personalized weight of the user is calculated.

[0109] S512, user personalized intention recognition: training a personalized intention dynamic recognition model, specifically training a personalized deep Q network PDQN. During training, the agent and the environment continuously interact, the agent continuously observes the user state, selects actions, receives rewards, and combines the personalized weight of each user to update the prediction value of the Q function.

[0110] S514, function operation: after training the DQN, the agent can select the action with the maximum Q value according to the user state, and then realize the dynamic recognition of the user's personalized intention. The personalized intention dynamic recognition system jumps to the corresponding function module according to the recognized user intention, and realizes the subsequent operation of the user's demand. The corresponding function module is the action taken by the personalized intention dynamic recognition system, that is, the candidate set of user intentions, which are product recommendation, business handling, knowledge question and answer, traffic query, product after-sales and transfer to artificial.

[0111] In particular, the embodiments of the present disclosure use deep reinforcement learning to realize dynamic user intention personalized recognition. The personalized intention dynamic recognition system is used as an agent, which is responsible for inferring the user's intention from the input data and making corresponding decisions. Each round of conversation between the user and the intelligent customer service is regarded as a time step, the keywords of the input conversation are regarded as the state of the agent, which contains information about the current environment and the behavior and observation history of the agent before. The action represents the behavior that the agent can take in a certain state. In the intention recognition task, the action is defined as a set of possible intention categories of the user. We set a total of 6 actions that the system can take, that is, the candidate set of user intentions, which are product recommendation, business handling, knowledge question and answer, traffic query, product after-sales and transfer to artificial. The reward is the feedback signal in reinforcement learning, which is used to guide the decision of the agent. In the personalized intention dynamic recognition task, positive rewards can be given according to the correct recognition of the user's intention by the system, and negative rewards can be given for incorrect recognition. In the embodiments of the present disclosure, the reward function is calculated according to the following formula (1).

[0112] R = R f + R s (1)

[0113] Wherein, R f is the reward based on task completion, and R sis the reward based on user satisfaction. The task completion reward is the prediction accuracy of the next action, and the action prediction accuracy is used as the reward score. If the prediction is incorrect, the reward score is negative. The user satisfaction reward is based on the user's feedback score. In DQN, the agent predicts the user's next action based on the user's input, and obtains the user's next state and immediate reward. These states, actions and rewards are used to construct experience tuples and stored in an experience replay buffer. These experience tuples are randomly sampled for training the neural network, and the weights of the neural network are updated by minimizing the difference between the predicted Q value and the target Q value to improve training efficiency and stability.

[0114] The traditional DQN does not consider the personalized intention of the user, and the embodiment of the present disclosure designs a personalized deep Q network (PDQN) to realize dynamic recognition of user intention. Specifically, we use a convolutional layer and multiple fully connected layers to build DQN, and the last fully connected layer is used to predict Q value. The predicted value Q' and the loss function are calculated according to the following formulas (2) and (3).

[0115] The predicted Q value of state s and action a, r is the actual reward obtained, s' is the next state after executing action a, and γ is the discount factor, is the maximum Q value of the target Q network for the next state s', where θ - is the parameter of the target Q network. The optimizer adjusts the network parameters θ to minimize the loss function, thereby gradually improving the performance of the Q network. In this way, the network can more accurately estimate the long-term return of executing each action in each state.

[0116] In actual dialog, the personalized intention dynamic recognition system uses the deep Q network to make online decisions according to the current state, selects the action with the highest Q value, and realizes dynamic recognition of user intention. In this way, the personalized intention dynamic recognition system can adjust the strategy in real time during the conversation, and dynamically recognize the user's intention according to the user's input.

[0117] As shown in FIG. 6, the natural language processing module based on large model is embedded in the dialogue management module, which can parse user input, generate output text, and maintain context dialogue, ensuring that the system can record and understand the context of the user's multi-round dialogue, and provide coherent and effective responses.

[0118] The keyword extraction module extracts keywords from user input based on the dialogue of user input, and obtains the representation of the corresponding word vector based on the pre-trained model, which is used for subsequent use.

[0119] The user feature module stores and manages basic information of the user, and performs construction of a user social graph, processing of a user portrait, and calculation of a personalized weight factor to provide personalized service support.

[0120] The intent recognition module includes a DQN-based personalized intent dynamic recognition method of the embodiment of the present disclosure. Based on the extracted keywords and corresponding representations, the action with the highest predicted Q value is taken as the user intent. According to the records stored in the dialogue management module and the historical interaction information of the user context, and after obtaining the user preference features in the user feature module, the response is weighted, which can more accurately identify the user intent.

[0121] The function operation module provides an interface for each operation. After the user's intent is identified, the function operation application corresponding to each intent is provided.

[0122] The personalized intent dynamic recognition method provided by the embodiment of the present disclosure is a telecom service personalized intent recognition method based on a large model and deep reinforcement learning. The user's social relationship is used to learn personalized features. By analyzing the relationship network between users, the user's interest, preference and demand are identified, so as to provide personalized services and suggestions for each user. The trained personalized intent dynamic recognition model combines the user's personalized features to enhance the ability of intent recognition. In addition, the embodiment of the present disclosure also uses a large model to enhance the understanding of the knowledge in the field of telecom service by the personalized intent dynamic recognition system. In summary, the embodiment of the present disclosure improves the accuracy of the intent recognition method and the personalized service ability, thereby providing a better user experience.

[0123] Based on the same inventive concept, the embodiment of the present disclosure also provides a personalized intent dynamic recognition device, as described in the following embodiments. Since the principle of solving the problem of the device embodiment is similar to that of the above-mentioned method embodiment, the implementation of the device embodiment can be referred to the implementation of the above-mentioned method embodiment, and the repeated parts will not be described here.

[0124] FIG. 7 shows a schematic diagram of a personalized intent dynamic recognition device in an embodiment of the present disclosure. As shown in FIG. 7, the device includes:

[0125] The historical dialogue data acquisition module 701 is configured to acquire historical dialogue data of at least one user and a customer service;

[0126] The experience tuple data construction module 702 is configured to construct experience tuple data for training a deep reinforcement learning model according to the historical dialogue data, wherein the deep reinforcement learning model is used to predict the value of an agent from a state to an action, and the experience tuple data includes a current state, an action, a reward value and a next state of the agent;

[0127] The personalized intention recognition model training module 703 is configured to obtain the personalized weight information of each user, and update the predicted value of the deep reinforcement learning model by using the personalized weight information of each user in the process of training the deep reinforcement learning model by the agent using the experience tuple data, until a personalized intention recognition model meeting a preset condition is obtained, wherein the personalized intention recognition model is used to recognize the personalized intention of each user.

[0128] In some embodiments of the present disclosure, the experience tuple data construction module 702 is further configured to take the historical dialogue data as the current state of the agent, determine the action of the agent according to the current state, calculate the reward value of the agent according to the feedback of the user based on the action of the agent and the pre-designed reward function, and obtain the next state of the agent.

[0129] In some embodiments of the present disclosure, the experience tuple data construction module 702 is further configured to obtain the feedback information of the user, wherein the feedback information includes the completion degree information and the satisfaction degree information, and calculate the reward value of the agent according to the completion degree information and the satisfaction degree information.

[0130] In some embodiments of the present disclosure, the personalized intention recognition model training module 703 is further configured to obtain the user feature information, and calculate the personalized weight of the user according to the user feature information.

[0131] In some embodiments of the present disclosure, the historical dialogue data obtaining module 701 is further configured to perform data preprocessing on the historical dialogue data, and extract keyword information from the historical dialogue data after data preprocessing, wherein the keyword is used to construct the experience tuple data for training the deep reinforcement learning model.

[0132] It should be noted that the above modules and the examples and application scenarios realized by the corresponding steps are the same, but are not limited to the content disclosed in the above method embodiments. It should be noted that the above modules as part of the device can be executed in a computer system such as a group of computer executable instructions.

[0133] Those skilled in the art can understand that each aspect of the present disclosure can be implemented as a system, a method or a program product. Therefore, each aspect of the present disclosure can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" here.

[0134] The electronic device 800 according to this embodiment of the present disclosure will be described below with reference to FIG. 8. FIG. 8 shows the electronic device 800 only as an example, and should not bring any limitation to the functions and use range of the embodiments of the present disclosure.

[0135] As shown in FIG. 8, electronic device 800 is in the form of a general computing device. Components of electronic device 800 can include, but are not limited to, at least one processing unit 810 as described above, at least one memory unit 820 as described above, and a bus 830 that connects different system components, including memory unit 820 and processing unit 810.

[0136] The memory unit stores program codes that can be executed by the processing unit 810, so that the processing unit 810 performs the steps of the various exemplary embodiments according to the present disclosure described in the above “Exemplary Method” section of the present specification. For example, the processing unit 810 can perform the following steps of the above method embodiments: obtaining historical dialogue data of at least one user and a customer service; constructing experience tuple data for training a deep reinforcement learning model according to the historical dialogue data, wherein the deep reinforcement learning model is used to predict the value of an agent from a state to an action, and the experience tuple data includes the current state, action, reward value and next state of the agent; obtaining personalized weight information of each user, and updating the predicted value of the deep reinforcement learning model using the personalized weight information of each user in the process of training the deep reinforcement learning model by the agent using the experience tuple data, until a personalized intent recognition model that meets a preset condition is obtained, wherein the personalized intent recognition model is used to identify the personalized intent of each user.

[0137] The memory unit 820 can include a readable medium in the form of volatile memory units, such as a random access memory (RAM) 8201 and / or a cache memory unit 8202, and can further include a read-only memory (ROM) 8203.

[0138] The memory unit 820 can further include program / utility 8204 having a set of program modules 8205, including but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof can include implementation of a network environment.

[0139] The bus 830 can represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus architectures.

[0140] The electronic device 800 can also communicate with one or more external devices 840 such as a keyboard or pointing device, a Bluetooth device, or a database, and / or one or more devices that enable a user to interact with the electronic device 800 and / or one or more devices (e.g., a router, a modem, a server, etc.) that enable the electronic device 800 to communicate with one or more other computing devices. Such communication can occur via an input / output (I / O) interface 850. Still yet, the electronic device 800 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the Internet, through a network adapter 860. As depicted, the network adapter 860 communicates with the other components of the electronic device 800 via the bus 830. It should be appreciated that the network adapter 860 and / or the bus 830 can be implemented using one or more types of technology, including, but not limited to, Ethernet, Bluetooth, and / or any other suitable technology.

[0141] From the above description of the embodiments, those skilled in the art will easily understand that the example embodiments described herein can be implemented by software and also can be implemented by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiments of the present disclosure.

[0142] In particular, according to the embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the method in the above embodiments.

[0143] In the example embodiments of the present disclosure, a computer readable storage medium is also provided, which can be a readable signal medium or a readable storage medium. A program product capable of implementing the method of the present disclosure is stored on the computer readable storage medium. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program codes for causing the terminal device to perform the steps according to various example embodiments of the present disclosure described in the above "example method" section of the specification when the program product is run on the terminal device.

[0144] More specific examples of the computer-readable storage medium in the present disclosure can include but are not limited to an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] In the present disclosure, a computer readable storage medium can include a data signal carrying the readable program code in a baseband or in a carrier wave. Such a propagated data signal can take a wide variety of forms, including but not limited to electro-magnetic, optical, or any suitable combination thereof. A computer readable storage medium can also be any non-transitory computer readable medium which can be considered to be a tangible storage medium. A computer readable storage medium can be any medium that can be read by a machine learner, artificial intelligence, or an instruction execution system, apparatus, or device.

[0146] Optionally, program code embodied on a computer readable storage medium can be transmitted by any data transmission techniques, including but not limited to radio frequency, wireless, cable, wire, optical fiber cable, or any suitable combination thereof.

[0147] In an implementation, the program code for carrying out operations of the present disclosure can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, and the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0148] It should be noted that, although several modules or units for a device to perform actions are mentioned in the above detailed description, such a division into modules or units is not mandatory. In fact, according to an embodiment of the present disclosure, features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, features and functions of one module or unit described above can be further divided into a plurality of modules or units.

[0149] Moreover, although individual steps of the methods in the present disclosure are described in a particular order in the drawings, this is not required or implied, nor is it necessary to perform all of the steps shown to achieve the desired result. Additionally or alternatively, certain steps can be omitted, combined into a single step, performed in a different order, broken down into multiple steps, and / or the like.

[0150] Through the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product. The software product can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash disk, a mobile hard disk, or the like) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to perform the methods according to the embodiments of the present disclosure.

[0151] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure following the general principles thereof and including such departures from the present disclosure that come within known use or custom in the art to which the present disclosure pertains. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A personalized intention dynamic identification method, comprising: obtaining historical dialogue data of at least one user and a customer service; constructing experience tuple data for training a deep reinforcement learning model according to the historical dialogue data, wherein the deep reinforcement learning model is used to predict the value of an agent from a state to an action, and the experience tuple data includes the current state, action, reward value and next state of the agent; obtaining personalized weight information of each user, and updating the predicted value of the deep reinforcement learning model using the personalized weight information of each user during the process of training the deep reinforcement learning model by the agent using the experience tuple data, until a personalized intention identification model that meets a preset condition is obtained, wherein the personalized intention identification model is used to identify the personalized intention of each user.

2. The personalized intent dynamic recognition method of claim 1, wherein, The customer service is an agent, and the experience tuple data for training the deep reinforcement learning model is constructed according to the historical dialogue data, comprising: taking the historical dialogue data as the current state of the agent; determining the action of the agent according to the current state; calculating the reward value of the agent according to the feedback of the user based on the action of the agent and a pre-designed reward function, and obtaining the next state of the agent.

3. The personalized intent dynamic recognition method of claim 2, wherein, The calculation of the reward value of the agent according to the feedback of the user based on the action of the agent and the pre-designed reward function comprises: obtaining feedback information of the user, the feedback information including completion degree information and satisfaction degree information; calculating the reward value of the agent according to the completion degree information and the satisfaction degree information. 4.The personalized intent dynamic identification method of claim 1, wherein, The obtaining of the personalized weight information of each user comprises: obtaining user feature information; calculating the personalized weight of the user according to the user feature information. 5.The personalized intent dynamic identification method of claim 1, wherein, After obtaining the historical dialogue data of at least one user and a customer service, the method further comprises: data preprocessing of the historical dialogue data; extracting keyword information from the historical dialogue data after data preprocessing, the keywords being used to construct experience tuple data for training a deep reinforcement learning model. 6.The personalized intent dynamic identification method of claim 1, wherein, The customer service is a dialogue robot, and the dialogue robot is obtained based on a pre-collected operation service field knowledge to train a semantic dialogue large model.

7. A personalized intention dynamic identification device, comprising: a historical dialogue data obtaining module configured to obtain historical dialogue data of at least one user and a customer service; an experience tuple data constructing module configured to construct experience tuple data for training a deep reinforcement learning model according to the historical dialogue data, wherein the deep reinforcement learning model is used to predict the value of an agent from a state to an action, and the experience tuple data includes the current state, action, reward value and next state of the agent; a personalized intention identification model training module configured to obtain personalized weight information of each user, and update the predicted value of the deep reinforcement learning model using the personalized weight information of each user during the process of training the deep reinforcement learning model by the agent using the experience tuple data, until a personalized intention identification model that meets a preset condition is obtained, wherein the personalized intention identification model is used to identify the personalized intention of each user. 8.An electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the personalized intention dynamic recognition method of any one of claims 1-6 via execution of the executable instructions. 9.A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the personalized intention dynamic recognition method of any one of claims 1-6.

10. A computer program product, comprising: computer program or instructions which, when executed by a processor, implement the personalized intention dynamic recognition method of any one of claims 1-6.

Images