Intelligent service dialogue method and system integrating generative big model and knowledge graph

Abstract

The present invention discloses an intelligent service dialogue method and system integrating a generative big model and a knowledge graph, including: a language model module, which constructs a sentence reply model based on BERT and uses a question-answering data set for training; a data extraction module, which generates a candidate item set and divides frequent and infrequent item sets; a knowledge graph module, which generates association rules and constructs a first and second knowledge graph; a hybrid retrieval module, which cross-retrieves two knowledge graphs to generate paths; a personalized user module, which determines demand intentions and sentiment tendency values; a sentence generation module, which uses sentiment tendency values ​​as dynamic weights and generates reply sentences in combination with user needs and retrieval paths; a habit perception module, which constructs a habit function, optimizes the model through reinforcement learning, and continuously learns user habit characteristics. The system can not only provide users with accurate and personalized dialogue services, but also better adapt to users' usage habits, improve user experience, and has good interpretability.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to an intelligent service dialogue method and system that integrates a generative big model and a knowledge graph. Background Art

[0002] With the rapid development of information technology and the widespread popularity of the Internet, people's demand for efficient and convenient intelligent services is growing. In various fields, such as e-commerce, financial services, healthcare, online education, etc., users expect to obtain accurate information and solutions through natural language interaction. As a technical means to meet user needs, intelligent service dialogue system has gradually become a hot topic in research and application.

[0003] Traditional intelligent dialogue methods are often completed through rule definition and template matching methods. The resulting sentences have large deviations in continuity and dialogue coherence, and are prone to answering irrelevant questions and generating wrong answers.

[0004] Although there are some dialogue systems or methods based on large language models, they lack detailed consideration in data processing, lack the use of knowledge graphs, and insufficient consideration for personalized users. This makes them unable to adapt well to user usage habits during long-term interactions with users, and unable to provide more intimate and efficient services. Summary of the invention

[0005] The purpose of the present invention is to provide an intelligent service dialogue method and system that integrates a generative big model and a knowledge graph.

[0006] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0007] The first aspect of the present invention provides an intelligent service dialogue method integrating a generative big model and a knowledge graph, comprising:

[0008] S10: scanning a professional database to generate a candidate item set, calculating weighted support of items in the candidate item set, dividing the candidate item set based on the weighted support, and obtaining frequent item sets and infrequent item sets;

[0009] S20 generates association rules according to the frequent itemsets, generates a first knowledge graph based on the association rules and the frequent itemsets, and generates a second knowledge graph based on the infrequent itemsets;

[0010] S30 obtains a user input statement, performs a cross search based on the first knowledge graph and the second knowledge graph, and generates a search path; determines the demand intention and the emotional tendency value according to the user input statement;

[0011] S40 builds a sentence reply model based on the BERT large language model, and trains the sentence reply model using the question-answering dataset; uses the sentiment tendency value as a dynamic weight value of the sentence reply model, and the sentence reply model obtains a reply sentence according to the demand intention and the retrieval path;

[0012] S50 constructs a habit function according to the context of the continuous conversation content, optimizes the sentence response model through reinforcement learning based on the habit function, and continuously learns the user's habit characteristics.

[0013] As a further method, in step S10, specifically:

[0014] Preprocess professional databases, including text cleaning, word segmentation, and stop word removal;

[0015] The FP-Growth algorithm is used to scan the preprocessed professional database to generate an initial candidate item set, and a multi-word phrase is generated based on the initial candidate item set using the sliding window method to obtain a candidate item set;

[0016] Calculate the weighted support of each item in the candidate set, the expression is:

[0017]

[0018] Among them, C represents the candidate item set, c i is the i-th candidate in the candidate set C, w f (c i ), w p (c i ) and w s (c i ) are candidate items c i The word frequency weight, position weight and semantic importance weight of the word are represented by D, which is a dataset formed after preprocessing of the professional database. j is the jth specific record in the data set D, and m is the total number of records in the data set D;

[0019] The candidate items whose weighted support is in the top 30% in the candidate set are classified into frequent item sets, and the candidate items whose weighted support is lower than the top 30% are classified into infrequent item sets.

[0020] As a further method, in step S20, specifically:

[0021] Use the Apriori algorithm to remove redundant rules from frequent item sets and obtain association rules;

[0022] Add frequent items as nodes to the first knowledge graph, and establish edges between nodes according to association rules;

[0023] Add infrequent items as nodes to the second knowledge graph and establish edges between nodes;

[0024] The semantic similarity of the nodes is calculated by the Word2Vec algorithm, and the nodes whose language similarity between the first knowledge graph and the second knowledge graph is greater than or equal to 0.6 are connected;

[0025] Attributes are assigned to nodes of the first knowledge graph and the second knowledge graph according to the professional database.

[0026] As a further method, in step S30, the method of performing cross-search based on the first knowledge graph and the second knowledge graph to generate a search path is specifically:

[0027] Using the key information input by the user as the starting point, initialize the search stack to orderly store the nodes to be visited, and push the starting node into the stack;

[0028] When searching, the nodes related to the key information are searched first before the nodes in the first knowledge graph. If the related nodes in the first knowledge graph are connected to the second knowledge graph, the corresponding nodes in the second knowledge graph are included in the search path. If the related nodes cannot be found in the first knowledge graph, the search is further conducted in the second knowledge graph.

[0029] During the entire retrieval process, the search stack continuously records the visited nodes and the edges connecting the nodes to form a retrieval path.

[0030] As a further method, in step S30, the method of determining the demand intention and the emotional tendency value according to the user input sentence is specifically:

[0031] Obtain input sentences with annotated demand intent as training data, select a machine learning algorithm to build a model, train the model with the training data, obtain a demand intent recognition model, and determine the user's demand intent through the demand intent recognition model based on the sentence input by the user;

[0032] The sentiment tendency value is calculated based on the lexical sentiment elements and tone elements in the user input sentence. The expression is:

[0033]

[0034] Among them, P is the set of active words, For active vocabulary i The basic score, For active vocabulary i The number of repetitions in the user input sentence, f is the decay function, where Where A is the number of times the word appears repeatedly, which means to avoid too many repetitions that lead to too high scores when the word appears repeatedly, k is the number of strong tone elements in the user's input sentence, and t i is the intensity score of the i-th strong tone element, φ is the negativity increase coefficient. When the strong tone and positive words appear at the same time, the positivity is enhanced. N is the set of negative words. For negative words i The basic score, For negative words i The number of repetitions in the user input sentence, ψ is the negativity enhancement coefficient, when a strong tone and negative words appear at the same time, the negativity level is enhanced.

[0035] As a further method, in step S40, the method of building a sentence response model based on the BERT large language model and using the question-answering dataset to train the sentence response model is specifically as follows:

[0036] Get the BERT-base model from the pre-trained model library, add a fully connected layer on top of the BERT-base model as the output layer, introduce the Transformer decoder, and use the output of the fully connected layer as the input of the Transformer decoder;

[0037] Divide the training process into multiple batches, each batch contains a specific number of question-answer pairs;

[0038] During the training process of each batch, the sentence reply model is used to predict the question, the loss function is calculated, and the loss function is back-propagated through the back-propagation algorithm to update the model parameters, where the loss function is expressed as:

[0039]

[0040] Where N is the number of question-answer pairs in a batch, T is the number of dialogue rounds or time steps, λ is the weight coefficient for balancing the importance of the loss term, and y i，t The one-hot encoding of the true label at time step t for the i-th question answer, is the predicted label for the answer to the ith question at time step t, τ is the smoothness control parameter, The predicted label for the ith question answer pair at time step t-1.

[0041] As a further method, in step S40, the method of using the sentiment tendency value as the dynamic weight value of the sentence reply model is specifically: before the softmax function of the output layer of the sentence reply model, the sentiment tendency value is normalized from 0 to 1 and then added to the output vector of the sentence reply model as the dynamic weight value.

[0042] As a further method, in step S40, the sentence reply model obtains a reply sentence according to the demand intention and the search path, specifically:

[0043] Based on the demand intention, the attention layer is introduced into the language response model to obtain the attention weight, which is expressed as:

[0044]

[0045] Among them, Q is the demand intention vector set, K is the key vector set, V is the value vector set, K i is the i-th key vector in the key vector set, d k is the dimension of the key vector, n is the total number of key vectors in the key vector set, K j is the jth key vector in the key vector set, V i is the i-th value vector in the value vector set;

[0046] The attention weights are weighted summed with the output of the middle layer of the language response model to obtain the attention-adjusted output, and the output is used as the input of the subsequent layer of the language response model;

[0047] The search results are obtained based on the search path, and the search results are provided as input to the optimized sentence reply model to output the reply sentence.

[0048] As a further method, in step S50, specifically:

[0049] Obtain all text data between the user and the intelligent service dialogue system and extract features, including lexical features, grammatical features and semantic features;

[0050] Use TF-IDF to convert features into feature vectors as conversation samples;

[0051] The habit function is constructed based on the dialogue samples. The expression of the habit function is:

[0052]

[0053] Among them, d i is the i-th dialogue sample, D is the number of dialogue samples, α is the weight coefficient of the difference between typical habit features and overall features, is d i The center vector of the cluster is the typical habit feature, C is d i The overall mean vector of the cluster is the overall feature, and β is d i The weight coefficient of the feature dimension between the reference vector C, m is the number of feature dimensions, v ik is the kth eigenvalue of the ith dialogue sample, C kis the kth eigenvalue of the reference vector C, 6 is the weight coefficient of the reference vector, v i is the feature vector of the i-th dialogue sample, C is the reference vector, ∈ k is the adjustment factor of the feature dimension importance, is the kth eigenvalue of the typical habit feature;

[0054] Taking the habit function as the state input of reinforcement learning, the update formula of the state action value function is:

[0055]

[0056] Among them, Q represents the state action value function, H(d i ) is the habit function, a is the action, ← is the update operation, α is the learning rate, R is the reward function, γ is the discount factor, H(d i+1 ) is the habit function in the next state, aξ is the set of all possible actions, It is expressed as the habit function value is H(d i+1 ), the set of all possible actions a ξ The maximum value of the state action-value function in .

[0057] The second aspect of the present invention further provides an intelligent service dialogue system integrating a generative large model and a knowledge graph, including:

[0058] A language model module is used to obtain a question-answering dataset, build a sentence answering model based on the BERT large language model, and train the sentence answering model using the question-answering dataset;

[0059] A data extraction module is used to scan a professional database to generate a candidate item set, calculate the weighted support of the candidate item set, and divide it based on the weighted support to obtain frequent item sets and infrequent item sets;

[0060] A knowledge graph module, configured to generate association rules according to the frequent itemsets, generate a first knowledge graph based on the association rules and the frequent itemsets, and generate a second knowledge graph based on the infrequent itemsets;

[0061] A hybrid retrieval module, used to perform a cross-search on the first knowledge graph and the second knowledge graph according to a user input statement to generate a retrieval path;

[0062] A personalized user module, used to determine the demand intention and emotional tendency value according to the user input sentence;

[0063] A sentence generation module, used to use the sentiment tendency value as a dynamic weight value of a sentence reply model, and the sentence reply model obtains a reply sentence according to the demand intention and the search path;

[0064] The habit perception module is used to build a habit function according to the context of continuous dialogue content, optimize the sentence response model through reinforcement learning based on the habit function, and continuously learn the user's habit characteristics.

[0065] Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

[0066] (1) The present invention provides an intelligent service dialogue method and system that integrates a generative big model and a knowledge graph. By combining the ability of the generative big model to capture complex language patterns and the accuracy of the knowledge graph, the system can not only generate smooth and natural dialogues, but also ensure that the information provided is accurate and reliable.

[0067] (2) The present invention can determine the demand intention and emotional tendency value based on the user's input statement by creating personalized users, so that the system can make targeted responses based on the user's specific needs and emotional state, providing a more personalized service experience.

[0068] (3) The present invention constructs a habit function and uses reinforcement learning to optimize the sentence response model, which can continuously learn the user's habit characteristics. As time goes by, the system's understanding of the user will become more and more in-depth, and the response will be more in line with the user's habits and preferences. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 This is a flowchart of the steps of the intelligent service dialogue method that integrates the generative big model and the knowledge graph of the present invention.

[0070] Figure 2 This is a logical schematic diagram of an embodiment of the intelligent service dialogue system that integrates the generative big model and the knowledge graph of the present invention. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0072] In one embodiment, an online travel platform hopes to provide users with a better service experience through an intelligent service dialogue system, answering users' questions about travel destinations, hotel reservations, attraction tickets, transportation, etc.

[0073] Reference Figure 1 As shown, the present invention provides an intelligent service dialogue method integrating a generative big model and a knowledge graph, including:

[0074] S10: scanning a professional database to generate a candidate item set, calculating weighted support of items in the candidate item set, dividing the candidate item set based on the weighted support, and obtaining frequent item sets and infrequent item sets;

[0075] It should be explained that the division of frequent itemsets and infrequent itemsets is helpful for subsequent data analysis and knowledge graph construction. Frequent itemsets usually represent important and common concepts or relationships in question-answering datasets, and can be used to generate association rules and build the core part of knowledge graphs. Infrequent itemsets can be used as supplementary information or for further analysis in specific cases.

[0076] In the actual evaluation, in the Chengdu tourism intelligent service scenario, the professional database related to Chengdu tourism is first preprocessed. Special symbols and non-standard expressions in the texts such as tourist evaluations, travel guides and scenic spot introductions in the database are removed, and word segmentation operations are performed to segment sentences such as "Chengdu's Wuhou Temple is very spectacular" into words such as "Chengdu", "Wuhou Temple", "very", "spectacular", etc., and stop words such as "of", "is", and "in" are removed. Then, the FP-Growth algorithm is used to scan the preprocessed database to generate an initial candidate item set, such as "Chengdu Hotel", "Qingcheng Mountain Attractions", "Subway", etc. Then, a richer candidate item set is generated through the sliding window method, such as "Chengdu City Luxury Hotel" and "Qingcheng Mountain Ticket Booking". The weighted support of each item in the candidate item set is calculated. For the candidate "Chengdu City Luxury Hotel", the word frequency weight is determined to be 0.3 based on its frequency of occurrence in the database. If it often appears in the hot search position, the position weight is set to 0.4. The semantic importance weight is determined to be 0.3 by analyzing the degree of association with other tourism keywords. According to statistics, 1800 of the 8000 records in the database contain this candidate item, and the value is 0.625 when substituted into the weighted support expression. According to the first 30% of the weighted support, the candidate items with higher support are classified as frequent item sets, and the rest are infrequent item sets.

[0077] S20 generates association rules according to the frequent itemsets, generates a first knowledge graph based on the association rules and the frequent itemsets, and generates a second knowledge graph based on the infrequent itemsets;

[0078] It is important to understand that the larger the data set used to construct the knowledge graph, the larger the content reserve of the resulting knowledge graph and the wider the scope of its coverage;

[0079] In the actual evaluation, the Apriori algorithm is used to remove redundant rules from frequent item sets and obtain association rules. Frequent items are added as nodes to the first knowledge graph, and edges between nodes are established according to association rules. For example, the nodes "Chengdu" and "Wuhou Temple" are connected by the "includes scenic spots" relationship edge, "Chengdu" and "Jinli" are connected by the "includes scenic spots" relationship edge, "Chengdu" and "hot pot" are connected by the "special food" relationship edge, and "Wuhou Temple" and "Jinli" are connected by the "close distance" relationship edge. Infrequent items are added as nodes to the second knowledge graph, and corresponding edges are established. For example, "homestay" and "Chengdu" establish a weak association edge of "accommodation options", and "convenient transportation" and "Chengdu" establish a similar weak association edge. The semantic similarity of nodes is calculated by the Word2Vec algorithm, and the nodes with language similarity greater than or equal to 0.6 in the first knowledge graph and the second knowledge graph are connected. At the same time, attributes are assigned to the nodes of the knowledge graph according to the professional database.

[0080] S30 obtains a user input statement, performs a cross search based on the first knowledge graph and the second knowledge graph, and generates a search path; determines the demand intention and the emotional tendency value according to the user input statement;

[0081] In the actual evaluation, when the user inputs "I want to travel to Chengdu, what fun places and delicious food are there?", the user input is first semantically analyzed to extract the keywords "Chengdu", "fun", and "delicious", and then the "Chengdu" node is used as the starting point for searching in the first knowledge graph to find nodes related to "fun" such as "Wuhou Temple" and "Jinli", as well as the node "hot pot" related to "delicious". At the same time, the second knowledge graph is also searched to see if there is any non-frequent information related to "Chengdu" and "fun" and "delicious" that can be supplemented; the generated search path is "Chengdu-including attractions-Wuhou Temple-relatively close-Jinli-special food-hot pot-accommodation options-[name of homestay]-convenient transportation-[traffic tips]".

[0082] S40 builds a sentence reply model based on the BERT large language model, and trains the sentence reply model using the question-answering dataset; uses the sentiment tendency value as a dynamic weight value of the sentence reply model, and the sentence reply model obtains a reply sentence according to the demand intention and the retrieval path;

[0083] It needs to be explained that by setting different weight adjustment strategies, for users with positive emotional tendencies, the enthusiasm and affirmative weight of the answers generated by the language model can be increased; for users with negative emotional tendencies, the weight of comforting and problem solving can be increased; when setting positive emotional tendencies, the probability of generating positive words and encouraging sentences in the language model is increased; when setting negative emotional tendencies, the probability of generating sentences such as apologies, solutions, and caring about the user's feelings is increased;

[0084] In the actual evaluation, we collected questions and corresponding answers from historical user consultation records on online travel platforms. We screened out 15,000 question and answer pairs related to travel destinations, hotel reservations, attraction tickets, transportation, etc. from the records of the past year. We also captured user questions and responses from well-known travel forums. After screening and sorting, we obtained another 5,000 pieces of relevant data, which were all included in the question-answering dataset.

[0085] In the actual evaluation, the BERT-base model is obtained from the pre-trained model library, a fully connected layer is added on top of it as the output layer, a Transformer decoder is introduced, and the output of the fully connected layer is used as the decoder input. The training process is divided into multiple batches, and each batch contains a certain number of question-answer pairs. In each batch training, the sentence reply model is used to predict the questions, the loss function is calculated, and the model parameters are updated through back propagation; the pre-trained BERT-base model is used as the basic architecture, which has a 12-layer Transformer encoder and a hidden layer dimension of 768; the model is trained using the collected 20,000 question-answer dataset. The dataset is divided into training set, validation set, and test set in a ratio of 7:2:1. During the training process, the stochastic gradient descent optimization algorithm is used, the learning rate is set to 0.001, the batch size is 32, and it is further trained for 50 epochs;

[0086] In the actual evaluation, when the sentiment tendency value is 0.8, this value is used as the weight adjustment factor of some parameters in the sentence response model. When generating the answer sentence, the model originally predicted the answer "Chengdu is a very distinctive tourist city with many fun places and delicious food." After considering the sentiment tendency value, the generated answer is "Chengdu is a great tourist city! There are many very fun places and delicious food waiting for you to explore." The positive words and expressions such as "great", "very", and "waiting for you to explore" here are added under the influence of the sentiment tendency value; if the sentiment tendency value is lower, the generated answer will be more objective or contain some soothing language, "Chengdu tourism has many different experiences. Although not everyone may find it very fun, there are many places worth visiting. You can choose attractions and food according to your interests and hobbies;

[0087] In the actual evaluation, input sentences with annotated demand intent are obtained as training data, and a suitable machine learning algorithm is selected to build a model and train it to obtain a demand intent recognition model. According to the user input sentence, the model is used to determine that the user's demand intent is to query for scenic spot information. The sentiment tendency value is calculated based on the lexical sentiment elements and tone elements in the user input sentence. Assuming that there are words such as "fun" in the positive vocabulary set, its basic score is set to 2, and it appears once in the sentence. The attenuation function is calculated according to a specific formula, the number of strong tone elements is 0, and the negative vocabulary set is empty. Substitute the sentiment tendency value expression to get the specific value;

[0088] In the actual evaluation, before the softmax function of the output layer of the sentence reply model, the sentiment tendency value is normalized from 0 to 1 and added to the output vector as a dynamic weight value. Based on the demand intention, an attention layer is introduced into the language reply model to obtain the attention weight, and the attention weight is weighted and summed with the output of the intermediate layer to obtain the output after attention adjustment, and then the output is used as the input of the subsequent layer. The retrieval results are obtained based on the retrieval path, and they are provided as input to the optimized sentence reply model to output the reply sentence;

[0089] Specifically, the user's demand intention (querying for tourist destinations and food information) and the information obtained in the search path are combined to generate a specific reply sentence: "Based on your needs, we recommend Wuhou Temple and Jinli in Chengdu. Both places are very unique and very fun. Chengdu's hot pot is also not to be missed. It tastes great! If you want to find a place to stay, you can consider some homestays, which will give you a different experience. In addition, Chengdu's transportation is relatively convenient, and you can easily reach various attractions."

[0090] S50 constructs a habit function according to the context of the continuous conversation content, optimizes the sentence response model through reinforcement learning based on the habit function, and continuously learns the user's habit characteristics.

[0091] It should be explained that the method of obtaining the content of the contextual continuous conversation is to collect historical user information and records of customer responses;

[0092] In actual evaluations, users often ask questions about special snacks during multiple interactions with the intelligent service dialogue system. By building a habit function, when users mention keywords such as "travel destination" or "food" in the conversation, the prediction probability of questions related to "special snacks" increases. At the same time, if users have already searched for scenic spots and food information in a certain city in previous conversations, in subsequent conversations, if the city or related keywords are mentioned again, the system can give priority to recommending content that the user has not mentioned before but may be interested in; the habit function can be continuously adjusted and optimized according to user behavior. If users begin to pay attention to natural scenery attractions over a period of time, the system can adjust the habit function accordingly to increase the prediction and recommendation of questions related to natural landscapes;

[0093] In the actual evaluation, the reinforcement learning algorithm Q-learning was used to optimize the sentence response model according to the habit function. Through continuous learning and optimization, the model can better adapt to the user's habit characteristics. When the user asks about a certain tourist destination again, the model can predict what the user may be interested in based on the habit function and give priority to providing relevant information and suggestions.

[0094] In the actual evaluation, as the interaction between users and the system continues to increase, the habit function is continuously updated to better capture changes in users' habits. Users have recently begun to pay attention to parent-child travel content. The system can adjust the habit function to increase predictions and optimizations for related issues such as parent-child hotels and child-friendly attractions. At the same time, the system can also continuously adjust and improve the habit function based on user feedback to improve service quality and user satisfaction.

[0095] In this embodiment, the specific conversation content is: User: "I want to travel to Chengdu. Are there any fun places and delicious food?"

[0096] System: "Chengdu is a great tourist city! There are many fun places and delicious food waiting for you to explore. Based on your needs, we recommend Wuhou Temple and Jinli in Chengdu. These two places are very unique and fun. Chengdu's hot pot is also not to be missed. It tastes great! If you want to find a place to stay, you can consider some homestays, which will give you a different experience. In addition, Chengdu's transportation is also relatively convenient, and you can easily reach various attractions."

[0097] User: “How is the B&B?”

[0098] System: "There are many different styles of B&Bs in Chengdu. Some B&Bs have very beautiful environments and allow you to experience the leisurely life in Chengdu. Some B&Bs are close to tourist attractions and are very convenient for travel. You can choose according to your preferences and budget."

[0099] User: "Do you have any special snacks to recommend?"

[0100] System: "There are many special snacks in Chengdu, such as Long Chao Shou, Zhong Dumplings, Dan Dan Noodles, etc. These snacks are delicious and will definitely satisfy your taste buds."

[0101] User: "What are some good places for family trips?"

[0102] System: "If you are traveling with your family, you can go to the Chengdu Research Base of Giant Panda Breeding, where children can see cute giant pandas. There is also the Chengdu Haichang Polar Ocean Park, which is also suitable for family travel. In addition, some hotels with parent-child theme rooms are also good choices."

[0103] User: "It sounds great. I decided to go to Chengdu."

[0104] System: "Great! I believe you will have a great time in Chengdu. Before traveling, you can book tickets for attractions and hotels on our platform, and there will be many discounts. If you have any other questions, feel free to contact me."

[0105] Among them, during the entire conversation process, the user's questions were proactive and showed a high interest and expectation for Chengdu tourism. There were more positive words and almost no negative words. The basic score of positive words was set to 2. "Great" and "Beautiful" were positive words, and the number of repetitions was small. The number of strong tone elements was also small. Substituting them into the sentiment tendency value expression, the sentiment tendency value was high, which was 0.71. It can be clearly seen that the user's initial demand intention was to query Chengdu's tourist attractions and food information. Then the user asked "How is the homestay?" The demand intention was extended to understand the accommodation situation in Chengdu. Then asked "What special snacks are recommended?" to further clarify the demand for Chengdu food, especially special snacks. Finally, asked "What are the good places for parent-child travel?" The demand intention was transformed into understanding information such as Chengdu attractions and accommodation suitable for parent-child travel. Based on the demand intention of each conversation content, different attention weight schemes were set for the sentence response model.

[0106] In another embodiment, the intelligent service dialogue system is applied in the field of e-commerce, and the specific dialogue content obtained is: User: "I want to buy a coat suitable for autumn, do you have any recommendations?"

[0107] System: "In autumn, a windbreaker is a very good choice. The classic khaki windbreaker is fashionable and versatile. There is also a denim jacket, which is very casual. If you want to keep warmer, you can consider a jacket, which has good windproof performance."

[0108] User: "How much does a windbreaker cost?"

[0109] System: "The price range of windbreakers is quite large. There are some brands with high cost performance, and the price is around [price range A]. If you are looking for quality and design, there are also some high-end brands of windbreakers, and the price may be in [price range B]. Our platform often has promotions, you can pay attention to them, you may be able to buy your favorite windbreaker at a more favorable price."

[0110] User: “Anything suitable for work?”

[0111] System: "Of course. Simple windbreakers are suitable for work. You can choose black, gray or dark blue, which are more stable. Paired with a shirt and trousers, it looks very capable. Or choose a slim denim jacket, which is also very good with a dress."

[0112] User: “What if the size doesn’t fit?”

[0113] System: "If the size is not suitable, you can apply for a return or exchange within the specified time. Our platform provides free return and exchange services, which is very convenient."

[0114] User: “Okay, I’ll take a look.”

[0115] System: "Okay, if you have any questions, feel free to ask me. I hope you can buy a satisfactory autumn coat."

[0116] Reference Figure 2 As shown, the present invention provides an intelligent service dialogue system integrating a generative big model and a knowledge graph, including:

[0117] S100 language model module, used to obtain a question-answering dataset, build a sentence reply model based on the BERT large language model, and train the sentence reply model using the question-answering dataset;

[0118] S200 data extraction module, used to scan professional databases to generate candidate item sets, calculate weighted support of the candidate item sets, divide them based on the weighted support, and obtain frequent item sets and infrequent item sets;

[0119] S300 knowledge graph module, used to generate association rules according to the frequent itemsets, generate a first knowledge graph based on the association rules and the frequent itemsets, and generate a second knowledge graph based on the infrequent itemsets;

[0120] S400 hybrid search module, used to perform cross-search on the first knowledge graph and the second knowledge graph according to the user input sentence to generate a search path;

[0121] S500 personalized user module, used to determine the demand intention and emotional tendency value according to the user input sentence;

[0122] S600 sentence generation module, used for taking the sentiment tendency value as the dynamic weight value of a sentence reply model, the sentence reply model obtains a reply sentence according to the demand intention and the search path;

[0123] S700 habit perception module is used to build a habit function according to the context of continuous dialogue content, optimize the sentence response model through reinforcement learning based on the habit function, and continuously learn the user's habit characteristics.

[0124] In this embodiment, in step S10, specifically:

[0125] Preprocess professional databases, including text cleaning, word segmentation, and stop word removal;

[0126] The FP-Growth algorithm is used to scan the preprocessed professional database to generate an initial candidate item set, and a multi-word phrase is generated based on the initial candidate item set using the sliding window method to obtain a candidate item set;

[0127] Calculate the weighted support of each item in the candidate set, the expression is:

[0128]

[0129] Among them, C represents the candidate item set, c i is the i-th candidate in the candidate set C, w f (c i ), w p (c i ) and w s (c i ) are candidate items c i The word frequency weight, position weight and semantic importance weight of the word are represented by D, which is a dataset formed after preprocessing of the professional database. j is the jth specific record in the data set D, and m is the total number of records in the data set D;

[0130] The candidate items whose weighted support is in the top 30% in the candidate set are classified into frequent item sets, and the candidate items whose weighted support is lower than the top 30% are classified into infrequent item sets.

[0131] In this embodiment, in step S20, specifically:

[0132] Use the Apriori algorithm to remove redundant rules from frequent item sets and obtain association rules;

[0133] Add frequent items as nodes to the first knowledge graph, and establish edges between nodes according to association rules;

[0134] Add infrequent items as nodes to the second knowledge graph and establish edges between nodes;

[0135] The semantic similarity of the nodes is calculated by the Word2Vec algorithm, and the nodes whose language similarity between the first knowledge graph and the second knowledge graph is greater than or equal to 0.6 are connected;

[0136] Attributes are assigned to nodes of the first knowledge graph and the second knowledge graph according to the professional database.

[0137] In this embodiment, in step S30, the method of performing cross-search based on the first knowledge graph and the second knowledge graph to generate a search path is specifically as follows:

[0138] Using the key information input by the user as the starting point, initialize the search stack to orderly store the nodes to be visited, and push the starting node into the stack;

[0139] When searching, the nodes related to the key information are searched first before the nodes in the first knowledge graph. If the related nodes in the first knowledge graph are connected to the second knowledge graph, the corresponding nodes in the second knowledge graph are included in the search path. If the related nodes cannot be found in the first knowledge graph, the search is further conducted in the second knowledge graph.

[0140] During the entire retrieval process, the search stack continuously records the visited nodes and the edges connecting the nodes to form a retrieval path.

[0141] In this embodiment, in step S30, the method for determining the demand intention and the emotional tendency value according to the user input sentence is specifically as follows:

[0142] Obtain input sentences with annotated demand intent as training data, select a machine learning algorithm to build a model, train the model with the training data, obtain a demand intent recognition model, and determine the user's demand intent through the demand intent recognition model based on the sentence input by the user;

[0143] The sentiment tendency value is calculated based on the lexical sentiment elements and tone elements in the user input sentence. The expression is:

[0144]

[0145] Among them, P is the set of active words, For active vocabulary i The basic score, For active vocabulary i The number of repetitions in the user input sentence, f is the decay function, where Where A is the number of times the word appears repeatedly, which means to avoid too many repetitions that lead to too high scores when the word appears repeatedly, k is the number of strong tone elements in the user's input sentence, and t i is the intensity score of the i-th strong tone element, φ is the negativity increase coefficient. When the strong tone and positive words appear at the same time, the positivity is enhanced. N is the set of negative words. For negative words i The basic score, For negative words i The number of repetitions in the user input sentence, ψ is the negativity enhancement coefficient, when a strong tone and negative words appear at the same time, the negativity level is enhanced.

[0146] In this embodiment, in step S40, the method of building a sentence reply model based on the BERT large language model and training the sentence reply model using the question-answering dataset is specifically as follows:

[0147] Get the BERT-base model from the pre-trained model library, add a fully connected layer on top of the BERT-base model as the output layer, introduce the Transformer decoder, and use the output of the fully connected layer as the input of the Transformer decoder;

[0148] Divide the training process into multiple batches, each batch contains a specific number of question-answer pairs;

[0149] During the training process of each batch, the sentence reply model is used to predict the question, the loss function is calculated, and the loss function is back-propagated through the back-propagation algorithm to update the model parameters, where the loss function is expressed as:

[0150]

[0151] Where N is the number of question-answer pairs in a batch, T is the number of dialogue rounds or time steps, λ is the weight coefficient for balancing the importance of the loss term, and y i，t The one-hot encoding of the true label at time step t for the i-th question answer, is the predicted label for the answer to the ith question at time step t, τ is the smoothness control parameter, The predicted label for the ith question answer pair at time step t-1.

[0152] In this embodiment, in step S40, the method of using the sentiment tendency value as the dynamic weight value of the sentence reply model is specifically: before the softmax function of the output layer of the sentence reply model, the sentiment tendency value is normalized from 0 to 1 and then added to the output vector of the sentence reply model as the dynamic weight value.

[0153] In this embodiment, in step S40, the method for the sentence reply model to obtain a reply sentence according to the demand intention and the search path is specifically as follows:

[0154] Based on the demand intention, the attention layer is introduced into the language response model to obtain the attention weight, which is expressed as:

[0155]

[0156] Among them, Q is the demand intention vector set, K is the key vector set, V is the value vector set, K i is the i-th key vector in the key vector set, d k is the dimension of the key vector, n is the total number of key vectors in the key vector set, K j is the jth key vector in the key vector set, V i is the i-th value vector in the value vector set;

[0157] The attention weights are weighted summed with the output of the middle layer of the language response model to obtain the attention-adjusted output, and the output is used as the input of the subsequent layer of the language response model;

[0158] The search results are obtained based on the search path, and the search results are provided as input to the optimized sentence reply model to output the reply sentence.

[0159] In this embodiment, in step S50, specifically:

[0160] Obtain all text data between the user and the intelligent service dialogue system and extract features, including lexical features, grammatical features and semantic features;

[0161] Use TF-IDF to convert features into feature vectors as conversation samples;

[0162] The habit function is constructed based on the dialogue samples. The expression of the habit function is:

[0163]

[0164] Among them, d i is the i-th dialogue sample, D is the number of dialogue samples, α is the weight coefficient of the difference between typical habit features and overall features, is d i The center vector of the cluster is the typical habit feature, C is d i The overall mean vector of the cluster is the overall feature, and β is d i The weight coefficient of the feature dimension between the reference vector C, m is the number of feature dimensions, v ik is the kth eigenvalue of the ith dialogue sample, C kis the kth eigenvalue of the reference vector C, 6 is the weight coefficient of the reference vector, v i is the feature vector of the i-th dialogue sample, C is the reference vector, ∈k is the adjustment factor of the feature dimension importance, is the kth eigenvalue of the typical habit feature;

[0165] Taking the habit function as the state input of reinforcement learning, the update formula of the state action value function is:

[0166]

[0167] Among them, Q represents the state action value function, H(d i ) is the habit function, a is the action, ← is the update operation, α is the learning rate, R is the reward function, γ is the discount factor, H(d i+1 ) is the habit function in the next state, a ξ is the set of all possible actions, It is expressed as the habit function value is H(d i+1 ), the set of all possible actions a ξ The maximum value of the state action-value function in .

[0168] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. An intelligent service dialogue method integrating a generative big model and a knowledge graph, characterized in that: include: S10: scanning a professional database to generate a candidate item set, calculating weighted support of items in the candidate item set, dividing the candidate item set based on the weighted support, and obtaining frequent item sets and infrequent item sets; S20 generates association rules according to the frequent itemsets, generates a first knowledge graph based on the association rules and the frequent itemsets, and generates a second knowledge graph based on the infrequent itemsets; S30 obtains a user input statement, performs a cross search based on the first knowledge graph and the second knowledge graph, and generates a search path; determines the demand intention and the emotional tendency value according to the user input statement; S40 builds a sentence response model based on the BERT large language model and uses the question-answering dataset to train the sentence response model; The sentiment tendency value is used as a dynamic weight value of a sentence reply model, and the sentence reply model obtains a reply sentence according to the demand intention and the search path; S50 constructs a habit function according to the context of the continuous conversation content, optimizes the sentence response model through reinforcement learning based on the habit function, and continuously learns the user's habit characteristics.

2. The intelligent service dialogue method integrating the generative big model and the knowledge graph according to claim 1 is characterized in that: In step S10, specifically: Preprocess professional databases, including text cleaning, word segmentation, and stop word removal; The FP-Growth algorithm is used to scan the preprocessed professional database to generate an initial candidate item set, and a multi-word phrase is generated based on the initial candidate item set using the sliding window method to obtain a candidate item set; Calculate the weighted support of each item in the candidate set, the expression is: Among them, C represents the candidate item set, c i is the i-th candidate in the candidate set C, w f (c i ), w p (c i ) and w s (c i ) are candidate c i The word frequency weight, position weight and semantic importance weight of the word are represented by D, which is a dataset formed after preprocessing of the professional database. j is the jth specific record in the data set D, and m is the total number of records in the data set D; The candidate items whose weighted support is in the top 30% in the candidate set are classified into frequent item sets, and the candidate items whose weighted support is lower than the top 30% are classified into infrequent item sets.

3. The intelligent service dialogue method integrating the generative big model and the knowledge graph according to claim 1 is characterized in that: In step S20, specifically: Use the Apriori algorithm to remove redundant rules from frequent item sets and obtain association rules; Add frequent items as nodes to the first knowledge graph, and establish edges between nodes according to association rules; Add infrequent items as nodes to the second knowledge graph and establish edges between nodes; The semantic similarity of the nodes is calculated by the Word2Vec algorithm, and the nodes whose language similarity between the first knowledge graph and the second knowledge graph is greater than or equal to 0.6 are connected; Attributes are assigned to nodes of the first knowledge graph and the second knowledge graph according to the professional database.

4. The intelligent service dialogue method integrating the generative big model and the knowledge graph according to claim 1 is characterized in that: In step S30, the method of performing cross-search based on the first knowledge graph and the second knowledge graph to generate a search path is specifically as follows: Using the key information input by the user as the starting point, initialize the search stack to orderly store the nodes to be visited, and push the starting node into the stack; When searching, the nodes related to the key information are searched first before the nodes in the first knowledge graph. If the related nodes in the first knowledge graph are connected to the second knowledge graph, the corresponding nodes in the second knowledge graph are included in the search path. If the related nodes cannot be found in the first knowledge graph, the search is further conducted in the second knowledge graph. During the entire retrieval process, the search stack continuously records the visited nodes and the edges connecting the nodes to form a retrieval path.

5. The intelligent service dialogue method integrating the generative big model and the knowledge graph according to claim 1 is characterized in that: In step S30, the method for determining the demand intention and the emotional tendency value according to the user input sentence is specifically as follows: Obtain input sentences with annotated demand intent as training data, select a machine learning algorithm to build a model, train the model with the training data, obtain a demand intent recognition model, and determine the user's demand intent through the demand intent recognition model based on the sentence input by the user; The sentiment tendency value is calculated based on the lexical sentiment elements and tone elements in the user input sentence. The expression is: Among them, P is the set of active words, For active vocabulary i The base score, n pi For active vocabulary i The number of repetitions in the user input sentence, f is the decay function, where Where A is the number of times the word appears repeatedly, which means to avoid too many repetitions that lead to too high scores when the word appears repeatedly, k is the number of strong tone elements in the user's input sentence, and t i is the intensity score of the i-th strong tone element, φ is the negativity increase coefficient. When the strong tone and positive words appear at the same time, the positivity is enhanced. N is the set of negative words. For negative words i The basic score, For negative words i The number of repetitions in the user input sentence, ψ is the negativity enhancement coefficient, when a strong tone and negative words appear at the same time, the negativity level is enhanced.

6. The intelligent service dialogue method integrating the generative big model and the knowledge graph according to claim 1 is characterized in that: In step S40, the method of building a sentence reply model based on the BERT large language model and training the sentence reply model using the question-answering dataset is specifically as follows: Get the BERT-base model from the pre-trained model library, add a fully connected layer on top of the BERT-base model as the output layer, introduce the Transformer decoder, and use the output of the fully connected layer as the input of the Transformer decoder; Divide the training process into multiple batches, each batch contains a specific number of question-answer pairs; During the training process of each batch, the sentence reply model is used to predict the question, the loss function is calculated, and the loss function is back-propagated through the back-propagation algorithm to update the model parameters, where the loss function is expressed as: Where N is the number of question-answer pairs in a batch, T is the number of dialogue rounds or time steps, λ is the weight coefficient for balancing the importance of the loss term, and y i,t The one-hot encoding of the true label at time step t for the i-th question answer, is the predicted label for the answer to the ith question at time step t, τ is the smoothness control parameter, The predicted label for the ith question answer pair at time step t-1.

7. The intelligent service dialogue method integrating the generative big model and the knowledge graph according to claim 1 is characterized in that: In step S40, the method of using the sentiment tendency value as the dynamic weight value of the sentence reply model is specifically as follows: before the softmax function of the output layer of the sentence reply model, the sentiment tendency value is normalized from 0 to 1 and then added to the output vector of the sentence reply model as the dynamic weight value.

8. The intelligent service dialogue method integrating the generative big model and the knowledge graph according to claim 1 is characterized in that: In step S40, the sentence reply model obtains a reply sentence according to the demand intention and the search path, specifically: Based on the demand intention, the attention layer is introduced into the language response model to obtain the attention weight, which is expressed as: Among them, Q is the demand intention vector set, K is the key vector set, V is the value vector set, K i is the i-th key vector in the key vector set, d k is the dimension of the key vector, n is the total number of key vectors in the key vector set, K j is the jth key vector in the key vector set, V i is the i-th value vector in the value vector set; The attention weights are weighted summed with the output of the middle layer of the language response model to obtain the attention-adjusted output, and the output is used as the input of the subsequent layer of the language response model; The search results are obtained based on the search path, and the search results are provided as input to the optimized sentence reply model to output the reply sentence.

9. The intelligent service dialogue method integrating a generative big model and a knowledge graph according to claim 1 is characterized in that: In step S50, specifically: Obtain all text data between the user and the intelligent service dialogue system and extract features, including lexical features, grammatical features and semantic features; Use TF-IDF to convert features into feature vectors as conversation samples; The habit function is constructed based on the dialogue samples. The expression of the habit function is: Among them, d i is the i-th dialogue sample, D is the number of dialogue samples, α is the weight coefficient of the difference between typical habit features and overall features, C j* is d i The center vector of the cluster is the typical habit feature, C is d i The overall mean vector of the cluster is the overall feature, and β is d i The weight coefficient of the feature dimension between the reference vector C, m is the number of feature dimensions, v ik is the kth eigenvalue of the ith dialogue sample, C k is the kth eigenvalue of the reference vector C, δ is the weight coefficient of the reference vector, v i is the feature vector of the i-th dialogue sample, C is the reference vector, ∈ k is the adjustment factor of the feature dimension importance, C j*k is the kth eigenvalue of the typical habit feature; Taking the habit function as the state input of reinforcement learning, the update formula of the state action value function is: Among them, Q represents the state action value function, H(d i ) is the habit function, a is the action, ← is the update operation, α is the learning rate, R is the reward function, γ is the discount factor, H(d i+1 ) is the habit function in the next state, a ξ is the set of all possible actions, max aξ Q(H(d i+1 ),a ξ ) is expressed as the habit function value H(d i+1 ), the set of all possible actions a ξ The maximum value of the state action-value function in .

10. An intelligent service dialogue system integrating generative big models and knowledge graphs, characterized by: The system comprises: A language model module is used to obtain a question-answering dataset, build a sentence answering model based on the BERT large language model, and train the sentence answering model using the question-answering dataset; A data extraction module is used to scan a professional database to generate a candidate item set, calculate the weighted support of the candidate item set, and divide it based on the weighted support to obtain frequent item sets and infrequent item sets; A knowledge graph module, configured to generate association rules according to the frequent itemsets, generate a first knowledge graph based on the association rules and the frequent itemsets, and generate a second knowledge graph based on the infrequent itemsets; A hybrid retrieval module, used to perform a cross-search on the first knowledge graph and the second knowledge graph according to a user input statement to generate a retrieval path; A personalized user module, used to determine the demand intention and emotional tendency value according to the user input sentence; A sentence generation module, used to use the sentiment tendency value as a dynamic weight value of a sentence reply model, and the sentence reply model obtains a reply sentence according to the demand intention and the search path; The habit perception module is used to build a habit function according to the context of continuous dialogue content, optimize the sentence response model through reinforcement learning based on the habit function, and continuously learn the user's habit characteristics.

Images