User consultation information automatic reply method and system based on natural language

Abstract

The invention relates to the technical field of big data processing, and discloses a natural language-based user consultation information automatic reply method and system, and the method comprises the steps: obtaining question information inputted by a user; inputting the question information into a semantic extraction model to obtain keyword features and user features; inputting the question information and the historical dialogue data into a context analysis model to obtain question scene features; obtaining question feature vectors through a vectorization method according to the keyword features, the user features and the question scene features; inputting the question feature vector into a graph reasoning model to obtain a predicted reply result; and automatically replying question information input by the user according to the predicted reply result. According to the method, the natural language analysis precision can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of big data processing, and in particular to a method and system for automatically replying to user consultation information based on natural language. Background Art

[0002] With the development of big data and Internet of Things technologies and the in-depth study of natural language understanding, artificial intelligence technology has been applied to different degrees in natural language processing fields such as intelligent chat systems, speech recognition, and machine translation. In recent years, under the influence of the "Internet +" and industrial Internet concepts, more and more companies and research institutions are committed to the combination of artificial intelligence and industry, hoping to use artificial intelligence technology to enhance the competitiveness of enterprises in "Internet +" and industrial Internet. For example, the intelligent consulting system using natural language processing technology can provide consulting services to enterprise R&D and management personnel on scientific and technological innovation and management decision-making. By combining the Internet of Things with speech recognition, production efficiency can be improved and workers can be assisted in on-site operations.

[0003] In one existing technology, traditional semantic analysis technology generally extracts semantic features through deep learning models when parsing natural language. At the same time, as the dimension of feature information increases, the interpretability of features deteriorates, the structural depth of deep learning models increases, the parameter scale becomes larger, the reasoning calculation becomes complex, the model transferability is poor, and the training efficiency is low. At the same time, traditional semantic analysis technology is difficult to integrate the rules, formulas and other characteristics of professional fields, and its application is limited.

[0004] Due to the diverse forms of natural language expression and complex logic, similar expressions are composed of different language components. When extracting semantic features, traditional vector word tables or word embedding technologies are difficult to reflect these differences, which leads to reduced analysis accuracy. In addition, when parsing natural language, traditional semantic analysis technology generally extracts semantic features through deep learning models. However, as the dimension of feature information increases, the interpretability of features deteriorates, the structural depth of deep learning models increases, the parameter scale becomes larger, the reasoning calculation becomes complex, the model transferability is poor, and the training efficiency is low.

[0005] In summary, the existing technology has the problem of low accuracy in natural language analysis, which leads to low accuracy in automatic replies to user inquiry information. Summary of the invention

[0006] The present invention provides a method and system for automatically replying to user consultation information based on natural language, so as to improve the analysis accuracy of natural language, thereby improving the accuracy of automatically replying to user consultation information.

[0007] In a first aspect, in order to solve the above technical problems, the present invention provides a method for automatically replying to user consultation information based on natural language, comprising:

[0008] Get the question information entered by the user;

[0009] Inputting the question information into a semantic extraction model to obtain keyword features and user features;

[0010] Inputting the question information and historical conversation data into a context parsing model to obtain question scene features;

[0011] Obtaining a question feature vector by a vectorization method according to the keyword feature, the user feature and the question scene feature;

[0012] Inputting the question feature vector into the graph reasoning model to obtain a predicted response result;

[0013] According to the predicted reply result, the question information input by the user is automatically replied.

[0014] In an optional implementation, the step of inputting the question information into a semantic extraction model to obtain keyword features and user features includes:

[0015] Cleaning the question information to obtain pure question information;

[0016] Inputting the pure question information into a semantic extraction model, and outputting keyword features and user features;

[0017] The training process of the semantic extraction model includes:

[0018] A training set is constructed based on historical natural language data to train the semantic extraction model.

[0019] In an optional implementation, the step of inputting the question information and the historical dialogue data into a context analysis model to obtain question scene features includes:

[0020] Inputting the question information and historical conversation data into a context parsing model to obtain question scene features, wherein the question scene features include time and place;

[0021] The training process of the context parsing model includes:

[0022] The historical conversation data is used as the training set, and the cross entropy loss is used as the loss function to train the context parsing model.

[0023] In an optional implementation, obtaining the question feature vector by a vectorization method according to the keyword feature, the user feature and the question scene feature includes:

[0024] Input the keyword features into a pre-trained word vector model, and output a keyword vector;

[0025] Performing one-hot encoding on the user features and the question scene features to obtain a user feature vector and a question scene feature vector;

[0026] The keyword vector, user feature vector and question scene feature vector are passed through a fully connected layer neural network to output a question feature vector.

[0027] In an optional implementation, inputting the question feature vector into a graph reasoning model to obtain a predicted response result includes:

[0028] The training process of the graph reasoning model includes:

[0029] Train graph reasoning models based on knowledge graphs;

[0030] Select the following loss function to train the graph reasoning model:

[0031]

[0032] Among them, L is the total loss function; D is the training data set, which contains all positive sample pairs (y i , p i ) and negative sample pairs yi represents the predicted entity vector; pi represents the number of paths between the head entity and the tail entity; R1(y i ) and R2(y i ) represent the entity vector y i Projection in two different relational spaces; represents the negative sample of the entity vector yi; K is the number of relationship types; and Represents the negative sample entity vectors Projection in two different relational spaces; Represents the transpose of the entity vector yi.

[0033] In an optional implementation, the process of constructing the knowledge graph includes:

[0034] Obtain historical user consultation information;

[0035] Constructing a historical corpus of the historical user consultation information by using a statistical method;

[0036] The knowledge graph structure is constructed using keyword features, user features or scene features in the historical corpus as nodes and reasoning relationships as edges.

[0037] In an optional implementation, after automatically replying to the question information input by the user according to the predicted reply result, the method further includes:

[0038] Obtain user feedback results;

[0039] Optimizing the knowledge graph according to the user feedback results;

[0040] A generative model building mechanism is used to optimize the graph reasoning model.

[0041] In a second aspect, the present invention provides a natural language-based automatic reply system for user consultation information, comprising:

[0042] A data acquisition module is used to obtain question information input by the user;

[0043] A semantic extraction module, used for inputting the question information into a semantic extraction model to obtain keyword features and user features;

[0044] A scene extraction module, used to input the question information and historical dialogue data into a context analysis model to obtain question scene features;

[0045] A feature fusion module, used to obtain a question feature vector by a vectorization method according to the keyword feature, the user feature and the question scene feature;

[0046] A result prediction module, used for inputting the question feature vector into the graph reasoning model to obtain a predicted reply result;

[0047] The reply module is used to automatically reply to the question information input by the user according to the predicted reply result.

[0048] In a third aspect, the present invention also provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements any one of the above-mentioned methods for automatically replying to user consultation information based on natural language.

[0049] In a fourth aspect, the present invention also provides a computer-readable storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned natural language-based automatic reply methods for user consultation information.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] The present invention relates to the technical field of big data processing, and discloses a method and system for automatically replying to user consultation information based on natural language. The method comprises the following steps: obtaining question information input by a user; inputting the question information into a semantic extraction model to obtain keyword features and user features; inputting the question information and historical conversation data into a context analysis model to obtain question scene features; obtaining a question feature vector by a vectorization method according to the keyword features, the user features and the question scene features; inputting the question feature vector into a graph reasoning model to obtain a predicted reply result; and automatically replying to the question information input by the user according to the predicted reply result. The method extracts keyword features and user features from the question through a semantic extraction model, and can more accurately understand the user's intention. Combined with historical conversation data, the question scene features are obtained, so that the system can better understand the context of the current question, thereby generating a more relevant reply. Using a graph reasoning model for prediction, complex reasoning tasks can be efficiently processed on a large-scale data set. Furthermore, the method can continuously accumulate a knowledge base and optimize model performance by collecting and analyzing user consultation data, and user feedback can be used to improve the model to form a closed-loop optimization process. In summary, this method can improve the accuracy of natural language analysis, thereby improving the accuracy of automatic replies to user consultation information. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flowchart of a method for automatically replying to user consultation information based on natural language provided by the first embodiment of the present invention;

[0053] Figure 2 It is a schematic diagram of the structure of a natural language-based automatic reply system for user consultation information provided by the second embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] With the development of big data and Internet of Things technologies and the in-depth study of natural language understanding, artificial intelligence technology has been applied to different degrees in natural language processing fields such as intelligent chat systems, speech recognition, and machine translation. In recent years, under the influence of the "Internet +" and industrial Internet concepts, more and more companies and research institutions are committed to the combination of artificial intelligence and industry, hoping to use artificial intelligence technology to enhance the competitiveness of enterprises in "Internet +" and industrial Internet. For example, the intelligent consulting system using natural language processing technology can provide consulting services to enterprise R&D and management personnel on scientific and technological innovation and management decision-making. By combining the Internet of Things with speech recognition, production efficiency can be improved and workers can be assisted in on-site operations.

[0056] In one existing technology, traditional semantic analysis technology generally extracts semantic features through deep learning models when parsing natural language. At the same time, as the dimension of feature information increases, the interpretability of features deteriorates, the structural depth of deep learning models increases, the parameter scale becomes larger, the reasoning calculation becomes complex, the model transferability is poor, and the training efficiency is low. At the same time, traditional semantic analysis technology is difficult to integrate the rules, formulas and other characteristics of professional fields, and its application is limited.

[0057] Since natural language has diverse expressions and complex logic, and similar expressions are composed of different language components, it is difficult for traditional vector word tables or word embedding technologies to reflect these differences when extracting semantic features, which leads to reduced analysis accuracy. In addition, when parsing natural language, traditional semantic analysis technology generally extracts semantic features through deep learning models. However, as the dimension of feature information increases, the interpretability of features deteriorates, the structural depth of deep learning models increases, the parameter scale becomes larger, the reasoning calculation becomes complex, the model transferability is poor, and the training efficiency is low. The analysis accuracy of natural language is low.

[0058] To solve the above problems, refer to Figure 1 The first embodiment of the present invention provides a method for automatically replying to user consultation information based on natural language, comprising the following steps:

[0059] S11, obtaining question information input by the user;

[0060] S12, inputting the question information into a semantic extraction model to obtain keyword features and user features;

[0061] S13, inputting the question information and historical conversation data into a context analysis model to obtain question scene features;

[0062] S14, obtaining a question feature vector by a vectorization method according to the keyword feature, the user feature and the question scenario feature;

[0063] S15, inputting the question feature vector into the graph reasoning model to obtain a predicted response result;

[0064] S16: Automatically reply to the question information input by the user according to the predicted reply result.

[0065] In step S11, question information input by the user is acquired.

[0066] In one implementation, the question information input by the user is obtained through the API interface, and the question information is in text format. For the user's voice input, a microphone is used as the audio source, and the Google Web Speech API is used for voice recognition. The natural language model is called for processing to obtain the text content of the question information. For example: "Where can I find a security guard?"

[0067] In another implementation, IBM Watson Speech to Text can be used, which has excellent transcription capabilities and supports multiple languages ​​and dialects, making it very suitable for enterprises that need to deal with multilingual environments. Microsoft's Azure Speech service is known for its comprehensive range of services, which not only includes speech-to-text functions, but also covers text-to-speech and speech translation, providing users with a one-stop voice solution. Amazon's Transcribe service is particularly suitable for processing large-scale voice data, such as call center recordings or meeting minutes, in terms of real-time and batch transcription. Baidu's speech recognition service focuses on the Chinese market and provides accurate Chinese speech recognition capabilities, which is very suitable for application development in the Chinese context. These technologies convert speech into actionable text data.

[0068] In step S12, the question information is input into a semantic extraction model to obtain keyword features and user features.

[0069] In one embodiment, the question information is cleaned to obtain pure question information; the pure question information is input into a semantic extraction model to output keyword features and user features; wherein the training process of the semantic extraction model includes: constructing a training set based on historical natural language data to train the semantic extraction model.

[0070] It should be noted that the purpose of cleaning the question information is to remove irrelevant information in the question, such as punctuation marks, stop words, etc., in order to simplify the question and reduce the complexity of subsequent processing. This includes commas, periods, exclamation marks, question marks, semicolons, colons, etc., which usually do not carry semantic information. In natural language processing, stop words, such as "de", "shi", "zai", etc., which do not carry important information, will be removed. Since the question information is extracted through voice input, situations such as stop words and stuttering may be recognized. For example, repeated words in the question, such as "today today" in "What's the weather like today today?". The clean-text in the python library can be used for data cleaning.

[0071] It should be noted that in the cleaned question, the semantics of the question are understood through word segmentation and part-of-speech tagging. Word segmentation is to split continuous natural language text into independent word units, and part-of-speech tagging is to label the grammatical function of each word. In addition, named entity recognition and sentiment analysis are also part of feature extraction. Named entity recognition (NER) can identify specific entities in the sentence, such as person names, place names or organization names, while sentiment analysis evaluates the user's emotional tendency as the content of subsequent user features. For different mood states, different reply modes should be adopted. For example: For a烦躁 user, it is necessary to comfort them when replying. For example: For the comment "The battery life of this mobile phone is really bad, and I am very dissatisfied.", sentiment analysis will identify negative sentiment and extract "The battery life is really bad" as the reason for dissatisfaction. In "The newly opened coffee shop is great. The barista is very friendly. Highly recommended!", sentiment analysis will identify positive sentiment and extract "The barista is very friendly" as the reason for recommendation. In the customer feedback "I waited for two hours to get a reply. This service is too slow.", sentiment analysis will identify negative sentiment and point out that "The service is too slow" is the area that needs improvement.

[0072] It should be noted that the training of the semantic extraction model is based on a training set constructed from historical natural language data. This training set contains a large number of questions and their corresponding semantic features, which are used to train the model to identify keyword features and user features in new questions. For example: For the question "What's the weather like in Beijing tomorrow?", feature vectors such as ("tomorrow", "Beijing", "weather") can be extracted, which are represented as (1, 1, 1) after numericalization. The training process can adopt the maximum entropy model, including: defining a feature function to describe the relationship between input features and output labels; setting constraint conditions; constructing an objective function; using the Lagrange multiplier method to solve the minimum value of the objective function to obtain model parameters.

[0073] In step S13, the question information and historical conversation data are input into the context parsing model to obtain question scenario features.

[0074] In one implementation, the question information and historical conversation data are input into a context parsing model to obtain question scene features, wherein the question scene features include time and place; wherein the training process of the context parsing model includes: using the historical conversation data as a training set, adopting the cross entropy loss as a loss function, and training to obtain the context parsing model.

[0075] It is worth noting that the historical conversation data is used as a training set. These data need to contain rich contextual information so that the model can learn the features in different scenarios. Before training, the data needs to be preprocessed, including cleaning, word segmentation, and removal of stop words, to improve the efficiency and effect of model training. For example, the processed data is given a label of "query weather" for the question "What is the weather like tomorrow" and a label of "book a flight" for the question "I want to book a flight to New York". The context parsing model can be selected as a recurrent neural network (RNN), which can capture complex patterns in sequence data and extract scene features. During the training process, the cross entropy loss function is usually used to optimize the model. The cross entropy loss function measures the difference between the probability distribution of the model prediction and the probability distribution of the true label. The model is trained using the training set, and the model parameters are optimized using the back propagation algorithm. During the training process, overfitting can be prevented and the generalization ability of the model can be improved by adjusting the learning rate and using regularization techniques. The trained model is evaluated using the validation set. Common evaluation indicators include accuracy, precision, recall, and F1 score. In addition, cross-validation methods can be used to evaluate the performance of the model to ensure that the model performs well on unseen data.

[0076] In one implementation, a BERT (Bidirectional Encoder Representations from Transformers) model is used as a context parsing model. BERT is a pre-trained language representation model based on the Transformer architecture. It learns deep bidirectional text representations by pre-training with a large amount of text data. The core of the BERT model is its ability to capture contextual information of words, phrases, and sentences. BERT is pre-trained on a large-scale corpus and usually includes two tasks: Masked Language Model (MLM) and Next Sentence Prediction (NSP). In the MLM task, the model randomly masks some words in the input sentence and tries to predict these masked words. In the NSP task, the model predicts whether two sentences are continuous texts. For example, the BERT model can be used to get the question "I ate a loaf of bread this morning and now I'm so hungry. What do you have here?" from a restaurant. That is, the scene feature contains a restaurant.

[0077] In step S14, a question feature vector is obtained by a vectorization method according to the keyword feature, the user feature and the question scenario feature.

[0078] In one implementation, the keyword feature is input into a pre-trained word vector model to output a keyword vector; the user feature and the question scene feature are one-hot encoded to obtain a user feature vector and a question scene feature vector; the keyword vector, user feature vector and question scene feature vector are passed through a fully connected layer neural network to output a question feature vector.

[0079] In one embodiment, Word2Vec is used as a word vector model to convert the input keyword features into keyword vectors. For example, for the keyword "where", a numerical vector will be output for subsequent semantic feature fusion. Suppose there is a user's feature set, including the user's gender (male, female) and age group (18-24, 25-34, 35-44, 45+). The get_dummies function in the pandas library can be used for unique hot encoding. Each user's features are converted into a set of binary features, of which only one feature is 1, indicating the existence of the feature, and the others are 0. In this way, user features and scene features can be effectively converted into numerical features that can be processed by machine learning models. User features include information such as the user's historical behavior and preferences.

[0080] Furthermore, after these features are digitized, they can be fused into a question feature vector through a fully connected layer neural network. The keyword vector, user feature vector, and question scene feature vector are merged to form a comprehensive feature representation of the question. The merged feature vector is input into a fully connected layer neural network. A fully connected layer is a layer in a neural network in which each neuron is connected to each neuron in the previous layer. Through the nonlinear transformation of the fully connected layer, the model can learn the complex relationship between features and output the final question feature vector. The final output question feature vector is a high-dimensional representation that integrates keyword semantics, user information, and scene context. Historical data is also used as a training set for neural network training.

[0081] In step S15, the question feature vector is input into the graph reasoning model to obtain a predicted reply result.

[0082] In one implementation, the training process of the graph reasoning model includes:

[0083] Train graph reasoning models based on knowledge graphs;

[0084] Select the following loss function to train the graph reasoning model:

[0085]

[0086] Among them, L is the total loss function; D is the training data set, which contains all positive sample pairs (y i , p i ) and negative sample pairs yi represents the predicted entity vector; pi represents the number of paths between the head entity and the tail entity; R1(y i ) and R2(y i ) represent the entity vector y i Projection in two different relational spaces; represents the negative sample of the entity vector yi; K is the number of relationship types; and Represents the negative sample entity vectors Projection in two different relational spaces; Represents the transpose of the entity vector yi.

[0087] It is worth noting that the graph reasoning model can use a graph convolutional network (GCN). This model can use graph structure information to learn the embedding representation of nodes and perform reasoning operations. Use the knowledge graph as training data to train the graph reasoning model. During the training process, the model learns how to reason in the knowledge graph based on the input question feature vector to generate accurate predicted response results. Define a loss function to measure the difference between the model prediction result and the true result. Common loss functions include cross entropy loss, mean square error loss (MSE) or contrast loss. Use gradient descent or other optimization algorithms to minimize the loss function to train the model. At the same time, regularization techniques can be used to prevent the model from overfitting. In graph reasoning tasks, contrastive loss is a common method that improves the performance of the model by maximizing the similarity between positive sample pairs and minimizing the similarity between negative sample pairs. The core idea of ​​contrastive loss is to enable the model to distinguish between positive sample pairs and negative sample pairs. Specifically: positive sample pairs are sample pairs with the same label or similar semantics; negative sample pairs are sample pairs with different labels or irrelevant semantics. By optimizing the contrast loss, the model can learn better node representations so that the representations of positive pairs are closer and the representations of negative pairs are farther apart.

[0088] In one embodiment, a graph convolutional network (GCN) is a deep learning model that is very suitable for processing graph structure data such as knowledge graphs. The form of the node in the graph structure data is (0, [1, 0, 0]), where 0 is the node number and [1, 0, 0] is the feature vector. The form of the edge is (0, 1) and represents its connection between node 0 and node 1. During the training process, the loss function and optimizer are defined by PyTorch, and each iteration is trained through forward propagation, back propagation and optimization. It updates the node features by aggregating the information of the node neighbors, effectively captures the complex relationship between entities, and learns insightful entity representations. GCN performs well in relational modeling, can understand the relationship between entities and is used for graph structure reasoning, such as link prediction and entity classification. This model structure is not only highly scalable and flexible, but also can fuse multi-source features and process variable-length input data. GCN has high training efficiency, can use the sparsity of graph data to reduce unnecessary calculations, and the trained model has good generalization ability and is suitable for a variety of downstream tasks. By optimizing the loss function, GCN can learn the high-order neighborhood information of nodes and output node representations suitable for knowledge graph reasoning, thereby improving the accuracy and efficiency of automatic reply systems or other related applications.

[0089] In one embodiment, the process of constructing the knowledge graph includes: obtaining historical user consultation information; constructing a historical corpus based on the historical user consultation information through statistical methods; and constructing a knowledge graph structure using keyword features, user features, or scene features in the historical corpus as nodes and reasoning relationships as edges.

[0090] It is worth noting that the construction of the knowledge graph requires the collection of a large amount of user consultation data. This data can come from customer service records, online question-and-answer platforms, user feedback, etc. For example, consulting companies may obtain information through channels such as expert meetings, paid data platforms, and free information platforms. Use statistical methods to process the collected user consultation information and construct a historical corpus. This corpus will contain a large number of questions and their corresponding semantic features, providing data support for training semantic extraction models. The construction of the corpus can be derived through probabilistic statistical methods using the methods of corpus linguistics. Identify keyword features, user features, or scene features as nodes in the corpus. Entity recognition can be accomplished through named entity recognition (NER) technology. Determine the relationship between entities, which are represented as edges in the knowledge graph. Relationship extraction can be achieved through rule-based methods, machine learning methods, and deep learning methods. Use the identified keyword features, user features, or scene features as nodes and the reasoning relationship as edges to construct the knowledge graph structure. This process can use automated ontology construction technology, including entity parallel relationship similarity calculation, entity hyponymy extraction, and ontology generation. After the knowledge graph is built, knowledge reasoning technology is used to complete the missing relationships or attributes to further improve the knowledge graph. Reasoning can be based on logical rules, graph structure, distributed representation learning, neural network, and hybrid reasoning.

[0091] It is worth noting that the key to ensuring the accuracy of knowledge graph data lies in taking a series of comprehensive measures. Select authoritative and reliable data sources and conduct thorough data cleaning, including removing duplicate records and correcting errors. Through named entity recognition and disambiguation technology, entities can be accurately identified and distinguished, and the assistance of machine learning models can improve the accuracy of relationship extraction. Furthermore, regular data verification, auditing, and user feedback mechanisms can discover and correct errors. Implementing version control and change management helps to record and review each data update, while continuous monitoring and expert review ensure that data quality meets standards. Using high-quality ontologies and automated reasoning technology can not only guide the organization of data, but also discover and correct inconsistencies in the data.

[0092] In step S16, an automatic reply is given to the question information input by the user according to the predicted reply result.

[0093] In one embodiment, after automatically replying to the question information input by the user according to the predicted reply result, it also includes: obtaining user feedback results; optimizing the knowledge graph according to the user feedback results; and optimizing the graph reasoning model using a generative model building mechanism.

[0094] It is worth noting that the feature vector of the question information input by the user is input into the graph reasoning model, the model performs reasoning based on the knowledge graph, and outputs the predicted reply result. The system automatically sends the predicted reply result to the user as a response to the user's question. After sending the automatic reply, the system needs to collect user feedback. This can be achieved in a variety of ways, such as: the user directly evaluates the relevance or satisfaction of the reply. The system monitors whether the user continues to ask questions or whether manual intervention is required. Based on the user's feedback results, the system can identify deficiencies in the knowledge graph, such as missing information, wrong associations, or inaccurate reasoning paths. Using this feedback information, the system can update and optimize the knowledge graph, add new entities and relationships, and correct wrong information, thereby improving the accuracy and reliability of the graph reasoning model. This process can be implemented through automated scripts or combined with manual review to ensure the accuracy of the update. The graph reasoning model is optimized using a generative model building mechanism. Generative models, such as generative adversarial networks (GANs) or variational autoencoders (VAEs), can be used to generate new training data or enhance existing data. These generative models can help the system generate similar data through simulation in the absence of certain types of data, thereby improving the generalization ability of the model.

[0095] In summary, the present invention proposes an automatic reply method and system based on natural language processing, aiming to improve the accuracy and efficiency of responding to user consultation information. This method converts the question information input by the user into vectorized features by constructing a knowledge graph, combining a semantic extraction model and a context parsing model, and uses a graph reasoning model for in-depth analysis, thereby obtaining a predicted reply result and realizing automatic reply. Specifically, the method first classifies and parses the user's question, extracts user information features, consultation information features, and knowledge dimension and formula dimension features. User information features include user's personal information, such as age, gender, etc., consultation information features cover the specific content of user questions, and knowledge dimension and formula dimension features involve professional knowledge and formula algorithms that may be involved in user consultation. In the feature extraction process, the method uses a pre-trained word vector model (such as Word2Vec, GloVe or FastText) to convert keywords into fixed-length vector representations. These word vectors can capture the semantic relationship between words, so as to better understand the content of the question. In addition, for classification features (such as gender), one-hot encoding is used to convert them into numerical form. For continuous features (such as age), standardization techniques (such as Z-score standardization) are used for processing. In this way, different types of features are effectively converted into numerical vectors that can be used for machine learning. At the same time, the above features are integrated into a comprehensive question feature vector through vectorization methods. This process not only considers the independence of individual features, but also focuses on the interaction between features. For example, the association between user age and consultation content will have an important impact on the final reply. Therefore, when constructing the question feature vector, special attention will be paid to the fusion method between features to ensure that each feature can be fully expressed. After obtaining the question feature vector, this method uses a graph embedding model to determine the fusion dimension of question information and corpus information. The graph embedding model is a powerful tool that can map nodes (i.e., question features) in a graph structure to a low-dimensional space while preserving the topological relationship between nodes. This technology enables the system to efficiently reason in complex graph structures. Through the graph embedding model, the connection between question features and related information in historical data can be determined, thereby improving the accuracy and relevance of the reply. Based on historical user consultation data and the preset information reply model, the system performs graph reasoning operations. The graph reasoning model gradually derives the best response result by passing messages and updating nodes and edges in the graph structure. In this process, the system calculates the probability of each predicted response and selects the output with the highest probability as the final response. This probability-based approach can ensure the quality and accuracy of the response, and can provide satisfactory answers even in the face of complex and ever-changing user inquiries. Furthermore, the present invention also takes user feedback into special consideration. User feedback is an important way to improve system performance.By collecting user feedback information, the system can continuously optimize the knowledge graph and graph reasoning model. In summary, the automatic reply method and system based on natural language processing proposed in the present invention realizes efficient user consultation information processing by constructing a knowledge graph and combining a semantic extraction model and a context parsing model. This method can improve the accuracy of natural language semantic analysis.

[0096] Reference Figure 2 The second embodiment of the present invention provides a natural language-based user consultation information automatic reply system, comprising:

[0097] A data acquisition module is used to obtain question information input by the user;

[0098] A semantic extraction module, used for inputting the question information into a semantic extraction model to obtain keyword features and user features;

[0099] A scene extraction module, used to input the question information and historical dialogue data into a context analysis model to obtain question scene features;

[0100] A feature fusion module, used to obtain a question feature vector by a vectorization method according to the keyword feature, the user feature and the question scene feature;

[0101] A result prediction module, used for inputting the question feature vector into the graph reasoning model to obtain a predicted reply result;

[0102] The reply module is used to automatically reply to the question information input by the user according to the predicted reply result.

[0103] Preferably, the data acquisition module is used to:

[0104] Get the question information entered by the user.

[0105] In one implementation, the question information input by the user is obtained through the API interface, and the question information is in text format. For the user's voice input, a microphone is used as the audio source, and the Google Web Speech API is used for voice recognition. The natural language model is called for processing to obtain the text content of the question information. For example: "Where can I find a security guard?"

[0106] Preferably, the semantic extraction module is used to:

[0107] The question information is input into the semantic extraction model to obtain keyword features and user features, including:

[0108] Cleaning the question information to obtain pure question information;

[0109] Inputting the pure question information into a semantic extraction model, and outputting keyword features and user features;

[0110] The training process of the semantic extraction model includes:

[0111] A training set is constructed based on historical natural language data to train the semantic extraction model.

[0112] Preferably, the scene extraction module is used to:

[0113] The question information and historical conversation data are input into the context parsing model to obtain question scene features, including:

[0114] Inputting the question information and historical conversation data into a context parsing model to obtain question scene features, wherein the question scene features include time and place;

[0115] The training process of the context parsing model includes:

[0116] The historical conversation data is used as the training set, and the cross entropy loss is used as the loss function to train the context parsing model.

[0117] Preferably, the feature fusion module is used to:

[0118] According to the keyword feature, the user feature and the question scene feature, a question feature vector is obtained by a vectorization method, including:

[0119] Input the keyword features into a pre-trained word vector model, and output a keyword vector;

[0120] Performing one-hot encoding on the user features and the question scene features to obtain a user feature vector and a question scene feature vector;

[0121] The keyword vector, user feature vector and question scene feature vector are passed through a fully connected layer neural network to output a question feature vector.

[0122] Preferably, the result prediction module is used to:

[0123] Inputting the question feature vector into the graph reasoning model to obtain a predicted response result includes:

[0124] The training process of the graph reasoning model includes:

[0125] Train graph reasoning models based on knowledge graphs;

[0126] Select the following loss function to train the graph reasoning model:

[0127]

[0128] Among them, L is the total loss function; D is the training data set, which contains all positive sample pairs (y i , p i ) and negative sample pairs y i represents the predicted entity vector; p i Indicates the number of paths between the head entity and the tail entity; R1(y i ) and R2(y i ) represent the entity vector y i Projection in two different relational spaces; Represents the entity vector y i Negative samples; K is the number of relationship types; and Represents the negative sample entity vectors Projection in two different relational spaces; Represents the entity vector y i The transpose of .

[0129] Preferably, the reply module is used to:

[0130] According to the predicted reply result, the question information input by the user is automatically replied.

[0131] Preferably, after automatically replying to the question information input by the user according to the predicted reply result, the method further includes: obtaining user feedback results; optimizing the knowledge graph according to the user feedback results; and optimizing the graph reasoning model using a generative model building mechanism.

[0132] It should be noted that the natural language-based automatic reply system for user consultation information provided in an embodiment of the present invention is used to execute all the process steps of the natural language-based automatic reply method for user consultation information in the above embodiment. The working principles and beneficial effects of the two correspond one to one, and therefore will not be repeated here.

[0133] The embodiment of the present invention further provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a data acquisition program. When the processor executes the computer program, the steps in the above-mentioned embodiments of the method for automatically replying to user consultation information based on natural language are implemented, such as Figure 1 Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are realized, such as the data acquisition module.

[0134] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0135] The electronic device may be a computing device such as a desktop computer, a notebook, a PDA, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the above components are merely examples of electronic devices and do not constitute a limitation on the electronic device. The electronic device may include more or fewer components than the above components, or may combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0136] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the electronic device, and uses various interfaces and lines to connect various parts of the entire electronic device.

[0137] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the electronic device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0138] Wherein, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-0nly Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0139] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0140] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatically replying to user consultation information based on natural language, characterized in that: Executed by a computer, including: Get the question information entered by the user; Inputting the question information into a semantic extraction model to obtain keyword features and user features; Inputting the question information and historical conversation data into a context parsing model to obtain question scene features; Obtaining a question feature vector by a vectorization method according to the keyword feature, the user feature and the question scene feature; Inputting the question feature vector into the graph reasoning model to obtain a predicted response result; According to the predicted reply result, the question information input by the user is automatically replied.

2. According to claim 1, a method for automatically replying to user consultation information based on natural language, characterized in that: The step of inputting the question information into a semantic extraction model to obtain keyword features and user features includes: Cleaning the question information to obtain pure question information; Inputting the pure question information into a semantic extraction model, and outputting keyword features and user features; The training process of the semantic extraction model includes: A training set is constructed based on historical natural language data to train the semantic extraction model.

3. The method for automatically replying user consultation information based on natural language according to claim 1, characterized in that: The step of inputting the question information and historical dialogue data into a context analysis model to obtain question scene features includes: Inputting the question information and historical conversation data into a context parsing model to obtain question scene features, wherein the question scene features include time and place; The training process of the context parsing model includes: The historical conversation data is used as the training set, and the cross entropy loss is used as the loss function to train the context parsing model.

4. The method for automatically replying user consultation information based on natural language according to claim 1, characterized in that: The step of obtaining a question feature vector by a vectorization method according to the keyword feature, the user feature and the question scene feature includes: Input the keyword features into a pre-trained word vector model, and output a keyword vector; Performing one-hot encoding on the user features and the question scene features to obtain a user feature vector and a question scene feature vector; The keyword vector, user feature vector and question scene feature vector are passed through a fully connected layer neural network to output a question feature vector.

5. The method for automatically replying user consultation information based on natural language according to claim 1, characterized in that: The step of inputting the question feature vector into the graph reasoning model to obtain a predicted response result includes: The training process of the graph reasoning model includes: Train graph reasoning models based on knowledge graphs; Select the following loss function to train the graph reasoning model: Among them, L is the total loss function; D is the training data set, which contains all positive sample pairs (y i , p i ) and negative sample pairs yi represents the predicted entity vector; pi represents the number of paths between the head entity and the tail entity; R1(y i ) and R2(y i ) represent the entity vector y i Projection in two different relational spaces; Represents the entity vector y i Negative samples; K is the number of relationship types; and Represents the negative sample entity vectors Projection in two different relational spaces; Represents the entity vector y i The transpose of .

6. The method for automatically replying user consultation information based on natural language according to claim 5, characterized in that: The construction process of the knowledge graph includes: Obtain historical user consultation information; Constructing a historical corpus of the historical user consultation information by using a statistical method; The knowledge graph structure is constructed using keyword features, user features or scene features in the historical corpus as nodes and reasoning relationships as edges.

7. The method for automatically replying user consultation information based on natural language according to claim 1, characterized in that: After automatically replying to the question information input by the user according to the predicted reply result, the method further includes: Obtain user feedback results; Optimizing the knowledge graph according to the user feedback results; A generative model building mechanism is used to optimize the graph reasoning model.

8. A natural language based automatic reply system for user consultation information, characterized in that: include: A data acquisition module is used to obtain question information input by the user; A semantic extraction module, used for inputting the question information into a semantic extraction model to obtain keyword features and user features; A scene extraction module, used to input the question information and historical dialogue data into a context analysis model to obtain question scene features; A feature fusion module, used to obtain a question feature vector by a vectorization method according to the keyword feature, the user feature and the question scene feature; A result prediction module, used for inputting the question feature vector into the graph reasoning model to obtain a predicted reply result; The reply module is used to automatically reply to the question information input by the user according to the predicted reply result.

9. An electronic device, characterized in that: It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements the natural language-based user consultation information automatic reply method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the natural language-based user consultation information automatic reply method as described in any one of claims 1 to 7.

Images