Conversation method and system of conversation robot and computer equipment

Abstract

The invention relates to a dialogue method and system of a dialogue robot, computer equipment, a storage medium and a computer program product. Relates to the technical field of artificial intelligence. The method comprises the following steps: receiving a to-be-processed text, and converting the to-be-processed text into a first word embedding vector based on a first word embedding model; converting the first word embedding vector into a second word embedding vector based on a second word embedding model; storing the second word embedding vector into a vector database, and determining a target question corresponding to the second word embedding vector in the vector database based on a preset question and answer mode; and in combination with the target question and a preset text generation mode, outputting a target answer of the to-be-processed text. By adopting the method, the question and answer accuracy of the dialogue robot when the text data is short can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a conversation method, system and computer device for a conversation robot. Background Art

[0002] In the field of artificial intelligence, conversational robots have a wide range of application scenarios, such as customer service and after-sales, which provide convenience for life. However, there is also the problem that conversational robots cannot accurately understand the user's expression and intention when the text data is short.

[0003] However, currently, when the text data is short, the method to help the conversational robot understand the user's intention is to perform deep retrieval of the text data through modular retrieval enhancement technology, that is, to add a sentiment recognition model or a rearrangement model in the retrieval stage to enhance the retrieval of text data, but at the same time, it will greatly increase the time consumption of the retrieval stage, causing the corresponding system of the conversational robot to run slower. Therefore, there is currently a problem that when the text data is short, the accuracy of the conversational robot in the question-and-answer process is low. Summary of the invention

[0004] Based on this, it is necessary to provide a conversation method, system, computer device, computer-readable storage medium and computer program product for a conversation robot to address the technical problem that the conversation robot has low accuracy in the question-and-answer process when the text data is short.

[0005] In a first aspect, the present application provides a conversational robot training method, comprising:

[0006] Receive a text to be processed, and convert the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training a word embedding model based on first training text data;

[0007] Converting the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on second training text data;

[0008] The second word embedding vector is stored in a vector database, and a target question corresponding to the second word embedding vector is determined in the vector database based on a preset question-answering mode; wherein the vector database includes an association relationship between the second word embedding vector and the target question;

[0009] Combining the target question with a preset text generation mode, a target answer for the text to be processed is output.

[0010] In one embodiment, the method further includes: inputting the first training text data into the word embedding model, and segmenting the first training text data through word segmentation technology, screening non-keywords in the segmented words as stop words, and forming a stop word library; screening keywords in the segmented words, and based on the occurrence frequency of each keyword, screening the keywords whose occurrence frequency is greater than or equal to a preset frequency as target keywords, and forming a keyword library; updating the first training text data based on the stop words to obtain first updated training data, and inputting the first updated training data into the word embedding model to complete the training of the word embedding model and obtain the first word embedding model.

[0011] In one embodiment, the method further includes: inputting the second training text data into the first word embedding model combined with preset core hyperparameters to complete the training of the first word embedding model and obtain the second word embedding model; wherein the core hyperparameters include calculation accuracy, learning rate, optimizer, temperature, pooling method and number of training times.

[0012] In one embodiment, the step of inputting the second training text data into the first word embedding model combined with preset core hyperparameters to complete the training of the first word embedding model and obtain the second word embedding model includes: matching a preset initial learning rate and a preset temperature to the first word embedding model; training the first word embedding model based on a preselected optimizer and in combination with a half-precision floating point quantization method and a classification label pooling method; inputting the second training text data into the first word embedding model for training, and completing the training of the first word embedding model when the number of training times is greater than or equal to a preset target training time and the loss value is within a preset target loss value range to obtain the second word embedding model.

[0013] In one of the embodiments, determining the target question corresponding to the second word embedding vector in the vector database based on a preset question-answering model includes: obtaining the similarity between the second word embedding vector and the vector in the vector database; determining multiple related questions of the second word embedding vector and a single identifier of the multiple related questions based on the similarity; and according to the single identifier, taking the single target question corresponding to the multiple related questions as the target question corresponding to the second word embedding vector.

[0014] In one embodiment, the preset text generation mode includes: a frequently asked question mode and a scenario mode; the generation of a target answer to the text to be processed by combining the target question and the preset text generation mode includes: in the frequently asked question mode and / or the scenario mode, when a preset target number of related questions retrieved from the vector database correspond to the same target question, the answer corresponding to the target question is used as the target answer to the text to be processed.

[0015] In a second aspect, the present application also provides a dialogue system of a dialogue robot, including:

[0016] A text processing module, configured to receive a text to be processed and convert the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training a word embedding model based on first training text data;

[0017] A word embedding vector conversion module, used to convert the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on second training text data;

[0018] A target question confirmation module, used to store the second word embedding vector into a vector database, and determine the target question corresponding to the second word embedding vector in the vector database based on a preset question-answering mode; wherein the vector database includes an association relationship between the second word embedding vector and the target question;

[0019] The target answer output module is used to output the target answer of the text to be processed in combination with the target question and a preset text generation mode.

[0020] In a third aspect, the present application further provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0021] Receive a text to be processed, and convert the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training a word embedding model based on first training text data;

[0022] Converting the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on second training text data;

[0023] The second word embedding vector is stored in a vector database, and a target question corresponding to the second word embedding vector is determined in the vector database based on a preset question-answering mode; wherein the vector database includes an association relationship between the second word embedding vector and the target question;

[0024] Combining the target question with a preset text generation mode, a target answer for the text to be processed is output.

[0025] In a fourth aspect, the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0026] Receive a text to be processed, and convert the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training a word embedding model based on first training text data;

[0027] Converting the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on second training text data;

[0028] The second word embedding vector is stored in a vector database, and a target question corresponding to the second word embedding vector is determined in the vector database based on a preset question-answering mode; wherein the vector database includes an association relationship between the second word embedding vector and the target question;

[0029] Combining the target question with a preset text generation mode, a target answer for the text to be processed is output.

[0030] In a fifth aspect, the present application further provides a computer program product, the computer program product comprising a computer program, which implements the following steps when executed by a processor:

[0031] Receive a text to be processed, and convert the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training a word embedding model based on first training text data;

[0032] Converting the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on second training text data;

[0033] The second word embedding vector is stored in a vector database, and a target question corresponding to the second word embedding vector is determined in the vector database based on a preset question-answering mode; wherein the vector database includes an association relationship between the second word embedding vector and the target question;

[0034] Combining the target question with a preset text generation mode, a target answer for the text to be processed is output.

[0035] The conversation method, system, computer device, storage medium and computer program product of the above-mentioned conversation robot, in the conversation method of the conversation robot, first receive the text to be processed, and convert the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training the word embedding model based on first training text data; then convert the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on second training text data; then store the second word embedding vector in a vector database, and determine the target question corresponding to the second word embedding vector in the vector database based on a preset question-answering mode; wherein the vector database includes the association relationship between the second word embedding vector and the target question; finally, combine the target question and the preset text generation mode to output the target answer of the text to be processed. In the above process, the text to be processed is converted into a first word embedding vector through the first word embedding model, and then the first word embedding vector is converted into a second word embedding vector through the second embedding model. This can improve the quality of the word embedding vector and improve the efficiency of determining the target question in the vector database in subsequent steps. Then, by combining the target question with the preset text pattern, the target answer is further output, which effectively improves the question and answer accuracy of the conversational robot when the text data is short. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic diagram of a flow chart of a conversation method of a conversation robot in one embodiment;

[0038] Figure 2 A flowchart of a conversation method of a conversation robot in another embodiment;

[0039] Figure 3 A timing diagram of a dialogue system of a dialogue robot in one embodiment;

[0040] Figure 4 A schematic diagram of the structure of a dialogue system of a dialogue robot in one embodiment;

[0041] Figure 5 A schematic diagram of the structure of a dialogue system of a dialogue robot in another embodiment;

[0042] Figure 6 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] In the field of conversational robots, a conversational robot needs to meet the user's questioning needs, so that it can be applied in a wide range of scenarios, such as customer service, after-sales, etc. However, understanding the user's needs for relatively short corpus data has become a major challenge for conversational robots in the field of natural language understanding. Most of the existing technologies can be divided into two technical routes: modular retrieval enhancement and text segmentation. Among them, modular retrieval enhancement technology mainly adds sentiment recognition models, rearrangement models, etc. in the retrieval stage. This architecture greatly increases the processing time of the retrieval stage, resulting in system slowdown; while the text segmentation technology route is more suitable for long text retrieval and processing, because this technical architecture mainly improves the global accuracy by changing the retrieval block size, and is more suitable for retrieval enhancement question and answer under long text from a practical application perspective. Therefore, there is currently a problem that when the text data is short, the accuracy of the conversational robot in the question and answer process is low.

[0045] In an exemplary embodiment, Figure 1 As shown, a conversation method of a conversation robot is provided, comprising the following steps S102 to S106. Among them:

[0046] Step S102, receiving the text to be processed, and converting the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training the word embedding model based on first training text data.

[0047] Among them, the text to be processed is the original text data that needs to be converted; word embedding is a technology that converts words in a text into vectors, and the word embedding model is a model used to generate word embedding vectors; the first training text is text data used to train the word embedding model to obtain the first word embedding model.

[0048] Specifically, word embedding technology can map the words in the text to be processed into a high-dimensional space, so that similar words are closer in the vector.

[0049] Step S104, converting the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on second training text data.

[0050] Among them, the second word embedding model is another word embedding model used to convert the first word embedding vector into the second word embedding vector; accordingly, the second training text is text data used to train the first word embedding model to obtain the second word embedding model.

[0051] Step S106, storing the second word embedding vector in a vector database, and determining a target question corresponding to the second word embedding vector in the vector database based on a preset question-answering model; wherein the vector database includes an association relationship between the second word embedding vector and the target question.

[0052] Among them, the vector database is a database that stores the second word embedding vector and its corresponding target question; the preset question-answering mode can be to query the target question in the vector database; the target question is the main question associated with the second word embedding vector stored in the vector database, usually a question that needs to be answered by the user.

[0053] Specifically, the association relationship between the second word embedding vector and the target question can be a similarity relationship, a label relationship, or a time relationship, etc. When the association relationship is a similarity relationship, the target question that is most similar to the current second word embedding vector can be determined by calculating the similarity between the second word embedding vector and other vectors in the vector database, such as cosine similarity, Euclidean distance, etc.; when the association relationship is a label relationship, each target question can be labeled as a specific category or topic, and the label can be associated with the second word embedding vector; when the association relationship is a time relationship, the creation time or last update time of the target question can be associated with the second word embedding vector.

[0054] Step S108, combining the target question and the preset text generation mode, outputting the target answer of the text to be processed.

[0055] Among them, the preset text generation modes may include FAQ mode (Frequently Asked Questions), scenario mode and large model mode; the target answer is the final generated answer, which combines the target question and the preset text generation mode to provide the required information or answer for the text to be processed.

[0056] In the above dialogue method of the dialogue robot, first, the text to be processed is received, and the text to be processed is converted into a first word embedding vector based on the first word embedding model; wherein, the first word embedding model is obtained by training the word embedding model based on the first training text data; then, the first word embedding vector is converted into a second word embedding vector based on the second word embedding model; wherein, the second word embedding model is obtained by training the first word embedding model based on the second training text data; then, the second word embedding vector is stored in the vector database, and the target question corresponding to the second word embedding vector is determined in the vector database based on the preset question and answer mode; wherein, the vector database includes the association relationship between the second word embedding vector and the target question; finally, the target answer of the text to be processed is output by combining the target question and the preset text generation mode. In the above process, by converting the text to be processed into the first word embedding vector through the first word embedding model, and then converting the first word embedding vector into the second word embedding vector through the second word embedding model, the quality of the word embedding vector can be improved, and the efficiency of determining the target question in the vector database in the subsequent steps can be improved. Then, by combining the target question and the preset text mode, the target answer is further output, effectively improving the question and answer accuracy of the dialogue robot when the text data is short.

[0057] In an exemplary embodiment, the above method further includes: inputting the first training text data into the word embedding model, segmenting the first training text data by using a word segmentation technique, screening out the non-keyword words in the segmentation as stop words to form a stop word library; screening out the keyword words in the segmentation, and based on the occurrence frequency of each keyword, screening out the keyword words with an occurrence frequency greater than or equal to a preset frequency as target keyword words to form a keyword library; updating the first training text data based on the stop words to obtain first updated training data, and inputting the first updated training data into the word embedding model to complete the training of the word embedding model and obtain the first word embedding model.

[0058] Among them, the word segmentation technique is a technique for splitting a continuous text into independent words or phrases; stop words are common words that are ignored during the text processing, such as "de", "le", "shi", etc. These words usually contribute less to the theme and meaning of the text; the stop word library refers to the database storing stop words; correspondingly, keyword words are words with important meanings in the text that can reflect the core content and theme of the text, and the keyword library refers to the database storing keyword words; the first updated training data is the text data remaining after removing the stop words in the first training text data.

[0059] Specifically, Jieba (结巴) word segmentation and word frequency statistics techniques can be used to segment the first training text.

[0060] In this embodiment, by deleting stop words, the first training data focuses more on important content, helping the word embedding model to learn more accurate semantic relationships, thereby improving the effectiveness of the first word embedding model; screening out target keywords with higher frequency enables the final first embedding model to enhance the quality of information retrieval and analysis.

[0061] In one embodiment, the above method also includes: inputting the second training text data into the first word embedding model combined with preset core hyperparameters to complete the training of the first word embedding model and obtain the second word embedding model; wherein the core hyperparameters include calculation accuracy, learning rate, optimizer, temperature, pooling method and number of training times.

[0062] Among them, calculation accuracy is the accuracy parameter of floating-point operations, which will affect the training effect of the model; learning rate is the parameter that controls the update of model weights; optimizer is the algorithm used to update model weights; temperature is used to adjust the output distribution of the model; pooling methods can include average pooling and maximum pooling, etc.; the number of training times is the number of iterative training rounds.

[0063] Specifically, the optimizer may use Adam (Adaptive Moment Estimation), SGD (Stochastic Gradient Descent), and AdamW (Adam with Weight Decay) is used in this application.

[0064] In this embodiment, by setting the core hyperparameters, the training process is made more flexible, so that the first word embedding model can perform more effective parameter adjustment on specific tasks, thereby obtaining better performance.

[0065] Furthermore, in one embodiment, the second training text data is input into the first word embedding model combined with preset core hyperparameters to complete the training of the first word embedding model and obtain the second word embedding model, including: matching the preset initial learning rate and the preset temperature to the first word embedding model; training the first word embedding model based on a preselected optimizer and in combination with a half-precision floating point quantization method and a classification label pooling method; inputting the second training text data into the first word embedding model for training, and completing the training of the first word embedding model when the number of training training is greater than or equal to the preset target training number and the loss value is within the preset target loss value range to obtain the second word embedding model.

[0066] Among them, the initial learning rate is the learning rate used by the first word embedding model at the beginning of training, which controls the pace of weight updating; half-precision floating-point quantization is a method to reduce model storage space and computational load, using 16 bits (half-precision) to represent floating-point numbers, which reduces memory usage and improves computational efficiency compared to 32 bits (single precision); the classification label pooling method is a method for aggregating information when processing embedded vectors; the loss value is an indicator for evaluating model performance during training, reflecting the gap between the model's predicted value and the true value. If it is within the preset target loss value range, it means that the model performance is within an acceptable range.

[0067] In this embodiment, by using reasonable settings of the initial learning rate and temperature, combined with the selection of the optimizer, the first word embedding model can be trained more effectively; the use of a half-precision floating-point quantization method reduces memory usage and computational load, making the training process more efficient; the classification label pooling method can better aggregate information and extract more representative features, allowing hyperparameters to be adjusted according to specific task requirements, making model training more flexible and adaptable to different application scenarios.

[0068] In an exemplary embodiment, a target question corresponding to a second word embedding vector is determined in a vector database based on a preset question-answering model, including: obtaining the similarity between the second word embedding vector and the vector in the vector database; determining multiple related questions of the second word embedding vector and a single identifier of the multiple related questions based on the similarity; and according to the single identifier, taking the single target question corresponding to the multiple related questions as the target question corresponding to the second word embedding vector.

[0069] Among them, the second word embedding vector is a vector converted by the second word embedding model, which is used for subsequent similarity calculation and matching of similar questions; similarity is an indicator to measure the degree of similarity between two vectors. Commonly used calculation methods include cosine similarity, Euclidean distance, etc. The higher the similarity, the closer the two vectors are semantically; related questions are a set of questions that are semantically similar or related to the second word embedding vector, which are usually retrieved from a vector database; a single identifier is an identifier used to uniquely identify each related question, usually a string or a number, which is convenient for retrieval and management in a vector database.

[0070] Specifically, by calculating the similarity between the second word embedding vector and the vector stored in the vector database, the content related to the word embedding vector can be identified, and then multiple related questions can be screened out according to the set threshold, and the most representative or relevant one of the multiple related questions can be selected as the target question based on a single identifier.

[0071] In this embodiment, by calculating the similarity between the second word embedding vector and the vector in the vector database, relevant questions can be quickly identified, significantly improving the speed and efficiency of information retrieval; by utilizing similarity calculation and identifier management, the high relevance of the target question to the user query can be ensured, reducing the interference of irrelevant information and improving the accuracy of the answer; the matching of related questions can be dynamically adjusted according to the new word embedding vector and the updated vector database, which is suitable for a variety of scenarios.

[0072] Furthermore, in one embodiment, the preset text generation modes include: FAQ mode and scenario mode; combining the target question and the preset text generation mode, generating a target answer for the text to be processed, including: in the FAQ mode and / or scenario mode, when a preset target number of related questions retrieved from the vector database correspond to the same target question, the answer corresponding to the target question is used as the target answer for the text to be processed.

[0073] Among them, the preset text generation modes also include the large model mode; the FAQ mode is a text generation mode, which is usually used to answer users' common questions and provide information in a structured manner to facilitate users to quickly obtain answers; the scenario mode is a mode that generates answers based on specific scenarios or contexts, usually considering the user's specific needs and background to provide more targeted answers.

[0074] For example, when three similar questions are retrieved from the vector database, and the three related questions all belong to the same target question, the answer to the target question is directly output; in other cases, the results of the target question are deduplicated after the retrieval is completed, and the number of target questions after deduplication is less than or equal to three, and then they are returned to the user; it also includes a mixed mode, which means that a user asks a question that may match the FAQ mode category or the scene mode category, and the system needs to finally decide which one to go for; the mixed mode has two decision algorithms: score rearrangement and ratio rearrangement. After the score rearrangement searches the results from the FAQ mode and scene mode vector databases respectively, it performs a mixed ranking based on the similarity score and returns the top three items with higher scores; the ratio ranking is because in some cases the difference in the scores between the FAQ mode and the scene mode is large, resulting in the possibility that the top three items may all belong to the FAQ mode or the scene mode if the top three items are returned based on the similarity score. In this case, the ratio method is flexibly adopted. If the ranking shows that the top three most similar results are all in the same category, the third item is replaced with another category.

[0075] In this embodiment, by combining the target question and the preset text generation mode, it is ensured that the generated answer directly responds to the user's core question, thereby improving the accuracy of the target answer; when multiple related questions correspond to the same target question, a unified answer is used as the target answer for the text to be processed to ensure the consistency and reliability of the information and reduce user confusion; by setting a preset target number, unnecessary calculation and processing time is reduced, thereby reducing the cost of generating answers.

[0076] This application provides a conversation method for a conversation robot. In order to better understand the process of the conversation method of the conversation robot, combined with Figure 2 As shown, the specific process of the conversation method of the conversation robot of the present application is described in detail below, including the following steps:

[0077] Step S202: Use the stop word technology to process the first training data to form a stop word library, and based on the stop word library, delete the stop words in the first training data to obtain first updated training data.

[0078] Step S204, training the word embedding model based on the first updated training data, extracting keywords according to the first updated training data and forming a keyword library, obtaining a first word embedding model based on the keyword library, and using the first word embedding model to convert the first training text data into a first word embedding vector and store it.

[0079] Step S206: input the second training text data into the first word embedding model to obtain a second word embedding model through training.

[0080] Step S208, comparing the second word embedding vector stored in the vector library with the vector in the vector library to obtain multiple results with high similarity.

[0081] Step S210, based on the multiple results, obtain a standard question corresponding to the multiple similar questions, and output a standard answer to the standard question.

[0082] In the above embodiment, by using the stop word technology to process the first training data, common words that do not contribute to semantic analysis are removed, making the training data cleaner and more effective, thereby improving the quality of subsequent model training, and by comparing the word embedding vectors stored in the vector library, multiple results with high similarity are quickly obtained, thereby improving the efficiency of information retrieval and matching.

[0083] Furthermore, in one embodiment, the training process and use method of the conversational robot of the present application may include four modules, namely: a text preprocessing module, a word embedding large model module, an information retrieval module, and a text generation module. The specific use process is as follows:

[0084] Each processing stage of the system will perform natural language processing on Chinese text (first training text data). Using stop word technology, a large amount of first training text data is used in the word embedding model training stage to enable the word embedding model to understand the conversation content of relevant scenarios. Jieba word segmentation and word frequency statistics technology are used to segment all first training text data. After manual review, the words with high frequency of occurrence and no effect on the semantics of any text are saved in a unified stop word library. The stop word library will continue to accumulate as the first training text data increases. All first training text data are deleted according to the stop words contained in the stop word library, and then the word embedding model is trained and optimized based on the cleaned data.

[0085] In the word embedding storage stage, the trained and optimized model (first word embedding model) needs to convert the real scene data (text to be processed) into the first word embedding vector and then store it. Before the conversion, the system will also use the stop word technology to delete all irrelevant words in the data to be stored. The system uses the text after deleting the stop words (the first updated training data) as the input of the first word embedding model, generates a word embedding vector, and then saves it.

[0086] During the user retrieval stage, the system needs to use a vector retrieval algorithm to find similar questions (related questions) based on the user's questions. Before performing the retrieval, the system will perform stop word processing on the user input text (text to be processed).

[0087] The second text preprocessing technology is keyword tokenization. This technology is applied in the fine-tuning step of the first word embedding model. First, Jieba word segmentation and word frequency statistics technology is used to extract and count the keywords and corresponding frequencies appearing in the second training text data, and the keywords (target keywords) with high frequency and consistent with the scene are manually added to the unified keyword library. This keyword library will continue to accumulate with the continuous increase of the second training text data and scenes, and then these keywords are added to the keyword table of the word embedding model tokenizer, so that special keywords can be processed separately when the second training text data is segmented, and no further splitting will deteriorate the semantics and model understanding.

[0088] After being added to the vocabulary, the second word embedding model needs to be fine-tuned according to the new vocabulary, because the model does not recognize the newly added words at this time. After fine-tuning based on the training data, the model absorbs and accommodates all keywords.

[0089] The system uses a large word embedding model with about 300M parameters as the word embedding model. The model's embedding (embedding vector) length is 762-1024, which means that a query text will become a vector of length 762-1024 after being processed by the word embedding model. Compared with other types of data, at this model parameter level and embedding length, the model can accurately capture and understand the expected contextual meaning of the conversation.

[0090] In order to make the second word embedding model understand the text content related to the scene requirements and produce high-quality second word embedding vectors for retrieval, the system needs to fine-tune the second word embedding model based on relevant scene data. The core hyperparameters in the training process are calculation accuracy, learning rate, optimizer, temperature, pooling method, and Epoch (number of training times). The model uses half-precision floating point calculation FP16 during the training process. Compared with other model accuracy quantization methods, this system uses relatively large accuracy to be able to train stably in more variable dialogue scene data. The initial learning rate of training is 1e-5, and this learning rate will be adjusted during the training process with random optimization in the training warm-up phase. The optimizer uses AdamW, which uses its excellent performance and appropriate weight decay to make weight regularization reach a suitable level. The temperature is set to 0.02 to ensure the stable output of the model. In terms of pooling, the second word embedding vector with contextual semantics that is finally output by the second word embedding model is determined by the CLS (Classification Token Pooling) method, which means that the second word embedding vector output by the second word embedding model is the vector of the CLS word unit. After the model is trained for 5-10 epochs, the training will be stopped when the loss reaches 1-2.

[0091] The loss calculation in the training process of the second word embedding model is composed of "positive documents" and "negative documents". For a piece of training data (the second training text data), the second word embedding model first obtains the second word embedding vectors of the positive document and the negative document of the data through forward reasoning, and then calculates the gap between the positive document and the negative document through the loss function to complete the loss calculation. The goal of the loss function is the goal of the second word embedding model, which is to make the distance between positive pairs (positive pairs) in the embedding space as small as possible, and to make the distance between negative pairs (negative pairs) in the embedding space as large as possible.

[0092] Suppose we have a query q and a corresponding positive document d + , and a series of negative example documents {d1 +- , d2 - ,...d n -}, the embedding vector is calculated: the embedding of the query is represented as f(q); the embedding of each document d is represented as f(d).

[0093] Calculate similarity score: Use cosine similarity to measure the similarity between two embedding vectors. Cosine similarity between query q and document d:

[0094]

[0095] in:

[0096] in include and all negative examples ,and is a temperature parameter that controls the smoothness of the distribution, and accordingly, Represents the loss value.

[0097] The training data includes all similar questions Q in a specific scenario and the standard question S to which each similar question belongs, for example, Q={'How to check the electricity bill? ', 'Calculation method of tiered electricity price'}.

[0098] Split the data Q into N subsets according to batch B, and then extract a piece of data from the subset as the query text query, and the remaining text as the positive text pos of the current query. Take the same number of pos belonging to other standard questions S as the negative text neg of the current query.

[0099] The training data is [

[0101] {

[0102] "query": "How to check electricity bill?",

[0103] "pos": [

[0104] How to check electricity bills?

[0105] ],

[0106] "neg": [

[0107] “How to report a charging pile failure?” ]

[0109] } ]

[0111] Then, query, random(pos), and random(neg) in the training data are used as the input of the model, and the model outputs three word embedding vectors, query, positive, and negative respectively. The loss function is used to calculate the loss, and the cosine similarity is used to calculate the similarity distance between the vectors. Finally, the model weights are updated to complete a model training.

[0112] This system uses the faiss (Facebook AI Similarity Search) vector database to store the text vectorized data generated by the word embedding model. First, the system uses the IndexFlatIP (Flat Inner Product Index) algorithm to create an index, that is, it directly stores the original vector without making any changes. When performing a query, the cosine similarity is used to compare the query vectorized query text with the vector in the library, and the top k results with the highest similarity are returned.

[0113] Suppose we have two word embedding vectors and , they are both d-dimensional vectors, then the cosine similarity between these two vectors is defined as follows:

[0114]

[0115] If two vectors have exactly the same direction, their cosine similarity is 1, indicating that they are very similar.

[0116] In this system, the two answer modes FAQ and scenario use a vector database to store the vectorized data of related questions, and then the unique ID configured in the vectorized database for the related questions is indexed to the corresponding standard questions. There is a one-to-many relationship between standard questions and related questions. Training more related questions in the data preparation stage is conducive to matching similar questions with real user questions to a greater extent, so as to find the corresponding standard questions and preset response methods in the relational database. Managers can create multiple different FAQ and scenario databases according to business scenarios. In the retrieval stage, joint retrieval can be configured in different FAQ and scenario databases. Compared with other quantized storage methods, direct storage and brute force search are conducive to matching all information in the text. Because the system mainly serves user questions and answers in various scenarios, and the text has been streamlined in the data processing stage, this query algorithm is fully matched with the system design requirements and data processing methods.

[0117] After the user submits the question, the system uses the word embedding model to encode the question, and uses the encoded word embedding vector to query similar questions in the vector database using cosine similarity, that is, questions with similar meanings to the question submitted. These similar questions have been pre-associated with standard questions. The system can find the corresponding standard questions from the database and the corresponding standard answers. The same is true for scenario questions and answers. After the user submits the question, the vector library is searched for similar questions, and then the corresponding standard questions (scenario standard questions) are found, and then the next system answer is determined. This system has three text generation modes: FAQ, scenario, and big model. FAQ and scenario are answered based on the retrieval technology mentioned above. If the user's question cannot find the corresponding content in the stored FAQ and scenario similar questions, it means that the user's question deviates too much from the predetermined scenario, and the big model is used to answer the user's question.

[0118] In FAQ and scenario Q&A, the system uses direct output and top3 (the highest three) methods. After the system searches, if three similar questions are found in the vector library and all three similar questions belong to the same standard question, the answer to the standard question will be directly output. In other cases, the standard question results will be deduplicated after the search is completed. If the number of standard questions after deduplication is less than or equal to three, they will be returned to the user for further clarification.

[0119] The hybrid mode means that when a user asks a question, it may match a FAQ-similar question or a scene-similar question, and the system needs to make the final decision on which one to go with. The hybrid mode has two decision algorithms, score rearrangement and ratio rearrangement. After searching the results from the FAQ and scene vector libraries respectively, score rearrangement performs a mixed ranking based on the similarity score and returns the top three items with higher scores. Ratio ranking is because in some cases the FAQ and scene scores differ greatly. If the top three items are returned based on score similarity, the top three items may all belong to FAQ or scene. In this case, the ratio method is used flexibly. If the ranking shows that the top three most similar results are all in the same category, the third item is replaced with another category.

[0120] And, if Figure 3 As shown in the figure, the main users of the system are system administrators. The system administrator is the developer of this system and has the management authority of the system. He can assign usage authority to all other users. The operator refers to the operator of the client company of this system, and his authority is assigned by the system administrator. The operator can use functions such as data upload, data training, and dialogue robot creation, and ultimately serve the dialogue robot to the end customers of their company. The end customer refers to the end user of the dialogue robot system, who performs question-and-answer applications based on the scenarios developed by the corresponding company.

[0121] This system has two stages of use. The first is that the operator prepares FAQ, scenario standard questions and similar questions data for the specific scenarios of their company and imports them into this system. This system trains and stores the data. If necessary, the manager needs to modify the data and some parameters according to the end customer's usage effect to achieve the best effect. The second stage is the end user's use stage. After the manager completes the data training and robot construction, the end customer can have a conversation with the robot in this system. During the stage of end users' dialogue use of the robot based on specific scenarios, the system matches the FAQ associated during the operation period according to the user's questions, retrieves relevant data from the scenario database, and returns the structured reply content stored in the relational database to the end user.

[0122] like Figure 4 As shown in the figure, this system has designed a 5-layer supporting technology architecture to meet the above business function requirements. Among them, the front-end layer refers to the specific interactive software used by the end user, not limited to HTML (HyperTextMarkup Language) pages that can be used for dialogue, dialogue interactions in the outbound call system, etc. The control layer refers to the summary of all available functions for system administrators, operators, and end users. The search engine layer refers to the summary of all natural language processing technology modules in this system, which consists of natural language processing modules, word embedding large model modules and vector database modules. All content in the system that requires natural language processing, word embedding processing and vector similarity processing passes through this layer. The business engine layer refers to the three core functions supported by this system: FAQ, scenario Q&A, and large model Q&A. All functions in the system are built on these three core functions. The database layer refers to the storage of all business data in this system. All training data, user permission data, user usage data, etc. of this system are stored in a relational database. This system uses PythonFastAPI (Fast Application Programming Interface) as a service middleware to provide related services to the outside.

[0123] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0124] Based on the same inventive concept, the embodiment of the present application also provides a conversation system of a conversation robot for implementing the conversation method of the conversation robot involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the conversation system of one or more conversation robots provided below can refer to the limitations of the conversation method of the conversation robot above, and will not be repeated here.

[0125] In an exemplary embodiment, Figure 5 As shown, a dialogue system of a dialogue robot is provided, including: a text processing module 501, a word embedding vector conversion module 502, a target question confirmation module 503 and a target answer output module 504, wherein:

[0126] The text processing module 501 is used to receive the text to be processed and convert the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training the word embedding model based on the first training text data;

[0127] A word embedding vector conversion module 502 is used to convert the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on the second training text data;

[0128] A target question confirmation module 503 is used to store the second word embedding vector in a vector database, and determine the target question corresponding to the second word embedding vector in the vector database based on a preset question-answering mode; wherein the vector database includes an association relationship between the second word embedding vector and the target question;

[0129] The target answer output module 504 is used to output the target answer of the text to be processed in combination with the target question and the preset text generation mode.

[0130] Furthermore, in one embodiment, the text processing module 501 is also used to input the first training text data into the word embedding model, and segment the first training text data through the word segmentation technology, filter out non-keywords in the segmented words as stop words, and form a stop word library; filter out keywords in the segmented words, and based on the occurrence frequency of each keyword, filter out keywords with an occurrence frequency greater than or equal to a preset frequency as target keywords to form a keyword library; update the first training text data based on the stop words to obtain first updated training data, and input the first updated training data into the word embedding model to complete the training of the word embedding model to obtain a first word embedding model.

[0131] Furthermore, in one embodiment, the word embedding vector conversion module 502 is also used to input the second training text data into the first word embedding model combined with preset core hyperparameters to complete the training of the first word embedding model and obtain the second word embedding model; wherein the core hyperparameters include calculation accuracy, learning rate, optimizer, temperature, pooling method and number of training times.

[0132] Furthermore, in one embodiment, the word embedding vector conversion module 502 is also used to match a preset initial learning rate and a preset temperature to the first word embedding model; based on a preselected optimizer, and in combination with a half-precision floating-point quantization method and a classification label pooling method, the first word embedding model is trained; the second training text data is input into the first word embedding model for training, and when the number of training training times is greater than or equal to the preset target training times and the loss value is within the preset target loss value range, the training of the first word embedding model is completed to obtain the second word embedding model.

[0133] Furthermore, in one embodiment, the target question confirmation module 503 is also used to obtain the similarity between the second word embedding vector and the vector in the vector database; determine multiple related questions of the second word embedding vector and a single identifier of the multiple related questions based on the similarity; and according to the single identifier, use the single target question corresponding to the multiple related questions as the target question corresponding to the second word embedding vector.

[0134] Furthermore, in one embodiment, the target answer output module 504 is also used to use the answer corresponding to the target question as the target answer for the text to be processed when a preset target number of related questions retrieved from the vector database correspond to the same target question in the FAQ mode and / or scenario mode.

[0135] Each module in the above-mentioned dialogue robot's dialogue system can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0136] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the conversation data of the dialogue robot. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a conversation method of a dialogue robot is implemented.

[0137] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0138] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0139] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0140] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0142] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0143] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A conversation method of a conversation robot, characterized in that: The method comprises: Receive a text to be processed, and convert the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training a word embedding model based on first training text data; Converting the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on second training text data; The second word embedding vector is stored in a vector database, and a target question corresponding to the second word embedding vector is determined in the vector database based on a preset question-answering mode; wherein the vector database includes an association relationship between the second word embedding vector and the target question; Combining the target question with a preset text generation mode, a target answer for the text to be processed is output.

2. The method according to claim 1, characterized in that Also includes: Inputting the first training text data into the word embedding model, segmenting the first training text data using a word segmentation technique, and filtering non-keywords in the segmented words as stop words to form a stop word library; Filtering the keywords in the segmented words, and based on the occurrence frequency of each of the keywords, filtering the keywords whose occurrence frequency is greater than or equal to a preset frequency as target keywords to form a keyword library; The first training text data is updated based on the stop words to obtain first updated training data, and the first updated training data is input into the word embedding model to complete the training of the word embedding model to obtain the first word embedding model.

3. The method according to claim 1, characterized in that Also includes: The second training text data is input into the first word embedding model combined with preset core hyperparameters to complete the training of the first word embedding model and obtain the second word embedding model; wherein the core hyperparameters include calculation accuracy, learning rate, optimizer, temperature, pooling method and number of training times.

4. The method according to claim 3, characterized in that The step of inputting the second training text data into the first word embedding model combined with preset core hyperparameters to complete the training of the first word embedding model to obtain the second word embedding model includes: Matching a preset initial learning rate and a preset temperature to the first word embedding model; Based on the pre-selected optimizer, the first word embedding model is trained in combination with a half-precision floating point quantization method and a classification label pooling method; The second training text data is input into the first word embedding model for training. When the number of training times is greater than or equal to the preset target training times and the loss value is within the preset target loss value range, the training of the first word embedding model is completed to obtain the second word embedding model.

5. The method according to claim 1, characterized in that The determining, in the vector database, a target question corresponding to the second word embedding vector based on a preset question-answering mode includes: Obtaining the similarity between the second word embedding vector and the vector in the vector database; Determine a plurality of related questions of the second word embedding vector based on the similarity and a single identifier of the plurality of related questions; According to the single identifier, a single target question corresponding to the multiple related questions is used as the target question corresponding to the second word embedding vector.

6. The method according to claim 5, characterized in that The preset text generation modes include: FAQ mode and scenario mode; The step of combining the target question with a preset text generation mode to generate a target answer to the text to be processed includes: In the FAQ mode and / or the scenario mode, when a preset target number of related questions retrieved from the vector database correspond to the same target question, the answer corresponding to the target question is used as the target answer for the text to be processed.

7. A dialogue system of a dialogue robot, characterized in that: The system comprises: A text processing module, configured to receive a text to be processed and convert the text to be processed into a first word embedding vector based on a first word embedding model; wherein the first word embedding model is obtained by training a word embedding model based on first training text data; A word embedding vector conversion module, used to convert the first word embedding vector into a second word embedding vector based on a second word embedding model; wherein the second word embedding model is obtained by training the first word embedding model based on second training text data; A target question confirmation module, used to store the second word embedding vector into a vector database, and determine the target question corresponding to the second word embedding vector in the vector database based on a preset question-answering mode; wherein the vector database includes an association relationship between the second word embedding vector and the target question; The target answer output module is used to output the target answer of the text to be processed in combination with the target question and a preset text generation mode.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images