A conversation intelligent generation method based on semantic matching and generation model

Abstract

The application discloses to the technical field of intelligent conversation, specifically a conversation intelligent generation method based on semantic matching and generation model, including model preparation stage, data transmission, dialogue skill base vector retrieval, semantic matching, dialogue skill generation, result display and sending and multiple steps, the application automatically matches the question in the dialogue skill base through the AI model auxiliary artificial customer service, reduces the requirement for artificial professional ability, and generates answers in multiple different expression modes, so that the artificial only needs to judge simply, and can return personalized answer service through one-key clicking, and based on a semantic model that can simultaneously respond to Chinese semantic matching and generation tasks, can greatly reduce the resource consumption of model training, fine tuning, online deployment, real-time prediction and the like which need very high cost.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of intelligent conversation, in particular to a conversation intelligent generation method based on semantic matching and a generation model. BACKGROUND

[0002] Intelligent conversation mainly refers to the communication scene in the complete sales process such as pre-sales, in-sales and after-sales. The AI model assists the entire process, such as intelligent matching of sales techniques, AI quality inspection, automatic extraction of labels and the like. The result returned by the intelligent customer service is too cold and rigid, and lacks personalized service, which greatly affects the user experience. In addition, if personalized service is provided based on manual work, it will be highly dependent on the professional ability of manual work, and a large amount of energy needs to be consumed to reply to problems individually. Semantic matching and generation model refers to a model that can simultaneously solve the Chinese semantic matching and generation task, supports Chinese text matching and text generation two tasks, and can realize one-time training for deployment response for the scene that needs to simultaneously solve the two tasks. For semantic matching and semantic generation, two models are generally needed to complete the corresponding functions. Thus, for training, fine-tuning, online deployment, real-time prediction and the like, the corresponding resources need to be repeatedly consumed. For the scene from training to deployment of the AI model, the resource cost is very high, and some existing multi-task models, such as the PALM model, have some defects, such as the task defects of the denoising autoencoder, or do not support Chinese, so that the effect cannot reach the premise of commercial use. SUMMARY

[0003] The application aims to provide a conversation intelligent generation method based on a semantic matching and generation model, to solve the problems of dependence on manual professional ability, consumption of a large amount of energy for individualized reply, repeated consumption of corresponding resources and existence of some defects in the background technology.

[0004] To achieve the above-mentioned purpose, the application provides the following technical scheme: a conversation intelligent generation method based on a semantic matching and generation model, comprising the following steps:

[0005] Step 1: model preparation stage: a separate shallow decoder is added to the PALM model, based on the CLUE news and encyclopedia data set, in each training batch, 1 / 4 of the data is used for bidirectional optimization, 1 / 4 of the data is used for Seq2Seq language model optimization, 1 / 4 of the data is used for right-to-left unidirectional optimization, 1 / 4 of the data is used for left-to-right unidirectional optimization, and 20% of the words in the sentence are masked to obtain a model with semantic matching and generation tasks;

[0006] Step 2: Data transmission: Since the means of communication in the sales process will be diverse, the content sent by the customer in real time is preprocessed: if the customer sends audio, it is converted into text through speech recognition, and then the continuous text is converted into understandable sentences through semantic segmentation; if the customer sends text, the above content is converted into a text format that can be processed by the model in step 1 through semantic segmentation and text correction processing;

[0007] Step 3: Vector retrieval of the script library: The existing questions in the script library are input into the semantic matching and generation model in step 1 in batches, and the semantic vectors corresponding to the questions are obtained respectively, and are managed by the corresponding vector retrieval engine for indexing;

[0008] Step 4: Semantic matching: The text question processed by the step 2 module is input into the semantic matching and generation model, and the text vector corresponding to the text is output, and then the text vector is found through the vector retrieval engine in step 3 to find the question with the highest semantic similarity in the script library;

[0009] Step 5: Script generation: The answer script corresponding to the question found in step 4 is input into the semantic matching and generation model to generate multiple answers expressing the same semantics but in different description ways;

[0010] Step 6: Result display and sending: The answers generated in step 5 are returned to the artificial display in real time, and the artificial selects the most suitable personalized description method based on the current scene and the customer's situation, and returns the customer through one-key clicking.

[0011] Preferably, the shallow decoder in step 1 is composed of 5 stacked Transformer layers, and a shared encoder is used as its input to solve the negative effects of the denoising autoencoder.

[0012] Preferably, the learning rate of the semantic matching and generation model in step 1 is 0.00002, the batch size is 80, and the total training is 140-160 thousand times.

[0013] Preferably, the script library in step 3 is an Internet-based sales team business reserve knowledge base, and each script in the script library corresponds to a specific question and an answer to the question.

[0014] Preferably, 70% of the 20% of the words masked in step 1 are replaced by masks, 15% are replaced by random words in the dictionary, and 15% remain unchanged.

[0015] Compared with the prior art, the beneficial effects of the present application are:

[0016] 1) The present application proposes a novel intelligent auxiliary conversation method, which can send questions in real time for customers, find the corresponding questions in the dialogue library based on the semantic matching function of the improved semantic model that can simultaneously respond to Chinese semantic matching and generation tasks, and then return multiple answers with the same semantics but different description methods through the generation module of the model. This can allow sales and customer service personnel to select personalized description answers according to the context scene and return them, reduce the requirement for the professional level of sales personnel, improve the reply efficiency, and provide customers with better personalized service experience.

[0017] 2) The present application can automatically match the questions in the dialogue library through the AI model assisted artificial customer service, reduce the requirement for the professional ability of the artificial, and generate multiple different expression methods of answers, so that the artificial can return personalized answer service through one-key click by simple judgment. At the same time, based on a semantic model that can simultaneously respond to Chinese semantic matching and generation tasks, the resource consumption of model training, fine-tuning, online deployment, real-time prediction and other high-cost resources can be greatly reduced. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The flowchart of the present application is shown. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0020] Embodiment:

[0021] Please refer to Figure 1 The present application provides a technical solution: a conversation intelligent generation method based on a semantic matching and generation model, including the following steps:

[0022] Step 1: Model preparation phase: Since the PALM model has some defects, as it adds a multi-language model task to the pre-trained encoder to make the encoder understand the meaning of the context, there is a risk of overfitting to the parameters initialized based on the autoencoder, and PALM does not support Chinese, so we improve it based on PALM. A separate shallow decoder is added to the PALM model, which consists of 5 stacked Transformer layers, and the shared encoder is used as its input to address the negative effects of the denoising autoencoder, which can achieve good performance on the text matching task. Based on the news and encyclopedia datasets of CLUE, in each training batch, use 1 / 4 of the data for bidirectional optimization, use 1 / 4 of the data for Seq2Seq language model optimization, use 1 / 4 of the data for right-to-left one-way optimization, and use 1 / 4 of the data for left-to-right one-way optimization. Mask 20% of the words in the sentence, 70% of which are replaced with masks, 15% are replaced with random words in the dictionary, and 15% remain unchanged. The learning rate is 0.00002, the batch size is 80, and the total training is 140-160 thousand times. Finally, a model with semantic matching and generation tasks is obtained;

[0023] Step 2: Data transmission: Since there are various means of communication in the sales process, the content sent by the customer in real time is preprocessed: if the customer sends audio, it is converted into text through speech recognition, and then the continuous text is converted into understandable sentences through semantic segmentation; if the customer sends text, it is processed through semantic segmentation and text correction to convert the above content into a text format that can be processed by the model in step 1;

[0024] Step 3: Vector retrieval of the script library: The existing questions in the script library are input into the semantic matching and generation model in step 1 in batches, and the corresponding semantic vectors are obtained, and are managed by the corresponding vector retrieval engine. The script library is an Internet-based sales team business knowledge base, and each script in the script library corresponds to a specific question and an answer to the question;

[0025] Step 4: Semantic matching: The text question processed by the step 2 module is input into the semantic matching and generation model, and the corresponding text vector is output. Then, the text vector is input into the vector retrieval engine in step 3 to find the question with the highest semantic similarity in the script library;

[0026] Step 5: Script generation: The answer script corresponding to the question found in step 4 is input into the semantic matching and generation model to generate multiple answers that express the same semantics but have different descriptions;

[0027] Step 6: Result display and sending: the answer generated in step 5 is returned to the artificial display in real time, and the artificial selects the most suitable personalized description method based on the current scene and the customer's situation, and returns it to the customer through one-key click.

[0028] The specific generation method of the model of the semantic matching and generation task in step 1 of the present application is as follows:

[0029] Step A: Based on the CLUE news and encyclopedia dataset, 20% of the words in the sentence are masked, 70% of the masked words are replaced with masks, 15% remain unchanged, and 15% are replaced with symbols in the dictionary in a random way;

[0030] Step B: Based on the Transformer encoder and decoder structure, the parameter encoder is designed based on the deep encoder, the matching decoder is based on the shallow decoder, and the generation decoder is based on the deep decoder;

[0031] Step C: Model pre-training: Use the denoising auto-encoding task to train the parameter encoder and the generation encoder to reconstruct the original text;

[0032] Step D: Model fine-tuning: Fine-tune the model based on the actual scene dataset;

[0033] Step E: Model deployment: Deploy the fine-tuned model in step 4 to the server, which can respond to text matching tasks and text generation tasks at the same time. For the text matching task, the similarity of different text vectors is calculated through the text vector generated by the matching decoder. For the text generation task, the text corresponding to the vector generated by the generation decoder is returned.

[0034] The following is a reference for this technical field. By comparing the methods in the reference literature of the prior art in this field with the present application, the advancement of the present application technology can be better embodied.

[0035] [1] B. Bi, C. Li, C. Wu, M. Yan, W. Wang, S. Huang, F. Huang, and L. Si, PALM: pre-training an autoencoding & autoregressive language model for context-conditioned generation, in EMNLP 2020.

[0036] [2] Mostafazadeh N, Brockett C, Dolan B, et al. Image-Grounded Conversations: Multimodal Context for Natural Question and Response Generation, IJCNLP. 2017.

[0037] [3] Woon Sang Cho, Yizhe Zhang, Sudha Rao, et al. Contrastive Multi-document Question Generation, European Chapter of the Association for Computational Linguistics (EACL), 2021.

[0038] [4] Multi-scene dialogue system and method for supporting rapid construction of training data and process customization - 202210944385.1; 115017919B.

[0039] The above shows and describes the basic principles and main features of the present application and the advantages of the present application. It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application; therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, the scope of the present application is defined by the appended claims rather than the above description, therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application, no figure reference in the claims should be regarded as limiting the claims involved.

[0040] Although embodiments of the present application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present application, the scope of the present application is defined by the appended claims and their equivalents.

Claims

1.A method for intelligent generation of a conversation based on semantic matching and generative model, characterized in that: The method comprises the following steps: Step 1: Model preparation phase: a separate shallow decoder is added to the PALM model, based on the CLUE news and encyclopedia data sets, in each training batch, using 1 / 4 of the data for bidirectional optimization, using 1 / 4 of the data for Seq2Seq language model optimization, using 1 / 4 of the data for right-to-left unidirectional optimization, using 1 / 4 of the data for left-to-right unidirectional optimization, and using a mask to mask 20% of the words in the sentence, finally obtaining a model with semantic matching and generation tasks; Step 2: Data transmission: since there are various means of communication in the sales process, the content sent by the customer in real time is preprocessed: if the customer sends audio, it is converted into text through speech recognition, and then the continuous text is converted into understandable sentences through semantic segmentation; if the customer sends text, the above content is converted into a text format that can be processed by the model in step 1 through semantic segmentation and text correction processing; Step 3: Vector retrieval of the script library: the existing questions in the script library are input into the semantic matching and generation model in step 1 in batches, and the corresponding semantic vectors of the questions are obtained, and the vector retrieval engine is used to establish index management; Step 4: Semantic matching: input the text question processed by the step 2 module into the semantic matching and generation model, output the text vector corresponding to the text, and then find the question with the highest semantic similarity in the script library through the vector retrieval engine in step 3; Step 5: Script generation: input the answer script corresponding to the question found in step 4 into the semantic matching and generation model to generate multiple answers expressing the same semantics but in different description ways; Step 6: Result display and sending: return the answers generated in step 5 to the artificial display in real time, and the artificial selects the most appropriate personalized description method based on the current scene and the customer's situation, and returns the customer through one-key click. 2.The method of claim 1, wherein: The shallow decoder in step 1 is composed of 5 layers of stacked Transformer layers, which uses a shared encoder as its input to solve the negative effects of the denoising autoencoder. 3.The method of claim 1, wherein: The learning rate of the semantic matching and generation model in step 1 is 0.00002, the batch size is 80, and the total training is 140-160 thousand times. 4.The method of claim 1, wherein: The script library in step 3 is an Internet-based sales team business knowledge base, and each script in the script library corresponds to a specific question and an answer to the question. 5.The method of claim 1, wherein: Among the 20% of the words masked in step 1, 70% are replaced with masks, 15% are replaced with random words in the dictionary, and 15% remain unchanged.

Images