Intelligent chat information processing method and device, electronic equipment and storage device

Abstract

The application provides an information processing method for intelligent chatting, comprising: obtaining a reply of a specific question by an intelligent robot; performing quality evaluation on the reply to determine whether the quality of the reply meets a set threshold; if not, performing question identification on the reply; and updating the reply according to the result of the question identification. The information processing method for intelligent chatting provided by the application can solve the problem that the answer sent by the intelligent robot to the user cannot solve the original question of the user due to the incorrect question intention identification of the user and the wide answer configuration, and optimize the chatting experience of the user and the intelligent robot.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to an information processing method and device for intelligent chatting, an electronic device and a storage device. BACKGROUND

[0002] With the continuous development of Internet technology, people's expectations for chat robot related products and services are also getting higher and higher. According to the types of products and services, they can be mainly divided into: customer service, entertainment, assistant, education, service, etc. Among them, the intelligent customer service product represented by the conversational intelligent robot has attracted widespread attention. The main function of intelligent customer service is to communicate with users and automatically reply to users' questions about products and services to solve users' problems, reduce enterprise customer service operation cost, and improve user experience, which brings great convenience to enterprises and users.

[0003] In the new retail field represented by e-commerce, there are requirements for the answers replied by intelligent robots, such as granularity, customization, timeliness, etc. Therefore, merchants who usually use intelligent customer service will configure the answers by themselves. The granularity of the configured answers is intended for the granularity of the intent. At present, the intelligent customer service product represented by the conversational intelligent robot usually solves the problem in the following way when answering the question raised by the user: the user inputs the question, and then the intent of the user is judged through intent recognition. For example, during the chatting process between the user and the intelligent robot, the user asks the intelligent robot a question: "When will the express be delivered?" Then, the intelligent robot can judge that the user's intent is to inquire about the delivery time of the express. After determining the user's intent, the intelligent robot matches the answer to the question in the knowledge base according to the user's intent, and sends the answer to the user after determining the answer.

[0004] The above solution chain can meet the demand of merchants for customized answers, but in the actual use process, it is inevitable that the answer sent by the intelligent robot to the user cannot solve the original problem of the user due to the error in the intent recognition of the question raised by the user, the wide configuration of the answer, etc., which will greatly affect the chatting experience between the user and the intelligent robot. SUMMARY

[0005] The embodiments of the present application provide an information processing method and device for intelligent chatting, an electronic device and a storage device to solve the above problems in the intelligent chatting in the prior art.

[0006] The information processing method for intelligent chatting provided by the present application comprises:

[0007] Obtaining a reply of an intelligent robot to a specific question;

[0008] Performing quality evaluation on the reply to determine whether the quality of the reply meets a set threshold;

[0009] If not, performing question identification on the reply;

[0010] Updating the reply according to a result of the question identification.

[0011] Optionally, the quality evaluation on the reply comprises:

[0012] Performing quality evaluation on the reply by an answer quality evaluation model to obtain a quality evaluation score of the reply.

[0013] Optionally, the answer quality evaluation model is obtained by:

[0014] Determining a target domain data pre-trained large-scale language model;

[0015] Taking the target domain data pre-trained large-scale language model as a pre-training model;

[0016] Fine-tuning the target domain data pre-trained large-scale language model according to multi-domain labeled data to obtain the answer quality evaluation model.

[0017] Optionally, the answer quality evaluation model is knowledge distilled.

[0018] Optionally, the updating of the reply according to the result of the question identification comprises:

[0019] Classifying problems existing in the reply according to the result of the question identification;

[0020] Reflowing the problems according to the result of the question classification;

[0021] Updating the reply according to the result of the reflowing.

[0022] Optionally, the determination that the quality of the reply does not meet the set threshold further comprises:

[0023] Recommending, according to conversation data of the intelligent robot and a user, associated questions associated with the specific question, the associated questions being not less than one associated question related to the specific question;

[0024] Determining a target associated question according to the associated questions;

[0025] Obtaining a reply of the intelligent robot to the target associated question according to the target associated question.

[0026] Optionally, the quality of the reply is evaluated to determine whether the quality of the reply meets a set threshold, including:

[0027] If the quality of the reply meets the set threshold, the reply is sent to the questioner.

[0028] Optionally, before obtaining the reply of the intelligent robot to the specific question, further including:

[0029] Obtaining the specific question input by the user;

[0030] Performing intent recognition on the specific question;

[0031] Obtaining the reply of the intelligent robot to the specific question according to the result of the intent recognition.

[0032] In addition, an information processing apparatus for intelligent chat is provided, including:

[0033] An obtaining unit, configured to obtain a reply of an intelligent robot to a specific question;

[0034] A quality evaluation unit, configured to evaluate the quality of the reply to determine whether the quality of the reply meets a set threshold;

[0035] A question identification unit, configured to perform question identification on the reply whose quality does not meet the set threshold;

[0036] An answer updating unit, configured to update the reply according to the result of the question identification.

[0037] In addition, an electronic device is provided, characterized in that, including:

[0038] A processor;

[0039] A memory, configured to store a program for processing information for intelligent chat, the program being read and executed by the processor to perform the following operations:

[0040] Obtaining a reply of an intelligent robot to a specific question;

[0041] Evaluating the quality of the reply to determine whether the quality of the reply meets a set threshold;

[0042] If not, performing question identification on the reply;

[0043] Updating the reply according to the result of the question identification.

[0044] In addition, a computer readable storage medium is provided, which stores computer instructions, characterized in that, the instructions are executed by a processor to implement the following steps:

[0045] obtaining a reply of an intelligent robot to a specific question;

[0046] performing quality evaluation on the reply to determine whether the quality of the reply meets a set threshold;

[0047] if not, performing question identification on the reply;

[0048] updating the reply according to the result of the question identification.

[0049] Compared with the prior art, the information processing method for intelligent chat provided by the application first obtains a reply of an intelligent robot to a specific question raised by a user, then performs quality evaluation on the reply to determine whether the quality of the reply meets a set threshold, if the quality of the reply does not meet the set threshold, question identification needs to be performed on the reply to determine the problems existing in the reply, and then the reply is updated according to the result of the question identification. The information processing method for intelligent chat provided by the application can detect the actual question of the user and the matching degree of the answer in real time by evaluating the quality of the reply, and the reply is updated in a feedback manner, instead of directly sending the answer that may have problems to the user, so that the problem that the answer sent by the intelligent robot to the user cannot solve the original question of the user due to the misrecognition of the question intention raised by the user and the wide configuration of the answer can be solved, and the experience of the user and the intelligent robot in chatting is optimized. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of the information processing method for intelligent chat provided by the embodiment of the application;

[0051] Figure 2 is a scene flowchart provided by the embodiment of the application. DETAILED DESCRIPTION

[0052] In order for those skilled in the art to better understand the technical solutions of the application, the application will be described in detail below with reference to the drawings in the embodiments of the application. However, the application can be implemented in many other ways different from the above description, therefore, based on the embodiments provided by the application, all other embodiments obtained by those skilled in the art without creative labor should belong to the protection scope of the application.

[0053] It should be noted that the terms "first", "second", "third", etc. in the claims, specification and drawings of the present application are used to distinguish similar objects and are not intended to describe a particular order or sequence. The data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include", "have" and their variants are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] In order for those skilled in the art to better understand the scheme of the present application, the specific application scenarios of the embodiments of the dialogue data processing method provided by the present application are described in detail below.

[0055] The present application provides an information processing method and device for intelligent chat, first obtaining the reply of a specific question raised by a user by an intelligent robot, then performing quality evaluation through the reply, judging whether the quality of the reply meets the set threshold, if the quality of the reply does not meet the set threshold, the reply needs to be identified for problems, judging the problems existing in the reply, and then updating the reply according to the result of the judgment of the reply. By designing the answer quality evaluation method in real-time dialogue scene, the reply effect of the full-automatic robot is effectively detected, the reply quality evaluation of the reply that does not meet the standard is updated, instead of directly sending the answer that may have problems to the user, which can solve the problem that the answer sent by the intelligent robot to the user cannot solve the original problem of the user due to the error of the problem intention recognition of the user and the wide answer configuration, achieve the goal of guiding the online answer exposure strategy and the answer configuration problem update, and form a closed loop for the use of dialogue data.

[0056] The first embodiment of the present application provides an information processing method for intelligent chat, as shown in Figure 1 The figure is the flow chart of the first embodiment of the present application. Figure 2 It is the scene flow chart provided by the first embodiment of the present application.

[0057] The first embodiment of the present application provides an information processing method for intelligent chat, as shown in Figure 1 , Figure 2 The first embodiment of the present application provides an information processing method for intelligent chat, as shown in

[0058] Step S101, obtaining the reply of a specific question by an intelligent robot.

[0059] The purpose of this step is to obtain the reply of a specific question by an intelligent robot.

[0060] The intelligent robot can be an intelligent customer service. The intelligent customer service is an online customer service system based on natural language processing technology and automatically solving user problems. The intelligent robot in the intelligent customer service replies answers of the user, which can be a text, an image, a voice, or a rich text answer.

[0061] The premise that the intelligent robot can accurately answer the user's question by using the knowledge base is that the robot can accurately understand the user's question, that is, can perform natural language understanding on the text or voice input by the user. The natural language understanding on the text or voice input by the user requires intent recognition.

[0062] In the embodiment of the present application, the reply of the intelligent robot to the specific question is obtained, that is, the intent of the user is determined according to the question input by the user, and then the answer matched with the intent of the user is selected from the knowledge base according to the intent recognition result.

[0063] The intent refers to the main request or action issued by the user in the chatting process with the intelligent robot, for example, the purpose of the user inputting a question and the answer the user wants to get for the question. The reply of the intelligent robot to the specific question refers to the reply made by the intelligent robot to the question according to the question input by the user.

[0064] As the name implies, intent recognition refers to recognizing the real intent of the language, that is, classifying the question input by the user by an algorithm model to determine the intent of the user. For example, the question of the user is: "How long does the express delivery take?" The recognition result of the intent is: express delivery arrival time.

[0065] Generally, the merchant using the intelligent customer service will configure the answer according to the question of the user. Therefore, the reply of the intelligent customer service to the question input by the user is selected from the answer configured by the merchant in the knowledge base, that is, the intent recognition is performed on the question input by the user, and then the appropriate answer is matched in the knowledge base according to the intent recognition result.

[0066] For example, the reply of the intelligent robot to the question input by the user: "How long does the express delivery take?" First, the intent recognition is performed on the question, and the recognition result of the intent of the question is: express delivery arrival time. Then, the appropriate answer is matched in the knowledge base according to the intent of the express delivery arrival time.

[0067] In step S102, the quality of the reply is evaluated, and it is judged whether the quality of the reply meets the set threshold.

[0068] The answer quality evaluation refers to a process of judging whether an answer can solve a user's problem based on the user's question and the answer automatically replied by the intelligent robot in real time online. In this process, the higher the matching degree between the answer automatically replied by the intelligent robot and the user's question, the higher the quality of the replied answer.

[0069] The answer quality evaluation can be realized by using an answer quality evaluation model. The effect of the answer quality evaluation depends on the accuracy of single sentence evaluation and the coverage of all industry questions. Since the questions input by the user and the answers configured by the merchant have diversity, it is difficult to train an answer quality evaluation model of a general industry. In the embodiments of the present application, in order to solve this generalization problem, a large language model (LLM) based on pre-training of data in a specific field (such as the e-commerce field) is used as a pre-training model, so that the language model is more fitted to the current field knowledge. At the same time, with the help of a large amount of other multi-field (such as all industries) labeled data, the language model is fine-tuned for specific downstream tasks, so that the model is more suitable for specific answer quality evaluation tasks.

[0070] The accuracy of single sentence evaluation refers to the matching degree between the question input by the user and the answer replied by the intelligent robot, that is, whether the reply can accurately solve the user's question. The higher the matching degree, the higher the accuracy of single sentence evaluation. The system based on the question and answer knowledge base has limited answering ability depending on the richness of the knowledge base. Therefore, the higher the coverage of the knowledge base on industry questions, the higher the accuracy of the evaluation.

[0071] The pre-trained language model can learn general language representation and capture the structural knowledge contained in the language. Especially in the low-resource scenario with less labeled data for downstream tasks, the pre-training + fine-tuning mode can bring significant improvement. The input of the pre-trained language model is usually a text sequence segment. The neural encoder encodes the input sequence, and each input unit encodes a corresponding vector representation. The vector representation obtained by pre-training is context-dependent. Since the vector is dynamically calculated by the encoder according to the input, it can capture the context semantic information.

[0072] For example, a certain model is first trained on a large dataset according to some unsupervised tasks, including a next sentence prediction (NSP) task and a masked language model task, which is called pre-training. Then in the fine-tuning stage, based on the pre-trained language model, the model is fine-tuned for subsequent downstream tasks, such as text classification, part-of-speech tagging, question and answer systems, etc. so that the model only needs to adjust the input and output data and the training part of the parameters to achieve good results on different tasks.

[0073] By pre-training a large-scale language model and then fine-tuning the model for each task using task-related labeled data, a task-related model can be obtained to complete the related task. In this embodiment, the model generated by the large-scale language model AliceMind described above is the answer quality evaluation model.

[0074] Model fine-tuning refers to fine-tuning the model on labeled data for a given natural language processing task to quickly improve the model's ability to complete the task, that is, applying the pre-trained model to a given data set so that the parameters adapt to the data of the given task. In this embodiment, the answer quality evaluation task is fine-tuned using the labeled data of all industry trainers to make the model more suitable for the specific answer quality evaluation task.

[0075] The answer quality evaluation model obtained in the above manner may be large in size and occupy a lot of computing resources, such as CPU resources or memory resources, during implementation. To facilitate flexible deployment, the model can be compressed to make it smaller, faster, and occupy fewer resources without affecting its performance. Knowledge distillation is a process of transferring the knowledge learned by a complex model to another simpler model (simple model) that is more suitable for deployment. The knowledge obtained by the complex model is generally obtained through a large amount of actual training data, and the simple model obtains the essence of the knowledge obtained by the complex model. Finally, the simple model will obtain a knowledge processing capability similar to that of the complex model, or at least greatly improve its knowledge processing capability.

[0076] Meanwhile, in the embodiments of the present application, considering the time-consuming problem of online implementation, the large model can be knowledge distilled to control the time consumption of online real-time prediction within 50 milliseconds. The knowledge distillation in this embodiment can use any existing knowledge distillation method, which will not be discussed here.

[0077] Through the above answer quality evaluation model, the quality of the reply of the intelligent robot can be evaluated. Whether the quality of the reply meets the set threshold is determined.

[0078] In step S102, the quality of the reply is evaluated, i.e. the accuracy of the reply of the intelligent robot to the user input question is determined. In one embodiment, the quality of the reply of the intelligent robot to the user input question is scored, and the score can be a probability value between 0 and 1, or other numerical value, which is not particularly limited herein.

[0079] The score of the quality evaluation of the reply can be classified by a pre-set threshold value, to determine whether the quality of the reply of the intelligent robot meets the set threshold value. By classifying the score of the quality evaluation by the threshold value, the answer of the intelligent robot can be divided into a high-quality answer and a non-high-quality answer. That is, if the score of the quality evaluation is greater than or equal to the threshold value, it means that the reply of the intelligent robot can solve the user's problem, and the reply of the intelligent robot is determined to be a high-quality answer. If the score of the quality evaluation is less than the threshold value, it means that the reply of the intelligent robot cannot solve the user's problem, and the reply of the intelligent robot is determined to be a non-high-quality answer.

[0080] For example, in one embodiment of the present application, the score is between 0 and 1, and the threshold value can be set to 0.5. A score greater than or equal to 0.5 is considered to be a high-quality answer, and a score less than 0.5 is considered to be a non-high-quality answer.

[0081] For example, the user input question is "What time will the express delivery be shipped?", and the user's intention is obviously to know the shipping time of the express delivery. The reply of the intelligent robot can be "The express delivery will be shipped tomorrow afternoon", or "The express delivery will be delivered tomorrow".

[0082] For the above answers, the score of the quality of "The express delivery will be shipped tomorrow afternoon" is greater than or equal to 0.5, which means that the reply is a high-quality answer. The score of the quality of "The express delivery will be delivered tomorrow" is less than 0.5, which means that the reply is a non-high-quality answer.

[0083] Step S103, if not, the problem of the reply is identified.

[0084] The purpose of step S103 is to identify the non-high-quality answer, to determine the problem of the non-high-quality answer, so that in the subsequent steps, the non-high-quality answer can be updated according to the problem identified in the non-high-quality answer, to optimize the non-high-quality answer.

[0085] In the new retail field represented by e-commerce, the answers replied by intelligent robots have requirements of fine granularity, customization, timeliness, etc. Therefore, the merchants who usually use intelligent customer service will configure the answers by themselves, and the granularity of the configured answers is intended for the granularity of the intent. In this way, although the demand of the merchants for customized answers can be met, in the actual use process, it is inevitable that the answer sent to the user by the intelligent robot cannot solve the problem input by the user due to the error of the intent recognition of the question raised by the user, the broad configuration of the answer, etc. In the embodiment of the present application, the reply of the intelligent robot which cannot solve the problem input by the user is determined as a non-quality answer, and the problem recognition is performed to determine the problems existing in the reply of the intelligent robot.

[0086] According to an embodiment of step S102, when the scoring result of the quality evaluation is less than a pre-set threshold value, it indicates that the reply of the intelligent robot cannot solve the problem of the user, and it is determined that the reply of the intelligent robot is a non-quality answer, and the problem recognition is performed on the reply of the intelligent robot.

[0087] The problem recognition refers to recognizing the problems existing in the reply of the intelligent robot according to the quality of the reply of the intelligent robot, such as recognizing the problems of the answer redundancy, the answer being too generalized, the intent recognition error, the inaccurate answer content, etc. In the embodiment of the present application, the above-mentioned problems existing in the reply are taken as examples, but the problems existing in the reply are not limited to the above-mentioned several problems.

[0088] In order to facilitate understanding, the following will be described in combination with specific examples, for example, the problem input by the user is: “What time will the express be delivered?”, and the real intent of the user is to know the delivery time of the express.

[0089] For the above-mentioned problem, if the result of the intent recognition is: “What express is delivered”, it is easy to understand that this kind of case is the error of the real intent recognition of the problem input by the user.

[0090] If the result of the intent recognition is: “The delivery time of the express”, the result of the intent recognition is accurate, and if the reply of the intelligent robot is the following several cases, the first case is: Hello, the warehouse delivers the express, please note the express when ordering, and the express will be delivered according to your note, and your express will be sent out at 15:00 tomorrow afternoon. The second case is: We will arrange the warehouse to deliver as soon as possible according to the order sequence of the customer, and the express will be delivered at the latest in 15 days, please wait patiently. The third case is: Your express will be delivered tomorrow.

[0091] For the above three cases, the reply of the first case can inform the user of the delivery time, but the reply content is mixed with irrelevant information such as express delivery, making the reply to the user too redundant and not immediately seeing the answer needed. The problem recognition result of this reply can be classified as answer redundancy. The reply of the second case is obviously that the user wants to know the specific delivery time of the express, and the reply is to arrange delivery as soon as possible according to the order placement order, and the latest delivery within 15 days. The problem of this kind of answer is too general, and the user cannot clearly know the specific time point of delivery. The problem recognition result of this reply can be classified as answer too general. The true intention of the reply of the third case is the delivery time of the express, and the user receives the arrival time of the express. Obviously, although this answer identifies the user's intention as the delivery time of the express, the answer is not for the delivery time. The problem recognition result of this reply is that the answer is not accurate, and the problem recognition result of this reply can be classified as answer not accurate.

[0092] The following describes the case when the quality evaluation score is greater than or equal to the pre-set threshold

[0093] If the quality evaluation score is greater than or equal to the threshold, it means that the reply of the intelligent robot can solve the user's problem, and the reply of the intelligent robot is determined to be a high-quality answer. That is, the reply with a score greater than or equal to 0.5 is considered to be a high-quality answer, and the problem reply is directly sent to the user.

[0094] The following is an example, for example, the user's input question is: "What time does the express deliver?", obviously, the user's intention is to know the delivery time of the express. If the reply of the intelligent robot is "the express will be delivered tomorrow afternoon", for this reply, the quality evaluation result is 0.8 points, which is greater than the set threshold, indicating that the reply of the intelligent robot can accurately solve the user's answer, therefore, the answer belongs to high-quality answer, when the reply belongs to high-quality answer, the reply is directly sent to the user.

[0095] On the other hand, online, when the quality evaluation score is less than the pre-set threshold, it means that the reply of the intelligent robot cannot solve the user's problem, and the reply of the intelligent robot is determined to be a non-high-quality answer. For non-high-quality answers, directly sending to the user cannot solve the user's problem to a certain extent, and will affect the user's question and answer experience.

[0096] Therefore, in the embodiments of the present application, when the scoring result of the quality evaluation is less than the set threshold, the function of knowledge recommendation is provided in the embodiments of the present application. When the quality evaluation result is less than the set threshold, the method of recommending similar associated questions to the user is adopted, so as to make up for the experience problem of the non-high-quality answer.

[0097] The similar question recommendation can be, according to the question input by the user, directly returning an answer to the user by using the similar question recommendation, that is, selecting the most similar question from the question set. The question set is reordered, for example, reordered by using rules, machine learning or deep learning model. Each question is marked with a score. The most similar question to the question input by the user is selected from the similar question set, and the answer to the question is directly returned to the user. For example, the question input by the user is: "What time does the express delivery send?" When the scoring result of the quality evaluation of the question is less than the pre-set threshold, the following questions are sorted, the most similar question is selected, and the answer to the question is sent to the user. For example, the question sorting can be: first, when does the express delivery arrive? Second, what express delivery? Third, how to modify the express delivery address? Correspondingly, the answer to the question of when does the express delivery arrive? is directly sent to the user.

[0098] In another embodiment, the similar question recommendation method can set a threshold to ensure the accuracy of the answer. The most similar question is selected from the question set. The top N questions higher than the threshold are directly sent to the user in the form of a list by sorting. The questions lower than the threshold are directly filtered. The user can select the question he wants from the question set, and then get the specific answer to the question. For example, the question sorting can be: first, when does the express delivery arrive? Second, what express delivery? Third, how to modify the express delivery address? Correspondingly, the user can select any one or more questions to get the answer to the question.

[0099] By the similar question recommendation, when the scoring result of the quality evaluation is less than the pre-set threshold, the experience problem of the non-high-quality answer is made up, and the question and answer experience of the user is improved.

[0100] Step S104: updating the answer according to the result of the question recognition.

[0101] The function of step S104 is to perform question recognition on the non-high-quality answer according to step S103, classify the problems existing in the answer, and then feed back the problems existing in the answer to the merchant, so that the merchant can find the problems existing in the dialogue, guide the merchant to maintain and update the knowledge base, and achieve the effect of data closed loop.

[0102] According to the result of the problem identification, the existing problems in the reply are classified, and the classification result is that the answer redundancy, the answer is too generalized, the intention recognition error, the answer content is inaccurate, and the like exist in the reply. After determining the existing problems in the reply, the result of the problem identification, that is, the existing problems in the reply, is fed back to the merchant, and the merchant re-edits and updates the reply according to the existing problems in the reply, so as to achieve the purpose of optimizing the reply.

[0103] In the above process, by evaluating the quality of the reply and updating the reply in a feedback manner, instead of directly sending the answer that may have problems to the user, the merchant automatically finds the problems from the dialogue, timely maintains and updates the knowledge base, and achieves the effect of data closed loop. The problem that the answer finally sent by the intelligent robot to the user cannot solve the original problem of the user due to the intention recognition error of the question raised by the user and the wide answer configuration can be solved.

[0104] Corresponding to the method for information processing of intelligent chat provided in the embodiments of the present application, the embodiments of the present application further provide an information processing device for intelligent chat. Since the device embodiments are basically similar to the method embodiments, the description is relatively simple, and the related parts refer to the part of the method embodiments. The device embodiments described below are only illustrative. The device for intelligent chat provided in the embodiments of the present application includes the following parts:

[0105] An acquisition unit is configured to acquire a reply of a specific question by an intelligent robot;

[0106] A quality evaluation unit is configured to evaluate the quality of the reply and determine whether the quality of the reply meets a set threshold;

[0107] A problem identification unit is configured to identify problems in the reply whose quality does not meet the set threshold;

[0108] An answer updating unit is configured to update the reply according to the result of the problem identification.

[0109] The embodiments of the present application further provide an electronic device for information processing of intelligent chat, which includes:

[0110] A processor;

[0111] A memory is configured to store a program for processing information of intelligent chat. When the program is read and executed by the processor, the following operations are performed:

[0112] Acquire a reply of a specific question by an intelligent robot;

[0113] performing a quality assessment on the reply to determine whether the quality of the reply meets a set threshold;

[0114] if not, performing a problem identification on the reply;

[0115] updating the reply according to the result of the problem identification.

[0116] The application discloses the above-mentioned preferred embodiments, but is not intended to limit the application. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the application. Therefore, the protection scope of the application should be subject to the scope defined by the claims of the application.

[0117] In one typical arrangement, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0118] The memory can include non-persistent memory, random access memory (RAM), and / or non-volatile memory, such as read only memory (ROM) or flash memory, among others. The memory is an example of computer readable media.

[0119] 1. Computer readable media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer readable media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store information which can be accessed by a computing device. According to the definition provided herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0120] 2. Those skilled in the art will appreciate that embodiments of the application can be practiced in a computer system comprising one or more computers or computing devices. Embodiments of the application can be practiced in a computing system that includes one or more processors, memory, and input / output interfaces such as those found in a computer or computing device. Therefore, embodiments of the application can be practiced in a computing system that includes one or more computers or computing devices. Embodiments of the application can be practiced in a computing system that includes one or more processors, memory, and input / output interfaces such as those found in a computer or computing device. Therefore, embodiments of the application can be practiced in a computing system that includes one or more computers or computing devices. Embodiments of the application can be practiced in a computing system that includes one or more processors, memory, and input / output interfaces such as those found in a computer or computing device. Therefore, embodiments of the application can be practiced in a computing system that includes one or more computers or computing devices. Embodiments of the application can be practiced in a computing system that includes one or more processors, memory, and input / output interfaces such as those found in a computer or computing device. Therefore, embodiments of the application can be practiced in a computing system that includes one or more computers or computing devices. Embodiments of the application can be practiced in a computing system that includes one or more processors, memory, and input / output interfaces such as those found in a computer or computing device. Therefore, embodiments of the application can be practiced in a computing system that includes one or more computers or computing devices. Embodiments of the application can be practiced in a computing system that includes one or more processors, memory, and input / output interfaces such as those found in a computer or computing device. Therefore, embodiments of the application can be practiced in a computing system that includes one or more computers or computing devices.

Claims

1. An information processing method for intelligent chat, characterized in that, include: Get answers from intelligent robots for specific questions; The response is evaluated using an answer quality assessment model to obtain a quality assessment score, and it is determined whether the quality of the response meets the set threshold. If the conditions are not met, then the response is subjected to problem identification to identify problems such as intent recognition error, redundant answer, overly generalized answer, or inaccurate answer content. The response is updated based on the results of the problem identification.

2. The information processing method for intelligent chat according to claim 1, characterized in that, The answer quality assessment model was obtained through the following methods: Identify large-scale language models pre-trained on target domain data; A large-scale language model pre-trained with the target domain data is used as the pre-trained model; The answer quality assessment model is obtained by fine-tuning a large-scale language model pre-trained on the target domain data based on labeled data from multiple domains.

3. The information processing method for intelligent chat according to claim 2, characterized in that... Also includes: Knowledge distillation is performed on the aforementioned answer quality assessment model.

4. The information processing method for intelligent chat according to any one of claims 1-3, characterized in that, The step of updating the response based on the result of the question identification includes: The problems in the response are classified according to the problem identification results; The questions will be returned based on the question classification results; The response is updated based on the feedback results.

5. The information processing method for intelligent chat according to claim 1, characterized in that, Determining that the quality of the response does not meet the set threshold also includes: Based on the dialogue data between the intelligent robot and the user, the system recommends related questions that are associated with the specific question, wherein there is at least one related question that is associated with the specific question; Determine the target related question based on the aforementioned related questions; The intelligent robot's response to the target association question is obtained based on the target association question.

6. The information processing method for intelligent chat according to claim 1, characterized in that, The response is evaluated for quality to determine whether its quality meets a set threshold, including: If the quality of the response meets the set threshold, the response will be sent to the questioner.

7. The information processing method for intelligent chat according to claim 1, characterized in that, Before obtaining the AI's answer to a specific question, the following steps are also included: Get specific questions from user input; Intent identification is performed on the specific question; The intelligent robot's response to a specific question is obtained based on the intent recognition results.

8. An information processing device for intelligent chat, characterized in that, include: The acquisition unit is used to acquire the intelligent robot's response to a specific question; The quality assessment unit is used to assess the quality of the response using an answer quality assessment model, obtain a quality assessment score for the response, and determine whether the quality of the response meets a set threshold. The problem identification unit is used to identify problems in the responses whose quality does not meet a set threshold, such as intention recognition errors, redundant answers, overly generalized answers, or inaccurate answer content. The answer update unit is used to update the response based on the result of the question identification.

9. An electronic device, characterized in that, include: processor; A memory for storing a processing program for intelligent chat, which, when read and executed by the processor, performs the following operations: Get answers from intelligent robots for specific questions; The response is evaluated using an answer quality assessment model to obtain a quality assessment score, and it is determined whether the quality of the response meets the set threshold. If the conditions are not met, then the response is subjected to problem identification to identify problems such as intent recognition error, redundant answer, overly generalized answer, or inaccurate answer content. The response is updated based on the results of the problem identification.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, When this instruction is executed by the processor, it performs the following steps: Get answers from intelligent robots for specific questions; The response is evaluated using an answer quality assessment model to obtain a quality assessment score, and it is determined whether the quality of the response meets the set threshold. If the conditions are not met, then the response is subjected to problem identification to identify problems such as intent recognition error, redundant answer, overly generalized answer, or inaccurate answer content. The response is updated based on the results of the problem identification.

Images