Method, device and terminal device for generating feedback information
By determining the answer generation model based on the preset corpus in the online intelligent consulting service, and using standard question and answer data and similar question data to generate target answer information, the problem of low matching of feedback information in online consulting services is solved, and the accuracy of information is improved and user experience.
Patent Information
- Application Number
- CN202010615084.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-06-30
AI Technical Summary
The existing online intelligent consulting services cannot adapt to changes in database data volume, resulting in poor matching of the accuracy of the output feedback information and user needs, affecting the user experience.
By determining the answer generation model based on the amount of corpus data in the preset corpus, using standard question and answer data and similar question data, the target answer information is generated, and the feedback information is determined based on the matching degree of the target answer information and the user's question information to improve the matching degree and accuracy Spend.
It improves the matching and accuracy of feedback information with user needs, improves the user experience, better adapts to the expression methods of different users, and enhances the efficiency and effectiveness of online consultation services.
Smart Images

Figure CN111767382B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent decision-making technology, and in particular to a method for generating feedback information, an apparatus for generating feedback information, a terminal device, and a computer-readable storage medium. Background Art
[0002] In order to better serve the public, various enterprises, government departments, etc. will provide online intelligent consulting services. However, the amount of data and other information in the database corresponding to the online intelligent consulting service will continue to change with the maintenance of the staff, but the current online intelligent consulting services often cannot adapt to the changes in the amount of data in the database, resulting in poor accuracy of the output feedback information and poor matching with user needs, affecting the user experience. Summary of the invention
[0003] In view of this, the embodiments of the present application provide a method for generating feedback information, an apparatus for generating feedback information, a terminal device and a computer-readable storage medium, which can improve the matching degree between the feedback information output during the online intelligent consultation process and the user needs.
[0004] In a first aspect, an embodiment of the present application provides a method for generating feedback information, including:
[0005] Determine an answer generation model according to the amount of corpus data of a preset corpus, wherein the preset corpus includes standard question and answer data and similar question data, the standard question and answer data includes at least one standard question and standard answers corresponding to each of the standard questions, and the similar question data includes similar questions corresponding to each of the standard questions;
[0006] Determining target answer information from the preset corpus according to the user question information through the answer generation model;
[0007] The matching degree between the target answer information and the user question information is used as the target matching degree;
[0008] Feedback information to the user is determined based on the target answer information and the target matching degree.
[0009] In a second aspect, an embodiment of the present application provides a device for generating feedback information, including:
[0010] A first determination module is used to determine an answer generation model according to the amount of corpus data of a preset corpus, wherein the preset corpus includes standard question and answer data and similar question data, the standard question and answer data includes at least one standard question and a standard answer corresponding to each of the standard questions, and the similar question data includes similar questions corresponding to each of the standard questions;
[0011] A second determination module is used to determine target answer information from the preset corpus according to the user question information through the answer generation model;
[0012] A processing module, configured to use the matching degree between the target answer information and the user question information as a target matching degree;
[0013] The third determination module is used to determine feedback information to the user according to the target answer information and the target matching degree.
[0014] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, a display, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for generating feedback information as described in the first aspect when executing the computer program.
[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for generating feedback information as described in the first aspect is implemented.
[0016] The method for generating feedback information, the device for generating feedback information, the terminal device, and the computer-readable storage medium provided by the embodiments of the present application have the following beneficial effects:
[0017] In an embodiment of the present application, an answer generation model can be determined based on the amount of corpus data in a preset corpus, and then the target answer information can be determined from the preset corpus based on the user's question information through the answer generation model, wherein, since the answer generation model is determined based on the amount of corpus data in the preset corpus, the adopted answer generation model can establish a more accurate correspondence between user questions and preset answers based on the preset corpus, thereby obtaining target answer information that better matches the user's question information based on the preset corpus, and the target corpus includes standard question and answer data and similar question data, wherein the standard question and answer data includes at least one standard question and the corresponding answers of each standard question. The corresponding standard answer, the similar question data includes the similar questions corresponding to each standard question. Therefore, any standard answer can have corresponding standard questions and similar questions, so as to better adapt to the differentiated expression methods of different users and more accurately match user questions; then, the matching degree between the target answer information and the user question information is used as the target matching degree, and then the feedback information to the user is determined according to the target answer information and the target matching degree. It can be determined whether the target answer information meets the expectations according to the target matching degree, thereby determining the feedback information to the user, so as to improve the matching degree between the output feedback information and the user needs, thereby improving the accuracy of the output feedback information and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0019] Figure 1 It is a flowchart of a method for generating feedback information provided by an embodiment of the present application;
[0020] Figure 2 is a flowchart of another method for generating feedback information provided by an embodiment of the present application;
[0021] Figure 3 is a flowchart of another method for generating feedback information provided by an embodiment of the present application;
[0022] Figure 4 is a structural diagram of a device for generating feedback information provided by an embodiment of the present application;
[0023] Figure 5 It is a structural diagram of a terminal device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0024] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0025] The method for generating feedback information provided in the embodiment of the present application can be applied to terminal devices such as servers, desktop computers, mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, personal digital assistants (PDA), etc. The embodiment of the present application does not impose any restrictions on the specific type of the terminal device.
[0026] See also Figure 1 , Figure 1 A flow chart of an information processing method provided by an embodiment of the present application is shown.
[0027] The embodiments of the present application can be applied to various business systems, wherein the specific application scenario of the business system can be determined according to actual needs. In some embodiments, the business system can be a shopping system, a government service system (such as a financial system), a management system for a specific application field, and the like.
[0028] In a specific example, a specific application scenario of this embodiment can be an online financial consulting service for a financial service department. Since the relevant processes and services in the financial field are highly professional, the service capabilities of online consulting are required to be high, resulting in high labor costs and time costs for related operations and maintenance. Through the embodiments of this application, online consulting services for related financial policies, processes and other matters can be efficiently and conveniently implemented, and the matching degree between the output feedback information and user needs can be improved, thereby greatly improving the user experience of related users.
[0029] like Figure 1 As shown, an information processing method provided by this embodiment may include:
[0030] Step S101, determining an answer generation model based on the amount of corpus data in a preset corpus, wherein the preset corpus includes standard question and answer data and similar question data, the standard question and answer data includes at least one standard question and standard answers corresponding to each of the standard questions, and the similar question data includes similar questions corresponding to each of the standard questions.
[0031] In the embodiment of the present application, the standard question and answer data and similar question and answer data in the preset corpus may be collected in advance based on manual experience, or generated based on an artificial intelligence algorithm such as a specific machine learning algorithm. There may be multiple types of answer generation models. For example, in some examples, it may be an answer generation model based on a statistical method, an answer generation model based on a probability network, or an answer generation model based on a specific machine learning algorithm, and other artificial intelligence algorithm models.
[0032] Among them, illustratively, the standard question and answer data can be manually sorted out based on the work experience of the operation and maintenance personnel, and can also be obtained based on the consultation records between the user and the operation and maintenance personnel. Specifically, based on the preset rule algorithm, the user's questions and the corresponding answers of the operation and maintenance personnel can be extracted from the consultation records, and which of the extracted questions and corresponding answers can be used as standard question and answer data can be manually confirmed, thereby combining the preset rule algorithm and manual semi-automatic extraction of standard question and answer data. The similar questions can be obtained by manually expanding the standard questions, or they can be generated by using the generative network in the trained generative adversarial network.
[0033] In an embodiment of the present application, different answer generation models are pre-selected according to the size of the corpus data in actual situations. A more appropriate answer generation model can be selected to generate answers according to the size of the corpus data in the preset corpus, thereby improving the degree of match between the obtained target answer information and the user's question.
[0034] In some embodiments, before determining the target answer information from the preset corpus according to the user question information by using the answer generation model, the method further includes:
[0035] Obtaining the standard question and answer data;
[0036] According to the standard question-answering data, similar questions corresponding to each standard question are generated by a generative network in a trained generative adversarial network;
[0037] The target corpus is obtained according to the standard question and answer data and the similar questions.
[0038] In an embodiment of the present application, the similar questions can be generated by using a generative network in a trained generative adversarial network. The generative adversarial network includes a generative network G (Generator) and a discriminator network D (Discriminator). The generative network G can be used to generate questions.
[0039] Specifically, the generator network G can receive a random input z and generate a sentence through this random input, which is recorded as G(z). The discriminant network D is used to determine whether the sentence generated by the generator network G is a standard question. Its input parameter is G(z), and the output D(x) represents the probability that G(z) is a standard question. If the output D(x) is 1, it means that the generator network G generates a sentence consistent with the standard question from the random input z; and if the output D(x) is 0, it means that the generated sentence G(z) cannot be a standard question. During the training process, the goal of the generator network G is to generate real questions as much as possible to deceive the discriminant network D, and the goal of D is to distinguish the sentences generated by the generator network G from the real standard questions as much as possible. After the training is completed, the generator network G can generate sentences G(z) with a high enough credibility. For the discriminant network D, it is difficult to determine whether the sentence generated by the generator network G is a real standard question. Specifically, the corresponding loss function can be used to determine whether the training is completed.
[0040] After the training is completed, similar questions of each standard question in the preset corpus can be generated by the generative network G in the trained generative adversarial network to expand the preset corpus.
[0041] During the consultation process, different users often have different ways of expressing the same question. Therefore, by obtaining similar questions corresponding to each standard question, the forms of questioning stored in the preset corpus can be expanded to better identify the differentiated expressions of different users and match more accurate target answer information.
[0042] In some embodiments, determining the answer generation model according to the amount of corpus data in a preset corpus includes:
[0043] If the amount of corpus data of the target corpus is in the first data interval, determining that the answer generation model is a preset data model based on the Jaccard coefficient;
[0044] If the amount of corpus data of the target corpus is in a second data interval, determining that the answer generation model is a preset probabilistic network model, wherein the lower limit of the second data interval is not less than the upper limit of the first data interval;
[0045] If the amount of corpus data in the target corpus is in a third data interval, the answer generation model is determined to be a hybrid model, wherein the hybrid model includes a generation model and a retrieval model, and the lower limit of the third data interval is not less than the upper limit of the second data interval.
[0046] In the embodiment of the present application, the preset data model based on the Jaccard coefficient can calculate the similarity between character strings by the Jaccard coefficient. At this time, when the amount of corpus data is small, the preset data model based on the Jaccard coefficient can quickly and accurately detect the target question and corresponding target answer information in the preset corpus that has the highest similarity with the user's question information and is the most matched.
[0047] The preset probability network model may be, for example, a Bayesian network, a Markov network, or a hidden Markov network, etc. In order to improve the performance of the preset probability network, the preset probability network model may be a preset probability network model based on Embedding. Among them, the Embedding is a way to convert discrete variables into continuous vectors. Through the Embedding, rapid screening and matching of similar problems can be achieved. Therefore, with a certain amount of corpus data, the target problem can still be detected relatively quickly and accurately. The specific setting of the second data interval can be determined based on actual experience, testing, etc.
[0048] The hybrid model may include a generation model and a retrieval model. Among them, the user question information can be input into the generation model, and the initial answer output by the generation model according to the user question information can be obtained. The initial answer is input into the retrieval model, and the retrieval model can use NLP semantic matching and other methods to retrieve the initial answer and the answer stored in the database, that is, select the answer with the greatest similarity as the output, that is, obtain the target answer information corresponding to the user question information and the target matching degree. Exemplarily, the generation model can adopt an Encoder-Decoder structure based on BiLSTM and attention mechanism. Compared with a single model, the hybrid model can achieve at least two levels of screening and matching. Therefore, the hybrid model can identify more matching target questions and target answer information from a preset corpus with a large amount of data.
[0049] Step S102, determining target answer information from the preset corpus according to the user's question information through the answer generation model.
[0050] In the embodiment of the present application, for example, the user question information may be input by the user into the corresponding business system through a specific front-end interface such as a dialog box; in addition, it may also be generated based on the user's display operation on the relevant display interface, for example, it may be generated based on the abnormal operation after the user's abnormal operation is detected.
[0051] In some examples, the user question information may be obtained after information extraction is performed on specific user input information. Specifically, the user input information may be obtained, and information extraction may be performed on the user input information to obtain the user question information. Wherein, the user input information may include text information and / or pictures. If the user input information includes text information, keyword extraction may be performed on the text information to obtain the user question information. If the user input information includes pictures, image feature extraction may be performed on the picture information or text content in the picture information may be obtained by optical character recognition (OCR) and the like, thereby obtaining the information in the picture as the user question information. Wherein, the picture may be stored locally, or the user may take a screenshot of the display interface containing the prompt information, or the business system may automatically take a screenshot after detecting a pop-up window containing prompt information about abnormal operations. There may be multiple specific ways to obtain the picture.
[0052] In an embodiment of the present application, there are multiple ways to determine the target answer information from the preset corpus. For example, the target question with the highest similarity to the user question information can be determined from the preset corpus by the answer generation model, the similarity between the target question and the user question information can be used as the target matching degree, and the target answer information corresponding to the target question can be obtained from the preset corpus. In addition, the user question information can also be input into the answer generation model to obtain the target answer information obtained by the answer generation model.
[0053] Among them, the specific acquisition method is different depending on the selected answer generation model. For example, if the answer generation model is a preset data model based on the Jaccard coefficient, the similarity between the user question information and each preset question in the preset corpus can be calculated by the Jaccard coefficient. If the answer generation model is a preset probabilistic network model, the user question information can be input into the preset probabilistic network model to obtain the target question with the highest similarity to the user question information output by the probabilistic network model, so as to determine the target answer information and the target matching degree according to the target question. If the answer generation model is a hybrid model, the user question information can be input into the hybrid model to obtain the target answer information and the target matching degree output by the hybrid model.
[0054] In some embodiments, before determining the target answer information from the preset corpus according to the user question information by using the answer generation model, the method further includes:
[0055] Step S201, if it is detected that a prompt window is displayed in the operation interface of the preset business system, a screenshot operation is performed on the operation interface to obtain a screenshot image, wherein the prompt window contains information about abnormal operations in the operation interface;
[0056] Step S202, performing edge detection on the screenshot image to determine the area where the prompt window is located;
[0057] Step S203, performing text recognition on the area where the prompt window is located to obtain target text information in the prompt window;
[0058] Step S204: extracting identification information of the abnormal operation from the target text information, and using the identification information of the abnormal operation as at least part of the user question information.
[0059] In the embodiment of the present application, illustratively, the abnormal operation can be an erroneous operation of the user, an operation with too fast frequency, or an operation with abnormal results, etc., and the classification of the abnormal operation can be determined according to the specific business scenario. After detecting the abnormal operation, the relevant business system often generates and displays a prompt window according to the pre-settings and the abnormal operation to notify the user. Exemplarily, the prompt window may include the abnormal number, abnormal operation content, abnormal operation result and other information corresponding to the user's abnormal operation. Therefore, by taking a screenshot of the operation interface to obtain a screenshot image, the operation information of the abnormal operation can be automatically captured and information can be extracted, and at least one of the obtained abnormal number, abnormal operation content, abnormal operation result and other information can be used as at least part of the content of the user's question information, without the user having to organize the language by himself and then input it into the relevant business system, which greatly improves the efficiency of information acquisition.
[0060] In the prior art, the user questioning method used in online consultation is usually only to input text information, that is, the corresponding application can often only receive text input information input by the user, but cannot receive and process other types of input information, let alone automatically obtain it. In the embodiment of the present application, the source of the user question information is not limited to text information, but can obtain the corresponding business system interface information and extract information as user question information through screenshots and other methods, which improves the diversity of online consultation methods and the convenience of online consultation operations.
[0061] In some embodiments, before determining the target answer information from the preset corpus according to the user question information by using the answer generation model, the method further includes:
[0062] Step S301, if a consultation dialog box pops up in the display interface, at least one estimated question is generated and displayed according to the user's historical browsing history;
[0063] Step S302: taking the estimated question selected by the user as at least part of the user question information.
[0064] The consultation dialog box may pop up after detecting a user's click operation on an intelligent customer service icon displayed in a business system. There are many specific ways to generate the estimated questions. For example, keywords of the relevant interface of the historical browsing record may be extracted, and the estimated questions may be generated based on the keywords. In addition, statistics may be collected on information such as the category of the browsing object corresponding to the user's historical browsing record, and the estimated questions may be generated based on the statistical results.
[0065] Step S103: taking the matching degree between the target answer information and the user question information as the target matching degree.
[0066] In an embodiment of the present application, the target matching degree can be used to reflect whether the target answer information meets expectations, so as to subsequently determine feedback information for the user to improve the matching degree between the output feedback information and the user's needs.
[0067] Step S104: Determine feedback information for the user according to the target answer information and the target matching degree.
[0068] In the embodiment of the present application, for target matching degrees in different preset intervals, different feedback methods can be adopted to obtain different feedback information. For example, if the target matching degree is high (such as greater than 90%), the feedback information may include the target answer information; but if the target matching degree is low, such as less than 40%, the feedback information may include other information, such as guidance information for guiding the user to ask further questions, etc.
[0069] In some embodiments, the determining of feedback information to the user based on the target answer information and the target matching degree includes:
[0070] If the target matching degree is less than the first matching degree, determining that the feedback information includes indication information indicating that the user's question cannot be answered, and / or includes inquiry information asking the user whether to transfer to manual service;
[0071] If the target matching degree is not less than the first matching degree and less than the second matching degree, searching the preset prediction library for N questions with the highest similarity to the user's question information, taking the N questions with the highest similarity as candidate questions and displaying them in the consultation dialog box to inquire whether the question the user wants to ask is among the displayed N candidate questions, wherein N is a positive integer;
[0072] If the target matching degree is not less than the second matching degree, it is determined that the feedback information includes the target answer information.
[0073] In an embodiment of the present application, whether the target answer information meets expectations is determined based on the target matching degree, thereby determining feedback information for the user. This can improve the matching degree between the output feedback information and user needs, thereby improving the accuracy of the output feedback information and improving the user experience.
[0074] In some embodiments, the method for generating feedback information further includes:
[0075] Counting session information within a first time period, wherein the session information includes user question information and corresponding feedback information within the first time period;
[0076] Comparing the session information with historical session information to detect whether the session information is abnormal, wherein the historical session information is generated earlier than the session information;
[0077] If an abnormality is detected in the session information, an abnormality prompt message is generated.
[0078] In the embodiments of the present application, for example, outlier analysis can be performed by means of Z-scores, etc., to detect whether the session information is abnormal. Among them, the Z-score, also known as the standard score, measures the degree of deviation between each data in a set of data and the average value of the entire set of data, that is, the distance or deviation of a specific data from the average value is expressed in units of standard deviation. In addition, other algorithms can also be used, such as calculating changes in characteristic data such as the amount of consultations, the number of consultations, etc., to detect whether the session information is abnormal. For example, the amount of data between the session information and the historical session information, the frequency of occurrence of keywords (such as negative reviews), etc., can be compared to determine whether the session information is abnormal. The method for determining whether the session information is abnormal can be determined according to the actual scenario requirements, etc. The abnormal prompt information can be used to prompt relevant management personnel to conduct investigations.
[0079] In some embodiments, the method for generating feedback information further includes:
[0080] Counting the number of occurrences of each question in the user question information within the second time period;
[0081] Sort the questions according to the order of the number of occurrences from most to least, and obtain the current sorting result;
[0082] A report is generated according to the current sorting result.
[0083] In addition, the current sorting result can be compared with the historical sorting result to obtain a comparison result, and a report can be generated based on the sorting result and the comparison result, so that relevant managers can find problems in time. For example, the report can point out that the number of occurrences of question A in business system X has increased significantly compared with the previous time period, so the administrator can prioritize the maintenance of the problems involved in question A in business system X based on the report to improve user satisfaction.
[0084] In some embodiments, unanswered user questions can be collected based on the user's reply to the feedback information, and the unanswered user questions can be sent to relevant managers in the form of reports, tables, documents, etc. at a certain period.
[0085] In some embodiments, the unresolved user question information and its corresponding answers reorganized by relevant managers can be further received to iteratively train the corresponding answer generation model, such as iteratively training the generative adversarial network, the preset probabilistic network model, the hybrid model including the generation model and the retrieval model, etc. Through iteration, new content can be added to the existing answer generation model to improve maintenance efficiency.
[0086] In an embodiment of the present application, an answer generation model can be determined based on the amount of corpus data in a preset corpus, and then the target answer information can be determined from the preset corpus based on the user's question information through the answer generation model, wherein, since the answer generation model is determined based on the amount of corpus data in the preset corpus, the adopted answer generation model can establish a more accurate correspondence between user questions and preset answers based on the preset corpus, thereby obtaining target answer information that better matches the user's question information based on the preset corpus, and the target corpus includes standard question and answer data and similar question data, wherein the standard question and answer data includes at least one standard question and the corresponding answers of each standard question. The corresponding standard answer, the similar question data includes the similar questions corresponding to each standard question. Therefore, any standard answer can have corresponding standard questions and similar questions, so as to better adapt to the differentiated expression methods of different users and more accurately match user questions; then, the matching degree between the target answer information and the user question information is used as the target matching degree, and then the feedback information to the user is determined according to the target answer information and the target matching degree. It can be determined whether the target answer information meets the expectations according to the target matching degree, thereby determining the feedback information to the user, so as to improve the matching degree between the output feedback information and the user needs, thereby improving the accuracy of the output feedback information and improving the user experience.
[0087] See also Figure 4 , Figure 4 1 is a structural block diagram of a device for generating feedback information provided by an embodiment of the present application. In this embodiment, the terminal device includes each unit for executing each step in each of the above-mentioned methods for generating feedback information. For details, please refer to the relevant description in the embodiment corresponding to the above-mentioned method for generating feedback information. For ease of explanation, only the part related to this embodiment is shown.
[0088] Reference Figure 4 , the device 4 for generating feedback information comprises:
[0089] A first determination module 401 is used to determine an answer generation model according to the amount of corpus data of a preset corpus, wherein the preset corpus includes standard question and answer data and similar question data, the standard question and answer data includes at least one standard question and a standard answer corresponding to each of the standard questions, and the similar question data includes similar questions corresponding to each of the standard questions;
[0090] A second determination module 402 is used to determine target answer information from the preset corpus according to the user question information through the answer generation model;
[0091] Processing module 403, used for taking the matching degree between the target answer information and the user question information as the target matching degree;
[0092] The third determination module 404 is used to determine feedback information to the user according to the target answer information and the target matching degree.
[0093] Optionally, the device 4 for generating feedback information further includes:
[0094] A screenshot module, for, if it is detected that a prompt window is displayed in the operation interface of the preset business system, taking a screenshot of the operation interface to obtain a screenshot image, wherein the prompt window contains information about abnormal operations in the operation interface;
[0095] An edge detection module, used to perform edge detection on the screenshot image to determine the area where the prompt window is located;
[0096] A text recognition module, used for performing text recognition on the area where the prompt window is located to obtain target text information in the prompt window;
[0097] The extraction module is used to extract the identification information of the abnormal operation from the target text information, and use the identification information of the abnormal operation as at least part of the user question information.
[0098] Optionally, the device 4 for generating feedback information further includes:
[0099] A second processing module is used to generate and display at least one estimated question based on the user's historical browsing history if a consultation dialog box pops up in the display interface;
[0100] A setting module is used to use the estimated question selected by the user as at least part of the user question information.
[0101] Optionally, the first determining module 401 specifically includes:
[0102] a first determining unit, configured to determine that the answer generation model is a preset data model based on a Jaccard coefficient if the amount of corpus data of the target corpus is in a first data interval;
[0103] a second determining unit, configured to determine that the answer generation model is a preset probabilistic network model if the amount of corpus data of the target corpus is in a second data interval, wherein a lower limit of the second data interval is not less than an upper limit of the first data interval;
[0104] The third determination unit is used to determine that the answer generation model is a hybrid model if the amount of corpus data in the target corpus is in a third data interval, wherein the hybrid model includes a generation model and a retrieval model, and the lower limit of the third data interval is not less than the upper limit of the second data interval.
[0105] Optionally, the device 4 for generating feedback information further includes:
[0106] An acquisition module, used to acquire the standard question and answer data;
[0107] A first generation module is used to generate similar questions corresponding to each standard question respectively through a generative network in a trained generative adversarial network according to the standard question-answering data;
[0108] The third processing module is used to obtain the target corpus according to the standard question and answer data and the similar questions.
[0109] Optionally, the second determining module 403 specifically includes:
[0110] a fourth determining unit, configured to determine, if the target matching degree is less than the first matching degree, that the feedback information includes indication information indicating that the user's question cannot be answered, and / or includes inquiry information asking the user whether to transfer to manual service;
[0111] A display unit, configured to search the preset prediction library for N questions with the highest similarity to the user's question information if the target matching degree is not less than the first matching degree and less than the second matching degree, use the N questions with the highest similarity as candidate questions and display them in a consultation dialog box to inquire whether the question to be asked by the user is among the displayed N candidate questions, wherein N is a positive integer;
[0112] A fifth determining unit is configured to determine that the feedback information includes the target answer information if the target matching degree is not less than the second matching degree.
[0113] Optionally, the device 4 for generating feedback information further includes:
[0114] A statistics module, used to count the session information in the first time period, wherein the session information includes user question information and corresponding feedback information in the first time period;
[0115] A comparison module, used for comparing the session information with historical session information to detect whether there is any anomaly in the session information, wherein the generation time of the historical session information is earlier than that of the session information;
[0116] The second generating module is used to generate abnormal prompt information if it is detected that the session information is abnormal.
[0117] In an embodiment of the present application, an answer generation model can be determined based on the amount of corpus data in a preset corpus, and then the target answer information can be determined from the preset corpus based on the user's question information through the answer generation model, wherein, since the answer generation model is determined based on the amount of corpus data in the preset corpus, the adopted answer generation model can establish a more accurate correspondence between user questions and preset answers based on the preset corpus, thereby obtaining target answer information that better matches the user's question information based on the preset corpus, and the target corpus includes standard question and answer data and similar question data, wherein the standard question and answer data includes at least one standard question and the corresponding answers of each standard question. The corresponding standard answer, the similar question data includes the similar questions corresponding to each standard question. Therefore, any standard answer can have corresponding standard questions and similar questions, so as to better adapt to the differentiated expression methods of different users and more accurately match user questions; then, the matching degree between the target answer information and the user question information is used as the target matching degree, and then the feedback information to the user is determined according to the target answer information and the target matching degree. It can be determined whether the target answer information meets the expectations according to the target matching degree, thereby determining the feedback information to the user, so as to improve the matching degree between the output feedback information and the user needs, thereby improving the accuracy of the output feedback information and improving the user experience.
[0118] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.
[0119] Figure 5 is a structural block diagram of a terminal device provided by another embodiment of the present application. Figure 5As shown, the terminal device 5 of this embodiment includes: a processor 51, a memory 52, and a computer program 53 stored in the memory 52 and executable on the processor 51, such as a program for a method for generating feedback information. When the processor 51 executes the computer program 53, the steps in each embodiment of the method for generating feedback information are implemented, such as Figure 1 S101 to S104 shown, or Figure 2 S201 to S204 shown, or Figure 3 Alternatively, the processor 51 implements the above when executing the computer program 53 Figure 4 The functions of each unit and / or each module in the corresponding embodiment are, for example, Figure 4 For details on the functions of modules 401 to 404, please refer to Figure 4 The relevant descriptions in the corresponding embodiments are not repeated here.
[0120] Exemplarily, the computer program 53 may be divided into one or more units, which are stored in the memory 52 and executed by the processor 51 to complete the present application. The one or more units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 53 in the terminal device 50. For example, the computer program 53 may be divided into a first determination module, a second determination module, a processing module, and a third determination module, and the specific functions of each unit are as described above.
[0121] The terminal device may include, but is not limited to, a processor 51 and a memory 52. Those skilled in the art will appreciate that Figure 5 It is only an example of the terminal device 5 and does not constitute a limitation on the terminal device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the turntable device may also include input and output devices, network access equipment, buses, etc.
[0122] The processor 51 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0123] The memory 52 may be an internal storage unit of the terminal device 5, such as a hard disk or memory of the terminal device 5. The memory 52 may also be an external storage device of the terminal device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 5. Further, the memory 52 may also include both an internal storage unit and an external storage device of the terminal device 5. The memory 52 is used to store the computer program and other programs and data required by the turntable device. The memory 52 may also be used to temporarily store data that has been output or is to be output.
[0124] In an embodiment of the present application, when the processor 50 executes the computer program 52 to implement the steps in any of the above-mentioned methods for generating feedback information, an answer generation model can be determined according to the amount of corpus data in a preset corpus, and then the target answer information can be determined from the preset corpus according to the user's question information through the answer generation model, wherein, since the answer generation model is determined according to the amount of corpus data in the preset corpus, the answer generation model adopted can establish a more accurate correspondence between user questions and preset answers based on the preset corpus, thereby obtaining target answer information that better matches the user's question information based on the preset prediction library, and the target corpus includes standard question and answer data and similar question data, and the standard question and answer data The system includes at least one standard question and a standard answer corresponding to each of the standard questions, and the similar question data includes similar questions corresponding to each of the standard questions. Therefore, any standard answer can have a corresponding standard question and a similar question, so as to better adapt to the differentiated expression methods of different users and more accurately match user questions. Then, the matching degree between the target answer information and the user question information is used as the target matching degree, and then the feedback information to the user is determined according to the target answer information and the target matching degree. It can be determined whether the target answer information meets the expectations according to the target matching degree, thereby determining the feedback information to the user, so as to improve the matching degree between the output feedback information and the user needs, thereby improving the accuracy of the output feedback information and improving the user experience.
[0125] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments for generating feedback information can be implemented.
[0126] An embodiment of the present application provides a computer program product. When the computer program product runs on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments for generating feedback information when executing the computer program product.
[0127] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.
Claims
1. A method for generating feedback information, characterized in that: The method is used for online intelligent consultation, and includes: Determine an answer generation model according to the amount of corpus data of a preset corpus, wherein the preset corpus includes standard question and answer data and similar question data, the standard question and answer data includes at least one standard question and standard answers corresponding to each of the standard questions, and the similar question data includes similar questions corresponding to each of the standard questions; Determining target answer information from the preset corpus according to the user question information through the answer generation model; The matching degree between the target answer information and the user question information is used as the target matching degree; Determining feedback information to the user according to the target answer information and the target matching degree; Before determining target answer information from the preset corpus according to the user question information by using the answer generation model, the method further includes: If it is detected that a prompt window is displayed in the operation interface of the preset business system, a screenshot operation is performed on the operation interface to obtain a screenshot image, wherein the prompt window contains information about abnormal operations in the operation interface; Performing edge detection on the screenshot image to determine the area where the prompt window is located; Performing text recognition on the area where the prompt window is located to obtain target text information in the prompt window; extracting identification information of the abnormal operation from the target text information, and using the identification information of the abnormal operation as at least part of the user question information; The step of determining the answer generation model according to the amount of corpus data in the preset corpus includes: If the amount of corpus data in the target corpus is in the first data interval, determining that the answer generation model is a preset data model based on the Jaccard coefficient; If the amount of corpus data of the target corpus is in a second data interval, determining that the answer generation model is a preset probabilistic network model, wherein the lower limit of the second data interval is not less than the upper limit of the first data interval; If the amount of corpus data in the target corpus is in a third data interval, the answer generation model is determined to be a hybrid model, wherein the hybrid model includes a generation model and a retrieval model, and the lower limit of the third data interval is not less than the upper limit of the second data interval.
2. The method for generating feedback information according to claim 1, characterized in that: Before determining target answer information from the preset corpus according to the user question information by using the answer generation model, the method further includes: If a consultation dialog box pops up in the display interface, at least one estimated question is generated and displayed based on the user's historical browsing history; The estimated question selected by the user is used as at least part of the user question information.
3. The method for generating feedback information according to claim 1, characterized in that: Before determining target answer information from the preset corpus according to the user question information by using the answer generation model, the method further includes: Obtaining the standard question and answer data; According to the standard question-answering data, similar questions corresponding to each standard question are generated by a generative network in a trained generative adversarial network; The target corpus is obtained according to the standard question and answer data and the similar questions.
4. The method for generating feedback information according to claim 1, characterized in that: The step of determining feedback information to the user based on the target answer information and the target matching degree includes: If the target matching degree is less than the first matching degree, determining that the feedback information includes indication information indicating that the user's question cannot be answered, and / or includes inquiry information asking the user whether to transfer to manual service; If the target matching degree is not less than the first matching degree and less than the second matching degree, searching the preset prediction library for N questions with the highest similarity to the user's question information, taking the N questions with the highest similarity as candidate questions and displaying them in the consultation dialog box to inquire whether the question the user wants to ask is among the displayed N candidate questions, wherein N is a positive integer; If the target matching degree is not less than the second matching degree, it is determined that the feedback information includes the target answer information.
5. The method for generating feedback information according to any one of claims 1 to 4, characterized in that: Also includes: Counting session information within a first time period, wherein the session information includes user question information and corresponding feedback information within the first time period; Comparing the session information with historical session information to detect whether the session information is abnormal, wherein the historical session information is generated earlier than the session information; If an abnormality is detected in the session information, an abnormality prompt message is generated.
6. A device for generating feedback information, characterized in that: The device is used for online intelligent consultation, including: A first determination module is used to determine an answer generation model according to the amount of corpus data of a preset corpus, wherein the preset corpus includes standard question and answer data and similar question data, the standard question and answer data includes at least one standard question and a standard answer corresponding to each of the standard questions, and the similar question data includes similar questions corresponding to each of the standard questions; A second determination module is used to determine target answer information from the preset corpus according to the user question information through the answer generation model; A processing module, configured to use the matching degree between the target answer information and the user question information as a target matching degree; A third determination module, configured to determine feedback information to the user according to the target answer information and the target matching degree; Before determining target answer information from the preset corpus according to the user question information by using the answer generation model, the method further includes: A screenshot module, for, if it is detected that a prompt window is displayed in the operation interface of the preset business system, taking a screenshot of the operation interface to obtain a screenshot image, wherein the prompt window contains information about abnormal operations in the operation interface; An edge detection module, used to perform edge detection on the screenshot image to determine the area where the prompt window is located; A text recognition module, used for performing text recognition on the area where the prompt window is located to obtain target text information in the prompt window; an extraction module, configured to extract identification information of the abnormal operation from the target text information, and use the identification information of the abnormal operation as at least part of the user question information; The first determining module specifically includes: a first determining unit, configured to determine that the answer generation model is a preset data model based on a Jaccard coefficient if the amount of corpus data of the target corpus is in a first data interval; a second determining unit, configured to determine that the answer generation model is a preset probabilistic network model if the amount of corpus data of the target corpus is in a second data interval, wherein a lower limit of the second data interval is not less than an upper limit of the first data interval; The third determination unit is used to determine that the answer generation model is a hybrid model if the amount of corpus data in the target corpus is in a third data interval, wherein the hybrid model includes a generation model and a retrieval model, and the lower limit of the third data interval is not less than the upper limit of the second data interval.
7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for generating feedback information according to any one of claims 1 to 5 is implemented.
8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for generating feedback information according to any one of claims 1 to 5 is implemented.
