Information processing method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention discloses an information processing method and device, equipment, a storage medium and a program product. The information processing method comprises the steps of obtaining a question-answer pair according to an input customer question and a pre-trained first question-answer model; obtaining a question-answer pair classification result according to the question-answer pair and a pre-trained question-answer pair classification model, wherein the question-answer pair classification result comprises a preset constraint relationship of a question-answer knowledge base to the question-answer pair; and determining target reply information according to the question and answer pair and the constraint relationship, and replying the customer question based on the target reply information. The question and answer pair including the preliminary reply information of the customer question is obtained according to the pre-trained first question and answer model and the customer question, and when it is determined that the question and answer pair needs to be retrieved according to the question and answer pair and the classification model, the preliminary reply information is further perfected according to the question and answer knowledge base to determine the target reply information of the customer question; and the accuracy of replying to the customer question can be improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent question answering, and in particular to an information processing method, apparatus, device, storage medium and program product. Background Art

[0002] Currently, the intelligent general dialogue system ChatGPT is popular all over the world, and its highly anthropomorphic answering ability is shocking. It has given people a glimpse of the dawn of AIGC. It mainly uses high-quality massive world knowledge to pre-train large-scale language models such as GPT3 to obtain pre-trained models, and then introduces "manually labeled data + reinforcement learning" to continuously fine-tune the pre-trained models, so that the LLM model can learn to understand the meaning of human commands and instructions, and achieve highly anthropomorphic answers. However, current models such as ChatGPT still have the problem of hallucination output, and the accuracy of intelligent question and answer is insufficient. Summary of the invention

[0003] The embodiments of the present application provide an information processing method, apparatus, device, storage medium and program product, which obtain question-answer pairs based on a pre-trained question-answer model and classification model, and use the question-answer pairs to search in a question-answer knowledge base to determine the target answer information for customer questions, thereby improving the accuracy of answers to customer questions.

[0004] In a first aspect, an embodiment of the present application provides an information processing method, including:

[0005] A question-answer pair is obtained based on an input customer question and a pre-trained first question-answer model; a question-answer pair classification result is obtained based on the question-answer pair and the pre-trained question-answer pair classification model, the question-answer pair classification result including a constraint relationship of a preset question-answer knowledge base on the question-answer pair; target answer information is determined based on the question-answer pair and the constraint relationship, and the customer question is replied based on the target answer information.

[0006] In a second aspect, an embodiment of the present application provides an information processing device, including:

[0007] A processing module, configured to obtain a question-answer pair based on an input customer question and a pre-trained first question-answer model; and to obtain a question-answer pair classification result based on the question-answer pair and the pre-trained question-answer pair classification model, wherein the question-answer pair classification result includes a constraint relationship between a preset question-answer knowledge base and the question-answer pair;

[0008] A determination module is used to determine target answer information according to the question-answer pair and the constraint relationship, and reply to the customer question based on the target answer information.

[0009] In a third aspect, an embodiment of the present application provides an information processing device, the device comprising:

[0010] A memory, a processor, and an information processing program stored in the memory and executable on the processor, wherein the information processing program is configured to implement part or all of the steps described in any method in the first aspect.

[0011] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which an information processing program is stored. When the information processing program is executed by a processor, some or all of the steps described in any method in the first aspect are implemented.

[0012] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in any method of the first aspect of the embodiment of the present application. The computer program product can be a software installation package.

[0013] By implementing the embodiment of the present application, the server of the information processing system obtains a question-answer pair based on the input customer question and the pre-trained first question-answer model; then obtains a question-answer pair classification result based on the question-answer pair and the pre-trained question-answer pair classification model, and the question-answer pair classification result includes the constraint relationship of the preset question-answer knowledge base on the question-answer pair; finally, the target reply information is determined based on the question-answer pair and the constraint relationship, and the customer question is replied based on the target reply information. By obtaining a question-answer pair including the customer question and the preliminary reply information of the customer question based on the pre-trained first question-answer model and the customer question, and then determining that the question-answer pair needs to be retrieved in the question-answer knowledge base based on the question-answer pair and the classification model, the preliminary reply information is further improved based on the question-answer knowledge base to determine the target reply information of the customer question, which is conducive to improving the accuracy of the reply to the customer question. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0015] Figure 1 It is a schematic diagram of the architecture of an information processing system provided by an embodiment of the present application;

[0016] Figure 2 is a flow chart of an information processing method provided by an embodiment of the present application;

[0017] Figure 3 It is a flowchart of a question-answering model training provided in an embodiment of the present application;

[0018] Figure 4 It is a flowchart of a question-answer pair search provided by an embodiment of the present application;

[0019] Figure 5is a structural diagram of an information processing device provided in an embodiment of the present application;

[0020] Figure 6 It is a structural diagram of a computer provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. According to the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0022] The terms "first", "second", "third", etc. in the specification, claims and drawings of this application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0023] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0024] The intelligent general dialogue system ChatGPT is popular all over the world, and its highly anthropomorphic answering ability is shocking. It has given people a glimpse of the dawn of AIGC, which mainly uses high-quality massive world knowledge to pre-train large-scale language models such as GPT3 to obtain pre-trained models, and then introduces "manually labeled data + reinforcement learning" to continuously fine-tune the pre-trained models, so that the LLM model can learn to understand the meaning of human commands and instructions and achieve highly anthropomorphic answers.

[0025] Currently, there are several methods to implement intelligent question answering through models such as ChatGPT:

[0026] Retrieval-based method: Match the question with a pre-built question-answering knowledge base or document to find the most relevant answer. Common methods include keyword matching, vector similarity calculation, etc. This method is simple and direct, but may be limited by the quality and coverage of the question-answering knowledge base, resulting in low answer accuracy.

[0027] Generation-based methods: Use pre-trained language models (such as BERT, GPT, etc.) to generate responses. These models can understand the context and generate coherent and reasonable responses. However, due to problems such as output uncertainty and lack of controllability of generative models, inaccurate or wrong responses may occur.

[0028] Interactive learning method: By interacting with users, it gradually obtains more information in multiple rounds of dialogue and dynamically adjusts the answer according to the context. This method can provide more personalized and accurate responses, but it takes a lot of time and resources to train and is less efficient.

[0029] It can be seen that the current intelligent question-answering through models such as ChatGPT still has the problem of hallucination output, and it is impossible to ensure the accuracy of intelligent question-answering without reducing the efficiency of question-answering.

[0030] In response to the above problems, the embodiments of the present application provide an information processing method, apparatus, device, storage medium and program product, wherein the server of the information processing system obtains a question-answer pair based on the input customer question and a pre-trained first question-answer model; then obtains a question-answer pair classification result based on the question-answer pair and the pre-trained question-answer pair classification model, and the question-answer pair classification result includes a constraint relationship between the preset question-answer knowledge base and the question-answer pair; finally, the target reply information is determined based on the question-answer pair and the constraint relationship, and the customer question is replied based on the target reply information. By obtaining a question-answer pair including the customer question and preliminary reply information of the customer question based on the pre-trained first question-answer model and the customer question, and then determining that the question-answer pair needs to be retrieved in the question-answer knowledge base based on the question-answer pair and the classification model, the preliminary reply information is further improved based on the question-answer knowledge base to determine the target reply information of the customer question, which is conducive to improving the accuracy of the reply to the customer question.

[0031] The information processing method, apparatus, device, storage medium and program product provided in the embodiments of the present application can be applied to Figure 1 In the information processing system shown, see Figure 1 , Figure 1It is a schematic diagram of the architecture of an information processing system provided in an embodiment of the present application. The information processing system 100 includes a terminal 101 and a server 102. The terminal 101 can communicate with the server 102 through a network. The terminal 101 refers to a device used by a user, such as a smart phone, a computer, etc. In this solution, the terminal 101 is mainly responsible for collecting training data including push information and question-and-answer information, and sending it to the server for processing. Users can interact with the system through the terminal; the server 102 refers to a remote computer used to process a large number of computing tasks and store data. In this solution, the server 102 receives training data from the terminal, and uses the training data to train the question-and-answer model and the classification model. Finally, the target answer information of the customer's question is determined based on the trained question-and-answer model and the classification model, and the target answer information is fed back to the terminal 101.

[0032] Based on this, the present application provides an information processing method, apparatus, device, storage medium and program product, and the present application is described in detail below in conjunction with the accompanying drawings.

[0033] See also Figure 2 , Figure 2 is a flow chart of an information processing method provided by an embodiment of the present application, such as Figure 2 As shown, the method comprises the following steps:

[0034] S201, obtaining a question-answer pair according to the input customer question and the pre-trained first question-answer model.

[0035] Among them, the first answer information output by the first question and answer model is obtained by taking the customer question as the input of the first question and answer model, and the customer question and the first answer information are combined to obtain the corresponding question and answer pair.

[0036] For example, the customer's question is: "What is the specific increase and reduction in the amount of credit limit and interest rate reduction in this A business interest rate increase activity?" The first answer information output by the first question-answering model is: "The specific increase in the amount of credit limit and interest rate reduction in the A business interest rate increase activity is 50,000 yuan, and the interest rate reduction is 13 percentage points." Finally, the customer's question and the first answer information are combined to obtain a question-answer pair: "What is the specific increase in the amount of credit limit and interest rate reduction in this A business interest rate increase activity?" "The specific increase in the amount of credit limit and interest rate reduction in the A business interest rate increase activity is 50,000 yuan, and the interest rate reduction is 13 percentage points."

[0037] In a possible implementation, before step S201, the method further includes:

[0038] Obtaining the pre-trained first question-answering model according to the question-answering training data;

[0039] The pre-trained question-answer pair classification model is obtained according to the question-answer training data and the initial classification model.

[0040] Among them, see Figure 3 , Figure 3 is a flowchart of a question-answering model training provided in an embodiment of the present application, such as Figure 3 As shown, in a possible implementation, obtaining a pre-trained first question-answering model according to question-answering training data includes:

[0041] Question and answer training data is obtained by combining push information and question and answer information, wherein the push information includes push background information, push process information and push object information; supervised fine-tuning training is performed on the initial question and answer model based on the question and answer training data to obtain a pre-trained first question and answer model.

[0042] Among them, the push information includes push background information, push process information and push object information. The push background information includes background description, the push process information includes push process introduction, such as opening remarks and closing remarks, etc. The push object information includes customer information description. For example, the push object information can be customer age, customer gender, and customer preferences, etc. Among them, the push information is constructed through prompt and is used as question and answer training data after being combined with question and answer information.

[0043] Among them, when the initial question-answering model is trained based on the question-answering training data, the question-answering training data can be preprocessed, including data cleaning, word segmentation, stop word removal, and language-specific processing. Specifically, data cleaning can remove duplicate data and unnecessary characters, stop word filtering can remove modal particles or common but meaningless prepositions, conjunctions, articles, etc. in the collected data; language-specific processing can convert different local dialects or foreign languages ​​into Mandarin.

[0044] For example, before stop word filtering, the question in the question and answer pair is: "Well, I just want to ask how much credit limit I can get when the credit limit is increased." After stop word filtering, the question in the question and answer pair is "I want to ask how much credit limit I can get when the credit limit is increased."

[0045] For example, at a certain moment, the push background information is: you are a customer service representative of business B of company A, and you need to guide customers to participate in the increase and interest rate reduction activity of business B based on their basic information. The push process information is: you are a gold medal customer service representative of business B of company A, and you can push based on the basic information of customers.

[0046] The main push steps include:

[0047] In the opening remarks, confirm whether it is the customer himself. If not, just hang up politely. The opening remarks are: Hello, is this Mr. Chen? Build a closer relationship with the customer. For example, the caller has something important to tell you. Is business B you repaying in person?

[0048] The steps of business introduction include explaining product information and activities. Business B has an activity of increasing credit limit and reducing interest rate. As long as the customer borrows the remaining credit limit in business B once, the customer will have the opportunity to increase the credit limit by 50,000 and reduce the interest rate by more than ten percentage points.

[0049] In the objection handling steps, if the customer has any questions, we will answer them accordingly. For example, what is the current interest rate, what is the interest rate for business B, or if the credit limit is too low, we will indicate that the call is to solve your problem, and you only need to withdraw the remaining credit limit to increase the customer's credit limit.

[0050] Conclusion step, OK, I won't bother you anymore, thank you for answering the call, goodbye! The push object information is: customer name, customer gender, customer loan amount and customer account balance. By combining the above push information with manual dialogue, question and answer training data is obtained.

[0051] Among them, after obtaining the question-answering training data, the first question-answering model, that is, the optimized question-answering model, is obtained by inputting the question-answering training data into the initial question-answering model for supervised fine-tuning training. Specifically, supervised fine-tuning (SFT) training is a semi-supervised learning method used to enhance the performance of the model. Its main steps include: initialization step: supervised training of the initial model using a small amount of labeled data. Pseudo-label generation step: use the initial model to predict a large amount of unlabeled data, and use the prediction results as pseudo-labels. Expanding the training set step: merge the unlabeled data with pseudo-labels with the original labeled data to form an expanded training set. Fine-tuning update step: use the expanded training set to retrain or fine-tune the model to improve performance.

[0052] In addition to using SFT training, the initial question-answering model can also be trained in the following ways:

[0053] Active Learning: Request human annotations by selecting representative, difficult, or uncertain samples and add them to known categories to improve the model.

[0054] Transfer Learning: Improve the target domain task by learning knowledge and experience from the source domain. You can use the model parameters that have been pre-trained on other related tasks as initial weights and fine-tune the parameters according to specific task requirements.

[0055] Reinforcement Learning: It is suitable for problems that require interaction with the environment and optimization of strategies and decisions through trial and error. It can be combined with technologies such as Generative Adversarial Networks (GANs) to introduce unsupervised signals to improve performance.

[0056] It can be seen that in this embodiment, by combining the collected multiple push information and question and answer information as training data for the subsequent question and answer model, and using supervised fine-tuning training to train the initial question and answer model, it is beneficial to improve the robustness and applicability of the first question and answer model after training.

[0057] In a possible implementation, obtaining a pre-trained question-answer pair classification model according to the question-answer training data and the initial classification model includes:

[0058] By splitting multiple questions and answers in the training data, multiple question-answer pairs and classification labels corresponding to the multiple question-answer pairs are obtained, and the classification labels are used to determine the steps of the question-answer pairs in the push process; the initial classification model is trained according to the multiple question-answer pairs and the classification labels corresponding to the multiple question-answer pairs to obtain a pre-trained question-answer pair classification model.

[0059] Among them, the classification model is a machine learning model that is used to classify input data into different categories or labels, and learn a decision boundary or probability distribution function based on the relationship between input features and corresponding labels, so that new unknown samples can be predicted and classified into appropriate categories. Common classification algorithms include logistic regression, support vector machine (SVM), decision tree, random forest, naive Bayes, etc.

[0060] The training data includes multiple sets of questions and answers, which are split into corresponding question-answer pairs. Each question-answer pair includes a question, an answer corresponding to the question, and a process classification label of the question-answer pair. The process classification label can be used to indicate the push process in which the question-answer pair is located. The question-answer pairs and the corresponding process classification labels are input into the first classification model for training to obtain an optimized first classification model, namely the second classification model.

[0061] It can be seen that in this embodiment, by splitting multiple questions and answers in the training data, classifying them according to different topics or categories to obtain multiple question-answer pairs and corresponding classification labels, and then using the question-answer pairs and corresponding classification labels to train a second classification model, it is beneficial to improve the accuracy of the second classification model.

[0062] S202: Obtain a question-answer pair classification result based on the question-answer pair and a pre-trained question-answer pair classification model.

[0063] Among them, the classification result of the question-answer pair can be obtained through the pre-trained classification model, and the classification result of the question-answer pair includes the constraint relationship of the preset question-answer knowledge base on the question-answer pair, and the constraint relationship is used to determine whether it is necessary to retrieve the question-answer pair in the question-answer knowledge base. Specifically, the classification result of the question-answer pair is used to determine the push process of the question-answer pair, and the push process that needs to retrieve the question-answer pair in the question-answer knowledge base and the push process that does not need to retrieve the question-answer pair in the question-answer knowledge base can be pre-set.

[0064] Exemplarily, the opening and closing stages of the push process do not need to be searched in the question and answer knowledge base. Therefore, if the classification result of the question and answer pair is the opening stage or the closing stage, there is no need to search the question and answer pair in the question and answer knowledge base; if the classification result of the question and answer pair is not the opening stage or the closing stage, it is necessary to search the question and answer pair in the question and answer knowledge base.

[0065] S203: Determine target response information according to the question-answer pair and the constraint relationship.

[0066] The target reply information is used to reply to the customer's question.

[0067] It can be seen that in this embodiment, by obtaining question-answer pairs based on the trained question-answer model and classification model and searching the question-answer knowledge base to determine the target answer information for the customer's question, it is helpful to improve the accuracy of the answer to the customer's question.

[0068] In a possible implementation, step S203 includes the following steps:

[0069] If it is detected that the constraint relationship requires a search in the question-answer knowledge base, the target reply information is determined based on the question-answer pair and the question-answer knowledge base; if it is detected that the constraint relationship does not require a search in the question-answer knowledge base, the target reply information is determined to be the first reply information.

[0070] Among them, the question and answer knowledge base includes multiple groups of question and answer pairs and corresponding classification labels. When it is determined that a search is needed in the question and answer knowledge base, the multiple groups of question and answer pairs corresponding to the search are first determined according to the classification labels of the question and answer pairs that need to be searched, and then relevant question and answer pairs are selected from the multiple groups of question and answer pairs as target reply information; when it is determined that there is no need to search in the question and answer knowledge base, the first reply information output by the first question and answer model is directly used as the target reply information.

[0071] It can be seen that in this embodiment, after obtaining the question-answer pair classification result issued by the classification model, it is determined whether it is necessary to search the question-answer knowledge base based on the question-answer pair classification result. When the classification result is simple, the first reply information can be directly output as the target reply information, which is conducive to improving the efficiency of question-answering.

[0072] In a possible implementation, determining target answer information according to the question-answer pair and the question-answer knowledge base includes:

[0073] Determine the first question and answer knowledge base in the question and answer knowledge base corresponding to the question and answer pair according to the length of the question and answer pair; perform best match retrieval based on the question and answer pair and the first question and answer knowledge base to obtain a first set of related question and answer pairs; obtain a vectorized question and answer pair and a second question and answer knowledge base by vectorizing the question and answer pair and the first question and answer knowledge base; perform cosine similarity retrieval based on the vectorized question and answer pair and the second question and answer knowledge base to obtain a second set of related question and answer pairs; obtain a retrieval result by inputting the question and answer pair, the first set of question and answer pairs and the second set of question and answer pairs into a refined ranking model, and the retrieval result is the output of the refined ranking model; determine the target reply information according to the retrieval result and a preset threshold.

[0074] Among them, the question and answer knowledge base includes a long text question and answer knowledge base and a short text question and answer knowledge base. After obtaining the length of the question and answer pair, the length of the question and answer pair can be compared with a preset length threshold. If the length of the question and answer pair does not exceed the preset length threshold, then the first question and answer knowledge base corresponding to the question and answer pair is the short text question and answer knowledge base. If the length of the question and answer pair exceeds the preset length threshold, then the first question and answer knowledge base corresponding to the question and answer pair is the long text question and answer knowledge base. Among them, the question and answer knowledge base is vectorized to obtain a vector database ES.

[0075] Among them, see Figure 4 , Figure 4 is a flowchart of a question-answer pair search provided by an embodiment of the present application, such as Figure 4 As shown, after obtaining the question-answer pair output by the model, the length of the question-answer pair is calculated to determine the first question-answer knowledge base corresponding to the question-answer pair. For example, the preset length threshold is 10. When the length of the question-answer pair is greater than 10, the first question-answer knowledge base is a long text question-answer knowledge base. When the length of the question-answer pair does not exceed 10, the first question-answer knowledge base is a short text question-answer knowledge base.

[0076] Among them, it is necessary to perform two similarity calculations on the question-answer pairs to determine two sets of related question-answer pairs. The first similarity calculation is the best match search, that is, the BM25 (Best Matching 25) keyword search. The BM25 keyword search is performed based on the question-answer pairs and the first question-answer knowledge base corresponding to the question-answer pairs to obtain the first set of related question-answer pairs. Here, the top 3 related question-answer pairs, that is, top3, are selected as the first set of related question-answer pairs; Among them, BM25 is a commonly used keyword-based information retrieval algorithm used to measure the relevance of text to the query. The BM25 algorithm calculates the degree of match between each query term and the document, and comprehensively considers factors such as word frequency, document length, and query term frequency to determine the relevance score. Specifically, BM25 evaluates the document frequency (that is, the number of documents containing the term), the inverse document frequency (that is, the proportion of documents containing the term to the total number of documents), and the word frequency of each query term in the inverted index, and finally generates a ranked list.

[0077] The second similarity calculation is the cosine similarity calculation. First, the question-answer pair and the corresponding question-answer knowledge base are vectorized to obtain the vectorized question-answer pair and the vector database ES, i.e., the second question-answer database. Then, the cosine similarity search is performed based on the vectorized question-answer pair and the second question-answer knowledge base to obtain the second set of related question-answer pairs. Here, the top 3 related question-answer pairs, i.e., top3, are selected as the first set of related question-answer pairs. Among them, cosine similarity calculation is a commonly used method to measure the similarity between vectors. It evaluates the degree of proximity in direction by calculating the cosine value of the angle between two vectors, thereby judging their similarity. Specifically, given two vectors A and B, the cosine similarity calculation formula is as follows:

[0078] cos(θ)=dot_product(A,B) / (norm(A)*norm(B)),

[0079] Among them, dot_product(A,B) represents the dot product of vectors A and B (that is, the sum of the products of corresponding elements), norm(A) and norm(B) represent the norms of vectors A and B (that is, the Euclidean distance), respectively.

[0080] Among them, the refined ranking model refers to the model used in the information retrieval system to further sort and filter the search results. In large-scale search engines or recommendation systems, the initial recall phase usually returns a large number of candidate results, and the refined ranking model is responsible for sorting these candidate results according to user needs and contextual information to provide an ordered list that better meets the user's intentions.

[0081] Exemplarily, after determining question and answer pair 1, the first set of related question and answer pairs (including question and answer pair 2, question and answer pair 3, and question and answer pair 4), and the second set of related question and answer pairs (including question and answer pair 5, question and answer pair 6, and question and answer pair 7), the question and answer pairs are sorted from high to low according to relevance using the refined ranking model, and the results are: question and answer pair 2, question and answer pair 3, question and answer pair 1, question and answer pair 5, question and answer pair 6, question and answer pair 4, question and answer pair 7. At this time, the most relevant question and answer pair is determined to be question and answer pair 2.

[0082] It can be seen that in this embodiment, the corresponding question and answer knowledge base is determined according to the length of the question and answer pair, and then the most relevant replies are retrieved using BM25 keyword retrieval and cosine similarity respectively, and finally the retrieval results are sorted by the refined ranking model to obtain the target answer information most relevant to the question, which is conducive to improving the accuracy of intelligent question and answer.

[0083] In a possible implementation, the search result includes a plurality of question-answer pairs ranked according to relevance, and the step of determining target answer information according to the search result and a preset threshold includes:

[0084] If it is detected that the retrieval result is less than the preset threshold, the target reply information is determined as the default reply; if it is detected that the retrieval result is not less than the preset threshold, the target reply information is determined based on the reply and label information of the question-answer pair with the highest relevance in the retrieval result.

[0085] Among them, the label information includes at least one of a customer label, an activity label or a holiday label. The customer label may be the customer's name, customer's gender, and customer's age, etc. The activity label may be an activity that is valid during the question-and-answer period, such as year-end promotions, credit limit increase activities, or interest rate reduction activities, etc. The holiday label may be the Dragon Boat Festival, Mid-Autumn Festival, Women's Day, etc.

[0086] Among them, when the search results are less than the preset threshold, it is considered that no relatively relevant replies have been retrieved, and a fallback phrase is used to reply at this time, which can be: "What did you say?", "Can you say it again?", etc. When the search results are not less than the preset threshold, the reply of the most relevant question-answer pair in the search results and the label information are combined to determine the target reply. For example, the reply of the most relevant question-answer pair in the search results is "the amount of the increase is 50,000", and the customer label in the label information is "surname is Zhang" and "female", and the target reply is "Ms. Zhang, the amount of the increase is 50,000"; the reply of the most relevant question-answer pair in the search results is "OK, then I won't bother you here", and the festival label in the label information is Dragon Boat Festival, and the target reply is "OK, then I won't bother you here, and I wish you a happy Dragon Boat Festival."

[0087] It can be seen that in this embodiment, after obtaining the retrieval results, by comparing the retrieval results with the preset threshold to determine whether the target reply information needs to be modified by collecting specific label information, it is beneficial to realize personalized question and answer services and increase user experience and satisfaction.

[0088] In a possible implementation, before obtaining a question-answer pair classification result according to the question-answer pair and the pre-trained question-answer pair classification model, the method further includes:

[0089] Determine whether there is manual dialing data in the current scenario; if it is detected that there is manual dialing data in the current scenario, obtain a first manual dialing question and answer pair according to the manual dialing data; deduplicate the question and answer pair according to clustering, and / or, intention recognition method, to obtain a second manual dialing question and answer pair; obtain a third manual dialing question and answer pair by replacing the customer information field in the second manual dialing question and answer pair with a preset identifier; construct a question and answer knowledge base by obtaining a set of the third manual dialing question and answer pairs; if it is detected that there is no manual dialing data in the current scenario, obtain a first non-manual dialing question and answer pair according to a customer question set in the current scenario and the answers corresponding to the customer question set; obtain a second non-manual dialing question and answer pair by replacing the customer information field in the first non-manual dialing question and answer pair with a preset identifier; construct a question and answer knowledge base by obtaining a set of the second non-manual dialing question and answer pairs.

[0090] Among them, before building the question-answering knowledge base, the data can be preprocessed, including word segmentation, stop word filtering, and language-specific processing. Specifically, stop word filtering can remove modal particles or common but meaningless prepositions, conjunctions, articles, etc. in the collected data; language-specific processing can convert different local dialects or foreign languages ​​into Mandarin.

[0091] For example, before stop word filtering, the question in the question and answer pair is: "Well, I just want to ask how much credit limit I can get when the credit limit is increased." After stop word filtering, the question in the question and answer pair is "I want to ask how much credit limit I can get when the credit limit is increased."

[0092] The method of deduplicating question and answer pairs may be clustering, intent recognition, feature extraction, and hash value comparison, etc., which are not limited here.

[0093] It can be seen that in this example, the question and answer knowledge base is constructed separately in the scenarios with manual dialing data and without manual dialing data. Different construction methods can be selected according to specific needs and scenarios, which is conducive to improving the quality and coverage of the question and answer knowledge base, thereby improving the accuracy of subsequent intelligent question and answer.

[0094] See also Figure 5 , Figure 5is a schematic diagram of the structure of an information processing device provided in an embodiment of the present application, such as Figure 5 As shown, the information processing device 500 includes:

[0095] Processing module 501, used to obtain a question-answer pair according to an input customer question and a pre-trained first question-answer model; and used to obtain a question-answer pair classification result according to the question-answer pair and the pre-trained question-answer pair classification model, wherein the question-answer pair classification result includes a constraint relationship between a preset question-answer knowledge base and the question-answer pair;

[0096] The determination module 502 is used to determine the target answer information according to the question-answer pair and the constraint relationship, and reply to the customer question based on the target answer information.

[0097] In one possible implementation, before obtaining a question-answer pair based on the input customer question and the pre-trained first question-answer model, the processing module 501 is also used to: obtain the pre-trained first question-answer model based on the question-answer training data; and obtain a pre-trained question-answer pair classification model based on the question-answer training data and the initial classification model.

[0098] In one possible implementation, in terms of obtaining a pre-trained first question-answering model based on question-answering training data, the processing module 501 is specifically used to: obtain question-answering training data by combining push information and question-answering information, wherein the push information includes push background information, push process information, and push object information; and perform supervised fine-tuning training on the initial question-answering model based on the question-answering training data to obtain a first question-answering model.

[0099] In one possible implementation, in terms of determining the target reply information based on the question and answer pair and the classification result of the question and answer pair, the determination module 502 is specifically used to: if it is detected that the constraint relationship requires retrieval in the question and answer knowledge base, determine the target reply information based on the question and answer pair and the question and answer knowledge base; if it is detected that the constraint relationship does not require retrieval in the question and answer knowledge base, determine the target reply information to be the first reply information.

[0100] In one possible implementation, in terms of determining the target reply information based on the question and answer pair and the question and answer knowledge base, the determination module 502 is specifically used to: determine the first question and answer knowledge base corresponding to the question and answer pair in the question and answer knowledge base based on the length of the question and answer pair, the question and answer knowledge base including a long text question and answer knowledge base and a short text question and answer knowledge base; perform best match retrieval based on the question and answer pair and the first question and answer knowledge base to obtain a first set of related question and answer pairs; obtain a vectorized question and answer pair and a second question and answer knowledge base by vectorizing the question and answer pair and the first question and answer knowledge base; perform cosine similarity retrieval based on the vectorized question and answer pair and the second question and answer knowledge base to obtain a second set of related question and answer pairs; obtain a retrieval result by inputting the question and answer pair, the first question and answer pair set and the second question and answer pair set into a refined ranking model; and determine the target reply information based on the retrieval result and a preset threshold.

[0101] In one possible implementation, the retrieval results include multiple question-answer pairs sorted according to relevance. In determining the target reply information based on the retrieval results and a preset threshold, the determination module 502 is specifically used to: if it is detected that the retrieval result is less than the preset threshold, determine the target reply information as a default reply; if it is detected that the retrieval result is not less than the preset threshold, determine the target reply information based on the reply and label information of the question-answer pair with the highest relevance in the retrieval result, wherein the label information includes at least one of a customer label, an activity label, or a holiday label.

[0102] In one possible implementation, in terms of obtaining a pre-trained question-answer pair classification model based on the question-answer training data and the initial classification model, the processing module 501 is specifically used to: obtain multiple question-answer pairs and classification labels corresponding to the multiple question-answer pairs by splitting the multiple questions and answers in the training data, and the classification labels are used to determine the steps of the question-answer pairs in the push process; train the initial classification model according to the multiple question-answer pairs and the classification labels corresponding to the multiple question-answer pairs to obtain a pre-trained question-answer pair classification model.

[0103] In a possible implementation, before obtaining the question and answer pair classification result according to the question and answer pair and the pre-trained question and answer pair classification model, the determination module 502 is also used to: determine whether there is manual dialing data in the current scenario; if it is detected that there is manual dialing data in the current scenario, obtain a first manual dialing question and answer pair according to the manual dialing data; deduplicate the question and answer pair according to clustering, and / or, intention recognition method to obtain a second manual dialing question and answer pair; obtain a third manual dialing question and answer pair by replacing the customer information field in the second manual dialing question and answer pair with a preset identifier; construct a question and answer knowledge base by obtaining a set of the third manual dialing question and answer pairs; if it is detected that there is no manual dialing data in the current scenario, obtain a first non-manual dialing question and answer pair according to the customer question set in the current scenario and the answers corresponding to the customer question set; obtain a second non-manual dialing question and answer pair by replacing the customer information field in the first non-manual dialing question and answer pair with a preset identifier; construct a question and answer knowledge base by obtaining a set of the second non-manual dialing question and answer pairs.

[0104] It is worth noting that the specific functional implementation of the information processing device 500 is shown in the above Figure 1 The description of the information processing method shown in the figure, for example, the processing module 501 is used to implement the relevant content of executing S201-S202, and the determination module 502 is used to implement the relevant content of executing S203. Each unit or module in the information processing device 500 can be separately or completely combined into one or more other units or modules to constitute, or one (some) of the units or modules can also be further divided into multiple smaller units or modules in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided according to logical functions. In practical applications, the function of one unit (or module) is realized by multiple units (or modules), or the function of multiple units (or modules) is realized by one unit (or module).

[0105] According to the description of the above method embodiment and related device embodiment, please refer to Figure 6 , Figure 6 It is a schematic diagram of the structure of a computer 600 provided in an embodiment of the present application. Figure 6 The computer 600 shown includes a processor 601, a memory 602, a communication interface 603 and a bus 604. The processor 601, the memory 602 and the communication interface 603 are connected to each other through the bus 604.

[0106] Optionally, the memory 602 is a ROM, a static storage device, a dynamic storage device or a RAM.

[0107] The memory 602 can store executable program codes. When the executable program codes stored in the memory 602 are executed by the processor 601, the processor 601 and the communication interface 603 are used to execute Figure 1 The various steps of the information processing method of the illustrated embodiment.

[0108] The processor 601 adopts a general-purpose CPU, a microprocessor, an application-specific integrated circuit ASIC, a GPU or one or more integrated circuits to execute relevant programs to perform the information processing method of the method embodiment of the present application.

[0109] The processor 601 can also be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the information processing method of the present application can be completed by the hardware integrated logic circuit in the processor 601 or the instruction in the form of software. Optionally, the processor 601 is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component. The processor can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor is a microprocessor or the processor is any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined to execute. The optional software module is located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602, and combines its hardware to complete the functions required to be executed by the modules included in the information processing device 500 of the embodiment of the present application, or executes the information processing method of the method embodiment of the present application.

[0110] The communication interface 603 uses a transceiver or other related device such as, but not limited to, a transceiver.

[0111] The bus 604 may include a path for transmitting information between various components of the computer 600 (eg, the memory 602 , the processor 601 , and the communication interface 603 ).

[0112] It should be noted that although Figure 6 The computer 600 shown only shows a memory, a processor, and a communication interface, but in the specific implementation process, those skilled in the art should understand that the computer 600 also includes other devices necessary for normal operation. At the same time, according to specific needs, those skilled in the art should understand that the computer 600 may also include hardware devices for implementing other additional functions. In addition, those skilled in the art should understand that the computer 600 may also include only the devices necessary for implementing the embodiments of the present application, and does not necessarily include Figure 6 All devices shown in .

[0113] An embodiment of the present application provides a computer-readable storage medium, in which a computer program for electronic data exchange is stored. The computer program includes execution instructions, and the execution instructions are used to execute part or all of the steps of any one of the information processing methods recorded in the above-mentioned information processing method embodiments. The above-mentioned computer includes an electronic terminal device.

[0114] An embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program can be operated to enable a computer to perform part or all of the steps of any information processing method recorded in the above method embodiments. The computer program product can be a software installation package.

[0115] It should be noted that, for the sake of simplicity, the embodiments of any of the aforementioned information processing methods are described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present application.

[0116] The above is a detailed introduction to the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of an information processing method, device, equipment, storage medium and program product of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for a general technician in this field, based on the idea of ​​an information processing method, device, equipment, storage medium and program product of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0117] The present application is described with reference to the flowcharts and / or block diagrams of the methods, hardware products, and computer program products of the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0118] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 The memory may include: a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0119] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude multiple situations. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0120] A person skilled in the art can understand that all or part of the steps in the various methods of the method embodiments of any of the above-mentioned information processing methods can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a disk or an optical disk, etc.

[0121] It can be understood that any product that is controlled or configured to execute the processing method of the flowchart described in an embodiment of an information processing method of the present application, such as the device and computer program product of the above flowchart, falls within the scope of the related products described in the present application.

[0122] Obviously, those skilled in the art can make various changes and modifications to the information processing method, device, equipment, storage medium and program product provided by this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include these modifications and variations.

Claims

1. An information processing method, characterized in that: The method comprises: Obtain a question-answer pair based on the input customer question and the pre-trained first question-answer model; Obtaining a question-answer pair classification result according to the question-answer pair and a pre-trained question-answer pair classification model, wherein the question-answer pair classification result includes a constraint relationship between a preset question-answer knowledge base and the question-answer pair; Target answer information is determined according to the question-answer pair and the constraint relationship, and the customer question is answered based on the target answer information.

2. The method according to claim 1, characterized in that The question-answer pair includes the customer question and the first answer information output by the first question-answer model, the constraint relationship includes that the question-answer pair needs to be searched in the question-answer knowledge base, and the question-answer pair does not need to be searched in the question-answer knowledge base; determining the target answer information according to the question-answer pair and the constraint relationship includes: If it is detected that the constraint relationship is that the question-answer pair needs to be searched in the question-answer knowledge base, the target answer information is determined according to the question-answer pair and the question-answer knowledge base; If it is detected that the constraint relationship is that the question-answer pair does not need to be searched in the question-answer knowledge base, the target answer information is determined to be the first answer information.

3. The method according to claim 2, characterized in that The determining target answer information according to the question-answer pair and the question-answer knowledge base includes: Determine, according to the length of the question-answer pair, a first question-answer knowledge base in the question-answer knowledge base corresponding to the question-answer pair, wherein the question-answer knowledge base includes a long text question-answer knowledge base and a short text question-answer knowledge base; Performing a best match search based on the question-answer pair and the first question-answer knowledge base to obtain a first set of related question-answer pairs; By vectorizing the question-answer pair and the first question-answer knowledge base, a vectorized question-answer pair and a second question-answer knowledge base are obtained; Perform a cosine similarity search based on the vectorized question-answer pair and the second question-answer knowledge base to obtain a second set of related question-answer pairs; Obtaining a retrieval result by inputting the question-answer pair, the first question-answer pair set, and the second question-answer pair set into a refined ranking model; The target reply information is determined according to the retrieval result and a preset threshold.

4. The method according to claim 3, characterized in that The search result includes a plurality of question-answer pairs sorted according to relevance, and the step of determining target answer information according to the search result and a preset threshold includes: If it is detected that the search result is less than the preset threshold, determining the target reply information as a default reply; If it is detected that the retrieval result is not less than the preset threshold, the target answer information is determined based on the answer and label information of the question-answer pair with the highest relevance in the retrieval result, wherein the label information includes at least one of a customer label, an activity label or a holiday label.

5. The method according to claim 1, characterized in that Before obtaining a question-answer pair according to the input customer question and the pre-trained first question-answer model, the method further includes: Obtaining the pre-trained first question-answering model according to the question-answering training data; The pre-trained question-answer pair classification model is obtained according to the question-answer training data and the initial classification model.

6. The method according to claim 5, characterized in that The step of obtaining a pre-trained first question-answering model according to the question-answering training data includes: The question-and-answer training data is obtained by combining the push information and the question-and-answer information, wherein the push information includes push background information, push process information, and push object information; The initial question-answering model is supervised and fine-tuned based on the question-answering training data to obtain a pre-trained first question-answering model.

7. The method according to claim 5, characterized in that The step of obtaining a pre-trained question-answer pair classification model according to the question-answer training data and the initial classification model includes: By splitting the multiple questions and answers in the training data, a plurality of question-answer pairs and classification labels corresponding to the multiple question-answer pairs are obtained, wherein the classification labels are used to determine the steps of the question-answer pairs in the push process; The initial classification model is trained according to the multiple question-answer pairs and the classification labels corresponding to the multiple question-answer pairs to obtain a pre-trained question-answer pair classification model.

8. The method according to any one of claims 1 to 7, characterized in that: Before obtaining the question-answer pair classification result according to the question-answer pair and the pre-trained question-answer pair classification model, the method further includes: Determine whether there is manual dialing data in the current scenario; If it is detected that there is manual dialing data in the current scenario, a first manual dialing question-answer pair is obtained according to the manual dialing data; the question-answer pair is deduplicated according to clustering and / or an intention recognition method to obtain a second manual dialing question-answer pair; a third manual dialing question-answer pair is obtained by replacing the customer information field in the second manual dialing question-answer pair with a preset identifier; a question-answer knowledge base is constructed by obtaining a set of the third manual dialing question-answer pairs; If it is detected that there is no manual dialing data in the current scenario, a first non-manual dialing question and answer pair is obtained based on the customer question set in the current scenario and the answers corresponding to the customer question set; a second non-manual dialing question and answer pair is obtained by replacing the customer information field in the first non-manual dialing question and answer pair with a preset identifier; and a question and answer knowledge base is constructed by obtaining the set of the second non-manual dialing question and answer pairs.

9. An information processing device, characterized in that: The device comprises: A processing module, configured to obtain a question-answer pair based on an input customer question and a pre-trained first question-answer model; and to obtain a question-answer pair classification result based on the question-answer pair and the pre-trained question-answer pair classification model, wherein the question-answer pair classification result includes a constraint relationship between a preset question-answer knowledge base and the question-answer pair; A determination module is used to determine target answer information according to the question-answer pair and the constraint relationship, and reply to the customer question based on the target answer information.

10. A computer, characterized in that: include: A memory, a processor, and an executable program code stored in the memory and executable on the processor, wherein the processor executes the steps of the information processing method according to any one of claims 1 to 8 when executing the executable program code.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores executable program code, wherein the executable program code includes execution instructions, and the execution instructions are used to execute the steps of the information processing method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is used to make a computer execute the steps of the information processing method according to any one of claims 1 to 8.

Images