User question and answer processing method and device, storage medium and electronic equipment

By using offset indicators to optimize the deletion and deletion of the Q&A model in the Q&A model, the problem of low response rate of the Q&A model is solved, the accuracy and user experience of the reply text are improved, and the number of human participation is reduced.

CN118917391APending Publication Date: 2024-11-08HANGZHOU PINGPONG INTELLIGENT TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410840015.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-26
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The question-and-answer model based on the big language model has a low response rate due to frequent updates of documents and overly reliance on specific keywords in the question, requiring manual participation in response, increasing labor costs and reducing business efficiency.

Method used

The initial Q&A model is optimized using offset indicators and training sets. By receiving questions input by users and generating reply text, the initial Q&A model is optimized to improve the accuracy of response by using offset indicators including question-and-answer matching, context recall and question-and-answer correlation. sex.

Benefits of technology

It improves the accuracy of the question-and-answer model to generate reply text, reduces the number of times human participation in question-and-answer, and improves the user experience and the work efficiency of business personnel.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118917391A_ABST
    Figure CN118917391A_ABST
Patent Text Reader

Abstract

The invention discloses a user question and answer processing method and device, a storage medium and electronic equipment, the method is applied to the field of artificial intelligence, and the method comprises the following steps: receiving text information input by a target user to obtain a target text; and inputting the target text into the target question and answer model to generate a reply text corresponding to the target text. By means of the question answering method and device, the problem that when a question answering model based on a large language model answers a user question in the related technology, the response rate of the question answering model is low due to the fact that documents are frequently updated and excessively depend on specific keywords in the question is solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a method, device, storage medium and electronic device for processing user questions and answers. Background Art

[0002] In order to improve the work efficiency of enterprises, a question-answering robot system based on large language models is proposed. The question-answering robot can collect and organize documents from various departments of the company, integrate them into a knowledge base, and then use the currently popular large language model (LLM) and retrieval-augmented generation (RAG) mechanism to automatically answer questions entered by users.

[0003] In the prior art, users can input questions through text, voice or other means. The question-answering robot pre-processes the user's question (for example, word segmentation, removal of stop words, part-of-speech tagging, etc.), uses machine learning methods (for example, deep learning) to identify the user's intention, and then extracts key information from the user's question (for example, time, place, people, etc.), understands the context of the key information, understands the semantics of the user's question, and retrieves relevant information in the knowledge base based on the understood semantic information. Finally, based on the retrieved information, the robot generates an answer and presents the generated answer to the user in text, voice or other forms.

[0004] However, since the knowledge information in the knowledge base is updated frequently and multiple people may update it at the same time, if the user does not enter specific keywords, the matching effect between the knowledge information in the knowledge base and the user's question will be poor. For more complex questions, the question-answering robot will refuse to answer, requiring manual participation in the reply work, which increases labor costs and reduces the work efficiency of business personnel.

[0005] In summary, in the prior art, when a question-answering model based on a large language model is used to answer user questions, the response rate of the question-answering model is low due to frequent document updates and excessive reliance on specific keywords in the questions. Summary of the invention

[0006] The main purpose of the present application is to provide a method, device, storage medium and electronic device for processing user questions and answers, so as to solve the problem in the related art that when a question and answer model based on a large language model answers user questions, the response rate of the question and answer model is low due to frequent document updates and excessive reliance on specific keywords in the question.

[0007] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for processing user questions and answers is provided, the method comprising: receiving text information input by a target user to obtain a target text, wherein the target text contains question information; inputting the target text into a target question and answer model to generate a reply text corresponding to the target text, wherein the target question and answer model refers to a model obtained by optimizing an initial question and answer model using an offset indicator and a training set, the offset indicator is used to characterize the difference between a reply generated by the question and answer model and an actual reply, and the training set contains a reply text generated according to a current business document.

[0008] Furthermore, before inputting the target text into the target question-answering model and generating a reply text corresponding to the target text, the method also includes: obtaining a target document set, and segmenting the documents in the target document set to obtain segmented documents, wherein the target document set includes target documents that record business information; inputting the segmented documents into the initial question-answering model to generate replies corresponding to historical questions to obtain a question-answering text set, wherein the historical questions are questions asked by users regarding the business information; calculating the offset value of the initial question-answering model based on the offset index and the question-answering text set, wherein the offset index includes at least the following indicators: question-answer matching, context recall, and question-answer relevance; when the offset value is greater than a preset threshold, generating the training set based on the segmented documents, and optimizing the initial question-answering model using the training set to obtain the target question-answering model.

[0009] Furthermore, the offset value of the initial question-answering model is calculated based on the offset index and the question-answer text set, including: calculating the question-answer matching degree based on the historical question, the reply corresponding to the historical question and the segmented document, wherein the question-answer matching degree is used to characterize the matching degree between the reply corresponding to the historical question and the segmented document; calculating the context recall rate based on the argument information in the reply corresponding to the historical question and the argument information in the context information, wherein the context information is the document information corresponding to the historical question in the segmented document, and the context recall rate is used to characterize the recall rate of the argument of the context information in the reply corresponding to the historical question; generating a similar question corresponding to the reply, and calculating the question-answer relevance based on the similarity between the historical question and the similar question; calculating the question-answer matching degree, the context recall rate and the question-answer relevance to obtain the offset value of the initial question-answering model.

[0010] Furthermore, the question-answer matching degree is calculated based on the historical questions, the replies corresponding to the historical questions and the segmented documents, including: determining answerable questions based on the context information, and determining the replies corresponding to the answerable questions; calculating the matching degree between the answerable questions and the replies corresponding to the answerable questions, and determining the number of questions and answers with the matching degree greater than a preset threshold, and obtaining the number of faithful answers; calculating the question-answer matching degree based on the number of faithful answers and the number of answerable questions.

[0011] Furthermore, the context recall rate is calculated based on the argument information in the reply corresponding to the historical question and the argument information in the context information, including: extracting the arguments contained in the context information to obtain a first number of arguments; extracting the arguments contained in the reply corresponding to the historical question to obtain a second number of arguments; calculating the context recall rate based on the first number of arguments and the second number of arguments.

[0012] Furthermore, a target document set is obtained, and the documents in the target document set are divided into blocks to obtain the divided documents, including: obtaining the target document from a storage medium through a data interface every preset time to obtain the target document set; dividing the target documents in the target document set into blocks and converting them into embedded vectors to obtain N document blocks, wherein there are overlapping characters between the N document blocks, and N is a positive integer greater than 1; and determining the divided documents based on the N document blocks.

[0013] Furthermore, the training set is generated based on the segmented documents, including: processing the historical questions and the segmented documents to generate M questions, where M is a positive integer greater than 1; processing the M questions and the segmented documents to generate replies corresponding to the M questions; and combining the M questions and the replies corresponding to each question into M question-answer texts to obtain the training set.

[0014] Furthermore, the historical questions and the segmented documents are processed to generate M questions, including: extracting keywords from the historical questions to obtain a keyword set; for each keyword in the keyword set, calculating the first similarity between the keyword and each document block in the segmented document; determining the first document block set corresponding to the keyword based on the first similarity that meets a preset condition, wherein the preset condition means that the sum of the first similarities is greater than a preset threshold and the number of the added first similarities is within a preset interval; and generating the M questions based on the first document block set corresponding to each keyword in the keyword set.

[0015] Furthermore, the M questions and the segmented documents are processed to generate replies corresponding to the M questions, including: for each of the M questions, calculating the second similarity between the question and each document block in the segmented documents; determining a second document block set corresponding to the question based on the second similarities that meet a preset condition, wherein the preset condition means that the sum of the second similarities is greater than a preset threshold and the number of added second similarities is within a preset interval; and generating replies corresponding to the M questions based on the second document block set corresponding to each of the M questions.

[0016] In order to achieve the above-mentioned purpose, according to another aspect of the present application, a user question and answer processing device is provided, which includes: a receiving unit, used to receive text information input by a target user to obtain a target text, wherein the target text contains question information; a first generating unit, used to input the target text into a target question and answer model to generate a reply text corresponding to the target text, wherein the target question and answer model refers to a model obtained by optimizing an initial question and answer model using an offset indicator and a training set, the offset indicator is used to characterize the difference between the reply generated by the question and answer model and the actual reply, and the training set contains the reply text generated according to the current business document.

[0017] Furthermore, the device also includes: an acquisition unit, which is used to acquire a target document set before inputting the target text into the target question and answer model and generating a reply text corresponding to the target text, and to block the documents in the target document set to obtain block documents, wherein the target document set includes target documents that record business information; a second generation unit, which is used to input the block documents into the initial question and answer model, generate replies corresponding to historical questions, and obtain a question and answer text set, wherein the historical questions are questions asked by users regarding the business information; a calculation unit, which is used to calculate the offset value of the initial question and answer model based on the offset index and the question and answer text set, wherein the offset index includes at least the following indicators: question and answer matching degree, context recall rate, and question and answer relevance; an optimization unit, which is used to generate the training set based on the block documents when the offset value is greater than a preset threshold, and optimize the initial question and answer model using the training set to obtain the target question and answer model.

[0018] Furthermore, the calculation unit includes: a first calculation subunit, used to calculate the question-answer matching degree based on the historical question, the reply corresponding to the historical question and the segmented document, wherein the question-answer matching degree is used to characterize the matching degree between the reply corresponding to the historical question and the segmented document; a second calculation subunit, used to calculate the context recall rate based on the argument information in the reply corresponding to the historical question and the argument information in the context information, wherein the context information is the document information corresponding to the historical question in the segmented document, and the context recall rate is used to characterize the recall rate of the argument of the context information in the reply corresponding to the historical question; a third calculation subunit, used to generate a similar question corresponding to the reply, and calculate the question-answer relevance based on the similarity between the historical question and the similar question; a fourth calculation subunit, used to calculate the question-answer matching degree, the context recall rate and the question-answer relevance to obtain the offset value of the initial question-answer model.

[0019] Furthermore, the first calculation subunit includes: a first determination module, used to determine answerable questions based on the context information, and determine replies corresponding to the answerable questions; a second determination module, used to calculate the degree of matching between the answerable questions and the replies corresponding to the answerable questions, and determine the number of questions and answers with the degree of matching greater than a preset threshold, and obtain the number of faithful answers; a first calculation module, used to calculate the question-answer matching degree based on the number of faithful answers and the number of answerable questions.

[0020] Furthermore, the second computing sub-unit includes: a first extraction module, used to extract the arguments contained in the context information to obtain a first number of arguments; a second extraction module, used to extract the arguments contained in the replies corresponding to the historical questions to obtain a second number of arguments; and a second computing module, used to calculate the context recall rate based on the first number of arguments and the second number of arguments.

[0021] Furthermore, the acquisition unit includes: an acquisition subunit, used to acquire the target document from the storage medium through the data interface every preset time to obtain the target document set; a first processing subunit, used to block the target documents in the target document set and convert them into embedded vectors to obtain N document blocks, wherein there are overlapping characters between the N document blocks, and N is a positive integer greater than 1; a determination subunit, used to determine the block-based documents based on the N document blocks.

[0022] Furthermore, the optimization unit includes: a second processing sub-unit, used to process the historical questions and the block-divided documents to generate M questions, where M is a positive integer greater than 1; a third processing sub-unit, used to process the M questions and the block-divided documents to generate replies corresponding to the M questions; and a fourth processing sub-unit, used to combine the M questions and the replies corresponding to each question into M question-answer texts to obtain the training set.

[0023] Furthermore, the second processing sub-unit includes: a third extraction module, used to extract keywords from the historical questions to obtain a keyword set; a third calculation module, used to calculate, for each keyword in the keyword set, the first similarity between the keyword and each document block in the segmented document; a third determination module, used to determine the first document block set corresponding to the keyword based on the first similarity that meets a preset condition, wherein the preset condition means that the sum of the first similarities is greater than a preset threshold and the number of the added first similarities is within a preset interval; a first generation module, used to generate the M questions based on the first document block set corresponding to each keyword in the keyword set.

[0024] Furthermore, the third processing sub-unit includes: a fourth calculation module, used to calculate, for each of the M questions, the second similarity between the question and each document block in the divided documents; a fourth determination module, used to determine the second document block set corresponding to the question based on the second similarity that meets a preset condition, wherein the preset condition means that the sum of the second similarities is greater than a preset threshold and the number of added second similarities is within a preset interval; a second generation module, used to generate replies corresponding to the M questions based on the second document block set corresponding to each of the M questions.

[0025] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for processing user questions and answers, and which, when executed by a processor, implements the steps of the method for processing user questions and answers described in each embodiment of the present application.

[0026] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes stored computer instructions, wherein when the computer instructions are executed by a processor, any one of the above-mentioned user question and answer processing methods is implemented.

[0027] In order to achieve the above-mentioned purpose, according to one aspect of the present application, an electronic device is provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned user question and answer processing methods.

[0028] Through this application, the following steps are adopted: receiving text information input by the target user to obtain a target text, wherein the target text contains question information; inputting the target text into the target question-answering model to generate a reply text corresponding to the target text, wherein the target question-answering model refers to a model obtained by optimizing the initial question-answering model using an offset index and a training set, the offset index is used to characterize the difference between the reply generated by the question-answering model and the actual reply, and the training set contains a reply text generated according to the current business document, which solves the problem that the question-answering model based on a large language model in the related art has a low response rate due to frequent document updates and excessive reliance on specific keywords in the question. By receiving the question input by the user, the reply text of the user's question can be automatically generated by the question-answering model, and the initial question-answering model can be optimized by using the offset index and the training set generated by the latest business document, and the optimized target question-answering model can be used to answer the user's question, thereby improving the accuracy of the reply text generated by the question-answering model, achieving the effect of improving the response rate of the question-answering model, and reducing the number of manual participation in the question-answering, and further achieving the effect of improving the user experience and the work efficiency of the business personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0030] Figure 1 is a flowchart of a method for processing user questions and answers provided in Embodiment 1 of the present application;

[0031] Figure 2 is a schematic diagram of an optional user question and answer processing method provided according to the first embodiment of the present application;

[0032] Figure 3 is a schematic diagram of a user question and answer processing device provided according to Embodiment 2 of the present application;

[0033] Figure 4 It is a schematic diagram of an electronic device for processing user questions and answers provided in Embodiment 5 of the present application. DETAILED DESCRIPTION

[0034] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0035] It should be noted that the user information (including but not limited to user device information, user personal information, collected data, used data, generated data, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, collected information, used information, generated information, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data are in compliance with the relevant laws, regulations and standards of relevant countries and regions, necessary confidentiality measures are taken, and public order and good morals are not violated, and corresponding operation portals are provided for users to choose to authorize or refuse. For example, an interface is set between this system and relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0036] It should be noted that this application provides users with corresponding operation entrances for them to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0037] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0039] Embodiment 1

[0040] The present invention is described below in conjunction with preferred implementation steps. Figure 1 is a flowchart of a method for processing user questions and answers provided in Example 1 of the present application, such as Figure 1 As shown, the method comprises the following steps:

[0041] Step S101, receiving text information input by a target user to obtain a target text, wherein the target text contains question information.

[0042] In the first embodiment, in order to answer questions raised by users or internal business personnel, it is necessary to receive the question text (i.e., the target text mentioned above) input by the user or business personnel (i.e., the target user mentioned above), so that the target text can be input into the question-answering model to automatically generate a reply text for the user's question, and then the received questions can be answered without human intervention, thereby achieving the effect of improving the work efficiency of the business personnel.

[0043] Step S102: input the target text into the target question-answering model to generate a reply text corresponding to the target text, wherein the target question-answering model refers to a model obtained by optimizing the initial question-answering model using an offset indicator and a training set, the offset indicator is used to characterize the difference between the reply generated by the question-answering model and the actual reply, and the training set contains the reply text generated according to the current business document.

[0044] In the first embodiment of the present invention, in order to avoid the problem that the frequently updated business documents make the reply text generated by the question-answering model less accurate, it is necessary to use a preset offset index to evaluate the initial question-answering model, optimize and adjust the initial question-answering model according to the evaluation results and the training set, and obtain the optimized target question-answering model, so as to use the target question-answering model to generate reply texts for user questions, thereby achieving the effect of improving the accuracy of the reply texts generated by the question-answering model, and further achieving the effect of improving the user experience and the work efficiency of business personnel, wherein the training set includes multiple groups of question-answering texts, and the multiple groups of question-answering texts are generated based on the knowledge base composed of the latest business documents, and each group of question-answering texts consists of a question text and its corresponding reply text. Specifically, the initial question-answering model can be an LLM, or it can be an LLM after fine-tuning or retraining.

[0045] In summary, the user question and answer processing method provided in the first embodiment of the present application receives the text information input by the target user to obtain the target text, wherein the target text contains the question information; the target text is input into the target question and answer model to generate the reply text corresponding to the target text, wherein the target question and answer model refers to the model obtained by optimizing the initial question and answer model by using the offset index and the training set, the offset index is used to characterize the difference between the reply generated by the question and answer model and the actual reply, and the training set contains the reply text generated according to the current business document, which solves the problem that the question and answer model based on the large language model in the related technology answers the user's question, due to the frequent update of the document and the over-reliance on specific keywords in the question, resulting in a low response rate of the question and answer model. By receiving the question input by the user, the reply text of the user's question can be automatically generated by the question and answer model, and the initial question and answer model can be optimized by using the training set generated by the offset index and the latest business document, and the optimized target question and answer model can be used to answer the user's question, thereby improving the accuracy of the reply text generated by the question and answer model, achieving the effect of improving the response rate of the question and answer model, and reducing the number of manual participation in the question and answer, and further achieving the effect of improving the user experience and the work efficiency of the business personnel.

[0046] Optionally, in the user question and answer processing method provided in Example 1 of the present application, before inputting the target text into the target question and answer model and generating a reply text corresponding to the target text, the above method also includes: obtaining a target document set, and segmenting the documents in the target document set to obtain segmented documents, wherein the target document set includes target documents that record business information; inputting the segmented documents into the initial question and answer model, generating replies corresponding to historical questions, and obtaining a question and answer text set, wherein the historical questions are questions asked by users regarding business information; calculating the offset value of the initial question and answer model based on the offset index and the question and answer text set, wherein the offset index includes at least the following indicators: question and answer matching degree, context recall rate, and question and answer relevance; when the offset value is greater than a preset threshold, generating a training set based on the segmented documents, and optimizing the initial question and answer model of the training set to obtain a target question and answer model.

[0047] In the first embodiment of the present invention, in order to improve the response rate and reply accuracy of the question and answer model, the offset value of the initial question and answer model of the reply text of the user's historical questions can be generated through the offset indicator, the user's historical questions, and the initial question and answer model. The initial question and answer model is optimized according to the training set generated by the offset value and the latest business document to obtain the target question and answer model, so as to use the target question and answer model to reply to the questions raised by the user.

[0048] In an optional embodiment, the document can be divided into document blocks so that the question-answering model can analyze and process the context information of the document and improve the computing efficiency at the same time, wherein the target document refers to the document edited by the business personnel according to the business needs or other business information, and the target document is also the latest document edited by the current business personnel, and the target document set (i.e., the above-mentioned knowledge base) contains multiple target documents. Then, the block-divided document is input into the initial question-answering model, and the reply text corresponding to the historical question is generated by the initial question-answering model. A group of question-answering texts is composed of historical questions and their corresponding reply texts, and the above-mentioned question-answering text set is composed of multiple groups of question-answering texts, wherein the historical question refers to the historical question that has been raised by the user or business personnel for various business information. Secondly, each group of question-answering texts in the question-answering text set is processed according to the calculation method of the offset index to obtain the offset value of the initial question-answering model, wherein the offset index includes at least the following indicators: question-answering matching degree, context recall rate, and question-answering relevance. Finally, when the offset value is greater than the preset threshold, a training set is generated based on the latest document (i.e., the above-mentioned segmented document), and the initial question-answering model is fine-tuned and optimized using the training set to obtain the above-mentioned target question-answering model.

[0049] Optionally, in the user question and answer processing method provided in Example 1 of the present application, the offset value of the initial question and answer model is calculated based on the offset index and the question and answer text set, including: calculating the question and answer matching degree based on historical questions, the replies corresponding to the historical questions and the block-based documents, wherein the question and answer matching degree is used to characterize the degree of matching between the replies corresponding to the historical questions and the block-based documents; calculating the context recall rate based on the argument information in the replies corresponding to the historical questions and the argument information in the context information, wherein the context information is the document information corresponding to the historical questions in the block-based documents, and the context recall rate is used to characterize the recall rate of the arguments of the context information in the replies corresponding to the historical questions; generating similar questions corresponding to the reply text, and calculating the question and answer relevance based on the similarity between the historical questions and the similar questions; calculating the question and answer matching degree, the context recall rate and the question and answer relevance to obtain the offset value of the initial question and answer model.

[0050] In the first embodiment of the present invention, the question-answer matching degree refers to the proportion of question-answer texts in which questions and replies match in the question-answer texts generated based on the context information, the context recall rate refers to the proportion of arguments in the reply information to the arguments in the context information, and the question-answer relevance refers to the degree of relevance between questions and replies in the question-answer text set.

[0051] Specifically, the question-answer matching degree and context recall rate measure the matching degree of each set of question-answer texts from the overall and specific levels respectively, and are used to characterize the factual validity of the questions and the traceability of the arguments. The question-answer relevance indicator excludes the factor of contextual information and only evaluates the correlation between questions and replies from the question-answer level, thereby ensuring the question-answer model's ability to understand the current target document set. At the same time, it avoids common sense errors caused by the decline of the basic functions of LLM due to model fine-tuning, thereby achieving the effect of improving the accuracy of the question-answer model.

[0052] In an optional embodiment, when calculating the question-answer relevance, the generated reply information can be input into the LLM, and the similar questions of each reply information can be calculated, so as to calculate the similarity between the similar questions of each reply information and the actual question of each reply information. The calculation formula can be shown as Formula 1:

[0053]

[0054] Among them, Answer Relevancy represents the above question-answer relevance, N represents the total number of questions, Q o Represents the embedded vector corresponding to the user's historical questions, Q i, represents the embedded vector of the i-th question generated by LLM based on the reply text, and cos represents the cosine similarity between embedded vectors. In actual situations, the number of arguments can be set to N = 3. On the one hand, it can reduce the computational cost. On the other hand, through experimental verification, N = 3 is an optimal boundary effect value, and the value does not change much after N>3. The calculated question-answer relevance is used to evaluate the completeness of the reply text and the redundancy of the reply text. For example, a reply text that is incomplete or contains redundant information will receive a lower question-answer relevance score.

[0055] In an optional embodiment, an example of generating similar questions based on the reply text may be as follows: {context information: "Only first-category accounts can receive rubles, and now new users can no longer apply to open first-category accounts"; question: "Does first-category trade support rubles?"; reply: "Old users can receive it through first-category accounts, but new users do not."; similar question 1: "How do old users of first-category trade receive rubles?"; similar question 2: "Can new users of first-category trade receive rubles?"; similar question 3: "Can first-category trade receive rubles?"}.

[0056] In an optional embodiment, in order to comprehensively evaluate the degree of deviation of the initial question-answering model, an indicator of an deviation value can be set to comprehensively evaluate the degree of deviation in the above three aspects. Specifically, the deviation value of the initial question-answering model can be calculated using the average value of the three indicators. If the change in the deviation value exceeds 0.3, or the absolute value of the deviation value is lower than 0.8, the initial question-answering model needs to be fine-tuned and optimized.

[0057] Optionally, in the user question and answer processing method provided in Example 1 of the present application, the question and answer matching degree is calculated based on historical questions, replies corresponding to historical questions and segmented documents, including: determining answerable questions based on context information, and determining replies corresponding to answerable questions; calculating the matching degree between answerable questions and replies corresponding to answerable questions, and determining the number of questions and answers with a matching degree greater than a preset threshold, and obtaining the number of faithful answers; calculating the question and answer matching degree based on the number of faithful answers and the number of answerable questions.

[0058] In the first embodiment of the present invention, the question-answer matching degree represents the degree of matching between the replies corresponding to the user's historical questions and the context information, which can also be called answer loyalty (Faithfulness). Based on the context information, answerable questions are determined, and reply texts corresponding to the answerable questions are generated. LLM is used to evaluate whether the answerable questions and reply texts corresponding to the answerable questions can form a set of appropriate questions and answers, so as to measure the factual consistency between the generated reply text (answer) and the given context information (context), and avoid answering irrelevant questions. The formula for calculating the question-answer matching degree can be shown in Formula 2,

[0059]

[0060] Among them, faithfulness represents the above-mentioned question-answer matching degree, "the number of questions matched based on the context" represents the number of questions that match the answerable questions and the reply texts corresponding to the answerable questions assessed by LLM (i.e. the number of faithful answers mentioned above, for example, the degree of match between the answerable questions and the replies corresponding to the answerable questions can be calculated, and the number of questions matched based on the context is determined based on the number of questions and answers with a matching degree greater than a preset threshold), and "total number of questions" refers to the total number of answerable questions.

[0061] In an optional embodiment, when the reply text generated by the initial question-answer model is loyal, the question-answer matching score is high; when the reply text generated by the initial question-answer model is not loyal, the question-answer matching score is low. An example of a reply text of the initial question-answer model that is not loyal (i.e., answering a question that is not asked) may be as follows: {context information: "Only first-class accounts can receive rubles, and now new users can no longer apply to open first-class accounts"; question: "Does first-class trade support rubles?"; reply: "First-class trade supports 200 currencies."}. An example of a reply text of the initial question-answer model that is loyal may be as follows: {context information: "Only first-class accounts can receive rubles, and now new users can no longer apply to open first-class accounts"; question: "Does first-class trade support rubles?"; reply: "Old users can receive through first-class accounts, but new users do not."}.

[0062] Optionally, in the user question and answer processing method provided in Example 1 of the present application, the context recall rate is calculated based on the argument information in the replies corresponding to the historical questions and the argument information in the context information, including: extracting the arguments contained in the context information to obtain a first number of arguments; extracting the arguments contained in the replies corresponding to the historical questions to obtain a second number of arguments; calculating the context recall rate based on the first number of arguments and the second number of arguments.

[0063] In the first embodiment, the context recall rate refers to the ratio of arguments in the reply text to arguments that can be supported by context information, which can also be called context recall support rate (Context Recall). Specifically, the reply text generated by the initial question-answering model can be split into several arguments through LLM, and each sentence argument in the reply text can be detected to see whether it can be attributed to the retrieved context information. The formula for calculating the context recall rate can be shown in Formula 3:

[0064]

[0065] Among them, Context Recall represents the context recall rate mentioned above, “the number of arguments obtained through context information” refers to the number of arguments extracted by the LLM model from the context information, that is, the first number of arguments mentioned above, and “the number of arguments in the reply” refers to the number of arguments extracted by the LLM model from the reply text generated by the initial question-answering model, that is, the second number of arguments mentioned above.

[0066] In an optional embodiment, when the reply text generated by the initial question-answering model recalls more arguments in the context information, the context recall rate score is higher; when the reply text generated by the initial question-answering model recalls fewer arguments in the context information, the context recall rate score is lower. An example of the initial question-answering model's reply text recalling some arguments in the context information may be as follows: {context information: "Only the first category of accounts can receive rubles, and now new users can no longer apply to open a first category of accounts"; question: "Does the first category of trade support rubles?"; reply: "Old users can receive through the first category of accounts."}. An example of the initial question-answering model's reply text recalling all arguments in the context information may be as follows: {context information: "Only the first category of accounts can receive rubles, and now new users can no longer apply to open a first category of accounts"; question: "Does the first category of trade support rubles?"; reply: "Old users can receive through the first category of accounts, but new users cannot."}.

[0067] Optionally, in the user question and answer processing method provided in Example 1 of the present application, a target document set is obtained, and the documents in the target document set are divided into blocks to obtain the divided documents, including: obtaining the target document from the storage medium through the data interface every preset time to obtain the target document set; dividing the target document in the target document set into blocks and converting them into embedded vectors to obtain N document blocks, wherein there are overlapping characters between the N document blocks, and N is a positive integer greater than 1; determining the divided documents based on the N document blocks.

[0068] In the first embodiment, in order to obtain data that can be processed by the question-answering model, it is necessary to perform a preprocessing operation on the target document.

[0069] In an optional embodiment, business documents (i.e., the above-mentioned target documents) can be retrieved from a network disk (i.e., the above-mentioned storage medium, not specifically limited in this embodiment) through an API interface at a preset time every day (e.g., 23:00 and / or 12:00 every day, not specifically limited in this embodiment 1) or every preset time (e.g., 6 hours, 12 hours, etc., not specifically limited in this embodiment 1) to obtain a target document set. Then, the target documents in the target document set are divided into blocks according to the preset string length (e.g., 500 tokens), and converted into embedded vectors through an embedded model to obtain the above-mentioned N document blocks, which constitute the above-mentioned divided documents.

[0070] It should be noted that some overlapping characters need to be retained between adjacent document blocks (for example, 20 tokens overlap between each block) to ensure the continuity between document blocks and improve the ability of the question-answering model to recognize and understand contextual information, thereby achieving the effect of improving the accuracy of the question-answering model in identifying and generating reply texts, as well as improving the response rate of the question-answering model.

[0071] Optionally, in the user question and answer processing method provided in Example 1 of the present application, a training set is generated based on the segmented documents, including: processing historical questions and segmented documents to generate M questions, where M is a positive integer greater than 1; processing the M questions and segmented documents to generate replies corresponding to the M questions; and combining the M questions and the replies corresponding to each question into M question and answer texts to obtain a training set.

[0072] In the first embodiment of the present invention, when the offset value is greater than a preset threshold, the latest business document (i.e., the above-mentioned segmented document) and the user's historical questions can be used to generate M questions, and a reply text for each of the M questions can be generated, thereby forming M question-answer texts based on the M questions and the replies corresponding to each question, and obtaining the above-mentioned training set.

[0073] In an optional embodiment, since some business information may appear simultaneously in multiple target documents, it is necessary to deduplicate the questions with high overlap among the M questions to reduce redundant data and improve data quality.

[0074] Optionally, in the user question and answer processing method provided in Example 1 of the present application, historical questions and segmented documents are processed to generate M questions, including: extracting keywords from historical questions to obtain a keyword set; for each keyword in the keyword set, calculating the first similarity between the keyword and each document block in the segmented document; determining the first document block set corresponding to the keyword based on the first similarity that meets a preset condition, wherein the preset condition means that the sum of the first similarities is greater than a preset threshold and the number of added first similarities is within a preset interval; and generating M questions based on the first document block set corresponding to each keyword in the keyword set.

[0075] In an optional embodiment, user history questions can be obtained, and keywords can be extracted from the user history questions through a preset model to obtain the above keyword set. Then, the similarity between each keyword and each document block after segmentation (i.e., the above first similarity) is calculated, and the L document blocks with the highest first similarity are determined, and the sum of the first similarities of the L document blocks is greater than a preset value (e.g., 0.9), forming the first document block set corresponding to each keyword (i.e., the above first document block set corresponding to the keyword is determined based on the first similarity that meets the preset conditions). Finally, M questions are generated based on the first document block set corresponding to each keyword.

[0076] By extracting keywords from historical questions and determining the document block with the highest similarity based on the keywords, we can generate questions with a high degree of content consistency with the current target document set, avoiding the problem of low response rate of the question-answering model due to document updates, thereby improving the response rate of the question-answering model and the accuracy of the reply text generated by the question-answering model.

[0077] Optionally, in the user question and answer processing method provided in Example 1 of the present application, M questions and block-based documents are processed to generate replies corresponding to the M questions, including: for each of the M questions, calculating the second similarity between the question and each document block in the block-based documents; determining a second document block set corresponding to the question based on the second similarities that meet a preset condition, wherein the preset condition means that the sum of the second similarities is greater than a preset threshold and the number of added second similarities is within a preset interval; and generating replies corresponding to the M questions based on the second document block set corresponding to each of the M questions.

[0078] In an optional embodiment, after generating M questions, the similarity between each question in the M questions and each document block after segmentation (i.e., the second similarity mentioned above) can be calculated, and the P document blocks with the highest second similarity corresponding to each question are determined, and the sum of the second similarities of the P document blocks is greater than a preset value (e.g., 0.9), forming a second block set corresponding to each question (i.e., the second document block set corresponding to the question is determined based on the second similarity that meets the preset conditions). Finally, the reply text corresponding to each question is generated based on the second document block set corresponding to each question, i.e., the replies corresponding to the M questions mentioned above.

[0079] By calculating the similarity between each question and the latest document block, and generating the corresponding reply text for each question based on the document block with the highest similarity, it is possible to generate reply text matching the question based on the current target document set, thereby improving the accuracy of the reply text generated by the question-answering model and avoiding the problem of low response rate of the question-answering model due to document updates.

[0080] Optionally, in the first embodiment, the process of processing user problems in this solution can be as follows: Figure 2 First, business personnel (such as Figure 2 The knowledge base administrator in the repository updates the target document collection (such as Figure 2 The target document in the knowledge base in the updated target document set is used to generate a question and answer text set for evaluating the offset value of the initial question and answer model according to the updated target document set. Then, the question and answer text set is processed and calculated according to the calculation method of the offset index to obtain the offset value of the initial question and answer model. When the offset value of the initial question and answer model is greater than the preset threshold, a training set is generated according to the updated target document set, and the initial question and answer model is optimized and fine-tuned according to the offset value and the training set (for example, the model parameters of the initial question and answer model are updated) to obtain the optimized target question and answer model. Finally, when a user question is received, the question text entered by the user is input into the target question and answer model, and a reply text is generated by the optimized target question and answer model, and the reply text is displayed to the user who input the question. In addition, if the offset value of the initial question and answer model is less than or equal to the preset threshold, there is no need to optimize and fine-tune the initial question and answer model. When a user question is received, the initial question and answer model can continue to be used to generate a reply text to reply to the user's question, and the reply text is displayed to the user who input the question.

[0081] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0082] Embodiment 2

[0083] The second embodiment of the present application also provides a user question and answer processing device. It should be noted that the user question and answer processing device of the second embodiment of the present application can be used to execute the user question and answer processing method provided in the first embodiment of the present application. The user question and answer processing device provided in the second embodiment of the present application is introduced below.

[0084] Figure 3 Schematic diagram of a user question and answer processing device according to the second embodiment of the present application. Figure 3 As shown, the device includes: a receiving unit 301 and a first generating unit 302.

[0085] Specifically, the receiving unit 301 is used to receive text information input by a target user to obtain a target text, wherein the target text contains question information.

[0086] The first generating unit 302 is used to input the target text into the target question-answering model to generate a reply text corresponding to the target text, wherein the target question-answering model refers to a model obtained by optimizing the initial question-answering model using an offset index and a training set, the offset index is used to characterize the difference between the reply generated by the question-answering model and the actual reply, and the training set contains the reply text generated according to the current business document.

[0087] The user question and answer processing device provided in the second embodiment of the present application receives the text information input by the target user through the receiving unit 301 to obtain the target text, wherein the target text contains the question information; the first generating unit 302 inputs the target text into the target question and answer model to generate the reply text corresponding to the target text, wherein the target question and answer model refers to the model obtained by optimizing the initial question and answer model by using the offset index and the training set, the offset index is used to characterize the difference between the reply generated by the question and answer model and the actual reply, and the training set contains the reply text generated according to the current business document, which solves the problem that the question and answer model based on the large language model in the related technology has a low response rate due to the frequent update of the document and the over-reliance on specific keywords in the question. By receiving the question input by the user, the reply text of the user question can be automatically generated by the question and answer model, and the initial question and answer model can be optimized by using the training set generated by the offset index and the latest business document, and the optimized target question and answer model can be used to answer the user question, thereby improving the accuracy of the reply text generated by the question and answer model, achieving the effect of improving the response rate of the question and answer model, and reducing the number of manual participation in the question and answer, and further achieving the effect of improving the user experience and the work efficiency of the business personnel.

[0088] Optionally, in the user question and answer processing device provided in Example 2 of the present application, the above-mentioned device also includes: an acquisition unit, which is used to acquire a target document set before inputting the target text into the target question and answer model and generating a reply text corresponding to the target text, and to block the documents in the target document set to obtain block documents, wherein the target document set includes target documents that record business information; a second generation unit, which is used to input the block documents into the initial question and answer model, generate replies corresponding to historical questions, and obtain a question and answer text set, wherein the historical questions are questions asked by users regarding business information; a calculation unit, which is used to calculate the offset value of the initial question and answer model based on the offset index and the question and answer text set, wherein the offset index includes at least the following indicators: question and answer matching degree, context recall rate, and question and answer relevance; an optimization unit, which is used to generate a training set based on the block documents when the offset value is greater than a preset threshold, and use the initial question and answer model of the training set for optimization to obtain a target question and answer model.

[0089] Optionally, in the user question and answer processing device provided in Example 2 of the present application, the above-mentioned calculation unit includes: a first calculation subunit, used to calculate the question and answer matching degree based on historical questions, replies corresponding to historical questions and block documents, wherein the question and answer matching degree is used to characterize the matching degree between the replies corresponding to historical questions and the block documents; a second calculation subunit, used to calculate the context recall rate based on the argument information in the replies corresponding to historical questions and the argument information in the context information, wherein the context information is the document information corresponding to the historical questions in the block documents, and the context recall rate is used to characterize the recall rate of the arguments of the context information in the replies corresponding to the historical questions; a third calculation subunit, used to generate similar questions corresponding to the replies, and calculate the question and answer relevance based on the similarity between the historical questions and the similar questions; a fourth calculation subunit, used to calculate the question and answer matching degree, the context recall rate and the question and answer relevance to obtain the offset value of the initial question and answer model.

[0090] Optionally, in the user question and answer processing device provided in Example 2 of the present application, the above-mentioned first calculation subunit includes: a first determination module, used to determine answerable questions based on context information, and determine replies corresponding to the answerable questions; a second determination module, used to calculate the degree of match between the answerable questions and the replies corresponding to the answerable questions, and determine the number of questions and answers with a matching degree greater than a preset threshold, and obtain the number of faithful answers; a first calculation module, used to calculate the question and answer matching degree based on the number of faithful answers and the number of answerable questions.

[0091] Optionally, in the user question and answer processing device provided in Example 2 of the present application, the above-mentioned second calculation sub-unit includes: a first extraction module, used to extract arguments contained in the context information to obtain a first number of arguments; a second extraction module, used to extract arguments contained in the replies corresponding to historical questions to obtain a second number of arguments; a second calculation module, used to calculate the context recall rate based on the first number of arguments and the second number of arguments.

[0092] Optionally, in the user question and answer processing device provided in Example 2 of the present application, the above-mentioned acquisition unit includes: an acquisition sub-unit, used to acquire the target document from the storage medium through the data interface every preset time to obtain a target document set; a first processing sub-unit, used to block the target document in the target document set and convert it into an embedded vector to obtain N document blocks, wherein there are overlapping characters between the N document blocks, and N is a positive integer greater than 1; a determination sub-unit, used to determine the block-based document based on the N document blocks.

[0093] Optionally, in the user question and answer processing device provided in Example 2 of the present application, the above-mentioned optimization unit includes: a second processing sub-unit, used to process historical questions and block-based documents to generate M questions, where M is a positive integer greater than 1; a third processing sub-unit, used to process the M questions and block-based documents to generate replies corresponding to the M questions; and a fourth processing sub-unit, used to combine the M questions and the replies corresponding to each question into M question and answer texts to obtain a training set.

[0094] Optionally, in the user question and answer processing device provided in Example 2 of the present application, the above-mentioned second processing sub-unit includes: a third extraction module, used to extract keywords from historical questions to obtain a keyword set; a third calculation module, used to calculate, for each keyword in the keyword set, the first similarity between the keyword and each document block in the block-divided document; a third determination module, used to determine the first document block set corresponding to the keyword based on the first similarity that meets a preset condition, wherein the preset condition means that the sum of the first similarities is greater than a preset threshold and the number of added first similarities is within a preset interval; a first generation module, used to generate M questions based on the first document block set corresponding to each keyword in the keyword set.

[0095] Optionally, in the user question and answer processing device provided in Example 2 of the present application, the above-mentioned third processing sub-unit includes: a fourth calculation module, used to calculate, for each question in the M questions, the second similarity between the question and each document block in the block-divided document; a fourth determination module, used to determine the second document block set corresponding to the question based on the second similarity that meets a preset condition, wherein the preset condition means that the sum of the second similarities is greater than a preset threshold and the number of added second similarities is within a preset interval; a second generation module, used to generate replies corresponding to the M questions based on the second document block set corresponding to each question in the M questions.

[0096] The user question and answer processing device includes a processor and a memory. The above-mentioned receiving unit 301 and the first generating unit 302 are stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0097] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the user experience and the work efficiency of business personnel can be improved by adjusting the kernel parameters.

[0098] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0099] Embodiment 3 of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, a method for processing user questions and answers is implemented.

[0100] A fourth embodiment of the present invention provides a processor, which is used to run a program, wherein the method for processing user questions and answers is executed when the program is running.

[0101] like Figure 4 As shown, embodiment 5 of the present invention provides an electronic device, the device includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor implements the following steps when executing the program: receiving text information input by a target user to obtain a target text, wherein the target text contains question information; inputting the target text into a target question-answering model to generate a reply text corresponding to the target text, wherein the target question-answering model refers to a model obtained by optimizing an initial question-answering model using an offset indicator and a training set, the offset indicator is used to characterize the difference between a reply generated by the question-answering model and an actual reply, and the training set contains a reply text generated according to a current business document.

[0102] When the processor executes the program, the following steps are also implemented: before the target text is input into the target question and answer model and the reply text corresponding to the target text is generated, the above method also includes: obtaining a target document set, and segmenting the documents in the target document set to obtain segmented documents, wherein the target document set includes target documents that record business information; inputting the segmented documents into the initial question and answer model, generating replies corresponding to historical questions, and obtaining a question and answer text set, wherein the historical questions are questions asked by users regarding business information; calculating the offset value of the initial question and answer model based on the offset index and the question and answer text set, wherein the offset index includes at least the following indicators: question and answer matching, context recall, and question and answer relevance; when the offset value is greater than a preset threshold, generating a training set based on the segmented documents, and optimizing the initial question and answer model of the training set to obtain a target question and answer model.

[0103] When the processor executes the program, the following steps are also implemented: calculating the offset value of the initial question-answering model based on the offset index and the question-answering text set, including: calculating the question-answering matching degree based on the historical questions, the replies corresponding to the historical questions and the block-based documents, wherein the question-answering matching degree is used to characterize the matching degree between the replies corresponding to the historical questions and the block-based documents; calculating the context recall rate based on the argument information in the replies corresponding to the historical questions and the argument information in the context information, wherein the context information is the document information corresponding to the historical questions in the block-based documents, and the context recall rate is used to characterize the recall rate of the arguments of the context information in the replies corresponding to the historical questions; generating similar questions corresponding to the replies, and calculating the question-answering relevance based on the similarity between the historical questions and the similar questions; calculating the question-answering matching degree, the context recall rate and the question-answering relevance to obtain the offset value of the initial question-answering model.

[0104] When the processor executes the program, the following steps are also implemented: the question-answer matching degree is calculated based on the historical questions, the replies corresponding to the historical questions and the segmented documents, including: determining the answerable questions based on the context information, and determining the replies corresponding to the answerable questions; calculating the matching degree between the answerable questions and the replies corresponding to the answerable questions, and determining the number of questions and answers with a matching degree greater than a preset threshold, and obtaining the number of faithful answers; calculating the question-answer matching degree based on the number of faithful answers and the number of answerable questions.

[0105] When the processor executes the program, the following steps are also implemented: calculating the context recall rate based on the argument information in the replies corresponding to the historical questions and the argument information in the context information, including: extracting the arguments contained in the context information to obtain a first number of arguments; extracting the arguments contained in the replies corresponding to the historical questions to obtain a second number of arguments; calculating the context recall rate based on the first number of arguments and the second number of arguments.

[0106] When the processor executes the program, the following steps are also implemented: obtaining a target document set, and dividing the documents in the target document set into blocks to obtain block-based documents, including: obtaining a target document from a storage medium through a data interface every preset time to obtain a target document set; dividing the target documents in the target document set into blocks and converting them into embedded vectors to obtain N document blocks, wherein there are overlapping characters between the N document blocks, and N is a positive integer greater than 1; determining the block-based documents based on the N document blocks.

[0107] When the processor executes the program, the following steps are also implemented: a training set is generated based on the segmented documents, including: processing the historical questions and the segmented documents to generate M questions, where M is a positive integer greater than 1; processing the M questions and the segmented documents to generate replies corresponding to the M questions; and combining the M questions and the replies corresponding to each question into M question-answer texts to obtain a training set.

[0108] When the processor executes the program, the following steps are also implemented: historical questions and segmented documents are processed to generate M questions, including: keywords are extracted from historical questions to obtain a keyword set; for each keyword in the keyword set, the first similarity between the keyword and each document block in the segmented document is calculated; the first document block set corresponding to the keyword is determined based on the first similarity that meets a preset condition, wherein the preset condition means that the sum of the first similarities is greater than a preset threshold value, and the number of added first similarities is within a preset interval; M questions are generated based on the first document block set corresponding to each keyword in the keyword set.

[0109] When the processor executes the program, the following steps are also implemented: M questions and the divided documents are processed to generate replies corresponding to the M questions, including: for each of the M questions, calculating the second similarity between the question and each document block in the divided documents; determining a second document block set corresponding to the question based on the second similarities that meet a preset condition, wherein the preset condition means that the sum of the second similarities is greater than a preset threshold and the number of added second similarities is within a preset interval; and generating replies corresponding to the M questions based on the second document block set corresponding to each of the M questions.

[0110] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0111] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: receiving text information input by a target user to obtain a target text, wherein the target text contains question information; inputting the target text into a target question-and-answer model to generate a reply text corresponding to the target text, wherein the target question-and-answer model refers to a model obtained by optimizing the initial question-and-answer model using an offset indicator and a training set, the offset indicator is used to characterize the difference between the reply generated by the question-and-answer model and the actual reply, and the training set contains the reply text generated according to the current business document.

[0112] When executed on a data processing device, it is also suitable for executing a program that is initialized with the following method steps: before inputting the target text into the target question and answer model and generating a reply text corresponding to the target text, the above method also includes: obtaining a target document set, and segmenting the documents in the target document set to obtain segmented documents, wherein the target document set includes target documents that record business information; inputting the segmented documents into the initial question and answer model to generate replies corresponding to historical questions to obtain a question and answer text set, wherein the historical questions are questions asked by users regarding business information; calculating the offset value of the initial question and answer model based on the offset index and the question and answer text set, wherein the offset index includes at least the following indicators: question and answer matching degree, context recall rate, and question and answer relevance; when the offset value is greater than a preset threshold, generating a training set based on the segmented documents, and optimizing the initial question and answer model of the training set to obtain a target question and answer model.

[0113] When executed on a data processing device, it is also suitable for executing a program that initializes the following method steps: calculating the offset value of the initial question-answering model based on the offset index and the question-answering text set, including: calculating the question-answering matching degree based on the historical questions, the replies corresponding to the historical questions and the block-based documents, wherein the question-answering matching degree is used to characterize the matching degree between the replies corresponding to the historical questions and the block-based documents; calculating the context recall rate based on the argument information in the replies corresponding to the historical questions and the argument information in the context information, wherein the context information is the document information corresponding to the historical questions in the block-based documents, and the context recall rate is used to characterize the recall rate of the arguments of the context information in the replies corresponding to the historical questions; generating similar questions corresponding to the replies, and calculating the question-answering relevance based on the similarity between the historical questions and the similar questions; calculating the question-answering matching degree, the context recall rate and the question-answering relevance to obtain the offset value of the initial question-answering model.

[0114] When executed on a data processing device, it is also suitable for executing a program initialized with the following method steps: calculating the question-answer matching degree based on historical questions, replies to historical questions and segmented documents, including: determining answerable questions based on context information, and determining replies to answerable questions; calculating the matching degree between answerable questions and replies to answerable questions, and determining the number of questions and answers with a matching degree greater than a preset threshold, and obtaining the number of faithful answers; calculating the question-answer matching degree based on the number of faithful answers and the number of answerable questions.

[0115] When executed on a data processing device, it is also suitable for executing an initialized program having the following method steps: calculating the context recall rate based on the argument information in the replies corresponding to the historical questions and the argument information in the context information, including: extracting the arguments contained in the context information to obtain a first number of arguments; extracting the arguments contained in the replies corresponding to the historical questions to obtain a second number of arguments; calculating the context recall rate based on the first number of arguments and the second number of arguments.

[0116] When executed on a data processing device, it is also suitable for executing a program that is initialized with the following method steps: obtaining a target document set, and dividing the documents in the target document set into blocks to obtain block-divided documents, including: obtaining a target document from a storage medium through a data interface every preset time to obtain a target document set; dividing the target document in the target document set into blocks, and converting them into embedded vectors to obtain N document blocks, wherein there are overlapping characters between the N document blocks, and N is a positive integer greater than 1; determining the block-divided documents based on the N document blocks.

[0117] When executed on a data processing device, it is also suitable for executing an initialization program having the following method steps: generating a training set based on the segmented documents, including: processing historical questions and segmented documents to generate M questions, where M is a positive integer greater than 1; processing the M questions and segmented documents to generate replies corresponding to the M questions; combining the M questions and the replies corresponding to each question into M question-answering texts to obtain a training set.

[0118] When executed on a data processing device, it is also suitable for executing an initialization program having the following method steps: processing historical questions and segmented documents to generate M questions, including: extracting keywords from historical questions to obtain a keyword set; for each keyword in the keyword set, calculating the first similarity between the keyword and each document block in the segmented document; determining the first document block set corresponding to the keyword based on the first similarity that meets a preset condition, wherein the preset condition means that the sum of the first similarities is greater than a preset threshold, and the number of added first similarities is within a preset interval; generating M questions based on the first document block set corresponding to each keyword in the keyword set.

[0119] When executed on a data processing device, it is also suitable for executing an initialized program having the following method steps: processing M questions and block-type documents to generate replies corresponding to the M questions, including: for each of the M questions, calculating the second similarity between the question and each document block in the block-type documents; determining a second document block set corresponding to the question based on the second similarity that meets a preset condition, wherein the preset condition means that the sum of the second similarities is greater than a preset threshold and the number of added second similarities is within a preset interval; generating replies corresponding to the M questions based on the second document block set corresponding to each of the M questions.

[0120] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0121] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0122] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0123] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0124] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0125] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0126] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0127] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0128] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0129] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for processing user questions and answers, characterized in that: include: Receiving text information input by a target user to obtain a target text, wherein the target text contains question information; The target text is input into a target question-answering model to generate a reply text corresponding to the target text, wherein the target question-answering model refers to a model obtained by optimizing an initial question-answering model using an offset index and a training set, the offset index is used to characterize the difference between a reply generated by the question-answering model and an actual reply, and the training set contains a reply text generated according to the current business document.

2. The method according to claim 1, characterized in that Before inputting the target text into the target question-answering model to generate a reply text corresponding to the target text, the method further includes: Acquire a target document set, and divide the documents in the target document set into blocks to obtain divided documents, wherein the target document set includes target documents recording business information; Inputting the segmented documents into the initial question-answering model, generating replies corresponding to historical questions, and obtaining a question-answering text set, wherein the historical questions are questions raised by users regarding the business information; Calculating the offset value of the initial question-answer model according to the offset index and the question-answer text set, wherein the offset index includes at least the following indicators: question-answer matching degree, context recall rate, and question-answer relevance; When the offset value is greater than a preset threshold, the training set is generated based on the segmented documents, and the initial question-answering model of the training set is optimized to obtain the target question-answering model.

3. The method according to claim 2, characterized in that Calculating the offset value of the initial question-answer model according to the offset index and the question-answer text set includes: Calculating the question-answer matching degree based on the historical question, the reply corresponding to the historical question and the segmented document, wherein the question-answer matching degree is used to represent the matching degree between the reply corresponding to the historical question and the segmented document; Calculating the context recall rate based on the argument information in the reply corresponding to the historical question and the argument information in the context information, wherein the context information is the document information corresponding to the historical question in the segmented documents, and the context recall rate is used to characterize the recall rate of the argument of the context information in the reply corresponding to the historical question; Generate a similar question corresponding to the reply, and calculate the question-answer relevance based on the similarity between the historical question and the similar question; The question-answer matching degree, the context recall rate, and the question-answer relevance are calculated to obtain an offset value of the initial question-answer model.

4. The method according to claim 3, characterized in that Calculating the question-answer matching degree according to the historical question, the reply corresponding to the historical question and the segmented document includes: Determining answerable questions based on the context information, and determining replies corresponding to the answerable questions; Calculating the degree of match between the answerable questions and the replies corresponding to the answerable questions, and determining the number of questions and answers with the degree of match greater than a preset threshold, to obtain the number of faithful answers; The question-answer matching degree is calculated based on the number of faithful answers and the number of answerable questions.

5. The method according to claim 3, characterized in that: Calculating the context recall rate according to the argument information in the reply corresponding to the historical question and the argument information in the context information includes: Extracting the arguments contained in the context information to obtain a first number of arguments; Extracting the arguments contained in the responses corresponding to the historical questions to obtain the second number of arguments; The context recall rate is calculated based on the first number of arguments and the second number of arguments.

6. The method according to claim 2, characterized in that Obtaining a target document set, and dividing the documents in the target document set into blocks to obtain divided documents, including: Obtaining the target document from the storage medium through the data interface every preset time to obtain the target document set; The target documents in the target document set are processed into blocks and converted into embedded vectors to obtain N document blocks, wherein there are overlapping characters between the N document blocks, and N is a positive integer greater than 1; The divided documents are determined according to the N document blocks.

7. The method according to claim 2, characterized in that Generating the training set according to the divided documents includes: Processing the historical questions and the divided documents to generate M questions, where M is a positive integer greater than 1; Processing the M questions and the divided documents to generate replies corresponding to the M questions; The M questions and the answers corresponding to each question are combined into M question-answer texts to obtain the training set.

8. The method according to claim 7, characterized in that The historical questions and the divided documents are processed to generate M questions, including: Extract keywords from the historical issues to obtain a keyword set; For each keyword in the keyword set, calculating a first similarity between the keyword and each document block in the divided document; Determining a first document block set corresponding to the keyword according to the first similarities that meet a preset condition, wherein the preset condition refers to that the sum of the first similarities is greater than a preset threshold and the sum of the first similarities is within a preset interval; The M questions are generated according to the first document block set corresponding to each keyword in the keyword set.

9. The method according to claim 7, characterized in that: Processing the M questions and the divided documents to generate replies corresponding to the M questions includes: For each of the M questions, calculating a second similarity between the question and each document block in the divided documents; Determining a second document block set corresponding to the question according to the second similarities that meet a preset condition, wherein the preset condition refers to that the sum of the second similarities is greater than a preset threshold and the number of the added second similarities is within a preset interval; Generate replies corresponding to the M questions based on the second document block set corresponding to each of the M questions.

10. A user question and answer processing device, characterized in that: include: A receiving unit, configured to receive text information input by a target user and obtain a target text, wherein the target text includes question information; The first generating unit is used to input the target text into the target question-answering model to generate a reply text corresponding to the target text, wherein the target question-answering model refers to a model obtained by optimizing the initial question-answering model using an offset indicator and a training set, the offset indicator is used to characterize the difference between the reply generated by the question-answering model and the actual reply, and the training set contains the reply text generated according to the current business document.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes stored computer instructions, wherein when the computer instructions are executed by a processor, the user question and answer processing method described in any one of claims 1 to 9 is implemented.

12. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the user question and answer processing method described in any one of claims 1 to 9.