Question and answer knowledge base updating method and device, equipment and medium

Abstract

The invention discloses a question and answer knowledge base updating method and device, equipment and a medium, and the method comprises the steps: inputting a platform help document into a large language model when it is monitored that the platform help document is updated, and controlling the large language model to generate a plurality of candidate question and answer entries; calculating the similarity between every two standard questions in the plurality of candidate question and answer items and standard questions in reference question and answer items stored in a question and answer knowledge base, and when the similarity meets a preset newly-added condition, storing the corresponding candidate question and answer items in the question and answer knowledge base; when the similarity does not meet a preset newly-added condition, calling a large language model to judge whether the full-text semantics of the corresponding candidate question entries and the reference question and answer entries are the same, and if not, storing the corresponding candidate question entries into a question and answer knowledge base; and if yes, marking the corresponding candidate question item as a to-be-audited state. According to the method, the updating efficiency and accuracy of the question and answer knowledge base are remarkably improved, and the user experience of an intelligent customer service system is enhanced.

Description

Technical Field

[0001] The present application relates to the field of intelligent customer service technology, and in particular to a method for updating a question-and-answer knowledge base and its apparatus, equipment, and medium. Background Art

[0002] With the rapid development of e-commerce and online service platforms, platform operators need to provide comprehensive and timely operational guidance and support to merchant users. Currently, platform help documents detail the operational procedures and business rules for each platform function. Key information is extracted from the platform help documents to construct a question-and-answer knowledge base to form standardized question-and-answer pairs. Intelligent customer service systems then use this knowledge base to provide merchant users with instant and accurate question-and-answer services.

[0003] However, when the iteration of platform functions leads to changes in the content of help documents, the traditional update method of the question-and-answer knowledge base mainly relies on manual methods. Specifically, business personnel need to regularly check the updates of the platform help documents, manually extract new questions and answers, and add them to the question-and-answer knowledge base. Traditional methods have many disadvantages. First, the manual processing speed is limited, resulting in the update of the question-and-answer knowledge base seriously lagging behind the actual changes in the help documents, and failing to meet the needs of merchant users for real-time guidance. Secondly, it is difficult for manual editors to fully predict the various ways of expressing questions that users may use, resulting in insufficient coverage of natural language questions in the question-and-answer knowledge base. In addition, due to the uneven professional level and experience of business personnel, the questions and answers they write may be biased, affecting the accuracy of the answers and user experience of the intelligent customer service system. Summary of the Invention

[0004] The primary purpose of this application is to solve at least one of the above problems and provide a question-and-answer knowledge base updating method and its device, equipment, and medium.

[0005] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0006] A method for updating a question-and-answer knowledge base provided to meet one of the purposes of this application includes the following steps:

[0007] When it is detected that the platform help document is updated, the platform help document is input into the large language model, and the large language model is controlled to generate multiple candidate question and answer entries, wherein the candidate question and answer entries include a standard question, multiple similar questions, and corresponding answer texts;

[0008] Calculating the similarity between the standard questions in the plurality of candidate question and answer entries and the standard questions in the benchmark question and answer entries stored in the question and answer knowledge base, and storing the corresponding candidate question and answer entry in the question and answer knowledge base when the similarity meets a preset additional condition;

[0009] When the similarity does not satisfy the preset addition condition, a large language model is called to determine whether the full-text semantics of the corresponding candidate question item and the benchmark question and answer item are the same, if not the same, the corresponding candidate question item is stored in the question and answer knowledge base;

[0010] If the same, the corresponding candidate question item is marked as a to-be-audited state, and whether to cover the corresponding benchmark question and answer item and store it in the question and answer knowledge base is determined according to the corresponding audit result data.

[0011] A question and answer knowledge base updating device is provided for one of the purposes of the present application, comprising:

[0012] A question and answer generation module is configured to input the platform help document into a large language model when it is monitored that the platform help document is updated, control the large language model to generate a plurality of candidate question and answer items, and the candidate question and answer items include a standard question, a plurality of similar questions and corresponding answer texts;

[0013] A similarity calculation module is configured to calculate the similarity between the standard questions in the plurality of candidate question and answer items and the standard questions in the benchmark question and answer items stored in the question and answer knowledge base, and when the similarity satisfies a preset addition condition, the corresponding candidate question and answer item is stored in the question and answer knowledge base;

[0014] A semantic judgment module is configured to call a large language model to determine whether the full-text semantics of the corresponding candidate question item and the benchmark question and answer item are the same when the similarity does not satisfy the preset addition condition, and if not the same, the corresponding candidate question item is stored in the question and answer knowledge base;

[0015] An audit processing module is configured to mark the corresponding candidate question item as a to-be-audited state if the same, and determine whether to cover the corresponding benchmark question and answer item and store it in the question and answer knowledge base according to the corresponding audit result data.

[0016] A computer device is provided for one of the purposes of the present application, comprising a central processing unit and a memory, and the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the question and answer knowledge base updating method described in the present application.

[0017] A computer readable storage medium is provided for another purpose of the present application, which stores a computer program implemented according to the question and answer knowledge base updating method in the form of computer readable instructions, and when the computer program is called and run by a computer, the steps included in the method are executed.

[0018] Compared with the prior art, the advantages of the present application are as follows:

[0019] First, this application generates candidate question and answer entries by introducing a large language model, which can quickly respond to updates to the platform help documents. When it is detected that the platform help documents are updated, the updated documents are immediately input into the large language model, and the large language model is controlled to generate multiple candidate question and answer entries containing standard questions, multiple similar questions and corresponding answer texts, which greatly improves the efficiency of updating the question and answer knowledge base, and can provide merchant users with the latest operational guidance support in a timely manner to meet their needs for real-time guidance.

[0020] Secondly, when judging whether candidate question and answer entries should be included in the question and answer knowledge base, this application adopts a dual mechanism of calculating similarity and calling a large language model for semantic judgment. This dual mechanism can accurately identify questions with the same semantics but different expressions, and effectively avoids the problems of missing important question variants and redundant entries caused by simple string matching or keyword comparison, thereby improving the coverage of natural language questions in the question and answer knowledge base, enhancing the accuracy and practicality of the question and answer knowledge base, and thus improving the service effect of the intelligent customer service system.

[0021] In addition, this application also sets up a pending review status mechanism, which ensures the quality and accuracy of the Q&A knowledge base content through manual review, so that the update of the Q&A knowledge base can not only fully utilize the powerful capabilities of the large language model, but also ensure the reliability and authority of the content through manual review at key links, further optimizing the update process of the Q&A knowledge base and improving the overall performance and user experience of the intelligent customer service system. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0023] Figure 1 The network architecture of the e-commerce platform exemplified in this application;

[0024] Figure 2 This is a flowchart of a typical embodiment of the question-answer knowledge base updating method of the present application;

[0025] Figure 3 A schematic diagram of the process of generating candidate question and answer entries in an embodiment of the present application;

[0026] Figure 4 This is a flowchart of the corresponding processing when the preset new conditions are met in the embodiment of the present application;

[0027] Figure 5 This is a flowchart of the corresponding processing when the preset new conditions are not met in the embodiment of the present application;

[0028] Figure 6A schematic diagram of a process for determining whether answer texts are similar to delete duplicate answer texts in an embodiment of the present application;

[0029] Figure 7 A flowchart of adding unmatched question text to corresponding similar questions in an embodiment of the present application;

[0030] Figure 8 A schematic diagram of the process of optimizing the answer text after user feedback in an embodiment of the present application;

[0031] Figure 9 This is a functional block diagram of the question-and-answer knowledge base updating device of this application;

[0032] Figure 10 This is a schematic diagram of the structure of a computer device used in this application. DETAILED DESCRIPTION

[0033] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

[0034] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0035] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0036] like Figure 1In the network architecture shown, the e-commerce platform 82 is deployed on the Internet to provide corresponding services to its users. Similarly, the devices 80 of the merchant users of the e-commerce platform 82 and the devices 81 of the consumer users are also connected to the Internet to use the services provided by the e-commerce platform.

[0037] The exemplary e-commerce platform 82 provides supply and demand matching of products and / or services to the general public with the help of Internet infrastructure. In the e-commerce platform 82, products and / or services are provided as commodity information. To simplify the description, the concepts of commodity, product, etc. are used in this application to refer to the products and / or services in the e-commerce platform 82, which may specifically be physical products, digital products, tickets, service subscriptions, other offline services, etc.

[0038] In reality, various entities can access the e-commerce platform 82 as users, use the various online services provided by the e-commerce platform 82, and achieve the purpose of participating in the business activities achieved by the e-commerce platform 82. These entities can be natural persons, legal persons, or social organizations. Corresponding to the two types of entities in business activities, merchants and consumers, the e-commerce platform 82 has two corresponding types of users: merchant users and consumer users. In business activities, all entities in the product distribution chain, including manufacturers, sellers, retailers, logistics providers, etc., can use online services on the e-commerce platform 82 as merchant users, while consumers in business activities, including real or potential consumers, can use online services on the e-commerce platform 82 as their corresponding consumer users. In actual business activities, the same entity can act as both a merchant user and a consumer user, and this should be understood flexibly.

[0039] The infrastructure used to deploy the e-commerce platform 82 primarily includes a backend architecture and frontend devices. The backend architecture runs various online services through a service cluster, including platform-facing middleware or frontend services, consumer-facing services, merchant-facing services, etc., to enrich and improve its service functions. The frontend devices primarily encompass the terminal devices used by users to access the e-commerce platform 82 as clients, including but not limited to various mobile terminals, personal computers, point-of-sale devices, etc. For example, a merchant user can use their terminal device 80 to enter product information for their online store, or generate their product information using an interface open to the e-commerce platform. A consumer user can use their terminal device 81 to access the webpage of the online store implemented by the e-commerce platform 82, trigger the shopping process through the shopping button provided on the webpage, and invoke various online services provided by the e-commerce platform 82 during the shopping process, thereby completing the purpose of placing a shopping order.

[0040] In some embodiments, the e-commerce platform 82 may be implemented by a processing facility including a processor and a memory, the processing facility storing a set of instructions that, when executed, cause the e-commerce platform 82 to perform the e-commerce and support functions described herein. The processing facility may be part of a server, client, network infrastructure, mobile computing platform, cloud computing platform, fixed computing platform, or other computing platform, and may provide electronic components of the e-commerce platform 82, merchant devices, payment gateways, application developers, marketing channels, transportation providers, customer devices, point-of-sale devices, and the like.

[0041] The e-commerce platform 82 can be implemented as an online service such as cloud computing, software as a service (SaaS), infrastructure as a service (IaaS), platform as a service (PaaS), desktop as a service (DaaS), hosted software as a service, mobile backend as a service (MBaaS), information technology management as a service (ITMaaS), etc. In some embodiments, the various functional components of the e-commerce platform 82 can be implemented to be suitable for operation on various platforms and operating systems. For example, for an online store, its administrator users can enjoy the same or similar functions regardless of various embodiments such as iOS, Android, HomonyOS, or web pages.

[0042] The e-commerce platform 82 can implement its corresponding independent website for each merchant to run its corresponding online store, and provide merchants with corresponding business management engine instances for merchants to establish, maintain, and run one or more online stores in one or more independent websites. The business management engine instance can be used for content management, task automation, and data management of one or more online stores, and can configure various specific business processes of the online store through interfaces or built-in components to support the implementation of business activities. The independent website is the infrastructure of the e-commerce platform 82 with cross-border service functions. Merchants can maintain their online stores more centrally and independently based on the independent website. The independent website usually has a domain name and storage space dedicated to the merchant, and different independent websites are relatively independent. The e-commerce platform 82 can provide standardized or personalized technical support for a large number of independent websites, so that merchant users can customize their own business management engine instance and use this business management engine instance to maintain one or more online stores they own.

[0043] The online store can implement backend configuration and maintenance by having the merchant user log in to its business management engine instance as an administrator. With the support of various online services provided by the infrastructure of the e-commerce platform 82, the merchant user can configure various functions in its online store as an administrator, view various data, etc. For example, the merchant user can manage various aspects of its online store, such as viewing the latest activities of the online store, updating the online store product catalog, managing orders, recent visit activities, total order activities, etc.; the merchant user can also view more detailed information about the business and visitors to the merchant's online store by obtaining reports or metrics, such as showing a sales summary of the merchant's overall business, specific sales and participation data of active sales marketing channels, etc.

[0044] The e-commerce platform 82 may provide communication facilities and associated merchant interfaces for providing electronic communications and marketing, such as utilizing electronic message aggregation facilities to collect and analyze communication interactions between merchants, consumers, merchant devices, customer devices, point-of-sale devices, etc., aggregating and analyzing communications, such as for increasing the potential for providing product sales, etc. For example, a consumer may have questions about a product, which may generate a conversation between the consumer and the merchant (or an automated processor-based agent on behalf of the merchant), wherein the communication facility is responsible for the interaction and provides the merchant with analysis on how to increase the probability of sales.

[0045] In some embodiments, applications suitable for installation on terminal devices can be provided to serve the access needs of different users, so that various users can access the e-commerce platform 82 by running applications on the terminal devices, such as the merchant backend module of the online store in the e-commerce platform 82. In the process of implementing business activities through these functions, the e-commerce platform 82 can implement various functions related to supporting business activities as middleware or online services and open corresponding interfaces, and then implant toolkits corresponding to the interface access functions into the application to realize functional expansion and task implementation. The business management engine can include a series of basic functions and expose these functions to online services and / or application calls through APIs. The online services and applications use the corresponding functions by remotely calling the corresponding APIs.

[0046] Supported by the various components of the business management engine instance, the e-commerce platform 82 provides online shopping functionality, enabling merchants to connect with customers in a flexible and transparent manner. Consumers can select items online, create an order, provide a delivery address in the order, and complete payment confirmation for the order. Merchants can then review and complete or cancel the order.

[0047] A question-and-answer knowledge base updating method of the present application can be programmed as a computer program product and deployed in a client or server for execution. For example, in the exemplary application scenario of the present application, it can be deployed and implemented in the server of an e-commerce customer service platform. The method can be executed by accessing an interface opened after the computer program product is run and performing human-computer interaction with the process of the computer program product through a graphical user interface.

[0048] See also Figure 2 The question-answer knowledge base updating method of the present application, in its typical embodiment, includes the following steps:

[0049] Step S1100: When the platform help document is detected to be updated, the platform help document is input into the large language model, and the large language model is controlled to generate multiple candidate question and answer entries, wherein the candidate question and answer entries include a standard question, multiple similar questions, and corresponding answer texts:

[0050] Platform help documentation is crucial for the intelligent customer service system, providing operational guidance and business support to merchants. It is continuously updated with platform functionality updates, business process optimizations, and policy adjustments. Whenever the platform help documentation is updated, the updated document content must be promptly retrieved and fed into the large language model to generate new candidate Q&A entries, ensuring that the intelligent customer service system can provide merchants with timely and accurate help information.

[0051] In one embodiment, a version monitoring mechanism is set up, which can be based on file system timestamp monitoring, version control system update notifications, or e-commerce platform document management system change logs. When a change in the platform help document version number is detected, the monitoring mechanism triggers an event and executes an operation to obtain the updated platform help document.

[0052] After obtaining updated platform help documents, further identify version change annotations within the document. These annotations are added by document editors when updating documents to clearly indicate which sections have been added, modified, or deleted. These annotations can be specific tags, comments, or metadata from the document management system. By analyzing these annotations, changes can be quickly located within the document.

[0053] In some embodiments, before inputting the platform help document into the large language model, the document content is preprocessed, including segmenting the document into sections and sentences, removing irrelevant formatting and markup, and extracting the core content. For example, the document can be divided into chapters, with each chapter as an independent text block; within each text block, the sentence structure is then segmented to form independent sentences. At the same time, formatting content such as headers, footers, and tables of contents, as well as irrelevant markup such as extra spaces and line breaks, are removed to ensure that the text input into the large language model is clear and accurate, helping to improve model understanding and generation.

[0054] The preprocessed platform help document content is then fed into a large language model. The large language model, built based on deep learning technology and possessing powerful language understanding and generation capabilities, generates multiple candidate Q&A entries based on the input platform help document content. In some embodiments, strategies are employed to guide the large language model's generation process to improve the quality and diversity of the generated candidate Q&A entries. For example, specific prompts or instructions can be provided to the large language model to indicate the focus and direction of the Q&A entries it generates. For example, when feeding the platform help document, a prompt can be added to instruct the large language model to focus on the introduction of new features and generate relevant Q&A entries. The generated candidate Q&A entries include a standard question, multiple similar questions, and corresponding answer text. The standard question is the question that best represents the core content of the Q&A entry after being refined and summarized by the model. Similar questions are questions that are semantically similar to the standard question but expressed differently. These similar questions can help the intelligent customer service system better understand the merchant's different questioning styles and improve the accuracy of question matching. The answer text is a detailed answer to the standard question and similar questions, providing accurate and comprehensive answers based on the information in the document.

[0055] Step S1200: Calculate the similarity between the standard questions in the multiple candidate Q&A entries and the standard questions in the benchmark Q&A entries stored in the Q&A knowledge base. When the similarity meets a preset additional condition, store the corresponding candidate Q&A entry in the Q&A knowledge base.

[0056] After the large language model generates multiple candidate Q&A entries based on the updated platform help documentation, these candidates need to be evaluated to determine whether they can be included in the Q&A knowledge base. Specifically, by calculating the similarity between the standard questions in the candidate Q&A entries and the standard questions in the benchmark Q&A entries stored in the Q&A knowledge base, it is determined whether the candidate Q&A entries provide new and valuable information or simply repeat content already in the knowledge base.

[0057] In one embodiment, it is first necessary to extract the standard questions of all benchmark question and answer items from the question and answer knowledge base to form a benchmark question set. Then, for each candidate question and answer item, the similarity between its standard question and each standard question in the benchmark question set is calculated. In this embodiment, a comprehensive judgment is made in combination with cosine similarity and Jaccard coefficient, that is, the new condition is preset to be when the cosine similarity is less than a preset first threshold and the Jaccard coefficient is less than a preset second threshold, confirming that the preset new condition is met. Among them, the preset first threshold and the second threshold are set according to actual needs and experience, and are used to measure the minimum acceptable level of cosine similarity and Jaccard coefficient. By combining cosine similarity and Jaccard similarity, the similarity between two questions can be more comprehensively evaluated. When the similarity meets the preset new condition, the corresponding candidate question and answer item is stored in the question and answer knowledge base.

[0058] Through the similarity calculation and judgment in this step, truly valuable and non-repetitive candidate question and answer entries can be effectively screened out and incorporated into the question and answer knowledge base, thereby realizing dynamic updating and optimization of the knowledge base and improving the performance and user experience of the intelligent customer service system.

[0059] Step S1300: When the similarity does not meet the preset additional condition, the large language model is called to determine whether the full text semantics of the corresponding candidate question item and the benchmark question and answer item are the same. If not, the corresponding candidate question item is stored in the question and answer knowledge base.

[0060] When the similarity between the standard question in a candidate Q&A entry and the standard question in a benchmark Q&A entry already stored in the Q&A knowledge base does not meet the preset additional conditions, a preliminary similarity calculation will be performed to determine that the candidate Q&A entry has a certain degree of similarity with the existing Q&A entries in the knowledge base. However, relying solely on this preliminary similarity calculation cannot fully and accurately determine whether the two are truly identical at the semantic level. Due to the complexity of natural language, different expressions may actually convey the same or very similar semantic information. Therefore, it is necessary to further use a large language model to determine whether the full text semantics of the corresponding candidate Q&A entry and the benchmark Q&A entry are identical.

[0061] In one embodiment, candidate Q&A entries that do not meet the pre-set additional conditions and their corresponding benchmark Q&A entries are fed into a large language model to determine whether the candidate Q&A entries and the benchmark Q&A entries are semantically identical in their full text. If the large language model determines that the candidate Q&A entries and the benchmark Q&A entries are semantically different in their full text, this indicates that the candidate Q&A entry is valuable and should be stored in the Q&A knowledge base, thereby enriching the knowledge base and improving the intelligent customer service system's coverage of merchant questions and the diversity of its responses.

[0062] Step S1400: If the two are the same, the corresponding candidate question entry is marked as pending review, and based on the corresponding review result data, it is determined whether to overwrite the corresponding benchmark question and answer entry and store it in the question and answer knowledge base:

[0063] When the large language model determines that a candidate Q&A entry has semantically identical semantics to a benchmark Q&A entry, the candidate Q&A entry is marked as pending review. Reviewers must carefully compare the candidate Q&A entry with the benchmark Q&A entry and evaluate it from multiple perspectives. In one embodiment, the candidate Q&A entry's answer text is examined to see whether it provides more detailed, accurate, or information that better meets current business needs. For example, if the platform has added advanced usage tips for a new feature, the candidate Q&A entry's answer text may include detailed explanations of these new tips, while the benchmark Q&A entry's answer text remains at the basic operational level. In this case, even with semantically identical answers, the candidate Q&A entry still has some value and may be considered for inclusion in the benchmark Q&A. Secondly, the candidate Q&A entry's standard questions and similar questions are reviewed to see whether they better align with user questioning habits. For example, if the candidate Q&A entry's standard questions use more accessible or more user-friendly vocabulary, better matching likely user questions, then inclusion in the benchmark Q&A entry may also be considered. Furthermore, consideration should be given to whether the candidate Q&A entry's answer text is more readable and logical, making it easier for users to understand and operate. If a candidate Q&A entry performs better in these areas, even if its semantics are identical to the baseline Q&A entry, it should be included in the Q&A knowledge base to improve the quality of answers and user experience of the intelligent customer service system. After completing the above evaluation, if the reviewer passes the review, the candidate Q&A entry will be stored in the Q&A knowledge base over the corresponding baseline Q&A entry to update and optimize the Q&A knowledge base. If the reviewer fails, the baseline Q&A entry will be retained, and the candidate Q&A entry will not be included in the database to ensure the accuracy and practicality of the Q&A knowledge base content.

[0064] Through the above description of the typical embodiments of the present application, it can be understood that the present application has many positive advantages, including but not limited to the following aspects:

[0065] First, this application generates candidate question and answer entries by introducing a large language model, which can quickly respond to updates to the platform help documents. When it is detected that the platform help documents are updated, the updated documents are immediately input into the large language model, and the large language model is controlled to generate multiple candidate question and answer entries containing standard questions, multiple similar questions and corresponding answer texts, which greatly improves the efficiency of updating the question and answer knowledge base, and can provide merchant users with the latest operational guidance support in a timely manner to meet their needs for real-time guidance.

[0066] Secondly, when judging whether candidate question and answer entries should be included in the question and answer knowledge base, this application adopts a dual mechanism of calculating similarity and calling a large language model for semantic judgment. This dual mechanism can accurately identify questions with the same semantics but different expressions, and effectively avoids the problems of missing important question variants and redundant entries caused by simple string matching or keyword comparison, thereby improving the coverage of natural language questions in the question and answer knowledge base, enhancing the accuracy and practicality of the question and answer knowledge base, and thus improving the service effect of the intelligent customer service system.

[0067] In addition, this application also sets up a pending review status mechanism, which ensures the quality and accuracy of the Q&A knowledge base content through manual review, so that the update of the Q&A knowledge base can not only fully utilize the powerful capabilities of the large language model, but also ensure the reliability and authority of the content through manual review at key links, further optimizing the update process of the Q&A knowledge base and improving the overall performance and user experience of the intelligent customer service system.

[0068] For further examples, please refer to Figure 3 When it is detected that the platform help document is updated, the platform help document is input into the large language model, and the large language model is controlled to generate multiple candidate question and answer entries, wherein the candidate question and answer entries include a standard question, multiple similar questions, and corresponding answer texts, including the following steps:

[0069] Step S1110: Obtain the platform help document with the version change description annotation, which is used to indicate the incremental modification content of the current update:

[0070] Monitor in real time whether the version number of the platform help document has changed. When a change occurs, obtain the platform help document with a version change description annotation. The version change description annotation is added by the document editor when updating the document, and is used to clearly indicate the incremental changes of the update, including new, modified or deleted parts. In one embodiment, the version number change is monitored by a document management system, and the version information of the platform help document is regularly checked and compared with the previous recorded version number. When a change in the version number is detected, the latest platform help document is automatically obtained.

[0071] Step S1120: Embed the obtained platform help document into a preset question and answer generation prompt template to obtain a question and answer generation prompt instruction. The question and answer generation prompt template includes natural language instructions for guiding the large language model to generate multiple candidate question and answer entries based on the incremental modification content of the platform help document:

[0072] The question and answer generation prompt template is preset by those skilled in the art as needed. It contains natural language instructions for guiding the large language model to generate multiple candidate question and answer entries based on the incremental modification content of the platform help document, that is, guiding the large language model to generate question and answer entries based on the incremental modification content of the platform help document. The obtained platform help document is embedded in the preset question and answer generation prompt template. Specifically, the incremental modification content in the document, that is, the content indicated by the version change description annotation, is replaced with the corresponding position in the template. A complete question and answer generation prompt instruction is obtained.

[0073] Step S1130: Input the question and answer generation prompt instruction into the large language model, and control the large language model to generate multiple candidate question and answer entries based on the incrementally modified content in the platform help document:

[0074] In one embodiment, the question-and-answer generation prompt generated in the previous step is preprocessed to ensure that its format and content meet the input requirements of the large language model, including removing irrelevant special characters, unifying text encoding, adjusting text length, and other operations. The preprocessed question-and-answer generation prompt is then input into the large language model, which begins generating candidate question-and-answer entries based on the incrementally modified content in the platform help document. During this process, the large language model uses its internal algorithms and rules to analyze and understand the content in the question-and-answer generation prompt. The large language model attempts to identify key information in the document, such as the name of the new feature, operating steps, precautions, etc., and converts it into the form of questions and answers.

[0075] When generating candidate Q&A entries, the large language model considers multiple factors to ensure high quality and relevance. For example, the large language model generates questions based on keywords, semantic structure, and contextual information from the platform's help documentation, and generates answers based on detailed information from the platform's help documentation. Furthermore, the large language model utilizes its internal algorithms and rules to optimize the generated Q&A entries, making them more consistent with natural language expression.

[0076] Through this step, the large language model can be effectively used to generate high-quality candidate question and answer entries, providing a basis for updating the question and answer knowledge base. This not only improves the updating efficiency of the question and answer knowledge base, but also ensures that the generated question and answer entries have high accuracy and relevance, and can better meet the needs of merchant users.

[0077] In this embodiment, by monitoring the version number changes of the platform help document in real time, documents with version change description annotations are obtained to ensure that only incremental modified content is processed, avoiding repeated processing of the entire document and improving efficiency. The obtained incremental modified content is then embedded in a preset question and answer generation prompt template to generate targeted question and answer generation prompt instructions, and then candidate question and answer entries containing standard questions, multiple similar questions and answer texts are generated through the large language model. This step not only utilizes the powerful natural language processing capabilities of the large language model, but also guides the model through the template to focus on incremental content, ensuring that the generated question and answer entries are closely related to the document updates and are of high quality and relevance.

[0078] For further examples, please refer to Figure 4 , calculating the similarity between the standard questions in the plurality of candidate question and answer entries and the standard questions in the benchmark question and answer entries stored in the question and answer knowledge base, and when the similarity meets the preset new condition, storing the corresponding candidate question and answer entry in the question and answer knowledge base, including the following steps:

[0079] Step S1210: For each candidate Q&A item, calculate the cosine similarity between the standard question of the candidate Q&A item and the standard question of each benchmark Q&A item:

[0080] Calculate the cosine similarity between the standard question of each candidate Q&A entry and the standard question of the benchmark Q&A entry stored in the Q&A knowledge base. Cosine similarity is a commonly used text similarity metric that measures the cosine of the angle between two text vectors, reflecting the semantic similarity between the two standard questions. Based on the cosine similarity, the similarity between the standard question of the candidate Q&A entry and the standard question of the benchmark Q&A entry can be determined. This will be used in subsequent implementations, along with the Jacquard coefficient, to determine whether the candidate Q&A entry meets the preset additional conditions.

[0081] Step S1220: Calculate the Jacquard coefficient between the standard question of the candidate Q&A item and the standard question of each benchmark Q&A item:

[0082] The Jaccard Index is a statistical method for measuring the similarity between two sets. It measures the corresponding similarity by calculating the ratio of the size of the intersection of the two sets to the size of the union. In text similarity calculations, the Jaccard Index is often used to compare the degree of lexical overlap between two texts, thereby evaluating the semantic similarity between two title questions. Specifically, the two standard questions first need to be converted into lexical sets. This is achieved by performing word segmentation on the text, that is, breaking the standard questions into individual words or tokens to obtain two lexical sets. The intersection and union of the two lexical sets are then calculated. The intersection refers to the words shared by the two sets, while the union refers to all words that appear in the two sets. Finally, the ratio of the intersection to the union is recorded as the corresponding Jaccard Index. The calculated Jaccard Index value ranges from 0 to 1. The closer the value is to 1, the higher the similarity between the two sets, that is, the closer the semantics of the two standard questions are. Based on the Jaccard Index, the similarity between the standard questions of the candidate Q&A items and the standard questions of the benchmark Q&A items is further evaluated.

[0083] The resulting Jaccard coefficient is used in conjunction with cosine similarity in subsequent implementations to more comprehensively evaluate the similarity between candidate Q&A entries and benchmark Q&A entries. For details, please refer to the subsequent implementations and will not be elaborated here.

[0084] Step S1230: When the cosine similarity is less than a preset first threshold and the Jacquard coefficient is less than a preset second threshold, it is determined that the preset new condition is met, and the corresponding candidate question and answer entry is stored in the question and answer knowledge base:

[0085] This step determines whether the candidate question and answer entries meet the preset new addition conditions based on the cosine similarity and the Jacquard coefficient. Specifically, the setting of the first threshold and the second threshold needs to comprehensively consider factors such as the update frequency of the knowledge base, the complexity of the document content, and the merchant's requirements for the accuracy of the questions and answers. For example, if the knowledge base needs to be updated frequently to reflect the latest changes in the platform help documents, the threshold should be appropriately lowered at this time so that new information can be incorporated into the knowledge base more quickly. On the contrary, if the knowledge base is updated relatively rarely, or the accuracy requirements for questions and answers are very high, the threshold should be raised at this time to ensure that only highly relevant question and answer entries are added.

[0086] Specifically, if the cosine similarity of a candidate Q&A entry is less than a preset first threshold, and the Jacquard coefficient is also less than a preset second threshold, the candidate Q&A entry is determined to meet the preset new condition. This indicates that the candidate Q&A entry is semantically sufficiently different from the existing benchmark Q&A entries in the knowledge base to be considered a new Q&A entry and thus stored in the Q&A knowledge base. This step effectively screens out candidate Q&A entries with unique value, avoiding the accumulation of large amounts of duplicate or highly similar content in the knowledge base.

[0087] In the embodiment, the semantic similarity is evaluated from different angles by combining the cosine similarity and the Jaccard coefficient, and when the cosine similarity is less than the first preset threshold and the Jaccard coefficient is less than the second preset threshold, it is determined that the candidate question and answer entry meets the new addition condition, and is stored in the knowledge base. The double evaluation mechanism can effectively identify new question and answer entries that are different in semantics, add new question and answer entries to the question and answer knowledge base, and increase the comprehensiveness of the question and answer knowledge base.

[0088] In further embodiments, please refer to Figure 5 When the similarity does not meet the preset new addition condition, the full-text semantics of the corresponding candidate question entry and the reference question and answer entry is judged by calling a large language model, and if they are not the same, the corresponding candidate question entry is stored in the question and answer knowledge base, including the following steps:

[0089] Step S1310, when the cosine similarity is greater than the first threshold and / or the Jaccard coefficient is greater than the second threshold, it is determined that the preset new addition condition is not met, and the standard question of the corresponding candidate question and answer entry and the standard question of the corresponding reference question and answer entry are input into the semantic judgment prompt template to generate a semantic judgment prompt instruction:

[0090] When the cosine similarity between the standard question of the candidate question and answer entry and the standard question of the reference question and answer entry is greater than the first preset threshold and / or the Jaccard coefficient is greater than the second preset threshold, it indicates that the two standard questions have semantic repetition or are too similar. At this time, the large language model needs to be further used for semantic judgment to determine whether the two standard questions truly express the same meaning. Specifically, the standard question of the candidate question and answer entry and the standard question of the corresponding reference question and answer entry are input into the preset semantic judgment prompt template to generate a semantic judgment prompt instruction for further judgment. The semantic judgment prompt instruction provides a clear task for the large language model, i.e., to judge whether the two questions are the same in semantics. For steps of using the large language model for judgment, please refer to the subsequent specific embodiments, which will not be described here.

[0091] Step S1320, inputting the semantic judgment prompt instruction into the large language model to control the large language model to judge whether the standard question of the candidate question and answer entry and the standard question of the corresponding reference question and answer entry are the same in semantics:

[0092] In one embodiment, the semantic judgment prompt template includes specific references to two standard questions and clear instructions, requiring the large language model to output whether the two standard questions are semantically identical. For example, the semantic judgment prompt template is set to: "Please determine whether the following two questions express the same meaning: Question 1: [Standard question of the candidate Q&A item]; Question 2: [Standard question of the benchmark Q&A item]. If the two questions are semantically identical, output 'identical'; if not, output 'not identical'."

[0093] The output of the large language model will be captured and used for subsequent processing. If the large language model determines that the semantics of two standard questions are the same, the candidate question and answer entry will be marked as pending for further manual review. If the large language model determines that the semantics of the two standard questions are different, the candidate question and answer entry will be directly added to the question and answer knowledge base.

[0094] Step S1330: If they are not the same, the corresponding candidate question and answer entry is stored in the question and answer knowledge base:

[0095] If the large language model determines that the semantics of two standard questions are different, the representation of the candidate question and answer entry provides information that is different from the existing question and answer knowledge base. At this time, the candidate question and answer entry is judged to be a question and answer entry that is not stored in the question and answer knowledge base and should be included in the question and answer knowledge base to enrich the content of the knowledge base and provide more comprehensive assistance to merchants.

[0096] Once a candidate Q&A entry is successfully stored in the Q&A knowledge base, it can be used by the intelligent customer service system to provide assistance to merchants. This step effectively incorporates new and valuable knowledge into the Q&A knowledge base, thereby improving the coverage and accuracy of the knowledge base.

[0097] In this embodiment, a semantic judgment mechanism based on a large language model is used to process candidate Q&A entries that fail to meet the pre-set additional criteria in the initial similarity assessment. If the judgment result indicates that the two standard questions are semantically different, the candidate Q&A entry is directly stored in the Q&A knowledge base, thereby enriching the knowledge base content and providing more comprehensive assistance to merchants. By leveraging the powerful semantic understanding capabilities of the large language model, this embodiment can avoid mistakenly rejecting valuable candidate Q&A entries due to superficial similarities.

[0098] For further examples, please refer to Figure 6 , the method further comprises the following steps:

[0099] Step S2100: Calculate the cosine similarity between the answer texts of each benchmark question and answer item in the question and answer knowledge base:

[0100] In one embodiment, the answer text in each benchmark question and answer entry in the question and answer knowledge base is converted into a vector form using a pre-trained word embedding model, wherein the word embedding model includes, for example, Word2Vec, GloVe, or BERT. After the answer text is converted into an answer vector using the word embedding model, the cosine similarity between the answer texts of each benchmark question and answer entry in the question and answer knowledge base is calculated. In some embodiments, the cosine similarity is calculated using a natural language processing library such as NLTK, spaCy, or TextBlob. The calculated cosine similarity value ranges from -1 to 1. The closer the value is to 1, the more similar the two vectors are, that is, the more semantically similar the two answer texts are.

[0101] By calculating the cosine similarity, we can effectively identify and process the duplication or high similarity of answer texts in the knowledge base, thereby improving the efficiency and accuracy of the knowledge base.

[0102] Step S2200: When the cosine similarity is greater than a preset third threshold, the corresponding benchmark question and answer item is input into the large language model, and the large language model is controlled to determine whether the benchmark question item is a semantically duplicate item that can be merged.

[0103] The third threshold is preset by those skilled in the art as needed. When the cosine similarity is greater than the preset third threshold, it indicates that the two answer texts have a high semantic similarity. After determining that the two answer texts have a high semantic similarity, the advanced natural language processing capabilities of the large language model are further used to determine whether the two answer texts are semantically repeated items that can be merged. In one embodiment, a semantic repetition judgment template is pre-designed, and the semantic repetition judgment template is used to guide the large language model to judge whether the two answer texts can be merged. For example, the content of the semantic repetition judgment template is set to "Please determine whether the following two benchmark question and answer items express The two benchmark Q&A entries have the same meaning, thereby determining whether they can be merged. Q&A entry 1: [the content of benchmark Q&A entry 1]; Q&A entry 2: [the content of benchmark Q&A entry 2]. If the two benchmark Q&A entries are semantically identical, output 'can be merged'; if they are different, output 'cannot be merged'. "Two benchmark Q&A entries corresponding to answer texts with high semantic similarity are input into the semantic repetition judgment template to obtain a semantic repetition judgment instruction. This semantic repetition judgment instruction is input into the large language model to control the large language model to determine whether the two benchmark Q&A entries express the same meaning, that is, whether they have the same semantics.

[0104] Through this step, the semantic analysis mechanism based on the large language model also provides a reliable method for the automated maintenance of the knowledge base, reducing the need for manual intervention and improving the automation level and operational efficiency of this application.

[0105] Step S2300: If yes, delete any one of the benchmark question entries from the question-answering knowledge base, and merge similar questions in the deleted benchmark question entry into multiple similar questions in the retained benchmark question entry:

[0106] If the large language model in the previous step determines that the two benchmark Q&A entries are semantically duplicate entries that can be merged, one of the benchmark Q&A entries will be deleted from the Q&A knowledge base. The deletion of the benchmark Q&A entry is based on various factors, such as deleting older entries, deleting entries with less complete content, or deleting entries that are slightly different in semantics but not sufficient to be retained. After the deletion operation is completed, similar questions in the deleted benchmark Q&A entry are merged into the retained benchmark Q&A entry. The purpose of this step is to enrich the question coverage of the retained entry and ensure that more user questioning methods can be matched with the correct answers. During the merging process, all similar questions in the deleted entry need to be extracted and added to the list of similar questions of the retained entry.

[0107] This embodiment effectively reduces redundant content in the question-and-answer knowledge base and, by merging similar questions, enhances the knowledge base's question coverage. This not only improves the efficiency and accuracy of the knowledge base, but also provides merchants with more comprehensive and accurate help information. This step, by combining cosine similarity calculations with the semantic analysis capabilities of large language models, enables automated identification and merging of duplicate or highly similar entries in the knowledge base, reducing the need for manual intervention and improving knowledge base maintenance efficiency. This ensures that the intelligent customer service system can operate at optimal performance and provide merchants with timely and accurate support.

[0108] For further examples, please refer to Figure 7 If they are the same, the corresponding candidate question entry is marked as pending review, and after determining whether to overwrite the corresponding benchmark question and answer entry based on the corresponding review result data and storing it in the question and answer knowledge base, the following steps are included:

[0109] Step S1500: Obtain the question text submitted by the merchant user. When the matching degree between the question text and all standard questions and similar questions in the question-answering knowledge base is lower than a preset matching threshold, calculate the semantic similarity between the question text and all standard questions and similar questions in the question-answering knowledge base, and generate a list of candidate questions sorted in descending order of semantic similarity:

[0110] In the intelligent customer service system, when the merchant user submits a question text, the question is first matched with all standard questions and similar questions in the question and answer knowledge base, and the matching process is based on a preset matching threshold to quickly determine the knowledge base entry most matched with the user question and provide the corresponding answer. However, if the matching degree of the question text submitted by the merchant user with all standard questions and similar questions in the knowledge base is lower than the preset matching threshold, it indicates that the intelligent customer service system cannot directly determine the highly relevant answer text. At this time, the semantic similarity of the question text with all standard questions and similar questions in the knowledge base is calculated, and a candidate question list ranked in descending order of semantic similarity is generated.

[0111] The generated candidate question list is used to show the merchant user the standard questions or similar questions in the candidate question list in the subsequent step, and ask the merchant user whether he wants to ask the questions in the candidate question list. If the user confirms that one of the questions is what he wants to ask, the corresponding answer text can be directly provided. For specific steps, please refer to the subsequent specific embodiments, which will not be described here.

[0112] Step S1600, filter a preset number of standard questions or similar questions ranked at the top from the candidate question list as a candidate recommended question set and push it to the merchant user:

[0113] From the descendingly sorted candidate question list, a preset number of standard questions or similar questions ranked at the top are filtered to form a candidate recommended question set, and the candidate question set is pushed to the merchant user, wherein the number of filtering is set by a person skilled in the art as needed. For example, the intelligent customer service system shows a dialog box to the merchant user, which lists three candidate questions and displays a prompt asking whether the user wants to ask these questions: "Are you asking one of the following questions?"

[0114] This step can effectively push the most relevant candidate question set to the merchant user, improving the efficiency and accuracy of problem solving.

[0115] Step S1700, receive the target question selected by the merchant user from the candidate recommended question set, and add the question text input by the user to the multiple similar questions of the reference question and answer entry corresponding to the target question:

[0116] Receive the target question selected by the merchant user from the candidate recommended question set, and add the question text originally entered by the user to multiple similar questions of the benchmark question and answer entry corresponding to the target question, so as to enrich and optimize the content of the question and answer knowledge base, so that the knowledge base can better cover the merchant user's question-asking method, thereby improving the accuracy and efficiency of future question matching. When the merchant user determines the target question from the candidate recommended question set, it indicates that the user believes that the question is the most relevant to the question originally submitted. Therefore, adding the merchant user's original question text to the list of similar questions of the target question can increase the knowledge base's coverage of different ways of expressing the question, so that the intelligent customer service system can more accurately identify and match similar questions in the future.

[0117] In this embodiment, not only can the current merchant user's problems be solved immediately, but the user feedback can also be used to optimize the content of the knowledge base, thereby improving the long-term performance and user experience of the intelligent customer service system. This dynamic update mechanism based on user feedback is an important way for the intelligent customer service system to continuously learn and improve, which helps the intelligent customer service system to better adapt to the needs of merchant users and provide more accurate and efficient services.

[0118] For further examples, please refer to Figure 8 , the method further comprises the following steps:

[0119] Step S3100: running a human-computer dialogue service based on the question-answer knowledge base, collecting feedback information submitted by online users after providing answer texts in the question-answer knowledge base to online users during the running of the human-computer dialogue service:

[0120] In an intelligent customer service system, maintaining the relevance and accuracy of the Q&A knowledge base over time requires continuous collection and analysis of user feedback to ensure that the information in the Q&A knowledge base meets the needs of online users. A human-computer dialogue service is run based on the Q&A knowledge base. During the operation of the human-computer dialogue service, the intelligent customer service system provides online users with corresponding answer texts from the Q&A knowledge base. After viewing the answer texts, online users can provide feedback on the answer texts using the feedback mechanism provided by the human-computer dialogue service. In some embodiments, this feedback is provided as an open text box, allowing users to describe their opinions and suggestions on the answer texts in detail.

[0121] The collected feedback information will be stored in a dedicated database for analysis and processing in subsequent implementations. In some embodiments, this feedback information includes not only direct user evaluations, but also user behavior data, contextual information about the questions, and detailed content of the answers. Subsequent comprehensive analysis of this feedback information will evaluate the performance of benchmark Q&A entries in the Q&A knowledge base, identifying which answers are effective, which need improvement, and which questions may be new areas not yet covered in the knowledge base.

[0122] Step S3200: Input the feedback information and the corresponding answer text into the large language model, and control the large language model to generate an optimized candidate answer text based on the feedback information:

[0123] Before inputting the feedback information and corresponding answer text collected in the previous step into the large language model, preprocess the data to ensure that the format and content of the input data meet the requirements of the large language model. This includes removing irrelevant special characters, unifying text encoding, and integrating the feedback information and answer text into a coherent input format. For example, the feedback information and answer text are combined into natural language instructions, such as: "Optimize the following answer based on user feedback: [original answer text]. User feedback: [feedback information]."

[0124] When the preprocessed data is input into the large language model, the large language model will analyze and optimize the original answer text based on the feedback information. The large language model considers the specific opinions and suggestions in the feedback information, as well as the semantic content of the original answer text, to generate optimized candidate answer text. This step can identify the shortcomings in the original answer text and generate optimized candidate answer text based on the user's feedback information. For example, suppose the original answer text is: "You can find the configuration options for the automatic reply function in the settings of the store backend, and just follow the prompts on the page." The user feedback is: "This answer is not detailed enough, and I hope there will be more specific step-by-step instructions." After receiving this information, the large language model generates an optimized candidate answer text: "You can find the configuration options for the automatic reply function in the settings of the store backend. The specific steps are as follows: 1. Log in to the store backend; 2. Click the settings menu; 3. Select the automatic reply function; 4. Fill in the relevant information according to the page prompts and save the settings."

[0125] Through this step, the powerful natural language processing capabilities of the large language model are utilized to generate optimized candidate answer texts based on user feedback. This not only improves the quality and satisfaction of the answer texts, but also provides important support for the dynamic update and optimization of the question-and-answer knowledge base.

[0126] Step S3300: Display the optimized candidate answer text to the user, and monitor whether the user provides secondary feedback on the optimized candidate answer text:

[0127] The optimized candidate answer text generated in the previous step is displayed to the user, and the user is monitored to see whether they provide secondary feedback on the optimized answer text to further verify whether the optimized answer text meets the user's needs, and to collect further user opinions for further optimization.

[0128] Specifically, the user's actions are monitored in real time to determine whether the user has provided secondary feedback. In some embodiments, if the user has provided secondary feedback, the secondary feedback information is recorded, and the secondary feedback information and the corresponding optimized candidate answer text are again input into the large language model. The answer text is further optimized and displayed to the online user. The user's actions are further monitored to determine whether the user has provided feedback again. This process is repeated until the user receives a satisfactory answer text (i.e., no feedback is received after the answer text is displayed to the user).

[0129] Step S3400: If no secondary feedback is received during the operation of the human-computer dialogue service, the optimized candidate answer text is marked as pending review, and it is determined based on the corresponding review result data whether to overwrite the corresponding answer text in the question-and-answer knowledge base with the optimized candidate answer text:

[0130] If no secondary feedback is received during the operation of the human-computer dialogue service, the optimized candidate answer text that has not received secondary feedback will be marked as pending review status, where the pending review status indicates that the corresponding answer text needs to be manually reviewed to determine whether the answer text can be included in the question and answer knowledge base to replace the original answer text.

[0131] The review process is performed by professional customer service personnel or a content review team. Reviewers evaluate the optimized candidate answer text based on a series of criteria, including the accuracy, completeness, comprehensibility, and compliance with the platform's policies and guidelines. For example, reviewers will check whether the answer provides sufficient details, whether it addresses the user's possible questions, and whether it uses clear and professional language.

[0132] This step ensures that only optimized answer texts that have undergone rigorous review will be included in the question-and-answer knowledge base, thereby ensuring the quality and accuracy of the knowledge base content.

[0133] In this embodiment, user feedback from the human-computer dialogue service is collected, and optimized candidate answer texts are generated using a large language model. These are then verified through secondary user feedback, and ultimately a manual review determines whether the optimized answer texts are included in the knowledge base. This embodiment enables dynamic updating and optimization of the question-and-answer knowledge base, improving the quality of the answer texts and user satisfaction. By collecting user feedback and optimizing them using a large language model, it is possible to identify and improve deficiencies in the answer texts in the knowledge base.

[0134] See also Figure 9, provided by one of the purposes of the present application, is a functional embodiment of the question and answer knowledge base updating method of the present application. The device comprises a question and answer generation module 1100, a similarity calculation module 1200, a semantic judgment module 1300, and an audit processing module 1400. The question and answer generation module 1100 is configured to input the platform help document into a large language model when it is monitored that the platform help document is updated, control the large language model to generate a plurality of candidate question and answer entries, and the candidate question and answer entries include standard questions, a plurality of similar questions and corresponding answer texts. The similarity calculation module 1200 is configured to calculate the similarity between the standard questions in the plurality of candidate question and answer entries and the standard questions in the reference question and answer entries already stored in the question and answer knowledge base. When the similarity meets the preset addition condition, the corresponding candidate question and answer entry is stored in the question and answer knowledge base. The semantic judgment module 1300 is configured to call the large language model to judge whether the full-text semantics of the corresponding candidate question entry and the reference question and answer entry are the same when the similarity does not meet the preset addition condition. If not, the corresponding candidate question entry is stored in the question and answer knowledge base. The audit processing module 1400 is configured to mark the corresponding candidate question entry as a to-be-audited state if the same, and determine whether to cover the corresponding reference question and answer entry according to the corresponding audit result data and store it in the question and answer knowledge base.

[0135] In a further embodiment, the question and answer generation module 1100 comprises a document acquisition unit configured to acquire a platform help document with version change description annotations when it is monitored that the version number of the platform help document is changed, the version change description annotations being used to indicate the incremental modification content of the current update; an instruction generation unit configured to embed the acquired platform help document into a preset question and answer generation prompt template to obtain a question and answer generation prompt instruction, the question and answer generation prompt template containing natural language instructions for guiding the large language model to generate a plurality of candidate question and answer entries based on the incremental modification content of the platform help document; and a model execution unit configured to input the question and answer generation prompt instruction into the large language model to control the large language model to generate a plurality of candidate question and answer entries based on the incremental modification content of the platform help document.

[0136] In a further embodiment, the similarity calculation module 1200 includes: a cosine similarity calculation unit, configured to calculate the cosine similarity between the standard question of the candidate question and answer item and the standard question of each benchmark question and answer item for each candidate question and answer item; a Jaccard coefficient calculation unit, configured to calculate the Jaccard coefficient between the standard question of the candidate question and answer item and the standard question of each benchmark question and answer item; and a storage determination unit, configured to confirm that the preset new condition is met when the cosine similarity is less than a preset first threshold and the Jaccard coefficient is less than a preset second threshold, and store the corresponding candidate question and answer item in the question and answer knowledge base.

[0137] In a further embodiment, the semantic judgment module 1300 includes: a prompt instruction generation unit, which is configured to, when the cosine similarity is greater than the first threshold and / or the Jacquard coefficient is greater than the second threshold, confirm that the preset new condition is not met, and input the standard question of the corresponding candidate question and answer item and the standard question of the corresponding benchmark question and answer item into the semantic judgment prompt template to generate a semantic judgment prompt instruction; a semantic analysis unit, which is configured to input the semantic judgment prompt instruction into the large language model to control the large language model to judge whether the standard question of the candidate question and answer item and the standard question of the corresponding benchmark question and answer item have the same semantics; a difference processing unit, which is configured to store the corresponding candidate question and answer item in the question and answer knowledge base if they are not the same.

[0138] In a further embodiment, the method also includes: an answer similarity detection module, configured to calculate the cosine similarity between the answer texts of each benchmark question and answer item in the question and answer knowledge base; a duplicate item judgment module, configured to input the corresponding benchmark question and answer item into the large language model when the cosine similarity is greater than a preset third threshold, and control the large language model to determine whether the benchmark question item is a mergeable semantically duplicate item; an item merging processing module, configured to, if so, delete any one of the benchmark question items from the question and answer knowledge base, and merge similar questions in the deleted benchmark question item into multiple similar questions of the retained benchmark question item.

[0139] In a further embodiment, after the review processing module 1400, it includes: a user question processing module, which is configured to obtain the question text submitted by the merchant user, and when the matching degree of the question text with all standard questions and similar questions in the question and answer knowledge base is lower than a preset matching threshold, calculate the semantic similarity between the question text and all standard questions and similar questions in the question and answer knowledge base, and generate a list of candidate questions arranged in descending order of semantic similarity; an intelligent recommendation module, which is configured to filter out a preset number of standard questions or similar questions with high ranking from the candidate question list and push them as a candidate recommended question set to the merchant user; a knowledge base update module, which is configured to receive the target question selected by the merchant user from the candidate recommended question set, and add the question text input by the user to the multiple similar questions of the benchmark question and answer entry corresponding to the target question.

[0140] In a further embodiment, the method also includes: a dialogue service monitoring module, which is configured to run a human-computer dialogue service based on the question-and-answer knowledge base, and collect feedback information submitted by the online user after the answer text in the question-and-answer knowledge base is provided to the online user during the operation of the human-computer dialogue service; an answer optimization generation module, which is configured to input the feedback information and the corresponding answer text into a large language model, and control the large language model to generate an optimized candidate answer text based on the feedback information; a user feedback monitoring module, which is configured to display the optimized candidate answer text to the user, and monitor whether the user provides secondary feedback on the optimized candidate answer text; a knowledge base optimization module, which is configured to mark the optimized candidate answer text as pending review if no secondary feedback is received during the operation of the human-computer dialogue service, and determine whether to overwrite the corresponding answer text in the question-and-answer knowledge base with the optimized candidate answer text based on the corresponding review result data.

[0141] In order to solve the above technical problems, the embodiment of the present application also provides a computer device. Figure 10 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions, and the database may store a control information sequence, and when the computer-readable instructions are executed by the processor, the processor may implement a question-and-answer knowledge base update method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor may execute the question-and-answer knowledge base update method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 10The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0142] The processor in the embodiment is configured to execute the specific functions of each module and the sub-modules thereof in the above method, and the memory stores the program codes and various data required for executing the above modules or sub-modules. The network interface is configured to transmit data between the user terminal and the server. The memory in the embodiment stores the program codes and data required for executing all modules / sub-modules in the question and answer knowledge base updating device of the present application, and the server can call the program codes and data of the server to execute the functions of all sub-modules. Figure 9

[0143] The present application also provides a storage medium storing computer readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the question and answer knowledge base updating method of any embodiment of the present application.

[0144] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments of each method. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0145] Those skilled in the art can understand that the steps, measures, and schemes in the various operations, methods, and processes discussed in the present application can be alternated, changed, combined, or deleted. Further, other steps, measures, and schemes in the various operations, methods, and processes discussed in the present application can also be alternated, changed, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and schemes in the prior art with the various operations, methods, and processes disclosed in the present application can also be alternated, changed, rearranged, decomposed, combined, or deleted.

[0146] The above is only some embodiments of the present application. It should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered as falling within the scope of protection of the present application.​

Claims

1. A question-answer knowledge base updating method, characterized in that: The steps include: When it is detected that the platform help document is updated, the platform help document is input into the large language model, and the large language model is controlled to generate multiple candidate question and answer entries, wherein the candidate question and answer entries include a standard question, multiple similar questions, and corresponding answer texts; Calculating the similarity between the standard questions in the plurality of candidate question and answer entries and the standard questions in the benchmark question and answer entries stored in the question and answer knowledge base, and storing the corresponding candidate question and answer entry in the question and answer knowledge base when the similarity meets a preset additional condition; When the similarity does not meet the preset additional condition, the large language model is called to determine whether the full text semantics of the corresponding candidate question item and the benchmark question and answer item are the same; if not, the corresponding candidate question item is stored in the question and answer knowledge base; If they are the same, the corresponding candidate question entry will be marked as pending review status, and it will be determined based on the corresponding review result data whether to overwrite the corresponding benchmark question and answer entry and store it in the question and answer knowledge base.

2. The question-answer knowledge base updating method according to claim 1, characterized in that: When the platform help document is detected to be updated, the platform help document is input into the large language model, and the large language model is controlled to generate multiple candidate question and answer entries. The candidate question and answer entries include a standard question, multiple similar questions, and corresponding answer texts, including: When a version change of the platform help document is detected, the platform help document with a version change description is obtained, where the version change description is used to indicate the incremental modification content of the update; Embedding the obtained platform help document into a preset question and answer generation prompt template to obtain a question and answer generation prompt instruction, wherein the question and answer generation prompt template includes natural language instructions for guiding the large language model to generate multiple candidate question and answer entries based on the incrementally modified content of the platform help document; The question and answer generation prompt instruction is input into the large language model, and the large language model is controlled to generate multiple candidate question and answer entries based on the incremental modified content in the platform help document.

3. The question-answer knowledge base updating method according to claim 1, characterized in that: Calculating the similarity between the standard questions in the plurality of candidate question and answer entries and the standard questions in the benchmark question and answer entries stored in the question and answer knowledge base, and storing the corresponding candidate question and answer entries in the question and answer knowledge base when the similarity meets a preset additional condition, including: For each candidate Q&A item, calculate the cosine similarity between the standard question of the candidate Q&A item and the standard question of each benchmark Q&A item; Calculate the Jacquard coefficient between the standard questions of the candidate question-and-answer items and the standard questions of each benchmark question-and-answer item; When the cosine similarity is less than a preset first threshold and the Jacquard coefficient is less than a preset second threshold, it is confirmed that the preset new condition is met, and the corresponding candidate question and answer entry is stored in the question and answer knowledge base.

4. The question-answer knowledge base updating method according to claim 3, characterized in that: When the similarity does not meet the preset additional condition, the large language model is called to determine whether the full text semantics of the corresponding candidate question item and the benchmark question and answer item are the same. If not, the corresponding candidate question item is stored in the question and answer knowledge base, including: When the cosine similarity is greater than the first threshold and / or the Jacquard coefficient is greater than the second threshold, it is determined that the preset additional condition is not met, and the standard question corresponding to the candidate question and answer item and the standard question corresponding to the benchmark question and answer item are input into the semantic judgment prompt template to generate a semantic judgment prompt instruction; Inputting the semantic judgment prompt instruction into the large language model to control the large language model to determine whether the standard question of the candidate question and answer item is semantically identical to the standard question of the corresponding benchmark question and answer item; If they are not the same, the corresponding candidate question and answer entries are stored in the question and answer knowledge base.

5. The question-answer knowledge base updating method according to any one of claims 1 to 4, characterized in that: The method further comprises: Calculate the cosine similarity between the answer texts of each benchmark question and answer entry in the question and answer knowledge base; When the cosine similarity is greater than a preset third threshold, inputting the corresponding benchmark question-answer item into the large language model, and controlling the large language model to determine whether the benchmark question item is a semantically duplicate item that can be merged; If so, any one of the benchmark question entries is deleted from the question-answering knowledge base, and similar questions in the deleted benchmark question entry are merged into multiple similar questions in the retained benchmark question entry.

6. The question-answer knowledge base updating method according to any one of claims 1 to 4, characterized in that: If they are the same, the corresponding candidate question entry is marked as pending review, and after determining whether to overwrite the corresponding benchmark question and answer entry based on the corresponding review result data and storing it in the question and answer knowledge base, the following steps are included: Obtain the question text submitted by the merchant user. When the matching degree between the question text and all standard questions and similar questions in the question-answering knowledge base is lower than a preset matching threshold, calculate the semantic similarity between the question text and all standard questions and similar questions in the question-answering knowledge base, and generate a list of candidate questions sorted in descending order of semantic similarity. Filtering a preset number of standard questions or similar questions ranked at the top from the candidate question list as a candidate recommended question set and pushing them to the merchant user; A target question selected by the merchant user from the candidate recommended question set is received, and the question text input by the user is added to multiple similar questions of the benchmark question and answer entry corresponding to the target question.

7. The question-answer knowledge base updating method according to any one of claims 1 to 4, characterized in that: The method further comprises: Running a human-computer dialogue service based on the question-and-answer knowledge base, and collecting feedback information submitted by online users after answer texts in the question-and-answer knowledge base are provided to online users during the running of the human-computer dialogue service; Inputting the feedback information and the corresponding answer text into a large language model, and controlling the large language model to generate an optimized candidate answer text based on the feedback information; Displaying the optimized candidate answer text to the user, and monitoring whether the user provides secondary feedback on the optimized candidate answer text; If no secondary feedback is received during the operation of the human-computer dialogue service, the optimized candidate answer text will be marked as pending review, and it will be determined based on the corresponding review result data whether the optimized candidate answer text will overwrite the corresponding answer text in the question and answer knowledge base.

8. A question-answer knowledge base updating device, characterized in that: include: a question and answer generation module configured to, when detecting that a platform help document has been updated, input the platform help document into the large language model and control the large language model to generate multiple candidate question and answer entries, wherein the candidate question and answer entries include a standard question, multiple similar questions, and corresponding answer texts; a similarity calculation module configured to calculate the similarity between the standard questions in the plurality of candidate question and answer entries and the standard questions in the benchmark question and answer entries stored in the question and answer knowledge base, and store the corresponding candidate question and answer entry in the question and answer knowledge base when the similarity satisfies a preset additional condition; a semantic judgment module configured to, when the similarity does not satisfy the preset additional condition, call the large language model to determine whether the full-text semantics of the corresponding candidate question item and the benchmark question and answer item are the same; if not, store the corresponding candidate question item in the question and answer knowledge base; The review processing module is configured to mark the corresponding candidate question entry as pending review if they are the same, and determine whether to overwrite the corresponding benchmark question and answer entry based on the corresponding review result data and store it in the question and answer knowledge base.

9. A computer device comprising a central processing unit and a memory, characterized in that: The central processing unit is configured to call and run a computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images