Customer service knowledge base preparation method and device, equipment and medium

Abstract

The invention relates to a customer service knowledge base preparation method and device, equipment and a medium in the technical field of e-commerce. The method comprises the following steps: determining a consultation matching degree between a user consultation text in each chat conversation pair in a chat database and each consultation text set in a consultation knowledge base; for each consultation text set, determining a user consultation text of which the consultation matching degree with the consultation text set exceeds a dynamic intention matching threshold value, and adding a corresponding artificial customer service text in the chat dialogue pair to an artificial reply set; manual customer service texts with similar expressions in the manual reply set are removed, and a corresponding customer service reply set is generated by adopting a large language model according to the manual reply set and the consultation text set after the manual reply set and the consultation text set are removed; and removing the customer service reply text, of which the consultation answer credibility with the consultation text set does not reach the standard, in the customer service reply set, and storing the removed customer service reply set in association with the consultation text set in the customer service knowledge base. Effective customer service reply knowledge storage can be dynamically constructed.

Description

Technical Field

[0001] This application relates to the field of e-commerce technology, and in particular to a method for preparing a customer service knowledge base and the corresponding apparatus, computer equipment, and computer-readable storage medium. Background Technology

[0002] In existing customer service knowledge base construction practices, companies typically use manual sorting or simple rule matching to convert historical dialogues into knowledge entries. Specifically, operations staff first select original "user question - customer service answer" pairs from massive chat logs based on their personal experience. Then, they perform keyword clustering or coarse-grained merging based on literal similarity to form several intent groups. Subsequently, standard answers corresponding to these intent groups are manually written or fine-tuned and finally incorporated into the knowledge base. The entire process is highly dependent on human judgment: on the one hand, question classification requires reviewing each one, and with diverse and highly colloquial user texts, subjective differences can easily lead to overly coarse or overly fine intent classifications; on the other hand, answer writing relies entirely on the operations staff's language organization skills, making it difficult to accommodate wording differences in different scenarios, resulting in an incomplete yet bloated knowledge base with a lengthy update cycle.

[0003] Because the above methods lack a quantitative evaluation mechanism for the mapping relationship between "user's true intent and effective customer service response," traditional knowledge bases often exhibit two typical defects after going live. First, the intent boundary is blurred: when a user's inquiry differs literally from a standard question but the intent... Figure 1 Firstly, the system fails to match correctly due to a fixed threshold, triggering a fallback mechanism that involves manual intervention, resulting in repetitive work. Secondly, answers are redundant or inaccurate: different customer service personnel exhibit significant differences in response styles and information density in similar scenarios. Simple deduplication only eliminates literal repetitions and cannot identify semantic similarities or missing information, leading to a large number of inefficient or even misleading answers in the knowledge base, further reducing the credibility of automated responses. These two defects jointly increase subsequent maintenance costs without substantially improving the generalization ability and accuracy of the knowledge base. Summary of the Invention

[0004] The primary objective of this application is to solve at least one of the above-mentioned problems by providing a method for preparing a customer service knowledge base and the corresponding apparatus, computer equipment, and computer program product.

[0005] To achieve the various objectives of this application, the following technical solution is adopted:

[0006] A method for preparing a customer service knowledge base, provided for one of the purposes of this application, includes the following steps:

[0007] Determine the consultation matching degree between the user consultation text in each chat dialogue pair stored in the chat database and the consultation text set stored in the consultation knowledge database. The consultation text set includes a standard consultation text and at least one similar consultation text with the same consultation intent.

[0008] For each set of consultation texts, identify user consultation texts whose consultation matching degree with the consultation text set exceeds the dynamic intent matching threshold, and add the corresponding human customer service text in the chat dialogue pair to the human response set. The dynamic intent matching threshold is determined based on the difficulty of expressing the consultation intent corresponding to the consultation text set.

[0009] Remove similar-sounding customer service texts from the set of manual replies, and use a large language model to generate a customer service reply set corresponding to the set of consultation texts based on the removed set of manual replies and the set of consultation texts.

[0010] Remove customer service reply texts that do not meet the credibility standard between the customer service reply set and the consultation text set, and store the customer service reply set after removal, along with the consultation text set, in the customer service knowledge base.

[0011] On the other hand, a customer service knowledge base preparation device provided to meet one of the purposes of this application includes a consultation matching module, a response set construction module, a response generation module, and a response storage module. The consultation matching module is used to determine the consultation matching degree between user consultation texts in various chat dialogue pairs stored in the chat database and various consultation text sets stored in the consultation knowledge base. Each consultation text set includes a standard consultation text and at least one similar consultation text with the same consultation intent. The response set construction module is used to, for each consultation text set, determine user consultation texts whose consultation matching degree with that consultation text set exceeds a dynamic intent matching threshold, and add the corresponding human customer service text in the chat dialogue pair to the human response set. The dynamic intent matching threshold is determined based on the difficulty of expressing the consultation intent corresponding to the consultation text set. The response generation module is used to remove similar human customer service texts from the human response set, and generate a customer service response set corresponding to the consultation text set using a large language model based on the removed human response set and the consultation text set. The response storage module is used to remove customer service response texts from the customer service response set whose consultation answer credibility does not meet the standard with the consultation text set, and store the removed customer service response set, associated with the consultation text set, in the customer service knowledge base.

[0012] In another aspect, a computer device provided for one of the purposes of this application includes a central processing unit and a memory, the central processing unit being used to invoke and run a computer program stored in the memory to perform the steps of the customer service knowledge base preparation method described in this application.

[0013] In another aspect, a computer program product provided for another purpose of this application includes a computer program / instructions that, when executed by a processor, implement the steps of the method described in any embodiment of this application.

[0014] The technical solution of this application has many advantages, including but not limited to the following aspects:

[0015] This application firstly implements a dynamic intent matching threshold mechanism, enabling the knowledge base to self-adjust based on the varying difficulty levels of the same consultation intent during the construction phase, thus avoiding the "one-size-fits-all" problem caused by traditional fixed thresholds. For intents with high colloquialism, rich ambiguity, or dense technical terminology, the matching threshold is automatically lowered, ensuring more potentially relevant dialogues are included in the training corpus; conversely, for straightforward intents with less ambiguity, the threshold is raised to eliminate noisy data. This fine-tuning strategy based on intent not only improves the recall and precision of corpus selection but also provides high-quality, highly relevant raw materials for subsequent generation stages, comprehensively enhancing the knowledge coverage depth and answer consistency of the customer service knowledge base.

[0016] Secondly, by first deduplicating similar expressions in the human customer service texts, and then generating responses using a large language model based on the refined set of human responses and consultation texts, a balance is achieved between the diversity, readability, and professionalism of the responses. The deduplication process effectively suppresses the "pattern hijacking" of the model by frequently repeated sentences, allowing the large language model to output richer and more natural expressions while retaining the authentic tone and business details of customer service. The secondary credibility filtering after model generation further filters out potential illusions or errors, ensuring that every response entering the knowledge base has verifiable business correctness. The resulting customer service response set inherits the experience accumulated by human service while also possessing the generalization and language organization advantages of the large model, significantly improving end-user satisfaction and trust when obtaining answers.

[0017] Furthermore, a three-tiered, progressive structure—"consultation text set—human response set—customer service response set"—is used to accumulate knowledge, enabling the knowledge base to iterate sustainably. When new business scenarios emerge or user questioning methods evolve, causing the chat database and consultation knowledge base to expand, the above process can be repeated for the newly added consultation text set to quickly expand the knowledge boundaries without requiring a large-scale rebuild of the entire database. This modular update method significantly reduces operational costs while ensuring the stability and traceability of the knowledge base version evolution. In the long run, it can continuously optimize itself with real dialogue data, forming a positive cycle of "the more data is used, the more accurate it becomes; the more knowledge is accumulated, the more comprehensive it becomes," providing a continuous high-quality, low-latency customer service capability. Attached Figure Description

[0018] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0019] Figure 1 The network architecture of the e-commerce platform exemplified in this application;

[0020] Figure 2 This is a flowchart illustrating a typical embodiment of the customer service knowledge base preparation method of this application;

[0021] Figure 3 This is a schematic diagram of the customer service knowledge base preparation device of this application;

[0022] Figure 4 This is a schematic diagram of the structure of a computer device used in this application. Detailed Implementation

[0023] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0024] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0025] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0026] like Figure 1In the network architecture shown, the e-commerce platform 82 is deployed on the Internet to provide corresponding services to its users. Similarly, the devices 80 of the merchant users and the devices 81 of the consumer users of the e-commerce platform 82 are also connected to the Internet to use the services provided by the e-commerce platform.

[0027] An exemplary e-commerce platform 82 provides supply and demand matching of products and / or services to the general public through the Internet infrastructure. In e-commerce platform 82, products and / or services are provided as commodity information. For the sake of simplicity, the concepts of commodity and product are used in this application to refer to the products and / or services in e-commerce platform 82. Specifically, these may be physical products, digital products, tickets, service subscriptions, other offline services, etc.

[0028] In reality, various entities can access e-commerce platform 82 as users and utilize its online services to participate in the business activities facilitated by the platform. These entities can be natural persons, legal persons, or social organizations. Corresponding to the two types of entities in business activities—merchants and consumers—e-commerce platform 82 has two corresponding categories of users: merchant users and consumer users. Entities involved in the product distribution chain in business activities, including manufacturers, sellers, retailers, and logistics providers, can all use online services on e-commerce platform 82 as merchant users. Similarly, consumers in business activities, including actual or potential consumers, can use online services on e-commerce platform 82 as consumer users. In actual business activities, the same entity can operate as both a merchant user and a consumer user; this should be interpreted flexibly.

[0029] The infrastructure used to deploy the e-commerce platform 82 mainly includes the backend architecture and frontend devices. The backend architecture runs various online services through a service cluster, including middleware or frontend services for the platform, services for consumers, and services for merchants, to enrich and improve its service functions. The frontend devices mainly cover the terminal devices used by users as clients to access the e-commerce platform 82, including but not limited to various mobile terminals, personal computers, and point-of-sale devices. For example, merchant users can use their terminal device 80 to enter product information for their online stores or use the interfaces opened by the e-commerce platform to generate their product information; consumer users can use their terminal device 81 to access the webpage of the online store implemented by the e-commerce platform 82, trigger the shopping process by clicking the shopping button provided on the webpage, and call various online services provided by the e-commerce platform 82 during the shopping process to achieve the purpose of placing an order.

[0030] In some embodiments, the e-commerce platform 82 may be implemented via a processing facility including a processor and memory, which stores a set of instructions that, when executed, cause the e-commerce platform 82 to perform the e-commerce and support functions as described in this application. The processing facility may be part of a server, client, network infrastructure, mobile computing platform, cloud computing platform, fixed computing platform, or other computing platform, and may provide electronic components, merchant devices, payment gateways, application developers, marketing channels, transportation providers, customer devices, point-of-sale devices, etc., for the e-commerce platform 82.

[0031] E-commerce platform 82 can provide online services such as cloud computing services, Software as a Service (SaaS), Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Desktop as a Service (DaaS), Hosted Software as a Service, Mobile Backend as a Service (MBaaS), and Information Technology Management as a Service (ITMaaS). In some embodiments, the various functional components of e-commerce platform 82 can be implemented to operate on various platforms and operating systems. For example, for an online store, its administrator user enjoys the same or similar functions regardless of whether it is on iOS, Android, HomonyOS, or a web page.

[0032] E-commerce platform 82 enables merchants to create their own independent websites to run their online stores. It provides merchants with corresponding business management engine instances, allowing them to establish, maintain, and operate one or more online stores across these independent websites. The business management engine instance can be used for content management, task automation, and data management for one or more online stores. It can be configured through interfaces or built-in components to support various specific business processes in the online store, supporting business activities. Independent websites are the infrastructure of e-commerce platform 82, which offers cross-border services. Merchants can maintain their online stores relatively independently and centrally based on these independent websites. Independent websites typically have dedicated domain names and storage space, and different independent websites are relatively independent. E-commerce platform 82 can provide standardized or customized technical support for a large number of independent websites, allowing merchants to customize a business management engine instance that suits their needs and use it to maintain one or more online stores.

[0033] Online stores can be configured and maintained in the backend by merchant users logging into their Business Management Engine instance as administrators. Supported by the various online services provided by the e-commerce platform 82's infrastructure, merchant users can configure various functions within their online stores and view various data as administrators. For example, merchant users can manage various aspects of their online stores, such as viewing recent online store activities, updating the online store's product catalog, managing orders, recent visit activity, and total order activity. Merchant users can also view more detailed information about their business and visitors to their online store by obtaining reports or metrics, such as displaying a sales summary of the merchant's overall business, specific sales and engagement data from promotional sales and marketing channels, etc.

[0034] E-commerce platforms 82 can provide communication facilities and associated merchant interfaces for electronic communication and marketing. For example, they can utilize electronic messaging aggregation facilities to collect and analyze communication interactions between merchants, consumers, merchant devices, customer devices, point-of-sale devices, etc., aggregating and analyzing communications to increase the potential for product sales. For instance, a consumer may have product-related questions, which could lead to a dialogue between the consumer and the merchant (or an automated processor-based agent representing the merchant), where the communication facilities handle the interaction and provide the merchant with analysis on how to increase the probability of a sale.

[0035] In some embodiments, applications suitable for installation on terminal devices can be provided to serve the access needs of different users, enabling various users to access the e-commerce platform 82 by running the application on their terminal devices. Examples include the merchant backend module of online stores within the e-commerce platform 82. During the process of conducting business activities through these functions, the e-commerce platform 82 can implement various functions related to business activities as middleware or online services and expose corresponding interfaces. Then, toolkits corresponding to the interface access functions are embedded into the application to achieve functional expansion and task completion. The business management engine can include a series of basic functions and expose these functions to online services and / or applications via APIs. Online services and applications use the corresponding functions by remotely calling the corresponding APIs.

[0036] With the support of various components of the Business Management Engine instance, E-commerce Platform 82 can provide online shopping functionality, enabling merchants to connect with customers in a flexible and transparent manner. Consumers can select items online, create orders, provide delivery addresses in the orders, and complete payment confirmation. Merchants can then review and complete or cancel orders.

[0037] The customer service knowledge base preparation method of this application can be programmed into a computer program product and deployed on a client or server. For example, in an exemplary application scenario of this application, it can be deployed on the server of an e-commerce customer service platform. In this way, the method can be executed by human-computer interaction with the process of the computer program product through a graphical user interface by accessing the interface opened after the computer program product is run.

[0038] Please see Figure 2 The customer service knowledge base preparation method of this application, in its typical embodiment, includes the following steps:

[0039] Step S1100: Determine the consultation matching degree between the user consultation text in each chat dialogue pair stored in the chat database and the consultation text set stored in the consultation knowledge base. The consultation text set includes a standard consultation text and at least one similar consultation text with the same consultation intent.

[0040] Chat dialogues are automatically captured, structured, and stored in the chat database as part of the daily customer service operations of the e-commerce platform. When a buyer inquires with customer service, triggering a chat session, a human customer service representative can intervene to provide a manual response to the buyer's inquiry. This typically occurs when automated customer service is unable to answer the inquiry, or when automated customer service is not yet available and manual responses are the only option. Each text message pushed in the chat session can be monitored and identified, allowing the text stream to be organized into one-inquiry-one-response pairs. For cases of multiple replies to one inquiry or one reply to multiple inquiries, consecutive replies or inquiries can be combined into a single sentence. Furthermore, for each inquiry-response pair, the sender of the reply text is identified as a human customer service representative. If a human representative is identified, the inquiry-response pair is stored in the chat database as a chat dialogue pair.

[0041] The consultation knowledge base can be pre-built by the customer service operations team of the e-commerce platform, and / or by the customer service operations personnel of the online stores within the e-commerce platform. Their main task is to create and maintain the consultation text sets stored in the knowledge base. Typically, the operations personnel consider covering all possible consultation intentions of buyer users at various stages of customer service, configuring a consultation text set for each consultation intention. This set contains a standard consultation text and its corresponding consultation intention. Figure 1At least one similar consultation text is required. The standard consultation text uses the most concise and accurate question to describe the consultation intent, such as "How to apply for a 7-day no-reason return?" Similar consultation texts cover various synonymous expressions that buyers might use in real-world scenarios, such as "How do I process a 7-day return?" or "Can I return the clothes I just bought?" These texts can be manually added by operations staff, or automatically generated using algorithms that utilize synonym expansion and template generalization, and then added after manual review. After constructing the consultation text set corresponding to these consultation intents, the consultation text set is stored in the consultation knowledge base.

[0042] When calculating the consultation matching degree, the user consultation text in each chat dialogue pair stored in the chat database is compared with each consultation text set stored in the consultation knowledge base. The comparison process first converts the user consultation text and all consultation texts in the consultation text set (including standard consultation texts and all similar consultation texts) into vector representations. This conversion can be achieved using OpenAI's embedding interface, Sentence-BERT, SimCSE, or other semantic encoding models. Then, the cosine similarity between the user consultation text vector and each consultation text vector in the consultation text set is calculated. The maximum value among all obtained cosine similarities is taken as the consultation matching degree between the buyer's consultation text and the consultation text set.

[0043] Step S1200: For each consultation text set, identify user consultation texts whose consultation matching degree with the consultation text set exceeds the dynamic intent matching threshold, and add the corresponding human customer service text in the chat dialogue pair to the human response set. The dynamic intent matching threshold is determined based on the difficulty of expressing the consultation intent corresponding to the consultation text set.

[0044] It is understandable that a dynamic intent matching threshold can be pre-determined for each set of consultation texts. This threshold is used to determine whether the user's consultation text and the consultation text set express the same consultation intent. The value of this threshold is not fixed, but is positively correlated with the difficulty of expressing the consultation intent corresponding to the consultation text set: the greater the difficulty of expression, the lower the threshold; the less difficult the expression, the higher the threshold. The difficulty of expression can be numerically represented as the average semantic similarity between the standard consultation text within the consultation text set and all similar consultation texts. The lower this average value, the more diverse the colloquial expression of the consultation intent, and the greater the difficulty; the higher this average value, the more straightforward and easy to understand the consultation intent, and the easier it is for ordinary people to express.

[0045] After determining the dynamic intent matching threshold for each set of consultation texts, a traversal is performed on all user consultation texts in the chat database. For the currently being processed consultation text set, the consultation matching degree between all consultation texts in the set and each user consultation text is compared one by one. If the maximum consultation matching degree between a user consultation text and the consultation text set is higher than the current dynamic intent matching threshold, then the consultation text expressed by that user consultation text and the consultation text set are considered to have a matching intent. Figure 1 At this point, locate the corresponding human customer service text in the chat dialogue pair for the user's inquiry text and add it to the dedicated human response set for that inquiry text set (initialized to empty). This process is executed independently for each inquiry text set, ultimately yielding the human response set for each inquiry text set.

[0046] Step S1300: Remove similar-sounding customer service texts from the set of manual replies, and use a large language model to generate a customer service reply set corresponding to the set of consultation texts based on the removed set of manual replies and the set of consultation texts;

[0047] Each customer service text in the manual response set is converted into a vector representation. This conversion can be achieved using OpenAI's embedding interface, Sentence-BERT, SimCSE, or other semantic encoding models. Therefore, in one embodiment, for each manual customer service text, the cosine similarity between its vector representation and the vector representations of all other manual customer service texts is calculated. The mean cosine similarity is then calculated for each of these cosine similarities. The calculated mean cosine similarity values ​​for each manual customer service text in the manual response set are then sorted from highest to lowest. The top N texts are identified as those expressing differences within the manual response set. Conversely, those not identified in the manual response set are similar in expression and need to be removed. N can be set as needed by those skilled in the art based on this disclosure. In another embodiment, the cosine similarity between each manual customer service text in the manual response set is calculated. Then, it is determined whether the cosine similarity between any two manual customer service texts exceeds a preset deduplication threshold. If it does, the text with the longer character length is retained, and the other text is considered redundant and removed from the manual response set. The deduplication threshold can be configured to 0.85, depending on the customer service's tolerance for "synonymous but different expressions" and / or the need to retain multiple polite or personalized expressions. The preset deduplication threshold can be further set as needed by those skilled in the art based on the disclosure herein.

[0048] Next, the set of manually edited responses, after removal, and their corresponding consultation text sets are embedded into a pre-configured response generation prompt template. This template is maintained by operations personnel in the user configuration interface of the visual image. An example of the template content is as follows:

[0049] "You are a senior e-commerce customer service representative. Based on the following buyer questions and sample replies from live customer service representatives, please standardize each manual reply in the list into a standard reply text. The standard reply text should be accurate and strictly adhere to the content of the original manual reply text that answered the user's inquiry, maintain the original manual reply text's style, and be clearly explained. Standardized user inquiries: {Placeholder: Standard inquiry text}; Other similar questions: {Placeholder: List of similar inquiry texts}; Sample replies from live customer service representatives: {Placeholder: Set of manual replies after removal}."

[0050] During embedding, corresponding data is filled into placeholders to obtain the response prompt text. The large language model is then invoked, guided by this prompt, to complete the generation process. Each standardized response text generated by the large language model is used as the customer service response text and added to the dedicated customer service response set (initialized to empty) for this consultation text set. The large language model is pre-trained to convergence using an extremely large corpus, acquiring the ability to generate text describing human language, possessing accurate basic text semantic understanding and basic logical reasoning capabilities. The large language model can be selected from Chinchilla, PaLM, LLaMA, Alpaca, Vicuna, etc., and any one of them can be chosen as needed.

[0051] Step S1400: Remove customer service reply texts from the customer service reply set that do not meet the credibility standard for consultation answers between the customer service reply set and the consultation text set, and store the customer service reply set after removal as an association with the consultation text set in the customer service knowledge base.

[0052] For each customer service reply text in the customer service reply set, the reply text and the standard consultation text in the consultation text set are input into a pre-trained, converged question-answering verification model. This model verifies whether the reply text can answer the consultation text and outputs a confidence level representing the answer. This confidence level falls within the value range [0,1], with higher confidence levels indicating a higher likelihood that the reply text can answer the consultation text. The question-answering verification model can be a deep learning model suitable for binary classification tasks in the field of NLP (Natural Language Processing). Multiple user consultation texts with clear consultation intentions are pre-collected. These user consultation texts and the standard reply texts that can answer the user consultation text form positive training samples, while the user consultation texts and the standard reply texts that cannot answer the user consultation text form negative training samples. Then, for each training sample, if the training sample is a positive example, the supervision label for the training sample is labeled as indicating that the reply text in the training sample can answer the consultation text, for example, 1; if the training sample is a negative example, the supervision label for the training sample is labeled as indicating that the reply text in the training sample cannot answer the consultation text, for example, 0. Finally, a training set is formed by combining all training samples and their supervision labels. The training set is then used to supervise the training of the question-answering verification model, which learns the verification basis or paradigm for whether the response text can answer the question text. This process continues until the model reaches a convergence state, at which point it learns to determine the credibility of the response text that represents the input question text.

[0053] When the credibility of a consultation answer exceeds the credibility threshold, the credibility of the consultation answer is considered to meet the standard, meaning that the corresponding customer service reply text can be used to answer the consultation text that expresses the consultation intent corresponding to the consultation text set; when the credibility of a consultation answer does not exceed the credibility threshold, the credibility of the consultation answer is considered to fail to meet the standard, meaning that the corresponding customer service reply text is not sufficient to answer the consultation text that expresses the consultation intent corresponding to the consultation text set, and accordingly, the customer service reply text is removed from the customer service reply set.

[0054] The customer service response sets that have been removed are mapped one-to-one with the consultation text sets, and then stored in the customer service knowledge base. In this way, the customer service knowledge base can store all the consultation text sets in the consultation knowledge base, as well as the customer service response sets required to answer the consultation intent corresponding to each consultation text set.

[0055] It is easy to understand from the above typical embodiments that, compared with the prior art, this application has many advantages, including at least:

[0056] This application firstly implements a dynamic intent matching threshold mechanism, enabling the knowledge base to self-adjust based on the varying difficulty levels of the same consultation intent during the construction phase, thus avoiding the "one-size-fits-all" problem caused by traditional fixed thresholds. For intents with high colloquialism, rich ambiguity, or dense technical terminology, the matching threshold is automatically lowered, ensuring more potentially relevant dialogues are included in the training corpus; conversely, for straightforward intents with less ambiguity, the threshold is raised to eliminate noisy data. This fine-tuning strategy based on intent not only improves the recall and precision of corpus selection but also provides high-quality, highly relevant raw materials for subsequent generation stages, comprehensively enhancing the knowledge coverage depth and answer consistency of the customer service knowledge base.

[0057] Secondly, by first deduplicating similar expressions in the human customer service texts, and then generating responses using a large language model based on the refined set of human responses and consultation texts, a balance is achieved between the diversity, readability, and professionalism of the responses. The deduplication process effectively suppresses the "pattern hijacking" of the model by frequently repeated sentences, allowing the large language model to output richer and more natural expressions while retaining the authentic tone and business details of customer service. The secondary credibility filtering after model generation further filters out potential illusions or errors, ensuring that every response entering the knowledge base has verifiable business correctness. The resulting customer service response set inherits the experience accumulated by human service while also possessing the generalization and language organization advantages of the large model, significantly improving end-user satisfaction and trust when obtaining answers.

[0058] Furthermore, a three-tiered, progressive structure—"consultation text set—human response set—customer service response set"—is used to accumulate knowledge, enabling the knowledge base to iterate sustainably. When new business scenarios emerge or user questioning methods evolve, causing the chat database and consultation knowledge base to expand, the above process can be repeated for the newly added consultation text set to quickly expand the knowledge boundaries without requiring a large-scale rebuild of the entire database. This modular update method significantly reduces operational costs while ensuring the stability and traceability of the knowledge base version evolution. In the long run, it can continuously optimize itself with real dialogue data, forming a positive cycle of "the more data is used, the more accurate it becomes; the more knowledge is accumulated, the more comprehensive it becomes," providing a continuous high-quality, low-latency customer service capability.

[0059] In a further embodiment, before step S1200, which determines user consultation texts for each consultation text set whose consultation matching degree exceeds the dynamic intent matching threshold, the following steps are included:

[0060] Step S1210: Determine the semantic similarity between the standard consultation text in the consultation text set and each similar consultation text, and calculate the average semantic similarity based on all semantic similarities;

[0061] A pre-trained, convergent text semantic model is used to determine the text feature vectors representing the semantics of the standard consultation text in the consultation text set, as well as the text feature vectors corresponding to the semantics of each similar consultation text in the set. The vector distance between the text feature vectors corresponding to the standard consultation text and each similar consultation text is then determined, serving as a measure and representation of the semantic similarity between the standard consultation text and each similar consultation text. Furthermore, the mean semantic similarity value corresponding to the semantic similarity between the standard consultation text and each similar consultation text is calculated.

[0062] The text semantic model is pre-trained to convergence, acquiring the ability to extract deep semantic information from the input text and output vectorized text feature vectors representing the semantics of the text. Specific models can include TextTransformer, RoBERTa, XLM-RoBERTa, MPNet, BERT, etc., which can be selected and implemented as needed by those skilled in the art. Since the training process of these models is known in the art, it will not be described in detail. Vector distance calculation can be implemented using any available vector distance algorithm, including but not limited to: cosine similarity algorithm, vector dot product algorithm, Manhattan distance, Euclidean distance algorithm, Pearson correlation coefficient, etc.

[0063] Step S1220: Using a preset language expression model, determine the corresponding expression difficulty coefficient based on the standard consultation texts in the consultation text set;

[0064] During training, the language expression model is input with standardized expressions of consultation intentions, while the supervision signal is the difficulty coefficient of expressing that intention among real buyers on the e-commerce platform. By minimizing the mean squared error or mean absolute error between the predicted values ​​and the supervision labels, the model gradually adjusts its internal parameters, ultimately implicitly learning the "intention-expression difficulty" mapping function. Specifically, the interpretable knowledge of this function allows the model to learn the differences in complexity of different consultation intentions in terms of vocabulary, sentence structure, terminology, and implicit background knowledge. For example, "How to apply for a 7-day no-reason return" uses everyday language and has a simple sentence structure, corresponding to low difficulty; "How to re-declare a failed customs clearance order in a cross-border bonded warehouse" contains professional terminology and multiple steps, corresponding to high difficulty. During training, the model typically considers compressing features such as terminology density, sentence length, word complexity, background knowledge threshold, or the possibility of missing information into a high-dimensional semantic space, forming an internal representation positively correlated with the difficulty coefficient.

[0065] During the inference phase, when a standard consultation text is input into the language expression model that has been trained to convergence, the model reuses the mapping relationship between "consultation intention and its expression difficulty" established by the knowledge learned above, and infers the corresponding expression difficulty coefficient between 0 and 1 for the consultation intention expressed by the standard consultation text.

[0066] Step S1230: Determine the dynamic intent matching threshold based on the expression difficulty coefficient and the mean semantic similarity.

[0067] The product of the expression difficulty coefficient and the mean semantic similarity is calculated and used as the dynamic intent matching threshold.

[0068] In this embodiment, the two originally independent dimensions of "mean semantic similarity" and "expression difficulty coefficient" are coupled into a dynamic intent matching threshold. This transforms the threshold from an empirical constant into a continuous variable that adaptively floats to match the characteristics of the consultation intent itself. This coupling method is not only highly interpretable and reliable, but also compresses two heterogeneous pieces of information—"diverse ways of accurately expressing consultation intent" and "the expression level of buyer groups on e-commerce platforms"—into an operable scalar, significantly reducing debugging costs. More importantly, the adaptive nature of the threshold allows the customer service knowledge base to lower the threshold for highly ambiguous and highly specialized consultation intents during the construction phase, preventing omissions; while raising the expression threshold for straightforward and easily understood consultation intents, suppressing noise from entering the database. This "adaptive consultation intent expression difficulty" mechanism is not publicly available in existing fixed threshold schemes, and its technical contribution is not limited to simple numerical mapping, but lies in fusing the semantic mean and expression difficulty to determine the threshold, thereby simultaneously improving recall and precision, significantly enhancing the credibility and accuracy of customer service knowledge in the customer service knowledge base.

[0069] In a further embodiment, before step S1220, which uses a preset language expression model to determine the corresponding expression difficulty coefficient based on the standard consultation texts in the consultation text set, the following steps are included:

[0070] Step S2220: Obtain a training set, which includes multiple training samples and their supervision labels. The training samples are standard consultation intentions, and the supervision labels represent the difficulty of a user expressing the consultation intention of a standard consultation text.

[0071] To train the language expression model to quantify the difficulty of expressing a specific inquiry intent from the perspective of the overall expressive ability of buyers across the entire e-commerce platform, a training set must first be constructed. The standard inquiry intents used as training samples in the training set can be manually refined, semantically clear, and unambiguous questions that accurately express an inquiry intent. These questions can be directly taken from standard inquiry texts in the inquiry knowledge base, such as "How to apply for a 7-day no-reason return policy." Alternatively, they can be descriptive tags or template texts abstracted from clusters of real inquiries from multiple buyers, as long as they uniquely and clearly express the inquiry intent. The expression difficulty coefficient is a real numerical value, typically normalized to 0–1. A higher value indicates that it is more difficult for buyers on a real e-commerce platform to express the inquiry intent in daily customer service inquiries.

[0072] There are three parallel or combined implementation paths for obtaining the expression difficulty coefficient. The first is crowdsourced scoring: Standard consultation intent is presented to multiple e-commerce customer service personnel for annotation. Annotators are required to rate the intent on a scale of 1-5 or 0-1, based on the "ease of an ordinary buyer accurately expressing this intent without any prompting." Higher scores indicate greater difficulty. The arithmetic mean of all scores is then calculated as the expression difficulty coefficient. The second is comparative analysis between original and standard consultations: A large number of original buyer expressions for the same consultation intent are collected from the platform. A pre-trained semantic encoding model encodes each original consultation text and the standard consultation intent into vectors, calculates the cosine or Euclidean distance between them, and then takes the arithmetic mean of all distances. The third is multi-task collaboration: Crowdsourced scoring and comparative analysis scoring are collected simultaneously. The final expression difficulty coefficient is output through linear weighting, with the sum of the weights of the two indicators being one. The specific values ​​of each weight can be configured as needed by those skilled in the art.

[0073] Step S2221: Use the training set to train the language expression model to a convergent state, and learn the ability to determine the expression difficulty coefficient corresponding to the user's consultation intent in the input consultation text.

[0074] After the training set is prepared, the language expression model is trained to convergence using this set. The training process employs supervised learning, where the input is the standard consultation intent and the output is the corresponding expression difficulty coefficient. The training objective is to minimize the mean squared error loss or mean absolute error loss between the model's predicted difficulty coefficient and the supervision label. The language expression model can be selected from deep learning architectures suitable for regression tasks, including fully connected neural networks, Transformer encoders, or fine-tuned pre-trained models such as BERT or RoBERTa. The training environment can be deployed on the e-commerce platform's server. Batch training iterations are performed by loading a pre-compiled training library onto the server. Each iteration includes forward propagation to calculate the predicted output and backpropagation applying gradient descent to adjust the model weights, until the training loss converges to a preset threshold or a specified number of iterations is reached. After training, the language expression model is capable of extracting semantic features from the input standard consultation text and outputting the expression difficulty coefficient. This expression difficulty coefficient can be directly used to calculate the dynamic intent matching threshold.

[0075] In this embodiment, by implementing crowdsourcing scoring, distance measurement of original corpus, and multi-task weighted fusion to support the deployment of supervised labels for training samples, the annotation bias can be overcome, enabling the model to truly learn to "stand from the perspective of buyers across the entire platform" to assess the difficulty of expressing specific consultation intentions.

[0076] In a further embodiment, after step S1400, which involves storing the removed customer service response set associated with the consultation text set into the customer service knowledge base, the following steps are included:

[0077] Step S1500: Perform the first clustering on the standard consultation texts in all consultation text sets in the consultation knowledge base to identify multiple customer service stage clusters;

[0078] First, the entire consultation text set in the consultation knowledge base is traversed, and standard consultation texts are extracted from each consultation text set. These standard consultation texts are then used as input for a clustering operation, which is called the first cluster. Clustering algorithms can include common unsupervised methods such as K-means, hierarchical clustering, DBSCAN, spectral clustering, or BERT-based Sentence-BERT+k-means. The goal of clustering is to divide the standard consultation texts into several customer service stage clusters based on semantic similarity. Each cluster corresponds to a customer service stage in the e-commerce platform's customer service process, including but not limited to pre-sales consultation, in-sales logistics, after-sales returns and exchanges, account and payment, and coupons and activities. In practice, operators can predefine labels for several customer service stages and then align the clustering results with the labels, or directly use the clustering results as stage definitions. The standard consultation texts within each customer service stage cluster have stage consistency; for example, "How to apply for a 7-day no-reason return" and "How long after returning an item will I receive a refund?" would both be classified as part of the after-sales returns and exchanges stage.

[0079] Step S1600: Determine the customer service stage cluster to which the user inquiry text in each chat dialogue pair stored in the chat database belongs;

[0080] After completing the stage division, all chat dialogue pairs in the chat database are read, user consultation texts are extracted, and the customer service stage clusters to which these user consultation texts belong are determined. Specifically, user consultation texts can be converted into vector representations, and the cosine similarity between the text and the cluster center vector of each customer service stage cluster is calculated. The user consultation text is then assigned to the customer service stage cluster with the highest similarity. The cluster center vector can be the average vector of all standard consultation text vectors within that stage cluster, or it can be the representative vector of that cluster. Thus, each user consultation text has its own customer service stage cluster.

[0081] Step S1700: For each customer service stage cluster, perform a second clustering on the human customer service text corresponding to all user inquiry texts belonging to that customer service stage cluster in the chat dialogue pair to determine multiple customer service knowledge sub-clusters.

[0082] Next, for each customer service stage cluster, all user inquiry texts belonging to that cluster are collected along with their corresponding human customer service texts in the chat dialogue pairs and then clustered. This clustering is called the second clustering. The second clustering can also employ methods such as K-means, hierarchical clustering, DBSCAN, spectral clustering, or Sentence-BERT+k-means. This clustering has a finer granularity, aiming to further subdivide the human customer service texts within the same stage into several customer service knowledge sub-clusters. Each sub-cluster represents a specific business scenario or question subtype. For example, in the after-sales return and exchange stage, sub-clusters might include "refund only," "return and refund," "exchange process," and "reissue goods." The human customer service texts within each customer service knowledge sub-cluster are semantically similar, covering multiple expressions from users within the same sub-scenario.

[0083] Step S1800: For each customer service knowledge subcategory, generate a corresponding customer service knowledge document based on the human customer service text belonging to that customer service knowledge subcategory and the consultation text set associated with its customer service response set.

[0084] For each customer service knowledge subcategory, a corresponding customer service knowledge document is generated based on the human customer service text belonging to that subcategory and the consultation text set associated with its corresponding customer service response set.

[0085] Based on each human customer service text belonging to the customer service knowledge subcategory, the system retrieves and obtains the consultation text set associated with the customer service response set of each human customer service text stored in the customer service knowledge base. Then, all human customer service texts and the standard consultation texts in the associated consultation text sets, along with at least one corresponding similar consultation text, are embedded into a structured prompt template according to fixed slots. This ensures that each human customer service text and its associated consultation text set are embedded within the same dialogue context. The prompt template uses natural language to describe the task, with an example example: "You are a senior e-commerce customer service knowledge engineer. Based on the following real human responses, the standard consultation texts for each response, and the corresponding list of similar questions, please write a customer service knowledge document for buyers on the e-commerce platform. Requirements: 1. Organize the content using four levels of headings: 'Problem Overview—Operation Steps—Precautions—Frequently Asked Questions'; 2. Maintain the business details and polite tone of the original human responses; 3. If the human responses contain image links or tables, retain them in Markdown format in the document."

[0086] Standardized user inquiries: {Placeholder: Standard inquiry text}; Other similar inquiries: {Placeholder: List of similar inquiry texts}; Real human responses: {Placeholder: Human customer service text 1}; ...

[0088] Standardized user inquiries: {Placeholder: Standard inquiry text}; Other similar inquiries: {Placeholder: List of similar inquiry texts}; Real human responses: {Placeholder: Human customer service text N}.

[0089] Step S1900: Store all customer service knowledge documents in the customer service knowledge base.

[0090] Finally, all generated customer service knowledge documents are written to the customer service knowledge base. These documents can be indexed with customer service stage clusters and customer service knowledge sub-clusters, enabling subsequent searches to be filtered by stage or located by sub-cluster. The customer service knowledge base provides an external query interface, supporting searches by keywords, semantic vectors, or stage / sub-clusters, providing knowledge support for online customer service.

[0091] In this embodiment, a first clustering method is used to divide customer service into multiple customer service stages based on all existing standard consultation texts, including pre-sales consultation, sales logistics, after-sales returns and exchanges, accounts and payments, and coupons and activities. A second clustering method further subdivides each stage into customer service knowledge sub-clusters such as "refund only" and "exchange process" based on specific business scenarios, thus constructing a hierarchical structure of "customer service stage - customer service knowledge sub-cluster". This structure not only fully aligns the knowledge document classification system with the actual customer service process, but also achieves automated organization from fragmented dialogue to systematic knowledge by embedding human customer service text and its corresponding consultation text set into a structured prompt template and generating standardized documents with four levels of headings using a large language model. The generated documents retain the business details and polite tone of human responses and are compatible with images and tables through Markdown format, allowing direct use in online help centers or customer service robot training, significantly reducing manual compilation costs. More importantly, when business scenarios expand, incremental updates can be performed by repeating the above process for newly clustered sub-clusters, without requiring a complete database rebuild, demonstrating high engineering feasibility.

[0092] In a further embodiment, after storing all customer service knowledge documents in the customer service knowledge base, step S1900 includes the following steps:

[0093] Step S2000: Obtain the target consultation text in the chat session triggered by the user's consultation with customer service, determine the matching confidence between the target consultation text and each consultation text set in the customer service knowledge base, and filter out the one with the highest matching confidence.

[0094] E-commerce platforms offer customer service to buyers, allowing them to consult with customer service for answers and assistance. Buyers typically need to contact customer service due to issues such as disputes over product quality, abnormal order status, logistics delays, unclear or untimely return and exchange policies, or ambiguous or unclear promotional rules.

[0095] E-commerce platforms can provide multiple customer service entry points, allowing users to contact customer service for assistance. These entry points can be a "Contact Customer Service" button on the page, a help center page, etc. When a user initiates a customer service inquiry, a chat session is created for communication between the user and customer service.

[0096] During the process of users consulting customer service, users send their consultation text to the chat session. It's understandable that users often use informal, colloquial, context-omitted, or even typo-filled expressions in their consultation texts, leading to a deviation between the consultation intent and the user's actual intended message. In other words, the consultation intent in the consultation text may be inaccurate. Therefore, the accuracy of the consultation intent in the consultation text is assessed. If accurate, the consultation text is confirmed as the target consultation text; if inaccurate, the consultation text is rewritten to correctly express the consultation intent, and then used as the target consultation text. Those skilled in the art can flexibly adapt and implement the assessment of the accuracy of the consultation intent in the consultation text, as well as the correction and rewriting of consultation texts with inaccurate intent.

[0097] In a further embodiment, determining the matching confidence between the target consultation text and each consultation text set in the customer service knowledge base includes the following steps:

[0098] Step S2001: For each consultation text set in the customer service knowledge base, calculate the semantic similarity between each consultation text in the consultation text set and the target consultation text.

[0099] A pre-trained text semantic model, which has reached a convergence state, can be used to encode each consultation text in the consultation text set and the target consultation text, respectively, to obtain the semantic corresponding vectorized representation of each consultation text, namely the text feature vector. This vector represents the corresponding text semantics in a high-dimensional dense vector.

[0100] Since the text semantic model has pre-learned to represent the semantics of the input text as high-dimensional dense vectors and map them to the semantic space, and the more similar different input texts are semantically, the closer the high-dimensional dense vectors corresponding to these texts are in the semantic space, the vector distance algorithm can be used to calculate the vector distance between the text feature vectors of each consultation text in the consultation text set and the text feature vector of the target consultation text, which can be used as the semantic similarity between each consultation text in the consultation text set and the target consultation text.

[0101] Step S2002: Select the maximum value among all semantic similarities as the matching confidence score.

[0102] From the semantic similarity between each consultation text in the consultation text set and the target consultation text, the highest semantic similarity can be selected. This takes into account the user's personalized expression and avoids underestimating the semantic similarity. Therefore, this semantic similarity is used as the matching confidence between the target consultation text and the consultation text set.

[0103] Step S2010: When the maximum matching confidence score meets the determination condition that represents the same consultation intent, obtain the customer service reply text in the customer service knowledge base that is associated with the consultation text set corresponding to the maximum matching confidence score and matches the target consultation text.

[0104] A confidence threshold can be pre-configured to measure whether the maximum match confidence is high enough to ensure that the consultation intent of the corresponding target consultation text is extremely close to, or even identical to, the consultation intent of the corresponding consultation text set, although there may be some differences in expression. Those skilled in the art can further set the confidence threshold as needed based on this disclosure to ensure the accuracy of the answer, for example, 0.9.

[0105] When the maximum matching confidence exceeds the confidence threshold, it is confirmed that the maximum matching confidence meets the judgment condition. The customer service response set associated with the consultation text set corresponding to the maximum matching confidence in the customer service knowledge base is determined. Then, the customer service response set and the consultation text set corresponding to the maximum matching confidence are embedded into a preset question-and-answer matching prompt template. After obtaining the question-and-answer matching prompt text, it guides the large language model to infer the customer service response text in the customer service response text set that is most similar in expression style to the target consultation text and can accurately answer the target consultation text.

[0106] Step S2020: When the maximum matching confidence does not meet the judgment condition, generate a corresponding customer service reply text based on the target customer service knowledge document in the customer service knowledge base that matches the target inquiry text;

[0107] When the maximum match confidence does not exceed the confidence threshold, it confirms that the maximum match confidence at this point meets the judgment criteria. It also confirms that the consultation intent between the target consultation text and all consultation text sets in the customer service knowledge base has a significant offset. This is understandable, as the customer service knowledge structure within the customer service knowledge documents in the customer service knowledge base differs from the customer service knowledge structure associated with consultation text sets and customer service response sets. It provides more comprehensive contextual information and more detailed knowledge content. Therefore, relevant knowledge applicable to the current customer service response can be found from all customer service knowledge documents in the customer service knowledge base. Then, by organizing the found relevant knowledge linguistically, the customer service response text can be obtained.

[0108] Step S2030: Send the customer service reply text to the chat session to reply to the target inquiry text.

[0109] By calling the session communication interface provided by the e-commerce platform, the matched or generated customer service reply text is sent to the currently active chat session, thereby completing the closed loop of responding to the target inquiry text.

[0110] In this embodiment, a dual-path closed-loop mechanism of "matching first, then generating" is implemented in the overall real-time response process: when the maximum matching confidence of the user's inquiry text with any inquiry text set in the customer service knowledge base is higher than the confidence threshold, the verified and usable customer service reply text is directly used to achieve millisecond-level accurate response; when the maximum matching confidence is insufficient, the system automatically switches to the generation path based on customer service knowledge documents, using a large language model to instantly construct a reply that matches the user's expression style and contains the required answer. This "confidence gating" not only avoids the coverage blind spots of a customer service system built on a single retrieval model when facing novel or colloquial expressions, but also prevents the illusion risk that may occur in a purely generative customer service system. More importantly, using the dialogue record as a dynamic context ensures that the model can fully understand the current user's intended inquiry when generating a reply, thereby outputting a reply that conforms to business rules and fits the personalized scenario. It is evident that using a static customer service knowledge base to dynamically support customer service replies in real dialogue contexts significantly improves the adaptability to personalized expressions of inquiry intent and the robustness to long-tail problems.

[0111] In a further embodiment, step S2020, generating a corresponding customer service response text based on the target customer service knowledge document in the customer service knowledge base that matches the target inquiry text, includes the following steps:

[0112] Step S2021: Perform word-granular matching and global matching between the target consultation text and each customer service knowledge document in the customer service knowledge base, and determine the corresponding word-granular matching score and global matching score;

[0113] The word-granular matching score can be implemented as follows: first, an inverted index is built for the customer service knowledge document, and then the BM25 algorithm is used to retrieve the matching score with the target consultation text. The global matching score can be implemented as follows: the target consultation text and the customer service knowledge document are respectively fed into the same pre-trained and converged text semantic model to obtain 512-dimensional or 768-dimensional vectors, and then the cosine similarity is calculated; if the customer service knowledge document is extremely large, approximate vector retrieval (FAISS, Milvus) can be used to quickly recall candidate documents first, and then the precise vector similarity can be calculated.

[0114] Step S2022: Based on the word granularity matching score and the global matching score, retrieve the corresponding first matching sequence and second matching sequence from the customer service knowledge base. Each sequence includes multiple customer service knowledge documents arranged in reverse order according to the corresponding score.

[0115] After determining the word-granularity matching score and the global matching score, the customer service knowledge documents in the customer service knowledge base are sorted in reverse order according to their respective scores from highest to lowest. Multiple knowledge documents corresponding to the reversed order are then recalled to form a first matching sequence (based on word-granularity matching score) and a second matching sequence (based on global matching score). For example, if the customer service knowledge base contains 10,000 documents, the 30 documents with the highest word-granularity matching scores constitute the first matching sequence, and the 30 documents with the highest global matching scores constitute the second matching sequence. The same customer service knowledge document may appear repeatedly in both sequences, but the order may differ. The number of recalled knowledge documents can be set as needed by those skilled in the art.

[0116] Step S2023: For each recalled customer service knowledge document, the inverse ranking fusion algorithm is used to determine the document recall score based on the ranking of the customer service knowledge document in the first matching sequence and the second matching sequence respectively.

[0117] For each recalled customer service knowledge document, a reciprocal ranking fusion algorithm is used to calculate the document recall score. An illustrative example of the algorithm formula is shown below:

[0118]

[0119] Where: score is the document recall score of a single recalled knowledge document, K is an adjustment constant usually taken as 60, and Rank1 and Rank2 correspond to the ranking of the customer service knowledge document in the first matching sequence and the second matching sequence.

[0120] Step S2024: Recall the target customer service knowledge documents in the customer service knowledge base that meet the preset conditions for document recall scoring;

[0121] The preset conditions can be either "retriev the top K documents with the highest document recall scores" or "retriev all documents with document recall scores exceeding a threshold T". For example, if Top-K = 3, then the three customer service knowledge documents with the highest document recall scores will be retrieved and concatenated into the target customer service knowledge document in descending order of score; if the threshold T = 0.75, then all customer service knowledge documents with a document recall score ≥ 0.75 will be retrieved and concatenated into the target customer service knowledge document in descending order of score.

[0122] Step S2025: Obtain the dialogue record corresponding to the current moment in the chat session, embed the dialogue record and the target document content in the target customer service knowledge document into a preset reply prompt template, and obtain the reply prompt text;

[0123] It is understandable that at the current moment, the last text sent in the chat session is the user's inquiry text. At this time, it is necessary to reply to the user's inquiry text as a customer service representative. In order to prevent the large language model from misunderstanding the user's inquiry text due to lack of context, based on the sending timestamp of the user's inquiry text, the chat records in the chat session before the sending timestamp, including all corresponding texts sent between the user and the customer service representative, are retrieved.

[0124] Since the input text length of the large language model is limited, the total number of characters in the target document content is calculated by subtracting the total number of characters in the response prompt template, the target consultation text, the dialogue record, and the estimated maximum number of characters in the model's output from the calculated upper limit of the input character count. If the total number of characters in the target customer service knowledge document exceeds the total number of characters in the target document content, the target document content with an equivalent total number of characters in the target customer service knowledge document is extracted. If the total number of characters in the target customer service knowledge document does not exceed the total number of characters in the target document content, no truncation is required, and all content in the target customer service knowledge document is directly used as the target document content.

[0125] As can be seen, the dialogue log is used to provide the complete contextual dependencies required for this round of customer service responses. The target document content is derived from the target customer service knowledge document, serving as an authoritative information source and providing factual basis for the model's output.

[0126] Response prompt templates typically include a fixed format, such as: "Based on the following authoritative information: {context}, please answer the user's question: {question}. Chat history: {chat_history}. Answers must avoid subjective speculation." Of course, those skilled in the art can further configure the response prompt template as needed based on this disclosure. Thus, by embedding the target document content into the {context} slot, the target consultation text into the {question} slot, and the chat history into the {chat_history} slot, a structured and context-complete response prompt text can be obtained.

[0127] Step S2026: Use a large language model to generate corresponding customer service reply text based on the reply prompt text.

[0128] Guided by the response prompt text, the large language model extracts content suitable for answering the target inquiry from the target document, and then organizes this content linguistically to output the response text. The response text output by the large language model is then used as the customer service response text.

[0129] In this embodiment, a "dual-layer retrieval + fusion ranking + context generation" technology is implemented on the customer service knowledge document side to achieve a refined connection between recall and generation, ultimately generating accurate customer service response text. First, parallel recall is achieved through BM25 word-granularity matching and global vector similarity, simultaneously capturing both keyword precision and semantic generalization relevance. Then, a reciprocal ranking fusion algorithm is introduced to merge the two heterogeneous scores into a single document recall score, avoiding the tedious manual weight tuning while ensuring both relevances are considered to guarantee document recall accuracy. Second, considering the limited input length of large language models in practical applications, a character budget-driven adaptive truncation strategy is proposed to ensure that the dialogue record, target consultation text, and target document content are presented completely and evenly in the prompt template, preventing information loss or misunderstanding by the large language model due to improper truncation. Finally, by treating the recalled target document knowledge as authoritative material and embedding it along with the dialogue record in the prompt template, the model can strictly adhere to the document facts while maintaining contextual coherence when generating responses, thus achieving fast and reliable automatic generation of customer service responses to user inquiries in high-concurrency customer service scenarios.

[0130] Please see Figure 3 This application provides a customer service knowledge base preparation device, which is a functional embodiment of the customer service knowledge base preparation method of this application. On another note, this device, also provided to fulfill one of the objectives of this application, includes a consultation matching module 1100, a response set construction module 1200, a response generation module 1300, and a response storage module 1400. The consultation matching module 1100 is used to determine the consultation matching degree between user consultation texts in various chat dialogue pairs stored in the chat database and various consultation text sets stored in the consultation knowledge database. Each consultation text set includes a standard consultation text and at least one similar consultation text with the same consultation intent. The response set construction module 1200 is used to, for each consultation text set… The system identifies user consultation texts whose consultation matching degree with the consultation text set exceeds the dynamic intent matching threshold, adds the corresponding human customer service text in the chat dialogue pair to the human response set, and the dynamic intent matching threshold is determined based on the difficulty of expressing the consultation intent corresponding to the consultation text set; the response generation module 1300 is used to remove human customer service texts with similar expressions in the human response set, and uses a large language model to generate a customer service response set corresponding to the consultation text set based on the removed human response set and the consultation text set; the response storage module 1400 is used to remove customer service response texts in the customer service response set whose consultation answer credibility does not meet the standard with the consultation text set, and stores the removed customer service response set as associated with the consultation text set in the customer service knowledge base.

[0131] In a further embodiment, before the response set construction module 1200, the module includes: a similarity determination submodule, used to determine the semantic similarity between the standard consultation text in the consultation text set and each similar consultation text, and to calculate the average semantic similarity based on all semantic similarities; a coefficient determination submodule, used to determine the corresponding expression difficulty coefficient based on the standard consultation text in the consultation text set using a preset language expression model; and a threshold determination submodule, used to determine the dynamic intent matching threshold based on the expression difficulty coefficient and the average semantic similarity.

[0132] In a further embodiment, before the coefficient determination submodule, the module includes: a training set acquisition submodule, used to acquire a training set, the training set including multiple training samples and their supervision labels, the training samples being standard consultation intentions, and the supervision labels representing the difficulty level of a user expressing the consultation intention of a standard consultation text; and a model training submodule, used to apply the training set to train the language expression model to a convergent state, thereby learning the ability to determine the expression difficulty coefficient corresponding to a user's consultation intention in expressing the input consultation text.

[0133] In a further embodiment, after the reply storage module 1400, the module includes: a first clustering module, used to perform a first clustering on the standard consultation texts in all consultation text sets in the consultation knowledge base to determine multiple customer service stage clusters; a cluster determination module, used to determine the customer service stage cluster to which the user consultation texts in each chat dialogue pair stored in the chat database belong; a second clustering module, used to perform a second clustering on the corresponding human customer service texts in the chat dialogue pair for each customer service stage cluster to determine multiple customer service knowledge sub-clusters; a document generation module, used to generate a corresponding customer service knowledge document for each customer service knowledge sub-cluster based on the human customer service texts belonging to that customer service knowledge sub-cluster and the consultation text set associated with its corresponding customer service reply set; and a document storage module, used to store all customer service knowledge documents in the customer service knowledge base.

[0134] In a further embodiment, after the document storage module, the module includes: a maximum filtering submodule, used to obtain the target consultation text in a chat session triggered by a user's consultation with customer service, determine the matching confidence between the target consultation text and each consultation text set in the customer service knowledge base, and filter out the maximum matching confidence; a reply invocation submodule, used to obtain, when the maximum matching confidence satisfies the judgment condition representing the same consultation intent, the customer service reply text associated with the consultation text set corresponding to the maximum matching confidence and matching the target consultation text in the customer service knowledge base; a reply generation submodule, used to generate a corresponding customer service reply text based on the target customer service knowledge document matching the target consultation text in the customer service knowledge base when the maximum matching confidence does not satisfy the judgment condition; and a reply sending submodule, used to send the customer service reply text to the chat session to reply to the target consultation text.

[0135] In a further embodiment, the maximum filtering submodule includes: a semantic comparison unit, used to calculate the semantic similarity between each consultation text in the consultation text set in the customer service knowledge base and the target consultation text; and a maximum filtering unit, used to filter out the maximum value among all semantic similarities as the matching confidence score.

[0136] In a further embodiment, the response generation submodule includes: a binary matching unit, used to perform word-granular matching and global matching between the target consultation text and each customer service knowledge document in the customer service knowledge base, and determine the corresponding word-granular matching score and global matching score; a sequence recall unit, used to recall the corresponding first matching sequence and second matching sequence from the customer service knowledge base according to the word-granular matching score and global matching score, each sequence including multiple customer service knowledge documents in reverse order of their corresponding scores; a score determination unit, used to determine the document recall score for each recalled customer service knowledge document by using a reciprocal ranking fusion algorithm based on the ranking of the customer service knowledge document in the first matching sequence and the second matching sequence; a document recall unit, used to recall the target customer service knowledge document in the customer service knowledge base whose document recall score meets a preset condition; a prompt construction unit, used to obtain the dialogue record corresponding to the current moment in the chat session, embed the dialogue record and the target document content in the target customer service knowledge document into a preset response prompt template, and obtain the response prompt text; and a response generation unit, used to generate the corresponding customer service response text using a large language model based on the response prompt text.

[0137] To address the aforementioned technical problems, embodiments of this application also provide computer equipment. For example... Figure 4The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store a sequence of control information. When the computer-readable instructions are executed by the processor, the processor can implement a customer service knowledge base preparation method. The processor of the computer device provides computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute the customer service knowledge base preparation method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0138] In this embodiment, the processor is used to execute... Figure 3 The specific functions of each module and its sub-modules are defined within the system. The memory stores the program code and various data required to execute the aforementioned modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / sub-modules in the customer service knowledge base preparation device of this application. The server can call the server's program code and data to execute the functions of all sub-modules.

[0139] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the customer service knowledge base preparation method of any embodiment of this application.

[0140] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0141] In summary, this application enables precise knowledge extraction that adapts to the difficulty of intent expression, document processing with a two-layer knowledge structure of customer service stage and knowledge subclass, and real-time response generation with dual matching fusion of word granularity and global scope, significantly improving the consultation coverage, answer accuracy, and intelligent response efficiency of the customer service knowledge base in complex intent scenarios.

[0142] Those skilled in the art will understand that the steps, measures, and solutions in the various operations, methods, and processes discussed in this application can be alternated, modified, combined, or deleted. Furthermore, other steps, measures, and solutions in the various operations, methods, and processes discussed in this application can also be alternated, modified, rearranged, decomposed, combined, or deleted. Furthermore, steps, measures, and solutions in the prior art that are similar to those in the open-source operations, methods, and processes of this application can also be alternated, modified, rearranged, decomposed, combined, or deleted.

[0143] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for preparing a customer service knowledge base, characterized in that, Includes the following steps: Determine the consultation matching degree between the user consultation text in each chat dialogue pair stored in the chat database and the consultation text set stored in the consultation knowledge database. The consultation text set includes a standard consultation text and at least one similar consultation text with the same consultation intent. For each set of consultation texts, identify user consultation texts whose consultation matching degree with the consultation text set exceeds the dynamic intent matching threshold, and add the corresponding human customer service text in the chat dialogue pair to the human response set. The dynamic intent matching threshold is determined based on the difficulty of expressing the consultation intent corresponding to the consultation text set. Remove similar-sounding customer service texts from the set of manual replies, and use a large language model to generate a customer service reply set corresponding to the set of consultation texts based on the removed set of manual replies and the set of consultation texts. Remove customer service reply texts that do not meet the credibility standard between the customer service reply set and the consultation text set, and store the customer service reply set after removal, along with the consultation text set, in the customer service knowledge base.

2. The customer service knowledge base preparation method according to claim 1, characterized in that, Before identifying user consultation texts that match the dynamic intent matching threshold for each consultation text set, the following steps are included: Determine the semantic similarity between the standard consultation text and each similar consultation text in the consultation text set, and calculate the mean semantic similarity based on all semantic similarities; A pre-defined language expression model is used to determine the corresponding expression difficulty coefficient based on the standard consultation texts in the consultation text set; The dynamic intent matching threshold is determined based on the expression difficulty coefficient and the mean semantic similarity.

3. The customer service knowledge base preparation method according to claim 2, characterized in that, Before determining the corresponding expression difficulty coefficient based on the standard consultation texts in the consultation text set using a pre-set language expression model, the following steps are included: Obtain a training set, which includes multiple training samples and their supervision labels. The training samples represent standard consultation intentions, and the supervision labels characterize the ease or difficulty with which a user expresses the consultation intention of a standard consultation text. The language expression model is trained to convergence using the training set, and learns the ability to determine the expression difficulty coefficient corresponding to the user's consultation intent in the input consultation text.

4. The method for preparing a customer service knowledge base according to claim 1, characterized in that, After storing the removed customer service response set along with the consultation text set in the customer service knowledge base, the following steps are included: The first clustering of standard consultation texts in all consultation text sets in the consultation knowledge base was performed to identify multiple customer service stage clusters; Identify the customer service stage cluster to which the user inquiry text in each chat dialogue pair stored in the chat database belongs; For each customer service stage cluster, a second clustering is performed on the corresponding human customer service text in the chat dialogue for all user inquiry texts belonging to that customer service stage cluster, to identify multiple customer service knowledge sub-clusters. For each customer service knowledge subcategory, a corresponding customer service knowledge document is generated based on the human customer service text belonging to that subcategory and the consultation text set associated with its corresponding customer service response set. All customer service knowledge documents are stored in the aforementioned customer service knowledge base.

5. The customer service knowledge base preparation method according to claim 4, characterized in that, After storing all customer service knowledge documents in the customer service knowledge base, the following steps are included: Obtain the target consultation text from the chat session triggered by the user's consultation with customer service, determine the matching confidence between the target consultation text and each consultation text set in the customer service knowledge base, and filter out the one with the highest matching confidence. When the maximum matching confidence score meets the judgment condition that represents the same consultation intent, the customer service reply text that is associated with the consultation text set corresponding to the maximum matching confidence score and matches the target consultation text is obtained from the customer service knowledge base; When the maximum matching confidence does not meet the judgment condition, a corresponding customer service response text is generated based on the target customer service knowledge document in the customer service knowledge base that matches the target inquiry text. The customer service reply text is sent to the chat session to respond to the target inquiry text.

6. The customer service knowledge base preparation method according to claim 5, characterized in that, Determining the matching confidence between the target consultation text and each consultation text set in the customer service knowledge base includes the following steps: For each set of consultation texts in the customer service knowledge base, calculate the semantic similarity between each consultation text in the set and the target consultation text. The maximum value among all semantic similarities is selected as the matching confidence score.

7. The customer service knowledge base preparation method according to claim 5, characterized in that, Based on the target customer service knowledge document in the customer service knowledge base that matches the target inquiry text, a corresponding customer service response text is generated, including the following steps: The target consultation text is matched with each customer service knowledge document in the customer service knowledge base using both granular and global matching methods to determine the corresponding granular matching score and global matching score. Based on the word granularity matching score and the global matching score, the corresponding first matching sequence and second matching sequence are retrieved from the customer service knowledge base. Each sequence includes multiple customer service knowledge documents arranged in reverse order according to the corresponding scores. For each recalled customer service knowledge document, a reciprocal ranking fusion algorithm is used to determine the document recall score based on the ranking of the customer service knowledge document in the first matching sequence and the second matching sequence, respectively. Recall the target customer service knowledge documents in the customer service knowledge base whose document recall scores meet preset conditions; Obtain the dialogue record corresponding to the current moment in the chat session, embed the dialogue record and the target document content in the target customer service knowledge document into a preset reply prompt template, and obtain the reply prompt text; A large language model is used to generate corresponding customer service response text based on the response prompt text.

8. A customer service knowledge base preparation device, characterized in that, include: The consultation matching module is used to determine the consultation matching degree between the user consultation text in each chat dialogue pair stored in the chat database and the consultation text set stored in the consultation knowledge database. The consultation text set includes a standard consultation text and at least one similar consultation text with the same consultation intent. The response set construction module is used to identify user consultation texts whose consultation matching degree with the consultation text set exceeds the dynamic intent matching threshold for each consultation text set, and add the corresponding human customer service text in the chat dialogue pair to the human response set. The dynamic intent matching threshold is determined based on the difficulty of expressing the consultation intent corresponding to the consultation text set. The response generation module is used to remove similar-sounding customer service texts from the set of manual responses, and to generate a customer service response set corresponding to the consultation text set using a large language model based on the removed set of manual responses and the consultation text set. The reply storage module is used to remove customer service reply texts that do not meet the credibility standards of the consultation answers between the customer service reply set and the consultation text set, and store the customer service reply set after removal as a related consultation text set in the customer service knowledge base.

9. A computer device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 7, which, when invoked by a computer, executes the steps included in the corresponding method.

Images