Customer service knowledge base optimization method based on multi-dimensional evaluation and related device

Abstract

The application relates to the technical field of knowledge base management, and discloses a customer service knowledge base optimization method based on multidimensional evaluation and related equipment. The method comprises the following steps: collecting response interaction data and evaluation data of a customer service system, wherein the response interaction data comprises system answer content and corresponding knowledge base entries; based on the evaluation data and quality evaluation dimensions, calculating the scores of each dimension of the system answer and obtaining a comprehensive quality score by weighting; screening out answers and corresponding knowledge base entries with a comprehensive score lower than a preset threshold as to-be-optimized entries, and determining a defect category according to the scores of each dimension; extracting correction information corresponding to the to-be-optimized entries from the evaluation data, and calculating the semantic difference degree between the correction information and the original content; when the semantic difference degree exceeds a preset threshold, generating a correction scheme according to the defect category and updating the knowledge base. The application realizes the continuous automatic optimization and accurate response of the content of the customer service knowledge base.

Description

Technical Field

[0001] This invention relates to the field of knowledge base management technology, and in particular to a method and related equipment for optimizing customer service knowledge bases based on multidimensional evaluation. Background Technology

[0002] With the deepening application of artificial intelligence technology in the enterprise service sector, intelligent customer service systems have become an important tool for improving service efficiency and reducing operating costs. Existing customer service systems generally rely on pre-built knowledge bases to provide answers to user questions. These knowledge bases are typically manually compiled by business personnel based on product documents, service manuals, and frequently asked questions. The quality of the knowledge base content directly determines the service level of the customer service system, thus requiring continuous maintenance and updates to adapt to dynamic factors such as product iterations, policy changes, and evolving user needs. However, the traditional knowledge base maintenance model primarily uses manual editing. Business personnel need to periodically check the accuracy of knowledge entries, manually correct errors based on user complaints or work order records, and add new knowledge entries based on experience. This maintenance method has a significant lag; from discovering a problem to completing the correction often takes weeks or even months, resulting in a long-term disconnect between the knowledge base content and actual business conditions.

[0003] While some existing systems incorporate user satisfaction rating features, allowing users to rate or comment on customer service responses, the utilization of this feedback data is extremely limited. These systems lack structured analysis capabilities for feedback, failing to transform scattered user opinions into specific knowledge correction instructions; a large amount of valuable feedback data is merely presented as statistical reports. Furthermore, existing systems have vague criteria for evaluating response quality, unable to quantify key indicators such as the accuracy and completeness of knowledge items, and struggling to identify which content urgently needs updating. Although some systems employ machine learning models for question-and-answer matching, these models also face performance degradation issues when the training data becomes outdated, and lack a mechanism for automatic adjustment based on online performance. Therefore, establishing a systematic mechanism that enables the knowledge base to continuously identify defects and automatically optimize content based on actual service data, while ensuring the accuracy and reliability of the optimization process, has become a pressing technical problem in the customer service system field. Summary of the Invention

[0004] The main objective of this invention is to solve the problem that existing customer service knowledge bases lack an automated optimization mechanism, resulting in the inability to update and correct knowledge content in a timely manner based on actual usage data.

[0005] The first aspect of this invention provides a customer service knowledge base optimization method based on multidimensional evaluation. The method includes: collecting response interaction data and evaluation data from a customer service system, wherein the response interaction data at least includes system answer content and corresponding knowledge base entries; calculating the sub-item scores for each dimension of the system answer content based on the evaluation data and preset quality evaluation dimensions, and weighting the sub-item scores to obtain a comprehensive quality score; selecting system answer content with a comprehensive quality score lower than a preset threshold and matching the corresponding knowledge base entries as items to be optimized based on the knowledge base entries, and determining the defect category corresponding to the items to be optimized according to the sub-item scores; extracting correction information corresponding to each item to be optimized from the evaluation data, and calculating the semantic difference between the correction information and the original content corresponding to the item to be optimized; when the semantic difference exceeds a preset difference threshold, generating a correction scheme according to the defect category, and updating the content to be optimized in the customer service knowledge base according to the correction scheme.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the quality assessment dimensions include at least a content factual dimension, a semantic matching degree dimension, and a language fluency dimension. The step of calculating the sub-item scores for each dimension of the system's response content based on the evaluation data and the preset quality assessment dimensions, and then weighting the sub-item scores to obtain a comprehensive quality score, includes: mapping the satisfaction score, answer credibility score, and quality review score in the evaluation data to a unified scoring interval to obtain normalized score values; selecting evaluation data for the corresponding dimension from the normalized score values ​​according to the quality assessment dimensions, and calculating the content factual sub-item scores, semantic matching degree sub-item scores, and language fluency sub-item scores respectively; and performing a weighted summation operation on the sub-item scores of each dimension and their corresponding weight coefficients based on the preset weight coefficients of each quality assessment dimension to obtain a comprehensive quality score.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the step of filtering out system answer content with a comprehensive quality score lower than a preset threshold and matching the corresponding knowledge base entries as items to be optimized based on the knowledge base entries, and determining the defect category corresponding to the item to be optimized based on the sub-item scores of each dimension, includes: filtering out system answer content with a comprehensive quality score lower than a preset threshold, extracting the knowledge base entries called when generating the system answer content from the response interaction data, and marking the knowledge base entries as items to be optimized; extracting the sub-item scores of each dimension of the corresponding system answer content based on each item to be optimized, comparing the numerical values ​​of the sub-item scores of each dimension, and taking the evaluation dimension corresponding to the sub-item score with the lowest numerical value as the defect category of the item to be optimized, wherein the defect category includes at least the content factual defect category, the semantic matching degree defect category, and the language fluency defect category.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of extracting correction information corresponding to each item to be optimized from the evaluation data and calculating the semantic difference between the correction information and the original content corresponding to the item to be optimized includes: extracting text feedback content corresponding to each item to be optimized from user evaluation information in the evaluation data, and using the correction content provided by the user in the text feedback content as the correction information of the item to be optimized; performing semantic vector encoding on the correction information and the original content corresponding to the item to be optimized respectively to obtain a semantic vector of the correction information and a semantic vector of the original content; calculating the cosine distance between the semantic vector of the correction information and the semantic vector of the original content to obtain the semantic difference, wherein the semantic difference is used to characterize the degree of change of the correction information relative to the original content.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of generating a correction scheme based on the defect category when the semantic difference exceeds a preset difference threshold includes: determining whether the semantic difference exceeds the preset difference threshold; when it exceeds the threshold, determining a corresponding optimization instruction based on the defect category of the item to be optimized; generating a factual content replacement instruction when the defect category is a content factual defect category; generating a context association optimization instruction when the defect category is a semantic matching defect category; generating a language reorganization instruction when the defect category is a language fluency defect category; combining the optimization instruction, the original content of the item to be optimized, and the correction information to generate a prompt text; generating corrected knowledge item content based on the prompt text input, and associating and storing the corrected knowledge item content with the corresponding knowledge base item identifier to form a correction scheme.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the step of combining the optimization instruction, the original content of the item to be optimized, and the correction information to generate a prompt text includes: formatting the optimization instruction into instruction description text, formatting the original content of the item to be optimized into reference content text, and formatting the correction information into correction requirement text; arranging the instruction description text, reference content text, and correction requirement text in a preset combination order, and inserting preset separators between each text; merging the arranged text to generate a prompt text.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, updating the content to be optimized in the customer service knowledge base according to the correction scheme includes: obtaining the corrected knowledge entry content and the knowledge base entry identifier corresponding to the content to be optimized from the correction scheme; locating the target entry in the customer service knowledge base according to the knowledge base entry identifier; replacing the original content of the target entry with the corrected knowledge entry content; and updating the revision timestamp and version number of the target entry to complete the knowledge base update.

[0012] A second aspect of the present invention provides a customer service knowledge base optimization device based on multidimensional evaluation. The device includes: a data acquisition module for acquiring response interaction data and evaluation data from a customer service system, wherein the response interaction data at least includes system answer content and corresponding knowledge base entries; a quality evaluation module for calculating the sub-scores of each dimension of the system answer content based on the evaluation data and preset quality evaluation dimensions, and performing a weighted calculation of the sub-scores to obtain a comprehensive quality score; a defect diagnosis module for filtering system answer content with a comprehensive quality score lower than a preset threshold and matching the corresponding knowledge base entries as items to be optimized based on the knowledge base entries, and determining the defect category corresponding to the items to be optimized according to the sub-scores of each dimension; a difference analysis module for extracting correction information corresponding to each item to be optimized from the evaluation data and calculating the semantic difference degree between the correction information and the original content corresponding to the item to be optimized; and a content optimization module for generating a correction scheme according to the defect category when the semantic difference degree exceeds a preset difference threshold, and updating the content to be optimized in the customer service knowledge base according to the correction scheme.

[0013] A third aspect of the present invention provides a customer service knowledge base optimization device based on multidimensional evaluation, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the customer service knowledge base optimization device based on multidimensional evaluation to perform the various steps of the customer service knowledge base optimization method based on multidimensional evaluation described above.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the steps of the above-described customer service knowledge base optimization method based on multidimensional evaluation.

[0015] The above-described customer service knowledge base optimization method and related equipment are based on multi-dimensional evaluation. In this embodiment, response interaction data and evaluation data from the customer service system are collected. The response interaction data includes system answer content and corresponding knowledge base entries. Based on the evaluation data and preset quality evaluation dimensions, the scores for each dimension of the system answer content are calculated, and a weighted average of these scores is used to obtain a comprehensive quality score. System answer content with a comprehensive quality score below a preset threshold is selected from the knowledge base entries, and the corresponding knowledge base entries are matched as items to be optimized. The defect category corresponding to the item to be optimized is determined based on the scores for each dimension. Correction information corresponding to each item to be optimized is extracted from the evaluation data, and the semantic difference between the correction information and the original content corresponding to the item to be optimized is calculated. When the semantic difference exceeds a preset difference threshold, a correction scheme is generated based on the defect category, and the content to be optimized in the customer service knowledge base is updated according to the correction scheme. This application, by constructing a multi-dimensional quality assessment system and a defect classification and diagnosis mechanism, achieves precise positioning and targeted repair of knowledge base problems. It solves the technical problem that the lack of an automated optimization mechanism in the existing customer service knowledge base leads to the inability to update and correct knowledge content in a timely manner based on actual usage data. This effectively improves the dynamic adaptability and content accuracy of the knowledge base, ensuring that the customer service system can continuously provide high-quality user services.

[0016] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0017] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the first embodiment of the customer service knowledge base optimization method based on multidimensional evaluation in this invention. Figure 2 This is a schematic diagram of one embodiment of the customer service knowledge base optimization device based on multidimensional evaluation in this invention. Figure 3 This is a schematic diagram of an embodiment of the customer service knowledge base optimization device based on multi-dimensional evaluation in this invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] The terms "comprising" and "having," and any variations thereof, used in the embodiments of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0021] To facilitate understanding of this embodiment, the specific process of this embodiment is described below. Please refer to [link / reference]. Figure 1 The first embodiment of the customer service knowledge base optimization method based on multidimensional evaluation in this invention includes: 101. Collect response interaction data and evaluation data from the customer service system. The response interaction data shall at least include the system's answer content and the corresponding knowledge base entries. The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0022] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0023] In this embodiment, when the customer service system responds to user inquiries, it records the user-inputted question text, the system-generated answer content, and the knowledge base entries called to generate the answer content; it collects the user's satisfaction rating and text feedback information regarding the answer content; it calls a language model to perform self-evaluation of the answer content, calculates the answer credibility score, and obtains the quality review score submitted by human reviewers regarding the answer content; it associates and stores the question text, answer content, and knowledge base entries as response interaction data, and associates and stores the satisfaction rating, text feedback information, answer credibility score, and quality review score as evaluation data.

[0024] In practical applications, when a user submits an inquiry through the customer service interface, the front-end service module of the customer service system captures the user's input text in real time and transmits it to the question-and-answer matching engine via a natural language processing interface. The engine searches the knowledge base based on the semantic features of the question text, locates the knowledge base entry with the highest matching degree, and extracts the answer content from that entry, returning it to the user. A knowledge base entry refers to a structured data unit stored in the knowledge base, containing fields such as entry identifier, title, answer content, and applicable scenario. The system associates and records the question text, answer content, and the identifier of the called knowledge base entry in a MySQL database. For example, if a user asks "How long does it take for a refund to arrive?", the system calls the knowledge base entry numbered FAQ_001 to generate the answer "3-5 business days." The system stores these three pieces of information as a single record as response interaction data. This ensures that each answer is traceable to a specific knowledge source, providing a data foundation for subsequent question entry location. After showing the answer to the user, the customer service interface simultaneously displays satisfaction rating options and a text feedback input box. Users can click the "Satisfied," "Partially Satisfied," or "Dissatisfied" buttons to submit a satisfaction rating, and the system maps these options to numerical values ​​for storage. Users can also add comments in the text feedback input box, such as entering "VIP customers are available 24 hours a day." The system collects user-submitted satisfaction ratings and text feedback information in real time through a front-end event listening mechanism, and establishes a connection between the unique identifier of the response interaction record and the corresponding response interaction data. This instant feedback mechanism directly obtains users' true evaluation of the answer quality. Furthermore, when generating an answer, the system calls a language model based on the Transformer architecture to perform self-evaluation of the answer content. The language model analyzes the semantic completeness and logical coherence of the answer content, outputting an answer credibility score, a value between 0 and 1, representing the model's confidence in the quality of its generated content. In addition, the system pushes some response interaction data to a human review queue, where reviewers view the question text and answer content through the review interface, assess its accuracy, and submit a quality review score. Introducing both model self-evaluation and human review creates a more comprehensive quality evaluation system. The system then stores the above data according to type. Question text, answer content, and knowledge base entry identifiers are stored as response interaction data in the Response Interaction Data Table, while satisfaction ratings, text feedback information, answer credibility scores, and quality review scores are stored as evaluation data in the Evaluation Data Table. The two tables are linked by the unique identifiers of the response interaction records. This categorized storage structure facilitates subsequent data retrieval and analysis according to different dimensions.

[0025] 102. Based on the evaluation data and the preset quality assessment dimensions, calculate the sub-item scores of each dimension of the system's response content, and perform weighted calculation of each sub-item score to obtain the comprehensive quality score; In this embodiment, the quality assessment dimensions include at least the content factuality dimension, semantic matching degree dimension, and language fluency dimension. Before mapping the satisfaction score, answer credibility score, and quality review score in the evaluation data to a unified scoring range, the method further includes: performing anomaly detection on the evaluation data to identify whether there are abnormal feedback patterns; abnormal feedback patterns include at least the following: a single user submitting more than a preset number of evaluations within a preset time window, multiple evaluations having a text content similarity exceeding a preset similarity threshold, or the proportion of extreme scores exceeding a preset proportion threshold within a preset time period; when an abnormal feedback pattern is detected, a weight reduction flag is set for the evaluation data that triggers the anomaly, and a weight reduction coefficient is applied to the evaluation data marked with the weight reduction flag in subsequent weighted calculations; when no abnormal feedback pattern is detected, the original weight coefficients of the evaluation data are maintained. Then, the satisfaction score, answer credibility score, and quality review score in the evaluation data are mapped to a unified scoring range to obtain normalized score values. Based on the quality assessment dimensions, evaluation data for the corresponding dimensions are selected from the normalized score values ​​to calculate the content factual sub-score, semantic matching sub-score, and language fluency sub-score. (The calculation of the content factual sub-score, semantic matching sub-score, and language fluency sub-score based on the quality assessment dimensions includes: extracting the answer credibility score and quality review score from the normalized score values, performing factual verification on the system's answer content to obtain a factual detection score, and then summing the factual detection score, answer credibility score, and quality review score.) The following steps are performed: First, a weighted sum is applied to obtain the content factual sub-score. Second, semantic vector encoding and cosine similarity calculation are performed on the question text and system response content in the response interaction data to obtain an objective matching score. A satisfaction score is extracted from the normalized score as a subjective evaluation score. The objective matching score and subjective evaluation score are then weighted and summed to obtain the semantic matching score. Third, grammar detection and coherence detection scores are calculated for the system response content. A quality review score is extracted from the normalized score. The grammar detection score, coherence detection score, and quality review score are then weighted and summed to obtain the language fluency score. Finally, based on the preset weight coefficients for each quality assessment dimension, a weighted summation of the sub-scores for each dimension with their corresponding weight coefficients is performed to obtain the comprehensive quality score.

[0026] In practical applications, before conducting quality assessments on the evaluation data, an anomaly detection process is first executed to identify abnormal feedback patterns in the evaluation data. Abnormal feedback patterns refer to data patterns that do not conform to normal user feedback behavior characteristics, including malicious review manipulation, false feedback submitted by automated tools, or concentrated abnormal behavior within a short period. Three detection rules are used to identify abnormal feedback patterns. The first is user frequency detection: the system extracts the evaluation submission records corresponding to each user identifier from the evaluation data table and counts the number of evaluations submitted by a single user within a preset time window. The preset time window is a time range set by the system to count user behavior frequency, for example, 60 minutes. When the number of evaluations submitted by a user within this time window exceeds a preset threshold, the user's evaluation record is marked as abnormal. For example, the preset threshold can be set to 10 times; if a user submits 15 evaluations within one hour, anomaly detection is triggered. The second is text similarity detection: text feedback information is extracted from multiple evaluations, and the Sentence-BERT model is used to encode the text feedback information into semantic vectors, calculating the cosine similarity between the vectors. Cosine similarity is a numerical value that measures the similarity between two vectors in terms of direction, ranging from 0 to 1. A higher value indicates greater similarity in the text content. When the text content similarity of multiple reviews exceeds a preset similarity threshold, these reviews are marked as abnormal. For example, if the preset similarity threshold is set to 0.85, and three reviews with the text feedback messages "The answer is inaccurate and needs updating," "The answer is inaccurate and needs modification," and "The answer is inaccurate and needs updating," all have a cosine similarity exceeding 0.85, then they are identified as an abnormal feedback pattern. The third method is extreme rating distribution detection, which counts the number of extreme ratings and the total number of reviews within a preset time period. Extreme ratings refer to the highest or lowest satisfaction ratings. The preset time period is, for example, 24 hours. The system calculates the proportion of extreme ratings to the total number of reviews. When this proportion exceeds a preset proportion threshold, anomaly detection is triggered. For example, if the preset proportion threshold is set to 0.7, and 100 reviews are received on a certain day, with 75 being the lowest ratings (a proportion of 0.75), exceeding the threshold, then the review data within that time period is marked as having an abnormal feedback pattern. This multi-rule anomaly detection mechanism can identify false feedback from three perspectives: behavior frequency, content characteristics, and distribution patterns, thus preventing malicious data from interfering with subsequent quality assessments.

[0027] When the system detects an abnormal feedback pattern, it sets a weight reduction flag for the triggered evaluation data. The weight reduction flag is a marker field added to the evaluation data table for abnormal evaluation records, used to identify these records during subsequent weighted calculations. The system adds a weight coefficient field to each triggered evaluation record in the evaluation data table and sets the value of this field to the weight reduction coefficient. The weight reduction coefficient is a value less than 1, for example, set to 0.3, indicating that the influence weight of this evaluation data in subsequent calculations is reduced. For example, the satisfaction rating of an evaluation originally had a weight coefficient of 1.0, but after triggering the abnormality, its weight coefficient changes to 0.3. When the system does not detect an abnormal feedback pattern, it maintains the original weight coefficients of the evaluation data, for example, the weight coefficient of ordinary users remains 1.0. This differentiated weighting ensures that the evaluation results are based on genuine and valid user feedback, avoiding malicious review manipulation that could lead to incorrect knowledge base optimization. Furthermore, mapping operations are performed on the satisfaction rating, answer credibility score, and quality review score in the evaluation data, transforming the rating data from different sources to a unified rating range to obtain normalized rating values. Different rating sources have different numerical ranges. For example, the original numerical range for satisfaction ratings is 0.2 to 1.0, for answer credibility scores it's 0 to 1.0, and for quality review scores it's 0 to 10. A linear mapping algorithm transforms these different ranges of rating values ​​into a unified standard range of 0 to 1. The linear mapping algorithm calculates the relative position of each original rating value within its original range and then maps it to the 0 to 1 range using the same relative position. For example, if the original value of the quality review score is 8, its relative position in the 0 to 10 range is 0.8, and its normalized rating value after mapping is 0.8. After this mapping process, all rating values ​​in the evaluation data are converted into normalized rating values ​​between 0 and 1, eliminating the dimensional differences between different rating sources. Then, based on the quality assessment dimensions, corresponding evaluation data are selected from the normalized rating values ​​to calculate the three sub-item scores. The quality assessment dimensions are different evaluation angles used to assess the quality of the system's answer content, including the content factuality dimension, semantic matching degree dimension, and language fluency dimension. The content factual dimension assesses whether the information stated in the answer is accurate and reliable; the semantic matching dimension assesses whether the answer truly answers the user's question; and the language fluency dimension assesses whether the expression of the answer is smooth and natural. The calculation of the content factual sub-score first extracts the answer credibility score and quality review score from the normalized score values. Then, the system's answer content undergoes factual verification to obtain a factual detection score. Factual verification is the process of comparing and verifying the factual statements in the system's answer content with authoritative data sources.The system uses a named entity recognition algorithm to extract key factual information, such as time, numerical values, and policy names, from the system's responses. This information is then matched against authoritative data sources within the enterprise, such as product document databases and policy rule bases. A higher factual detection score is awarded when the factual information in the response matches the authoritative data source; a lower score is awarded when there are discrepancies. For example, if the system's response is "Refunds will arrive in 3-5 business days," the system extracts the time entity "3-5 business days" and compares it with the latest refund policy in the policy rule base. If the policy rule base records a refund timeframe of "3-5 business days," the factual detection score is 0.95; if the policy rule base has updated the refund timeframe to "within 24 hours," the factual detection score is 0.3. The system multiplies the factual detection score, response credibility score, and quality review score by their respective weighting coefficients and then sums them to obtain the content factual score. The weighting coefficients are pre-set based on the reliability of each scoring source. For example, the weighting coefficient for factual detection scores is 0.5, for answer credibility scores it is 0.3, and for quality review scores it is 0.2. This multi-source fusion scoring method combines the advantages of automatic detection and human evaluation, improving the reliability of the assessment of the factual accuracy of the answer content. For the semantic matching score, the system performs semantic vector encoding and cosine similarity calculation on the question text and system answer content in the response interaction data to obtain an objective matching score. The system uses a BERT pre-trained language model to encode the question text and system answer content separately, obtaining two semantic vectors. Then, it calculates the cosine similarity between the two semantic vectors to obtain the objective matching score. For example, if the question text is "How long does it take for a VIP member refund to arrive?" and the system answer is "A VIP member refund will arrive within 24 hours", the cosine similarity of their semantic vectors is 0.92, meaning the objective matching score is 0.92. The system extracts a satisfaction score from the normalized score as a subjective evaluation score, which reflects the user's intuitive feeling about the quality of the answer. The system multiplies the objective matching score and subjective evaluation score by their respective weighting coefficients and then sums them to obtain the semantic matching score. For example, the weighting coefficient for the objective matching score is set to 0.6, and the weighting coefficient for the subjective evaluation score is set to 0.4. Combining objective semantic calculation and subjective user evaluation, the system can comprehensively assess whether the answer truly solves the user's actual problem. For the language fluency score, the system calculates the grammar detection score and coherence detection score for the system's answer. The grammar detection score is obtained by using a grammar analysis tool to detect grammatical errors in the answer. The system uses the LanguageTool grammar checker to scan the answer, identifying grammatical errors, spelling errors, punctuation errors, etc., and calculates the grammar detection score based on the number and severity of errors.When the answer is grammatically correct, the grammar detection score is 1.0; when grammatical errors are found, a corresponding score is deducted for each error. The coherence detection score assesses the logical coherence and semantic fluency of the answer. The system segments the answer into sentences, calculates the semantic relevance between adjacent sentences, uses sentence-level semantic vectors to calculate the cosine similarity of adjacent sentence vectors, and then averages the similarities of all adjacent sentence pairs to obtain the coherence detection score. A quality review score is extracted from the normalized score. The grammar detection score, coherence detection score, and quality review score are multiplied by their respective weighting coefficients and then summed to obtain the language fluency sub-score. For example, the weighting coefficient for the grammar detection score is 0.4, the weighting coefficient for the coherence detection score is 0.3, and the weighting coefficient for the quality review score is 0.3. This fine-grained language quality assessment can accurately identify deficiencies in the expression of the answer.

[0028] Then, based on the preset weight coefficients of each quality assessment dimension, the scores of each dimension are weighted and summed with their corresponding weight coefficients to obtain a comprehensive quality score. The preset weight coefficients reflect the importance of different quality assessment dimensions in the overall quality evaluation and are set according to the actual needs of customer service operations. For example, in a financial customer service scenario where information accuracy is emphasized, the weight coefficient for the factual content dimension is set to 0.5, the weight coefficient for the semantic matching dimension is set to 0.3, and the weight coefficient for the language fluency dimension is set to 0.2. The system multiplies the factual content score by 0.5, the semantic matching score by 0.3, and the language fluency score by 0.2, and then adds the three products to obtain the comprehensive quality score. The comprehensive quality score integrates the evaluation results of multiple assessment dimensions, providing a quantitative basis for subsequently identifying low-quality answers and locating knowledge base defects, solving the problems of vague evaluation standards and the inability to quantify the quality of knowledge items in traditional systems. The above-described comprehensive quality score calculation method based on preset weight coefficients is a basic implementation method that can meet the needs of most application scenarios. In some application scenarios where higher scoring discrimination is required, this invention also provides an optional method for optimizing and determining weight coefficients to further improve the overall quality score's ability to identify different items to be evaluated. This method optimizes and adjusts preset weight coefficients by analyzing the statistical characteristics of actual evaluation data, making the weight allocation more reasonable. It should be noted that this optimization method is an optional technical solution, and those skilled in the art can choose whether to adopt it based on actual business needs and system resources. The optional embodiment is described in detail below. In this optional embodiment, an adaptive weight adjustment algorithm is used to optimize the preset weight coefficients. The core idea of ​​this algorithm is: by statistically analyzing the dispersion of the sub-scores of each quality evaluation dimension in a batch of items to be evaluated, the evaluation dimensions with stronger discrimination ability are identified, and their weight coefficients are increased accordingly, thereby making the overall quality score more discriminative. Specifically, when the score of a certain dimension differs significantly among different items (high standard deviation), it indicates that the dimension can effectively distinguish between high-quality and low-quality items, and the weight of the dimension is automatically increased; conversely, when the scores of a certain dimension are generally similar (low standard deviation), it indicates that the dimension has limited discriminative effect in the current dataset, and the system correspondingly reduces its weight. The formula for calculating the overall quality score using the adaptive weight adjustment algorithm is as follows: The formula for calculating the dynamic weighting coefficient is: , in, For the overall quality score, Score the sub-items of the i-th dimension (i=1 represents factual content, i=2 represents semantic matching degree, and i=3 represents language fluency). Let i be the dynamic weight coefficient of the i-th dimension. This represents the basic weight coefficient (preset value) for the i-th dimension. Let β be the standard deviation of the score for the i-th dimension across all items to be evaluated, β be the dispersion sensitivity coefficient ranging from 0.1 to 0.5, and n be the total number of quality assessment dimensions. For example: A customer service system generates 100 response records within 24 hours, and it is necessary to calculate the overall quality score of one of these records. Known conditions include: the three dimension scores for this record are: factual content S1 = 0.65, semantic matching S2 = 0.82, and language fluency S3 = 0.78; the preset basic weight coefficients are: α1 = 0.5, α2 = 0.3, α3 = 0.2; by statistically analyzing 100 records, the standard deviations of the scores for each dimension are calculated as: σ1 = 0.25, σ2 = 0.12, σ3 = 0.08; the dispersion sensitivity coefficient is set to β = 0.3. The calculation process is as follows: First, calculate the weight adjustment factors for each dimension: Dimension 1: α1×(1+β×σ1) = 0.5×(1+0.3×0.25) = 0.5375, Dimension 2: α2×(1+β×σ2) = 0.3×(1+0.3×0.12) = 0.3108, Dimension 3: α3×(1+β×σ3) = 0.2×(1+0.3×0.08) = 0.2048; Second, normalize to obtain the dynamic weight coefficients: Sum = 0.5375 + 0.3108 + 0.2048 = 1.0531, w1 = 0.5375 / 1.0531 = 0.510, w2 = 0.3108 / 1.0531 = 0.295, w3 = 0.2048 / 1.0531 = 0.195; Recalculate the overall quality score: Q total = 0.510 × 0.65 + 0.295 × 0.82 + 0.195 × 0.78 = 0.726. The adaptive weighting algorithm and calculation have the following technical advantages: 1. Enhanced dynamic adaptability: By introducing the standard deviation σ... iAs a weight adjustment factor, the algorithm can automatically identify evaluation dimensions with high discriminative power in practical applications. When the score of a certain dimension differs significantly among different items (high standard deviation), it indicates that the dimension is more effective in distinguishing between high-quality and low-quality items. The system automatically increases the weight of this dimension, making the overall score more discriminative. 2. Avoiding evaluation blind spots: Traditional fixed-weight methods may lose the evaluation value of a dimension due to the general convergence of scores. This algorithm ensures effective discriminative power across all dimensions by monitoring the dispersion of each dimension in real time, avoiding blind spots in the evaluation system. 3. Adaptive to business scenarios: The importance of each quality dimension may differ in different business scenarios. For example, in financial customer service scenarios, the dispersion of factual content is usually high, and the system will automatically increase its weight; while in consulting customer service scenarios, the dispersion of semantic matching may be more significant, and the system will adjust the weight allocation accordingly to achieve automatic adaptation to different business characteristics. Through the above adaptive weight coefficient optimization method, the system can dynamically adjust the specific value of the weight coefficient according to the statistical characteristics of the actual evaluation data, making the weight allocation more consistent with the feature distribution of the current dataset, further improving the accuracy and discriminative power of the overall quality score. Those skilled in the art should understand that this adaptive optimization method is an optional technical solution, primarily applicable to application scenarios with high requirements for scoring accuracy and sufficient computing resources. In standard application scenarios, directly using the aforementioned preset weight coefficients for weighted calculation is sufficient to meet the needs of knowledge base optimization. In actual deployment, system administrators can select whether to enable this optimization function by configuring parameters based on business characteristics, data scale, and computing resource availability.

[0029] 103. Based on knowledge base entries, filter out system answer content with a comprehensive quality score lower than a preset threshold and match the corresponding knowledge base entries as entries to be optimized, and determine the defect category corresponding to the entries to be optimized based on the sub-item scores of each dimension. In this embodiment, system response content with a comprehensive quality score lower than a preset threshold is selected, and knowledge base entries called when generating system response content are extracted from response interaction data. These knowledge base entries are marked as items to be optimized. Based on each item to be optimized, the sub-item scores of each dimension of the corresponding system response content are extracted, and the numerical values ​​of each sub-item score are compared. The evaluation dimension corresponding to the sub-item score with the lowest value is taken as the defect category of the item to be optimized. The defect category includes at least the content factual defect category, the semantic matching degree defect category, and the language fluency defect category.

[0030] In practical applications, the system filters out system responses with scores below a preset threshold from the overall quality score data and traces the corresponding knowledge base entries for these low-quality responses. The preset threshold is a minimum acceptable value for the overall quality score set in advance by the system, used to distinguish between qualified responses and those requiring optimization. By traversing all response interaction records with calculated overall quality scores, the system compares the overall quality score of each record with the preset threshold, filtering out records with overall quality scores below the preset threshold. For example, if the overall quality score of a response interaction record is below the preset threshold, that record is filtered out. The system extracts the knowledge base entry identifiers corresponding to these low-quality responses from the response interaction data table and locates the specific knowledge base entry based on the identifier. The knowledge base entry identifier is a unique number for each entry in the knowledge base, such as FAQ_001. The system marks these located knowledge base entries as entries to be optimized by adding a status field to the knowledge base data table and setting the value of the status field to "to be optimized". For example, the knowledge base entry numbered FAQ_001 is marked as an entry to be optimized because the overall quality score of the corresponding system response content is below the preset threshold. This step enables precise tracing from low-quality answers to the knowledge source of the question, ensuring that optimization work directly affects entries with genuine defects in the knowledge base, avoiding unnecessary modifications to qualified entries. Then, based on each entry to be optimized, the system extracts sub-scores for each dimension of the corresponding system answer content, and determines the defect category by comparing the numerical values ​​of these sub-scores. The system queries the system answer content corresponding to each entry to be optimized from the scoring data table, extracting the content factuality sub-score, semantic matching sub-score, and language fluency sub-score. For example, the three dimension sub-scores for the system answer content corresponding to a certain entry to be optimized are low content factuality, medium semantic matching, and high language fluency. The system compares the numerical values ​​of these three sub-scores and identifies the lowest sub-score. In the example above, the content factuality sub-score is the minimum of the three. The system uses the evaluation dimension corresponding to the lowest sub-score as the defect category of the entry to be optimized. In this example, the lowest sub-score corresponds to the content factuality dimension, therefore the defect category of the entry to be optimized is determined to be the content factuality defect category. Defect categories refer to the specific types of quality problems present in the items to be optimized, including at least content factual defects, semantic matching defects, and language fluency defects. Content factual defects indicate that the information in the knowledge base item contains factual errors or is outdated; semantic matching defects indicate that the content of the knowledge base item deviates from the user's actual problem; and language fluency defects indicate that the expression of the knowledge base item is not clear or natural enough. The identified defect categories are then associated with the corresponding items to be optimized and stored in the knowledge base data table.By using this defect classification method based on itemized scoring, the system can not only identify problematic knowledge base entries, but also accurately locate the root cause of the problem. This enables subsequent corrective work to address the actual defects in a targeted manner, significantly improving the accuracy and effectiveness of knowledge base optimization.

[0031] 104. Extract the correction information corresponding to each item to be optimized from the evaluation data, and calculate the semantic difference between the correction information and the original content corresponding to the item to be optimized; In this embodiment, text feedback content corresponding to each item to be optimized is extracted from user evaluation information in the evaluation data, and the correction content provided by the user in the text feedback content is used as the correction information of the item to be optimized; semantic vector encoding is performed on the correction information and the original content corresponding to the item to be optimized to obtain the semantic vector of the correction information and the semantic vector of the original content; the cosine distance between the semantic vector of the correction information and the semantic vector of the original content is calculated to obtain the semantic difference degree, wherein the semantic difference degree is used to characterize the degree of change of the correction information relative to the original content.

[0032] In practical applications, text feedback content corresponding to each item to be optimized is extracted from the evaluation data table, and user-provided corrections are identified as correction information. Based on the identifier of the item to be optimized, all evaluation records corresponding to that item are queried in the evaluation data table, and the content of the text feedback information field is extracted from these records. Text feedback information consists of supplementary opinions or modification suggestions filled in by users in the text input boxes on the evaluation interface. Text analysis algorithms are used to identify corrective content within the text feedback information. Corrective content refers to statements where users explicitly point out errors in the original answer and provide correct information. For example, if the system answer for an item to be optimized is "Refund arrives in 3-5 business days," and the user enters "VIP customers receive it 24 hours" in the text feedback, the system identifies "VIP customers receive it 24 hours" as corrective content and extracts it as correction information for that item to be optimized. Correction information refers to specific suggestions for improvement or supplementary information made by users regarding the system answer. This step directly obtains improvement suggestions from real users, providing a clear direction for the optimization of knowledge base items. Then, semantic vector encoding is performed on the correction information and the original content corresponding to the item to be optimized, resulting in semantic vectors for the correction information and the original content. Semantic vector encoding is the process of converting text into high-dimensional numerical vectors that represent the semantic features of the text. By calling the BERT pre-trained language model, the corrected information text is input into the model for encoding, and the model outputs a fixed-dimensional numerical vector as the corrected information semantic vector. Similarly, the system inputs the original content corresponding to the item to be optimized into the BERT model for encoding, obtaining the original content semantic vector. The original content refers to the answer text stored in the knowledge base for the item to be optimized. For example, if the corrected information is "VIP customers are available 24 hours," and the original content is "Refunds arrive in 3-5 business days," encoding these two texts separately yields two semantic vectors. By using a unified encoding method to convert the text into a computable numerical representation, it becomes possible to quantify and compare the semantic differences between the two texts. Furthermore, the cosine distance between the corrected information semantic vector and the original content semantic vector is calculated to obtain the semantic difference degree. The cosine distance is a mathematical measure of the angular difference between two vectors, ranging from 0 to 2; a larger value indicates a greater directional difference between the two vectors. By taking the semantic vectors of the corrected information and the original content as input, the cosine similarity between the two vectors is calculated. Then, the cosine distance, or semantic dissimilarity, is obtained by subtracting the cosine similarity from 1. The semantic dissimilarity characterizes the degree of change of the corrected information relative to the original content. The larger the semantic dissimilarity, the more significant the semantic difference between the corrected information and the original content. For example, the corrected information "VIP customers receive refunds within 24 hours" and the original content "Refunds arrive in 3-5 business days" show a significant difference in refund timeliness, resulting in a larger calculated semantic dissimilarity value.By quantifying the degree of difference between the corrected information and the original content, it is possible to determine whether substantive modifications to the knowledge base entries are necessary, thus avoiding over-optimization of minor differences.

[0033] The aforementioned semantic difference calculation method based on cosine distance is a basic implementation method, which can effectively measure the degree of difference between the corrected information and the original content in terms of overall semantics. In some application scenarios with higher requirements for the accuracy of semantic difference measurement, especially those requiring precise differentiation between expression optimization and factual changes, this invention also provides an optional multi-level semantic difference measurement method. This method, based on the basic cosine distance calculation, adds difference analysis at the key entity level. By integrating the difference information from both levels, it can more accurately identify substantive content modifications. It should be noted that this enhancement method is an optional technical solution, and those skilled in the art can choose whether to adopt it based on actual accuracy requirements and computing resources. In most application scenarios, the basic cosine distance calculation method is sufficient to meet the needs of semantic difference measurement. The optional embodiment is described in detail below.

[0034] In this optional embodiment, a multi-level semantic difference measurement algorithm is employed. To further improve the accuracy of semantic difference measurement, the system introduces difference analysis at the key entity level in addition to calculating cosine distance. The core idea of ​​this method is that cosine distance can reflect the overall semantic similarity, but it may not be sensitive enough to changes in key factual information; while the difference of key entities can directly reflect changes in core information elements. By integrating the difference information at both levels, the system can more accurately determine whether the revised information contains substantial content changes. Specifically, the system first uses a named entity recognition algorithm to extract key entities from the revised information and the original content respectively. Key entities refer to keyword groups in the text that express core information, such as time expressions ("3-5 working days", "24 hours"), object limitations ("VIP members", "ordinary users"), numerical information, etc. The system calculates the intersection and union of the two entity sets, measures the similarity at the entity level using the Jaccard similarity coefficient, and subtracts the similarity coefficient from 1 to obtain the difference at the entity level. The comprehensive semantic difference calculation formula of the multi-level semantic difference measurement algorithm is as follows: The formula for calculating semantic dissimilarity at the vector level is: , The formula for calculating the difference at the entity level is: , in, To measure the semantic difference, the value ranges from 0 to 1. This represents the semantic dissimilarity (cosine distance) at the vector level. For the degree of difference at the entity level, To correct the semantic vector of information This is the semantic vector of the original content. To correct the set of key entities extracted from the information, The set of key entities extracted from the original content; λ is the balance coefficient, ranging from 0.6 to 0.8, and 0.7 in this embodiment; |·| is the modulus of the vector or the number of elements in the set; ∩ is the intersection operation of sets; ∪ is the union operation of sets. For example: the original content of an item to be optimized is "[Refund will arrive in 3-5 business days]", and the user-provided correction information is "[VIP members will receive their refund within 24 hours]". The semantic difference between the two needs to be calculated. The known conditions are: the semantic vector (simplified to 3-dimensional) obtained through the BERT model encoding: It equals [0.82, 0.65, 0.43]. The values ​​are [0.71, 0.58, 0.39]. Key entities extracted using named entity recognition are: This equates to a refund, which takes 3-5 business days. Equal to {VIP member, 24 hours}, the balance coefficient λ=0.7, and its calculation process is as follows: First, calculate the semantic difference at the vector level: vector dot product: · =0.71×0.82 + 0.58×0.65 + 0.39×0.43 = 1.1269, vector magnitude: | | = √(0.82²+0.65²+0.43²) = 1.1068, vector magnitude: | = √(0.71²+0.58²+0.39²) = 0.9899, ​​cosine similarity = 1.1269 / (1.1068×0.9899) = 0.9373, = 1 - 0.9373 = 0.0627; Next, calculate the difference at the entity level: intersection: ∩ = The union of elements with zero is: ∪ = {Refund, 3-5 business days, VIP members, 24 hours}, with 4 elements. = 1 - 0 / 4 = 1.0, and then calculate the comprehensive semantic dissimilarity: = 0.7×0.0627 + 0.3×1.0 = 0.0439 + 0.3000 = 0.3439. This multi-level semantic difference measurement algorithm has the following technical advantages: 1. Accurate identification of substantive modifications: By combining vector-level and entity-level measurements, the algorithm can accurately distinguish between minor adjustments in expression and substantive changes in factual content. In the above embodiment, although the vector similarity is high (0.9373), because the key entities are completely different (refund timeout changed from [3-5 working days] to [24 hours], and the applicable target group was expanded to [VIP members]), the entity-level difference reached 1.0, and the overall difference was finally 0.3439, accurately reflecting that the corrected information contained important new information, triggering the knowledge base update process. 2. Reduced false trigger rate: Traditional methods based solely on vector similarity are prone to misjudging synonymous rewriting as substantive modifications. This algorithm uses entity-level auxiliary judgment. When users only optimize the expression while keeping key information entities consistent, the entity difference is close to 0, and the overall difference remains at a low level. This avoids unnecessary update operations on essentially identical content, reducing system computational overhead. 3. Adjustable sensitivity: By setting the balance coefficient λ, the system can adjust its sensitivity to semantic fine-tuning and entity changes according to business needs. When λ is larger (e.g., 0.8), the system pays more attention to changes in overall semantics, suitable for scenarios with high requirements for expression; when λ is smaller (e.g., 0.6), the system pays more attention to changes in key facts, suitable for fields with extremely high accuracy requirements such as finance and healthcare, achieving flexible configuration capabilities for the algorithm. Through the above multi-level semantic difference measurement method, the system can comprehensively consider the overall semantic similarity and the consistency of key information elements, more accurately identifying the substantive modifications that truly need to trigger knowledge base updates, while avoiding misjudging user feedback involving only expression optimization as content changes requiring updates. Those skilled in the art should understand that this multi-level measurement method is an optional technical solution, mainly suitable for application scenarios with high requirements for semantic difference judgment accuracy. In standard application scenarios, the basic cosine distance calculation method can already provide sufficient accuracy for difference measurement. In actual deployment, system administrators can choose whether to enable this enhanced feature through configuration parameters, based on business characteristics and accuracy requirements. When this feature is enabled, the system needs to additionally call the named entity recognition module, which increases computational overhead.

[0035] 105. When the semantic difference exceeds the preset difference threshold, generate a correction plan based on the defect category, and update the content to be optimized in the customer service knowledge base according to the correction plan.

[0036] In this embodiment, it is determined whether the semantic difference exceeds a preset difference threshold. When it exceeds the threshold, the corresponding optimization instruction is determined according to the defect category of the item to be optimized. When the defect category is a content factual defect category, a factual content replacement instruction is generated. When the defect category is a semantic matching defect category, a context association optimization instruction is generated. When the defect category is a language fluency defect category, a language reorganization instruction is generated. The optimization instruction, the original content of the item to be optimized, and the correction information are combined to generate a prompt text (wherein, combining the optimization instruction, the original content of the item to be optimized, and the correction information to generate the prompt text includes: formatting the optimization instruction into instruction description text, formatting the original content of the item to be optimized into reference content text, and formatting the correction information into...). The information is formatted as a correction requirement text; the instruction description text, reference content text, and correction requirement text are arranged in a preset combination order, and preset separators are inserted between each text; the arranged texts are merged to generate a prompt text; based on the prompt text input, the corrected knowledge entry content is generated, and the corrected knowledge entry content is associated with the corresponding knowledge base entry identifier and stored to form a correction scheme; the corrected knowledge entry content and the knowledge base entry identifier corresponding to the content to be optimized are obtained from the correction scheme; the target entry in the customer service knowledge base is located according to the knowledge base entry identifier, the original content of the target entry is replaced with the corrected knowledge entry content, and the revision timestamp and version number of the target entry are updated to complete the knowledge base update. In addition, after updating the corresponding entries in the customer service knowledge base according to the correction plan, the following steps are also included: continuously collecting newly generated response interaction data and evaluation data during the continuous operation of the customer service system; when the amount of data corresponding to the response interaction data and evaluation data reaches a preset quantity threshold or the time since the last optimization process reaches a preset interval, the following operations are repeatedly performed on the response interaction data and evaluation data: quality assessment, comprehensive score calculation, defect classification, correction information extraction, semantic difference calculation, correction plan generation, and knowledge base update, forming a continuous loop knowledge base optimization mechanism.

[0037] In practical applications, the semantic difference is determined by whether it exceeds a preset difference threshold. When it does, the corresponding optimization instruction is determined based on the defect category of the item to be optimized. The preset difference threshold is a critical value pre-set by the system to determine whether the corrected information needs to trigger knowledge base optimization. The system compares the calculated semantic difference value with the preset difference threshold. When the semantic difference value is greater than the preset difference threshold, it is determined that the corrected information has a substantial difference from the original content and knowledge base optimization is required. The system reads the identified defect category of the item to be optimized from the database and generates corresponding optimization instructions based on different defect categories. Optimization instructions are guiding text that directs the language model to generate corrected content in a specific direction. When the defect category is a factual content defect, a factual content replacement instruction is generated, which requires the language model to update outdated or incorrect factual information. For example, if the defect category of an item to be optimized is a factual content defect, the generated factual content replacement instruction would be "Please update the factual information in the following content according to the latest policy." When the defect category is a semantic matching defect, a contextual association optimization instruction is generated, which requires the language model to improve the relevance between the answer content and the user's question. For example, the system-generated contextual optimization instruction is "Please optimize the following content to more accurately answer the user's actual question." When the defect category is language fluency defect, the system generates a language reorganization instruction, which requires the language model to improve the expression of the answer content. For example, the system-generated language reorganization instruction is "Please reorganize the language expression of the following content to make it clearer and more fluent." By generating targeted optimization instructions based on the defect category, the system can accurately guide the direction of subsequent content generation, ensuring that the correction work directly solves the specific problem type identified. Then, the optimization instruction, the original content of the item to be optimized, and the correction information are combined to generate prompt text. First, the three types of content are formatted separately. The optimization instruction is formatted into instruction description text. The formatting process adds marker text before and after the optimization instruction. For example, adding "[Task Requirements]" before the optimization instruction forms "[Task Requirements] Please update the factual information in the following content according to the latest policy." Then, the original content of the item to be optimized is formatted into reference content text. The formatting process adds marker text before and after the original content. For example, adding "[Original Content]" before the original content forms "[Original Content] Refund will arrive in 3-5 business days." The system formats the correction information into correction request text. The formatting process involves adding marker text before and after the correction information; for example, adding "[User Feedback]" before the correction information creates "[User Feedback] VIP customers are available 24 hours a day." Then, the system arranges the instruction description text, reference content text, and correction request text in a preset order. This preset order is a pre-determined text arrangement to ensure the language model can understand the task logically.The system inserts preset separators between text segments. These separators are special characters or line breaks used to distinguish different text paragraphs; for example, double line breaks are used as separators. The arranged text is then merged to generate prompt text. This prompt text is a structured text containing complete task instructions, reference information, and correction requirements, used as input to the language model for content generation. For example, the generated prompt text might be: "[Task Requirements] Please update the factual information in the following content according to the latest policy\n\n[Original Content] Refunds will arrive in 3-5 business days\n\n[User Feedback] VIP customers receive it within 24 hours." This structured prompt text generation method ensures that the language model can clearly understand the task objective, existing content, and correction direction, improving the accuracy and relevance of the generated content. Based on the prompt text input, corrected knowledge entry content is generated and associated with the corresponding knowledge base entry identifier, forming a correction scheme. The generated prompt text is then input into a large language model based on the GPT architecture. The language model generates corrected knowledge entry content that meets the requirements based on the task requirements, original content, and user feedback in the prompt text. The corrected knowledge entry content is the optimized answer text output by the language model. For example, based on the above prompt text, the language model generates the corrected knowledge entry content as "VIP member refunds will arrive within 24 hours, and regular member refunds will arrive within 3-5 business days." The corrected knowledge entry content is then associated with the knowledge base entry identifier corresponding to the entry to be optimized and stored in the correction scheme data table. The correction scheme is a data record containing the knowledge base entry identifier and the corrected knowledge entry content, used for subsequent knowledge base update operations. Through the language model's generation capabilities, the system can automatically generate high-quality content that meets the correction requirements, significantly reducing the workload of manual writing.

[0038] The system then retrieves the corrected knowledge entry content and the corresponding knowledge base entry identifier for the content to be optimized from the correction scheme data table. By reading records from the correction scheme data table, the system extracts the knowledge base entry identifier field and the corrected knowledge entry content field from each record. Based on the knowledge base entry identifier, the system locates the target entry in the customer service knowledge base. The knowledge base entry identifier is a unique number for each entry in the knowledge base. The system searches the knowledge base data table for records matching the knowledge base entry identifier to locate the target entry that needs to be updated. The target entry refers to the specific entry record in the knowledge base that needs to be replaced by the corrected content. The system then replaces the original content of the target entry with the corrected knowledge entry content. The replacement operation updates the data in the answer content field of the target entry record with the corrected knowledge entry content. This updates the revision timestamp and version number of the target entry. The revision timestamp records the last modification time of the entry, which the system updates to the current system time. The version number is a numerical value that identifies the number of times the entry has been revised; the system increments the original version number by 1. For example, when the system locates the target entry numbered FAQ_001, it changes the answer from "Refunds will arrive in 3-5 business days" to "VIP member refunds will arrive within 24 hours, and regular member refunds will arrive in 3-5 business days," updates the revision timestamp to the current time, and updates the version number, thus completing the knowledge base update. Updating the revision timestamp and version number allows the system to track the evolution history of knowledge base entries, facilitating subsequent version management and issue tracing.

[0039] Secondly, after updating the corresponding entries in the customer service knowledge base according to the correction plan, the system continuously collects newly generated response interaction data and evaluation data during the ongoing operation of the customer service system. The system maintains its data collection operation, monitors the interaction process between the customer service system and users in real time, and stores each new question-and-answer interaction and user evaluation information in the corresponding data table. The system sets two trigger conditions to initiate a new round of optimization. The first trigger condition is that the amount of response interaction data and evaluation data reaches a preset threshold, which is a pre-set standard for data accumulation. The system counts the total number of new response interaction data and evaluation data records since the last optimization process; when the total reaches the preset threshold, the optimization process is triggered. The second trigger condition is that the time since the last optimization process reaches a preset interval, which is a pre-set optimization cycle length. The system calculates the time difference between the current system time and the last optimization time; when the time difference reaches the preset interval, the optimization process is triggered. When either trigger condition is met, the system repeatedly performs quality assessment, comprehensive score calculation, defect classification, correction information extraction, semantic difference calculation, correction plan generation, and knowledge base update operations on the newly collected response interaction data and evaluation data. Following the complete process outlined above, the system processes new data, identifies new items requiring optimization, generates new correction schemes, and updates the knowledge base content. This continuous, iterative knowledge base optimization mechanism enables the system to constantly improve itself as business develops and user needs change, transforming the knowledge base from static maintenance to dynamic evolution, ensuring that the customer service system always provides services based on the latest and most accurate knowledge.

[0040] In this embodiment of the invention, response interaction data and evaluation data from a customer service system are collected. The response interaction data includes system answer content and corresponding knowledge base entries. Based on the evaluation data and preset quality assessment dimensions, scores for each dimension of the system answer content are calculated, and a weighted average of these scores is used to obtain a comprehensive quality score. System answer content with a comprehensive quality score below a preset threshold is selected from the knowledge base entries, and the corresponding knowledge base entries are matched as items to be optimized. The defect category corresponding to each item to be optimized is determined based on the scores for each dimension. Correction information corresponding to each item to be optimized is extracted from the evaluation data, and the semantic difference between the correction information and the original content corresponding to the item to be optimized is calculated. When the semantic difference exceeds a preset difference threshold, a correction scheme is generated based on the defect category, and the content to be optimized in the customer service knowledge base is updated according to the correction scheme. This application, by constructing a multi-dimensional quality assessment system and a defect classification and diagnosis mechanism, achieves accurate positioning and targeted repair of knowledge base problems. It solves the technical problem that the lack of an automated optimization mechanism in existing customer service knowledge bases leads to the inability to update and correct knowledge content in a timely manner based on actual usage data. This effectively improves the dynamic adaptability and content accuracy of the knowledge base, ensuring that the customer service system can continuously provide high-quality user services.

[0041] The above describes the customer service knowledge base optimization method based on multi-dimensional evaluation in the embodiments of the present invention. The following describes the customer service knowledge base optimization device based on multi-dimensional evaluation in the embodiments of the present invention. Please refer to [link / reference]. Figure 2 One embodiment of the customer service knowledge base optimization device based on multi-dimensional evaluation in this invention includes: Data acquisition module 201 is used to collect response interaction data and evaluation data of the customer service system. The response interaction data includes at least the system's answer content and the corresponding knowledge base entries. The quality assessment module 202 is used to calculate the sub-item scores of each dimension of the system's answer content based on the evaluation data and preset quality assessment dimensions, and to perform weighted calculation of each sub-item score to obtain a comprehensive quality score. The defect diagnosis module 203 is used to filter out system answer content with a comprehensive quality score lower than a preset threshold and match the corresponding knowledge base entries as entries to be optimized based on the knowledge base entries, and determine the defect category corresponding to the entries to be optimized based on the sub-item scores of each dimension. The difference analysis module 204 is used to extract the correction information corresponding to each item to be optimized from the evaluation data, and to calculate the semantic difference between the correction information and the original content corresponding to the item to be optimized. The content optimization module 205 is used to generate a correction scheme based on the defect category when the semantic difference exceeds a preset difference threshold, and to update the content to be optimized in the customer service knowledge base according to the correction scheme.

[0042] In this embodiment of the invention, response interaction data and evaluation data from a customer service system are collected. The response interaction data includes system answer content and corresponding knowledge base entries. Based on the evaluation data and preset quality assessment dimensions, scores for each dimension of the system answer content are calculated, and a weighted average of these scores is used to obtain a comprehensive quality score. System answer content with a comprehensive quality score below a preset threshold is selected from the knowledge base entries, and the corresponding knowledge base entries are matched as items to be optimized. The defect category corresponding to each item to be optimized is determined based on the scores for each dimension. Correction information corresponding to each item to be optimized is extracted from the evaluation data, and the semantic difference between the correction information and the original content corresponding to the item to be optimized is calculated. When the semantic difference exceeds a preset difference threshold, a correction scheme is generated based on the defect category, and the content to be optimized in the customer service knowledge base is updated according to the correction scheme. This application, by constructing a multi-dimensional quality assessment system and a defect classification and diagnosis mechanism, achieves accurate positioning and targeted repair of knowledge base problems. It solves the technical problem that the lack of an automated optimization mechanism in existing customer service knowledge bases leads to the inability to update and correct knowledge content in a timely manner based on actual usage data. This effectively improves the dynamic adaptability and content accuracy of the knowledge base, ensuring that the customer service system can continuously provide high-quality user services.

[0043] above Figure 2 The customer service knowledge base optimization device based on multidimensional evaluation in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The customer service knowledge base optimization device based on multidimensional evaluation in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0044] Figure 3 This is a schematic diagram of a customer service knowledge base optimization device based on multidimensional evaluation, provided in an embodiment of the present invention. This multidimensional evaluation-based customer service knowledge base optimization device 300 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the multidimensional evaluation-based customer service knowledge base optimization device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the multidimensional evaluation-based customer service knowledge base optimization device 300.

[0045] The customer service knowledge base optimization device 300 based on multidimensional evaluation may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The illustrated customer service knowledge base optimization device structure based on multidimensional evaluation does not constitute a limitation on the customer service knowledge base optimization device based on multidimensional evaluation. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0046] The present invention also provides a customer service knowledge base optimization device based on multidimensional evaluation. The computer device includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs each step of the customer service knowledge base optimization method based on multidimensional evaluation in the above embodiments.

[0047] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the various steps of the customer service knowledge base optimization method based on multi-dimensional evaluation.

[0048] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0049] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0050] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0051] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A customer service knowledge base optimization method based on multidimensional evaluation, characterized in that, The customer service knowledge base optimization method based on multidimensional evaluation includes: Collect response interaction data and evaluation data from the customer service system. The response interaction data shall at least include the system's answer content and the corresponding knowledge base entries. Based on the evaluation data and the preset quality assessment dimensions, the scores of each dimension of the system's response content are calculated, and the scores of each dimension are weighted to obtain a comprehensive quality score. Based on the knowledge base entries, system answer content with a comprehensive quality score lower than a preset threshold is selected and the corresponding knowledge base entries are matched as entries to be optimized. Based on the sub-item scores of each dimension, the defect category corresponding to the entries to be optimized is determined. Extract the correction information corresponding to each item to be optimized from the evaluation data, and calculate the semantic difference between the correction information and the original content corresponding to the item to be optimized; When the semantic difference exceeds a preset difference threshold, a correction plan is generated based on the defect category, and the content to be optimized in the customer service knowledge base is updated according to the correction plan.

2. The customer service knowledge base optimization method based on multi-dimensional evaluation according to claim 1, characterized in that, The quality assessment dimensions include at least the content factuality dimension, semantic matching degree dimension, and language fluency dimension. Based on the evaluation data and the preset quality assessment dimensions, the system calculates the sub-scores for each dimension of the response content, and then weights these sub-scores to obtain a comprehensive quality score, including: The satisfaction score, answer credibility score, and quality review score in the evaluation data are mapped to a unified scoring range to obtain normalized score values. Based on the quality assessment dimensions, the assessment data corresponding to the dimensions are selected from the normalized score values, and the content factuality score, semantic matching score, and language fluency score are calculated respectively. Based on the preset weight coefficients of each quality assessment dimension, the scores of each dimension and their corresponding weight coefficients are weighted and summed to obtain the comprehensive quality score.

3. The customer service knowledge base optimization method based on multi-dimensional evaluation according to claim 1, characterized in that, Based on the knowledge base entries, system answer content with a comprehensive quality score lower than a preset threshold is selected and matched with the corresponding knowledge base entries as items to be optimized. Then, based on the sub-scores for each dimension, the defect categories corresponding to the items to be optimized are determined, including: Filter out system response content with a comprehensive quality score lower than a preset threshold, and extract the knowledge base entries called when generating the system response content from the response interaction data, and mark the knowledge base entries as entries to be optimized; Based on each item to be optimized, extract the sub-scores of each dimension of the corresponding system's response content, compare the numerical values ​​of each sub-score, and take the evaluation dimension corresponding to the sub-score with the lowest numerical value as the defect category of the item to be optimized. The defect category includes at least the content factual defect category, the semantic matching degree defect category, and the language fluency defect category.

4. The customer service knowledge base optimization method based on multi-dimensional evaluation according to claim 1, characterized in that, The step of extracting correction information corresponding to each item to be optimized from the evaluation data and calculating the semantic difference between the correction information and the original content corresponding to the item to be optimized includes: Extract the text feedback content corresponding to each item to be optimized from the user evaluation information in the evaluation data, and use the correction content provided by the user in the text feedback content as the correction information for the item to be optimized; The original content corresponding to the correction information and the item to be optimized is respectively encoded with semantic vectors to obtain the semantic vector of the correction information and the semantic vector of the original content; The cosine distance between the semantic vector of the corrected information and the semantic vector of the original content is calculated to obtain the semantic difference degree, wherein the semantic difference degree is used to characterize the degree of change of the corrected information relative to the original content.

5. The customer service knowledge base optimization method based on multi-dimensional evaluation according to claim 1, characterized in that, When the semantic difference exceeds a preset difference threshold, generating a correction scheme based on the defect category includes: Determine whether the semantic difference exceeds a preset difference threshold. If it exceeds the threshold, determine the corresponding optimization instruction based on the defect category of the item to be optimized. When the defect category is a content factual defect category, generate a factual content replacement instruction. When the defect category is a semantic matching defect category, generate a context association optimization instruction. When the defect category is a language fluency defect category, generate a language reorganization instruction. The optimization instruction, the original content of the item to be optimized, and the correction information are combined to generate a prompt text; Based on the input prompt text, the corrected knowledge entry content is generated, and the corrected knowledge entry content is associated with and stored with the corresponding knowledge base entry identifier to form a correction scheme.

6. The customer service knowledge base optimization method based on multi-dimensional evaluation according to claim 5, characterized in that, The step of combining the optimization instruction, the original content of the item to be optimized, and the correction information to generate the prompt text includes: The optimization instructions are formatted into instruction description text, the original content of the item to be optimized is formatted into reference content text, and the correction information is formatted into correction requirement text. The instruction description text, reference content text, and correction requirement text are arranged in a preset combination order, and preset separators are inserted between each text. The sorted text is merged to generate the prompt text.

7. The customer service knowledge base optimization method based on multi-dimensional evaluation according to claim 1, characterized in that, The steps to update the content to be optimized in the customer service knowledge base according to the proposed correction plan include: Obtain the corrected knowledge entry content and the knowledge base entry identifier corresponding to the content to be optimized from the correction scheme; The target entry in the customer service knowledge base is located based on the knowledge base entry identifier. The original content of the target entry is replaced with the corrected knowledge entry content, and the revision timestamp and version number of the target entry are updated to complete the knowledge base update.

8. A customer service knowledge base optimization device based on multi-dimensional evaluation, characterized in that, The customer service knowledge base optimization device based on multidimensional evaluation includes: The data acquisition module is used to collect response interaction data and evaluation data from the customer service system. The response interaction data includes at least the system's answer content and the corresponding knowledge base entries. The quality assessment module is used to calculate the sub-item scores of each dimension of the system's answer content based on the evaluation data and preset quality assessment dimensions, and to perform weighted calculation of each sub-item score to obtain a comprehensive quality score. The defect diagnosis module is used to filter out system answer content with a comprehensive quality score lower than a preset threshold and match the corresponding knowledge base entries as entries to be optimized based on the knowledge base entries. It also determines the defect category of the entries to be optimized based on the sub-item scores of each dimension. The difference analysis module is used to extract the correction information corresponding to each item to be optimized from the evaluation data, and to calculate the semantic difference between the correction information and the original content corresponding to the item to be optimized. The content optimization module is used to generate a correction plan based on the defect category when the semantic difference exceeds a preset difference threshold, and to update the content to be optimized in the customer service knowledge base according to the correction plan.

9. A customer service knowledge base optimization device based on multi-dimensional evaluation, characterized in that, The customer service knowledge base optimization device based on multidimensional evaluation includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the customer service knowledge base optimization device based on multidimensional evaluation to perform the steps of the customer service knowledge base optimization method based on multidimensional evaluation as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the customer service knowledge base optimization method based on multidimensional evaluation as described in any one of claims 1-7.

Images