Chat standard reply delta sedimentation method based on artificial confirmation of sending result and evidence playback

Abstract

The application relates to the technical field of artificial intelligence application, and provides a chat standard reply incremental sedimentation method based on artificial confirmation of sending results and evidence playback, which comprises the following steps: obtaining a process result in response to a trigger condition of a chat reply process; constructing a standard reply candidate according to the obtained process result; and writing the standard reply candidate into a candidate area isolated from a formal standard reply area in a knowledge base; performing a duplicate checking process in the candidate area according to a candidate number in the standard reply candidate; newly creating a candidate record or updating an existing candidate record according to a duplicate checking process result; performing artificial auditing on the standard reply candidate; and in response to an auditing result, converting the standard reply candidate that passes the auditing into a formal standard reply page and writing the standard reply candidate into the formal standard reply area. The application can safely and efficiently convert high-quality chat replies that have been confirmed by artificial confirmation into reusable standard reply knowledge under the premise of ensuring that un-audited content does not pollute a formal knowledge base and that an auditing process is traceable.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence application technology, and in particular to a method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback. Background Technology

[0002] In enterprise chat assistance, customer response, and internal collaboration scenarios, desktop intelligent reply systems typically identify chat window content, retrieve relevant information from local or team knowledge bases, and then call upon a large model to generate candidate replies. Users complete the reply process by confirming, editing, copying, or sending the generated replies. During this process, a large number of high-quality replies to repetitive questions such as refund procedures, document submissions, and configuration instructions, after repeated manual revisions to form stable scripts, should ideally become a source for the continuous growth of the knowledge base.

[0003] However, existing technologies have the following significant drawbacks: High-quality responses manually verified in real chats cannot be fed back into the knowledge base. When similar issues arise later, re-retrieval, generation, and correction are still required, leading to resource waste and inconsistencies in wording. Directly writing generated responses into the formal knowledge base causes knowledge pollution. Generated results often contain temporary context, over-promises, or incomplete evidence. Entering the formal knowledge base without review will result in repeated citations in subsequent searches, solidifying errors. Simply saving the response text lacks evidence playback capabilities. Without retaining the source of the hit, matching score, screenshot path, risk assessment, and sending context, reviewers cannot determine the universality and reliability of the response, making accurate inclusion decisions difficult. Frequently occurring questions lack deduplication and value accumulation mechanisms. When the same questions and responses appear repeatedly, creating new entries each time generates significant redundancy; completely ignoring recurrence fails to reflect the high-frequency value of the candidate repeatedly verified in real-world scenarios. There is a lack of isolation boundaries between candidates awaiting review and the formal knowledge base. If unreviewed candidates directly participate in the search, the system will provide unconfirmed content to the model, leading to the premature use of erroneous knowledge and contaminating subsequent generated results. The knowledge lifecycle before and after review lacks complete status records. If the status, time, reason, and entry path are not recorded after a candidate is approved, rejected, or modified, it is impossible to track the source and quality changes of the candidate, which is detrimental to the continuous governance of the knowledge base.

[0004] Chinese patent CN112131338A discloses a method for creating question-and-answer pairs. This method monitors human customer service representatives' pasting operations within the current period to determine if the frequency of repeated use of the same reply text reaches a threshold. If so, it directly generates a question-and-answer pair with the current user's question and stores it in a knowledge base, thereby achieving automatic discovery and reuse of high-frequency replies in customer service scenarios. However, this method relies solely on the human customer service representative's pasting operation as a trigger, directly using the frequency of reply text repetition as the basis for data entry. It lacks the freezing and replay of data generation criteria such as retrieval evidence, risk assessment, and sending context, resulting in a lack of traceability in data entry decisions. Furthermore, it does not establish an isolation zone between pending candidates and the official knowledge base. Unreviewed question-and-answer pairs are directly entered into the knowledge base, or rely solely on human review without supporting evidence, posing a risk of knowledge contamination.

[0005] Therefore, how to provide a standard incremental accumulation method for replies that takes manually confirmed chat replies as input, evidence snapshots and candidate isolation as constraints, and manual review as the formal entry threshold, so that desktop chat assistants can be upgraded from a one-way ability to generate replies by consuming knowledge to a closed-loop system that continuously accumulates reusable knowledge within the boundaries of manual confirmation, has become an urgent technical problem to be solved. Summary of the Invention

[0006] In view of this, in order to overcome the shortcomings of the prior art, the present invention aims to provide a method and system.

[0007] This invention provides a method for incremental accumulation of standard chat responses based on manual confirmation of sending results and evidence playback. The method includes:

[0008] Step S1: Respond to the triggering conditions of the chat reply process to obtain the process result, construct standard reply candidates based on the obtained process result, and write the standard reply candidates into the candidate area isolated from the formal standard reply area in the knowledge base;

[0009] Step S2: Perform deduplication processing in the candidate area based on the candidate number in the standard response candidates, and create new candidate records or update existing candidate records based on the deduplication processing results;

[0010] Step S3: Perform manual review of the standard response candidates. In response to the review results, convert the approved standard response candidates into official standard response pages and write them into the official standard response area.

[0011] Optionally, in the incremental sedimentation method for standard chat replies based on manual confirmation of sending results and evidence playback of the present invention, in step S1, the trigger condition for obtaining the process result is that the final reply in the chat reply process has been manually confirmed and sent, or the final reply has been explicitly selected to be saved by the user. The process result includes user questions, final replies, search hit sets, screenshot paths, risk assessment results, and sending context.

[0012] Optionally, in the incremental accumulation method for standard chat responses based on manual confirmation of sending results and evidence playback of the present invention, in step S1, standard response candidates are constructed according to the obtained process results in the following manner:

[0013] The process results are filtered and validated, and a set of candidate evidence snapshots is generated based on the search hit set in the process results that pass the filtering and validation.

[0014] Generate deduplication keys based on user questions and final responses in the process results, and generate corresponding candidate numbers based on the deduplication keys;

[0015] The constructed standard response candidates are structured data objects, including user questions, final responses, candidate evidence snapshot sets, screenshot paths, risk assessment results, sending context, deduplication keys, and candidate numbers.

[0016] Optionally, in the incremental sedimentation method for chat standard replies based on manual confirmation of sending results and evidence playback of the present invention, in step S1, the candidate evidence snapshot set is generated in the following manner: traversing each search hit in the search hit set, extracting the source path, source title, source type, source tag, hit fragment summary and original relevance; normalizing the original relevance to a knowledge matching score within a preset score range; assembling it into a candidate evidence snapshot corresponding to the search hit, and all successfully converted candidate evidence snapshots constitute the candidate evidence snapshot set.

[0017] Optionally, in the incremental sedimentation method for chat standard replies based on manual confirmation of sending results and evidence playback of the present invention, in step S1, a deduplication key and a corresponding candidate number are generated in the following manner: the user question and the final reply are normalized respectively, the normalized user question and the normalized final reply are combined into a deduplication payload, the hash value of the deduplication payload is calculated as the deduplication key, and the first few characters of the deduplication key are extracted as candidate numbers.

[0018] Optionally, in the incremental sedimentation method for chat standard replies based on manual confirmation of sending results and evidence playback of the present invention, in step S2, if there are no identical candidates, a new candidate record is created; if there are identical candidates, the adoption count and weight of the existing candidate record are updated. The status of the candidate record includes at least pending review, passed, and rejected. Newly created candidates default to the pending review status.

[0019] Optionally, in the incremental accumulation method for chat standard replies based on manual confirmation of sending results and evidence playback of the present invention, in step S3, manual review is performed in the review interface. The review interface at least displays the user question, the final reply, the set of candidate evidence snapshots and the source path and knowledge matching score of each evidence snapshot, the screenshot corresponding to the screenshot path, the risk judgment result, the sending object, and the number of adoptions and weights. The review interface also provides an editing entry, allowing reviewers to adjust the standard reply candidates before they are added to the database. The pre-entry adjustment includes at least one or more of the following: editing the candidate title, applicable scenario, reply key points, suggested reply, and classification tag.

[0020] Optionally, in the incremental accumulation method for standard chat replies based on manual confirmation of sending results and evidence playback of the present invention, in step S3, the approved standard reply candidates are converted into formal standard reply pages and written into the formal standard reply area in the following manner: the user questions, final replies, candidate evidence snapshot sets and classification tags in the standard reply candidates are mapped to the corresponding fields of the formal standard reply page, a type identifier and a source candidate number are added, and after being written into the formal standard reply area, the knowledge index is refreshed so that the formal standard reply page can be hit in the knowledge retrieval of subsequent chat replies.

[0021] Optionally, in the incremental sedimentation method for chat standard replies based on manual confirmation of sending results and evidence playback of the present invention, in step S3, in response to the review rejection, the reason for rejection is recorded and the candidate record is marked as rejected, so that the corresponding standard reply candidate does not enter the formal standard reply area.

[0022] Optionally, in the incremental sedimentation method for chat standard replies based on manual confirmation of sending results and evidence playback of the present invention, in step S3, when writing to the formal standard reply page or refreshing the knowledge index fails, the passed status and failure record of the candidate record are retained, and the formal standard reply page is not generated again; after recovery, the candidate record is located according to the candidate number, and an idempotent retry is performed based on the retained passed status, and the writing and refreshing operations are re-executed until successful completion.

[0023] The incremental accumulation method for chat standard replies based on manual confirmation of sending results and evidence playback of the present invention has the following beneficial technical effects: by writing standard reply candidates into a candidate area isolated from the formal standard reply area, unreviewed content is not included in the formal retrieval, thus avoiding the pollution of the formal knowledge base by erroneous knowledge; duplicate candidates are merged and accumulated through deduplication processing to avoid redundant storage; and the quality of data entry is controlled by manual review, ensuring that all formal standard replies have been verified in real scenarios and manually confirmed, forming a secure accumulation closed loop from manually confirmed replies to the standard knowledge base. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating the incremental accumulation method for chat standard replies based on manual confirmation of sending results and evidence playback according to an exemplary embodiment 1 of the present invention.

[0026] Figure 2 This is a flowchart illustrating the process of generating deduplication keys and corresponding candidate numbers for the incremental sedimentation method of chat standard replies based on manual confirmation of sending results and evidence playback according to an exemplary embodiment 1 of the present invention.

[0027] Figure 3 This is a flowchart illustrating the process of creating or updating candidate records for the incremental accumulation method of chat standard replies based on manual confirmation of sending results and evidence playback according to an exemplary embodiment 1 of the present invention.

[0028] Figure 4 This is a flowchart illustrating the fault-tolerant processing of the chat standard reply incremental sedimentation method based on manual confirmation of sending results and evidence playback according to Exemplary Embodiment 1 of the present invention when writing fails. Detailed Implementation

[0029] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] It should be noted that, in the absence of conflict, the following embodiments and features can be combined with each other; and, based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0031] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0032] Example 1

[0033] Exemplary embodiment 1 of the present invention provides a method for incremental accumulation of standard chat responses based on manual confirmation of sending results and evidence playback. Figure 1 This is a flowchart illustrating the incremental accumulation method for standard chat responses based on manual confirmation of sending results and evidence playback according to Exemplary Embodiment 1 of the present invention. Figure 1 As shown, in this embodiment, the method of the present invention is implemented in the following manner:

[0034] Step S1: In response to the triggering conditions of the chat reply process, obtain the process result, construct standard reply candidates based on the obtained process result, and write the standard reply candidates into the candidate area of ​​the knowledge base that is isolated from the formal standard reply area.

[0035] In this embodiment, the triggering condition for obtaining the process result is that the final reply in the chat reply process has been manually confirmed and sent, or the final reply has been manually selected to be saved. The process result includes user question, final reply, search hit set, screenshot path, risk assessment result and sending context.

[0036] This embodiment limits the triggering condition to a final reply that has been manually confirmed and sent or has been explicitly selected to be saved by a person. This ensures that only high-quality replies from real chats and verified by a person enter the accumulation process, providing a reliable input source for subsequent candidate construction. At the same time, by carrying user questions, final replies, search hits, screenshot paths, risk assessments, and sending contexts in the process results, the candidates have complete generation basis and scenario information.

[0037] This embodiment filters and verifies the process results to exclude empty responses, risk-blocked responses, and non-compliant sources from the source, ensuring candidate quality; it generates a set of candidate evidence snapshots to freeze the retrieval basis at the time of generation, so that the candidates have complete evidence support; and it generates deduplication keys and candidate numbers to provide unique identifiers for subsequent deduplication accumulation and audit traceability.

[0038] Specifically, in this embodiment, standard response candidates are constructed based on the obtained process results in the following manner: the process results are filtered and verified, and a set of candidate evidence snapshots is generated based on the search hit set in the process results that have passed the filtering and verification; a deduplication key is generated based on the user question and the final response in the process results, and a corresponding candidate number is generated based on the deduplication key; the constructed standard response candidates are structured data objects, including user question, final response, set of candidate evidence snapshots, screenshot path, risk assessment result, sending context, deduplication key, and candidate number.

[0039] In this embodiment, the candidate evidence snapshot set is generated as follows: Each search hit in the search hit set is traversed, and the source path, source title, source type, source tag, hit fragment summary, and original relevance are extracted; the original relevance is normalized to a knowledge matching score within a preset score range; and the snapshots are assembled into candidate evidence snapshots corresponding to that search hit. All successfully converted candidate evidence snapshots constitute the candidate evidence snapshot set. This embodiment normalizes the original relevance in the search hits to a knowledge matching score within a preset score range, making the search results from different sources and algorithms uniformly quantitatively comparable, facilitating reviewers to intuitively judge the reliability of each piece of evidence during review; and by assembling candidate evidence snapshots in a structured manner, it ensures that the key fields of each search hit are completely frozen and retained, providing accurate and traceable original evidence for review playback.

[0040] Figure 2 This is a flowchart illustrating the process of generating deduplication keys and corresponding candidate numbers for the incremental sedimentation method of chat standard replies based on manual confirmation of sending results and evidence playback according to Exemplary Embodiment 1 of the present invention. Figure 2 As shown, in this embodiment, the deduplication key and corresponding candidate number are generated in the following manner: the user question and the final reply are normalized respectively, the normalized user question and the normalized final reply are combined into a deduplication payload, the hash value of the deduplication payload is calculated as the deduplication key, and the first few characters of the deduplication key are extracted as candidate numbers.

[0041] This embodiment combines the hash values ​​of the user's question and the final reply after normalization to ensure that the same question and the same reply can be stably mapped to the same deduplication key under any conditions, so that duplicate candidates can be accurately located. By truncating the first few digits of the deduplication key as a candidate number, a length-controllable identifier is obtained while ensuring uniqueness, which is convenient for system indexing and manual identification, and improves the computational efficiency and operability of the deduplication process.

[0042] Step S2: Perform deduplication processing in the candidate area based on the candidate number in the standard response candidates, and create new candidate records or update existing candidate records based on the deduplication processing results.

[0043] Figure 3 This is a schematic diagram illustrating the process of creating or updating candidate records in the incremental accumulation method for standard chat replies based on manual confirmation of sending results and evidence playback according to Exemplary Embodiment 1 of the present invention. Figure 3As shown, in this embodiment, if no identical candidate exists, a new candidate record is created; if identical candidates exist, the adoption count and weight of the existing candidate record are updated. The status of a candidate record includes at least pending review, passed, and rejected. Newly created candidates default to the pending review status. This embodiment ensures that newly created candidates do not become publicly available before review by defaulting to the pending review status, thus continuing the candidate isolation security mechanism. Through the branching process of creating new candidate records and updating existing candidate records, duplicate records are not created when the same question and the same response occur repeatedly, effectively reducing knowledge redundancy. By accumulating the adoption count and weight, the high-frequency usage value of candidates is continuously recorded, providing a quantitative basis for review ranking and knowledge governance.

[0044] Step S3: Perform manual review of the standard response candidates. In response to the review results, convert the approved standard response candidates into official standard response pages and write them into the official standard response area.

[0045] In practical applications, the manual review in this embodiment is performed in the review interface. The review interface at least displays the user's question, the final reply, the collection of candidate evidence snapshots and the source path and knowledge matching score of each evidence snapshot, the screenshot corresponding to the screenshot path, the risk assessment result, the recipient, and the number of adoptions and weights. The review interface also provides an editing entry, allowing reviewers to adjust the standard reply candidates before they are added to the database. The pre-entry adjustment includes at least one or more of the following: editing the candidate title, applicable scenario, reply key points, suggested reply, and classification tag.

[0046] This embodiment establishes a two-way traceability link between candidates and formal knowledge by mapping candidate content to formal standard response pages and attaching type identifiers and source candidate numbers. This allows formal standard responses to trace back to the original candidates and the evidence snapshot at that time. By refreshing the knowledge index, newly added standard responses can be immediately retrieved and hit in subsequent chat responses, achieving immediate effectiveness and reuse of knowledge.

[0047] Specifically, in this embodiment, the approved standard response candidates are converted into formal standard response pages and written into the formal standard response area in the following manner: the user questions, final responses, candidate evidence snapshot sets, and classification tags in the standard response candidates are mapped to the corresponding fields of the formal standard response page, a type identifier and a source candidate number are added, and after being written into the formal standard response area, the knowledge index is refreshed so that the formal standard response page can be hit in the knowledge retrieval of subsequent chat replies.

[0048] In this embodiment, in response to a review rejection, the reason for rejection is recorded and the candidate record is marked as rejected, preventing the corresponding standard response candidate from entering the formal standard response area and protecting the formal knowledge base from contamination by inappropriate wording. The persistence of rejection reasons and status records facilitates subsequent analysis of rejection patterns and knowledge gaps, providing data support for knowledge base governance and response generation strategy optimization.

[0049] Figure 4 This is a flowchart illustrating the fault-tolerant processing of the incremental sedimentation method for chat standard replies based on manual confirmation of sending results and evidence playback according to Exemplary Embodiment 1 of the present invention when writing fails. Figure 4 As shown in this embodiment, when writing to the formal standard response page or refreshing the knowledge index fails, the passed status and failed records of the candidate records are retained, and the formal standard response page is not generated repeatedly. This avoids the loss of review results or the need for repeated review due to temporary failures during the data entry process. After recovery, the candidate record is located according to the candidate number, and an idempotent retry is performed based on the retained passed status. The write and refresh operations are re-executed until successful completion, ensuring that each candidate that passes the review can be reliably entered into the database without generating duplicate formal pages, thus enhancing fault tolerance and data consistency.

[0050] Example 2

[0051] Exemplary Example 2 of this invention provides a method for incremental accumulation of standard chat responses based on manually confirmed sending results and evidence playback. This embodiment uses customer service personnel handling user inquiries about the "refund process" as a typical scenario, detailing the complete process from the generation of standard response candidates to their formal entry into the database. The system in this embodiment is a chat assistance software system deployed on desktops (such as Windows, macOS, etc.) to implement the method of this invention in enterprise chat, customer replies, internal collaboration, and knowledge base question-and-answer scenarios. It assists users in quickly generating, confirming, and sending chat responses, and accumulates high-quality responses that have been manually confirmed into standard knowledge.

[0052] 1. Process Result Collection

[0053] Customer service personnel open a chat session with a user in the desktop chat intelligent reply system. The system uses a chat recognition module to screenshot and identify the chat window, obtaining the chat context and recognizing the user's latest question as, "I want to apply for a refund, how do I do that?" Subsequently, the system calls the knowledge retrieval module to search the local knowledge base for information related to "refund process," obtaining several search results, such as relevant paragraphs from the "Refund Policy and Operation Guide" document. The system uses these search results as context input to a large model, generating candidate reply text: "Hello, please provide your order number to apply for a refund. We will process your request within 1-3 business days after verification."

[0054] The system uses its risk strategy module to assess the risk of the reply, outputting a "low risk" risk level and setting the action to "allow sending." Customer service personnel review the generated candidate reply, deem the content accurate, and without editing, directly click the send button to send the reply to the user. This triggers the conditions for the message retention process (the final reply has been manually confirmed and sent), and the system uses its process result collection module to uniformly obtain the process result of this chat reply.

[0055] The process results are carried by a unified process result object, which specifically includes: the user question "I want to apply for a refund, how do I do that?", the final reply "Hello, to apply for a refund, please provide your order number. We will process it for you within 1-3 business days after verification.", the search hit list (including the path to the hit knowledge base file, the fragment summary, and the original relevance score), the screenshot path of the current chat window, the risk assessment result output by the risk strategy module (risk level is low risk, action is allow sending), the sending mode is "direct sending", the sending target is the user in the current session, the sending result is "sent successfully", and the source session name is the current session name.

[0056] 2. Verify before saving

[0057] The system passes the process results to the candidate accumulation module, first executing the pre-save verification submodule. The verification logic checks the following conditions in sequence: whether the final response field is empty (not empty, verification passes); whether the risk judgment result indicates that sending is allowed (the policy judgment result indicates that sending is allowed, verification passes); whether the system has configured the knowledge base directory (configured, verification passes); whether the user's question can be effectively extracted from the process results (extractable, verification passes); whether the source of the process results is the generated result or the session draft result (from the generated result, verification passes); whether the response has been marked as blocked by the risk policy (not blocked, verification passes). If all six verifications pass, the system allows the process to proceed to the subsequent candidate generation step.

[0058] If any of the above checks fail, for example, if the response is blocked by a risk policy, the system will record the reason for rejection in the log ("The response has been blocked by a risk policy, terminate the sedimentation process") and terminate the process directly without generating any candidate records.

[0059] 3. Generation of candidate evidence snapshot set

[0060] After successful verification, the system iterates through each search hit in the search hit set and performs evidence snapshot transformation. For the first search hit, the system extracts its source path (e.g., " / knowledge / policies / refund_policy.md"), source title ("Refund Policy and Operation Guide"), source type ("Markdown Document"), source tag ("Policy Category"), hit fragment summary ("Refund applications require users to provide order numbers, and processing will take 1-3 business days after verification"), and original relevance (e.g., 0.87). The system converts this original relevance into a knowledge matching score within the 0-100 range (e.g., 87 points) according to a preset normalization rule. Subsequently, the system assembles all extracted and converted information into a candidate evidence snapshot corresponding to this search hit. If the search hit set contains multiple hits, the system performs the above transformation on each hit. All successfully transformed snapshots together constitute a candidate evidence snapshot set and are bound and saved with the subsequently constructed standard response candidates. If a search hit fails to be transformed due to formatting errors, the system marks it as unparseable, but this does not affect the generation of evidence snapshots for other search hits.

[0061] 4. Deduplication key and candidate number generation

[0062] The system standardizes user questions by removing leading and trailing whitespace, merging consecutive whitespace, and unifying character case, resulting in the standardized user question, "I want to apply for a refund, how do I do that?". The system performs the same standardization on the final reply, resulting in the standardized final reply, "Hello, to apply for a refund, please provide your order number. We will process your request within 1-3 business days after verification." The system combines the standardized user question, line breaks, and standardized final reply in sequence into a deduplication payload, calculates its SHA-256 hash value, and uses the hash value as the deduplication key. The system then extracts the first 8 characters of this deduplication key as candidate numbers (e.g., "a3f5c8e9").

[0063] 5. Construction of Standard Response Candidates and Writing of Candidate Regions

[0064] The system constructs standard response candidates, which are structured data objects containing the following fields: candidate number ("a3f5c8e9"), deduplication key (complete hash value), user question ("I want to apply for a refund, how do I do that?"), final reply ("Hello, please provide your order number to apply for a refund. We will process it for you within 1-3 business days after verification."), candidate evidence snapshot set, source screenshot path, source session name, sending mode ("direct sending"), risk judgment summary ("low risk, allow sending"), sending object (current user identifier), sending result ("sent successfully"), candidate status (set to "pending review"), creation time (current timestamp), number of adoptions (initial value 1), weight (initial value set to 1), and unassigned fields such as review time, number of modifications, rejection reason, and formal entry path.

[0065] The system writes standard response candidates into the candidate area of ​​the knowledge base as candidate files. These candidate files are stored in the "wiki / candidates / " directory, which is physically isolated from the official standard response area "wiki / playbooks / ". Each candidate file contains both structured metadata (YAML format, including candidate number, deduplication key, status, creation time, adoption count, and other system-scanned fields) and human-readable text (Markdown format, including user questions, final responses, evidence snapshot lists, screenshot paths, risk assessment summaries, and other content directly viewable by reviewers).

[0066] While writing to the candidate area, the system searches for other candidate records with the same or similar user questions. If other candidate records are found, the system outputs a related candidate suggestion for reviewers to compare in the subsequent review interface, avoiding wording conflicts. Simultaneously, the system searches the formal standard response area for a relevant formal standard response. If found, it outputs a prompt indicating that an existing standard response can be reused, without automatically overwriting the existing formal standard response.

[0067] Once a candidate is written, it is in a pending review state and will not participate in the formal standard response retrieval. This means that the candidate will not be found in knowledge retrieval for other chat sessions in the future.

[0068] 6. Repeated accumulation processing

[0069] The system searches the candidate area for a candidate with the same ID "a3f5c8e9". Since this is the first time this question and response have appeared, there is no candidate record with the same ID in the candidate area. The system creates a new candidate record, and the candidate status defaults to pending review, with the adoption count set to 1 and the weight set to 1.

[0070] Consider a variant scenario: In another session, another customer service representative encounters the same user question, "I want to apply for a refund, how do I do that?" and the system generates the same reply. After manual confirmation and sending, the system again generates the same deduplication key and candidate number "a3f5c8e9". At this point, the system searches the candidate area based on this candidate number and finds that the same candidate record already exists. Instead of creating a new candidate file, the system performs an update operation: increasing the adoption count of the existing candidate record from 1 to 2, updating the most recent adoption time to the current time, and increasing the weight accordingly based on the adoption count (e.g., the weight increases linearly with the adoption count, set to equal the adoption count of 2). Simultaneously, the recipient, sending result, and session name from this saved request are added to the candidate file, and then rewritten to the candidate file. This mechanism ensures that repeated occurrences of the same question and the same reply do not generate multiple redundant files, while its high-frequency value is continuously reflected through the accumulation of adoption counts and weights.

[0071] 7. Manual review

[0072] The system displays the candidate record in the candidate review queue, and reviewers (such as knowledge base administrators or team leaders) can view the candidate details through the review interface. The review interface displays the following information: user question ("I want to apply for a refund, how do I do that?"), final reply ("Hello, please provide your order number to apply for a refund. We will process it for you within 1-3 business days after verification."), candidate evidence snapshot collection and the source path and knowledge matching score of each evidence snapshot (e.g., "Source: / knowledge / policies / refund_policy.md, matching score: 87 points"), screenshot of the chat window corresponding to the screenshot path (reviewers can click to view the original chat context), risk judgment summary ("Low risk, allowed to send"), sending target, sending mode, number of adoptions (1 time), and weight (1).

[0073] Based on the complete information above, the reviewers made a comprehensive judgment: the response was accurate, the wording was general, the risks were controllable, the evidence was sufficient, and the source was authentic. The reviewers believed that the response was suitable for inclusion as a standard response. In the review interface, the reviewers used the editing entry to standardize the candidate title (changing it from "Refund Application Response" to "Standard Reply Script for Refund Process"), applicable scenarios (filling in "Applicable when users inquire about the refund process"), key points of the response (filling in "1. Please provide the order number; 2. Inform the user that the processing time is 1-3 business days"), and category tags (filling in "After-sales / Refund") before adding them to the database. Finally, they clicked the "Approved" button.

[0074] 8. Formal warehousing

[0075] In response to the manual review approval, the system performs the following transformations: It maps the user questions, final responses, candidate evidence snapshot sets, and classification tags in the standard response candidates to corresponding fields on the official standard response page, appends a type identifier (“standard response”) and a source candidate number (“a3f5c8e9”), and generates a complete official standard response page. The system writes this official standard response page to the “wiki / playbooks / ” directory in the official standard response area, generating a new knowledge base page file. Subsequently, the system updates the status of the candidate record to “approved”, records the review time (current time), and the official entry path (“wiki / playbooks / refund_standard_reply.md”).

[0076] The system then refreshed its knowledge index, incorporating the official standard response page into the knowledge retrieval system's index. Subsequently, when other customer service personnel encounter similar user questions (such as "how to get a refund" or "refund application process"), the knowledge retrieval module can locate this official standard response page, presenting it as a search result to customer service personnel or directly using it when generating responses in the large model. Thus, a secure incremental data accumulation chain from manually confirmed responses to official standard responses is completed.

[0077] 9. Fault tolerance for write failures

[0078] If a temporary failure occurs during the writing to the formal standard response area or the refresh of the knowledge index (such as insufficient disk space, index service restart, etc.), the system will retain the approved status and failure record of the candidate record after detecting the write failure, and will not generate a duplicate formal standard response page. The system records the failure status according to the candidate number "a3f5c8e9". After the system recovers, the system will locate the candidate record according to the candidate number, perform an idempotent retry based on the retained approved status and candidate content, and re-execute the operations of writing to the formal standard response area and refreshing the knowledge index until successful completion. This mechanism ensures that every approved candidate is reliably stored in the database, and that the review results are not lost due to temporary failures, nor are duplicate formal pages generated.

[0079] Example 3

[0080] Exemplary Example 3 of this invention provides a method for incremental accumulation of standard chat responses based on manually confirmed sending results and evidence playback. This embodiment uses the scenario of customer service personnel handling multiple unread contact conversation drafts as an example to detail the complete process of standard response candidates from draft result generation, repeated accumulation to review and rejection. The system in this embodiment is a chat assistance software system deployed on desktop (such as Windows, macOS, etc.) to implement the method of this invention in enterprise chat, customer replies, internal collaboration, and knowledge base question and answer scenarios. It assists users in quickly generating, confirming, and sending chat responses, and accumulates high-quality responses confirmed by humans into standard knowledge.

[0081] 1. Flowchart for collecting conversation draft results

[0082] Customer service personnel open multiple unread contact conversations in the multi-contact task panel. The system identifies one of the conversation drafts and reads the following information from it: contact information (User B), latest message content ("I need to submit materials, where is the upload entry?"), draft reply text ("Hello, the material upload entry is in the 'Material Center' module on the system homepage. Click the 'Upload Materials' button to operate."), search hit set (including relevant paragraphs of "Material Upload Operation Guide"), screenshot path, risk assessment result (risk level is "low risk", action is "allow sending"), and current status ("draft not sent"). After reviewing the draft reply and deeming the content complete and accurate, the customer service personnel click the "Save as Candidate" button. At this point, the trigger condition for the sedimentation process (after manual selection to save) is triggered, and the system obtains the process result corresponding to the draft.

[0083] 2. Pre-save verification and evidence snapshot generation

[0084] The system performs pre-save validation on the process results: The final reply is not empty ("Hello, the document upload entry is in the 'Document Center' module on the system homepage..."), validation passes; the risk policy judgment result is allowed to send, validation passes; the knowledge base directory is configured, validation passes; the user question is extractable ("I need to submit documents, where is the upload entry?"), validation passes; the process result originates from the conversation draft result (belonging to an allowed source type), validation passes; the reply is not blocked by the risk policy, validation passes. After all validations pass, the system iterates through the hit set, generates a candidate evidence snapshot set, and calculates the deduplication key and candidate number after normalizing the user question and final reply.

[0085] 3. Repeated accumulation processing

[0086] The system searches the candidate area for a matching candidate based on the generated candidate number. In this example, the question and response combination does not exist in the candidate area, so the system creates a new candidate record, sets its status to "pending review," initially sets the adoption count to 1, the initial weight to 1, and writes it into the candidate area.

[0087] If the candidate reappears in other conversations (for example, another customer service representative also encounters the same problem with the "data upload portal" and generates the same reply text in the draft), the system will find the existing candidate record based on the same candidate number, without creating a new file, but will update the number of adoptions from 1 to 2, update the most recent adoption time to the current time, and increase the weight to 2.

[0088] 4. Manual review

[0089] The system displays the candidate in the candidate review queue. Reviewers can view the candidate details through the review interface, including: user question ("I need to submit materials, where is the upload entry?"), final reply ("Hello, the material upload entry is in the 'Material Center' module on the system homepage. Click the 'Upload Materials' button to operate."), evidence snapshot and knowledge matching score, screenshot corresponding to the screenshot path, risk judgment summary ("Low risk, allowed to send"), recipient (user B), sending mode ("draft saved"), number of adoptions (1 time) and weight (1).

[0090] After reviewing the evidence snapshot, the reviewers found that while the response was generally accurate, the "Data Center" module's name had recently been updated to "File Management," indicating outdated information. Furthermore, the response scored only 62 points, indicating weak evidence support. The reviewers deemed this response unsuitable for use as a standard, universally accepted response (due to its outdated module name, which could mislead subsequent users).

[0091] The reviewer selects the "Reject" option on the review interface and enters in the rejection reason input box: "The 'Data Center' module name has been updated to 'File Management' in the reply. The wording is outdated and it is not recommended to add it to the database." The system responds to the review rejection operation by recording the rejection reason and review time, updating the status of the candidate record from "Pending Review" to "Rejected," and reducing the weight of the candidate (e.g., from 1 to 0.5), causing the candidate to drop in the recommendation ranking in the subsequent review queue.

[0092] Rejected candidates will not enter the formal standard response area, and will not be retrieved in subsequent formal standard response searches. However, the candidate will remain in the candidate area, and its rejection reason and status record will be persisted. This will facilitate subsequent analysis of rejection patterns by knowledge governance personnel (such as the "outdated wording due to module name change" type), providing data basis for the system to prompt users to update knowledge sources or optimize response generation strategies.

[0093] 5. Modifications approved after review

[0094] As a variation of this embodiment, if the reviewer, while reviewing the uploaded materials, deems the main direction of the response correct but requires correction, they can modify the suggested response field in the editing section of the review interface, changing "Data Center" to "File Management," and adjusting the applicable scenario and key points of the response. The system detects that the reviewer modified the final response before approval, records the number of modifications (e.g., changing from 0 to 1), and adjusts the candidate weights based on the magnitude of the modification (e.g., the initial weight of 1 is adjusted to 0.8 after a significant modification). This adjustment differentiates the candidates whose "original responses can be directly saved" from those whose "require significant manual revision before saving" in terms of weight, providing a quantitative basis for subsequent quality analysis. Subsequently, the reviewer clicks "Approved," and the system writes the revised response content as the official standard response page into the official standard response area, rather than writing it into the original draft response.

[0095] 6. Related Candidate Hints

[0096] During the aforementioned review process, the system automatically searches for other candidate records with the same or similar user questions in the candidate area based on the user question, "I need to submit materials, where is the upload portal?". If the system finds another candidate record in the candidate area with the same user question, "Where is the material upload portal?", but a different final response (e.g., "Hello, please log in to the system and find the 'File Management' module in the left navigation bar to upload"), the system will output a related candidate prompt on the review interface, reminding reviewers of other candidate versions with the same question and suggesting a comparison of the wording. Reviewers can simultaneously view the response content, evidence snapshots, and adoption counts of both candidates, comprehensively judging which version is more accurate and universal, and then deciding which to approve and which to reject, thereby avoiding conflicting expressions from multiple candidates after they are officially entered into the database.

[0097] Similarly, if the system finds an existing formal standard response (such as "Document Upload Standard Response") in the formal standard response area, the system will output a prompt indicating that a formal standard response already exists, allowing the reviewer to refer to and reuse the response. The system will not automatically overwrite the existing formal standard response. This mechanism further reduces the risk of duplicate entries.

[0098] 7. Closed-loop treatment at the source

[0099] In this embodiment, draft results in the multi-contact task platform and single-chat generation results are processed uniformly through the same set of sedimentation processes, enabling high-quality responses in draft scenarios to also enter the standard response incremental sedimentation process. After a draft is rejected, the candidate remains in the candidate area but its status is "rejected," its weight has been reduced, and it will not enter the formal search; the original draft content itself is not automatically deleted due to rejection, and customer service personnel can still view and use the draft response in the draft box for the current conversation. However, the draft content will not be passively converted into formal knowledge unless it is modified, resubmitted for review, and approved.

[0100] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0101] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback, characterized in that, The method includes: Step S1: Respond to the triggering conditions of the chat reply process to obtain the process result, construct standard reply candidates based on the obtained process result, and write the standard reply candidates into the candidate area isolated from the formal standard reply area in the knowledge base; Step S2: Perform deduplication processing in the candidate area based on the candidate number in the standard response candidates, and create new candidate records or update existing candidate records based on the deduplication processing results; Step S3: Perform manual review of the standard response candidates. In response to the review results, convert the approved standard response candidates into official standard response pages and write them into the official standard response area.

2. The method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback as described in claim 1, characterized in that, In step S1, the trigger condition for obtaining the process result is that the final reply in the chat reply process has been manually confirmed and sent, or the final reply has been manually selected to be saved. The process result includes user question, final reply, search hit set, screenshot path, risk assessment result and sending context.

3. The method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback as described in claim 1, characterized in that, In step S1, standard response candidates are constructed based on the obtained process results as follows: The process results are filtered and validated, and a set of candidate evidence snapshots is generated based on the search hit set in the process results that pass the filtering and validation. Generate deduplication keys based on user questions and final responses in the process results, and generate corresponding candidate numbers based on the deduplication keys; The constructed standard response candidates are structured data objects, including user questions, final responses, candidate evidence snapshot sets, screenshot paths, risk assessment results, sending context, deduplication keys, and candidate numbers.

4. The method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback as described in claim 3, characterized in that, In step S1, the candidate evidence snapshot set is generated as follows: traverse each search hit in the search hit set, extract the source path, source title, source type, source tag, hit fragment summary and original relevance; normalize the original relevance to a knowledge matching score within a preset score range; The candidate evidence snapshots are assembled into a snapshot corresponding to the search result, and all successfully converted candidate evidence snapshots constitute a candidate evidence snapshot set.

5. The method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback as described in claim 1, characterized in that, In step S1, the deduplication key and corresponding candidate number are generated as follows: the user question and the final reply are normalized respectively, the normalized user question and the normalized final reply are combined into a deduplication payload, the hash value of the deduplication payload is calculated as the deduplication key, and the first few characters of the deduplication key are extracted as candidate numbers.

6. The method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback as described in claim 1, characterized in that, In step S2, if no identical candidate exists, a new candidate record is created. If identical candidates exist, the adoption count and weight of the existing candidate record are updated. The status of a candidate record includes at least pending review, approved, and rejected. Newly created candidates are put into the pending review status by default.

7. The method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback as described in claim 1, characterized in that, In step S3, manual review is performed in the review interface. The review interface displays at least the user's question, the final reply, the collection of candidate evidence snapshots and the source path and knowledge matching score of each evidence snapshot, the screenshot corresponding to the screenshot path, the risk assessment result, the recipient, and the number of adoptions and weights. The review interface also provides an editing entry point for reviewers to adjust the standard reply candidates before they are added to the database. The pre-entry adjustment includes at least one or more of the following: editing the candidate title, applicable scenario, reply key points, suggested reply, and classification tag.

8. The method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback as described in claim 1, characterized in that, In step S3, the approved standard response candidates are converted into formal standard response pages and written into the formal standard response area in the following way: the user questions, final responses, candidate evidence snapshot sets and classification tags in the standard response candidates are mapped to the corresponding fields of the formal standard response page, type identifiers and source candidate numbers are added, and after being written into the formal standard response area, the knowledge index is refreshed so that the formal standard response page can be hit in the knowledge retrieval of subsequent chat replies.

9. The method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback as described in claim 1, characterized in that, In step S3, in response to the review rejection, the reason for rejection is recorded and the candidate record is marked as rejected, so that the corresponding standard response candidate does not enter the formal standard response area.

10. The method for incremental accumulation of standard chat replies based on manual confirmation of sending results and evidence playback as described in claim 1, characterized in that, In step S3, when writing to the formal standard response page or refreshing the knowledge index fails, the passed status and failed record of the candidate record are retained, and the formal standard response page is not generated again; after recovery, the candidate record is located according to the candidate number, and an idempotent retry is performed based on the retained passed status, and the write and refresh operations are re-executed until successful completion.

Images