Intelligent customer service dialogue generation and optimization method and system based on large model

Abstract

The invention provides an intelligent customer service dialogue generation and optimization method and system based on a large model. According to the method, data such as repeated questions, manual rewriting, cursor tracks, text deletion and voice interruption of a user are collected, and timestamps are marked. And calculating an operation density value and an error frequency value to generate a corrected heat map. And defining a strength node and a triggering condition edge according to the dynamic error graph. The dynamic error graph is updated when data are newly added, and an adversarial sample is generated based on the characteristics of the dynamic error graph. And the large model response module is finely tuned through reinforcement learning negative feedback, optimization content is output, the updated atlas is fed back, and closed-loop iterative optimization is realized. According to the method, a dynamic error graph is constructed by utilizing user interaction data to generate an adversarial sample, a large model is driven to perform closed-loop optimization on customer service dialogue generation through reinforcement learning, and errors are accurately positioned and continuously improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent customer service dialogue technology, and in particular to a method and system for generating and optimizing intelligent customer service dialogue based on a large model. Background Art

[0002] In the actual application of intelligent customer service, faced with the massive amount of conversational data generated by user interactions, the system needs to possess the key capabilities of autonomously identifying and locating high-frequency error responses and driving model optimization. This requires a technical solution that can dynamically capture error patterns directly from user behavioral feedback signals without human intervention, and form a closed-loop mechanism to achieve continuous and rapid self-iteration and optimization of the model to respond to ever-changing user needs and emerging error types, thereby improving service quality and efficiency.

[0003] One currently available targeted technical solution utilizes offline collected user complaints and annotated bad case data to generate adversarial examples in batches using preset rules or templates, and then regularly fine-tunes the large language model offline. This solution extracts erroneous snippets from historical conversations, uses automated methods to generate additional negative training data, and then readjusts model parameters within a fixed training cycle in order to correct the model's tendency to make errors on specific problems.

[0004] However, this existing solution has significant flaws: its optimization process lags seriously behind the interaction and cannot respond promptly to errors that occur immediately during the conversation; the method of relying on historical data and preset rules to generate adversarial samples has weak generalization ability and cannot effectively capture new complex, dynamic or context-dependent error patterns; more importantly, it completely ignores the user's operational behavior data during the conversation, resulting in rough error positioning and lack of targeted optimization, making it impossible to achieve truly autonomous and precise iteration. Summary of the Invention

[0005] This application provides a large-model-based intelligent customer service dialogue generation and optimization method and system to solve the problems of severe lag in error response optimization, detachment from user feedback, and weak generalization ability in the existing offline adversarial training scheme.

[0006] In a first aspect, this application provides a method for generating and optimizing intelligent customer service dialogues based on a large model, including: Collect data on repeated user questions, manual rewriting, cursor traces, text deletions, and voice interruptions, and associate them with conversation timestamps to form the original dataset. Calculating an operation intensity value and an error frequency value based on the original data set to generate a modified heat map with time annotations, operation intensity annotations, and error frequency annotations; The error location distribution is determined by analyzing the correlation between the correction heat map and the conversation context. Node elements carrying intensity values ​​are defined based on the operation density values ​​of the error location distribution. At the same time, edge elements recording trigger conditions are defined based on the error frequency values ​​of the error location distribution. The node elements and edge elements are integrated to construct a dynamic error map for large model optimization. Calling the dynamic error graph, updating the strength values ​​of the node elements and the error frequency values ​​of the edge elements when new user correction data is added, and generating an adversarial sample adapted to the large model input based on the highest strength value and the error frequency value of the corresponding edge element in the updated dynamic error graph; The adversarial samples are input into the large model as negative rewards using a reinforcement learning mechanism. The response module is fine-tuned for the error type corresponding to the highest intensity value. The fine-tuned response module is driven to output optimized dialogue content based on the current dialogue process. At the same time, the dialogue content is fed back to the update process of the dynamic error map, forming a closed-loop control of dialogue generation and optimization.

[0007] Optionally, the reinforcement learning mechanism is used to input the adversarial examples as negative rewards into the large model, fine-tune the response module for the error type corresponding to the highest intensity value, and drive the fine-tuned response module to output optimized dialogue content based on the current dialogue process. At the same time, the dialogue content is fed back to the update process of the dynamic error map, forming a closed-loop control of dialogue generation and optimization, including: Input the adversarial examples as negative rewards into the large model training environment through the reinforcement learning mechanism; Locating the node element corresponding to the highest intensity value in the dynamic error graph, extracting the error type corresponding to the node element as the target error type, and modifying the parameters of the response module according to the target error type to reduce the probability of errors in the dialogue generation; Based on the context of the current conversation process, the response module with adjusted parameters is driven to output optimized conversation content; The conversation content is fed back to the dynamic error map update process as newly added user correction data, forming a closed-loop control of conversation generation and optimization.

[0008] Optionally, feeding back the conversation content as newly added user correction data to the dynamic error map update process to form a closed-loop control of conversation generation and optimization includes: Convert the conversation content into newly added user correction data containing the conversation text and corresponding timestamp; Inputting the newly added user correction data into the update process of the dynamic error map to trigger the update operation of the strength value of the node element and the error frequency value of the edge element; The adversarial samples are regenerated through the updated dynamic error map, and the reinforcement learning optimization process is carried out to form a closed-loop control with continuous optimization.

[0009] Optionally, calling the dynamic error graph, updating the strength values ​​of the node elements and the error frequency values ​​of the edge elements when new user correction data is added, and generating an adversarial sample adapted to the large model input based on the highest strength value and the error frequency value of the corresponding edge element in the updated dynamic error graph, includes: The dynamic error map is called. When new user correction data is added, the operation intensity value of the newly added error position is used as the strength value of the new node element, the latest operation intensity value of the existing error position is used as the strength value of the corresponding node element, and the latest error frequency value of the adjacent position is used as the trigger condition value of the edge element; Traverse all node elements in the updated dynamic error graph and locate the node element with the highest intensity value as the target node; Extracting the original conversation text corresponding to the error position of the target node, and obtaining the maximum error frequency value in the edge elements associated with the target node, and using the error type corresponding to the maximum value as the core error feature; The original conversation text is modified, and the core error features are injected to generate adversarial samples that contain similar errors and are adapted to the input format of the large model.

[0010] Optionally, the error location distribution is determined by analyzing the association between the correction heat map and the conversation context, a node element carrying an intensity value is defined based on the operation density value of the error location distribution, and an edge element recording a trigger condition is defined based on the error frequency value of the error location distribution, and the node elements and the edge elements are integrated to construct a dynamic error map for large model optimization, including: Analyze the correlation between each round label and the conversation context in the correction heat map, locate the rounds in the conversation process where the operation density value exceeds the density threshold or the error frequency value exceeds the frequency threshold as error positions, and integrate the error positions to form an error position distribution; Creating a node element based on the operation intensive value of the error position distribution, and assigning the operation intensive value of each error position as a strength value to the corresponding node element; Based on the error frequency values ​​of the error position distribution, and taking the error frequency value of the previous position in the adjacent error positions as a trigger condition, creating an edge element connecting two node elements; The node elements and the edge elements are integrated in a dialog flow order to construct a dynamic error graph for large model parameter optimization.

[0011] Optionally, the calculating the operation intensity value and the error frequency value based on the original data set to generate a modified heat map with time annotation, operation intensity annotation, and error frequency annotation includes: Divide the original data set into conversation turns starting from user input and ending with intelligent customer service output; The operation intensity value is calculated by counting the number of manual rewrites, text deletions, and total cursor movement distance in a single round of conversation. The error frequency value is also calculated by counting the number of repeated questions and voice interruptions in a single round of conversation. The start and end timestamps of each round of dialogue are used as time labels, the operation intensity value is used as the operation intensity label, and the error frequency value is used as the error frequency label; A two-dimensional grid is constructed with the time axis as the horizontal axis and the conversation round as the vertical axis, and the time annotation, the operation intensive annotation, and the error frequency annotation of each conversation round are mapped to the corresponding two-dimensional grid area to generate a correction heat map.

[0012] Optionally, the collecting of user repeated question data, manual rewriting data, cursor trajectory data, text deletion data, and voice interruption data, and associating and recording conversation timestamps to form an original data set includes: Monitor the conversation interface between users and intelligent customer service, collecting data on repeated questions, manual rewriting, cursor tracks, text deletions, and voice interruptions; Independent operation units with operation type identifiers are generated based on the user repeated question data, the manual rewriting data, the cursor track, the text deletion data and the voice interruption data, and the independent operation units are merged in timestamp order to generate an original data set.

[0013] In a second aspect, this application provides a large-model-based intelligent customer service dialogue generation and optimization system, including: The collection module is used to collect data on repeated questions, manual rewriting, cursor tracks, text deletions, and voice interruptions, and associates the recorded conversation timestamps to form the original data set; A generation module, configured to calculate an operation intensity value and an error frequency value based on the original data set to generate a corrected heat map with time annotations, operation intensity annotations, and error frequency annotations; A construction module is configured to determine an error location distribution by analyzing the correlation between the correction heat map and the conversation context, define a node element carrying an intensity value based on the operation density value of the error location distribution, and define an edge element recording a trigger condition based on the error frequency value of the error location distribution, and integrate the node elements and the edge elements to construct a dynamic error map for large model optimization; An updating module, configured to call the dynamic error graph, update the strength values ​​of the node elements and the error frequency values ​​of the edge elements when new user correction data is added, and generate an adversarial sample adapted to the large model input based on the highest strength value and the error frequency value of the corresponding edge element in the updated dynamic error graph; A feedback module is used to input the adversarial samples as negative rewards into the large model using a reinforcement learning mechanism, fine-tune the response module according to the error type corresponding to the highest intensity value, and drive the fine-tuned response module to output optimized dialogue content based on the current dialogue process. At the same time, the dialogue content is fed back to the update process of the dynamic error map, forming a closed-loop control of dialogue generation and optimization.

[0014] In a third aspect, the present application provides a computing device comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a large-model-based intelligent customer service dialogue generation and optimization method as described in the first aspect above.

[0015] In a fourth aspect, the present application provides a computer storage medium storing a computer program. When the computer program is executed by a computer, it implements a large model-based intelligent customer service dialogue generation and optimization method as described in the first aspect.

[0016] In the example of the present application, the user's repeated question data, manual rewriting data, cursor trajectory, text deletion data and voice interruption data are collected, and the conversation timestamps are associated and recorded to form an original data set; the operation intensity value and error frequency value are calculated based on the original data set to generate a correction heat map with time annotation, operation intensity annotation and error frequency annotation; the error position distribution is determined by analyzing the association between the correction heat map and the conversation context, and the node elements carrying intensity values ​​are defined based on the operation intensity values ​​of the error position distribution, and the edge elements of the recording trigger conditions are defined based on the error frequency values ​​of the error position distribution, and the node elements and the edge elements are integrated to construct a large-scale model. The method is to generate an optimized dynamic error map; call the dynamic error map, update the strength value of the node element and the error frequency value of the edge element when new user correction data is added, and generate an adversarial sample adapted to the large model input according to the highest strength value and the error frequency value of the corresponding edge element in the updated dynamic error map; use the reinforcement learning mechanism to input the adversarial sample as a negative reward into the large model, fine-tune the response module according to the error type corresponding to the highest strength value, and drive the fine-tuned response module to output the optimized dialogue content based on the current dialogue process, and at the same time feed the dialogue content back to the update process of the dynamic error map, forming a closed-loop control of dialogue generation and optimization.

[0017] The technical solution of this application has the following beneficial effects: This application constructs the original data set by collecting user interaction behavior data and associating timestamps, and generates a multi-dimensionally labeled correction heat map based on operation intensity values ​​and error frequency values; forms a dynamic error map by analyzing the relationship between the heat map and the conversation context, and quantifies the error mode with intensity value nodes and trigger condition edges; updates the map when new user correction data is added, and dynamically generates adversarial samples based on the peak characteristics of the map; uses the reinforcement learning negative feedback mechanism to drive the large model to fine-tune the response module in seconds, specifically optimizes the error type corresponding to the highest intensity value, and feeds the optimization results back to the map update, ultimately forming a full-process closed-loop self-evolution system from data collection, heat map analysis, map construction, sample generation, model optimization to effect verification, to achieve precise positioning, dynamic response and continuous autonomous optimization of intelligent customer service conversation errors.

[0018] Furthermore, by inputting adversarial samples into the large model training environment as negative rewards, the error type corresponding to the node with the highest intensity value in the dynamic error graph is located, and the response module parameters are modified in a targeted manner to reduce the probability of generating such errors. The fine-tuned response module outputs optimized content based on the current conversation context, and the optimization results are finally fed back into the graph update as new correction data to achieve closed-loop control. This solution accurately suppresses the tendency to generate target errors through a negative reward mechanism, outputs optimized content based on the conversation context, and feeds the optimization results back into the graph update, forming a self-driven error location, parameter adjustment, effect verification, and graph evolution cycle system, achieving targeted, scenario-adaptive, and continuously evolving capabilities for intelligent customer service error correction.

[0019] These and other aspects of the present application will become more readily apparent from the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A flowchart of a method for generating and optimizing intelligent customer service dialogues based on a large model provided by this application is shown; Figure 2 A scenario diagram showing a method for generating and optimizing intelligent customer service dialogues based on a large model provided by this application is shown; Figure 3 The following is a schematic diagram of the structure of a large-model-based intelligent customer service dialogue generation and optimization system provided by this application; Figure 4A schematic structural diagram of a computing device provided by the present application is shown. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0023] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0024] Research has shown that current error-repair technologies in intelligent customer service systems suffer from three major problems: First, they only analyze manually curated historical complaint records, ignoring actual user behavior during conversations (such as repeatedly revising questions or abandoning conversations mid-sentence). This leads to a mismatch between the system's understanding of errors and actual usage; second, the systems are updated on a fixed schedule (e.g., weekly), making them unable to quickly identify emerging error types, resulting in a constant delay in problem resolution; and finally, the systems rely on pre-set rules to simulate possible error scenarios, but these simulations are overly simplistic and unable to cope with the ever-changing and complex error situations in real conversations. These issues trap existing systems in a vicious cycle of "finding errors, simple fixes, and then generating new errors."

[0025] To address these issues, a new intelligent customer service optimization method has been developed. First, the system records various user actions during conversations (such as cursor movements and text deletions). This behavioral data creates an "error heat map," visualizing which steps are most prone to errors. Next, the system automatically generates training materials based on the most common error types, using these materials to continuously optimize the customer service system's response capabilities. Finally, the system uses the optimized conversation results as new learning material to continuously update the error heat map. This method effectively addresses the shortcomings of existing technologies: accurately locating errors through real user behavior, avoiding repair delays through updates, and adapting to various complex errors through continuous learning, significantly improving the accuracy and self-improvement capabilities of intelligent customer service.

[0026] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0027] Figure 1 A flowchart of a method for generating and optimizing intelligent customer service dialogues based on a large model is provided for the embodiment of this application. Figure 1 As shown, the method includes: 101. Collect data on repeated user questions, manual rewriting, cursor traces, text deletions, and voice interruptions, and associate them with recorded conversation timestamps to form the original dataset; Optionally, step 101 may specifically include the following steps: 1011. Monitor the conversation interface between users and intelligent customer service, and collect data on repeated questions, manual rewriting, cursor tracks, text deletions, and voice interruptions; 1012. Generate independent operation units with operation type identifiers based on the user repeated question data, the manual rewriting data, the cursor track, the text deletion data, and the voice interruption data, and merge the independent operation units in timestamp order to generate an original data set.

[0028] In the above solution, user repeated question data refers to behavioral signals that indicate users have not received satisfactory answers. This data is used to identify key instances of inadequate responses or misunderstandings. Manual rewrite data records user actions that proactively request changes or rewrites of customer service responses (e.g., clicking "Rewrite" or "Unsatisfied" buttons or entering rewrite suggestions in the feedback box). This data is used to identify inaccurate or missing paragraphs. Cursor trajectory data captures the path of the user's cursor over the text of customer service responses, revealing points of comprehension barriers or areas of focus for key information. Text deletion data refers to behavioral features that mark when users delete their own input (questions or feedback) within the input box. This data is used to identify changes in user intent, difficulties in expression, or dissatisfaction with previous interactions. Voice interruption data records the time points when users interrupt customer service during voice conversations. This data is used to detect key points where responses do not meet user expectations. The original dataset is a time-structured collection of the aforementioned behavioral signals, consisting of sequences of time-stamped action units.

[0029] In this embodiment, step 1011 utilizes an embedded front-end monitoring module to perform multi-dimensional data collection. When a user submits a text question, the system records the continuously entered content to collect repeated user question data. When a user triggers an interaction to rewrite a customer service response (e.g., clicking a "rewrite" button or entering a suggested change in a designated area), the system captures the user-initiated rewrite request or suggested content to collect manual rewrite data. When the cursor moves within the customer service response area, the system samples the cursor's absolute screen coordinate sequence at 100 millisecond intervals to collect cursor trajectory data. When a user deletes within their input box, the system monitors native browser input events and extracts the selection anchor and focus offset to collect text deletion data. When a voice conversation is interrupted, the system marks the state transition time of the speech synthesis interface and the associated audio segment identifier to collect voice interruption data. All collected raw behavioral signals are associated with a high-precision time counter to generate an unprocessed signal stream. For example, if user D enters a refund-related question twice in a ticket customer service conversation, the system records the two textual entries as repeated user question data. When user D clicks the "unsatisfied" button in response to a non-refundable clause, the system captures the rewrite request. .

[0030] Next, in step 1012, based on the original signal stream collected in step 1011, the streaming processing engine performs structured conversion and merging: Text similarity calculation is performed on the collected user repeated question data, and valid repeated question instances are identified using TF-IDF vectorization and cosine similarity algorithms to generate structured user repeated question data units with operation type identifiers; the collected manual rewrite data (rewrite requests / suggestions) is parsed to extract the user-requested modifications or feedback text to generate standardized manual rewrite data units; the original coordinate sequence of the collected cursor trajectory data is mapped to the row and column positions of the customer service reply text area to generate cursor trajectory data units with temporal path features; the native selection offsets in the collected text deletion data are converted to start and end index values ​​to generate deletion information units for the precisely located user input content; and the timestamps and voice segment identifiers in the collected voice interruption data are associated to generate traceable voice interruption data units. Event time window processing technology is then used to dynamically sort the five types of structured data units in the order of occurrence based on high-precision timestamps, eliminating the disorder caused by network transmission. Finally, the ordered data unit sequence is encoded into JSONLines format. Each record contains the ISO 8601 standard timestamp, operation type enumeration value and feature object, generating a strictly time-series original data set.

[0031] For example, the original signal collected by user D, two text submissions ["Logistics Delay Reason", "Logistics Delay Reason"] are converted into dataset entries: {"timestamp":T1,"operation_type":"repeat","features":{"content":"Reasons for logistics delays"}}.

[0032] For the operation of clicking "dissatisfaction", the generated dataset entries are as follows: {"timestamp":T2,"operation_type":"rewrite_request","features":{"trigger":"dissatisfaction_button"}}.

[0033] In practical applications, taking the scenario of users shopping on e-commerce platforms as an example, the complete interaction chain between users and intelligent customer service is clearly demonstrated how multi-dimensional behavioral data is converted into structured data sets: The first is the interaction process and data collection stage: 2023-05-14 10:30:15 User C enters the following for the first time: "Order number FH20230514001 shows it has been out of stock for three days. Can it be delivered today?" (The system records this as the initial question) 2023-05-14 10:30:22 Customer service AI response: [System reply] Latest status of order FH20230514001: 05-12 08:30 East China Transshipment Warehouse dispatched 05-13 14:20 Arrived at North China Distribution Center Due to the holiday peak, delivery is expected to be delayed by 1-3 days 2023-05-14 10:30:25 Text deletion: The user deleted the text "I want to complain about logistics!" in the input box (selected position: anchor 0 - focus 6). The system detected the input event: {inputType: "deleteContentBackward", data: null}.

[0034] 2023-05-14 10:30:28 User cursor trajectory: The mouse moves back and forth over "Delay 1-3 days" (generating high-density coordinate points), and the system collects the coordinate sequence [{x:205,y:378,t:1684038628400}...{x:220,y:378,t:1684038632400}].

[0035] 2023-05-14 10:30:35 The user suddenly inputs: "No! It must be delivered today!" (interrupting the subsequent text broadcast by the AI), and the system marks the time when the speech synthesis was interrupted.

[0036] 2023-05-14 10:30:40 The user rewrites the request. For example, if the "urgent processing" button is clicked below the reply, the system captures the customized rewrite request.

[0037] 2023-05-14 10:31:05 Repeated question: User input: "What is the contact number of the North China Distribution Center? I will follow up on the shipment myself."

[0038] Next, the data is converted into a stream processing engine. The stream processing engine captures the corresponding data through the above interactive process and converts it into structured data units, as shown in Table 1 below: Table 1 Original signal Conversion Logic Structured data unit Initial question: Text content Direct storage semantics { "timestamp":"2023-05-14T10:30:15.200Z", "operation_type":"initial_question", "features":{ "content":"Order number FH20230514001 shows it has been out of stock for three days. Can it be delivered today?", "urgency_flag":true } } Text deletion: Delete 6 characters 1. Restore deleted content 2. Associated emotional intensity { "timestamp":"2023-05-14T10:30:25.800Z", "operation_type":"deletion", "features":{ "deleted_content":"I want to complain about logistics!", "emotion_level":2 / / Anger level } } Cursor track: high-density coordinate points 1. Map to reply line 3 2. Calculate the scan frequency { "timestamp":"2023-05-14T10:30:28.400Z", "operation_type":"cursor", "features":{ "line":3, "target_text":"Delay 1-3 days", "scan_freq":4.2Hz / / scan 4.2 times per second } } Voice interruption: interruption time point Associate the current broadcast field { "timestamp":"2023-05-14T10:30:35.100Z", "operation_type":"voice_interrupt", "features":{ "related_text":"Expected delivery delay of 1-3 days", "interrupt_type":"demand_override" / / demand override interrupt } } Rewrite request: Click the "Urgent Processing" button Parsing button semantics { "timestamp":"2023-05-14T10:30:40.600Z", "operation_type":"rewrite_request", "features":{ "trigger":"emergency_button", "expected_action":"contact_logistics" } } Repeat question: New input 1. Semantic analysis 2. Associated distribution center nodes { "timestamp": "2023-05-14T10:31:05.300Z", "operation_type": "repeat", "features": { "content": "What is the contact number for the North China Distribution Center? I will urge the shipment myself." "focus_node": "North China Distribution Center" } } Finally, the system generates the original data set (JSON Lines) based on the obtained structured data units, as shown in the following JSON code: {"timestamp":"2023-05-14T10:30:15.200Z","operation_type":"initial_question","features":{"content":"Order number FH20230514001 shows it has been out of stock for three days. Can it be delivered today?","urgency_flag":true}} {"timestamp":"2023-05-14T10:30:25.800Z","operation_type":"deletion","features":{"deleted_content":"I want to complain about the logistics!","emotion_level":2}} {"timestamp":"2023-05-14T10:30:28.400Z","operation_type":"cursor","features":{"line":3,"target_text":"Delay 1-3 days","scan_freq":4.2}} {"timestamp":"2023-05-14T10:30:35.100Z","operation_type":"voice_interrupt","features":{"related_text":"Expected delivery delay of 1-3 days","interrupt_type":"demand_override"}} {"timestamp":"2023-05-14T10:30:40.600Z","operation_type":"rewrite_request","features":{"trigger":"emergency_button","expected_action":"contact_logistics"}} {"timestamp":"2023-05-14T10:31:05.300Z","operation_type":"repeat","features":{"content":"What is the contact number for the North China Distribution Center? I will urge the shipment myself","focus_node":"North China Distribution Center"}} The above-mentioned 101 overall solution collects multi-dimensional user interaction behaviors (focused on correcting them to only operate the user's own input or trigger feedback on customer service replies) and constructs a time-series raw data set. This provides basic data support that comprehensively and truly reflects user intentions for subsequent error analysis, significantly improving the accuracy of error location and the reliability of data-driven optimization.

[0039] 102. Calculate an operation intensity value and an error frequency value based on the original data set to generate a modified heat map with time annotations, operation intensity annotations, and error frequency annotations; Optionally, step 102 may specifically include the following steps: 1021. Divide the original data set into conversation rounds starting from user input to intelligent customer service output; 1022. Count the number of manual rewrites, text deletions, and total cursor movement distance in a single round of conversation to calculate the operation intensity value. Also count the number of repeated questions and voice interruptions in a single round of conversation to calculate the error frequency value. 1023. Use the start and end timestamps of each round of dialogue as time labels, use the operation intensity value as an operation intensity label, and use the error frequency value as an error frequency label; 1024. Construct a two-dimensional grid with the time axis as the horizontal axis and the conversation round as the vertical axis, and map the time annotation, the operation intensive annotation, and the error frequency annotation of each conversation round to corresponding two-dimensional grid areas to generate a correction heat map.

[0040] In the above scheme, the operation intensity value refers to a dynamic quantitative indicator that reflects the intensity of user modification of the intelligent customer service reply content. It includes comprehensive operation characteristics of manual rewriting behavior, text deletion behavior and cursor movement behavior, and can be used to locate high-risk reply segments with user comprehension barriers or missing information. The error frequency value refers to a frequency indicator that characterizes the degree of user dissatisfaction with customer service replies. It includes trigger characteristics of repeated questioning behavior and voice interruption behavior, and can be used to identify systemic defects such as inaccurate answers or delayed responses. The correction heat map refers to a two-way visual data carrier that integrates the time dimension and dialogue turns. It contains three-dimensional information of time annotation, operation intensity annotation and error frequency annotation. It can be used to reveal the error distribution pattern and provide a spatiotemporal basis for the construction of dynamic error maps.

[0041] In the embodiment of the present application, step 1021 is used to read the timestamp sequence in the original data set, and the user input start event is used as the start point of the round. If there is no new user event for 5 consecutive seconds after the customer service outputs the end event, the original data is cut into independent conversation rounds. The session timeout detection algorithm is used during cutting. If the interval between adjacent user inputs is less than 5 seconds, they are merged into the same round to avoid splitting continuous questions. For example, user A asks "How do I refund the ticket?" at 10:00:01, which is marked as the start point of the round. The customer service replies at 10:00:05 and ends. User A then asks "Is there a handling fee?" at 10:00:06. The system merges them into the same round until the customer service replies for the second time at 10:00:10 and there is no new input for 5 seconds. Finally, a single-round conversation data packet containing two user questions is generated.

[0042] Next, based on the single-round conversation data packets segmented in step 1021, the operation intensity value and error frequency value are calculated simultaneously. The user operation behavior within a single round is analyzed. For the operation intensity value, the number of manual rewrites is counted, and the number of text deletions with a single deletion of ≥5 characters is accumulated. The cursor movement trajectory coordinate sequence is extracted. The total cursor movement distance is obtained by calculating the Euclidean distance between adjacent coordinate points and accumulating them. The operation intensity value is then calculated using a weighted calculation formula. The calculation formula is as follows: , where 12 is the standard character width in pixels. For example, if a round records three deletion operations and two rewrite operations, and the total cursor trajectory distance is 360 pixels, then the operation intensity value calculated by the formula is 10.8. Simultaneously analyze the conversation event stream, identify repeated questions using a semantic similarity model, and count the number of times the customer service voice output is interrupted by the user's voice input. The formula: Calculate the error frequency value. For example, if a user asks a question about refund policy for the first time in a round and the customer service representative interrupts them once during their response, and the user asks a similar question again, the error frequency value is calculated to be 3.

[0043] Then, step 1023 receives the single-round dialogue operation density value and error frequency value output by step 1022, and synchronously combines the original round time information segmented by step 1021 to perform three-dimensional data binding, extracts the precise start and end timestamps from the individual round data packets, and rounds the millisecond timestamps to the second level through time normalization and formats them into a "start time-end time" string; then, the operation density value is written as an operation density annotation, and the error frequency value is written as an error frequency annotation; finally, these three types of data are merged into a structured annotation triple. For example, in the flight cancellation dialogue, user B's second round of interaction is divided into independent rounds within the time range of 10:00:30-10:00:40. Based on the four deletions and 480 pixel cursor movements in the second round, the corresponding operation intensity value is calculated to be 12.0. At the same time, based on the two repeated questions, the error frequency value is calculated to be 4. These three types of data are combined to generate a triplet {time: "10:00:30-40", operation intensity value: 12.0, error frequency value: 4}. This triplet is stored in the cache queue as the basic unit for heat map generation.

[0044] Finally, if Figure 2 As shown, step 1024 constructs a corrected heat map based on the annotated triples. First, the global time range of the conversation is determined, and the horizontal axis is divided into 300 continuous time grids at 1-second intervals. The vertical axis generates a number sequence according to the order of the conversation turns. Then, all triples are traversed to perform grid mapping. Taking the second round of user B's data as an example, its time annotation "10:00:30-40" corresponds to the 31st to 40th seconds on the horizontal axis and the second row on the vertical axis. The system writes the operation density value 12.0 and the error frequency value 4 for this round into each cell of these 10 grids. Finally, the grid matrix is ​​rendered in two channels through the visualization engine. The difference in operation density values ​​can be set to: low density (0-5): sparse thin gray lines and large-spaced dashed lines; medium density (5-10): medium-density solid line grid; high density (>10): dense thick black lines with densely crossed grids. The difference in error frequency values ​​can be controlled by controlling the transparency. The higher the error frequency value, the less transparent it is, generating a directly recognizable spatiotemporal error distribution heat map. For example, in Figure 2 Medium, low density and low error: gray thin line + long dashed line, the grid is sparse; medium density and low error: gray medium line + medium dashed line, the grid is medium dense; medium density and medium error: black solid line, the grid is medium dense; high density and high error: black thick solid line, the grid is the densest.

[0045] In a real-world application, in a certain intelligent customer service system, user C started asking about ticket changes at 14:00:01.500, and the customer service representative finished replying at 14:00:05.200. User C then asked, "How can I avoid the handling fee?" at 14:00:05.800, and the customer service representative replied again at 14:00:10.300. Because the interval between the follow-up questions was less than 5 seconds, the two questions were combined into a single round of dialogue, with a time range of 14:00:01-14:00:10. The system analyzed the behavior of this round and found that the user deleted the original input 5 times, rewrote it manually 3 times, and the cursor trajectory coordinate sequence [(100,200), (120,210), (150,230)] moved a distance of Pixels, calculations ;in It means square root. At the same time, it is detected that after the user asked the first question, he was interrupted once in the customer service's first reply. The semantic similarity between the follow-up question and the first question reached 90%. The system then binds the time information "14:00:01-11" to generate the triplet {time: "14:00:01-11", operation intensity value: 4.36, error frequency value: 3}; finally, a heat map is constructed: the horizontal axis is divided into 60 grids from 14:00:00 to 14:01:00, the vertical axis is the first row, and this round occupies grids 2-12 on the horizontal axis. Each grid is written with (4.36, 3), corresponding to an operation intensity value of 4.36 and an error frequency value of 3, and is rendered as a yellow semi-transparent block.

[0046] The overall solution described above, 102, transforms user interactions into spatiotemporally annotated correction heatmaps, enabling intuitive and visual identification of conversation error patterns. Based on quantitative calculations of operation intensity and error frequency, the system automatically identifies conversation turns and time periods with high modification intensity and high error frequency, providing precise spatiotemporal data support for the subsequent construction of dynamic error maps. This significantly improves the objectivity of error detection and the efficiency of localization, eliminating reliance on subjective judgment.

[0047] 103. Determine the error location distribution by analyzing the correlation between the correction heat map and the conversation context, define node elements carrying intensity values ​​based on the operation density values ​​of the error location distribution, and define edge elements recording trigger conditions based on the error frequency values ​​of the error location distribution, and integrate the node elements and edge elements to construct a dynamic error map for large model optimization; Optionally, step 103 may specifically include the following steps: 1031. Analyze the association between each round label and the conversation context in the correction heat map, locate the rounds in the conversation process where the operation density value exceeds the density threshold or the error frequency value exceeds the frequency threshold as error locations, and integrate the error locations to form an error location distribution; 1032. Create a node element based on the operation intensive value of the error position distribution, and assign the operation intensive value of each error position as a strength value to the corresponding node element; 1033. Based on the error frequency values ​​of the error position distribution, and with the error frequency value of the previous position among adjacent error positions as a trigger condition, create an edge element connecting two node elements; 1034. Integrate the node elements and the edge elements in the order of the dialogue process to construct a dynamic error graph for large model parameter optimization.

[0048] In the above scheme, the error location distribution is the set of abnormal rounds located by analyzing the temporal association between the heat map and the conversation context, which meets the judgment conditions that the operation density value exceeds the preset density threshold or the error frequency value exceeds the frequency threshold. The node element is a topological unit that carries the error location. The strength value it carries directly maps the quantitative result of the operation density value of the location and is used to characterize the urgency of error correction. The edge element is a directed association unit connecting adjacent nodes. The trigger condition it records is derived from the statistical output of the error frequency value of the previous node and is used to describe the induction probability between error types. The dynamic error map is a mesh structure constructed by integrating node elements and their associated edge elements. The node strength value and edge trigger condition can be updated as new user data is added, providing an adaptive error relationship model for generating adversarial samples.

[0049] In this embodiment of the present application, step 1031 is first used to extract the operation density value and error frequency value of each round of dialogue from the corrected heat map. Then, a dual-threshold judgment mechanism is used to compare the heat map annotation with the dialogue context in chronological order. If the operation density value of a round exceeds the preset density threshold of 2.0 or the error frequency value exceeds the frequency threshold 5 times, the round is marked as an error position. Finally, all marked rounds are integrated to form an error position distribution set. For example, in a medical customer service scenario, the operation density value of the fifth round of dialogue is 2.2, which is greater than the density threshold, and the error frequency value is 7, which is greater than the error threshold. The system includes it in the error position distribution set.

[0050] Next, based on the error location distribution set output in step 1031, a separate node element is created for each marked error location. The node ID is bound to the specific conversation turn and error type, and the operation intensity value of the location is directly converted into a node strength value, forming a topological unit that quantifies the urgency of error correction. For example, in an insurance customer service scenario, the operation intensity value of 2.0 for the claim process error in turn 5 of the error location distribution is directly assigned a node strength value, generating a node element with the attributes {turn: 5, error type: "claim process", strength: 2.0}, achieving a precise mapping of error severity and node weight.

[0051] Then, in step 1033, the node elements generated in step 1032 are scanned sequentially according to the dialog flow. Directed edge elements are created for consecutively occurring adjacent error node elements. The triggering conditions for the edge are defined based on the error frequency value of the previous node element. Specifically, the error frequency value of the node element is directly converted into an edge attribute, forming association rules that describe the patterns of induction between error types. For example, in a government customer service scenario, when nodes A and B are adjacent, the system creates a directed edge from A to B and sets the error frequency value of node A at 8 as the trigger condition. This generates a rule chain: "When household registration errors accumulate to 8 times, a social security transfer consultation error will be triggered." This achieves quantitative modeling of error propagation relationships.

[0052] Finally, in step 1034, based on the node elements generated in step 1032 and the edge elements constructed in step 1033, all node elements are topologically sorted according to the order of conversation turns, and each node element with a strength value is arranged along the time axis. Furthermore, according to the association rules defined in step 1033, adjacent nodes with causal relationships are connected using edge elements carrying trigger conditions. Ultimately, this integration forms a dynamic error graph, which uses nodes to represent the specific error location and urgency, and edges to describe the triggering rules for error propagation. For example, in a bank customer service scenario, the graph connects the "annual fee policy error" node, ranked as strength 2.1 in round 3, with the "transfer limit error" node, ranked as strength 1.8 in round 7, through an edge with the trigger condition "accumulated six annual fee errors," thus constructing an updateable error evolution network model.

[0053] In a practical application, in an e-commerce customer service conversation scenario, the system first analyzes the correction heatmap and detects that the user's response to the "shipping rules" in the third round has an action density of 1.5, which is less than the density threshold of 2.0. However, this error type has occurred 7 times in nearly 200 conversations, exceeding the frequency threshold of 5. Therefore, the third round is still marked as an error location. Subsequently, the action density of the fifth round is 2.4, which is greater than the threshold of 2.0, so the fifth round is also marked, resulting in the error location distribution set {Round 3, Round 5}. Nodes are then created for both locations, with the Round 3 node's strength value directly inheriting its action density value of 1.5 and the Round 5 node's strength value of 2.4. Adjacent nodes are then processed in conversation order. Because the Round 3 node has an error frequency of 7, a directed edge is created to the Round 5 node, with the trigger condition set to 7 occurrences. Finally, the node and edge elements are integrated to construct a dynamic error graph, forming a causal chain model: "When a user expresses dissatisfaction with the explanation of the shipping rules seven times, the probability of misunderstanding in subsequent inquiries about insurance policies increases significantly."

[0054] The overall solution of 103 mentioned above achieves three core optimizations by converting user interaction behaviors into structured error networks. First, it establishes a visual mapping of error severity, using node strength values ​​to intuitively identify dialogue defects that need to be addressed first. Second, it reveals the error chain patterns and accurately predicts potential related errors through edge triggering conditions. Finally, it forms a dynamically evolving error relationship model, enabling the system to actively track error propagation paths and predict risks, providing a causally explainable decision-making framework for targeted optimization, significantly improving the timeliness and continuous evolution of error correction.

[0055] 104. Calling the dynamic error graph, updating the strength values ​​of the node elements and the error frequency values ​​of the edge elements when new user correction data is added, and generating an adversarial sample adapted to the large model input based on the highest strength value and the error frequency value of the corresponding edge element in the updated dynamic error graph; Optionally, step 104 may specifically include the following steps: 1041. Call the dynamic error map. When new user correction data is added, the operation intensity value of the newly added error position is used as the strength value of the new node element, the latest operation intensity value of the existing error position is used as the strength value of the corresponding node element, and the latest error frequency value of the adjacent position is used as the trigger condition value of the edge element. 1042. Traverse all node elements in the updated dynamic error graph and locate the node element corresponding to the highest intensity value as the target node; 1043. Extract the original conversation text corresponding to the error position of the target node, and simultaneously obtain the maximum error frequency value among the edge elements associated with the target node, and use the error type corresponding to the maximum value as the core error feature; 1044. Modify the original conversation text and inject the core error features to generate an adversarial sample that contains similar errors and is adapted to the input format of the large model.

[0056] In the above solution, core error features refer to the essential properties of high-frequency error patterns extracted from the dynamic error graph. These include the error type corresponding to the maximum error frequency among the edge elements associated with the target node. These features can be used to guide the precise error injection of adversarial examples. Adversarial examples are specially constructed input data designed to expose flaws in the large model. These contain core error features intentionally implanted in the original conversation text. They can be used to trigger the large model to generate specific error responses to drive reinforcement learning optimization.

[0057] In the embodiment of the present application, first, when the system detects newly added user correction data through step 1041, the specific position of the error in the conversation text is located; if the position does not exist in the existing graph, the same operation intensive value calculation process as step 102 is executed, and the calculation result is directly used as the strength value of the new node element; if a corresponding node already exists at the position, the latest operation intensive value is synchronized from the updated correction heat map; at the same time, the error operations adjacent to the position are scanned, and the latest statistical error co-occurrence frequency value is obtained from the heat map, and this value is directly assigned to the associated edge element as the trigger condition value, ultimately achieving dynamic updates of the graph node strength and edge error frequency in seconds. For example, the user continuously deletes the word "power cord" in the customer service reply "Please disconnect the power cord first". After determining it as a new error position, the system calculates the operation intensive value as 1.2 and creates a new node element with a strength value of 1.2; it is simultaneously detected that the user subsequently adds "an insulated screwdriver is required" at the adjacent position, obtains the latest co-occurrence frequency value of 85% of the "delete-add" operation from the heat map, and updates the associated edge attributes.

[0058] Next, all node elements in the updated dynamic error graph are traversed through step 1042: the maximum strength value variable and the temporary target node storage pointer are initialized; starting from the root node of the graph, each node is recursively visited, and the magnitude relationship between the current node strength value and the maximum strength value is compared; when traversing to node B, if its strength value 1.2 is greater than the current maximum value 0.8, the maximum value is updated to 1.2 and the target candidate is switched to node B; after all nodes are visited, the node corresponding to the final recorded maximum strength value is output as the target node, which represents the most prominent error location in the current user correction behavior and urgently needs to be optimized. For example, the updated graph contains three nodes, node X corresponding to strength 0.6, node Y corresponding to strength 0.9, and node Z corresponding to strength 1.1; when the traversal starts, the maximum strength value is initially set to 0. When visiting node X, 0.6>0, the maximum value is updated to 0.6 and temporarily stored in X; when visiting node Y, 0.9>0.6, the maximum value is updated to 0.9 and stored in Y; finally, when visiting node Z, 1.1>0.9, the maximum value is updated to 1.1, and Z is locked as the target node.

[0059] After determining the target node, two key operations are performed to prepare for adversarial sample generation: Based on the location index information recorded in the target node, the original text content before the user performed the correction operation is accurately retrieved from the conversation history database; all edge elements connected to the target node are retrieved, and the edge element with the highest error frequency value is selected using a maximum comparison algorithm. The error type recorded by this edge element is then identified as the core error feature to be injected into the adversarial sample. For example, if the target node is located at the position of the sentence "Thawing takes 30 minutes," which the user has repeatedly deleted, the system retrieves the original text "Thawing takes 30 minutes"; searching its associated edges reveals that the error frequency of "missing time unit" is 95%, higher than the 80% error rate of "missing temperature value", thus identifying "omitted time unit" as the core error feature.

[0060] Finally, in step 1044, adversarial examples are constructed based on the target node's original text and the core error features: the original text content is completely copied; targeted semantic deletions are performed based on the core error feature type; and the modified text is packaged into the standard input format of the large model to generate the final adversarial example. For example, after the original text "Charging time requires 2 hours" is injected with the core feature "missing time unit", the "hour" is deleted to generate the adversarial example "[QUERY]Charging time requires 2[CONTEXT]User asks about charging time..." This example will trick the large model into exposing the flaw of ignoring time units in its response.

[0061] In actual application, when user C deleted the version number "v3.2" three times in a row in the intelligent customer service dialogue in response to the reply "To upgrade the printer firmware, you need to download the v3.2 installation package", the system located the error position as the first sentence of the second paragraph, judged it as a new location, calculated the operation intensive value and created a new node element; it simultaneously detected that the user subsequently added "USB connection to the computer is required" at an adjacent location, obtained the latest error frequency value from the heat map and updated the associated edge; then traversed the updated graph, the strength of node A was 0.9, the strength of node B was 1.2, and the strength of node C was 0.7. After depth-first comparison, the highest strength value of 1.2 was locked. Node B is the target node; then the original text of the node is retrieved, "Upgrading the printer firmware requires downloading the v3.2 installation package", and the associated edges are retrieved. It is found that the error frequency of "missing version number" is 90%, which is higher than the 80% of "missing connection method", and the core error feature is determined to be "omitted version parameter"; finally, the original text is copied, and the version number is deleted according to the core feature, and it is changed to "Upgrading the printer firmware requires downloading the version installation package", and a label is added to generate an adversarial sample "[QUERY]Upgrading the printer firmware requires downloading the version installation package[CONTEXT]historical dialogue" to induce the large model to expose similar errors.

[0062] The above 104 overall solutions accurately capture the user's latest correction intentions by updating the dynamic error map, and lock the core error source based on the highest intensity node; generate targeted adversarial samples by extracting high-frequency error features, and convert ambiguous user operation behavior signals into executable optimization instructions; finally, combine the reinforcement learning mechanism to drive large-scale model fine-tuning in seconds, achieving targeted suppression of error responses and continuous improvement of dialogue quality.

[0063] 105. Use a reinforcement learning mechanism to input the adversarial sample as a negative reward into the large model, fine-tune the response module according to the error type corresponding to the highest intensity value, and drive the fine-tuned response module to output optimized dialogue content based on the current dialogue process. At the same time, feed the dialogue content back to the update process of the dynamic error map, forming a closed-loop control of dialogue generation and optimization.

[0064] Optionally, step 105 may specifically include the following steps: 1051. Input the adversarial example as a negative reward into the large model training environment through a reinforcement learning mechanism; 1052. Locate the node element corresponding to the highest intensity value in the dynamic error map, extract the error type corresponding to the node element as the target error type, and modify the parameters of the response module according to the target error type to reduce the probability of errors in the dialogue generation; 1053. Based on the context of the current conversation process, drive the response module with adjusted parameters to output optimized conversation content; 1054. Feedback the conversation content as newly added user correction data to the dynamic error map update process, forming a closed-loop control of conversation generation and optimization.

[0065] Among them, step 1054 may specifically include the following processes: converting the conversation content into newly added user correction data containing the conversation text and the corresponding timestamp; inputting the newly added user correction data into the update process of the dynamic error map, triggering the update operation of the strength value of the node element and the error frequency value of the edge element; regenerating adversarial samples through the updated dynamic error map, performing a reinforcement learning optimization process, and forming a closed-loop control with continuous optimization.

[0066] In the above scheme, the target error type refers to the essential attributes of the most frequent error pattern identified from the dynamic error graph, including the error classification characteristics associated with the highest intensity nodes, including basic error categories such as semantic loss, logical contradiction, and actual deviation, which can be used to guide the targeted optimization of the response module parameters. The conversation content refers to the interactive text data stream generated by the intelligent customer service system, which contains the context-related replies in the current conversation process, including the response text, generation timestamp and user operation association mark, which can be used to trigger user operation behavior analysis and dynamic error graph updates. New user correction data refers to the feedback signal that drives system optimization and can be used for iterative updates of the node strength and edge error frequency of the dynamic error graph.

[0067] In the embodiment of the present application, the adversarial sample is input into the large model training environment through the reinforcement learning mechanism. When the large model generates a response containing the target error for the sample, the system detects the error feature in the response and triggers a negative reward signal; this signal is input into the proximal policy optimization algorithm to calculate the policy gradient loss. The specific calculation formula is: in represents the model parameters, is the probability ratio of the new and old strategies, is the reward advantage estimate, The clipping threshold is typically set at 0.2; the loss value is used to adjust model parameters through backpropagation, thereby reducing the probability of generating similar erroneous responses in the future. This process transforms the error signals implicit in user operation behavior into a quantifiable driving force for model optimization, achieving a precise mapping of error patterns to penalty mechanisms. For example, if an adversarial example with a deliberately deleted version number, "[QUERY] To upgrade your printer, you need to download the version installation package," is input into the large model, and the model returns "Please download the installation package directly" without the supplemental version information, the system detects a "missing version parameter" error and triggers a negative reward of -1. The gradient loss is then calculated to reduce the tendency to generate such erroneous responses.

[0068] Next, step 1052 locates the node element corresponding to the highest intensity value in the dynamic error map based on the negative reward signal, and extracts the core error type associated with the node as the target error type; then, in the parameter space of the response module, the gradient descent method is used to accurately adjust the generation path weight related to the target error type: the characteristic expression pattern of the error type is analyzed, and the corresponding neuron path is located in the softmax probability distribution of the model output layer; then, according to the formula Execute parameter modification, where is the learning rate, L is the loss function, when the path p belongs to the error type e =1 otherwise 0, specifically reducing the activation weight of the incorrect path and increasing the weight of the correct path; ultimately achieving targeted suppression of a single error type, avoiding knowledge forgetting caused by global parameter adjustments. For example, for the "missing version parameter" error type corresponding to the highest-intensity node in the graph, the system reduces the probability weight of the large model generating a "Please download the installation package" message without a version number, while increasing the probability of generating "Please download the v3.2 installation package," enabling the model to accurately avoid this type of error in subsequent conversations.

[0069] Then, in step 1053, the parameter-adjusted response module, based on the context of the current conversation process, generates an optimized response: loading the conversation context data stream; calling the response module that has been targeted and fine-tuned in step 1052, and forcibly injecting error-proofing constraints when generating candidate responses through the self-attention mechanism; finally, performing rejection sampling verification. If the candidate response still contains the target error type, it is discarded and regenerated until the output strictly meets the optimized conversation content, ensuring that the result both avoids specific errors and meets the needs of the current conversation scenario. For example, in response to a user's question "B router restart steps", the fine-tuned module generates a candidate response "Please wait and try again". After detecting the "missing time value" error, the system discards it and regenerates and outputs a compliant response "Please wait 5 minutes and try again".

[0070] Finally, step 1054 converts the optimized conversation content and its generated timestamp into a new user correction data packet; the data packet is input into the dynamic error graph update process, triggering the update mechanism of step 1041: recording the user's subsequent operation behavior; if the user performs a new correction operation, the operation type, location and timestamp are recorded, and the new operation density value is calculated; finally, based on these behavioral data, the strength value of the graph node element and the error frequency value of the edge element are updated. When the strength of any node exceeds the threshold, the adversarial sample regeneration and reinforcement learning optimization cycle are automatically triggered, forming a self-evolution closed loop from model output to user feedback to graph iteration.

[0071] In actual applications, when a user repeatedly deletes the "Type A" device model in response to the customer service reply "Device firmware upgrade requires downloading an installation package", the system inputs the generated adversarial sample "[QUERY] Firmware upgrade requires downloading an installation package" into the large model, and the model returns an error response "Please download the universal installation package", triggering a negative reward signal of -1; the "device model missing" error type corresponding to the highest intensity node in the positioning graph is located, and the weight of the generation path without model description is reduced through gradient descent; combined with the user's current question "How to upgrade device B" and historical context, the fine-tuning module is driven to output the optimized reply "Please download the dedicated installation package for Type A"; after the user accepts the reply and stays at "Type A" for 6 seconds, the system synchronously updates the graph to reduce the strength of the model missing node from 1.2 to 0.9, and then the user deletes the "5" in the new reply "Wait 5 minutes", triggering a new time parameter missing node and starting the next round of optimization cycle.

[0072] The overall solution of 105 mentioned above converts adversarial samples into optimization signals that can be understood by the model through a negative reward mechanism, accurately suppressing the target error types locked by the graph; combines the dialogue context to output scenario-adapted optimization content to ensure that replies are immediately available; and finally converts user feedback into a driving force for graph iteration, forming a self-evolutionary closed loop of "error exposure-targeted correction-effect verification-strategy evolution", realizing the dynamic eradication and continuous prevention of erroneous responses of intelligent customer service, breaking through the lag and blindness of traditional offline optimization.

[0073] The following is a complete example for steps 101-105. In the customer service system, user E submitted a similar issue, "C-type router frequently disconnects," five times, each with an interval of less than 90 seconds. The system captured the entire sequence of his actions: when the automatic reply "Try restarting the device" was received, the user deleted the word "restart" three times, each deletion taking 0.3 seconds. The cursor remained in the "Check network cable interface" section for a total of 42 seconds, and the user interrupted twice with a voice message: "Specific diagnostic steps are required!" All actions were associated with millisecond-accurate timestamps. The first deletion occurred at 01:12:30.115, and the last voice message ended at 01:15:22.704, forming a multimodal raw dataset.

[0074] Based on this data, the system generates a dynamically corrected heat map, with the X-axis locating the location of the conversation text and the Y-axis marking the time series of operations. Operation intensity is rendered in a thermal color temperature. For example, the "restart" location, where users frequently delete, has an operation intensity of 3 times × 0.4 + cursor dwell time of 42 seconds × 0.05 / second = 1.2 + 2.1 = 3.3, appearing as a dark red block. Error frequency is mapped to a transparency gradient, with the transparency of the area associated with "Required diagnostic steps after deletion and restart" reduced to 20%, creating a spatiotemporal diagnostic image that combines color temperature and transparency.

[0075] Based on the modified heat map data, a dynamic error graph is constructed, and the dark red block with an operation density value of 3.3 is converted into an "operation suggestion fuzzy" node element with an intensity value of 3.3, and connected to the "technical details missing" node through an edge element. When the user adds an operation at 01:22:17, deletes the reply "Contact the operator" and asks for the "specific fault code", the graph is updated, a new "responsibility attribution fuzzy" node is created, and the node element is calculated. At the same time, because the user accepts the optimization reply, the cursor stays on the original node for 25 seconds × 0.05 / second = 1.25 positive feedback, and the intensity is reduced to 3.3-1.25=2.05.

[0076] At this point, the highest intensity node in the graph is still "Action suggestion blur," corresponding to an intensity of 2.05. The system extracts features corresponding to the lowest transparency area of ​​its associated edges and removes the action body from the original text "Try to restart" to generate the adversarial example "[QUERY]Try." This example, when fed into the larger model, induces a more ambiguous response, "Suggest action," triggering a negative reinforcement learning reward of -1.

[0077] Targeted optimization was initiated based on the negative reward signal, reducing the weight of fuzzy verb generation by 35%. Based on the user's question, "Troubleshooting packet loss rate on router D," the system generated a precise response, "Please log in to 192.168.1.1 to check QoS status." When the user deleted the "100" from the new response, "Delay threshold 100ms," this action was recorded in the correction data stream with a timestamp of 01:30:05, triggering the creation of a "Missing Parameter Quantization" node in the graph. The system then shifted to optimizing for numerical accuracy.

[0078] Figure 3 The present application provides a structural diagram of an intelligent customer service dialogue generation and optimization system based on a large model, such as Figure 3 As shown, the system includes: The collection module 31 is used to collect the user's repeated question data, manual rewriting data, cursor trajectory data, text deletion data and voice interruption data, and associate the recorded conversation timestamps to form the original data set; A generating module 32 is configured to calculate an operation intensity value and an error frequency value based on the original data set to generate a corrected heat map with time annotations, operation intensity annotations, and error frequency annotations; A construction module 33 is configured to determine an error location distribution by analyzing the correlation between the correction heat map and the conversation context, define a node element carrying an intensity value based on the operation density value of the error location distribution, and define an edge element recording a trigger condition based on the error frequency value of the error location distribution, and integrate the node elements and the edge elements to construct a dynamic error map for large model optimization; An updating module 34 is configured to call the dynamic error graph, update the strength values ​​of the node elements and the error frequency values ​​of the edge elements when new user correction data is added, and generate an adversarial sample adapted to the large model input based on the highest strength value and the error frequency value of the corresponding edge element in the updated dynamic error graph; The feedback module 35 is used to input the adversarial sample as a negative reward into the large model using a reinforcement learning mechanism, fine-tune the response module according to the error type corresponding to the highest intensity value, and drive the fine-tuned response module to output optimized dialogue content based on the current dialogue process. At the same time, the dialogue content is fed back to the update process of the dynamic error map, forming a closed-loop control of dialogue generation and optimization.

[0079] Figure 3 The intelligent customer service dialogue generation and optimization system based on a large model can be executed Figure 1 The implementation principles and technical effects of the large-scale model-based intelligent customer service dialogue generation and optimization method described in the illustrated embodiment will not be elaborated on here. The specific manner in which each module and unit performs operations in the large-scale model-based intelligent customer service dialogue generation and optimization system described in the above embodiment has been described in detail in the relevant embodiments of the method and will not be elaborated on here.

[0080] In one possible design, Figure 3 The embodiment shown is a large model-based intelligent customer service dialogue generation and optimization system that can be implemented as a computing device, such as Figure 4 As shown, the computing device may include a storage component 41 and a processing component 42; The storage component 41 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 42 .

[0081] The processing component 42 is used for the above Figure 1 The embodiment provides a method for generating and optimizing intelligent customer service dialogues based on a large model.

[0082] The processing component 42 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0083] The storage component 41 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0084] Of course, a computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0085] The input / output interface provides an interface between the processing component and the peripheral interface module, which can be an output device, an input device, etc.

[0086] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0087] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0088] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 The illustrated embodiment is a method for generating and optimizing intelligent customer service dialogues based on a large model.

[0089] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0090] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0091] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer or server) to execute the methods described in each embodiment or certain portions of the embodiments.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating and optimizing intelligent customer service dialogues based on a large model, characterized in that: include: Collect data on repeated user questions, manual rewriting, cursor traces, text deletions, and voice interruptions, and associate them with conversation timestamps to form the original dataset. Calculating an operation intensity value and an error frequency value based on the original data set to generate a modified heat map with time annotations, operation intensity annotations, and error frequency annotations; The error location distribution is determined by analyzing the correlation between the correction heat map and the conversation context. Node elements carrying intensity values ​​are defined based on the operation density values ​​of the error location distribution. At the same time, edge elements recording trigger conditions are defined based on the error frequency values ​​of the error location distribution. The node elements and edge elements are integrated to construct a dynamic error map for large model optimization. Calling the dynamic error graph, updating the strength values ​​of the node elements and the error frequency values ​​of the edge elements when new user correction data is added, and generating an adversarial sample adapted to the large model input based on the highest strength value and the error frequency value of the corresponding edge element in the updated dynamic error graph; The adversarial samples are input into the large model as negative rewards using a reinforcement learning mechanism. The response module is fine-tuned for the error type corresponding to the highest intensity value. The fine-tuned response module is driven to output optimized dialogue content based on the current dialogue process. At the same time, the dialogue content is fed back to the update process of the dynamic error map, forming a closed-loop control of dialogue generation and optimization.

2. The method according to claim 1, characterized in that The reinforcement learning mechanism uses the adversarial examples as negative rewards to input into the large model, fine-tunes the response module for the error type corresponding to the highest intensity value, and drives the fine-tuned response module to output optimized dialogue content based on the current dialogue process. The dialogue content is fed back into the update process of the dynamic error map, forming a closed-loop control of dialogue generation and optimization, including: Input the adversarial examples as negative rewards into the large model training environment through the reinforcement learning mechanism; Locating the node element corresponding to the highest intensity value in the dynamic error graph, extracting the error type corresponding to the node element as the target error type, and modifying the parameters of the response module according to the target error type to reduce the probability of errors in the dialogue generation; Based on the context of the current conversation process, the response module with adjusted parameters is driven to output optimized conversation content; The conversation content is fed back to the dynamic error map update process as newly added user correction data, forming a closed-loop control of conversation generation and optimization.

3. The method according to claim 2, characterized in that Feeding back the conversation content as newly added user correction data to the dynamic error map update process to form a closed-loop control of conversation generation and optimization includes: Convert the conversation content into newly added user correction data containing the conversation text and corresponding timestamp; Inputting the newly added user correction data into the update process of the dynamic error map to trigger the update operation of the strength value of the node element and the error frequency value of the edge element; The adversarial samples are regenerated through the updated dynamic error map, and the reinforcement learning optimization process is carried out to form a closed-loop control with continuous optimization.

4. The method according to claim 1, wherein The calling of the dynamic error graph, updating the strength value of the node element and the error frequency value of the edge element when new user correction data is added, and generating an adversarial sample adapted to the large model input based on the highest strength value and the error frequency value of the corresponding edge element in the updated dynamic error graph, includes: The dynamic error map is called. When new user correction data is added, the operation intensity value of the newly added error position is used as the strength value of the new node element, the latest operation intensity value of the existing error position is used as the strength value of the corresponding node element, and the latest error frequency value of the adjacent position is used as the trigger condition value of the edge element; Traverse all node elements in the updated dynamic error graph and locate the node element with the highest intensity value as the target node; Extracting the original conversation text corresponding to the error position of the target node, and obtaining the maximum error frequency value in the edge elements associated with the target node, and using the error type corresponding to the maximum value as the core error feature; The original conversation text is modified, and the core error features are injected to generate adversarial samples that contain similar errors and are adapted to the input format of the large model.

5. The method according to claim 1, wherein The error location distribution is determined by analyzing the correlation between the correction heat map and the conversation context, a node element carrying an intensity value is defined based on the operation density value of the error location distribution, and an edge element recording a trigger condition is defined based on the error frequency value of the error location distribution. The node elements and the edge elements are integrated to construct a dynamic error map for large model optimization, including: Analyze the correlation between each round label and the conversation context in the correction heat map, locate the rounds in the conversation process where the operation density value exceeds the density threshold or the error frequency value exceeds the frequency threshold as error positions, and integrate the error positions to form an error position distribution; Creating a node element based on the operation intensive value of the error position distribution, and assigning the operation intensive value of each error position as a strength value to the corresponding node element; Based on the error frequency values ​​of the error position distribution, and with the error frequency value of the previous position in the adjacent error positions as a trigger condition, creating an edge element connecting two node elements; The node elements and the edge elements are integrated in a dialog flow order to construct a dynamic error graph for large model parameter optimization.

6. The method according to claim 1, characterized in that The calculating the operation intensity value and the error frequency value based on the original data set to generate a modified heat map with time annotation, operation intensity annotation and error frequency annotation includes: Divide the original data set into conversation turns starting from user input and ending with intelligent customer service output; The operation intensity value is calculated by counting the number of manual rewrites, text deletions, and total cursor movement distance in a single round of conversation. The error frequency value is also calculated by counting the number of repeated questions and voice interruptions in a single round of conversation. The start and end timestamps of each round of dialogue are used as time labels, the operation intensity value is used as the operation intensity label, and the error frequency value is used as the error frequency label; A two-dimensional grid is constructed with the time axis as the horizontal axis and the conversation round as the vertical axis, and the time annotation, the operation intensive annotation, and the error frequency annotation of each conversation round are mapped to the corresponding two-dimensional grid area to generate a correction heat map.

7. The method according to claim 1, characterized in that The data collected includes repeated user questions, manual rewriting data, cursor tracks, text deletion data, and voice interruption data, and associated conversation timestamps to form an original data set, including: Monitor the conversation interface between users and intelligent customer service, collecting data on repeated questions, manual rewriting, cursor tracks, text deletions, and voice interruptions; Independent operation units with operation type identifiers are generated based on the user repeated question data, the manual rewriting data, the cursor track, the text deletion data and the voice interruption data, and the independent operation units are merged in timestamp order to generate an original data set.

8. A large-model-based intelligent customer service dialogue generation and optimization system, characterized by: include: The collection module is used to collect data on repeated questions, manual rewriting, cursor tracks, text deletions, and voice interruptions, and associates the recorded conversation timestamps to form the original data set; A generation module, configured to calculate an operation intensity value and an error frequency value based on the original data set to generate a corrected heat map with time annotations, operation intensity annotations, and error frequency annotations; A construction module is configured to determine an error location distribution by analyzing the correlation between the correction heat map and the conversation context, define a node element carrying an intensity value based on the operation density value of the error location distribution, and define an edge element recording a trigger condition based on the error frequency value of the error location distribution, and integrate the node elements and the edge elements to construct a dynamic error map for large model optimization; An updating module, configured to call the dynamic error graph, update the strength values ​​of the node elements and the error frequency values ​​of the edge elements when new user correction data is added, and generate an adversarial sample adapted to the large model input based on the highest strength value and the error frequency value of the corresponding edge element in the updated dynamic error graph; A feedback module is used to input the adversarial samples as negative rewards into the large model using a reinforcement learning mechanism, fine-tune the response module according to the error type corresponding to the highest intensity value, and drive the fine-tuned response module to output optimized dialogue content based on the current dialogue process. At the same time, the dialogue content is fed back to the update process of the dynamic error map, forming a closed-loop control of dialogue generation and optimization.

9. A computing device, characterized in that It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a large model-based intelligent customer service dialogue generation and optimization method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that A computer program is stored, and when the computer program is executed by a computer, it implements a large model-based intelligent customer service dialogue generation and optimization method as described in any one of claims 1 to 7.

Images