Intelligent debugging system and method for chat robot

Abstract

The invention provides an intelligent debugging system and method for a chat robot, and relates to the technical field of artificial intelligence and natural language processing. A predictive prompt engine, a natural language understanding service and an enhanced root cause analysis service are further integrated, and the engines / services cooperate with a process service, a dialogue management service and the like, so that more intelligent, deeper and more comprehensive support is provided in the whole life cycle of design, test and debugging of the chat robot; the chat robot development quality and efficiency are remarkably improved, and the technical threshold is reduced.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing technology, and in particular to a chatbot intelligent debugging system and method. Background Technology

[0002] Currently, various intelligent conversational AI platforms are available for developers and operators. These platforms typically provide visual interfaces (such as drag-and-drop components) to build chatbots. Such platforms generally come equipped with basic functional modules, including a "slot acquisition module," an "API call module," and a "dialogue response module," used to build a complete dialogue interaction process. After system deployment, users need to conduct simulated dialogues with the chatbot through a dedicated testing interface to verify the logical accuracy and process integrity of each functional module.

[0003] However, analysis revealed the following five problems with the aforementioned intelligent conversational AI platform: 1. Debugging is inefficient and reactive: The current solution relies heavily on identifying problems during the testing phase, a process with significant limitations. Specifically, when users encounter errors during testing (such as process interruption, incorrect intent recognition, or incorrectly filled slots), they typically need to interrupt the test and return to the complex setup interface for troubleshooting and modification. Because such operations need to be performed repeatedly, the entire process is time-consuming and inefficient.

[0004] 2. Difficulty in error localization: Error messages are often presented as simple error messages or JSON code that requires specialized knowledge to interpret, which poses a significant obstacle for users without a technical background. They struggle to quickly and accurately pinpoint the specific error point in the process and its root cause.

[0005] 3. Lack of in-depth insight: While some improvements have provided the ability to convert JSON into "natural language" for analysis, their functionality remains relatively superficial, primarily limited to demonstrating "what happened" (e.g., which intent was identified) rather than explaining "why this happened" (e.g., why the model made that judgment and what its basis was). This limitation makes optimization for deeper problems difficult.

[0006] 4. Insufficient test coverage: Manual testing methods cannot fully cover all possible paths and boundary conditions in complex dialogue processes, which may lead to unexpected errors after the system goes live.

[0007] 5. Delay in problem detection: There may be some potential logical flaws in the process design (such as infinite loops, unreachable nodes, or conflicting intents), which are often only discovered in the later stages of testing or even after the system goes live. This not only increases the cost of fixing the flaws but also introduces higher operational risks. Summary of the Invention

[0008] The purpose of this invention is to provide a chatbot intelligent debugging system and method to at least solve one of the above-mentioned problems.

[0009] In a first aspect, the present invention provides a chatbot intelligent debugging system, including a front-end application module and a back-end service cluster with communication connection; The front-end application module includes a process editor and a test panel. The test panel includes a dialogue interaction area, a predictive prompt display area, a natural language state parsing area, and an enhanced root cause analysis display area. The backend service cluster includes process services, natural language understanding services, dialogue management services, debugging services, and enhanced root cause analysis services. The debugging services include a state parser and a predictive prompting engine. The process editor provides a visual canvas for users to design chatbot dialogue processes visually. The process service stores and manages the dialogue flowcharts in the visual canvas and sends them to the predictive hint engine of the debugging service when a process analysis event is triggered. The predictive hint engine performs defect detection on the dialogue flowcharts and displays the detected defect information in the predictive hint display area. The dialogue interaction area is used for users to input statements to test the chatbot; the natural language understanding service and the dialogue management service are used to perform intent recognition and dialogue state management on the statements to be tested, respectively; the state parser of the debugging service is used to convert the state of the natural language understanding service and the dialogue management service into natural language descriptions and display them in the natural language state parsing area. The Natural Language Understanding Service or Dialogue Management Service is also used to invoke the Enhanced Root Cause Analysis Service to generate interpretable decision-making basis when a viewing and analysis request for the Natural Language State Parsing area is received, and to display it in the Enhanced Root Cause Analysis Display Area.

[0010] In an optional implementation, the test panel also includes a simulator control and coverage visualization area, the backend service cluster also includes a test service, and the debugging service also includes a simulation execution engine; The simulator control and coverage visualization area allows users to initiate multi-turn dialogue simulations for chatbots based on preset test cases or defined test objectives. The test service, upon receiving a multi-turn dialogue simulation request from the simulator control and coverage visualization area, drives the simulation execution engine to execute multi-turn dialogue simulations according to the defined multi-turn user interaction path or user behavior model. After the simulation is completed, the coverage calculator analyzes the simulation trajectory, calculates the test coverage, and displays the test coverage in the simulator control and coverage visualization area.

[0011] In an optional implementation, the test panel also includes a context debugging tool area, and the debugging service also includes a context editor interface; Both the context debugging tool area and the natural language state parsing area are used for users to modify intents and / or edit question-answer pairs; the context editor interface is used to update the corresponding data when it receives an editing request from the context debugging tool area or the natural language state parsing area. The corresponding data includes the question-answer configuration in the process service and / or the sample data of the natural language understanding service.

[0012] In an optional implementation, the predictive prompting engine is used to: perform one or more of the following on the dialogue flowchart: reachability analysis, loop detection, intent conflict detection, and rule checking, to obtain defect information; The reachability analysis includes traversing the dialogue flowchart from the starting node to identify the first unreachable node, and traversing the dialogue flowchart in reverse from all ending nodes to identify the second unreachable node. Intent conflict detection involves obtaining the intent associated with the third node and its training samples for the third node configured with intent recognition in the dialogue flowchart, and determining the intent conflict risk based on the similarity between samples under different intents. Rule checking involves scanning and matching the dialogue flowchart based on a predefined set of rules to obtain rule matching results.

[0013] In an optional implementation, the enhanced root cause analysis service, upon receiving a call request from the natural language understanding service, acquires the model information, input features, and prediction results from the call request; calculates the contribution of each input feature to the prediction intent, and returns the calculated contribution data to the natural language understanding service; the natural language understanding service sends the contribution data to the front-end application module; the front-end application module renders the input text in the enhanced root cause analysis display area based on the contribution data to differentiate the contribution of different words in the input text to the current prediction result.

[0014] In an optional implementation, the test service is used to obtain the test objectives defined by the user in the simulator control and coverage visualization area, obtain the starting statement based on the test objectives, and drive the simulation execution engine to perform dialogue simulation starting from the starting statement; The simulation execution engine starts from the initial statement and processes simulated user input by calling the natural language understanding service and dialogue management service according to the current state of the maintained dialogue state machine and the chatbot's flow definition. It obtains the chatbot's response and state transitions, and generates the next user input based on the multi-turn user interaction path or user behavior model, performing multi-turn dialogue simulation. During the simulation, it records all traversed nodes and edges, forming a simulation trajectory. After the simulation, the coverage calculator reads the simulation trajectory, counts the number of unique nodes and edges visited, and calculates the test coverage, including node coverage and edge coverage, based on the statistical results and the total number of nodes and edges in the chatbot's dialogue flow graph. When a coverage acquisition request is received from the front-end application module, it returns one or more of the following: a list of visited nodes, a list of visited edges, and the test coverage. This allows for the differentiation of visited and unvisited nodes and / or edges in the flow editor, or the display of the test coverage in the simulator control and coverage visualization area.

[0015] In an optional implementation, the front-end application module communicates with the back-end service cluster via an API gateway.

[0016] Secondly, the present invention provides a chatbot intelligent debugging method, applied to the chatbot intelligent debugging system of any of the foregoing embodiments; the chatbot intelligent debugging method includes: In response to flow analysis events triggered during the dialogue flow design process of the chatbot, the predictive prompt engine performs defect detection on the dialogue flow diagram and displays the detected defect information in the predictive prompt display area. In response to a statement test request, the Natural Language Understanding Service and the Dialogue Management Service perform intent recognition and dialogue state management on the statement to be tested, respectively; the state parser of the Debug Service converts the states of the Natural Language Understanding Service and the Dialogue Management Service into natural language descriptions and displays them in the natural language state parsing area. In response to a request to view and analyze the Natural Language State Parsing area, the Natural Language Understanding Service or the Dialogue Management Service invokes the Enhanced Root Cause Analysis Service to generate interpretable decision-making evidence, which is then displayed in the Enhanced Root Cause Analysis Display Area.

[0017] In an optional implementation, the test panel further includes a simulator control and coverage visualization area, the backend service cluster includes a test service, and the debugging service includes a simulation execution engine; the chatbot intelligent debugging method also includes: In response to multi-turn dialogue simulation requests from the simulator control and coverage visualization area based on preset test cases or defined test objectives, the test service drives the simulation execution engine to execute multi-turn dialogue simulations according to the defined multi-turn user interaction path or user behavior model. After the simulation is completed, the simulation trajectory is analyzed through the coverage calculator to calculate the test coverage, and the test coverage is displayed in the simulator control and coverage visualization area.

[0018] In an optional implementation, the test panel further includes a context debugging tool area, and the debugging service further includes a context editor interface; the chatbot intelligent debugging method also includes: In response to an edit request from the context debugging tool area or the natural language state parsing area, the context editor interface updates the corresponding data, including the question-and-answer configuration in the process service and / or sample data from the natural language understanding service.

[0019] The intelligent chatbot debugging system and method provided by this invention, in addition to providing a visual process building and integrated test and debugging interface, further integrates a predictive prompt engine, natural language understanding service, and enhanced root cause analysis service. These engines / services work together with process services and dialogue management services to provide more intelligent, in-depth, and comprehensive support throughout the entire lifecycle of chatbot design, testing, and debugging, significantly improving the quality and efficiency of chatbot development and lowering the technical threshold. Attached Figure Description

[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the structure of a chatbot intelligent debugging system provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating an application scenario of an intelligent debugging system for chatbots provided in an embodiment of the present invention; Figure 3 A flowchart illustrating a chatbot intelligent debugging method provided in an embodiment of the present invention; Figure 4 This is a flowchart illustrating another intelligent debugging method for chatbots provided in an embodiment of the present invention.

[0022] Icons: 100 - Front-end application module; 110 - Process editor; 120 - Test panel; 121 - Dialogue interaction area; 122 - Predictive prompt display area; 123 - Natural language state parsing area; 124 - Enhanced root cause analysis display area; 125 - Simulator control and coverage visualization area; 126 - Context debugging tool area; 200 - Back-end service cluster; 210 - Process service; 220 - Natural language understanding service; 230 - Dialogue management service; 240 - Debugging service; 241 - State parser; 242 - Predictive prompt engine; 243 - Simulation execution engine; 244 - Context editor interface; 250 - Enhanced root cause analysis service; 260 - Test service; 270 - API gateway; 300 - Data storage layer; 310 - Process database; 320 - Model library; 330 - Test database. Detailed Implementation

[0023] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Analysis of existing technologies reveals the following user needs: Basic requirements: Operations / business personnel without a technical background want to build and debug chatbots with low barriers to entry, be able to intuitively understand the chatbot's operation process, and quickly locate and fix errors.

[0025] Advanced requirements: ① Early problem detection: Users want to receive alerts about potential problems during the setup process, rather than waiting until the testing phase, to reduce rework; ② Understanding decision-making reasons: When the chatbot performs below expectations (e.g., incorrect intent recognition, irrelevant answers), users want to understand the specific reasons why the system made that judgment for more accurate optimization; ③ Comprehensive testing assurance: Users want to ensure that all branches and critical paths of the dialogue flow are fully tested to avoid unexpected situations after deployment; ④ Efficiency optimization: Users are concerned about the chatbot's response efficiency and API (Application Programming Interface) call costs, and hope to identify potential performance bottlenecks during the debugging phase.

[0026] Based on this, the present invention provides an intelligent chatbot debugging system and method that combines predictive error detection, interpretable model diagnosis, and automated test coverage evaluation. It is suitable for optimizing low-threshold dialogue systems used by non-technical users. The present invention aims to solve the technical problems existing in current chatbot debugging methods, such as passive debugging, low efficiency, difficulty in error localization, lack of deep insight, incomplete test coverage, and delayed problem discovery. It provides an intelligent debugging system and method that integrates predictive analysis, interpretable insight, and simulation testing to significantly improve the quality and efficiency of chatbot development and lower the technical threshold.

[0027] To facilitate understanding of this embodiment, a detailed description of the intelligent debugging system for chatbots disclosed in this embodiment of the invention will be provided first.

[0028] The intelligent chatbot debugging system provided in this invention is an intelligent, integrated chatbot design, debugging, and optimization platform that combines predictive analytics, deep interpretability, and full-path simulation. It not only simplifies the debugging process but also helps users build and iterate chatbots with higher quality, greater efficiency, and lower risk through proactive prediction, deep insight, and comprehensive testing.

[0029] like Figure 1 As shown, the chatbot intelligent debugging system includes a front-end application module 100 and a back-end service cluster 200 with communication connection. The front-end application module 100 includes a process editor 110 and a test panel 120. The test panel 120 includes a dialogue interaction area 121, a predictive prompt display area 122, a natural language state parsing area 123, and an enhanced root cause analysis display area 124. The backend service cluster 200 includes a process service 210, a natural language understanding service 220, a dialogue management service 230, a debugging service 240, and an enhanced root cause analysis service 250. The debugging service 240 includes a state parser 241 and a predictive prompting engine 242. The process editor 110 provides a visual canvas for users to design a visual chatbot dialogue process; the process service 210 stores and manages the dialogue flowcharts in the visual canvas, and sends the dialogue flowcharts to the predictive prompt engine 242 of the debugging service 240 when a process analysis event is triggered; the predictive prompt engine 242 performs defect detection on the dialogue flowcharts and displays the detected defect information in the predictive prompt display area 122. The dialogue interaction area 121 is used for users to input sentences to test the chatbot; the natural language understanding service 220 and the dialogue management service 230 are used to perform intent recognition and dialogue state management on the sentences to be tested, respectively; the state parser 241 of the debugging service 240 is used to convert the state of the natural language understanding service 220 and the dialogue management service 230 into natural language descriptions and display them in the natural language state parsing area 123. Natural Language Understanding Service 220 or Dialogue Management Service 230 is also used to invoke Enhanced Root Cause Analysis Service 250 to generate interpretable decision-making basis when a viewing and analysis request for Natural Language State Parsing Area 123 is received, and to display it in Enhanced Root Cause Analysis Display Area 124.

[0030] In this embodiment, the aforementioned front-end application module 100 provides a user interface, and its process editor 110 allows users to configure processes visually by dragging and dropping nodes. The test panel 120 is an integrated test and debugging panel that allows for interactive testing and result analysis on the same interface. When a user clicks the analysis control or save control on the test panel 120, a process analysis event is triggered.

[0031] In the aforementioned backend service cluster 200, the process service 210 is responsible for the creation, storage, and version management of the dialogue flowchart. The natural language understanding service 220 is responsible for intent recognition and slot extraction, and it integrates the calling capability of the enhanced root cause analysis service 250 interface, which can display the internal state (intent, slot, action) of the chatbot in easy-to-understand language. The dialogue management service 230 is responsible for dialogue state management and process transitions, and it integrates the calling capability of the enhanced root cause analysis service 250 interface. The state parser 241 is used to convert the runtime state of the natural language understanding service 220 / dialogue management service 230 into a natural language description. The predictive prompting engine 242 is used to analyze the process definition, detect potential structural, logical, and efficiency problems, and can automatically detect and prompt potential problems in the canvas building or test results, such as isolated nodes, infinite loops, unprocessed user branches, intent conflicts / incomplete coverage, inefficient API calls, etc. For example, it can perform breakpoint detection and prompt the user based on the dialogue flowchart structure. The enhanced root cause analysis service 250 provides model explanation capabilities, receives requests from the natural language understanding service 220 / dialogue management service 230, and returns explanation results (such as feature importance, decision paths, etc.). It can generate a misjudgment cause graph based on semantic similarity calculation and training sample analysis. The enhanced root cause analysis service 250 can provide interpretability analysis for the natural language understanding service 220 / dialogue management service 230 stages such as intent recognition and slot extraction, such as displaying key trigger words, confidence scores, similar intent comparisons, and decision path visualization.

[0032] The aforementioned intelligent chatbot debugging system can adopt a microservice-based front-end and back-end separation architecture. Microservices facilitate the independent development, deployment, and expansion of various functional modules (such as enhanced root cause analysis services, debugging services, and testing services), offer more flexible technology stack selection, and can isolate faults. With front-end and back-end separation, responsibilities are clear, facilitating parallel development and independent iteration, and providing a better user experience.

[0033] The intelligent chatbot debugging system provided in this invention, in addition to providing a visual process building and integrated test and debugging interface, further integrates a predictive prompt engine, natural language understanding service, and enhanced root cause analysis service. These engines / services work together with process services and dialogue management services to provide more intelligent, in-depth, and comprehensive support throughout the entire lifecycle of chatbot design, testing, and debugging, significantly improving the quality and efficiency of chatbot development and lowering the technical threshold.

[0034] Further optional, such as Figure 1 As shown, the test panel 120 also includes a simulator control and coverage visualization area 125, the backend service cluster 200 also includes a test service 260, and the debugging service 240 also includes a simulation execution engine 243. The simulator control and coverage visualization area 125 is used by users to initiate multi-turn dialogue simulations for chatbots based on preset test cases or defined test objectives. The test service 260, upon receiving a multi-turn dialogue simulation request from the simulator control and coverage visualization area 125, drives the simulation execution engine 243 to execute multi-turn dialogue simulations according to the defined multi-turn user interaction path or user behavior model. After the simulation is completed, the simulation trajectory is analyzed through the coverage calculator to calculate the test coverage, and the test coverage is displayed in the simulator control and coverage visualization area 125.

[0035] In this embodiment, the simulation execution engine 243 is used to execute multi-turn dialogue simulations. The test service 260 manages test cases, drives the simulation execution engine 243, and includes a coverage calculator. The coverage calculator analyzes the simulation trajectory and calculates test coverage. The simulation execution engine 243 allows users to define or automatically generate typical multi-turn user interaction paths, simulates execution, and displays the state changes of the complete process. Test coverage visualization is possible: tested and untested nodes and paths are highlighted on the dialogue flowchart, and coverage is statistically analyzed.

[0036] Further optional, such as Figure 1 As shown, the test panel 120 also includes a context debugging tool area 126, and the debugging service 240 also includes a context editor interface 244. Both the context debugging tool area 126 and the natural language state parsing area 123 are used for users to perform intent correction and / or question-answer pair editing; the context editor interface 244 is used to update the corresponding data when it receives an editing request from the context debugging tool area 126 or the natural language state parsing area 123, which includes the question-answer configuration in the process service 210 and / or the sample data of the natural language understanding service 220.

[0037] In this embodiment, the aforementioned context editor interface 244 is used to process intent correction requests and question-and-answer pair editing requests from the front-end application module 100. It can directly correct intent recognition and edit question-and-answer pairs in the test interface and synchronize in real time.

[0038] Further optional, such as Figure 1 As shown, the aforementioned backend service cluster 200 also includes an API gateway 270, through which the frontend application module 100 communicates with the backend service cluster 200. The API gateway 270 is a unified interface entry point.

[0039] Furthermore, such as Figure 1 As shown, the chatbot intelligent debugging system also includes a data storage layer 300, which includes a process database 310, a model library 320, and a test database 330. The process database 310 is used to store the dialogue flowcharts saved by the process service 210, the model library 320 is used to store the relevant model data of the natural language understanding service 220 and the dialogue management service 230, and the test database 330 is used to store the relevant test data of the test service 260 (such as test cases, user behavior models, etc.).

[0040] Optionally, the predictive prompting engine 242 described above is used to: perform one or more of the following on the dialogue flowchart: reachability analysis, loop detection, intent conflict detection, and rule checking, to obtain defect information; The reachability analysis includes traversing the dialogue flowchart from the starting node to identify the first unreachable node, and traversing the dialogue flowchart in reverse from all ending nodes to identify the second unreachable node. Intent conflict detection involves obtaining the intent associated with the third node and its training samples for the third node configured with intent recognition in the dialogue flowchart, and determining the intent conflict risk based on the similarity between samples under different intents. Rule checking involves scanning and matching the dialogue flowchart based on a predefined set of rules to obtain rule matching results.

[0041] The aforementioned predictive prompting engine 242 can identify structural problems (islands, loops) in the dialogue flowchart, detect intent conflicts (multiple intent samples are highly similar) or insufficient coverage (some key branches lack triggering conditions), and discover potential inefficiency patterns (too many consecutive API calls). Related algorithms may include: 1) Structural problem detection: Algorithm: Graph traversal (Depth-first Search / Breadth-first Search) Logical description: Perform a traversal starting from the "starting node" and mark all reachable nodes; Perform a reverse traversal starting from the "end node" to find paths that cannot reach the end point; If a path repeatedly returns to itself, it is considered a "loop".

[0042] Definitional formula: ; The function `is_reachable(v)` is used to determine whether a path exists from the start node (StartNode) to the node. v The path checks whether it is possible to reach a node from the starting node. v is_orphan(v): This function is used to determine the orphan node. v Is it an orphan node, meaning there is no path from the starting node to it? By definition, if a node is not reachable (i.e., is_reachable(v) returns false), it is orphaned. has_path_to_end(v): This function checks if there is a follower node. v The path to the end node; this represents the path from the node. v Starting point: Is it possible to reach the destination? is_dead_end(v): This function is used to identify dead ends; a node is considered dead if and only if it is reachable (reachable from the starting node), but there is no path from that node to the destination.

[0043] 2) Intent conflict detection: algorithm: Text similarity calculation: TF-IDF (Term Frequency-Inverse Document Frequency) + Cosine, Sentence-BERT Embedding + Cosine; Vector clustering analysis: K-Means or DBSCAN (Density-Based Spatial Clustering of Applications with Noise); logic: Extract training samples from all intentions and calculate their pairwise similarity; if the similarity between different intentions is higher than a set threshold (e.g., 0.85), it is judged as "conflict risk".

[0044] Cosine similarity formula: ; Where A and B are vector representations of two input statements; if Similarity > 0.85 and A and B belong to different intentions, a semantic conflict is indicated.

[0045] 3) Inefficient mode detection: Algorithm: Rule-based Pattern Detection; Example of a rule: "More than 3 consecutive API call nodes"; "The default fallback path for branches without conditional configuration"; "Redirect to the end node immediately after the user enters a node", etc.

[0046] In its specific implementation, after receiving the flow definition of the dialogue flow graph (which can be represented as a directed graph G=(V, E), where V is a node and E is an edge), the predictive prompting engine 242 can perform the following checks: Reachability analysis: Using the DFS (Depth-First Search) or BFS (Breadth-First Search) algorithm, traverse the graph from the starting node and mark all reachable nodes; then traverse in reverse from all ending nodes and mark all nodes that can be reached from the ending nodes; thus identifying nodes that cannot be reached from the starting node (islands) and nodes that cannot be reached from any ending node.

[0047] Cycle detection: During DFS traversal, a stack of nodes along the current path is maintained; if a node already in the stack is visited, a cycle is detected. The maximum allowed number of cycles or the path length can be configured.

[0048] Intent conflict detection: For nodes configured with intent recognition, obtain their associated intents and their training samples; use models such as Sentence-BERT to convert the samples into vector embeddings. Calculate the cosine similarity between sample vectors under different intents. If a large number of sample pairs between different intents have similarities exceeding a preset threshold (e.g., 0.85), an alert for "Intent conflict risk" will be displayed in the predictive prompt area 122.

[0049] Rule checking: Apply predefined rule sets, such as "more than 3 consecutive API call nodes" and "conditional branches lack default processing paths", to scan and match the dialog flowchart.

[0050] Optionally, the enhanced root cause analysis service 250, upon receiving a call request from the natural language understanding service 220, obtains the model information, input features, and prediction results from the call request; calculates the contribution of each input feature to the prediction intent, and returns the calculated contribution data to the natural language understanding service 220; the natural language understanding service 220 sends the contribution data to the front-end application module 100; the front-end application module 100 renders the input text in the enhanced root cause analysis display area 124 based on the contribution data to differentiate and display the contribution of different words in the input text to the current prediction result.

[0051] The enhanced root cause analysis service 250 described above can explain why the NLU model identifies a user input as a specific intent / extracts a specific slot, and explain why the DM model selects a specific response or redirect path. Related algorithms may include: LIME: By generating slight perturbations near the prediction point, a simple, interpretable local surrogate model is trained to simulate the behavior of complex models; suitable for interpreting single predictions. SHAP: Based on the Shapley value in game theory, it calculates the contribution of each feature to the final prediction result; it has a stronger theoretical foundation and can provide global and local explanations. Attention Visualization (for Transformer-based NLU models): Displays the degree of attention (weight) given to each word in the Transformer model, that is, visualizes which words the model gives more attention to when processing input text.

[0052] Taking SHAP as an example, the formula is as follows: ; in: Representation of features i The SHAP value; F Represents the set of all features; S Indicates feature removali Subset of features; f ( S ) indicates inputting only a subset S The predicted value of the time model.

[0053] In practical implementation, the interpretability of NLU (Natural Language Understanding) based on SHAP (SHapley Additive exPlanations) can be as follows: If a user encounters unsatisfactory NLU results during testing (such as an intent recognition error), they can click "View Analysis." The front-end application module 100 then sends a request containing the current input text and the NLU model identifier to the NLU service.

[0054] The NLU service calls the XAI (eXplainable AI) service to request an explanation of the prediction. The request includes model information, input features (such as tokenized text), and the prediction result.

[0055] The XAI service uses SHAP (such as KernelSHAP or SHAP variants for specific model architectures) to calculate the contribution of each input token to the prediction intent (Shapley value).

[0056] The XAI service returns the calculated SHAP value array to the NLU service, which then passes it to the front-end application module 100.

[0057] The front-end application module 100 renders the input text in the enhanced root cause analysis display area 124. Based on the positive and negative values ​​and magnitudes of the SHAP values, it highlights the words that contribute the most to the current prediction results with different colors or backgrounds, helping users understand the basis of the model's decision-making.

[0058] Optionally, the aforementioned test service 260 is used to obtain the test objectives defined by the user in the simulator control and coverage visualization area 125, obtain the starting statement according to the test objectives, and drive the simulation execution engine 243 to perform dialogue simulation starting from the starting statement; The simulation execution engine 243 is used to start from the initial statement, and by calling the natural language understanding service 220 and the dialogue management service 230 to process the simulated user input according to the current state of the maintained dialogue state machine and the chatbot's flow definition, obtain the chatbot's response and state transition, and generate the next user input according to the multi-turn user interaction path or user behavior model, to perform multi-turn dialogue simulation, and record all the nodes and edges traversed during the simulation to form a simulation trajectory; after the simulation ends, the coverage calculator reads the simulation trajectory, counts the number of unique nodes and edges visited, obtains the statistical results, and calculates the test coverage including node coverage and edge coverage based on the statistical results and the total number of nodes and edges in the chatbot's dialogue flow graph; when a coverage acquisition request is received from the front-end application module 100, it returns one or more of the following: a list of visited nodes, a list of visited edges, and test coverage, so as to distinguish between visited and unvisited nodes and / or edges in the flow editor 110, or to display the test coverage in the simulator control and coverage visualization area 125.

[0059] The aforementioned multi-turn dialogue simulation and coverage analysis can generate realistic multi-turn dialogue interaction sequences based on user goals or preset scenarios, trace the simulated path on the flowchart, and calculate the coverage of nodes / edges. Related algorithms may include: 1) Simulation Execution Engine: Algorithm: State machine simulator + probabilistic user model; Logic: Starting from the initial node, a dialogue trajectory is generated based on the test objective; in each round of simulation, the NLU / DM service is invoked to respond and drive state transitions; during the simulation execution, nodes, paths, and response behaviors are recorded.

[0060] 2): Coverage calculation: Algorithm: Graph Path Tracing + Statistical Analysis Logic: Mark all process nodes and edges during the simulation; calculate node coverage and path coverage (i.e., edge probability).

[0061] Coverage formula: Node coverage = number of visited nodes / total number of nodes; Edge coverage = number of visited edges / total number of edges.

[0062] In practice, state machine-based simulation and coverage calculation can be performed as follows: In the simulator control and coverage visualization area 125, the user defines a test objective, such as "the user successfully completes the leave application".

[0063] Depending on the test objectives, TestService260 can request users to provide a starting statement or automatically generate a typical starting statement.

[0064] Test service 260 drives simulation execution engine 243 to start simulation. Simulation execution engine 243 maintains a dialogue state machine. Based on the current state and process definition, it calls NLU / DM (Dialog Management) service to process simulated user input, obtains robot response and state transition, and generates the next input based on user behavior model.

[0065] During the simulation, the IDs of all nodes (Identifications) and edges traversed are recorded to form the simulated trajectory.

[0066] After the simulation ends (either by reaching the end node or triggering the termination condition), the coverage calculator reads the simulation trajectory and counts the number of unique nodes and edges visited.

[0067] The front-end application module 100 requests coverage results, and the test service 260 returns a list of covered nodes / edges or a coverage percentage. On the canvas of the process editor 110, the front-end application module 100 can highlight visited nodes / edges with one color (such as green) and unvisited nodes / edges with another color (such as red), or display the coverage value in the simulator control and coverage visualization area 125.

[0068] Test service 260 can generate process heatmaps through automated simulation, mark unreached nodes, and return the process heatmaps to front-end application module 100; front-end application module 100 displays the process heatmaps in simulator control and coverage visualization area 125.

[0069] It should be noted that the algorithm used in the XAI service can be replaced with LIME (Local Interpretable Model-agnostic Explanations) or other model interpretation techniques. The predictive prompt engine 242 can integrate machine learning-based models to predict high-risk design patterns by learning from historical data. The simulator's user behavior model can be replaced from simple random selection with a more complex behavior model trained based on user profiles or historical dialogue data. Coverage calculation can employ more complex criteria, such as conditional coverage and path coverage.

[0070] Regarding the aforementioned predictive suggestion engine 242, enhanced root cause analysis service 250, and simulation execution engine 243, this embodiment of the invention also provides a training and evaluation process, as follows: 1. Data Requirements and Preparation: 1) Predictive hint engine 242: Data: User-constructed flowchart data (nodes, edges, configurations); intent and sample data.

[0071] Source: Generated directly from user actions on the platform.

[0072] Preparation: No additional annotations are required, but structured flowchart data and intent data are needed.

[0073] 2) Enhanced Root Cause Analysis Service 250: Data: Access to the NLU / DM model itself and its training data (or at least the feature space definition) is required; the input data for a single prediction needs to be interpreted.

[0074] Source: Input from NLU / DM service and user testing.

[0075] Preparation: The XAI algorithm typically needs to be able to call the model to make multiple predictions (such as perturbations of LIME / SHAP).

[0076] 3) Simulation Execution Engine 243: Data: Flowchart data; optional test case definitions (user objectives, start statements); optional user behavior model parameters.

[0077] Source: User-built flowcharts; user-defined test cases.

[0078] Preparation: Structured flowchart data.

[0079] 2. Training and Assessment: 1) Predictive hint engine 242: If you are using ML (Machine Learning) models to predict potential problems (e.g., training a model based on historical data to predict which process designs are prone to errors), then you need to: Training: Supervised learning is performed using a large amount of labeled historical process data (whether it is a problem process or not).

[0080] Evaluation: Accuracy, precision, recall, and F1 score, assessing the model's ability to identify real-world problems.

[0081] 2) Enhanced Root Cause Analysis Service 250: XAI methods themselves typically do not require "training," but they do need to be configured and applied for a specific model. Metrics for evaluating XAI can include: Fidelity: The degree of consistency between the interpretation and the original model's predictions.

[0082] Comprehensibility: The ease with which users can understand and interpret the explanation (assessed through user surveys).

[0083] Usefulness: Explain whether it is helpful for users to debug or trust the model (evaluated through user surveys).

[0084] 3) Simulation Execution Engine 243: If the user behavior model is learned, it requires user interaction data for training. When evaluating the simulated execution engine 243, the distribution characteristics (such as dialogue turns, intent transition patterns, etc.) of simulated dialogues and real user dialogues can be compared.

[0085] For the algorithm integration and service-oriented architecture of the predictive suggestion engine 242 and the enhanced root cause analysis service 250: Predictive suggestion engine 242: Can be part of a debugging service or a standalone microservice. Provides an API to receive flowchart data and return a list of issues. Can be triggered asynchronously when the user saves the flowchart, or invoked in real time in the test interface.

[0086] Enhanced Root Cause Analysis Service 250: This is a standalone microservice. After making predictions, the NLU / DM service calls the XAI service's API if interpretation is needed, passing in model information and input data. The XAI service then returns the interpretation results. After obtaining test results, the debug service calls the XAI service to retrieve the interpretations and integrates them into the final data returned to the front end.

[0087] In summary, the core idea of ​​this invention is to further integrate a predictive prompt engine, NLU service, XAI service, simulation execution engine, and test service coverage calculation, based on providing a visual process building and integrated test and debugging interface. These engines / services work in conjunction with basic process services, DM services, and other modules to provide more intelligent, in-depth, and comprehensive support throughout the entire lifecycle of chatbot design, testing, and debugging.

[0088] For ease of understanding, please refer to the following: Figure 2 This paper introduces the application scenarios of the intelligent debugging system for chatbots.

[0089] Users enter the front-end application module; process setup: users build / edit the chatbot's dialogue process in the process editor; after users click save or analyze, the predictive prompt engine is invoked to analyze structural problems; when prompting results, structural anomaly prompts are displayed in the predictive prompt display area.

[0090] Dialogue Test: The user inputs simulated utterances on the test panel; the NLU service is invoked to identify intent and entities; the DM service is invoked to determine the response logic; the state parser is invoked to generate readable state explanations; and the parsing results are displayed in the Natural Language State Parsing Area.

[0091] Users click the "View Analysis" button; XAI service: generates explanations for model decisions; displays key features, confidence levels, and attention weights.

[0092] Determine if debugging is complete; if not, re-execute the steps of "Process Setup: Users build / edit the chatbot's dialogue flow in the process editor"; if yes, submit the go-live process or publish the process.

[0093] The user edits the intent / sample / configuration; calls the context debugging interface to update the sample or process; and returns to the test process for verification.

[0094] The user initiates a multi-turn dialogue simulation; the simulation execution engine runs according to the process; the coverage of process nodes / edges is calculated; the coverage is highlighted and visualized on the process canvas.

[0095] For example, the detailed process of the above-mentioned intelligent chatbot debugging system is as follows: S1: Process Design and Predictive Analytics.

[0096] Users build or modify dialogue flows through the flow editor 110 of the front-end application module 100 (hereinafter referred to as the front-end). During this process, when clicking the analysis control or save control, the flow service 210 sends the flow definition data (i.e., the dialogue flow diagram) to the predictive prompt engine 242 of the debugging service 240. The predictive prompt engine 242 analyzes the flow using graph algorithms, rule checks, text similarity calculations, etc., and returns the detected potential problems (such as isolated nodes, loops, and intent conflicts) to the front-end through the API gateway 270, which are then displayed to the user in the predictive prompt display area 122.

[0097] S2: Dialogue Testing and State Analysis.

[0098] Users input statements in the dialogue interaction area 121 of the test panel 120 to conduct tests. The front end sends the statements to the backend service cluster 200 (hereinafter referred to as the backend) through the API gateway 270. The NLU service and DM service process the statements, determine the intent, slots, and next action. The state parser 241 of the debug service 240 obtains the internal state of the NLU / DM, converts it into a natural language description, and displays it in the natural language state parsing area 123.

[0099] S3: Enhanced Root Cause Analysis (XAI).

[0100] When users are confused by the results of NLU / DM (such as incorrect intent recognition), they can click buttons such as "View Analysis" on the front end to trigger an analysis request for the NLU / DM results. The analysis request triggered by the front end goes through API gateway 270 to the NLU / DM service, and then to the XAI service. During the processing of the analysis request, the NLU / DM service calls the XAI service. The XAI service uses algorithms such as LIME, SHAP, or Attention mechanisms to analyze the model's decision-making basis and returns the interpretation results (such as key influencing words, confidence comparisons, feature contribution, etc.). The interpretation results are returned to the front end through API gateway 270, and the front end visualizes this information for the user in the enhanced root cause analysis display area 124.

[0101] S4: Context debugging.

[0102] Users can directly perform operations such as intent modification and question-and-answer pair editing in the context debugging tool area 126 or the natural language state parsing area 123 of the test panel 120. Editing requests reach the context editor interface 244 of the debugging service 240 via the API gateway 270. The context editor interface 244 processes these requests and updates the corresponding data (such as the question-and-answer configuration in the process service 210 or the sample data of the NLU service). Users can immediately continue to the next round of testing on the current interface to verify the effect of the modifications.

[0103] S5: Multi-turn dialogue simulation and coverage analysis.

[0104] Users initiate multi-turn dialogue simulations via the simulator control and coverage visualization area 125, based on preset test cases or user objectives. The multi-turn dialogue simulation request reaches the test service 260 via the API gateway 270. The test service 260 drives the simulation execution engine 243 of the debugging service 240 to execute multi-turn dialogues according to the defined multi-turn user interaction path or user behavior model, interacting with the NLU / DM service and recording the simulation trajectory. After the simulation is complete, the coverage calculator of the test service 260 analyzes the simulation trajectory, calculates the coverage of nodes and edges, and stores the coverage results in the test database 330. The coverage results are returned to the front end via the API gateway 270, where the coverage status is highlighted in the simulator control and coverage visualization area 125 or directly on the canvas of the process editor 110.

[0105] S6: Iterative optimization.

[0106] This is a decision point. Based on predictive prompts, natural language parsing, XAI insights, contextual debugging results, and simulated coverage reports, the user determines whether further optimization is needed. If so, depending on the nature of the optimization, the user chooses to return to step S1 (modify the process design) or S4 (adjust data / configuration, such as through contextual editing) for continuous iterative optimization of the chatbot. If not, the process ends, potentially indicating that the chatbot meets the requirements and is ready for deployment.

[0107] The key technical points of the embodiments of the present invention mainly include: (1) Predictive prompting engine 242: Proactively discover potential problems during the design phase, rather than passively waiting for them to be exposed during testing; (2) Enhanced Root Cause Analysis Service 250: Provides interpretability of model decisions, enabling in-depth debugging and optimization; (3) Simulation Execution Engine 243 and Coverage Calculator: Supports automated multi-turn dialogue testing and quantifies test coverage; (4) System integration: The above technical points are seamlessly integrated with functions such as visual construction, integrated test and debugging interface, natural language state parsing, and context editing to provide an end-to-end intelligent debugging experience.

[0108] Alternatively, the aforementioned chatbot intelligent debugging system can also perform the following functions: Performance and cost estimation: Simulate API calls to estimate the average response time and potential cost of the dialogue process; Sentiment / Conversation Quality Analysis: Analyze the sentiment and fluency of the chatbot's responses during testing; A / B testing support: Conveniently compare the effects of different process versions in the debugging environment; A / B testing is used to compare two or more versions of the product to determine which version performs better on a specific metric. Batch test case management: Supports importing / exporting test cases for regression testing.

[0109] To facilitate understanding, several specific application scenarios of the above-mentioned intelligent chatbot debugging system are provided below: Scenario 1: Predictive prompts during setup.

[0110] User: Operations Xiaoming is setting up a leave application process.

[0111] Process: He dragged a "Inquire about leave type" node but forgot to connect it to a follow-up node. The system (predictive hint) highlighted the node on the canvas in real time and prompted, "This node lacks a follow-up process connection, which may cause the conversation to be interrupted." After seeing the prompt, Xiaoming immediately added the follow-up node.

[0112] Scenario 2: Root cause analysis during testing.

[0113] User: Product manager Xiaohong is testing a ticket booking robot.

[0114] Process: She typed "Book me a ticket to Shanghai tomorrow," but the robot misinterpreted it as a "check the weather" intent. The incorrect intent was displayed in the Natural Language Processing (NLP) state analysis area, and she clicked "View Analysis" (Enhanced Root Cause Analysis). The system displayed: "The word 'ticket' also appeared in the training data for the 'check the weather' intent (e.g., 'typhoon ticket'), and the current model's confidence in extracting the location and time entities for 'going to Shanghai tomorrow' is low, leading to misjudgment. The confidence score for the 'book ticket' intent is 0.55, and for 'check the weather' it is 0.62." Xiaohong decided to adjust the training samples or model parameters for the "book ticket" intent.

[0115] Scenario 3: Conduct comprehensive testing using a simulator.

[0116] User: Test engineer Daqiang is responsible for the final verification before going live.

[0117] The process: He entered several key user objectives (such as "Successfully book a meeting room," "Booking failed due to incomplete information," and "Transfer to human operator midway") in the simulator control and coverage visualization area. The simulation execution engine automatically generated and executed multiple dialogue paths covering these objectives. After the simulation, the (test coverage visualization) showed on the flowchart that a node for handling exceptions was never triggered. Daqiang supplemented the test cases accordingly.

[0118] Optionally, the performance of the above-mentioned chatbot intelligent debugging system can be quantitatively evaluated using the following metrics: Debugging efficiency: Average time taken for a single debugging task; Usability rating: Satisfaction score of non-technical users with the platform; such as NPS (Net Promoter Score) / SUS (System Usability Scale). Prediction accuracy / recall: The proportion of real-world problems found by predictive suggestions; Root cause analysis effectiveness: The percentage of users who adopt XAI analysis results for optimization; Test coverage: Average test coverage after multi-turn dialogue simulation using the test service; Post-launch failure rate: Due to more comprehensive testing, the failure rate in the production environment caused by process design has decreased.

[0119] The embodiments of the present invention have the following significant beneficial effects: 1. Proactive prevention reduces rework costs: Through predictive intelligent prompts, potential problems can be identified and fixed during the design phase, avoiding high rework costs caused by discovering problems late in testing or after launch.

[0120] 2. Deep insights, precise optimization: Enhanced Root Cause Analysis (XAI) reveals the internal reasons for model decisions, helping users (even non-experts) understand model behavior, thereby enabling them to more accurately adjust training data or model parameters and improve chatbot performance.

[0121] 3. Comprehensive testing to improve quality: Multi-turn dialogue simulation and coverage analysis can systematically test various paths of the dialogue process, quantify the degree of test coverage, significantly reduce online failures caused by insufficient testing, and improve the robustness and reliability of chatbots.

[0122] 4. Improved efficiency and shortened cycle: The prediction, testing, parsing, interpretation, debugging, and simulation verification processes are integrated into a unified interface and intelligent support is provided, which greatly shortens the development and debugging cycle from design to high-quality deployment.

[0123] 5. Lowering the barrier to entry and empowering business: Through natural language parsing, visual explanation, context editing, and intelligent prompts, the barrier to entry for non-technical personnel to build and optimize high-quality chatbots has been further lowered.

[0124] This invention also provides a chatbot intelligent debugging method, which is applied to the aforementioned chatbot intelligent debugging system. See also... Figure 3 The diagram shows a flowchart of a chatbot intelligent debugging method, which mainly includes the following steps S310 to S330: Step S310: In response to the process analysis event triggered during the dialogue flow design process of the chatbot, the predictive prompt engine performs defect detection on the dialogue flow diagram and displays the detected defect information in the predictive prompt display area. In step S320, in response to the statement test request, the Natural Language Understanding Service and the Dialogue Management Service perform intent recognition and dialogue state management on the statement to be tested, respectively; the state parser of the debugging service converts the state of the Natural Language Understanding Service and the Dialogue Management Service into a natural language description and displays it in the natural language state parsing area. In step S330, in response to a request to view and analyze the natural language state parsing area, the natural language understanding service or dialogue management service calls the enhanced root cause analysis service to generate interpretable decision basis and displays it in the enhanced root cause analysis display area.

[0125] Furthermore, the aforementioned test panel also includes a context debugging tool area, and the debugging service also includes a context editor interface; such as Figure 4 As shown, the above-mentioned intelligent debugging method for chatbots includes: In step S410, in response to an edit request from the context debugging tool area or the natural language state parsing area, the context editor interface updates the corresponding data, which includes the question-and-answer configuration in the process service and / or sample data from the natural language understanding service.

[0126] Furthermore, the aforementioned test panel also includes a simulator control and coverage visualization area, the backend service cluster includes a test service, and the debugging service includes a simulation execution engine; such as Figure 4 As shown, the above-mentioned intelligent debugging method for chatbots includes: In step S420, in response to a multi-turn dialogue simulation request from the simulator control and coverage visualization area based on preset test cases or defined test objectives, the test service drives the simulation execution engine to execute multi-turn dialogue simulations according to the defined multi-turn user interaction path or user behavior model. After the simulation is completed, the simulation trajectory is analyzed through the coverage calculator to calculate the test coverage and display the test coverage in the simulator control and coverage visualization area.

[0127] The chatbot intelligent debugging device provided in this embodiment has the same implementation principle and technical effect as the aforementioned chatbot intelligent debugging method embodiment. For the sake of brevity, any parts not mentioned in the chatbot intelligent debugging device embodiment can be referred to the corresponding content in the aforementioned chatbot intelligent debugging method embodiment.

[0128] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0129] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0130] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A chatbot intelligent debugging system, characterized in that, This includes the front-end application module and back-end service cluster for communication connections; The front-end application module includes a process editor and a test panel. The test panel includes a dialogue interaction area, a predictive prompt display area, a natural language state parsing area, and an enhanced root cause analysis display area. The backend service cluster includes process services, natural language understanding services, dialogue management services, debugging services, and enhanced root cause analysis services. The debugging services include a state parser and a predictive prompting engine. The process editor provides a visual canvas for users to design a visual chatbot dialogue process; the process service stores and manages the dialogue flowcharts in the visual canvas, and sends the dialogue flowcharts to the predictive prompting engine of the debugging service when a process analysis event is triggered. The predictive prompting engine is used to detect defects in the dialogue flowchart and display the detected defect information in the predictive prompting display area. The dialogue interaction area is used for users to input statements to test the chatbot; The Natural Language Understanding Service and the Dialogue Management Service are used to perform intent recognition and dialogue state management on the statements to be tested, respectively; the state parser of the debugging service is used to convert the states of the Natural Language Understanding Service and the Dialogue Management Service into natural language descriptions and display them in the natural language state parsing area. The Natural Language Understanding Service or the Dialogue Management Service is further configured to, upon receiving a viewing and analysis request for the Natural Language State Parsing Area, invoke the Enhanced Root Cause Analysis Service to generate interpretable decision-making basis and display it in the Enhanced Root Cause Analysis Display Area.

2. The chatbot intelligent debugging system according to claim 1, characterized in that, The test panel also includes a simulator control and coverage visualization area, the backend service cluster also includes a test service, and the debugging service also includes a simulation execution engine; The simulator control and coverage visualization area is used for users to initiate multi-turn dialogue simulations for chatbots based on preset test cases or defined test objectives. The test service is used to drive the simulation execution engine to execute multi-turn dialogue simulations according to the defined multi-turn user interaction path or user behavior model after receiving a multi-turn dialogue simulation request from the simulator control and coverage visualization area. After the simulation is completed, the test service analyzes the simulation trajectory through the coverage calculator, calculates the test coverage, and displays the test coverage in the simulator control and coverage visualization area.

3. The chatbot intelligent debugging system according to claim 1 or 2, characterized in that, The test panel also includes a context debugging tool area, and the debugging service also includes a context editor interface; Both the context debugging tool area and the natural language state parsing area are used for users to perform intent correction and / or question-answer pair editing; the context editor interface is used to update the corresponding data when it receives an editing request from the context debugging tool area or the natural language state parsing area, the corresponding data including the question-answer configuration in the process service and / or the sample data of the natural language understanding service.

4. The chatbot intelligent debugging system according to claim 1, characterized in that, The predictive prompting engine is used to perform one or more of the following on the dialogue flowchart: reachability analysis, loop detection, intent conflict detection, and rule checking, to obtain the defect information; The reachability analysis includes traversing the dialogue flowchart from the starting node to identify the first unreachable node, and traversing the dialogue flowchart in reverse from all ending nodes to identify the second unreachable node. The intent conflict detection includes obtaining the intent associated with the third node and its training samples for the third node configured with intent recognition in the dialogue flowchart, and determining the intent conflict risk based on the similarity between samples under different intents. The rule check includes scanning and matching the dialogue flowchart based on a predefined set of rules to obtain rule matching results.

5. The chatbot intelligent debugging system according to claim 1, characterized in that, The enhanced root cause analysis service is used to, upon receiving a call request from the natural language understanding service, obtain model information, input features, and prediction results from the call request; calculate the contribution of each input feature to the prediction intent; and return the calculated contribution data to the natural language understanding service. The natural language understanding service is used to send the contribution data to the front-end application module; The front-end application module is used to render the input text in the enhanced root cause analysis display area according to the contribution data, so as to distinguish and display the contribution of different words in the input text to the current prediction result.

6. The chatbot intelligent debugging system according to claim 2, characterized in that, The testing service is used to obtain the test objectives defined by the user in the simulator control and coverage visualization area, obtain the starting statement according to the test objectives, and drive the simulation execution engine to start the dialogue simulation from the starting statement. The simulation execution engine is used to start from the initial statement, and by calling the natural language understanding service and the dialogue management service to process the simulated user input according to the current state of the maintained dialogue state machine and the flow definition of the chatbot, obtain the chatbot's response and state transition, and generate the next user input according to the multi-round user interaction path or the user behavior model, to perform multi-round dialogue simulation, and record all the nodes and edges passed during the simulation to form a simulation trajectory. After the simulation ends, the coverage calculator reads the simulation trajectory, counts the number of unique nodes and edges visited, obtains the statistical results, and calculates the test coverage including node coverage and edge coverage based on the statistical results and the total number of nodes and edges in the chatbot's dialogue flowchart. When a coverage acquisition request is received from the front-end application module, one or more of the following are returned: a list of visited nodes, a list of visited edges, and the test coverage rate. This is used to distinguish between visited and unvisited nodes and / or edges in the process editor, or to display the test coverage rate in the simulator control and coverage visualization area.

7. The chatbot intelligent debugging system according to claim 1, characterized in that, The front-end application module communicates with the back-end service cluster through an API gateway.

8. A method for intelligent debugging of a chatbot, characterized in that, The chatbot intelligent debugging system applicable to any one of claims 1-7; The intelligent debugging method for the chatbot includes: In response to a process analysis event triggered during the dialogue flow design process of the chatbot, the predictive prompt engine performs defect detection on the dialogue flow diagram and displays the detected defect information in the predictive prompt display area. In response to a statement test request, the Natural Language Understanding Service and the Dialogue Management Service perform intent recognition and dialogue state management on the statement to be tested, respectively; the state parser of the debugging service converts the states of the Natural Language Understanding Service and the Dialogue Management Service into natural language descriptions and displays them in the natural language state parsing area; In response to a request to view and analyze the natural language state parsing area, the natural language understanding service or the dialogue management service invokes the enhanced root cause analysis service to generate interpretable decision-making basis and displays it in the enhanced root cause analysis display area.

9. The intelligent debugging method for chatbots according to claim 8, characterized in that, The test panel also includes a simulator control and coverage visualization area; the backend service cluster also includes a test service; the debugging service also includes a simulation execution engine; the chatbot intelligent debugging method also includes: In response to a multi-turn dialogue simulation request from the simulator control and coverage visualization area based on preset test cases or defined test objectives, the test service drives the simulation execution engine to perform multi-turn dialogue simulations according to defined multi-turn user interaction paths or user behavior models. After the simulation is completed, the simulation trajectory is analyzed through the coverage calculator to calculate the test coverage, and the test coverage is displayed in the simulator control and coverage visualization area.

10. The intelligent debugging method for chatbots according to claim 8 or 9, characterized in that, The test panel also includes a context debugging tool area, and the debugging service also includes a context editor interface; The intelligent debugging method for chatbots also includes: In response to an edit request from the context debugging tool area or the natural language state parsing area, the context editor interface updates the corresponding data, which includes the question-and-answer configuration in the process service and / or the sample data of the natural language understanding service.

Images