High-concurrency multi-turn conversation risk detection system for large model service

Abstract

This invention discloses a high-concurrency multi-turn dialogue risk detection system for large-model services, comprising an access scheduling layer, a heterogeneous computing cluster layer, and a global decision layer. The heterogeneous computing cluster layer consists of multiple computing nodes, each including: a network interface module for capturing dialogue data streams distributed by the access scheduling layer; an intent recognition acceleration card for classifying execution intents for each turn of dialogue and outputting a basic risk score; a graph construction and storage unit for maintaining local intent graph fragments; a nonlinear risk calculation unit for performing risk accumulation calculations; a dynamic threshold controller for comparing accumulated risk values ​​with dynamic thresholds; a log and audit storage unit for recording risk judgment logs; and a management interface for receiving policy configuration instructions. This high-concurrency multi-turn dialogue risk detection system for large-model services enables high-concurrency, low-latency real-time risk detection.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence security technology, specifically relating to a high-concurrency multi-turn dialogue risk detection system for large model services. Background Technology

[0002] With the widespread application of large-scale language models in scenarios such as intelligent customer service, financial consulting, and government services, multi-turn dialogue interaction systems face severe content security challenges. Attackers employ progressive inducement strategies, breaking down malicious intent into multiple seemingly harmless steps, distributing them across dozens of dialogue rounds to carry out attacks. Traditional single-turn detection methods struggle to identify such threats.

[0003] Existing multi-turn dialogue risk detection technologies mainly fall into two categories. One category is based on single-turn detection strategies, where keyword matching or classification models are used to score each round of user input independently. This approach fails to capture cross-turn intent relationships and is easily bypassed by gradual, "boiling frog" tactics. The other category attempts to introduce context awareness, extracting risk features through deep learning models. However, this approach has the following limitations: first, it relies on large-scale labeled data, limiting its generalization ability to emerging or rare attack patterns; second, it is a black box, lacking interpretability in the risk assessment process and making it difficult to trace the specific logical path of risk accumulation; third, it does not explicitly model intent logic dependencies, making it impossible to inject domain knowledge; and fourth, it lacks a gradual decay over time, failing to consider the impact of dialogue intervals on risk accumulation.

[0004] More importantly, existing solutions mostly adopt a software-based deployment model, which faces severe performance bottlenecks in high-concurrency scenarios. When the system needs to handle thousands of concurrent dialogues simultaneously, the CPU computing power and memory capacity of a single-machine architecture become bottlenecks, causing risk detection latency to deteriorate from milliseconds to hundreds of milliseconds, failing to meet the stringent real-time requirements of scenarios such as financial transactions. In addition, software-based deployments lack reliability; a single point of failure can cause the entire risk control system to fail, which does not meet the high availability requirements of core financial businesses.

[0005] Therefore, there is an urgent need for a high-performance risk detection system for high-concurrency scenarios that can maintain method-level accuracy while achieving distributed horizontal scaling, hardware-level acceleration, and telecom-grade reliability. Summary of the Invention

[0006] This invention provides a high-concurrency multi-turn dialogue risk detection system for large model services to solve the aforementioned technical problems, specifically adopting the following technical solution: A high-concurrency multi-turn dialogue risk detection system for large model services includes: The access scheduling layer is configured with a load balancer and a session affinity routing module, which are used to receive multiple dialogue data streams and distribute them to downstream processing nodes according to user identifiers. A heterogeneous computing cluster layer, composed of multiple computing nodes coupled via a high-speed interconnect network, is configured to maintain a dynamic intent graph in a distributed manner and perform risk accumulation calculations in parallel. Each computing node includes: The network interface module is used to capture the dialogue data stream distributed by the access scheduling layer; An intent recognition acceleration card, coupled to the network interface module, is used to classify the execution intent of each round of dialogue and output a basic risk score. The graph construction and storage unit is coupled to the intent recognition acceleration card and is used to maintain local intent graph fragments; A nonlinear risk calculation unit, coupled to the map construction and storage unit, is used to perform risk accumulation calculation; A dynamic threshold controller, coupled to the nonlinear risk calculation unit, is used to compare the cumulative risk value with the dynamic threshold. Logs and audit storage are used to record risk assessment logs; The management interface is used to receive policy configuration instructions; The global decision-making layer, coupled with the heterogeneous computing cluster layer through a message bus, is configured to aggregate the risk calculation results of each computing node and output unified defense instructions. In this context, each computing node in the heterogeneous computing cluster layer exchanges graph status data with the graph construction and storage unit to achieve cross-node risk situation synchronization.

[0007] Furthermore, the network interface module is configured with a DPDK-accelerated packet processing engine, supporting 10Gbps line-speed capture, and encapsulates the dialogue data into an internal message format before sending it to the intent recognition acceleration card.

[0008] Furthermore, the intent recognition acceleration card integrates the BERT inference engine, uses GPU or NPU to achieve millisecond-level intent classification, with a processing time of less than 2ms per round, and outputs intent type identifier, basic risk score and timestamp.

[0009] Furthermore, the graph construction and storage unit is implemented based on the GalaxyBase graph database and uses an in-memory computing engine to maintain a dynamic directed graph G=(V,E). The node attributes of the graph include intent type, basic risk score and timestamp, and the edge weights are determined by a predefined logical dependency matrix.

[0010] Furthermore, the nonlinear risk calculation unit is configured to perform the following operations: Query all paths from historical nodes to the current node from the graph construction and storage unit; The comprehensive risk value is calculated based on a nonlinear cumulative formula that combines fusion time decay and intent path accumulation. The computation employs a parallel pipeline architecture, completing the summation of contributions from all historical nodes within 1 microsecond.

[0011] Furthermore, the nonlinear accumulation formula is as follows: in, This is the comprehensive risk assessment value for round N. This is the base risk score for the current round. Let α be the base risk score for the k-th round, α be the historical cumulative amplification factor, and λ be the path length attenuation factor. Let be the path length from round k to round N, used to suppress the influence of excessively long paths. w is a composite term of the product of intentional transfer weights and the product of time decay. j→j+1 The intended weight is Δt, where γ is the time decay factor and 0 < γ < 1. j The time interval between two adjacent rounds of dialogue. This is the predefined weight factor corresponding to the intent type in the k-th round.

[0012] Furthermore, the dynamic threshold controller is configured as follows: Periodically read graph statistics from the graph construction and storage unit to calculate graph connectivity; Dynamic threshold adjustment is performed based on dialogue depth and graph connectivity; the greater the dialogue depth or the higher the connectivity, the lower the interception threshold. The adjusted threshold is fed into the comparator for risk assessment.

[0013] Furthermore, the dynamic threshold adjustment adopts the following formula: in, For dynamic blocking thresholds, Based on the threshold, The term represents the dialogue depth penalty, where N is the current dialogue depth and α is the depth decay coefficient. δ is the connectivity penalty term, C is the number of strongly connected paths, δ is the saturation coefficient, h(U) is the user's historical behavior penalty term, and β, η, and κ are adjustment coefficients.

[0014] Furthermore, the log and audit storage includes a solid-state array composed of multiple NVMe SSDs, with RAID redundancy configuration. The log format includes user identifier, dialogue round, intent sequence, risk accumulation process, judgment result, and timestamp. The management interface includes an independent management network port, providing Web services, REST APIs, and a graphical management interface, supporting online updates of the logical dependency matrix, adjustment of risk calculation parameters, and configuration of defense strategies.

[0015] Furthermore, the heterogeneous computing cluster layer adopts a heterogeneous computing architecture in which CPU, GPU, and NPU work in concert: The CPU is responsible for control flow scheduling and lightweight computing; The GPU or NPU is responsible for accelerating BERT inference on the intent recognition acceleration card; The graph construction and storage unit combines the in-memory computing and indexing technology of the GalaxyBase graph database to achieve high-performance graph operations; Optimal system performance is achieved through the collaborative work of the CPU, GPU, and NPU, and the efficient querying of GalaxyBase.

[0016] The advantage of this invention lies in the fact that the high-concurrency multi-turn dialogue risk detection system for large model services it provides adopts a three-layer distributed architecture. It achieves traffic distribution through the access scheduling layer, performs parallel risk detection through the heterogeneous computing cluster layer, and coordinates unified defense through the global decision layer, thus solving the technical defects of existing solutions in terms of high concurrency, low latency, and high reliability. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the high-concurrency multi-turn dialogue risk detection system for large model services proposed in this application; Figure 2 This is a schematic diagram of the computing nodes in this application. Detailed Implementation

[0019] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0020] In the description of this application, it should be noted that, unless otherwise specified and limited, the terms "installation", "connection" and "linkage" should be interpreted broadly, and can refer to mechanical or electrical connections, or internal connections between two components, or direct connections. "Up", "down", "left", "right", etc., are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may change.

[0021] like Figure 1 The image shows a high-concurrency multi-turn dialogue risk detection system for large model services according to this application, including: an access scheduling layer, a heterogeneous computing cluster layer, and a global decision layer.

[0022] The access scheduling layer is configured with a load balancer and a session affinity routing module to receive multiple dialogue data streams and distribute them to downstream processing nodes according to user identifiers. As the system's traffic entry point, the access scheduling layer is responsible for distributing and scheduling multiple concurrent dialogue data streams. The load balancer configured in this layer uses a round-robin or hash algorithm to evenly distribute massive external dialogue requests to downstream processing nodes; the session affinity routing module ensures that multiple rounds of dialogue from the same user are continuously routed to fixed computing nodes, avoiding context loss caused by session state migration between nodes. This layer achieves horizontal scalability by decoupling access and processing, supporting concurrent access at tens of thousands of QPS.

[0023] Specifically, the load balancer is deployed at the system traffic ingress, employing hardware load balancing devices or software load balancing based on DPDK, supporting network access capabilities of 10Gbps and even 100Gbps. The load balancer maintains the health status and load of each computing node in the downstream heterogeneous computing cluster layer, distributing external dialogue requests to processing nodes using weighted round-robin or consistent hashing algorithms. The key mechanism of the session affinity routing module is: extracting user identifiers (such as user ID, session token, or IP address combinations) from the dialogue request, determining the target computing node through hash calculation, and consistently routing all subsequent dialogue requests from the same user to that node throughout the session's duration. This mechanism ensures that the context state of multi-turn dialogues is processed in a closed loop within a single node, avoiding latency and consistency risks associated with inter-node state synchronization. For extremely long sessions (e.g., exceeding 100 turns), the session affinity routing module supports a session migration protocol, smoothly migrating the session state to a lighter-loaded node while maintaining the continuity of the risk graph.

[0024] The heterogeneous computing cluster layer consists of multiple computing nodes coupled through a high-speed interconnect network. The cluster layer is configured to maintain a distributed dynamic intent graph and perform risk accumulation calculations in parallel. Specifically, the heterogeneous computing cluster layer comprises a distributed processing array of multiple physical computing nodes. Each node achieves low-latency coupling through a high-speed interconnect network (such as InfiniBand or RoCE), forming a unified computing resource pool. This layer adopts a sharded architecture, with each node independently maintaining its local intent graph shard and incrementally synchronizing the graph state through a distributed protocol. Internally, each node integrates seven functions: network interface, intent recognition, graph storage, risk calculation, threshold control, log storage, and management interface, forming a complete risk detection pipeline. Specifically, such as... Figure 2 As shown, each computing node comprises seven functional modules: a network interface module, an intent recognition accelerator card, a graph construction and storage unit, a nonlinear risk calculation unit, a dynamic threshold controller, and a log and audit storage and management interface, forming a complete risk detection pipeline. The hardware implementation and working principle of each module are explained in detail below.

[0025] The network interface module is used to capture dialogue data streams distributed by the access scheduling layer. It is equipped with a DPDK-accelerated packet processing engine, supporting 10Gbps line-rate capture, and encapsulates the dialogue data into an internal message format before sending it to the intent recognition acceleration card. Specifically, the network interface module is deployed at the traffic ingress location of each compute node, employing the DPDK (Data Plane Development Kit) user-space packet processing framework. It bypasses the operating system kernel protocol stack and directly manipulates the network card hardware to complete packet capture. This module supports 10Gbps Ethernet line-rate processing, receives network traffic in polling mode, parses raw packets into structured dialogue metadata (user identifier, timestamp, message content), and encapsulates it into an internal system message format before sending it to the downstream intent recognition acceleration card. The module's zero-copy design avoids the data transfer overhead between kernel and user space.

[0026] The network interface module, accelerated by DPDK, reduces packet capture latency from milliseconds to microseconds compared to traditional kernel-mode network processing, meeting the stringent real-time requirements of financial transaction scenarios. Its 10Gbps line-speed processing capability ensures no packet loss under high traffic surges, providing complete data input for risk detection. The standardized encapsulation of internal message formats decouples the network access layer from the business processing layer, facilitating independent upgrades of subsequent modules.

[0027] The intent recognition accelerator card is coupled to the network interface module to classify the execution intent for each round of dialogue and output a basic risk score. In the embodiments of this application, the intent recognition accelerator card integrates the BERT inference engine, uses a GPU or NPU to achieve millisecond-level intent classification, and has a processing time of less than 2ms per round. The output includes an intent type identifier, a basic risk score, and a timestamp.

[0028] Specifically, the intent recognition accelerator card is the system's AI inference execution unit, integrating a BERT-based pre-trained model for intent classification tasks. This accelerator card uses dedicated AI accelerators (GPUs or NPUs) for model inference, and through operator fusion and batch processing optimization, it compresses the intent recognition latency for single-turn dialogues to less than 2ms. The output uses a structured triple format: intent type identifier (mapped to a predefined intent category), basic risk score (a continuous value of 0-1, output by the single-turn detection model), and timestamp (used for subsequent time decay calculations). This module supports hot model updates, allowing new versions of the intent classification model to be loaded online via a management interface.

[0029] The intent recognition acceleration card, which uses a dedicated AI accelerator, achieves more than 10 times the throughput improvement compared to CPU inference, reducing the unit inference cost while meeting real-time constraints; millisecond-level response ensures that risk detection does not block normal business dialogue; standardized output format provides a unified input interface for downstream graph construction and risk calculation; hot update capability enables the system to quickly respond to new attack modes and upgrade detection capabilities without downtime.

[0030] The graph construction and storage unit is coupled with the intent recognition acceleration card to maintain local intent graph fragments. In the embodiments of this application, the graph construction and storage unit is implemented based on the GalaxyBase graph database and uses an in-memory computing engine to maintain a dynamic directed graph G=(V,E). The node attributes of the graph include intent type, basic risk score, and timestamp, and the edge weights are determined by a predefined logical dependency matrix. The graph construction and storage unit is the core data management component of the system and is implemented based on the GalaxyBase native distributed graph database. This unit uses an in-memory computing engine to maintain a dynamic directed graph G=(V,E), where node V represents a single-turn dialogue, and node attributes include intent type, basic risk score, and timestamp; edge E represents intent transition relationships, and the edge weights are determined by a predefined logical dependency matrix. This unit supports real-time graph updates (adding nodes, establishing edge connections) and subgraph queries (retrieving all paths from historical nodes to the current node), stores adjacency relationships in CSR (Compressed Sparse Row) format, and uses the SIMD instruction set to accelerate graph traversal operations.

[0031] The graph construction and storage units of the GalaxyBase graph database reduce the complexity of multi-hop path queries compared to general relational databases and support millisecond-level risk path retrieval. The in-memory computing engine ensures that high-frequency graph operations do not require disk I / O, meeting real-time processing requirements. The injection of predefined logical dependency matrices makes domain knowledge explicit, providing complete interpretability compared to black-box neural network solutions. The distributed architecture supports graph sharding expansion, breaking through single-machine memory limitations.

[0032] A nonlinear risk calculation unit is coupled to a map construction and storage unit for performing risk accumulation calculations. In an embodiment of this application, the nonlinear risk calculation unit is configured to perform the following operations: The graph construction and storage unit queries all paths from historical nodes to the current node. The comprehensive risk value is calculated based on the nonlinear cumulative formula of fusion time decay and intent path accumulation. The calculation adopts a parallel pipeline architecture and completes the summation of the contributions of all historical nodes within 1 microsecond.

[0033] Specifically, the nonlinear risk calculation unit, as the system's risk quantification execution component, retrieves the historical path set of the current session from the graph construction and storage unit, and performs nonlinear cumulative calculations that integrate time decay and path multiplication. This unit adopts a parallel pipeline architecture: the path retrieval phase utilizes graph database indexes to obtain multi-hop paths in parallel; the weight calculation phase uses a lookup table method to quickly obtain the time decay factor; and the cumulative summation phase uses a tree-like reduction structure to accumulate the contribution of each historical path in parallel. This architecture supports the summation of the risk contributions of all historical nodes within 1 microsecond, outputting the comprehensive risk value for the current round. The parallel pipeline architecture of the nonlinear risk calculation unit achieves orders-of-magnitude latency compression compared to serial computation, meeting the stringent latency requirements of financial-grade real-time risk control; the introduction of the time decay factor effectively distinguishes closely connected induced questions from those unrelated to history, avoiding accidental noise interference; the path multiplication mechanism forms a logic gate effect, ensuring that only dialogues with coherent intent logic chains can accumulate risk, accurately identifying progressive attacks; and the microsecond-level computation latency makes risk detection transparent to the business system, without affecting user experience.

[0034] In the embodiments of this application, the nonlinear accumulation formula is: in, This is the comprehensive risk assessment value for round N. This is the base risk score for the current round. Let α be the base risk score for the k-th round, α be the historical cumulative amplification factor, and λ be the path length attenuation factor. Let be the path length from round k to round N, used to suppress the influence of excessively long paths. w is a composite term of the product of intentional transfer weights and the product of time decay.j→j+1 The intended weight is Δt, where γ is the time decay factor and 0 < γ < 1. j The time interval between two adjacent rounds of dialogue. This is the predefined weight factor corresponding to the intent type in the k-th round.

[0035] The nonlinear risk calculation unit employs an improved cumulative formula to quantify the combined impact of historical dialogues on current risk. This formula introduces a triple attenuation mechanism into its basic structure: a path length attenuation factor. The contribution of excessively long historical paths is suppressed through exponential means to avoid interference from early irrelevant dialogues in the current judgment; time decay multiplication is used. Simulating the cognitive forgetting curve, the longer the dialogue interval, the weaker the risk transmission; intention type weighting factor. Differentiated base weights are predefined for each type of intent to strengthen the historical impact of high-risk intents. The triple decay and intent transfer weights are multiplied together to form a composite gating effect. Only when the intent logic chain is coherent, the time connection is close, and the path length is moderate can historical risks be effectively accumulated to the current round.

[0036] This nonlinear cumulative formula achieves refined risk quantification and scenario adaptability through the synergistic design of a triple attenuation mechanism. Path length attenuation effectively defends against historical noise interference in ultra-long sessions, time attenuation distinguishes between closely induced and casual questions, and intent weighting factors inject domain knowledge to strengthen high-risk identification.

[0037] A dynamic threshold controller is coupled to a nonlinear risk calculation unit to compare the cumulative risk value with a dynamic threshold. In an embodiment of this application, the dynamic threshold controller is configured to: periodically read graph statistics from the graph construction and storage unit, calculate graph connectivity, perform dynamic threshold adjustment based on dialogue depth and graph connectivity (the higher the dialogue depth or connectivity, the lower the interception threshold), and send the adjusted threshold to a comparator for risk determination.

[0038] In the embodiments of this application, the dynamic threshold adjustment adopts the following formula: in, For dynamic blocking thresholds, Based on the threshold, The term represents the dialogue depth penalty, where N is the current dialogue depth and α is the depth decay coefficient. The term represents the connectivity penalty, where C is the number of strongly connected paths (defined as the path product of all paths with length ≥ 2 that exceeds a threshold). (the number of paths), where δ is the saturation coefficient. This refers to the user's historical behavior penalty items (the system should be more vigilant for users who have repeatedly triggered risk interception or frequently expressed high-risk intentions; personalized defense, different users have different risk profiles, using a uniform threshold will lead to insufficient sensitivity to high-risk users and excessive sensitivity to normal users; attackers may continuously probe in multiple sessions, and the accumulated information of cross-session behavior helps to identify malicious intent earlier), and U is the user identifier. This represents the number of sessions that user U has had blocked within a certain period (typically 7 days). K is the saturation threshold (when the number of blocks reaches this value, h(U)=1, and the penalty no longer increases. For example, if K=2, two blocks within 7 days will reach the maximum penalty, prompting a rapid increase in vigilance). make sure Prevent overflow. η and κ are adjustment coefficients.

[0039] The dynamic threshold controller uses a saturated nonlinear formula to adaptively adjust the interception sensitivity. Dialogue depth penalty term. An exponential decay mechanism is employed, with the threshold decreasing rapidly in the early stages of the dialogue to increase sensitivity, and then stabilizing in the later stages to avoid excessive interception; a connectivity penalty term is also included. A saturation-type growth model is adopted, where the penalty contribution diminishes marginally as the number of strongly connected paths increases, preventing excessive threshold drop due to overfitting. The user's historical behavior penalty term h(U) incorporates a user profile dimension. These three factors are coupled with the basic threshold through linear combination to form a personalized dynamic defense curve.

[0040] This dynamic threshold formula achieves smooth evolution and personalized adaptation of defense strategies through a saturated nonlinear design. Exponential depth penalties quickly establish protection in the early stages, while saturated connectivity penalties avoid extreme thresholds. User history factors enable differentiated risk control tailored to each individual user. This design ensures the system maintains reasonable sensitivity across different dialogue stages, risk profiles, and user groups.

[0041] The log and audit storage is used to record risk assessment logs. The management interface is used to receive policy configuration instructions. In the embodiments of this application, the log and audit storage includes a solid-state array composed of multiple NVMe SSDs, using RAID redundancy configuration. The log format includes user identifier, dialogue round, intent sequence, risk accumulation process, assessment result, and timestamp. The management interface includes an independent management network port, providing web services, REST API, and a graphical management interface, supporting online updates of the logical dependency matrix, adjustment of risk calculation parameters, and configuration of defense policies.

[0042] The log and audit storage and management interface constitute the system's operation and maintenance support subsystem. The log and audit storage uses a redundant RAID array composed of multiple NVMe SSDs to persistently store three types of data: complete dialogue graphs (for attack tracing), risk assessment records (including cumulative calculation processes), and system alarm logs. The management interface provides out-of-band management capabilities, physically isolated from the production network through a dedicated management port. It exposes three types of interfaces: Web services, REST APIs, and a graphical management interface, supporting operations personnel in configuring policies (updating logical dependency matrices, adjusting risk parameters, and modifying defense rules) and monitoring the system (real-time risk situation visualization and historical session playback).

[0043] The RAID redundancy design of log and audit storage ensures high reliability of audit data and meets the immutability requirements of financial compliance; the high throughput of NVMe SSDs supports real-time writing of massive logs, avoiding log blockage that could affect the core processing chain; the out-of-band design of the management interface achieves physical isolation between operation and maintenance traffic and production traffic, preventing the management channel from becoming an attack entry point; the graphical interface lowers the threshold for operation and maintenance, and the REST API supports automated operation and maintenance integration, forming a closed-loop security operation system.

[0044] In the embodiments of this application, the heterogeneous computing cluster layer adopts a heterogeneous computing architecture in which CPU, GPU, and NPU work in collaboration. The CPU is responsible for control flow scheduling and lightweight computing. The GPU or NPU is responsible for accelerating BERT inference on the intent recognition accelerator card. The graph construction and storage unit combines the in-memory computing and indexing technology of the GalaxyBase graph database to achieve high-performance graph operations. Optimal system performance is achieved through the collaborative work of the CPU, GPU, and NPU and the efficient querying of GalaxyBase. Specifically, the heterogeneous computing cluster layer adopts a collaborative architecture of three types of processors: CPU, GPU, and NPU. The CPU is responsible for task scheduling, control flow management, and lightweight computing (such as threshold comparison and log formatting); the GPU or NPU is responsible for accelerating BERT inference on the intent recognition accelerator card, using tensor cores to achieve parallelization of matrix operations; the graph construction and storage unit combines the in-memory computing engine of GalaxyBase, accelerating graph operations through a dedicated index structure and SIMD optimization. The three types of processors share data through a unified memory architecture or high-speed interconnect protocol, avoiding cross-device data transfer overhead and achieving optimal overall system performance.

[0045] The heterogeneous collaborative architecture of CPU / GPU / NPU achieves a 3-5 times improvement in energy efficiency compared to homogeneous CPU clusters, reducing data center operating costs while meeting computing power requirements; hierarchical task scheduling ensures that each processor performs the type of computation it is best at, avoiding resource mismatch; deep adaptation of GalaxyBase to heterogeneous computing reduces pipeline latency for graph query and AI inference to the theoretical lower limit; the architecture supports independent expansion (adding GPU nodes to increase AI throughput, or adding memory nodes to increase graph capacity), adapting to the evolving needs of different business workloads.

[0046] The global decision-making layer and the heterogeneous computing cluster layer are coupled via a message bus, configured to aggregate the risk calculation results of each computing node and output unified defense commands. As the system's coordination hub, the global decision-making layer collects the risk calculation results of each computing node through the message bus and executes cross-node risk posture aggregation. This layer is configured with a conflict resolution strategy; when multiple nodes have differing judgments on the same session, a conservative strategy is adopted to prioritize the higher-risk judgment. Simultaneously, this layer is responsible for outputting unified defense commands to external business systems, achieving decoupling between distributed computing and centralized decision-making. The global decision-making layer is deployed on an independent high-availability server cluster and coupled with each computing node through the message bus. The core functions of the global decision-making layer are cross-node risk posture aggregation and unified defense decision-making. When attackers attempt to induce attacks through multiple accounts and multiple sessions in parallel, their risk signals are dispersed across multiple computing nodes. The global decision-making layer subscribes to the risk outputs of each node through the message bus and identifies cross-session coordinated attacks based on correlation features such as user profiles, device fingerprints, and behavioral patterns. The conflict resolution strategy adopts a conservative approach: when multiple nodes disagree on the risk assessment of the same entity (user, device, IP range), the assessment result with the highest risk level is adopted to ensure no cases are missed. The global decision-making layer is also responsible for outputting unified defense instructions to external business systems. The instruction format follows standardized risk control interfaces and supports various defense actions such as refusing to answer, transferring to manual review, topic shifting, session termination, and account freezing. The instruction output latency is controlled within 5 milliseconds, and the total end-to-end latency from user inquiry to defense triggering does not exceed 10 milliseconds.

[0047] In a heterogeneous computing cluster layer, each computing node exchanges graph state data with the graph construction and storage units to achieve cross-node risk posture synchronization. Specifically, the graph construction and storage units of each computing node establish a point-to-point communication channel through a high-speed interconnect network, and use an incremental synchronization protocol to transmit changes to the risk graph. When a computing node needs to obtain graph data maintained by other nodes due to session migration, load balancing adjustments, or global risk analysis requirements, the graph construction and storage units of the source node serialize the subgraph structure, node attributes, edge weights, and risk accumulation status of the target session into a binary data stream, and write it directly into the target node's memory space using RDMA technology, avoiding the data copying overhead of the traditional TCP / IP protocol stack. The synchronization process uses a version vector mechanism to record the timing of data changes, ensuring the consistency of the graph state in a distributed environment; at the same time, differential compression algorithms are used to transmit only incremental changes, keeping network bandwidth usage within 5% of the total bandwidth between nodes, ensuring the transmission priority of core business traffic.

[0048] The cross-node risk situation synchronization mechanism solves the state consistency problem in distributed architectures, enabling the system to maintain continuous risk detection while ensuring horizontal scalability. When attackers attempt to circumvent single-node detection by distributing conversations across multiple accounts, the global decision-making layer can use this mechanism to quickly aggregate risk signals from each node and identify cross-session collaborative attack patterns. When a computing node fails, the session can be seamlessly migrated to a backup node, and the complete inheritance of the risk graph ensures uninterrupted detection and no loss of accumulated risks. Real-world testing shows that in a 100Gbps high-speed network environment, this synchronization mechanism controls the cross-node migration latency of a million-node graph to within 50 milliseconds, with no impact on business operations.

[0049] This application's high-concurrency multi-turn dialogue risk detection system for large-model services adopts a three-layer distributed architecture, achieving three technological breakthroughs compared to a single-machine deployment: First, by utilizing the session affinity mechanism at the access scheduling layer, it maintains the contextual continuity of multi-turn dialogues in a distributed environment, resolving the contradiction between horizontal scaling and session state consistency; second, by using sharded graph storage at the heterogeneous computing cluster layer, it distributes the storage pressure of large-scale graphs across nodes, overcoming the limitations of single-machine memory capacity; and third, by using a risk aggregation mechanism at the global decision layer, it achieves collaborative detection of cross-session related attacks. This architecture supports elastic scaling, dynamically adding or removing computing nodes based on business load, meeting financial-grade high-concurrency requirements while maintaining risk detection accuracy.

[0050] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A high-concurrency multi-turn dialogue risk detection system for large model services, characterized in that, include: The access scheduling layer is configured with a load balancer and a session affinity routing module, which are used to receive multiple dialogue data streams and distribute them to downstream processing nodes according to user identifiers. A heterogeneous computing cluster layer, composed of multiple computing nodes coupled via a high-speed interconnect network, is configured to maintain a dynamic intent graph in a distributed manner and perform risk accumulation calculations in parallel. Each computing node includes: The network interface module is used to capture the dialogue data stream distributed by the access scheduling layer; An intent recognition acceleration card, coupled to the network interface module, is used to classify the execution intent of each round of dialogue and output a basic risk score. The graph construction and storage unit is coupled to the intent recognition acceleration card and is used to maintain local intent graph fragments; A nonlinear risk calculation unit, coupled to the map construction and storage unit, is used to perform risk accumulation calculation; A dynamic threshold controller, coupled to the nonlinear risk calculation unit, is used to compare the cumulative risk value with the dynamic threshold. Logs and audit storage are used to record risk assessment logs; The management interface is used to receive policy configuration instructions; The global decision-making layer, coupled with the heterogeneous computing cluster layer via a message bus, is configured to aggregate the risk calculation results of each computing node and output unified defense commands.

2. The high-concurrency multi-turn dialogue risk detection system for large model services according to claim 1, characterized in that, Each computing node in the heterogeneous computing cluster layer exchanges graph status data with the graph construction and storage unit to achieve cross-node risk situation synchronization.

3. The high-concurrency multi-turn dialogue risk detection system for large model services according to claim 1, characterized in that, The network interface module is equipped with a DPDK-accelerated packet processing engine, which supports 10Gbps line-speed capture and encapsulates the dialogue data into an internal message format before sending it to the intent recognition acceleration card. The intent recognition acceleration card integrates a BERT inference engine and uses a GPU or NPU to achieve millisecond-level intent classification. Each round of processing takes less than 2ms, and the output includes an intent type identifier, a basic risk score, and a timestamp.

4. The high-concurrency multi-turn dialogue risk detection system for large model services according to claim 1, characterized in that, The graph construction and storage unit is implemented based on the GalaxyBase graph database and uses an in-memory computing engine to maintain a dynamic directed graph G=(V,E). The node attributes of the graph include intent type, basic risk score and timestamp, and the edge weight is determined by a predefined logical dependency matrix.

5. The high-concurrency multi-turn dialogue risk detection system for large model services according to claim 1, characterized in that, The nonlinear risk calculation unit is configured to perform the following operations: Query all paths from historical nodes to the current node from the graph construction and storage unit; The comprehensive risk value is calculated based on a nonlinear cumulative formula that combines fusion time decay and intent path accumulation. The computation employs a parallel pipeline architecture, completing the summation of contributions from all historical nodes within 1 microsecond.

6. The high-concurrency multi-turn dialogue risk detection system for large model services according to claim 5, characterized in that, The nonlinear accumulation formula is as follows: in, This is the comprehensive risk assessment value for round N. This is the base risk score for the current round. Let α be the base risk score for the k-th round, α be the historical cumulative amplification factor, and λ be the path length attenuation factor. Let be the path length from round k to round N, used to suppress the influence of excessively long paths. w is a composite term consisting of the product of the intended transfer weights and the product of time decay. j→j+1 The intended weight is Δt, where γ is the time decay factor and 0 < γ < 1. j The time interval between two adjacent rounds of dialogue. This is the predefined weight factor corresponding to the intent type in the kth round.

7. The high-concurrency multi-turn dialogue risk detection system for large model services according to claim 1, characterized in that, The dynamic threshold controller is configured as follows: Periodically read graph statistics from the graph construction and storage unit to calculate graph connectivity; Dynamic threshold adjustment is performed based on dialogue depth and graph connectivity; the greater the dialogue depth or the higher the connectivity, the lower the interception threshold. The adjusted threshold is fed into the comparator for risk assessment.

8. The high-concurrency multi-turn dialogue risk detection system for large model services according to claim 7, characterized in that, The dynamic threshold adjustment uses the following formula: in, For dynamic blocking thresholds, Based on the threshold, The term represents the dialogue depth penalty, where N is the current dialogue depth and α is the depth decay coefficient. δ is the connectivity penalty term, C is the number of strongly connected paths, δ is the saturation coefficient, h(U) is the user's historical behavior penalty term, and β, η, and κ are adjustment coefficients.

9. The high-concurrency multi-turn dialogue risk detection system for large model services according to claim 1, characterized in that, The log and audit storage includes a solid-state array composed of multiple NVMe SSDs, with RAID redundancy configuration. The log format includes user identifier, dialogue round, intent sequence, risk accumulation process, judgment result and timestamp. The management interface includes an independent management network port, providing Web services, REST APIs, and a graphical management interface, supporting online updates of the logical dependency matrix, adjustment of risk calculation parameters, and configuration of defense strategies.

10. The high-concurrency multi-turn dialogue risk detection system for large model services according to claim 1, characterized in that, The heterogeneous computing cluster layer adopts a heterogeneous computing architecture in which CPU, GPU, and NPU work together: The CPU is responsible for control flow scheduling and lightweight computing; The GPU or NPU is responsible for accelerating BERT inference on the intent recognition acceleration card; The graph construction and storage unit combines the in-memory computing and indexing technology of the GalaxyBase graph database to achieve high-performance graph operations; Optimal system performance is achieved through the collaborative work of the CPU, GPU, and NPU, and the efficient querying of GalaxyBase.

Images