Concurrent debugging method and system based on sandbox-llm session bidirectional binding

Abstract

The application provides a concurrent debugging method and system based on sandbox-LLM session bidirectional binding, the method comprising: allocating a globally unique session identifier for each large language model (LLM) session; writing the globally unique session identifier and the sandbox instance identifier into a distributed memory database to form a bidirectional binding table, and marking the initial state as a binding state; realizing the transmission of the globally unique session identifier through an environment variable and an HTTP / gRPC header field; when the sandbox is hot migrated, the old instance atomically updates the binding table entry state to a migration state through a distributed mutex, and the new instance writes the binding state after completing the preheating within a preset time; a log agent aggregates indexes according to the sandbox instance identifier, collects the average generation delay, the cumulative data collection number and the GPU memory occupation according to a preset frequency; and periodically polling the binding table, and executing cleaning on the entries that exceed the preset threshold of the distributed mutex and have a recycling state.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electronic digital data processing, in particular to a concurrent debugging method and system based on sandbox-LLM session bidirectional binding. BACKGROUND

[0002] When concurrently debugging large language model (LLM) applications in a multi-container or sandbox environment, common pain points include:

[0003] Session string package: LLM request links span containers, Sidecars, and reverse proxies, and logs are difficult to split by session;

[0004] Hot migration loses context: sandboxes are migrated due to resource scheduling or faults, and debug Session IDs are invalid;

[0005] Cross-link tracking is missing: distributed link tracking (OpenTelemetry, etc.) lacks the semantic layer of "LLM session", making it difficult to quickly associate prompts, fine-tuning weights, and in-sandbox logs.

[0006] Existing solutions (such as Jaeger + Sidecar log collection) can only guarantee HTTP-level traceId association, and cannot establish a one-to-one mapping between LLM Token-level and sandbox lifecycles.

[0007] Patent application document CN119026632A discloses a dynamic constraint multi-objective evolution method based on large language model interaction, including: initializing a population; dividing the population into target and constraint types; passing non-dominated individuals to a large language model to save historical information; detecting whether the environment has changed, and if it has changed, classifying the environment change type; if the environment has changed, starting an interaction strategy with the large language model to produce a response individual according to the environment change type; if the environment has not changed, using LLM interaction to produce offspring; connecting a deep learning framework to return a reward value as a target function for individual evaluation, selecting non-dominated individuals and updating the population and archive; finally, determining whether the number of fitness evaluations is less than the maximum number of function evaluations, if it is, outputting the optimal solution and optimal value of the population; otherwise, entering the next generation population evolution. However, this patent cannot completely solve the existing technical problems and cannot meet the needs of the present application. SUMMARY

[0008] In view of the defects in the prior art, the purpose of the present application is to provide a concurrent debugging method and system based on sandbox-LLM session bidirectional binding.

[0009] The concurrent debugging method based on sandbox-LLM session bidirectional binding provided by the present application comprises:

[0010] Step 1: Assign a globally unique session identifier to each large language model LLM session;

[0011] Step 2: Write the globally unique session identifier and sandbox instance identifier into the distributed memory database to form a bidirectional binding table, and mark the initial state as a binding state;

[0012] Step 3: Realize the transmission of the globally unique session identifier through environment variables and HTTP / gRPC header fields;

[0013] Step 4: When the sandbox is hot migrated, the old instance atomically updates the binding table entry state to a migration state through a distributed mutex, and the new instance writes the binding state after completing the preheating within a preset time;

[0014] Step 5: The log agent aggregates indicators according to the sandbox instance identifier, collects average generation delay, cumulative data collection number, and GPU memory usage at a preset frequency; when a violation is detected, a debug error report is generated, which includes the globally unique session identifier, the new and old sandbox instance identifier, the migration timestamp, the data collection rate, the GPU utilization, and the error stack;

[0015] Step 6: Periodically poll the binding table, and perform cleaning on entries that exceed the preset threshold of the distributed mutex and are in the recycling state.

[0016] Preferably, the preheating of the new instance includes: loading large language model LLM weight cache and session index identifier, synchronizing text instructions and query history cache through memory database stream, and performing empty reasoning operation on GPU to avoid cold start delay.

[0017] Preferably, the violation detection adopts a dynamic threshold model:

[0018]

[0019] wherein μ(t) is the baseline data collection rate generated by the exponential weighted moving average, σ(t) is the standard deviation, and α is the sensitivity parameter; when the data collection rate of two consecutive time windows exceeds the threshold, an abnormal mark is triggered.

[0020] Preferably, GPU utilization mutation analysis is performed: calculate the cosine similarity between the current session GPU usage curve and the normal baseline of the same type of text instruction and query; when the similarity is lower than the preset threshold and accompanied by a data collection rate that does not meet the preset range, a joint alarm is triggered.

[0021] Preferably, the log agent comprises: adopting a bucketing strategy to process back pressure, bucketing by global unique session identifier and sliding time window, suspending the log agent and returning a corresponding response when a single bucket accumulates more than a preset number of data collection events; instructing the client to retry the request after a specified time through the response header, and temporarily storing the bucketed data using a lightweight compression algorithm.

[0022] The concurrent debugging system based on sandbox-LLM session bidirectional binding provided by the application comprises:

[0023] Module M1: assigning a global unique session identifier to each large language model LLM session;

[0024] Module M2: writing the global unique session identifier and the sandbox instance identifier into a distributed memory database to form a bidirectional binding table, and marking the initial state as a binding state;

[0025] Module M3: realizing the transmission of the global unique session identifier through an environment variable and an HTTP / gRPC header field;

[0026] Module M4: when the sandbox is hot-migrated, the old instance atomically updates the binding table entry state to a migration state through a distributed mutex, and the new instance writes the binding state after completing the warm-up within a preset time;

[0027] Module M5: the log agent aggregates indicators according to the sandbox instance identifier, collects the average generation delay, the cumulative data collection number and the GPU memory occupation at a preset frequency, generates a debugging error report when a violation is detected, and the report contains the global unique session identifier, the new and old sandbox instance identifiers, the migration timestamp, the data collection rate, the GPU utilization and the error stack;

[0028] Module M6: periodically polling the binding table, and performing cleaning on the entries that exceed the preset threshold of the distributed mutex and are in the recycling state.

[0029] Preferably, the new instance warm-up comprises: loading the large language model LLM weight cache and the session index identifier, synchronizing the text instruction and the query history cache through the memory database stream, and performing an empty reasoning operation on the GPU to avoid the cold start delay.

[0030] Preferably, the violation detection adopts a dynamic threshold model:

[0031]

[0032] wherein μ(t) is a baseline data collection rate generated by an exponential weighted moving average, σ(t) is a standard deviation, and α is a sensitivity parameter; when the data collection rates of two consecutive time windows exceed the threshold, an abnormality mark is triggered.

[0033] Preferably, GPU utilization mutation analysis is performed: the cosine similarity of the current session GPU usage curve with the normal baseline of the same type of text instruction and query is calculated, when the similarity is lower than the preset threshold and accompanied by a data collection rate that does not meet the preset range, a joint alarm is triggered.

[0034] Preferably, the log agent comprises: adopting a bucketing strategy to process back pressure, and performing bucketing according to a globally unique session identifier and a sliding time window; when a single bucket accumulates more than a preset number of data collection events, the log agent is suspended and a corresponding response is returned; the client is instructed by a response header to retry the request after a specified time, and the bucketed data is temporarily stored using a lightweight compression algorithm.

[0035] Compared with the prior art, the present application has the following beneficial effects:

[0036] The present application uses the "sandbox-LLM session bidirectional binding + MOVING state migration protocol + full-link session transparent transmission" technology, solves the problems of session string package and migration context loss in concurrent debugging, realizes one-to-one binding of LLM session and sandbox instance, automatic renewal and traceable log aggregation, and provides millisecond-level debugging query capability. BRIEF DESCRIPTION OF DRAWINGS

[0037] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings:

[0038] Figure 1 is a method flowchart;

[0039] Figure 2 is a MOVING state timing diagram. DETAILED DESCRIPTION

[0040] The present application will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that, for those skilled in the art, without departing from the concept of the present application, a number of changes and improvements can be made. These all belong to the protection scope of the present application.

[0041] Example 1

[0042] As Figure 1 , the present application provides a concurrent debugging method based on sandbox-LLM session bidirectional binding, comprising:

[0043] A) Session initialization: assigning a globally unique session identifier SessionID to each LLM session;

[0044] B) Bidirectional binding registration: At the start of the sandbox instance, write the SessionID and the sandbox instance identification SandboxID into the distributed binding table, and record the initial state as BOUND;

[0045] C) Full link transmission: In the sandbox and its Sidecar process, inject the SessionID into the environment variable and / or add it to the HTTP or gRPC header field "X-LLM-Session" to achieve request link transmission;

[0046] D) Hot migration renewal:

[0047] D1) When the orchestration scheduling system triggers the sandbox hot migration, the old sandbox instance updates the binding table entry state atomically to MOVING through heartbeat detection combined with a distributed mutex, and returns an acknowledgement signal; the MOVING state timing diagram is as follows Figure 2 ;

[0048] To ensure the atomicity and consistency of the sandbox hot migration process, the present application introduces the following collaborative mechanism and fault tolerance design between the distributed mutex and the heartbeat detection:

[0049] 1. Mutex implementation mechanism:

[0050] The distributed mutex is implemented through the SETNX command of Redis, and the key format is llm:lock: <sessionid>, set lock holding time expire = 300 ms, ensure single instance holding lock and automatic expiration to prevent deadlock.

[0051] 2. MOVING state write logic:

[0052] The old sandbox detects the scheduling event in every 50 ms heartbeat cycle, and first attempts to acquire the above mutex; only the successful lock holder can atomically update the binding table entry to MOVING, and immediately release the lock; this process is wrapped by the Lua script to ensure the atomicity of Redis operations.

[0053] 3. Crash boundary scenario processing:

[0054] If the old instance crashes or loses connection unexpectedly after successfully marking MOVING, the Watchdog process will consider the entry as "non-renewable" after the continuous heartbeat timeout (such as losing 3 consecutive heartbeats), allowing the new instance to force renewal to BOUND after detecting MOVING state timeout (default P95 < 200 ms), and recording the partial migration event for debugging system to leave traces.

[0055] 4. Anti-lock mechanism:

[0056] To avoid deadlock caused by multiple node competition for lock, all locks are forced to set expire TTL (such as 300 ms), and any form of infinite retry is prohibited; if the old instance fails to complete state update within TTL, it is considered as lock acquisition failure, and the scheduler automatically delays the migration action by 1 second for retry or switches to the backup migration path.

[0057] 5. Backoff mechanism for failed write to MOVING state:

[0058] If the old instance fails to successfully write to MOVING state (such as Redis write failure or lock conflict), the current hot migration action is interrupted, and the scheduler retry process will bring the retry identifier attempt = x and record it in the event log, ensuring that the system behavior is auditable.

[0059] To ensure that the old sandbox instance has strong consistency and atomicity guarantee when atomically updating the binding table entry state to MOVING, the present application adopts the following multi-level technical mechanisms:

[0060] 1. Atomic write guarantee:

[0061] The state update operation is implemented through the Lua script (EVAL) of Redis, which integrates operations such as acquiring mutex, judging current state, setting new state MOVING, recording timestamp, etc. into a single Redis atomic execution transaction, preventing concurrent write conflicts in the "read-modify-write" process.

[0062] Example operation:

[0063] if redis.call('get', KEYS[1]) == 'BOUND' then

[0064] redis.call('set', KEYS[1], 'MOVING')

[0065] redis.call('set', KEYS[2], ARGV[1])-- Set timestamp

[0066] return 1

[0067] else

[0068] return 0

[0069] end

[0070] 2. Mutual exclusion mechanism cooperation:

[0071] Before executing the above Lua script, the old instance must first obtain llm:lock through the SETNX command of Redis: <sessionid>Distributed mutex lock of the binding table, avoiding multiple sandbox competition leading to repeated writing;

[0072] Lock setting survival time for 300 ms, preventing deadlock (automatic release), and identifying unique holder through UUID, ensuring that only the lock holder can release.

[0073] 3. Combination point with CRDT:

[0074] Although RedisLua script is used for atomic writing operation within a single data center, the present application allows deployment in a multi-data center environment, for which the Redis cluster enables CRDT (Conflict-free Replicated Data Type) replica mechanism;

[0075] The <SessionID, {SandboxID, Status, Timestamp}> structure in the binding table is defined as a CRDT type (such as Last-Write-Wins Element Set, LWW-Set), which can realize distributed asynchronous replication and eventual consistency;

[0076] After the old instance is marked as MOVING, the change is immediately written to the local Redis shard and automatically propagated to other data center replica nodes through the CRDT engine, realizing "remote readable consistency".

[0077] 4. Boundary guarantee and failure handling:

[0078] If the Lua script execution fails (such as state conflict or Redis exception), the old instance immediately records the migration failure event and notifies the orchestration system to back off;

[0079] If CRDT propagation is not completed, the new instance can read the "old state" when trying to renew, and the system design has a maximum delay threshold (default 100 ms) to force override, avoiding blocking the migration process due to replica unsynchronization.

[0080] D2) After the new sandbox instance completes preheating, write {SessionID, SandboxID_new, BOUND} in the binding table and release the mutex, making the overall migration window less than 200 milliseconds;

[0081] To ensure that sandbox hot migration is completed within the 200-millisecond P95 window, the system designs the following guarantee mechanisms during the new instance renewal process:

[0082] 1. LLM context preloading strategy in the preheating phase:

[0083] New instance enters pre-warming phase immediately after startup, which does not directly process requests, but loads the following contexts in advance:

[0084] Corresponding LLM weight cache and Session Token index;

[0085] Prompt history cache (session context synchronization through Redis Stream);

[0086] Perform several empty inferences (warmup) on GPU to avoid cold start delay of the first request;

[0087] Only when the above three preparations are completed, {SessionID, SandboxID_new, BOUND} will be attempted to write.

[0088] 2. "Preemptive" renewal write mechanism (Preemptive BOUND Write):

[0089] The system introduces "preemptive renewal" logic, that is, when the new instance attempts to write the BOUND state in the binding table, if it finds that the current state is MOVING and the MOVING marker time exceeds 100 ms, it will forcibly overwrite to avoid the old instance from being stuck in migration due to deadlock;

[0090] This process still needs to acquire a mutex to ensure consistency under concurrent competition.

[0091] 3. Backoff strategy for renewal failure:

[0092] If the new instance fails to write BOUND within 200 ms (e.g., Redis is unavailable, lock conflict, etc.), it will immediately:

[0093] Report renewal failure events to the console and scheduler, marking the Session as DEGRADED mode;

[0094] Trigger the "migration retry" process by Watchdog, mark the current new instance as INVALID, and release resources;

[0095] And preferentially reschedule to the cold standby instance within the node to minimize the delay caused by cross-machine migration.

[0096] 4. Measurement method and monitoring mechanism of P95 migration window:

[0097] The system records the T1=MOVING setting time and T2=BOUND write time for each Session migration, and calculates ΔT=T2-T1;

[0098] All migration events are entered into a time series database (such as Prometheus + Grafana), and P95, P99 migration time consumption indicators are counted. If 5 events exceeding 200 ms appear in continuous sampling, the system will automatically perform configuration rollback or degradation processing.

[0099] E) Log aggregation and debugging report: The log agent aggregates link logs and performance indicators collected by Token granularity based on the SessionID. If interface violations or abnormalities are detected, a debugging error report is generated, which includes at least SessionID, old / new SandboxID, migration timestamp, Token rate, GPU utilization, and error stack;

[0100] To effectively identify abnormal call behavior in high-concurrency scenarios, the present application designs a multi-dimensional "violation detection" judgment mechanism in the log agent, combining dynamic threshold modeling and associated feature analysis, which specifically includes:

[0101] 1. Dynamic threshold model based on Token rate (Token Rate Anomaly Detection):

[0102] The system uses a sliding window mechanism (default 3 seconds) to record the Token output rate (Token / s) of each Session;

[0103] An exponential weighted moving average (EWMA) is used to generate a dynamic baseline rate μ(t) for the historical window, and an adaptive threshold is set: Where α is the sensitivity parameter (default 2.5), and σ(t) is the standard deviation within the window;

[0104] If the current Token rate is higher than the threshold for two consecutive windows, it is marked as "rate anomaly", and further analysis is performed to determine whether there is a Prompt injection or model generalization failure risk in the context.

[0105] 2. Associated analysis of GPU utilization mutation and context anomaly:

[0106] Each log bucket records the real-time GPU utilization (%), video memory usage (MB), and video memory surge frequency;

[0107] The system calculates the cosine similarity between the GPU usage curve of the current Session and the normal baseline Session (same Prompt type). If it is less than a set threshold (such as 0.75), a joint alarm is triggered in combination with the abnormal Token rate;

[0108] Special handling for "mutant-low utilization" patterns (e.g. GPU utilization drops to <10% for a short period, but log volume continues to increase) as potential "dead loop generation" or "Prompt defect" risks.

[0109] 3. Content-index joint rule engine:

[0110] Upon all metric exceptions being triggered, enter the rule-based content analysis module:

[0111] Contains sensitive function call stack (such as eval, exec);

[0112] Contains abnormal keywords in session context (such as infinite loop, redundant nesting, etc.);

[0113] The number of SessionToken exceeds the set upper limit (such as 4,096);

[0114] Trigger any joint rule, generate "debug error report" and immediately push to the front-end console and CI blocking interface through WebSocket.

[0115] 4. False positive suppression mechanism:

[0116] Each rule item adopts a confidence scoring mechanism, and needs to reach a comprehensive score threshold (such as ≥0.7) to generate the final alarm;

[0117] In the training stage, unsupervised clustering (such as DBSCAN) is formed by collecting historical Session data to optimize the threshold setting, ensuring that the false positive rate is controlled below 1%.

[0118] F) Failure detection: Perform periodic polling on the binding table, and when the entry exceeds the preset TTL and the state is DROPPED, delete the entry to release resources.

[0119] The distributed binding table adopts Cluster implementation, key format is llm:sandbox: <sessionid>, the value contains {SandboxID, Status, Timestamp}.

[0120] The mutex is implemented by the Redis SETNX command; the heartbeat packet period is 50 milliseconds, and the old sandbox instance is determined to be invalid if 3 consecutive heartbeats are missing.

[0121] The measurement method of the migration window is: the time experienced from the state being set to MOVING to the new sandbox instance being set to BOUND, ensuring that P95 does not exceed 200 milliseconds.

[0122] The debug error report is returned to the continuous integration pipeline through the HTTP 409 status code or the Shell exit code 3, and is archived to the object storage service.

[0123] The log agent outputs a metric every 50 tokens when the LLM generates a response, including the average generation delay, the cumulative token number, and the current GPU memory usage.

[0124] When the state is MOVING, the downstream caller receives a 302 redirection or a response with a retry-after header to avoid 404 errors.

[0125] The polling period of the watchdog process is 60 seconds, and the TTL is 30 minutes.

[0126] The debug error report is pushed to the debug console in real time through WebSocket to support interactive debugging of the IDE.

[0127] The method of the application is suitable for containers, virtual machines or bare metal sandboxes, and supports orchestration platforms such as Kubernetes, DockerCompose or Nomad.

[0128] Embodiment 2

[0129] The application provides a concurrent debugging system based on sandbox-LLM session bidirectional binding, comprising:

[0130] a) a session ID allocation unit for generating a globally unique SessionID;

[0131] b) a bidirectional binding registration unit for writing SessionID and sandbox instance SandboxID into a Redis cluster and maintaining BOUND, MOVING and DROPPED states;

[0132] c) a link transparent transmission unit for injecting SessionID in environment variables or HTTP / gRPC headers;

[0133] d) a migration renewal unit for detecting the completion of the state change from MOVING to BOUND within a 200 ms window by heartbeats and mutual exclusion control at sandbox migration;

[0134] e) a log aggregation unit for aggregating logs by SessionID and collecting Token level metrics;

[0135] f) a debug query unit for generating debug error reports and outputting blocking instructions to continuous integration systems.

[0136] The Redis cluster enables multi-master CRDT replication to support cross-datacenter session migration.

[0137] The debug query unit provides a SessionID-based REST interface / debug / {sid} and a WebSocket-based real-time subscription interface.

[0138] The log aggregation unit writes debug data in Parquet format in batches to a data lake for offline analysis.

[0139] Those skilled in the art know that, in addition to implementing the system, device and each module thereof provided by the present application in a pure computer readable program code manner, the same program can also be implemented by logically programming the method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. Therefore, the system, device and each module thereof provided by the present application can be considered as a hardware component, and the modules included therein for implementing various programs can also be considered as structures within the hardware component; the modules for implementing various functions can also be considered as both software programs for implementing methods and structures within the hardware component.

[0140] The specific embodiments of the present application are described above. It needs to be understood that the present application is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. The embodiments of the present application and the features in the embodiments can be arbitrarily combined with each other without conflict.< / sessionid> < / sessionid> < / sessionid>

Claims

1. A concurrent debugging method based on sandbox-LLM session bidirectional binding, characterized in that, Comprising: Step 1: Assign a globally unique session identifier for each large language model LLM session; Step 2: Write the globally unique session identifier and sandbox instance identifier into a distributed in-memory database to form a bidirectional binding table, and mark the initial state as a binding state; Step 3: Realize the transmission of the globally unique session identifier through the environment variable and HTTP / gRPC header field in the whole link; Step 4: When the sandbox is hot migrated, the old instance atomically updates the binding table entry state to the migration state through the distributed mutex, and the new instance writes the binding state after completing the preheating within the preset time; Step 5: The log agent aggregates the indicators according to the sandbox instance identifier, collects the average generation delay, accumulates the data collection number, and collects the GPU memory occupation at a preset frequency; when a violation is detected, a debugging error report is generated, which includes the globally unique session identifier, the new and old sandbox instance identifier, the migration timestamp, the data collection rate, the GPU utilization rate, and the error stack; Step 6: Periodically poll the binding table, and execute cleaning for the entries that exceed the preset threshold of the distributed mutex and are in the recycling state.

2. The concurrent debugging method based on sandbox-LLM session bidirectional binding according to claim 1, characterized in that, The preheating of the new instance includes: loading the large language model LLM weight cache and session index identifier, synchronizing the text instruction and query history cache through the in-memory database stream, and performing empty reasoning operation on the GPU to avoid cold start delay.

3. The concurrent debugging method based on sandbox-LLM session bidirectional binding according to claim 1, characterized in that, The violation detection adopts a dynamic threshold model: Wherein, μ(t) is the baseline data collection rate generated by the exponential weighted moving average, σ(t) is the standard deviation, and α is the sensitivity parameter; when the data collection rate of two consecutive time windows exceeds the threshold, an abnormal mark is triggered.

4. The concurrent debugging method based on sandbox-LLM session bidirectional binding according to claim 3, characterized in that, Perform GPU utilization mutation analysis: calculate the cosine similarity between the current session GPU usage curve and the normal baseline of the same type of text instruction and query, when the similarity is lower than the preset threshold and accompanied by a data collection rate that does not meet the preset range, trigger a joint alarm.

5. The concurrent debugging method based on sandbox-LLM session bidirectional binding according to claim 1, characterized in that, The log agent includes: adopting a bucketing strategy to handle back pressure, bucketing according to the globally unique session identifier and sliding time window, suspending the log agent and returning the corresponding response when a single bucket accumulates more than a preset number of data collection events; through the response header, instruct the client to retry the request after a specified time, and temporarily store the bucketed data using a lightweight compression algorithm.

6. A concurrent debugging system based on sandbox-LLM session bidirectional binding, characterized in that, Comprising: Module M1: Assign a globally unique session identifier for each large language model LLM session; Module M2: Write the globally unique session identifier and sandbox instance identifier into a distributed in-memory database to form a bidirectional binding table, and mark the initial state as a binding state; Module M3: Realize the transmission of the globally unique session identifier through the environment variable and HTTP / gRPC header field in the whole link; Module M4: When the sandbox is hot migrated, the old instance atomically updates the binding table entry state to the migration state through the distributed mutex, and the new instance writes the binding state after completing the preheating within the preset time; Module M5: The log agent aggregates the indicators according to the sandbox instance identifier, collects the average generation delay, accumulates the data collection number, and collects the GPU memory occupation at a preset frequency; when a violation is detected, a debugging error report is generated, which includes the globally unique session identifier, the new and old sandbox instance identifier, the migration timestamp, the data collection rate, the GPU utilization rate, and the error stack; Module M6: Periodic polling of the binding table, performing cleanup on entries that exceed a preset threshold of the distributed mutex and are in the state of recycling.

7. The concurrent debugging system based on sandbox-LLM session bidirectional binding according to claim 6, characterized in that, The new instance preheating includes: loading large language model LLM weight cache and session index identifier, synchronizing text instruction and query history cache through in-memory database stream, and performing empty inference operation on GPU to avoid cold start delay.

8. The concurrent debugging system based on sandbox-LLM session bidirectional binding according to claim 6, characterized in that, The violation detection adopts a dynamic threshold model: Wherein, μ(t) is the baseline data collection rate generated by the exponential weighted moving average, σ(t) is the standard deviation, and a is the sensitivity parameter; When the data collection rate of the two consecutive time windows exceeds the threshold, an abnormal mark is triggered.

9. The concurrent debugging system based on sandbox-LLM session bidirectional binding according to claim 8, characterized in that, Perform GPU utilization mutation analysis: calculate the cosine similarity between the current session GPU usage curve and the normal baseline of the same type of text instruction and query, when the similarity is lower than the preset threshold and accompanied by a data collection rate that does not meet the preset range, trigger a joint alarm.

10. The concurrent debugging system based on sandbox-LLM session bidirectional binding of claim 6, wherein, The log agent includes: using a bucketing strategy to handle back pressure, bucketing by globally unique session identifier and sliding time window, pausing the log agent and returning the corresponding response when a single bucket accumulates more than a preset number of data collection events; Through the response header, instruct the client to retry the request after a specified time, and temporarily store the bucketed data using a lightweight compression algorithm.

Images