A Blind-State Feature Tree Speculative Verification Inference Method and System for Untrusted Computing Power Nodes

By generating candidate token trees locally and blinding them, combined with batch verification and blind key-value management of remote blind computing nodes, the privacy protection and network latency issues in remote inference of large models are solved, realizing a low-latency and efficient distributed inference process.

CN123117312APending Publication Date: 2026-11-17SICHUAN YUMU TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610884762.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-06-18
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies suffer from problems such as the risk of exposing plaintext semantics, high network round-trip latency, and chaotic state management in remote inference of large models. They are particularly difficult to effectively protect privacy and reduce computational overhead in distributed inference scenarios.

Method used

By generating a candidate token tree locally and blinding it, batch verification is performed using remote blind computing nodes. Combined with Blind KV branch management, the number of network round trips is reduced and the plaintext context is not seen by the remote nodes. The state is managed by a Blind KV branch commit and rollback mechanism.

Benefits of technology

It effectively reduces network latency in distributed inference, protects privacy, reduces computational overhead on remote nodes, and ensures auditability and recoverability of state management.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123117312A_ABST
    Figure CN123117312A_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for speculative verification inference using blind feature trees for untrusted computing nodes. The method includes: a local secure node performing word segmentation, embedding, and prefix layer computation of the target large model on the inference request, generating a candidate token tree or candidate feature tree based on the current hidden state; performing structure-preserving blinding, session layout perturbation, differential encoding, quantization compression, and authentication encryption on the hidden tensors corresponding to the candidate trees to form a blind feature tree frame; a remote blind computing node performing batch verification on multiple candidate branches based on the blinded intermediate layer model package and the blind KV Cache, and returning an acceptance path or a corrected blind tensor; and the local secure node performing suffix layer, language model header, and sampling, and submitting or rolling back the remote blind cache based on the acceptance or rejection result. This invention reduces the token-by-token network round-trip latency of distributed blind inference and reduces the exposure of plaintext semantics, real tokens, and the complete model to untrusted remote nodes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, large model inference, privacy computing, distributed heterogeneous computing power scheduling and computer security technology. Specifically, it relates to a method and system for reducing cross-network inference latency by utilizing local candidate generation and remote batch verification when performing blind inference of the intermediate layer of a large language model or a multimodal large model on a third-party or untrusted computing power node. Background Technology

[0002] Existing large-scale remote inference typically places the Prompt, Tokenizer, Embedding, complete model, and KV Cache at the remote end. Although network links can be protected through transmission encryption, the remote computation state may still expose plaintext semantics, the true token sequence, business context, and model assets. Solutions such as multi-party secure computation, homomorphic encryption, and trusted execution environments can improve security, but they often face computational overhead, hardware dependencies, or deployability issues in commercial inference scenarios. Traditional speculative decoding can generate candidate tokens using a small model, which are then verified by the target model. However, in local / remote distributed inference scenarios, token-by-total cross-network verification introduces significant network round-trip latency; furthermore, ordinary speculative decoding does not solve the problem of remote nodes seeing the plaintext context or the complete model. Summary of the Invention

[0003] This invention addresses at least the following technical problems: how to reduce the number of network round trips during the remote Decode stage without the plaintext Prompt and the real Token leaving the local trusted domain; how to enable remote batch verification of candidate trees but not to directly interpret candidate Tokens, candidate features, and the final output semantics; and how to synchronously manage the remote Blind KV state when the speculative branch accepts or rejects, avoiding large-scale memory erasure and state chaos.

[0004] This invention generates candidate token trees or candidate feature trees by a local security node, and performs prefix layer calculation, structure-preserving blinding, differential quantization, and authentication encryption locally. The remote blind computing node only receives blind feature tree frames and performs batch tree verification based on the remote blinding model package and Blind KV. The remote node returns the acceptance path, correction tensor, or verification digest, and the local security node performs suffix layer, LM Head, and sampling, and sends KV_COMMIT or KV_DROP control frames to the remote node according to the acceptance path.

[0005] The verification of multiple candidate tokens is merged into one or a small number of remote blind forward computations, reducing the RTT overhead of the distributed Decode phase; remote nodes do not hold real tokenizers, embeddings, LM heads and plaintext outputs, reducing the risk of remote recovery of business semantics; through the Blind KV branch commit and rollback mechanism, the state management of speculative branches is auditable, revocable and recoverable. Attached Figure Description

[0006] Figure 1 The structure diagram of the blind-state feature tree speculative verification inference system is shown. Figure 2 The flowchart shows the speculative verification method for blind-state feature trees. Figure 3 This diagram illustrates candidate tree validation and Blind KV submission / rollback. The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of the invention. Module numbers, step numbers, and arrow directions in the drawings are used to indicate the flow of data, control commands, or audit events. Implementation

[0007] The embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the described embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Where there is no conflict, the technical features of the various embodiments can be combined with each other; those skilled in the art can make equivalent substitutions or parameter adjustments to the model size, number of slices, network connections, quantization format, hardware type, and security level without departing from the concept of the present invention. Example 1: Candidate Token Tree Verification Based on a Local Small Draft Model

[0008] See Figure 1 and Figure 2 After receiving a user request, the local security node performs word segmentation, embedding, and the first K layers of the target model on the request text to obtain the current context hiding state.

[0009] The local security node calls a small draft model of size 0.3B to 0.8B to generate k candidate tokens and a multi-level candidate token tree; then, it performs a target model prefix layer mapping on each candidate token to form a candidate hidden state.

[0010] The local security node performs structure-preserving blinding, differential coding, quantization compression, and authentication encryption on the candidate hidden states to form a blind state feature tree frame, which is then sent to the remote blind computing node via RBP Speculative Stream.

[0011] The remote blind computing node performs tree-structured batch verification using a remote blinding intermediate layer and returns the acceptance path or the correction blind state tensor. The local secure node performs the suffix layer, LM Head, and sampling, and finally outputs a natural language token. Example 2: Feature-level candidate tree verification based on DraftHead

[0012] Based on Example 1, the local candidate generation module is replaced by DraftHead, which is bound to the target model. DraftHead takes the local hidden state of the target model as input and directly outputs candidate features, candidate tokens, candidate tree topology, and branch confidence.

[0013] DraftHead training is tied to the target model, slice location, hidden size, TransformSpec, and task domain. DraftHead can be retrained or calibrated when the model is upgraded or the slice strategy changes.

[0014] Since the candidate features are directly derived from the target model's hidden space, this embodiment can reduce the rejection rate caused by the inconsistency between the distribution of the small draft model and the target model, and increase the number of effective tokens for remote batch verification. Example 3: Dynamic Layout and Delta Frames in High-Security Sessions

[0015] See Figure 2 At the L3 security level, the local security node further derives session-level layout parameters based on the tenant key, model identifier, session identifier, and random number.

[0016] After the local security node maintains the blindness of the static structure, it superimposes session-level layout perturbation and calculates Delta frames between adjacent candidate nodes or adjacent time steps in the blind state space.

[0017] When the quantization error exceeds the threshold, a branch rollback occurs, a remote node switch occurs, or a session resynchronization occurs, the system sends a complete blind frame; otherwise, it sends an incremental blind frame to reduce bandwidth and suppress cross-session stable alignment. Example 4: Cache Commit and Rollback in Case of Speculation Failure

[0018] See Figure 3 The remote blind computing node maintains a Blind KV branch reference for each branch of the candidate tree.

[0019] When the local LM Head determines that a certain consecutive candidate prefix is ​​accepted, the local security node sends a KV_COMMIT control frame to the remote end, promoting the corresponding branch KV to the main path cache.

[0020] When a candidate branch is rejected, the local secure node sends a KV_DROP control frame to the remote end. The remote end only updates the reference count, branch visibility, and free list, and performs physical erasure as an asynchronous background operation.

Claims

1. A method for speculative verification reasoning using blind-state feature trees for untrusted computing power nodes, characterized in that, An application to a distributed large-model inference system including local secure nodes and remote blind computing nodes, wherein the local secure nodes are located within a trusted domain and the remote blind computing nodes are located within a non-fully trusted computing domain, the method includes: 1) The local secure node receives a user inference request and performs word segmentation, embedding layer computation, and local forward computation of the first few layers of the target large model locally to obtain a first hidden state tensor; 2) The local secure node generates a candidate feature tree containing multiple candidate time steps and / or multiple candidate branches based on the first hidden state tensor; 3) The local secure node performs a blind state transformation on the candidate feature tree to generate a blind state candidate feature tree, whose... The blind state transformation mentioned above includes at least one or more of the following: structure-preserving reversible transformation, session-level perturbation, differential coding, and quantization compression; 4) The local security node sends the blind state candidate feature tree to the remote blind computing node via an encrypted binary tensor transmission protocol; 5) The remote blind computing node loads the blinded intermediate layer model parameters, performs parallel branch verification on the blind state candidate feature tree, and generates verification results and corresponding blind state intermediate results; 6) The local security node receives the verification results and the blind state intermediate results, and performs boundary restoration, calculation of several layers after the target large model, calculation of the language model head, and sampling locally to obtain the output token or output content; 7) Based on the verification results, cache submission is performed on the accepted candidate branches, and cache revocation or rollback is performed on the unaccepted candidate branches.

2. The method according to claim 1, characterized in that, The candidate feature tree is generated by a local speculative small model or a feature-level prediction head; wherein, the local speculative small model is used to generate a sequence of candidate tokens, and the feature-level prediction head is used to generate candidate hidden features, candidate tokens, candidate tree structure and confidence based on the hidden state of the target large model.

3. The method according to claim 1, characterized in that, The blind state transformation includes deriving a session key based on the tenant identifier, model identifier, session identifier, and random number, and generating a session-level tensor layout, permutation parameters, padding parameters, or differential coding parameters based on the session key, so that the same plaintext context in different sessions corresponds to different blind state tensor representations.

4. The method according to claim 1, characterized in that, The transmission frame of the encrypted binary tensor transmission protocol includes at least a tenant identifier digest, session identifier, request identifier, stream identifier, sequence number, layer range, data type, quantization type, compression type, security level, tensor shape, and ciphertext payload; the ciphertext payload is obtained by encrypting the compressed blind-state tensor data using an authentication encryption algorithm.

5. The method according to claim 1, characterized in that, The remote blind computing node does not store the real word segmenter, the real embedding layer, the first few layers of the target large model, the last few layers of the target large model, the language model header, and the plaintext output mapping. It only performs intermediate layer inference based on the blinded intermediate layer model parameters, the blind candidate feature tree, and the blind key-value cache.

6. The method according to claim 1, characterized in that, The cache commit and cache revocation include: assigning a branch identifier, session round identifier, and blind cache block identifier to each candidate branch; sending a cache commit control frame when a candidate branch is accepted and sending a cache revocation control frame when a candidate branch is rejected; the remote blind computing node updates the reference state of the blind cache block based on the control frame without performing a synchronous full memory data rewrite.

7. The method according to claim 1, characterized in that, During the long context pre-filling stage, the local security node divides the first hidden state tensor or candidate feature tree into multiple tensor blocks, and performs simultaneous computation, blinding, compression, and transmission on each tensor block; the remote blind computation node performs simultaneous reception, decryption, reconstruction, and intermediate layer computation on the received tensor blocks.

8. The method according to claim 1, characterized in that, The local security node and / or the remote blind computing node generate audit records, which include at least the request identifier, session identifier, security level, candidate tree width, candidate tree depth, branch acceptance rate, tensor block count, cache commit event, cache revocation event, remote node identifier, and abnormal status.

9. A blind-state feature tree speculative verification inference system for untrusted computing power nodes, characterized in that, The system includes a local security node and a remote blind computing node; the local security node includes a request processing module, a local forward computation module, a speculative feature tree generation module, a blind state transformation module, a tensor transfer module, a local backward boundary computation module, and a sampling module; the remote blind computing node includes a blind model loading module, a feature tree verification module, a blind state key-value cache management module, and a verification result return module; the system is configured to perform the method described in any one of claims 1 to 8.

10. An electronic device or computer-readable storage medium, characterized in that, The electronic device includes a processor and a memory, wherein a computer program is stored in the memory; or the computer-readable storage medium stores a computer program; wherein the computer program, when executed by the processor, implements the method according to any one of claims 1 to 8.