A large language model ciphertext cache processing method, device and medium based on pseudo token ID

Abstract

This invention discloses a method for ciphertext caching of large language models based on pseudo-token IDs, comprising: obtaining ciphertext generated after encrypting the original text, wherein the ciphertext is an embedding vector adapted to the input of a large language model; performing a hash operation on the embedding vector using a preset hash algorithm to generate a pseudo-token ID with a fixed format; and storing the pseudo-token ID as a cache key for the key-value cache of the large language model, associated with the ciphertext embedding vector, in the cache to achieve ciphertext cache reuse. This invention builds an adaptation bridge between ciphertext and caching mechanisms through pseudo-token IDs, solving the technical pain points of ciphertext embedding vectors lacking a standard key structure and being unable to reuse caches. It effectively reduces redundant computation overhead, lowers hardware resource consumption, significantly improves the inference response speed of large language models in encrypted scenarios, and adapts to high-frequency interaction needs such as multi-turn dialogues and high-frequency queries.

Description

Technical Field

[0001] This invention relates to the field of natural language processing and data security, and in particular to a method, device and medium for ciphertext caching of large language models based on pseudo-token IDs. Background Technology

[0002] In the processing flow of existing large language models, input text must go through steps such as word segmentation, generating token IDs (identifiers), and converting into embedding vectors before it can enter the model inference stage. To improve inference efficiency, existing technologies generally adopt a key-value cache mechanism to cache the calculated keys and values, thereby avoiding redundant calculations and reducing resource consumption.

[0003] However, when the input text is ciphertext, its direct output is the embedding. This ciphertext embedding lacks a standard key structure corresponding to the token ID in the plaintext scenario, making it unsuitable for direct adaptation to the KV Cache mechanism. This means that the ciphertext data must be completely reprocessed for each model inference, making it impossible to accelerate the process using caching. This not only results in the ineffective consumption of computing resources but also significantly increases system response latency. This performance bottleneck is particularly pronounced in high-frequency interaction scenarios such as multi-turn dialogues and repeated queries, severely impacting the user experience.

[0004] Therefore, how to effectively adapt encrypted embeddings to the KV Cache mechanism to achieve cache reuse and inference acceleration has become a core technical challenge for improving the efficiency of encrypted text processing. Summary of the Invention

[0005] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows:

[0006] According to a first aspect of the present invention, a method for ciphertext caching of large language models based on pseudo-token IDs is provided, comprising the following steps:

[0007] S100, Obtain the ciphertext generated after encrypting the original text, wherein the ciphertext is an embedding vector used for input processing of a large language model.

[0008] S200, the embedded vector is hashed using a preset hash algorithm to generate a pseudo token ID with a fixed format.

[0009] S300, the pseudo token ID is used as the cache key of the key-value cache of the large language model, and the embedding vector of the ciphertext is associated with it and stored in the key-value cache, so as to realize the cache reuse of the ciphertext by the large language model.

[0010] According to a second aspect of the present invention, an electronic device is provided, including a processor and a memory; the processor executes the steps of the method described in the first aspect of the present invention by invoking a program or instructions stored in the memory.

[0011] According to a third aspect of the present invention, a computer-readable storage medium is provided that stores a program or instructions that cause a computer to perform the steps of the method described in the first aspect of the present invention.

[0012] The present invention has at least the following beneficial effects:

[0013] 1. Overcoming the bottleneck of ciphertext caching adaptation: Addressing the core issues of existing technologies where ciphertext embedding vectors lack a standard key structure corresponding to plaintext token IDs and cannot adapt to KV Cache, this paper establishes an adaptation channel between ciphertext and caching mechanisms through the standardized format design of pseudo token IDs. This enables the first-ever caching reuse of ciphertext data by a large language model, filling the gap in caching acceleration technology in encrypted scenarios.

[0014] 2. Significantly improves inference efficiency: The pseudo token ID and the ciphertext embedding vector form a unique mapping (or similar association). Subsequent requests with the same / similar ciphertext can directly hit the cache and call the associated vector to participate in the calculation, without having to repeatedly perform encryption, embedding conversion and vector operation, which greatly reduces the overhead of invalid calculation and significantly shortens the response latency of a single request, especially suitable for high-frequency repetitive interaction scenarios.

[0015] 3. Optimize resource utilization efficiency: Avoid hardware resource waste caused by repeated processing of encrypted text, reduce CPU / GPU computing power consumption and memory usage, effectively improve hardware resource utilization, support more encrypted text processing requests under the same hardware configuration, and enhance the system's concurrent processing capabilities.

[0016] 4. Balancing Data Security and Adaptability: Processing is based entirely on encrypted embedding vectors, ensuring no leakage of original text information and guaranteeing data transmission and interaction security. Simultaneously, the pseudo-token ID format adapts to the KV cache key structure requirements of mainstream large language models, requiring no modification to the underlying model architecture, resulting in strong compatibility and low implementation costs. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The flowchart illustrates a method for ciphertext caching of a large language model based on a pseudo-token ID, provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0021] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0022] This invention provides a method for ciphertext caching of large language models based on pseudo-token IDs. It is suitable for application scenarios where data intermediaries interact with large language models via ciphertext, and is particularly well-suited for encrypted text interaction requirements that demand secure data transmission / storage while prioritizing efficient model inference response. Figure 1 As shown, the method specifically includes the following steps: S100, obtaining the ciphertext generated after encrypting the original text, wherein the ciphertext is an embedding vector used for input processing of a large language model, specifically:

[0023] S101, the data intermediary first obtains the original text to be input into the large language model.

[0024] The original text includes various types of text data that need to be processed by the large language model, such as natural language interactive text, business data text, and search query text. The large language model is an artificial intelligence model built on the Transformer architecture and pre-trained on large-scale text data to complete natural language understanding and generation tasks. Its core achieves functions such as question answering, retrieval, and content generation through semantic modeling of text, and its underlying training logic does not need to be elaborated further.

[0025] S102, the original text is encrypted; after encryption, ciphertext is directly generated, and the carrier form of the ciphertext is an embedding vector.

[0026] The specific principle of encryption processing can be a technology known in the field. The encryption method can adopt well-known symmetric / asymmetric encryption algorithms such as AES and RSA, or well-known encryption schemes such as transmission encryption under the HTTPS protocol. The core purpose is to ensure the data privacy and security of the original text during transmission, storage and interaction with the large language model, and to avoid leakage of the original text.

[0027] The embedding vectors are generated by a pre-set text embedding model (such as BERT, GPT series pre-trained models or dedicated embedding generation models), and have dimensional features that are adapted to the input requirements of large language models (such as 768-dimensional, 1024-dimensional, 2048-dimensional, etc.). They can be directly used as input data for large language models without additional embedding transformation processing.

[0028] S200, the embedded vector is hashed using a preset hash algorithm to generate a pseudo token ID with a fixed format.

[0029] The preset hash algorithm is selected from well-known irreversible hash algorithms or locality-sensitive hash algorithms. Irreversible hash algorithms include well-known algorithms such as MD5 and SHA256. Their core characteristic is a one-to-one mapping between input data and hash results; different inputs are almost impossible to generate the same hash value (collision resistance), and the original data cannot be deduced from the hash result. This type of algorithm is suitable for exact matching tasks, such as ciphertext deduplication and data integrity verification. It only requires complete consistency of the pseudo-token ID to achieve cache hits, without needing to retain the similarity relationship between the original vectors. Locality-sensitive hash algorithms (LSH) include well-known algorithms such as SimHash, E2LSH, BitSampling, SpectralHashing, and MinHash. Their core characteristic is that similar input vectors will generate similar hash results (small Hamming distance), preserving the similarity relationship between the original vectors. This type of algorithm is suitable for similarity-related tasks, such as semantic retrieval, similar ciphertext matching, and classification clustering. It can achieve cache reuse through pseudo-token ID similarity judgment, avoiding duplicate calculations caused by minor differences in ciphertext.

[0030] As is known to those skilled in the art, the core principle of hash operation is to map, compress, and encode the numerical features of each dimension of the embedded vector. The specific operation logic is well known in the art and does not need to be elaborated further.

[0031] The fixed format of the pseudo token ID is a standardized identifier of a preset length (such as a 32-bit / 64-bit hexadecimal string, a 256-bit binary string, or a Base64 encoded string). Its format is fully compatible with the key structure requirements of the KVCache (key-value cache) of the large language model, and can be recognized and stored by the caching mechanism. At the same time, the pseudo token ID has a unique mapping relationship with the input ciphertext embedding vector (in irreversible hash algorithm scenarios) or an association relationship between similar vectors and similar IDs (in locality sensitive hash algorithm scenarios). This ensures the accuracy of cache association and can also adapt to the subsequent cache reuse and semantic matching requirements.

[0032] S300, the pseudo token ID is used as the cache key of the key-value cache of the large language model, and the embedding vector of the ciphertext is associated with it and stored in the key-value cache, so as to realize the cache reuse of the ciphertext by the large language model.

[0033] The pseudo token ID generated by S200 is used as the unique cache key of the large language model KVCache (key-value cache). A one-to-one key-value pair relationship is established with the corresponding ciphertext embedding vector. Then, the key-value pair is stored in the KVCache storage space of the large language model. Cache management can adopt cache eviction strategies known in the field, such as LRU and FIFO, to balance cache capacity and data validity.

[0034] The KVCache is a core caching module used to store computed vector data during the inference phase of the large language model. Its preset key structure is fully compatible with the fixed format of the pseudo-token ID, allowing direct identification and indexing of the key-value pairs corresponding to the pseudo-token ID. When a subsequent data intermediary initiates the same or similar encrypted interaction request, the large language model first performs the same hash operation on the newly input encrypted embedding vector to generate the corresponding target pseudo-token ID, and then checks whether a matching cache key exists in the KVCache. If the query is successful (i.e., the same pseudo-token ID exists), the encrypted embedding vector associated with that cache key is directly called to participate in the inference calculation, without the need for repeated encryption, embedding transformation, and vector operations on the encrypted text. If the query fails, the complete encrypted processing and cache storage process is executed.

[0035] This approach enables efficient caching and reuse of ciphertext by large language models, effectively reducing redundant computation overhead, lowering hardware resource consumption, and significantly improving the inference response speed of large language models in encrypted scenarios. It is particularly suitable for scenarios requiring continuous ciphertext interaction, such as multi-turn dialogues and high-frequency repeated queries.

[0036] The following section will describe in detail the specific selection process, core basis, and adaptation logic of the preset hash algorithm, taking into account the characteristics of the ciphertext embedding vector, actual task requirements, and technical constraints, to ensure the scientific and practical nature of the algorithm selection.

[0037] In this invention, to ensure that the algorithm selection accurately matches the actual needs, a four-dimensional evaluation + constraint optimization screening mechanism is adopted. Specifically, the preset hash algorithm is determined through a multi-dimensional evaluation system. The multi-dimensional evaluation system includes:

[0038] 1. Accuracy Dimension: This dimension primarily measures the ability of a hash algorithm to preserve the original similarity relationship between ciphertext embedding vectors, avoiding the association deviation between ciphertext and pseudo token ID caused by hash mapping, which could lead to mismatch problems in the key-value cache of large language models.

[0039] The evaluation metrics for this dimension are as follows: First, determine the similarity calculation type (one or more of Euclidean distance similarity, cosine similarity, and Jaccard similarity) of the original ciphertext embedding vectors based on actual business needs. Then, specifically calculate the similarity difference of the same set of ciphertext embedding vectors before and after the hash operation, thereby quantifying the similarity loss caused by hash mapping. The quantification formula is:

[0040] A = 1 - (1 / T)∑ （vi，vj）∈T |simV(vi, vj)−simH(H(vi), H(vj))|.

[0041] Where A is the evaluation value of the hash algorithm in the accuracy dimension, i.e., the similarity preservation accuracy, with a value range of [0,1]. T is the ciphertext embedding vector test set, which consists of several vector pairs {(vi, vj) | i ≠ j} selected from ciphertext embedding vector samples in actual application scenarios. The test set needs to cover vector pairs with different similarity levels of high, medium, and low to ensure the objectivity and comprehensiveness of the evaluation results. vi and vj are any two different ciphertext embedding vectors in the test set T, both of which are vector data that can be directly input into the large language model. simV(vi, vj) is the similarity calculation result of ciphertext embedding vectors vi and vj in the original vector space, and its calculation method is consistent with the preset original similarity type. H() is the mapping function of the hash algorithm to be evaluated. The input is the ciphertext embedding vector, and the output is the hash value of the vector after hash operation. simH(H(vi), H(vj)) represents the similarity calculation results corresponding to the hash values ​​H(vi) and H(vj) of the ciphertext embedding vectors vi and vj after hash mapping in the hash space. |simV(vi, vj)−simH(H(vi), H(vj))| represents the absolute similarity error of a single vector pair before and after the hash operation, representing the degree of similarity loss of that pair of vectors.

[0042] The closer the A value is to 1, the stronger the hash algorithm's ability to maintain the original similarity relationship between ciphertext embedding vectors, and the smaller the similarity loss caused by hash mapping. When generating pseudo token IDs based on this algorithm, the association accuracy between ciphertext and pseudo token IDs is higher, which can effectively reduce the probability of mismatch in key-value caching of large language models. Conversely, the closer the A value is to 0, the worse the similarity maintenance effect of the hash algorithm is, and the higher the risk of cache mismatch.

[0043] 2. Computational complexity dimension:

[0044] This dimension is designed to meet the real-time inference requirements of large language models, avoiding the additional performance overhead introduced by hash operations and ensuring that the inference response efficiency of large language models in encrypted scenarios meets business expectations.

[0045] The evaluation metrics for this dimension include time complexity (reflecting the time cost of hash operations), space complexity (reflecting the memory cost of hash operations), and energy consumption estimation (resource consumption metrics derived from time and space complexity). These three metrics together quantify the overall computational cost of the hash algorithm. Specific values ​​for time and space complexity can be found in the computational complexity models of common hash algorithms, as shown in the table below.

[0046]

[0047] Where n is the number of ciphertext embedding vectors to be processed, d is the dimension of the ciphertext embedding vector, k is the number of hash bits, L is the number of hash tables, and |S| is the size of the set corresponding to the ciphertext embedding vector.

[0048] The quantification formula for this dimension is: C=wt·T(n,d)+wm·M(n,d)+we·E(n,d).

[0049] Where C is the evaluation value of the hash algorithm in terms of computational complexity, representing the overall computational cost of the algorithm; wt, wm, and we are the weight coefficients for time complexity, space complexity, and energy consumption estimation, satisfying wt+wm+we=1; the weights can be dynamically adjusted according to the actual business scenario. For example, in real-time high-concurrency scenarios, priority is given to ensuring response speed, so wt=0.6, wm=0.3, and we=0.1 can be set; in memory-constrained scenarios, wt=0.5, wm=0.4, and we=0.1 can be set. T(n, d) is the time complexity for processing n d-dimensional ciphertext embedding vectors. Its specific form is shown in the table above. For example, SimHash's T(n, d)=O(n·d·k), which means that its time consumption increases linearly with the increase of the number of vectors, vector dimension, and hash bit length. M(n, d) represents the space complexity of processing n d-dimensional ciphertext embedding vectors. Its specific form is shown in the table above. For example, for E2LSH, M(n, d) = O(L·k·d), meaning its memory usage increases linearly with the number of hash tables, hash bit length, and vector dimension. E(n, d) is the energy consumption estimate for hash operations, usually derived from a time complexity and hardware power consumption model. For example, E(n, d) = P·t, where P is the hardware power consumption per unit time, taken as a standard value for general-purpose servers (e.g., 300W). t is the actual time consumed by the hash operation, which can be obtained by converting time complexity to hardware computing power, for example: t = time complexity coefficient × vector size ÷ hardware computing power.

[0050] Based on the resource constraints of the actual scenario (such as the inference response time of a large language model needing to be <100ms and the cache memory usage needing to be <2GB), the time complexity is first converted into actual time consumption, and the space complexity is converted into actual memory usage. Then, the C value is calculated by substituting these values ​​into the quantization formula. Finally, those with a C value lower than a preset threshold (e.g., C0) are selected. threshold =0.8 (the threshold is calculated from the comprehensive overhead limit corresponding to the resource constraint) hash algorithm to ensure that the computational overhead of the algorithm will not affect the real-time inference efficiency of the large language model.

[0051] 3. User Purpose Dimension: This dimension is guided by the actual business task type. Through quantitative encoding and vector matching, it evaluates the suitability of the hash algorithm with the task requirements to ensure that the algorithm selection is highly consistent with the business objectives.

[0052] (1) Classification and coding of user purposes

[0053] First, the user's business tasks are divided into four categories, and one-hot vectors are used for standardized encoding (encoding rules are as follows):

[0054] P1=[1,0,0,0]: Complete matching task. The core requirement is that "a match can be determined only when the ciphertext embedding vectors to be matched are completely identical". Typical scenarios include ciphertext deduplication (determining whether two ciphertexts are completely identical to avoid duplicate storage) and data integrity verification (verifying whether the current ciphertext is completely identical to the original encrypted ciphertext to confirm that the data has not been tampered with).

[0055] P2=[0,1,0,0]: Similarity search tasks, including semantic retrieval, multi-turn dialogue association matching, and other scenarios;

[0056] P3=[0,0,1,0]: Classification tasks, including scenarios such as encrypted topic classification and business type labeling;

[0057] P4=[0,0,0,1]: Clustering tasks, including clustering of similar encrypted data, grouping of user intent, and other scenarios.

[0058] (2) Adaptability vector of hash algorithm

[0059] For each candidate hash algorithm Hr (where r ranges from 1 to m, and m is the number of candidate hash algorithms), its adaptability to the above four types of tasks is evaluated through experiments, resulting in the corresponding adaptability vector Padapt(Hr), with the structure: Padapt(Hr) = [p1, p2, p3, p4]. T .

[0060] Where ps∈[0,1] (s=1,2,3,4) represents the algorithm’s fit score for the s-th type of task (the closer the score is to 1, the stronger the fit). The score is quantified by the algorithm’s performance on the corresponding task in the experiment (such as exact matching accuracy and similarity search recall).

[0061] (3) Quantitative calculation of matching degree

[0062] In one embodiment, the matching degree evaluation value Φ between the candidate algorithm and the target task is calculated by combining vector dot product with the Kronecker delta function. P :

[0063] If the one-hot vector corresponding to the target task is P req (e.g., a perfectly matching task corresponds to P) req =P1);

[0064] The formula for calculating the matching degree is: Φ P =P req ×Padapt(Hr)=∑ s=1 4 P req[s]·Padapt(Hr)[s].

[0065] Because of P req It is a one-hot vector. This formula is actually equivalent to extracting the dimension score of Padapt(Hr) that corresponds to the target task. That is, only the algorithm's fit score to the target task is retained, avoiding interference from other task dimensions.

[0066] In another embodiment, to further improve computational efficiency and feasibility, the present invention also provides an equivalent quantization implementation based on the Kronecker delta function:

[0067] Φ P =δ(Padapt(Hr), P req )×b1+[1-δ(Padapt(Hr), P req )]×b2.

[0068] Where δ() is the Kronecker delta function, and its perfect match criterion is: in the candidate algorithm adaptability vector Padapt(Hr), the match between the target task requirement vector P and the target task requirement vector P. req If the adaptation score corresponding to a non-zero dimension meets a preset threshold (e.g., ≥0.7), it means that the core adaptation dimension of the algorithm is consistent with the business task requirement dimension. In this case, the function takes the value of 1; otherwise, it takes the value of 0. b1 and b2 are preset scores (satisfying b1>b2 and b1+3b2=1 to achieve normalization).

[0069] This function determines whether the algorithm's adaptation vector and the demand vector are a perfect match and outputs a matching degree value in the interval [0,1]. A perfect match results in a high score b1, while a partial match results in a low score b2. This is completely equivalent to the calculation result of the vector dot product, and it does not require summation, making it more computationally efficient.

[0070] (4) The directional adaptation relationship between tasks and algorithms

[0071] Based on the matching metric results (regardless of whether vector dot product or Kronecker delta function is used for calculation), the corresponding adaptive hash algorithms for different task types are as follows:

[0072] Completely consistent matching tasks (P) req =P1): Adaptable to irreversible hash algorithms (such as MD5, SHA256). For this type of algorithm, Padapt(Hr)≈1, and the scores of other dimensions are close to 0, with a matching degree Φ. P ≈1;

[0073] Similarity search tasks (P) req=P2): Adapted to locality-sensitive hashing algorithms (such as SimHash, E2LSH). For this type of algorithm, p2 ≥ 0.8 in Padapt(Hr), and the matching degree Φ P A score of ≥0.8 can efficiently recall similar ciphertext;

[0074] Classification tasks (Preq=P3): Adapting locality-sensitive hashing algorithms (such as Spectral Hashing), where p3 ≥ 0.75 in Padapt(Hr) and the matching degree Φ. P A value of ≥0.75 can support the category labeling of encrypted text;

[0075] Clustering tasks (P) req =P4): Suitable for locality-sensitive hashing algorithms (such as BitSampling). For this type of algorithm, p4 ≥ 0.75 in Padapt(Hr), and the matching degree Φ P A value ≥0.75 enables grouping and clustering of similar ciphertexts.

[0076] A higher matching degree indicates that the hash algorithm is more adaptable to the target business task and can better meet the core requirements of the task (such as the accuracy of exact matching and the recall rate of similarity search).

[0077] 4. Embedding Dimension: This dimension is the core evaluation tool for assessing the efficiency of hash algorithms in adapting to ciphertext embedding vectors of different dimensions. It avoids a decrease in the accuracy of hash operations or a surge in computational load due to a mismatch between the vector dimension and the algorithm's adaptation range.

[0078] (1) Piecewise adaptation function of embedding dimension

[0079] First, based on the dimension d of the current ciphertext embedding vector (such as 768-dimensional, 1024-dimensional, or 2048-dimensional vectors commonly used in large language models), a piecewise adaptation function D is used. class (d) Determine the standardized interval to which it belongs.

[0080] For example, if d < 100, then D class (d) = D1; if 100 ≤ d < 500, then D class (d) = D²; if 500 ≤ d < 2000, then D class (d) = D3; if d ≥ 2000, then D class (d)=D4.

[0081] Among them, D1 to D4 correspond to the four dimensional intervals of low dimension, low-to-medium dimension, medium-to-high dimension, and high dimension, respectively.

[0082] (2) Dimensionality efficiency function of hash algorithm

[0083] For a candidate hash algorithm H, based on its type (dimensionality-sensitive / linearly dependent / insensitive / dimensionally projective), substitute the current dimension d to calculate the corresponding dimensional efficiency value eff. H (d), this value is the matching evaluation value Φ of the embedding dimension. D . eff H The range of (d) is [0,1]. The closer the value is to 1, the stronger the algorithm's adaptability to the vector of that dimension. The efficiency function forms of different types of algorithms are as follows:

[0084] Dimension-sensitive algorithms (such as E2LSH and BitSampling): The efficiency function is exponentially decaying.

[0085] Φ D =eff H (d) = α × exp (-β × d); where α and β are experimental fitting parameters, and β > 0 represents the rate of efficiency decay as the dimension increases;

[0086] Dimensionally linearly dependent algorithms (such as MinHash): The efficiency function is of linear form: Φ D =eff H (d)=γ×(1-d / d max ); where γ is the baseline efficiency coefficient, d max The maximum dimension that this algorithm can adapt to;

[0087] Dimension-insensitive algorithms (such as SimHash): The efficiency function is a constant: Φ D =eff H (d) = δ; where δ is the experimentally measured steady-state efficiency value;

[0088] Dimensional projection algorithms (such as PCA Hashing): The efficiency function is in piecewise linear form: Φ D =eff H (d)=u×(1-(dd opt ) / (d max -d opt Where u is the projection efficiency coefficient, d opt For the optimal adaptation dimension of the algorithm (usually 1000~1500), d max The maximum dimension that the algorithm is adapted to (usually ≥3000) represents the algorithm's efficiency, which is close to 1 within the optimal dimension range and decreases linearly as the dimension deviates from the optimal value.

[0089] To further simplify the calculation process of adaptation efficiency and reduce the complexity of engineering implementation, this invention also provides a method for using the continuous efficiency function eff. H (d) Logically equivalent interval matching quantization implementation, as follows:

[0090] .

[0091] Among them, D req D is the dimensional range of the ciphertext embedding vector in the business scenario. class (d) Output D1-D4; F1 is the optimal dimension fit interval of the candidate algorithm (and the piecewise fitness function D). class (d) The correspondence between D1-D4 is determined by experiments), F2 is the extended compatibility interval of the candidate algorithm (the dimensional range adjacent to the optimal interval, the algorithm adaptation efficiency is slightly reduced but still meets the basic requirements).

[0092] The core logic of this discrete function and its continuous efficiency value eff H (d) Completely identical: F1 corresponds to eff H (d) ≥ 0.85 dimensional range (optimal fit), F2 corresponds to 0.5 ≤ eff H (d) Dimension range < 0.85 (compatibility adaptation), other cases correspond to eff H (d) < 0.5 (invalid fit), only the continuous values ​​are simplified to discrete scores, which improves computational efficiency without losing the accuracy of the fit.

[0093] (3) Adaptation logic of dimensions and algorithms

[0094] Combining the piecewise adaptive function D class (d) and the dimensional efficiency function eff H (d) The appropriate hash algorithms for different dimensional intervals and their basis are as follows:

[0095] Low-dimensional interval (D) class (d)=D1, d<100): Adaptable to algorithms such as E2LSH and BitSampling; Basis: These algorithms are mostly dimension-sensitive, and the efficiency is low in the low-dimensional interval. H (d)≈1, such as E2LSH with a small β, the exponential decay is not obvious in low dimensions, and it has strong adaptability and low computational cost.

[0096] Low-to-medium dimensional interval (D) class (d) = D², 100 ≤ d < 500): Adaptable to algorithms such as MinHash; Basis: This type of algorithm is linearly dependent on dimension, and d < d² within this interval. max , usually d max ≥500, eff H (d)≥0.85, which can balance computational efficiency and accuracy.

[0097] Mid-to-high dimensional intervals (D) class (d)=D3, 500≤d<2000): Adaptable to algorithms such as SimHash; Basis: This type of algorithm is dimension-insensitive, and within this interval, eff H(d)≈0.9, achieving the optimal balance between accuracy and computational cost. PCAHashing is a dimensional projection type, and this range covers its optimal fit dimension (1000~1500), eff H (d)≥0.88, the computational cost of high-dimensional vectors can be further reduced through dimensional projection, and its adaptability is better than that of traditional algorithms.

[0098] High-dimensional interval (D) class (d)=D4, d≥2000): Adapted to SpectralHashing, PCAHashing, or adapted to other algorithms after dimensionality reduction preprocessing; Basis: SpectralHashing is a data-driven projection algorithm, and its efficiency in high-dimensional intervals is... H (d) The decay rate is slow, approaching the efficiency characteristics of linearly dependent algorithms. max ≥4096, excellent high-dimensional compression effect; PCAHashing, as a dimensional projection algorithm, has an efficiency of ≥4096 in high-dimensional intervals. H (d)≥0.75, it can be directly adapted to high-dimensional vectors without additional PCA dimensionality reduction, resulting in lower implementation costs; if other algorithms are used, high-dimensional vectors need to be compressed to the medium-high dimension range through PCA dimensionality reduction before adapting to the corresponding algorithm.

[0099] This adaptation logic ensures that the dimensions of the hash algorithm and the ciphertext embedding vector are precisely matched, avoiding performance loss caused by dimension incompatibility.

[0100] Furthermore, the process of selecting the preset hash algorithm through a multi-dimensional evaluation system is specifically transformed into a constrained multi-objective optimization problem, realizing the quantification and precision of algorithm selection. The steps are as follows:

[0101] S210, based on the matching degree evaluation values ​​of each dimension of the multidimensional evaluation system, calculate the comprehensive matching degree evaluation value of each candidate hash algorithm according to the preset dynamic weight.

[0102] In this invention, a matching vector F(Hr, R) is defined corresponding to the candidate hash algorithm Hr and the target scenario requirement R, and its form is:

[0103] F(Hr, R) = [A, C, Φ] P , Φ D , ε] T .

[0104] Where, Φ P The degree of matching for the user's purpose dimension, Φ D For the adaptation efficiency of the embedding dimension, ε is a quantitative indicator corresponding to other engineering constraints (such as algorithm deployment compatibility).

[0105] In this invention, based on the core requirements of the target scenario, the dynamic weight ratio of the four-dimensional evaluation is adjusted to construct a comprehensive matching degree evaluation function S(Hr, L):

[0106] S(Hr,L)=w A ×Φ A +w C ×Φ C +w P ×Φ P +w D ×Φ D .

[0107] Where, Φ A Φ C Φ P and Φ D These represent the matching scores for each dimension, with a value range of [0, 1]. Wherein, Φ A =1-|AA req | / max(A, Areq). Where A req This represents the accuracy threshold required for the specific business scenario. Φ C =exp(-h·|CC req | / C req ), C req This represents the maximum allowable computational cost threshold for the business scenario. h is a sensitivity adjustment coefficient; h > 0, and the larger the value, the faster the matching evaluation value decreases when the computational cost deviates from the threshold.

[0108] w A w C w P and w D These are the weights for each dimension, summing to 1. A typical weight adjustment rule is as follows:

[0109] Real-time high-concurrency scenarios (such as high-frequency encrypted queries): computational complexity, dimension weight w C =0.6, prioritizing response speed;

[0110] High-precision retrieval scenarios (such as precise semantic matching): accuracy dimension weight w A =0.55, prioritizing the ability to maintain similarity.

[0111] S220, Based on preset constraints, select the algorithm with the best comprehensive matching degree evaluation value and that satisfies all constraints from the candidate hash algorithm set as the preset hash algorithm.

[0112] In this invention, a set of constraint conditions {g} is defined based on the matching vector F(Hr, R). k} k=1到5 To ensure that candidate algorithms meet the resource and performance requirements of the scenario:

[0113] g1: A > A threshold Accuracy constraints. A threshold This is the accuracy threshold, representing the lowest acceptable evaluation value for the hash algorithm in terms of accuracy. It signifies the algorithm's minimum ability to preserve the similarity relationship between ciphertext embedding vectors, and is used to avoid the risk of cache mismatches caused by hash mapping. The value is determined by the business scenario's tolerance for cache mismatch rates. For example, if the business allows a cache mismatch rate ≤15%, then A needs to be set. threshold =0.85, to ensure that the similarity accuracy remains ≥85%.

[0114] g2:C≤C threshold Computational constraints. C threshold This represents the computational constraint, specifically the highest acceptable evaluation value for the hash algorithm in terms of computational complexity. It signifies the upper limit of the algorithm's overall computational overhead, used to ensure the real-time inference efficiency of large language models. The value is derived from the resource constraints of the scenario. For example, if the hash operation response time for a single ciphertext is required to be <100ms, the time constraint can be converted into a quantized threshold C for computational complexity, taking into account hardware computing power. threshold If memory usage is required to be less than 2GB, the space constraint can be converted into a computational threshold.

[0115] g3:Φ P >P threshold User purpose matching constraint. threshold This is the user-defined matching threshold, representing the minimum fit between the hash algorithm and the target business task. It ensures the algorithm meets the core requirements of the task (such as exact matching or similarity search). The value is determined by the minimum fit requirement of the task type. For example, if the business requirement is an algorithm with a fit ≥70% for similarity search tasks, then P is set to... threshold =0.7; for core business scenarios, it can be increased to P. threshold =0.8.

[0116] g4: Φ D >D threshold Dimensional adaptation constraints. D threshold This is the dimensionality adaptation threshold, representing the minimum adaptation efficiency of the hash algorithm for the target ciphertext embedding vector dimension. It is used to avoid performance degradation due to dimensionality mismatch. The value is determined by the adaptation requirements of the vector dimension range. For example, if the algorithm is required to completely adapt to the target dimension range, set D... threshold =1.0; If partial adaptation is allowed, D can be set to 1.0. threshold =0.8.

[0117] g5: ε∈E, other engineering constraints, such as hardware resource adaptability, where E is the constraint interval. To maximize the comprehensive matching degree evaluation function S(Hr, L) while satisfying the constraints, the Lagrange multiplier method is used to construct the Lagrange function:

[0118] L(Hr, λ) = S(Hr, L) + ∑ 5 k=1 λ k ·(g k -threshold k ).

[0119] Among them, λ=[λ1, λ2, λ3, λ4, λ5] T , where is a Lagrange multiplier vector, threshold k The threshold for the corresponding constraint (e.g., threshold1=Athreshold).

[0120] By iteratively optimizing the Lagrange multiplier λ, the constraint condition {g k All conditions are met, and L(Hr, λ) is maximized. Finally, from the set of candidate hash algorithms, the algorithm that maximizes L(Hr, λ) is selected as the preset hash algorithm. The steps of iteratively optimizing the Lagrange multipliers belong to the well-known constrained optimization solution process in this field. The typical iterative steps can be briefly summarized as follows:

[0121] Initialization: Set the initial value of the Lagrange multiplier λ (usually a non-negative initial value, such as λ0=[0,0,0,0,0]), and set the convergence threshold and the maximum number of iterations. The maximum number of iterations is determined according to the real-time requirements of the scenario, such as 100~500 times. The convergence threshold is determined based on the accuracy requirements of the ciphertext vector similarity calculation, for example, 1e-6.

[0122] Gradient and optimality judgment: based on the multiplier λ of the current iteration t Calculate the gradient information of the Lagrangian function L(Hr,λ), and combine it with the KKT conditions (complementary relaxation conditions and feasibility conditions) to determine whether the current solution satisfies optimality.

[0123] Multiplier Update: Based on gradient information and the degree of constraint violation, the multiplier is updated using known methods such as gradient ascent and sequential quadratic programming to obtain λ. t+1 For example, for dimensions that do not meet the constraints, the corresponding multiplier is increased to strengthen the constraint penalty.

[0124] Convergence criterion: Compare the changes in multipliers or Lagrange function values ​​between two consecutive iterations. If the changes are less than the convergence threshold or the maximum number of iterations has been reached, terminate the iteration; otherwise, return to the gradient calculation step and continue iterating.

[0125] Output: After the iteration terminates, the candidate hash algorithm that satisfies all constraints and maximizes the overall matching degree is output as the preset hash algorithm.

[0126] To further improve the adaptability of the selected preset hash algorithm to the ciphertext embedding vector features and business task constraints, the core parameters of the algorithm need to be automatically optimized.

[0127] When the preset hash algorithm is Locality Sensitive Hash (LSH), automatic optimization is performed based on key indicators such as target similarity error, false alarm rate, and precision error threshold. The specific logic is as follows: 1. SimHash parameter optimization

[0128] Optimization target: hash bit length k.

[0129] Optimization formula: Optimal hash bit length k opt =rounddown(log2(1 / σ)×d / p), where rounddown() represents rounding down, σ is the allowed similarity error with a value range of [0.05, 0.2], representing the maximum deviation between the hashed vector similarity and the original similarity. d is the dimension of the ciphertext embedding vector, and p is the resource adjustment coefficient with a value range of [4, 16], used to balance accuracy and storage overhead. The larger p is, the smaller the hash bit length and the lower the storage overhead. Example values: when σ=0.15, d=768, p=8, k opt =rounddown(log2(6.67)×96)≈rounddown(2.73×96)=262. In practice, we take 2 as a power of 2, which is 256 bits. This balances accuracy and storage overhead while ensuring that the similarity error is ≤15%.

[0130] 2. E2LSH parameter tuning

[0131] Optimization targets: bucket width w, number of hash functions q, number of hash tables L.

[0132] Optimization formula:

[0133] Bucket width: Optimal bucket width w opt =μ×σ d ×d -1 / 2 , where σ d denoted as d, where d is the standard deviation of the distance between the ciphertext embedding vectors and μ is an adjustment coefficient, taking values ​​from 1 to 2, used to adapt to the distance distribution in different scenarios.

[0134] Number of hash functions: q, the optimal number of hash functions opt =roundup(log(1 / P fp ) / log(1 / P same ), where P fpP represents the target false positive rate (e.g., 0.05, representing the upper limit of the probability of a false match). same The probability of similar vectors falling into the same hash bucket is given by 0.6 (e.g., 0.6 represents the probability that similar vectors fall into the same hash bucket). roundup() means rounding up.

[0135] Number of hash tables: Optimal number of hash tables L opt =roundup((N×f) / (1+f)), where N is the total number of ciphertext embedding vectors, and f is the time-space tradeoff coefficient, ranging from 0.3 to 0.7. A larger f value results in more hash tables, higher recall, but also higher storage overhead. Example value: When N=10⁶ and f=0.5, L... opt =roundup(106×0.5 / 1.5)≈333334, balancing recall rate and storage cost.

[0136] 3. SpectralHashing parameter tuning

[0137] Optimization target: hash bit length k.

[0138] Optimization formula: Optimal hash bit length k opt =min(rounddown(d / t c ), k max ).

[0139] Among them, t c This is the information compression ratio parameter, ranging from 0.2 to 0.5, representing the retention of 20% to 50% of the original vector's information. k max The maximum number of bits for hashing, such as 512, represents the upper limit of storage overhead.

[0140] The optimization objective is to avoid information loss after high-dimensional vector hashing, while controlling the hash bit length to not exceed the storage limit.

[0141] Example values: When d=2048, t c =0.4、k max =512, k opt =min(rounddown(2048 / 0.4),512)=min(5120,512)=512.

[0142] 4. MinHash parameter tuning

[0143] Optimization target: permutations.

[0144] Optimization formula: Optimal permutation count opt =roundup(log2(1 / a) / log2(2)) =roundup(log2(1 / a)).

[0145] Where 'a' is the precision error threshold, ranging from 0.01 to 0.1, representing the maximum error of Jaccard similarity. Example value: When a = 0.01, permutations... opt =roundup(log2(100))=7, meaning the optimal number of permutations is 7, which meets the general high-precision requirement of Jaccard similarity error ≤1%. If higher precision is required (e.g., a=0.001), then 10 permutations are taken (log2(1000)≈9.97, rounded up to 10), avoiding excessive permutations that would cause a surge in computation. In engineering practice, to further redundancy and ensure ultra-high precision, permutations of 128 (powers of 2) can be taken, maximizing similarity while maintaining accuracy within an acceptable computational cost range.

[0146] When the preset hash algorithm is an irreversible hash algorithm, the automatic optimization adjusts at least one parameter of the algorithm, such as output length, block size, or number of iterations, based on business security level, ciphertext embedding vector dimension, and storage / computation overhead constraints. For irreversible hash algorithms such as MD5, SHA-256, and SHA-512, the core optimization logic is as follows:

[0147] 5. Irreversible hash algorithms (MD5 / SHA series)

[0148] Parameter tuning targets: output length, group size, number of iterations (for HMAC class extended irreversible hashing).

[0149] Tuning rules:

[0150] Output length: Select according to the dimension of the ciphertext embedding vector and the security level. For low security requirements (such as ciphertext deduplication), select 128 bits (MD5); for medium to high security requirements (such as data integrity verification), select 256 bits (SHA-256); and for very high security requirements (such as sensitive ciphertext verification), select 512 bits (SHA-512).

[0151] Group size: Adapts to the dimension of the ciphertext embedding vector, such as a 768-dimensional vector grouped into 64-byte (512-bit) groups (default grouping for MD5 / SHA-256), and a 2048-dimensional vector grouped into 128-byte (1024-bit) groups (default grouping for SHA-512), to avoid increased computation due to group splitting;

[0152] Number of iterations (HMAC scenario): Optimal number of iterations (iter) opt=max(10000, ⌈d / 1024⌉×1000), where ⌈·⌉ is the floor function, and the number of iterations increases linearly with the dimension (e.g., 10000 iterations for 768 dimensions, 20000 iterations for 2048 dimensions), balancing security strength and computational cost. The optimization objective is to adapt to the dimensional features of the ciphertext embedding vector while meeting business security requirements, avoiding excessive encryption that leads to a surge in computation, and ensuring the unique mapping between the ciphertext and the pseudo-token ID.

[0153] Furthermore, when a single candidate hash algorithm that can simultaneously satisfy all constraints is not found after screening based on the multidimensional evaluation system and preset constraints (for example, in high-dimensional vector scenarios, it is difficult to find a single algorithm that simultaneously satisfies the constraints of "high precision + low computational cost"), a hybrid hashing strategy is adopted to generate the pseudo token ID, balancing computational efficiency and matching accuracy. The hybrid hashing strategy is implemented step by step according to the process of "preprocessing → preliminary screening → accuracy guarantee", as follows:

[0154] 1. Preprocessing stage: Dimensionality reduction of high-dimensional vectors to reduce computational overhead.

[0155] To address the issue of high computational cost and low efficiency in hash operations caused by excessively high dimensionality (e.g., d≥2000) in the ciphertext embedding vector, dimensionality reduction is performed on the ciphertext embedding vector to obtain a low-dimensional vector suitable for subsequent hash operations. Preferably, the PCA (Principal Component Analysis) dimensionality reduction algorithm is used. This PCA dimensionality reduction algorithm is a well-known technique in the field. Its core principle is to preserve the core semantic features of the ciphertext embedding vector, eliminate redundant dimensions, and compress the vector dimension to a preset suitable range (the preset range is determined based on the suitable dimension of the subsequent preliminary screening algorithm, typically 256 dimensions, and can be dynamically adjusted according to the actual scenario). Through dimensionality reduction, the time and space overhead of subsequent hash operations is significantly reduced without significantly losing the core semantic features of the vector, providing support for real-time inference requirements.

[0156] 2. Preliminary screening stage: Low-dimensional vector fast hashing to achieve candidate set filtering.

[0157] The first hash algorithm is used to perform hash operation on the low-dimensional vector obtained after preprocessing to generate a preliminary pseudo token ID. The core purpose is to quickly filter out the candidate set with high similarity and reduce the amount of data for subsequent high-precision processing.

[0158] The first hash algorithm is preferably the SimHash algorithm, which has strong low-dimensional adaptability, high computational efficiency, and can preserve the basic similarity relationship between vectors. The specific operation is as follows: the SimHash algorithm is used to perform hash operation on the low-dimensional vector to generate a preliminary pseudo-token ID. The preliminary pseudo-token ID is used as an index to query the KVCache of the large language model and quickly filter out the set of preliminary pseudo-token IDs with high similarity to the current preliminary pseudo-token ID, i.e., the candidate set. The screening criterion is preset to a candidate set recall rate of ≥85% to ensure that no potentially matching ciphertext vectors are missed, while realizing the rapid filtering of invalid data and improving the overall processing efficiency.

[0159] 3. Accuracy Assurance Stage: Secondary verification and calibration, generating target pseudo-token ID.

[0160] The candidate set (i.e., multiple candidate pseudo-token IDs) obtained from the initial screening is processed a second time to calibrate the matching accuracy and avoid accuracy loss in the initial screening stage. Finally, the target pseudo-token ID is generated. One of the following two methods can be used, which can be selected according to the business accuracy requirements:

[0161] Method 1: Precise Similarity Calculation and Verification. Extract the original ciphertext embedding vector corresponding to each candidate pseudo-token ID in the candidate set, calculate the precise similarity between each original ciphertext embedding vector and the currently input ciphertext embedding vector (preferably cosine similarity, but Euclidean distance, Jaccard similarity, etc. can be selected based on vector features), and filter out the vectors whose precise similarity meets the preset threshold (e.g., ≥0.9) to obtain the preliminary pseudo-token IDs, and calibrate them as the target pseudo-token IDs;

[0162] Method 2: Secondary hash verification. A secondary hash algorithm is used to perform a secondary hash operation on the low-dimensional vectors in the candidate set. The preferred secondary hash algorithm is the SpectralHashing algorithm, which has high precision and strong adaptability and can accurately preserve the similarity relationship between vectors. The matching precision of the initial pseudo-token ID is adjusted through secondary hash operation, and invalid candidates with low similarity are eliminated, finally generating the target pseudo-token ID.

[0163] The hybrid hashing strategy described above not only ensures the efficiency of hash operations through the preprocessing and preliminary screening stages to meet the real-time inference requirements of large language models, but also calibrates the matching accuracy through the accuracy guarantee stage to avoid cache mismatches, effectively solving the problem that a single hashing algorithm cannot take into account multiple constraints.

[0164] The large language model ciphertext caching method based on pseudo-token ID of the present invention can be applied to the following typical interaction scenarios, but is not limited thereto:

[0165] Scenario 1: Real-time encrypted semantic retrieval (e.g., secure query of enterprise knowledge base)

[0166] Users submit query requests to the large language model through an encrypted channel. The original query text is encrypted and then directly generates a ciphertext embedding vector. In traditional solutions, each query requires a complete calculation of the embedding vector and model inference, which cannot be accelerated by caching.

[0167] After adopting the method of this invention: the system generates a pseudo-token ID with a fixed format from the ciphertext embedding vector using a preset hash algorithm (such as locality-sensitive hashing), and uses it as the cache key of the key-value cache (KV Cache), storing it in association with the embedding vector. When subsequent queries for the same or semantically similar ciphertext are received, the cache can be directly hit through the pseudo-token ID, reusing the already stored embedding vector to participate in model inference, significantly reducing redundant calculations.

[0168] This scenario is particularly suitable for interactive tasks with high real-time requirements, such as high-frequency queries and multi-turn dialogues.

[0169] Scenario 2: Precise deduplication of encrypted documents (e.g., medical or legal file processing)

[0170] The data intermediary needs to deduplicatize a large number of encrypted documents (such as medical records and legal files) to avoid wasting resources by repeatedly inputting them into a large language model. The original documents are encrypted to generate ciphertext embedding vectors, but due to the lack of a standard key structure, it is impossible to use caching to detect duplicates.

[0171] This invention employs an irreversible hash algorithm (such as SHA-256) to generate a unique pseudo-token ID from the ciphertext embedding vector. Ciphertext embedding vectors from the same document will generate the same pseudo-token ID, allowing the system to directly identify duplicates through cache hits without needing to repeatedly perform encryption and model input processing. This method achieves high-precision ciphertext deduplication without revealing the original document content.

[0172] Scenario 3: Hybrid hash caching of high-dimensional ciphertext vectors (suitable for tasks that balance high precision and real-time response)

[0173] When the ciphertext embedding vector has a high dimension (e.g., exceeding 2000 dimensions) and the business requires both high matching accuracy and low response latency, a single hash algorithm may be insufficient to achieve both. This invention provides a hybrid hashing strategy:

[0174] First, the high-dimensional ciphertext embedding vector is reduced in dimensionality (e.g., using PCA principal component analysis). Then, a first hash algorithm (e.g., SimHash) is used to quickly generate preliminary pseudo-token IDs and filter the candidate set. Finally, the original vectors in the candidate set are subjected to secondary verification (e.g., precise similarity calculation or a second hash algorithm). Through this process, the time consumption and resource overhead of hash operations can be controlled while ensuring cache matching accuracy, thus meeting the real-time inference requirements in high-dimensional ciphertext scenarios.

[0175] It should be noted that the above application scenarios are merely illustrative examples, used to demonstrate the implementation logic and applicable environment of the method of the present invention under different business requirements. The specific performance indicators achieved in the actual system (such as response time, cache hit rate, throughput, etc.) depend on the specific hardware configuration, data distribution, hash algorithm parameters, and task load. Those skilled in the art can obtain the implementation effect that meets their own business needs based on the technical solution disclosed in this invention and combined with conventional engineering optimization, without needing to limit it through specific numerical examples.

[0176] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0177] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0178] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0179] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for ciphertext caching of large language models based on pseudo-token IDs, characterized in that, Includes the following steps: S100, Obtain the ciphertext generated after encrypting the original text, wherein the ciphertext is an embedding vector used for input processing of a large language model; S200, perform a hash operation on the embedded vector using a preset hash algorithm to generate a pseudo token ID with a fixed format; S300, the pseudo token ID is used as the cache key of the key-value cache of the large language model, and the embedding vector of the ciphertext is associated with it and stored in the key-value cache, so as to realize the cache reuse of the ciphertext by the large language model.

2. The method according to claim 1, characterized in that, The preset hash algorithm is selected from at least one of irreversible hash algorithms and locality-sensitive hash algorithms.

3. The method according to claim 1, characterized in that, The preset hash algorithm is determined through a multi-dimensional evaluation system, which includes: accuracy dimension, computational complexity dimension, user purpose dimension, and embedding dimension.

4. The method according to claim 3, characterized in that, The accuracy dimension is used to evaluate the ability of the hash algorithm to preserve the similarity relationship between the original embedding vectors; the computational complexity dimension is quantitatively evaluated based on at least one of time complexity, space complexity, and energy consumption estimation; the user purpose dimension evaluates the matching degree of the hash algorithm according to the task type, which includes at least one of complete matching, similarity search, classification, and clustering; the embedding dimension evaluates the adaptability of the hash algorithm according to the dimensional range of the embedding vectors.

5. The method according to claim 4, characterized in that, The preset hash algorithm is determined through a multi-dimensional evaluation system, specifically including: Based on the matching degree evaluation values ​​of each dimension of the multidimensional evaluation system, the comprehensive matching degree evaluation value of each candidate hash algorithm is calculated according to the preset dynamic weights. Based on preset constraints, the algorithm with the best comprehensive matching degree evaluation value and that satisfies all constraints is selected from the candidate hash algorithm set as the preset hash algorithm.

6. The method according to claim 2, characterized in that, The method further includes: Based on the characteristics of the embedded vector and the task requirements, the parameters of the preset hash algorithm are automatically optimized.

7. The method according to claim 6, characterized in that, When the preset hash algorithm is a locality-sensitive hash algorithm, the automatic optimization is based on at least one parameter among the target similarity error, false alarm rate, or accuracy error threshold.

8. The method according to claim 5, characterized in that, When a single candidate hash algorithm that can simultaneously satisfy all constraints is not found after screening according to the multidimensional evaluation system and preset constraints, a hybrid hashing strategy is adopted to generate the pseudo token ID. The hybrid hashing strategy includes: The embedding vector is reduced in dimensionality to obtain a low-dimensional vector; The low-dimensional vector is processed using the first hash algorithm to generate a preliminary pseudo-token ID for initial filtering. The multiple candidate pseudo-token IDs obtained after initial filtering are aggregated using a second hash algorithm or by calculating the similarity between the original embedding vectors corresponding to the multiple candidate pseudo-token IDs to generate the final target pseudo-token ID.

9. An electronic device, characterized in that, Including processor and memory; The processor executes the steps of the method as described in any one of claims 1 to 8 by invoking programs or instructions stored in the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a program or instructions that cause a computer to perform the steps of the method as described in any one of claims 1 to 8.

Images