A large model prompt compression method based on dynamic budget allocation

Abstract

The application provides a large model prompt compression method based on dynamic budget allocation. The method aims to optimize the length of the prompt of the large language model, improve the computational efficiency of the model and reduce the computational cost. Through the cooperation of the dynamic budget allocator and the word selector, the prompt can be compressed while ensuring the integrity of the semantics. First, the dynamic budget allocator dynamically allocates the word retention budget of each sentence based on the relevance of each sentence to the question. Then, the word selector calculates the retention probability of each word through the strategy network, and preferentially retains the key words. The reward function module evaluates the fidelity and similarity of the compressed prompt to guide the strategy gradient to update the network parameters. The method significantly improves the performance of the large language model and is suitable for prompt optimization in scenarios such as multi-turn dialogue and long text analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing and large model prompt compression, and particularly relates to a large model prompt compression method based on dynamic budget allocation. BACKGROUND

[0002] In recent years, there have been significant advances in the field of natural language processing, particularly in the development of large language models (LLMs) represented by ChatGPT. As the integration and application of large language models in daily scenarios continue to deepen, the demand for handling complex situations such as multi-turn conversations and comprehensive long text analysis has also increased. To address these challenges and improve the model's ability to accurately understand and reason human intent, researchers have introduced advanced prompting techniques. In particular, techniques such as Chain of Thought (CoT) and Retrieval-Augmented Generation (RAG) provide large language models with richer contextual cues and background information, thereby enhancing their capabilities.

[0003] However, the rapid development of these prompting techniques has led to an increase in prompt length, which presents significant challenges. The increase in prompt length directly affects the computational efficiency of large language models, as there is a quadratic complexity relationship between the performance of the model and the length of the input prompt. This extended prompt prefix not only reduces the generation efficiency of large language models, but also significantly increases the computational cost. In addition, existing large models are usually constrained by context window limitations, which severely limits their potential in tasks that require a large amount of or continuous information prompts. Therefore, how to optimize the performance of LLMs under the condition of long text input has become a key research area.

[0004] Recent research has shown that large language models can often generate correct responses with only the core portion of a prompt. Lengthy and irrelevant prompts can be inadvertently incorporated into the latent representation via the attention mechanism of large language models, potentially negatively impacting the model's ability to predict the next word. Therefore, simplifying prompts to reduce input length has emerged as a promising solution. However, while several prompt word compression techniques have been proposed, they still face several challenges. First, some techniques use smaller language models to calculate the information entropy of words, attempting to identify and remove unnecessary information. However, these methods typically only consider the preceding context, neglecting the impact of the entire context on word importance. This neglect is particularly problematic in scenarios involving long text prompts, such as multi-turn conversations or long-document question-answering, as the relevance of different sentences in the historical conversation or retrieved text can vary significantly with text length. Second, task-aware prompt compression methods often rely heavily on the large models for which they were developed, as well as on specific downstream tasks or compression ratios. This tight integration limits the flexibility and scalability of these techniques, posing challenges for cross-task application, especially for tasks with diverse requirements and constraints. Summary of the Invention

[0005] In response to the limitations of the existing technology, the present invention proposes a large-model hint word compression method framework based on dynamic budget allocation. The framework includes a dynamic budget allocator and a vocabulary selector, which aims to perform compression without destroying the semantics of the sentence as much as possible. Among them, the dynamic budget allocator uses a novel sentence-level fine-grained dynamic budget allocation strategy. By maintaining a consistent compression rate, the strategy can flexibly capture and retain important details related to each question, thereby improving compression efficiency and information retention. In addition, the strategy allows the weights of the comparison strategy to be adjusted to adapt to various downstream tasks, significantly enhancing the flexibility and applicability of the method. The vocabulary selector calculates the retention probability of a vocabulary based on the overall context. This method ensures that the selection of vocabulary is not based solely on its individual meaning, but also takes into account its important role in a broader context. This method significantly improves the quality of the compression results.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] A large model prompt word compression method based on dynamic budget allocation includes the following steps:

[0008] S1 obtains the prompt word to be compressed, which consists of a question part and a demonstration part; the question part is the last core question in the prompt word to be compressed, and the other parts of the prompt word to be compressed except the question are the demonstration part.

[0009] S2 constructs a vocabulary selector based on the attention mechanism, and uses the vocabulary selector to calculate the retention probability P of each word in the prompt word to be compressed i .

[0010] The vocabulary selector includes a DistilRoBERTa model and a linear classifier; a context association analysis is performed on each vocabulary in the prompt words to be compressed through an attention mechanism, and its retention probability in the process of compressing the prompt words is calculated.

[0011] S3 builds a dynamic budget allocator for allocating sentence retention budgets:

[0012] 1) The demonstration part of the prompt word to be compressed is formed into a set consisting of several sentences: {sen1, sen2, sen3, ..., sen n}, each of the sentences ends with any one of a period, a question mark, and an exclamation mark; set a target compression rate R, which represents the number of words B allowed to be retained after the presentation part is compressed total The original vocabulary number N before compression of the demo part total proportion;

[0013] 2) Use the character overlap strategy and sentence embedding similarity strategy to calculate the relevance score of each sentence to the question part i ;

[0014] 3) Using relevance score i Perform iterative allocation of the reservation budget, including the following steps:

[0015] Ⅰ) Initial allocation: based on the relevance score of each sentence i , calculate the initial budget ratio of each sentence according to the following formula:

[0016]

[0017] Subsequently, according to the total budget B total , assign a corresponding vocabulary budget B to each sentence i :

[0018]

[0019] Among them, min() represents the minimum value function, B i is assigned to the sentence sen i Number of reserved words,|sen i | indicates the sentence sen i The number of original words;

[0020] II) Iterative Adjustment: After the initial allocation is completed, the reserved budget may not be fully allocated or may exceed the total budget. To this end, an iterative algorithm is used to fine-tune the allocation results; if The remaining budget is allocated to the most relevant sentences in descending order of relevance scores until the budget is exhausted; if Then sort the sentences by relevance, starting with the sentence with the lowest relevance score, and gradually deduct the budget B allocated to the sentence. i , until the total budget requirement is met; a retention budget for each sentence is obtained, representing the number of words that should be retained during the compression process. This method implements a sentence-level fine-grained dynamic budget allocator, which can retain as much important information related to the question as possible while maintaining a fixed compression rate, improving the effectiveness of prompt word compression and the accuracy of information retention.

[0021] Combined with S2, the retention probability P of each word is obtained i , for each sentence in the demonstration part, select the reserved words in descending order of probability until the word retention budget B of the sentence is reached i ; Considering the importance of the question part to the generated results, the question part is not compressed; the compressed demonstration part is combined with the question part to obtain the compressed prompt words.

[0022] Preferably, after step S3, the following steps are further included:

[0023] S4 optimizes the vocabulary selector through reinforcement learning training:

[0024] Obtain a number of prompt words to be compressed as a training set, input each prompt word to be compressed in the training set into the large language model, and obtain the response of the large language model as a reference answer;

[0025] Each prompt word to be compressed in the training set is processed in steps S2 and S3 to obtain a compressed prompt word, which is input into the large language model to obtain the response of the large language model as the answer;

[0026] A reward function is constructed based on fidelity and vector similarity. Fidelity evaluates the overlap between an answer and a reference answer, while fidelity evaluates the overlap between the answer derived from the compressed prompt and the reference answer. The fidelity metric measures the degree to which responses generated by the large model avoid hallucinations. Fidelity is quantitatively evaluated using the following formula:

[0027]

[0028] Where I is the indicator function used to calculate the answer S ans Each word ω in the reference answer Sref The number of times |S ans |Yes the answer S ans Number of words;

[0029] The similarity refers to the cosine similarity distance between the vector representation of the answer and the vector representation of the reference answer, and the formula is as follows:

[0030] Similarity=cos(E(ans),E(ref))

[0031] Where E(ans) and E(ref) represent the vector representations of the answer and reference answer, respectively. The similarity reward function helps the training model select vocabulary, thereby effectively guiding large language models to generate accurate and relevant answers.

[0032] The policy network parameter θ is used to represent all parameters in the vocabulary selector that need to be obtained through reinforcement learning. According to the following formula, the policy network parameter θ of the vocabulary selector is adjusted using the policy gradient method; the goal of the policy gradient method is to maximize the expected reward;

[0033]

[0034] Where J(θ) is the expected return of the strategy, is the gradient of J(θ) to the policy network parameter θ, P θ (τ) represents the probability distribution of trajectories based on the policy network parameters θ, With probability P θ (τ) is the expected value of all possible trajectories generated, R(τ) is the reward of the corresponding trajectory, and the policy gradient guides the network update through the information fed back by the reward function;

[0035] After each update, the vocabulary selector readjusts the retention probability p of each vocabulary based on the new policy network parameters θ i , so that the retained vocabulary can maximize the quality of the generated response while generating compressed prompts. After iterative training, the optimized vocabulary selector is obtained.

[0036] Preferably, the step S2 specifically includes the following steps:

[0037] First, the compressed prompt words are segmented and decomposed into several words {w1,w2,...,w m}, each word w i Corresponding to a vocabulary ID, recorded as ID i , get the vocabulary ID sequence X = {ID1, ID2, ..., ID m},

[0038] According to the following formula, we can get the embedded vocabulary vector E=

[0039] {E1,E2,...E i ,...,E m};

[0040] E i =Embed(ID i )+Pos i

[0041] Among them, Embed(ID i ) is the vocabulary embedding function, which transforms the vocabulary ID i , converted to a fixed-dimensional vocabulary vector, Pos i is the positional encoding of the word, in the form of a vector with the same length as the word ID sequence X, representing the word ID i Location information:

[0042] The DistilRoBERTa model includes several layers of Transformer encoders, each of which contains two core components: a multi-head self-attention mechanism and a feedforward neural network.

[0043] The embedded vocabulary vector E is passed through the DistilRoBERTa model and then passed to the linear classifier to calculate the retention probability P of each vocabulary. i .

[0044] Preferably, in step S3, the character overlap strategy refers to using a unary character overlap function f inc (sen i ,que) calculate each sentence sen i The character-level overlap between the sentence and the question, que, ranges from 0 to 1. The higher the character overlap, the stronger the relevance between the sentence and the question. The formula is as follows:

[0045]

[0046] Among them, |common_chars(sen i ,que)| represents the length of the overlapping character set of the i-th sentence and the question part, |sen i | represents the length of all character sets that make up the i-th sentence;

[0047] Preferably, in step S3, the sentence embedding similarity strategy refers to using the all-MiniLM-L6-v2 model in the Sentence-Transformers library to embed each sentence into iand the question part que are converted into embedding vectors, and then their vector similarity sim(sen i ,que); the all-MiniLM-L6-v2 model is good at extracting deep semantic information from sentences without additional training.

[0048] Preferably, in step S3, the character overlap strategy and sentence embedding similarity strategy are used to calculate the relevance score of each sentence to the question part. i , further comprising the following steps:

[0049] The weighted average of the above two strategies is used to obtain the final relevance score of each sentence. i :

[0050] Score i =ω1·f inc (sen i ,que)+ω2·sim(sen i ,que)

[0051] Among them, ω1 and ω2 are adjustable preset weight parameters to adapt to the different requirements of different tasks for character matching and semantic similarity.

[0052] Preferably, in step S2, the DistilRoBERTa model includes a 6-layer Transformer encoder.

[0053] Preferably, in step S2, the processing formula of the linear classifier is as follows:

[0054] P i =sigmoid(O i W p +b p )

[0055] Among them O i Represents vocabulary w i Semantic information in context, W p and b p is the parameter of the linear classifier. After the sigmoid function, the output P i is the retention probability of each word between 0 and 1; the retention probability is used as the basis for compressing the prompt words, and words with higher retention probabilities are retained first to ensure that the compressed prompt words still contain important semantic information.

[0056] Preferably, the demonstration part of the prompt word to be compressed is a legal provision; and the question part of the prompt word to be compressed is a legal question that has a semantic association with the legal provision in the demonstration part.

[0057] Compared with the prior art, the beneficial technical effects of the present invention are:

[0058] 1. This paper proposes a novel sentence-level fine-grained dynamic budget allocation strategy. By maintaining a consistent compression rate, this technique can skillfully capture and preserve essential details unique to each question, thereby improving compression efficiency and information retention. In addition, it allows the comparison strategy weights to be adjusted to accommodate various downstream tasks, significantly improving the flexibility and applicability of the method. While maintaining the quality of model generation, this paper achieves efficient compression of prompt words, significantly improving the performance of large language models, and is suitable for prompt optimization in scenarios such as multi-round dialogues and long text analysis.

[0059] 2. This invention also proposes a context-aware word selection method. The word selector calculates the probability of word retention based on the overall context. This method ensures that words are selected not only based on their individual meaning but also consider their overall role in the broader context. This method significantly improves the quality of the compression results. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0061] Figure 1 This is an overall flow chart of the large model prompt word compression method based on dynamic budget allocation in the present invention;

[0062] Figure 2 is a schematic diagram of the dynamic budget allocator in the present invention; DETAILED DESCRIPTION

[0063] The specific embodiments of the present invention are described below in conjunction with the accompanying drawings and examples. However, the following examples are only used to illustrate the present invention in detail and do not limit the scope of the present invention in any way. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making creative work are within the scope of protection of the present invention.

[0064] Example 1

[0065] The embodiment of the present invention provides a large model prompt word compression method framework based on dynamic budget allocation. Figure 1 As shown, the framework of the prompt word compression method includes a dynamic budget allocator and a vocabulary selector. The present invention is described in detail below in conjunction with an embodiment. As shown in the figure, it specifically includes the following steps:

[0066] Step 1: Preprocessing training data: Use the SQuAD (Stanford Question Answering Dataset) training set as training data and convert it into a format that matches the input user question.

[0067] In this example, the SQuAD training set was first selected as the training data source for the model and converted to a standard format suitable for the input requirements of the large model, namely, the format of user questions and responses. To ensure high-quality data input, this example uses the datasets module in the Transformers library developed by HuggingFace to facilitate fast loading and processing of the SQuAD dataset.

[0068] During the preprocessing process, the original data set is first cleaned to remove redundant information contained in the data, such as HTML tags, special characters, irrelevant symbols, and other information that may interfere with model training. At the same time, considering the impact of the quality of the data set on model training, this step also deletes repeated question-answer pairs in the data set to ensure the uniqueness and diversity of the training data. This step can improve the generalization ability of the model and avoid overfitting due to data redundancy. In order to further improve the effect of model training and evaluate its performance, this embodiment divides the processed data set into a training set and a validation set. The training set is used to train the model to ensure that it can learn effective patterns for question-answer matching; the validation set is used to perform performance evaluation during the training process, and the generalization ability of the model is evaluated by the accuracy of the validation set, providing a basis for subsequent parameter adjustment and optimization. This data division helps to ensure the accuracy and robustness of the model in actual application scenarios.

[0069] Step 2: Construct a vocabulary selector based on the attention mechanism to calculate the retention probability of the vocabulary in the question sentence.

[0070] In this embodiment, the vocabulary selector consists of a DistilRoBERTa model and a linear classifier. It mainly uses the attention mechanism to perform contextual analysis on each word and calculate its retention probability during the process of compressing the prompt word. The specific steps are as follows:

[0071] The input prompt word to be compressed is first processed by the word segmenter and decomposed into several words {w1,w2,...,w m}, m represents the number of words in the prompt words to be compressed. Each word w i Corresponding to a vocabulary ID ID i These vocabulary IDs are organized into a sequence X = {ID1, ID2, ..., ID m} and used as the input of the model. At the same time, add the position code Pos, which is a vector with the same length as the vocabulary sequence, indicating the position information of each vocabulary:

[0072] E i =Embed(ID i )+Pos i

[0073] Among them, Embed(ID i ) is the vocabulary embedding function, which converts each vocabulary ID into a vocabulary vector of fixed dimension, Pos i Is the position code of the word. The word vector after embedding E={E1,E2,...,E m} Pass through 6 layers of Transformer encoder, each layer contains two core components: multi-head self-attention mechanism and feedforward neural network. Then pass it to the linear classifier to calculate the retention probability P of each word i The processing formula of the linear classifier is:

[0074] P i =sigmoid(O i W p +b p )

[0075] Among them O i Represents vocabulary w i Semantic information in context, W p and b p is the parameter of the linear classifier. After the sigmoid function, the output P i is the retention probability of each word between 0 and 1. The higher the retention probability value, the more likely the word should be retained during the prompt word compression process. The final output is the retention probability sequence of each word {P1, P2, ..., P m These retention probabilities will be used as the basis for compressing the prompt words, and words with higher retention probabilities will be retained first to ensure that the compressed prompt words still contain important semantic information.

[0076] Step 3: Build a dynamic budget allocator for allocating sentence retention budgets.

[0077] During the prompt word compression process, because different parts of the prompt have different levels of importance, it is necessary to dynamically allocate the retention budget based on the relevance of the sentence to the question. In this embodiment, the dynamic budget allocator calculates the importance of sentences using multiple relevance strategies and then dynamically adjusts the retention budget of each sentence during the compression process based on these relevance scores. The specific implementation steps are as follows:

[0078] 4) The prompt word to be compressed consists of a question part and a demonstration part. The question part is the last core question sentence in the prompt word to be compressed, and the rest of the part excluding the question is the demonstration part. The demonstration part consists of several sentences {sen1, sen2, sen3, ..., sen n}, which are separated by periods, question marks, or exclamation marks. The goal of the dynamic budget allocator is to allocate a retention budget to each sentence based on its relevance to the question part. Set a target compression ratio R, which represents the ratio of the total number of words in the presentation part to the original number of words. For example, if the compression ratio is 0.7, it means that 70% of the original words are retained. Calculate the total budget B for the presentation part based on the compression ratio. total , that is, the total number of words allowed to be retained:

[0079]

[0080] where N total is the total number of words in the demonstration part.

[0081] 5) Use multiple strategies to calculate the relevance of each sentence to the question part.

[0082] Ⅰ) Character overlap strategy: Each sentence will be assigned a relevance score Score i , which will play a key role in allocating the retention budget. Calculate the overlap between each sentence and the question at the character level, using the unary character overlap function f inc (sen i ,que), the formula is as follows:

[0083]

[0084] Among them, que represents the question part, common_chars(sen i ,que) represents the overlapping character set of the i-th sentence and the question part.

[0085] II) Sentence Embedding Similarity Strategy: Use the all-MiniLM-L6-v2 model from the Sentence-Transformers library to convert each sentence and question into an embedding vector. Then, calculate their vector similarity using cosine similarity. The higher the similarity, the greater the semantic relevance between the sentence and the question, and the higher the budget allocated.

[0086] The scores of all relevance strategies are weighted averaged to obtain the final relevance score of each sentence. i :

[0087] Score i =ω1·f inc (seni ,que)+ω2·sim(sen i ,que)

[0088] Among them, ω1 and ω2 are adjustable weight parameters to adapt to the different requirements of different tasks for character matching and semantic similarity.

[0089] 6) Calculate the relevance score of each sentence i Afterwards, the next step is to iteratively allocate the reservation budget based on these scores:

[0090] Ⅰ) Initial allocation: Based on the relevance score of each sentence, calculate the initial budget allocation ratio for each sentence:

[0091]

[0092] Subsequently, according to the total budget B total , assign a corresponding vocabulary budget B to each sentence i :

[0093]

[0094] Among them, min() represents the minimum value function, B i is assigned to the sentence sen i Number of reserved words,|sen i | indicates the sentence sen i The number of original words.

[0095] II) Iterative Adjustment: After the initial allocation is completed, the reserved budget may not be fully allocated or may exceed the total budget. To this end, an iterative algorithm is used to fine-tune the allocation results. The remaining budget is then allocated to the most relevant sentences in descending order of relevance scores until the budget is exhausted. Start with the sentences with the lowest relevance scores and gradually reduce their budgets until the total budget requirement is met.

[0096] The result is a retention budget for each sentence, which indicates the number of words that should be retained during the compression process. This approach implements a dynamic budget allocator based on sentence-level fine-grainedness. This approach maximizes the preservation of important information related to the question while maintaining a fixed compression rate, improving both the effectiveness of prompt word compression and the accuracy of information retention.

[0097] Combined with the retention probability P of each word obtained in step 2 i , each sentence in the demonstration part selects the reserved vocabulary in order from high to low probability until the vocabulary retention budget B of the sentence is reachedi Considering the importance of the question part to the generated results, we do not compress it. Finally, the compressed demonstration part is combined with the question part to obtain the compressed prompt word.

[0098] Step 4: Optimize the vocabulary selector through reinforcement learning training.

[0099] First, the input demonstration is processed by a dynamic budget allocator, which determines the number of words B allowed to be retained for each sentence. i , this step is based on the aforementioned dynamic budget allocation algorithm. On this basis, the policy network in the vocabulary selector is used for each word x in the demonstration. i Calculate the retention probability P i . This probability is based on the importance of the word in the context. The policy network gives priority to those words that are crucial to generating the final output through semantic understanding of the context. According to the retention probability of each word, the words with higher probability are retained first to generate compressed prompt words. These compressed prompt words are input into a large language model (such as Tongyi Qianwen or ChatGPT, etc.) to generate corresponding replies. At the same time, the original prompt words are also input into the same large language model to generate baseline replies. By comparing the replies generated by the compressed prompts with the original prompts, the training process can evaluate the effect of compression and adjust the parameters of the policy network accordingly. This process depends on the evaluation results of the reward function module. The reward function module includes the aforementioned faithfulness (Faithfulness) and vector similarity (Similarity). Based on the evaluation results of faithfulness and similarity, the training process uses the policy gradient method to adjust the policy network parameters θ of the vocabulary selector. The goal of the policy gradient is to maximize the expected reward, thereby improving the effectiveness of the compressed prompts in the generation process. The policy gradient update formula is:

[0100]

[0101] Among them, J(θ) is the expected return of the strategy, P θ (τ) represents the probability distribution of trajectories based on the policy parameter θ, R(τ) is the reward for the corresponding trajectory, and the policy gradient guides the network update through the information fed back by the reward function.

[0102] After each update, the policy network readjusts the retention probability P of each word based on the new parameter θ i , so that the retained vocabulary can maximize the quality of the generated response while generating compressed prompts. After multiple iterative training, the policy network is gradually optimized, enabling it to accurately select those words that are crucial to the generated results.

[0103] The entire training process utilizes policy gradient optimization methods from reinforcement learning. By continuously optimizing the vocabulary selector's policy network, it increases its sensitivity to retaining important words. Once the model converges, the vocabulary selector can significantly compress the cue words without compromising generation quality. This not only improves the model's computational efficiency but also effectively reduces the computational cost of large models.

[0104] Example 2

[0105] In order to more fully understand the specific execution steps of the method proposed in the present invention, this embodiment uses a legal text interpretation case to illustrate the application of the present invention in the legal text interpretation scenario.

[0106] Step 1: After being processed by the method described in step 1 of Example 1, the input prompt word to be compressed is as follows:

[0107] "Labor contracts can be categorized as fixed-term, open-ended, or based on the completion of a specific job. If an employee has worked for the same employer for more than ten consecutive years and both parties agree to extend the labor contract, an open-ended labor contract shall be entered into if the employee requests one. A labor contract may stipulate a probationary period, which shall not exceed six months. The parties to a labor contract may also stipulate in the labor contract matters regarding the confidentiality of the employer's business secrets. A labor contract terminates upon expiration or the occurrence of the agreed-upon termination conditions. An employee must provide the employer with thirty days' written notice to terminate a labor contract. If an employer is nearing bankruptcy and undergoing statutory restructuring or faces serious production and operational difficulties and genuinely needs to reduce staff, the employer must explain the situation to the trade union or all employees thirty days in advance, consult the union or employees, and report to the labor administrative department before making staff reductions. Based on the above information, how much advance notice should an employee provide to the employer if they need to terminate a labor contract?"

[0108] In this case, the prompt "Based on the above information, if an employee needs to terminate their labor contract, how long in advance should they notify their employer?" is the question. The demonstration includes 124 words and 8 sentences. With a predefined compression ratio of 0.7, the compressed vocabulary budget is 87 words.

[0109] Step 2: The presentation part of the prompt word to be compressed is calculated by the vocabulary selector to obtain the vocabulary retention probability.

[0110] Step 3: The dynamic budget allocator calculates the retention budget for each sentence based on the presentation portion of the prompt word to be compressed. The dynamic budget allocator sets the relevance strategy weight parameters ω1 to 1 and ω2 to 5. The results are shown in Table 1:

[0111] Table 1. The retention budget of each sentence calculated by the dynamic budget allocator

[0112]

[0113] Combining the retention probability from step 2 and the retention budget from step 3, and combining the question part, the compressed prompt words are:

[0114] "Labor contracts can have fixed terms, open-ended terms, or terms for completion of work. If an employee has worked for the same employer for ten years and the parties agree to extend the labor contract, and the employee proposes an open-ended labor contract, the contract shall be open-ended. The labor contract may stipulate a probationary period, which can be up to six months. The parties to a labor contract may agree in the labor contract on matters related to the employer's business secrets. Upon expiration of the labor contract, the labor contract terminates if the conditions agreed upon by the parties arise. An employee who terminates a labor contract must give the employer thirty days' written notice in advance. If an employer is on the verge of bankruptcy and undergoing statutory reorganization or is facing difficulties and is reducing staff, the employer must inform the union thirty days in advance, listen to the union and employees' opinions, and report to the labor administrative department before making staff reductions. Based on the above information, how long in advance should an employee notify the employer if they need to terminate their labor contract?"

[0115] This embodiment demonstrates the application of a large-model prompt word compression method based on dynamic budget allocation in a legal text interpretation scenario. The present invention can also be extended to other professional fields that require the input of a large amount of long text information prompts, such as medical and financial fields, providing new solutions for the application of large models in these fields.

Claims

1. A large model prompt word compression method based on dynamic budget allocation, characterized in that: The steps include: S1 obtains a prompt word to be compressed, wherein the prompt word to be compressed consists of a question part and a demonstration part; the question part is the last core question sentence in the prompt word to be compressed, and the other parts of the prompt word to be compressed except the question are the demonstration part; S2 constructs a vocabulary selector based on the attention mechanism, and uses the vocabulary selector to calculate the retention probability P of each word in the prompt word to be compressed i ; The vocabulary selector includes a DistilRoBERTa model and a linear classifier; S3 builds a dynamic budget allocator for allocating sentence retention budgets: 1) The demonstration part of the prompt word to be compressed is represented as a set consisting of several sentences: {sen1, sen2, sen3, ..., sen n }, each of the sentences ends with any one of a period, a question mark, and an exclamation mark; set a target compression rate R, which represents the number of words B allowed to be retained after the presentation part is compressed total The original vocabulary number N before compression with the demo part total proportion; 2) Use the character overlap strategy and sentence embedding similarity strategy to calculate the relevance score of each sentence to the question part i ; 3) Using relevance score i Perform iterative allocation of the reservation budget, including the following steps: Ⅰ) Initial allocation: based on the relevance score of each sentence i , calculate the initial budget ratio of each sentence according to the following formula: Subsequently, according to the total budget B total , assign a corresponding vocabulary budget B to each sentence i : Among them, min() represents the minimum value function, B i is assigned to the sentence sen i Number of reserved words,|sen i | indicates the sentence sen i The number of original words; II) Use iterative algorithm to fine-tune the allocation result; if The remaining budget is allocated to the most relevant sentences in descending order of relevance scores; if Then sort the sentences by relevance, starting with the sentence with the lowest relevance score, and gradually deduct the budget B allocated to the sentence. i , until the total budget requirement is met; get the retention budget of each sentence, which represents the number of words that should be retained in each sentence during the compression process; Combined with S2, the retention probability P of each word is obtained i , for each sentence in the demonstration part, select the reserved words in descending order of probability until the word retention budget B of the sentence is reached i ; Do not compress the question part; Combine the compressed demonstration part with the question part to obtain the compressed prompt words; In step S3, the character overlap strategy is to use the unary character overlap function f inc (sen i ,que) calculate each sentence sen i The degree of overlap with the question part que at the character level is calculated as follows: Among them, |common_chars(sen i ,que)| represents the length of the overlapping character set of the i-th sentence and the question part, |sen i | represents the length of all character sets that make up the i-th sentence; In step S3, the sentence embedding similarity strategy refers to using the all-MiniLM-L6-v2 model in the Sentence-Transformers library to embed each sentence into i and the question part que are converted into embedding vectors, and then their vector similarity sim(sen i ,que); In step S3, the character overlap strategy and sentence embedding similarity strategy are used to calculate the relevance score of each sentence to the question part. i , further comprising the following steps: The above two strategies are weighted averaged to obtain the final relevance score of each sentence. i : Score i =ω1·f inc (sen i ,that)+ω2·sim(sen i ,what) Among them, ω1 and ω2 are adjustable preset weight parameters to adapt to the different requirements of different tasks for character matching and semantic similarity.

2. The large model prompt word compression method based on dynamic budget allocation according to claim 1, characterized in that: After step S3, the following steps are also included: S4 optimizes the vocabulary selector through reinforcement learning training: Obtain a number of prompt words to be compressed as a training set, input each prompt word to be compressed in the training set into the large language model, and obtain the response of the large language model as a reference answer; Each prompt word to be compressed in the training set is processed in steps S2 and S3 to obtain a compressed prompt word, which is input into the large language model to obtain the response of the large language model as the answer; A reward function is constructed based on fidelity and vector similarity. The fidelity refers to the degree of overlap between the evaluation answer and the reference answer. The fidelity is quantitatively evaluated using the following formula: Where I is the indicator function used to calculate the answer S ans Each word ω in the reference answer S ref The number of times |S ans | is the answer S ans Number of words; The similarity refers to the cosine similarity distance between the vector representation of the answer and the vector representation of the reference answer, and the formula is as follows: Similarity=cos(E(ans),E(ref)) Where E(ans) and E(ref) represent the vector representation of the answer and reference answer respectively; The policy network parameter θ is used to represent all parameters in the vocabulary selector that need to be obtained through reinforcement learning. According to the following formula, the policy network parameter θ of the vocabulary selector is adjusted using the policy gradient method; the goal of the policy gradient method is to maximize the expected reward; Where J(θ) is the expected return of the strategy, is the gradient of J(θ) to the policy network parameter θ, P θ (τ) represents the probability distribution of trajectories based on the policy network parameters θ, With probability P θ (τ) is the expected value of all possible trajectories generated, R(τ) is the reward of the corresponding trajectory, and the policy gradient guides the network update through the information fed back by the reward function; After each update, the vocabulary selector readjusts the retention probability P of each vocabulary based on the new policy network parameters θ i ,After iterative training, the optimized vocabulary selector is obtained.

3. The large model prompt word compression method based on dynamic budget allocation according to claim 1, characterized in that: The step S2 specifically includes the following steps: First, the compressed prompt words are segmented and decomposed into several words {w1,w2,...,w m }, each word w i Corresponding to a vocabulary ID, recorded as ID i , get the vocabulary ID sequence X = {ID1, ID2, ..., ID m }, According to the following formula, the word vector after embedding is obtained: E = {E1, E2, ... E i ,...,E m }; E i =Embed(ID i )+Pos i Among them, Embed(ID i ) is the vocabulary embedding function, which transforms the vocabulary ID i , converted to a fixed-dimensional vocabulary vector, Pos i Is the position code of the word, indicating the word ID i location information; The DistilRoBERTa model includes several layers of Transformer encoders, each of which contains two core components: a multi-head self-attention mechanism and a feedforward neural network. The embedded vocabulary vector E is passed through the DistilRoBERTa model and then passed to the linear classifier to calculate the retention probability P of each vocabulary. i .

4. The large model prompt word compression method based on dynamic budget allocation according to claim 3, characterized in that: In step S2, The DistilRoBERTa model includes a 6-layer Transformer encoder.

5. The large model prompt word compression method based on dynamic budget allocation according to claim 3, characterized in that: In step S2, the processing formula of the linear classifier is as follows: P i =sigmoid(O i W p +b p ) Among them O i Represents vocabulary w i Semantic information in context, W p and b p is the parameter of the linear classifier. After the sigmoid function, the output P i is the retention probability of each word between 0 and 1.

6. The large model prompt word compression method based on dynamic budget allocation according to claim 1, characterized in that: The demonstration part of the prompt word to be compressed is a legal provision; the question part of the prompt word to be compressed is a legal question that has a semantic association with the legal provision in the demonstration part.

Images